TL;DR: Agile Failure Patterns — Why Agile is Simple and Complex at the Same Time
Agile failure seems to be increasingly more prominent nowadays despite all the efforts undertaken by numerous organization embarking on their journeys to become agile.
The funny thing is: Who would disagree that the four core principles of the Agile Manifesto —
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
— are derived from applying common sense to a challenging problem? Moreover, the application of those principles might be suited to fix numerous organizational dysfunctions and reduce an error-prone and complex social setting to maybe just a complicated one?
🗞 Do you want to get this article in your inbox in the future? You can sign up here for our weekly ‘Food of Agile Thought newsletter’ and join 29k other subscribers.
Update 2019-03-27: The Video of the Webinar Agile Failure Patterns 2.0 Is Available
The video of the webinar is available now:
Note: If the browser will not play the playlist automatically, click here to watch the replay of the webinar agile failure patterns 2.0 directly on Youtube.
Reasons for Adopting Agile
There are various good reasons to adopt Agile, for example:
- Low productivity,
- Low morale,
- Problems hiring senior people in an increasingly competitive war for talent,
- Loss of senior people,
- Budget constraints (no more funds to waste on waterfall projects), or
- The competition drives innovation at a high pace, and the organization starts lagging behind.
But the fact is also, that the scope of an Agile organizational transformation is often wholly underestimated. That Agile is not the quick fix for everything that is going wrong. Each organization has its own set of dysfunctions, and hence solutions dealing with them need to be tailored specifically to that organization.
Agile Failure Patterns
Analyzing my past projects, I identified the following four cross-organizational patterns that are making Agile transitions to much more harder, less effective and significantly more expensive:
I. Agile Failure At Organizational Level:
- Not having a (product) vision in the first place: If you don't know where you are going, any road will get you there.
- The fallacy of "We know what we need to build." There is no need for product discovery or hypotheses testing; the senior management can define what is relevant to the product backlog.
- A perceived loss of control at management level leads to micro-management. (The ‘what-is-in-for-me-syndrome.’)
- The organization is not transparent about vision and strategy hence the teams are hindered to become self-organizing.
- There is no culture of failure: Teams, therefore, do not move out of their comfort zones but instead play safe. (Mathgeek on HN suggests to replace "culture of failure" with "culture of learning to get back up again after falling.")
- The organization is not optimized for a rapid build-test-learn culture, and thus departments are moving at different speed levels. The resulting friction caused is likely to equalize previous Agile gains.
- Senior management is not participating in Agile processes, for example, the sprint demos, despite being a role model. Instead, they do expect a different form of (push) reporting.
- Not making organizational flaws visible: The good thing about Agile is that it will identify all organizational problems sooner or later. (“When you put problem in a computer, box hide answer. Problem must be visible!” Hideshi Yokoi, former President of the Toyota Production System Support Center in Erlanger, Kentucky, USA.)
- Product management is not perceived as the “problem solver and domain expert” within the organization, but as the guys who turn requirements into deliverables, aka “Jira monkeys.”
- Other departments fail to involve product management from the start. Typical behavior in larger organizations is a kind of silo thinking, featured by local optimization efforts without regard to the overall company strategy, often driven by individual incentives, e.g., bonuses. (Personal agendas are not always aligned with the company strategy.)
- The sales organization guards access to customers, thus preventing teams from learning.
- Core responsibilities of product management are covered by other departments, e.g., tracking metrics, thus leaving product dependent on others for data-driven decisions.
- Product managers w/o a dedicated team can be problematic if the product management team is oversized by comparison to the size of the engineering team.
- Engineering teams are not free to choose “their” tech stack.
Cannot see the subscription form?
Please click here
II. Agile Failure At Team Level:
- There are too many junior engineers on an engineering team. They tend to appreciate micro-management as part of their training. Usually, they have no or little experience with Agile methodologies. Hence they hardly can live up to processes, particularly they fail to say “No.”
- Engineers with an open source coding mentality: Tasks are discussed on pull requests once they're finished, but not in advance during grooming or sprint planning sessions. (They tend to operate in a distributed team mentality.)
- Teams are too small and hence not cross-functional. Example: A team is only working on frontend issues and lacks backend competence. That team will always rely on another team delivering functionality to build upon.
- Teams are not adequately staffed, e.g., scrum master positions are not filled and product owners have to serve two roles at the same time.
- Team members reject agile methodologies openly, as they do not want to be pushed out of their comfort zones.
- Teams are not self-organizing. That would require accepting responsibility for the team’s performance and a sense of urgency for delivery and value creation. A "team" in this mode is more acting like a group—for example, people waiting at a bus-stop—: They are at the same time in the same place for the same purpose, but that does not result in forming a team. (The norming, storming, forming, performing cycle seems to be missing.)
- Even worse, team members abandon Agile quietly, believing it is a management fad that will go away sooner or later.
- Faux Agile: Teams follow the “Agile rules” mechanically without understanding why those are defined in the first place. This level of adoption often results in a phenomenon called "Peak Scrum": There is no improvement over the previous process, despite all Agile rules are being followed to the letter. Or even worse: morale and productivity go down, as the enthusiasm after the initial agile training wears off quickly in the trenches.
- Moving people among teams upon short notice. Even if required for technical reasons, this has a negative impact on a team's performance & morale.
Share your own experience w/ the poll: What Is the Most Important Agile Challenge You Are Facing At the Moment?
III. Agile Failure At Process Level:
- Agile light: The management abandons self-organization the moment a critical problem appears to form ‘task forces’ instead.
- Agile processes are either bent or ignored whenever it seems appropriate. There is a lack of process discipline.
- Agile processes are tempered with, e.g., the scrum product owner role is reduced to a project manager role. Often this is done so by assigning the task of backlog ownership to a different entity at the management level. ("Scrum" without a product owner makes a magnificent Waterfall 2.0 process.)
- Stakeholders are bypassing product management to get things done and get away with it in the eyes of the senior management, as they would show initiative.
- The organization is not spending enough time on team communication and workshops to create a shared understanding of what is to be built.
IV. Agile Failure At Facility Level:
- A team is not co-located, not working in the same room, but scattered across different floors, or worse, different locations.
- The work environment is lacking spots for formal and – more important – informal communication: cafeterias, tea kitchens, sofas, etc.
- The work environment is lacking whiteboards. The absence of whiteboards on every available wall within the office should be questionable, not having them.
- Agile requires suitable offices to further collaboration: spacy, with plenty of air and light. But they should not be a mere open space, which tends to get too noisy, particularly when several Scrum teams have stand-ups at the same time.
Agile Failure Patterns: What Is the Most Important Agile Challenge You are Facing at the Moment?
Every one of us is dealing with numerous issues related to becoming, improving, or even staying Agile every day. I would like to learn more about the spectrum of your Agile challenges. If you invest three minutes of your time, that would mean the world to me:
Click here to complete "Your #1 Agile Challenge" at Google Forms
📖 Related Posts
Download the Scrum Anti-Patterns Guide: 160-plus ways to improve your way of Scrum.
📅 Scrum Training Classes, Workshops, and Events
You can secure your seat for Scrum training classes, workshops, and meetups directly by following the corresponding link in the table below:
See all upcoming classes here.
You can book your seat for the training directly by following the corresponding links to the ticket shop. If the procurement process of your organization requires a different purchasing process, please contact Berlin Product People GmbH directly.
✋ Do Not Miss Out and Learn more about Agile Failure: Join the 12,000-plus Strong ‘Hands-on Agile’ Slack Team
I invite you to join the “Hands-on Agile” Slack team and enjoy the benefits of a fast-growing, vibrant community of agile practitioners from around the world.
If you like to join now all you have to do now is provide your credentials via this Google form, and I will sign you up. By the way, it’s free.
View Comments (13)
As someone who worked as a software developer in the 1980s before lightweight methodologies appeared on the scene, I should point out that projects with sequential phases were nowhere near as unsuccessful as people believe them to have been. It is not a fallacy to know what needs to be built. Where you understand the business domain and how any system fits into that domain, it is possible to work out what needs to be built with a good degree of precision. Agile asks a business to change to support agile modes of thought with no clear supporting arguments as to how they address the problems that said business is facing. Therefore, it is justified that every aspect of agile is analysed and questioned. The support of agile by inductive reasoning is not enough.
Hi Stefan, good points, although I'm not sure if I agree whit this one:
"Engineers with an open source coding mentality: Tasks are discussed on pull requests once they're finished, but not in advance during grooming or sprint planning sessions. (They tend to operate in a distributed team mentality.)"
Over the years, I started to believe that the principles and approaches of open source applied to internal development - InnerSource - are very beneficial to agile because they help to improve some very important aspects of agility, such as:
- Collective code ownership. (Avoids: "I can't change that because this belongs to team XYZ. I'm blocked"
- Cross-functionality (Avoids: "I don't know nothing about that module. There's no documentation and the person who made it already left the company. We need to rewrite it").
- Reduced cross-team dependencies (Avoids: "We're blocked. Team XYZ don't have time to add the functionality we need in order to deliver our part. We'll have to wait till next week")
- Increase overall product knowledge (Avoids: "We're not responsible for that part. We don't know nothing about it").
I have seen many times, issues that could be avoided if engineers have a more open source mentality. I'm not saying that all code should be open sourced or that all code should have a clear owner, but sometimes, even having all the skills inside the team is not enough to completely remove dependencies. Sometimes you have to use other approaches to go one step forward and increase even more the level of dependency between teams.
I was not referring to open source in general, just a particular communication form that was taking its heavy toll on that organization.
Good article. Would be great to have some social media sharing buttons specifically for the article to make it easily shareable to.
Thanks! The sharing buttons… There’s a technical issue at the moment. But we are at it!
Scrum makes employees leave the company
At first sight this sounds very dramatic, but it does not have to be. Scrum creates transparency and this quickly prevents anyone hiding behind or pointing at the (supposed) mistakes of others. Through constraining the work to a fixed Time Box, the tasks within the team and the transparency in respect to what is delivered removes the potential for common misunderstandings or excuses such as ”I’ve been waiting for this or for that”, or “I had to do so many other things for the customer that I simply didn’t get to it”. Yes, the successful introduction of Scrum may lead to the fact that some employees and manager don’t feel comfortable in their company. Perhaps some of these employees will leave because not everyone goes along with the changes that Scrum brings. But if we are honest it’s often best for both parties - Scrum brings a lot of transparency, and the impact of avoiding transparency has a detrimental impact on both employees and the company, with or without Scrum.
Scrum leads to conflicts
Scrum is a very simple framework, but the combination of transparency with short fixed-duration sprints often exposes pre-existing conflicts or inefficiencies between development teams, business owners and management.
The simple rules of Scrum mean that every cause of difficulties in the software and/or product development process in a company will be recognized. This is where the courageous companies win, and some companies that are uncomfortable with recognising and tackling the problems will often step back and fail to remove them. That Scrum cannot function in this environment should be clear. However this failure to recognise or tackle systemic problems often leads to the statements or excuses that ”Scrum is chaotic” or ”Scrum doesn’t work for us”.
The spectrum of possible conflicts that can appear through the introduction of Scrum is broad. It starts with managers who like transparency in development, but who don’t want to share their visions or who think that the rules are only for others. The traditional project leaders first need to understand how their role will change with Scrum, and ensure full cooperation with the customers of the development teams.
As more and more companies transition to Scrum, experience shows that professional support and advice is needed. Although the Scrum framework is simple, it is still a major change, and the advantage of an independent and experienced guide while making the transition has been proven over and over again.
(At this point I’d like to express my gratitude and thanks to all my colleagues and advisors that, through ”Scrumtisch,” ”Scrum Breakfasts” and other initiatives, continue to make Scrum successful, and answer questions and offer support at anytime.)
Scrum changes the role perception
Just yesterday I finished a presentation for project leaders and developers and recognised that the most important question from the audience was: How does Scrum fit with our established structures and roles? We already covered the three Scrum roles, the Product Owner, the Scrum Master and the Team. Unfortunately, the nearest equivalent to be found in traditional projects: Product Owner instead of Product Manager, Scrum Master instead of Team- or Project leader, or Development Team instead of developers and testers does not properly differentiate between the responsibilities of each Scrum role.
Product Owner
The nomenclature is definite. In Scrum, the PO is not only the manager of a product, but also the owner of the product, from vision to conception to delivery. Therefore he/she is the one responsible for the creation of the product. Being a Product Owner means:
You are responsible for the successful outcome of the product delivered by the team.
You make business decisions around importance and priorities.
You deliver the vision of the product to the team.
You prepare the User Stories for the team to develop.
You should possess extensive domain knowledge.
You validate the work delivered and test it for quality.
You communicate on a continual base with all stakeholders, financiers and the team.
You control the financial side of the project.
You are responsible for the return-on-investment of the product development.
Scrum Master and Team
Unlike many other development methods, a Scrum development team is not simply the executive organ that receives its tasks from the project leader. Rather, it decides itself which requirements or User Stories it can accomplish in one Sprint. The team constructs the task list themselves and is responsible for any permutation to those; in other words, the team becomes a manager. This new self-conception of the team and the alignment of tasks and responsibilities necessarily change the role of the team leader/project leader.
The Scrum Master does not need to delegate all the work or to plan the project. Rather he/she ensures that the team is able to reach the self-made goals, removing impediments and providing an ideal working environment for the team. This change in roles is one of the most important aspects to understanding Scrum with the intent of introducing it in your own company.
Scrum Anti-Patterns
Scrum works! Innumerable projects prove that. But, and of course there must be a ”but,” Scrum only works when one has the courage to really realize the rules and ideas behind Scrum. Scrum is more than just the summation of all the roles, ceremonies and artifacts.
We do not expect a perfectly smooth introduction of Scrum. Everyone involved needs to get used to the new framework and will be unsure what needs to be done if something unexpected happens. Problems often lead to ”known” mechanisms ”just for this one case” instead of solving the root cause of the problem, as required in the Scrum method.
Once the synergy between teams that is created by Scrum is broken, the impact of Scrum, the outstanding increase of productivity, gets lost rapidly.
Here are some of the worst, as in most destructive, Scrum Anti-Patterns we have seen: - The Scrum Master assigns tasks, thereby destroying the self navigating and self organized team. - The sprint gets disturbed externally. New tasks are included or changes to the committed User Stories are allowed. - There are no Review Meetings. - There are no Retrospectives. - There are no Daily Stand-ups. - The Product Owner doesn’t have adequate expertise or experience and can’t stand up for his role. - There is no ‚”1-to-1 Agreement‚” between the PO and the Team. - The participants aren’t trained well enough in Scrum. - All meetings aren’t time-boxed. - Instead of features, activities are monitored. - Organisations think that Scrum can be rolled out after a few Scrum Masters are sent to certification courses.
Thanks a lot for this awesome comment, Karki!
Would you be interested in turning this into a guest-post?
Good write-up.
Empowerment comes from the top down: The best run team has a clear mandate i.e. Strategic Initiative, a sponsor who partners with the team and a clear prioritized program level backlog and roadmap. This allows the team to focus on delivery from the get go.
What are your keys to success?
@Chris: Apologies, you're absolutely right, we are working at it already.
This page is very difficult to read with the small grey-on-white font, I'd suggest changing it to something with a bit more contrast.
@Vikash: So far on two occasions. To my observation, it's mainly a personal issue, e.g. sticking to a 5-to-9 mentality (structured way of living) or being uncomfortable committing to an outcome that's time-boxed. In one case, we set up a Kanban team that is actually working well for all involved: The one in question is no longer stressed and the agile team is no longer facing methodology discussions on a daily basis.
What patterns have you encountered so far?
Stefan, good article - you listed practical anti patterns that prevails. Regarding 'Team members reject Agile methodologies openly, as they do not want to be pushed out of their comfort zones.'; a question, in how many instances did you found team rejecting Agile approach?