TL; DR: Technical Debt & Scrum
If technical debt is the plague of our industry, why isn’t the Scrum Guide addressing the question of who is responsibly dealing with it? To make things worse, if the Product Owner’s responsibility is to maximize the value customers derive from the Development Team’s work, and the Development Team’s responsibility is to deliver a product Increment (at least) at the end of the sprint adhering to the definition of “Done,” aren’t those two responsibilities possibly causing a conflict of interest?
This post analyzes the situation by going back to first principles, as laid out in the Scrum Guide to answer a simple question: Who is responsible for keeping technical debt at bay in a Scrum Team?
🗞 Do you want to get this article in your inbox in the future? You can sign up here for our weekly ‘Food of Agile Thought newsletter’ and join 29k other subscribers.
What Is Technical Debt?
According to Wikipedia,
“Technical debt (also known as design debt or code debt) is a concept in software development that reflects the implied cost of additional rework caused by choosing an easy solution now instead of using a better approach that would take longer.”
The issue is that there is not just the typical hack, the technical shortcut that is beneficial today, but expensive tomorrow that creates technical debt. (A not uncommon tactic in feature factories.)
There is also a kind of technical debt that is passively created when the Scrum Team learns more about the problem it is trying to solve. Today, the Development Team might prefer a different solution by comparison to the one the team implemented just six months ago. Or, the Development Team upgrades the definition of “Done,” thus introducing rework in former product Increments.
No matter from what angle you look at the problem, you cannot escape it, and Scrum does not offer a silver bullet either.
Technical Debt & Scrum — The Scrum Guide
First of all, the Scrum Guide does not mention technical debt.
According to the Scrum Guide:
- The Product Owner is responsible for maximizing of the value of the work of the Development Team.
- Product Owners do so by managing the Product Backlog, visible in its content and ordering of Product Backlog items.
- The Product Backlog represents the only set of requirements the Development Team will work on and shall comprise of everything the product requires to be valuable.
- The Scrum Team never compromises on quality.
- The definition of “Done” is either provided by the engineering organization or the Development Team.
- During the Sprint Planning, the Product Owner discusses Product Backlog items suitable to achieve the Sprint Goal.
- However, only the Development Team picks the Product Backlog items it deems necessary to achieve the Sprint Goal.
- If necessary, the Development Team adds new Product Backlog items to the Sprint Backlog as more is learned.
- If the Development Team improves the definition of “Done,” rework of former product Increments may become necessary.
Therefore, I believe the Scrum Guide is deliberately vague on the question of who is responsible for the technical debt to foster collaboration and self-organization, starting with the Scrum values–courage, and openness come to mind—leading straight to transparency and Scrum’s inherent system of checks & balances.
Cannot see the subscription form?
Please click here
How to Deal with Technical Debt as a Scrum Team
I am convinced that dealing with technical debt should be a concern of the whole Scrum Team. There are a few proven techniques that will make this task more manageable:
- Be transparent about technical debt. Visualize existing technical debt prominently so that everyone is constantly reminded of the nature of your code-base. Also, address the technical debt at the Sprint Review events regularly so that the stakeholders are aware of the state of the application.
- Use code metrics to track technical debt, for example, cyclomatic complexity, code coverage, SQALE-rating, rule violations. (There are numerous tools available for that purpose.) At least, count the number of bugs.
- Pay down technical debt regularly every single sprint. Consider allocating 15 to 20 percent of the Development Team’s capacity each Sprint to handle refactoring and bug fixing. (Learn more: Scrum: 19 Sprint Planning Anti-Patterns.)
- Make sure that all tasks related to deal with technical debt are part of the Product Backlog—there is no shadow accounting in Scrum.
- Adapt your definition of “Done” to meet your understanding of product quality, for example, by defining code quality requirements that contribute to keeping technical debt at a manageable level in the long run.
- Create a standard procedure on how to handle experiments that temporarily will introduce technical debt to speed up the learning in a critical field.
In my experience, dealing with technical debt becomes much simpler when you consider transparency to be the linchpin of any useful strategy.
Technical Debt & Scrum — Conclusion
Given that solving a problem in a complex environment always creates new insights and learnings along the way that will make past decisions look ill-informed, creating technical debt is inevitable.
Handling technical debt, therefore, requires a trade-off. The long-term goal of achieving business agility cannot be reached as a feature factory churning out new features. At the same time, an application in a perfect technical condition without customers is of no value either.
Consequently, dealing with technical debt in Scrum is a responsibility for the Scrum Team as a whole and as such an excellent example of Scrum’s built-in checks & balances.
How are you handling technical debt in your daily work? Please share with us in the comments.
📺 Join 1,175-plus Agile Peers on Youtube
Now available on the Age-of-Product Youtube channel:
📖 Technical Debt & Scrum — Related Articles
Martin Fowler: Technical Debt.
Ward Cunningham: 📺 (Technical) Debt Metaphor.
📅 Scrum Training Classes, Workshops, and Events
You can secure your seat for Scrum training classes, workshops, and meetups directly by following the corresponding link in the table below:
See all upcoming classes here.
You can book your seat for the training directly by following the corresponding links to the ticket shop. If the procurement process of your organization requires a different purchasing process, please contact Berlin Product People GmbH directly.
✋ Do Not Miss Out and Learn more about Technical Debt and Scrum: Join the 12,000-plus Strong ‘Hands-on Agile’ Slack Team
I invite you to join the “Hands-on Agile” Slack team and enjoy the benefits of a fast-growing, vibrant community of agile practitioners from around the world.
If you like to join now all you have to do now is provide your credentials via this Google form, and I will sign you up. By the way, it’s free.
View Comments (12)
I fully agree that technical debt (TD) has to be presented during sprint review. Should I describe TD full life cycle, I would say :
Main 2 root causes with 2 distinct life cycles:
1)priority to business feature delivery against quality code dev and architecture
2) evolution that requires to review previous code.
Identification steps
In theory, in both root case, TD are detected before causing issue
1) during sprint poker planning because there will be 2 opposite estimations, a quick and dirty one & a longer but clean one. In this perfect world, a TD user story is initiated in the backlog and will be handle later one.
2) also during sprint poker planning because there will be a heavy estimation for the new user story , or there will be to 2 opposite estimations, one forgetting the TD side effect & one taking into account the TD side effect. Case 1 can apply at this time for this new user story.
Execution phase : Once in the backlog, it follows usual process
None Nominal case : It's none nominal when it is not detected during the poker planing
Even with a dream team, it happens that TD are not identified so early in the process.
In such case, if you listen carefully, first signs appear during daily meetings through when dev team will talk about problem. As SCM, when I have such feeling, I ask the dev team if it could be due to a TD. Doing so minimize the impact of the TD by detecting the cause early.
Worst case, you detected it because some US take far more time than expectation or worst because some US estimation are infinite (= longer that 1 sprint to be deliver), while they should not. In such case, 99% chance you have a TD, a bid & deep one !
Identification is a key issue of TD. It can jeopardize the scrum team mood as well as Scrum image because efficiency will decrease little by little, but surely.
TD identification is not natural nor easy because technic is a mean not an objective,
The whole team is involved to handle TD, but I think the SCM is the best person to think about it and help the team to identify when there is TD. Once identified, it becomes rather PO and dev team responsibility to manage it.
Feel free to comment.
As soon as TD is detected, fog disappears, sun shines again :-)
I suggest moving away from the term "Technical Debt" and instead refer to it as "Technical Health". Many of us accept "debt" in our daily lives. We borrow money to buy a home. Countries are running up tremendous debts to booster their economies or provide national security. On the other hand, Technical Health implies we are concerned about our well being. We understand that if we have a disease that it potentially will get worse, and we will become more ill over time. We understand the importance of taking care of ourselves if we are to maintain our health over time..
Please view my blog posting at http://www.agile2success.com/2017/12/18/technical-health-for-the-new-year/ for more about this topic.
One critical aspect related to technical debt, even if fully visible in the backlog, is how to properly articulate the business value of paying it off relative to the value of new features. When technical debt stories go head to head with other user stories, often those win because their value is often more easily understood.
Absolutely! Being a feature factory ignoring technical debt is an unsmart decision in the long run. Being focused on a debt-free code base is often similarly problematic as the return on investment is questionable. The most successful companies manage to find the right equilibrium between those two forces.
One of the biggest recurring issue I'm seeing is the shadow accounting as mentioned in this article. It causes issues in multiple ends. I'll not talk about the pain points but I'll briefly mention a very easy , quick solution that would help to realize the problems.
What we're currently experimenting is introducing Developer and System as personas. A sample US,
"As a developer I would like components to be in story book so that it would be easier to reuse them"
"
React storybook is a tool for writing styleguides.
Background
We have started writing more and more components and it's important to be able to keep track of them, not to write the same things twice.
Benefits
helps new developers to use the existing components faster
makes the components more encapsulated
fascilitates the communication with the designers
helps to keep the design consistent
Acceptance criteria
All X components are put in storybook
Documentation written how to use the storybook
Notes/Questions/Uncertainties
No components UI tests (storyshots)
No deployment of storybook"
and it goes through the flow that a feature request would go. Story has to be clear, refined, discussed, estimated, and prioritized by the Product Owner. Creating such channel would align the team members in any time to put their hidden ideas to one and only backlog. Also because it goes to through the same flow, the technical only things would be easily readable by anyone in the project. Also story splitting can be applied to limit the improvement at one time, and do the iterations.
I agree with dealing with it every sprint within some percentage levels as this is one of the strategy. Also what I see helpful in case of huge technical debt is dedicating more time, or dedicating a sprint. When it comes to the business value, well, the PO would not have a business if the system fails :)
I don't really think that the Product Owners only care about the features and nothing else when they're working with the team. The issue is they don't have enough visibility and control over the technical tasks. When they would see, for example 20 backlog items, which are all understandable by him/her with the goals and the impacts, they would be the one who would arrange time/resources in roadmap for that. Also because of the lack of visibility and control, frustrations, and lack of trust happen which leads to a harder situation for dealing with the actual work - dealing with technical debt.
One last thing from this approach is that it would show the impacts of the things done earlier. What I mean, PO would/should know if they would get more users the amount of reported bugs might increase, or amount of feature requests. It's a good thing because the business is growing. But, while doing that, The system and The developer personas would also generate requests because they are in the same environment, and they definitely shouldn't be forgotten. And this all eventually would educate the PO with foreseeing the technical debt as it's giving transparency and control earlier.
So, to sum up, in my opinion the real issue is communication. And improving communication practices for technical tasks would ease many things.
I am convinced that there are many forces out there which drive technical debt taking. My observations of the organisational dynamics leading to a technical debt have inspired me to design a board game which explains the related issues to non-technical staff. This game has been the technical debt eye-opener at c-level ever since. I am sure we need 3 things to do in order to tackle the issue: explain, explain, explain
While it would be helpful if the scrum bible, the Scrum Guide, addressed technical debt, perhaps even more powerful, the product management bible, Marty Cagan's Inspired: How To Create Products Customers Love, does address it. Inspired (p.28) says, "The deal with engineering goes like this: Product management takes 20% of the team's capacity right off the top and gives this to engineering to spend on... whatever they believe is necessary to avoid ever having to come to the team and say, 'we need to stop and rewrite'.... I get nervous when I find teams that think they can get away with much less than 20%." Cagan's comments TL;DR? If you're not spending 20% at least on paying down technical debt, you're marching toward technical bankruptcy.
I totally agree, Ron!
Joshua adds per email:
Stefan – for your "related articles" I recommend a nod to the book "The Art of Agile Development" by James Shore and Shane Warden, which has a lot to stay about tech debt.
For example…
https://www.jamesshore.com/Agile-Book/done_done.html
https://www.jamesshore.com/Agile-Book/collective_code_ownership.html
https://www.jamesshore.com/Agile-Book/refactoring.html
I would not talk about rework due to quick but bad coding. This is not a good practice , nether in V project nor in SCRUM. The solution as a SCM in that case is to change or educate the team.
I'm both an architect for decades and a SCM for nearly 4years now. My architecture job & skills help me understanding source of technical or functional debt and the key is there :
DEPENDENCY and STRUCTURAL CHANGE.
When user stories are independent from each others, you don't have debt. You just give up older stories and develop new one. When you have dependencies, then you start having large blocks of code having a structural organization (an architecture) and when a change happens that required a change of the previous architecture, all of a sudden you have a large debt. I have not other solution than understanding the overall impact and reviewing all the all technical of functional blocks that need to be rethink and re coded.
This happens for example when the initial backlog was done for one type of customer and is extended to several types. All what was done for one required to be able to support N. In such case, we freeze new US until the change is done -properly -. Spreading such change on several sprints leads to far more issues and delay than focusing on the debt recovery. Should the PO really requires to develop new US, I dedicate 1 ou 2 dev max for the new US and the rest of the team for the debt recovery. I do NOT change the team focusing on the debt until it is recovered.
Because Agile is responsible and especially Scrum. When you have iterations smaller than 6 weeks which Scrum prescribes is hard to make a coherent arhitecture. Also the way this is tested means that bugs that appear due to requirement dependencies go unnoticed until the release to final customer.
A significant source of technical debt are third-party dependencies going end of life.
These are not by-products of speed vs quality, but still part of debt.