Beyond Technical Debt

What happens to code when you don’t refactor?  Anyone with any experience knows the answer.  It gets messy.  It becomes hard to change and the rate at which you can add features slows to a crawl.

In the Agile Development community, we use the term “technical debt” to refer to this problem.  When you don’t keep your code clean, you incur debt, and as in other areas of life debt reduces your options – it can become crippling over time.

More and more, however, I wonder whether there could be a different way of looking at the messy code problem, a way that takes people's behavior into account.

The other day, I saw a news report that offered some advice for people who were cutting back on their spending.  The advice was to throw away your credit cards and use cash.  It turns out that people spend less when they are handing cash over to a cashier. 

When you think about it, it makes sense.  There is something fundamentally different about parting with something valuable that you own.  I have no doubt that credit card companies know this also.  People feel much freer spending money that, technically, isn’t theirs.  And, to the degree that the transaction doesn’t feel “lossy”, people will spend.   If recent history tells us anything, it tells us that in general people just don’t seem to mind accumulating debt.

The idea that I have in mind is to flip technical debt around.  We need to see clean flexible code as an asset and count it as an asset.  When a non-technical person wants something faster, we need to be able to talk about how much of their asset they would be losing if they want a rush job.  They also need to know how much it would cost to restore the asset, and it’s almost always far more than the original development cost.

The devil, of course, is in the details, but I do believe that we’re better off aligning with human nature.  People like to accumulate assets and they hate losing them.  We need to recognize the value of clean, supple code and make sure people understand the business impact of losing it.

Comments

[C. Keith Ray]

One of the common misperceptions of maintainable, clean, supple, testable code is that it is a luxury.

I'm not sure that "code is an asset" is going to combat that notion, since some people would have lots and lots of "cheap" assets rather than a few "expensive" assets.

It's kind of the same attitude that values lots and lots of "cheap" programmers, rather than valuing fewer, more capable, "expensive" programmers. Just about every company _says_ "our people are our greatest asset", but many companies don't act that way.

Perhaps "investments" might work better than "assets".

[Marcel Weiher]

Code is not an asset: http://www.metaobject.com/blog/2008/08/code-is-not-asset.html

[johnstok]

I think most developers would agree that there are two ways to deliver code:
* The right way (TDD; RedGreenRefactor; with documentation; etc.)
* The fast way (hack something together and get it out the door).
The latter will incur a more significant technical debt.

To explore your analogy further, what if we give the technical team the role of 'creditor' and the product manager the role of 'debtor'. When the product manager asks for a feature 'the fast way'. The technical team can allow this 'credit' but with the proviso that at some point in the future the product manager must repay the credit. The interest on the credit maps nicely to the fact that any additional work now undertaken is more expensive because developers must work with a sub-optimal code-base.

This state of affairs will break most teams for the following reasons:
1. The technical team fails to record the 'technical credit' that they have afforded the product manager. Thus it is forgotten _and never repaid_.
2. Unlike with a bank, the more credit that is given the higher the rate of interest should become. In other words, the more often that we choose 'the fast way' the more expensive it becomes to add any feature to the code-base.
3. Even if the technical team does record the credit it is typically very difficult to make the product manager pay the credit back.

[Christian Romney]

The problem is that code is intangible to most everyone but the programmer. It's much easier (psychologically) to compromise its integrity and quality and take on debt because the mess lacks visibility.

[Mike Bria]

[Re: above comments, Product Owner as "debtor"]

The "right way vs fast way" take by 'johnstok' implies that programmers allow accumulation of messy code largely to satisfy the PM's requests for "faster"; that programmers know better, but choose on a case by case basis to incur debt or not.

In my experience, this isn't really the case. Most programmers either write clean code all the time (or at least try), or they don't.

Sure, ignoring good coding practice is sometimes "faster" (in the short term), but it's also simply wrong.

Programmers are responsible for the state of the code, just as I am for the balance of my credit card. Calling the Product Owner the "debtor" is somewhat akin to me blaming the advertising companies for my credit card problems - it's denial.

[Peter Ritchie]

I agree. "Technical Debt" provides a crutch for members of the dev team to provide an "out" for stakeholders. Stakeholders can't conceive of tangible consequences when they're presented with "technical debt" as a consequence.

But, I think it's wrong to communicate "Technical Debt" beyond the dev team. Devs shouldn't rely on these buzzwords and should accurately communicate tangible consequences to the stakeholder. Sometimes this requires some research and thought.

Rather than "Technical Debt" what is often communicated to stakeholders are equally intangible; phrases like "increased fragility", "increased rigidity". "lower quality", "reduced responsiveness to requirements changes", etc.

[William Pietri]

Interesting article, Michael.

I'd always thought of code as an asset, and the debt as modifying the value, like damage to a car, or repairs needed on a house. When the debt is too great, I refer to a code base as bankrupt or, like a car, totaled. Like many others I explain increased bugs and slower development as interest: an ongoing cost you are compelled to pay until you get rid of the debt. You can also talk about particular areas of the code base as totaled, requiring rewrites.

I think an accountant would have some quibbles, but it seems to give people a pretty good intuitive grasp.

To make the debt more visible, a lot of the teams I coach have part of a whiteboard where they list known technical debt. And if a team ever decides to explicitly take on debt, I insist that they create a card for the payback and put it in the backlog.

Credit where due: I think Ward Cunningham is the originator of the "technical debt" metaphor.

[Erich von Hauske]

Just from the top of my head:

What about having a "Global Corruption Factor" and an "Application Corruption Factor", you'll use them when giving estimates so your estimate will be:

Effort = "Estimated Effort" * ("Global Corruption Factor" + "Application Corruption Factor") / 2

Both factors start in 1 and each time one stakeholder wants you to incur in technical debt you increase both factors. How much? tricky question. Again, from the top of my head, in the proportion of Lines Of Code affected, yes you read correctly LOCs!, jejeje. So if you affect 100 LOCs and yur application code base is 10,000 LOC and the entire company's code base is 600,000 LOC you'll increase your Application Corruption Factor in .01 and your Global Factor in .000167

You should affect your SLAs/OLAs in the same way.

Now you have 2 assets that stakeholders value and can understand: Development response times and SLAs/OLAs. Just be sure to have all these Debt Agreements and remember that you are co-signing it.

Probably there are better names, but no being a native English speaker really constrains me on that.

And the always important questions: is this a practical approach? does it seems like it should work? is there a more efficient way to get the stakeholders to realize the consecuencies of Tech Debt?

[Erich von Hauske]

A few more points:

1. Don't count test code lines.
2. I think it should be easier for the user to understand all this after he has his requirement completed, so he doesn't feel extortionated by a Corruption Nazi. Team work should be the first option.
3. The formula needs some work as the cost doesn't grow linearly, it grows exponentially?

[Mark Nijhof]

I think in different scenarios there are different reasons for allowing technical debt:

I know a situation where software projects are delivered to clients whom also have a maintenance agreement. There the actual quality of the delivered software is quite low, but the service provider figures that when the client needs something changed they will have to pay for these changes. So if I am a cynical person I would even go as far as saying, delivering bad quality will earn the service provider more money in the long run. Of course this is in the very long run not the best strategy to play, but hey they deal with that later. So the client wasn’t told.

Then there is the wanting to win a contract and because of that having very low estimates, of course the contract states the usual asserts about quality so when the potential client sees these offers they will most likely pick the cheapest one if both offer the same functionality. Here again a case where the client wasn’t told.

With internal development this is a different case, here the developers (if they know) will speak up and tell their boss or PM that if they do this and that in a ‘faster’ way that this will mean that at a later date it needs to be corrected. They do this because it has bitten them in the *** before so they learned to cover their own ***. Here the problem probably lies in the pressure of bringing something to market, everybody knows that it isn’t pretty and needs to be fixed in the near future, but hey it is a new release and before the competition. So they were told, but perhaps not what is would actually cost them in the long run.

So could we say that we are only living in the now, and we miss the capability to look further into the future? And even if we see far enough into the future to see the collision about to happen, we tend to ignore it, think that we will deal with that when the time comes. So I agree we need to change the behavior, and I believe that we need to change the expectations that our stakeholders have from software development. Isn’t it weird that a client expects that a software development project will run over time, will costs more than estimated and perhaps not even do what was needed?

But I am not sure that putting the emphasis on the value they lose by choosing the shortcut will do it. It wasn’t there to begin with; I have to pay more to get more value, right? In many cases they don’t even get to make the decision with all the facts presented to them.

In the end I think it is about trust, our stakeholders need to trust us to deliver the appropriate quality, and we need to trust them that when they tell us to take a short-cut that we get the opportunity to fix this later on. So maybe before that trust is there we have to put in writing (a backlog item or something like that) the extra work associated with bringing back the quality in the system after having taken this short-cut. And add it to the backlog straight away, that way it needs to be paid right then and there (when the decision is made) the only thing they then influence is the initial delivery date, with a clear indication that this will cost them x amount more?

(sorry for the long comment without really saying anything)

[Dan Bergh Johnsson]

If a "rush job" means you loose value in the asset, then logically there will be a point when the asset has no value any longer - the point of bankrupt. The code analogy of this would in that case be when there are no "structural quality" left in the code-base - meaning you could equally well throw it away and start over. This is well in line with my experience.

[Paul]

It's a valid line of reasoning — code is indeed a liability, especially bad code.

Consider, however, that very few real businesses function without debt. Even businesses that are not highly leveraged still rely on a line of credit or some similar structure to smooth out the bumps of week-to-week cash flow.

In the same way, maintaining a low level of technical debt — for prototyping, trying out an approach, getting a first release in front of users who can comment on its usability — is essential to getting software projects done fast.

The trick for software engineers is to manage technical debt wisely. That involves communicating its costs to the non-engineers, and not letting costs run out of control.

Agile developers often fall into a trap of purity at the expense of practicality. Zealously eliminating all technical debt would fall into that category.