Everything I learned about technical debt
I just made it to Sweden suffering from jet lag induced insomnia, but this blog post will not cover that. Instead, I will talk a little bit about technical debt.
The concept of technical debt always resonated with me, partly because I always like the analogy with “real” debt. If you take the analogy really far, there are some curious implications. I always like to think of the “interest rate” of software development. Debt is really just borrowing from the future, with some interest rate. You are getting a free lunch right now, but you need to pay back 1.2 free lunches in a few months. That's the interest rate. In a software project the equivalent could be to pick a database that will have scalability issues later, or to make all member variables of some class public. You are doing it because it makes it easier to do things now but you will have to pay the cost of that later.
A recent paper from Google stretches the analogy in its title: Machine Learning: The High-Interest Credit Card of Technical Debt. It focuses specifically on machine learning, but definitely read it if you are interested. A recent blog post challenges if tech debt is really “debt” in the strict sense (you borrow fixed amount and pay back slightly more) or if it has a more complicated structure: Bad code isn't Technical Debt, it's an Unhedged Call Option.
I like the blog post because it brings up something I have noticed many times. A lot of developers have this intuitive aversion towards tech debt and always want to fix anything that's perceived as “hacky”. FooBarController is a 1,000 line mayhem that no one understands, we need to refactor it! But say FooBarController is a well separated component that you have no intent on ever modifying, then there's really no reason to fix it. It's almost always a waste of time to try to fix bad code or bad architecture unless you at least some idea of why it helps you in the future.
So in some cases it makes sense not to fix technical debt. In other cases, it makes sense to take on tech debt deliberately. Back to the interest rate analogy: if the interest rate is lower than the return of investment, you should borrow money from the bank. It's fine to ship a product a year earlier with a hacky code, if you make a lot of money, and hire a ton of developers to clean it up. The concept of interest rate applies both to financing and software engineering.
In my experience, the biggest issues isn't taking on technical debt or not. As long as you make a conscious decision to take on tech debt, and everyone agrees it's tech debt that you might need to fix later, you're in the clear. You will get problems if you build up technical debt without acknowledging it. I made a chart to make it clear:
| Are you taking on tech debt? |
|---|
| Yes |
The bottom left picture is Tony Soprano knocking on your door because he's here to collect the debt you owe him. What happened is, you saw this investment (real estate?) that you thought would appreciate 10% year on year. You borrowed money from Tony, but you never realized you might have to pay it back. It turns out the interest rate was a lot more hefty than you thought, and now he wants it back a year later with 50% interest rate.
The bottom right picture is you going to the bank because you want to buy real estate. You examine the interest rates and make a decision to get a mortgage.
These pictures might not illustrate the point super well, because the bottom right also covers this situation: borrowing at a high interest rate because the return on investment is even higher. Maybe you know of this boxing match that's already rigged, and it's 5:1 odds. You won't be able to borrow money from the bank, so you go to Tony Soprano and borrow it for a few days. Next week, you pay it back with some interest, but you still made a ton of money.
Back to software engineering. The example above is like shipping the v2.0 of your web shop on time, and it turns out to be much better for users. You sell twice as much now! But you also have a bunch of scripts you have to run manually every day. You clearly should automate those scripts later, and it might be really messy to do so, but it's also clear that you can do that later. You made a deliberate decision to borrow some resources from the future, because the return of your investment was really high.