Why organizations fail

One of my favorite business hobbies is to reduce some nasty decision down to its absolute core objective, decide the most basic strategy, and then add more and more modifications as you have to confront the complexity of reality (yes I have very lame hobbies thanks I know).

Software engineers want to deliver features quickly without adding too much tech debt. But what's the point of delivering features? Business value. And what's the point of not adding too much tech debt? It's so that we don't add a bunch of crap that slows down future iteration speed, i.e. we don't want to slow down the future rate of delivering business value (modulo some kind of discounting factor). There is no intrinsic value of beautiful code and no intrinsic value of tech debt. I probably sound like a nihilist, but everything really just boils down to: what's the most efficient way to deliver business value?

So let's start with the assumption that it's everyone's job to deliver business value. Why can't we just make everyone's job to do this? Strip all titles and all management structure – just tell everyone to act in the company's best interest. My dream is that one day I made myself useless. Everyone in the team just comes in every morning and asks themselves: what is the highest ROI thing I can do today? And they just do it. Nothing else is needed. Why can't that just work?

It's of course an incredibly naïve idea – but where does it fail, exactly? It turns out there's in two places. Nothing more, nothing less:

1. Incentive problems (agents do not want to act in the organization's interests)
2. Bounded rationality problems (agents do not have the necessary information to do so)

I can't stress enough that there is no other reason. I had this exact theory for many years but wasn't able to express it in the same eloquent way as above, which is something I copied verbatim from a fantastic paper I just encountered: Why Organizations Fail: Models and Cases.

With the two bullets above our model is basically complete. The beauty of thinking about it this way is that it breaks down management recursively. As a manager the best way to get value out of other people is to:

1. Make sure people's interests are aligned with the company's
2. Make sure everyone in the team has the necessary information they need

Everything else follows. Anything else that 1. and 2. is useless to spend time on as a manager. For instance it explains why micromanaging is useless: all you need to do is give people the information they need to make the right decisions, and reward good behavior. Similarly the model also highlights why it's important to delegate projects with their full context: without enough information people will make the wrong decisions. And it points out that you should reward people for delivering business value: rewarding people for anything else and you misalign people's interests with something else than the company's.

The paper above is pure game theory and I love its mathematical clarity. Of course it's just a model of reality and like any game theoretical model it assumes (incorrectly) that humans are 100% rational. But I think it's a good first order approximation of what to do and it's a good reality check.