The mathematical principles of management

The eigenvector of "Why we moved from language X to language Y"

Why I went into the mortgage industry

I just realized last Thursday that I have spent two full years at Better, incidentally on the same day as we announced a $15M round led by Kleiner Perkins. So it was a good point to reflect a bit and think back – what the F led me to abandon my role managing the machine learning team at Spotify? To join some random startup in the world’s most boring industry? So here’s my justification why I love being where I am:

Language pitch

Functional programming is the libertarianism of software engineering

• Both ideologies strive to eliminate [the] state. (ok, dumb dad joke)
• Both ideologies are driven by a set of dogmatic axioms rather than a practical goal:
• Libertarianism wants to reduce the government because any involvement distorts free markets. I always struggled to see what the underlying objective function is (it doesn’t seem to be maximization of people’s utility). 🤔
• Functional programming wants to reduce side effects and make everything pure, often by enforcing onerous type systems. But why? Again I don’t see an ultimate objective here. IMO it should start from the principle that the goal of a programming language should be to make the programmers as productive as possible. For instance, the little research that exists has shown that most bugs have little to with typing and I’d expect something similar to apply to mutable state. In fact the largest class seems to be poor error handling (ok, typing isn’t necessarily related to FP, but in practice I find that strong typing and FP have highly overlapping fan clubs).
• Both camps invoke obscure cases in history as a proof of success: libertarianists (more so anarchists I guess) often talks about Spain during the civil war, Somalia, or sometimes Singapore. Haskell acolytes are very eager to bring up Facebook’s spam filtering.
• YET – and this is the surprising part imho – both ideologies are ~90% correct (source: my opinion). Which really surprises me given that they start from a (imo) arbitrary set of axioms.
• Even if you are a die hard bolshevik, you benefit from an understanding of how interventions distort markets, how incentives matter, and how entrepreneurship is the driver of progress.
• Even if you are coding in Visual Basic, you can level up your skills by learning FP: making functions pure when needed, avoid state, avoid reassign variables, avoid mutable data structures, write pipelines of data transformations, and all that jazz that FP has taught us to cherish.

The half-life of code & the ship of Theseus

As a project evolves, does the new code just add on top of the old code? Or does it replace the old code slowly over time? In order to understand this, I built a little thing to analyze Git projects, with help from the formidable GitPython project. The idea is to go back in history historical and run a git blame (making this somewhat fast was a bit nontrivial, as it turns out, but I’ll spare you the details, which involve some opportunistic caching of files, pick historical points spread out in time, use git diff to invalidate changed files, etc).

Are data sets the new server rooms?

This blog post Data sets are the new server rooms makes the point that a bunch of companies raise a ton of money to go get really proprietary awesome data as a competitive moat. Because once you have the data, you can build a better product, and no one can copy it (at least not very cheaply). Ideally you hit a virtuous cycle as well, where usage of your system once it takes of gives even more data, which makes the system even better, which attracts more users…

Pareto efficency

Pareto efficiency is a useful concept I like to think about. It often comes up when you compare items on multiple dimensions. Say you want to buy a new TV. To simplify it let’s assume you only care about two factors: price and quality. We don’t know what you are willing to pay for quality – but we know that everything else equals:

State drift

When machine learning matters