Storm in the stratosphere: how the cloud will be reshuffled

cloud atmosphere layers

Here's a theory I have about cloud vendors (AWS, Azure, GCP):

  1. Cloud vendors will increasingly focus on the lowest layers in the stack: basically leasing capacity in their data centers through an API.
  2. Other pure-software providers will build all the stuff on top of it. Databases, running code, you name it.

We currently have cloud vendors that offer end-to-end solutions from the developer experience down to the hardware:

cloud aws stack

What if cloud vendors focus on the lowest layer, and other (pure software) vendors on the layer above?

cloud new stack

Feel free to bring this up in five years to make me embarassed about how wrong I turned out to be. But let me walk you through my thinking—I think some of it is quite well illustrated through the story of Redshift.

Redshift and what happened

Redshift is a data warehouse (aka OLAP database) offered by AWS. Before Redshift, it was the dark ages. The main player was Teradata, which had an on-prem offering. Startups said no to SQL and used Hadoop—SQL was kind of lame back then, for reasons that in hindsight appear absurd. I'm very happy we're out of this era.

Anyway, one vendor was a company called ParAccel. AWS licensed their technology, rebranded it Redshift, and launched in 2012.

Redshift at the time was the first data warehouse running in the cloud. It was a brilliant move by AWS, because it immediately lowered the bar for a small company to start doing analytics. You didn't have to set up any infrastructure yourself, or write custom mapreduce and reload the jobtracker all day. You could spin up a Redshift cluster in AWS, feed it humongous amounts of data, and it would … sort of just work.

Enter Snowflake

Snowflake1 is a $100B+ publicly traded company. Basically their entire offering is a data warehouse that looks fairly similar to Redshift.2

If you looked at Redshift in 2012, there was a lot of things that favored it. AWS had large economies of scale, had control of the underlying substrate (EC2), and could make larger investments in building the software. Maybe because of the value of lock-in, they could even subsidize the development of Redshift and make up the money through other products. Anyway, this is what it looks like nine years later3:

cloud db ranking

What happened? But my more general question is: what are the forces that favor a company like Snowflake? And what does that mean for other cloud products?

What if….?

There's some sort of folk wisdom that a lot of cloud services are pure commodity services, and that the only way for a cloud vendor to make money is to build a bunch of high-margin software services on top of it.

I'm not sure it even passes a basic smell test? EC2 margins are roughly the same as overall AWS margins4. So at least there's something counteracting this theory. What I think is happening:

  • The “software layer on top” is getting incredibly competitive. There's so many startups going after it, fueled by cheap VC money and willing to burn billions of dollars on building software.
  • Cloud vendors might be pretty happy making money just in the lowest layer. Margins aren't so bad and vendor lock-in is still pretty high.

Startups are coming for the cloud

There's never been this many companies going after services that traditionally belonged to the cloud vendors:

cloud services

What's going on? Probably a confluence of a lot of things. If I was tired, I would just shrug and say something like “startup circle of life, whatever”. But let's elaborate:

  • The developer experience has been an attack vector for a while. I personally have many bones to pick here—just as an example I just wasted a few hours last week debugging S3 bucket policies. Some of the cloud stuff has become extremely complicated (not to name any names, but… IAM).
  • Software vendors can build for all the cloud vendors at the same time. I think this was a real benefit for Snowflake, since a lot of their early customers were banks who care about multi-cloud, but more generally it also expands the market size vs the reach of any cloud vendor.
  • VCs are pouring in money into the segment. If you're an ambitious person, do you go work at AWS? Or do you join an early stage startup, or create your own?

Maybe owning the lowest layer isn't so bad?

Let's say a customer is spending $1M/year on Redshift. That nets AWS about4 $500-700k in gross profits, after paying for EC2 operational cost and depreciation. If that customer switches their $1M/year budget to Snowflake, then about $400k5 goes back to AWS, making AWS about $200k in gross profits.

That seems kind of bad for AWS? I don't know, we ignored a bunch of stuff here. Snowflake's projected6 2022 research and development costs are 20% of revenue, and their sales and marketing costs are 48%! For a million bucks revenue, that's $700k. Translated back to AWS, maybe AWS would have spent $300-400k for the same thing? Seems reasonable.

Now the math suddenly adds up! AWS basically ends up with the same bottom line impact, but effectively “outsources” to Snowflake all the cost of building software and selling it. That seems like a good deal for them!

What about lock-in?

The other argument for why AWS should build their own software services is that it increases lock-in. So maybe Redshift in itself isn't a cash cow, but it decreases the churn on EC2.

I'm not so sure? I spent six years as a CTO and moving from one cloud to another isn't something I even remotely considered. My company, like most, spent far more money on engineer salaries than the cloud itself. Putting precious eng time on a cloud migration isn't worth it unless you're operating at a very extreme scale. Which some companies do! But those are in a minority.

A significant factor in all of this is that existing infra provides significant “gravity”. It's not like you can just pick whichever cloud vendor has the cheapest database and run your DB there. You want to run things in the same data center7. When you sign up for any of these vendor, they standard question is (a) what's your cloud vendor (AWS/GCP/Azure) (b) what region?

cloud sign up flows

The other question is—how much money you can save? And I think the truth it's never going to be substantial, since no one wants to start a price war. Being in a fairly stable oligopoly seems pretty nice and cozy, and if I was cloud vendor, I wouldn't try rock the boat.

Startups are coming for your code

This trend has been the clearest when you look at Snowflake vs Redshift, but you can see it in other places too. Transactional databases is another very exciting area. But where I think we'll see the most change is how software vendors will increasingly run customer code.

This isn't exactly a new idea—Heroku launched in 2007, and AWS Lambda in 2014. Kubernetes has been this interesting trend in the last few years on what I think is still essentially an inevitable march towards a fully “serverless” world, whatever that means to you.

I think the battlefield here is a lot more complicated, for at least a few reasons.

  • I generally think function-as-a-service (FaaS) offerings are sufficiently different from running data centers and it seems like software vendors will go after this segment too. But Netlify and Vercel offer cloud functions that use AWS Lambda under the hood! My guess is this is not the long-term equilibrium.
  • The CDN world seems like where the cloud was 5-10 years ago: and companies like Cloudflare own the whole thing from the developer experience to operating CDN. Cloudflare is clearly kicking ass right now in terms of delivering on developer experience. But I'm not sure if there's any foundational difference between operating 300 small data centers or 30 large data centers? I could see Cloudflare going in a similar direction as what I'm projecting for the cloud vendors.

The cloud in 2030

We're roughly 10 years into the shift to the cloud, and even though it feels like it's transformed how we build software, I think we're still just getting started. I really wouldn't be surprised to wake up in a world where most developers don't interact with cloud vendors.

Big transformations tend to happen in two stages. The first step happens when some new technology arrives and people adopt to it in the simplest way that lets them retain their conceptual models from the existing world. The real transformations happens later, when you rethink the consumption model because the new world opens up new ways to create value. The way we consumed music didn't change materially when Apple started selling songs online. The real transformation happened when providers like Spotify realized the whole notion of ownership didn't matter anymore.

If you think about it from that angle, the last 10-15 years look a bit like a dumb “lift and shift”. Crudely speaking, we took computers and we put them in the cloud. Is that really the right abstraction for where we are? I don't think so. I think new higher level tools will let us focus on building application code and not worry about the underlying infrastructure.

I wonder if a weird corollary of this is… maybe it's actually really good for the planet? The computers sitting in the cloud are ridiculously underutilized—my guesstimate of average CPU utilization is that it's maybe 10%? If I had a Ph. D. in bin packing, I'd go looking for a job at some serverless infrastructure provider right now.

One way to tell the story in 2030 looking backwards is that the cloud vendors needed software running on top of it, so they had to build it themselves first, in order to drive cloud adoption. But eventually the market matured, and they could focus on the place in the stack where they had the strongest advantage.

Predictions

  • The cloud market will grow to $1T/year in revenue. Ok, that's almost entirely noncontroversial.
  • The core three cloud vendors will see somewhat slower revenue growth, but faster earnings growth.
  • Most engineers will never interact directly with cloud vendors, but through services on top of those.
  • The database market (OLAP, OLTP, you name it) will be dominated by vendors running on top of cloud vendors, where the underlying layer is completely abstract.
  • We will have some amazing new runtimes, finally figuring out the developer experience problems that are currently holding serverless solutions back.
  • Kubernetes will be some weird thing people loved for five years, just like Hadoop was 2009-2013, but the world will move on.
  • Resource utilization in the cloud will be much better, and engineers will spend an order of magnitude less time thinking about resource allocation and provisioning.
  • IBM has finally given up on “hybrid multi-cloud”
  • YAML will be something old jaded developers bring up after a few drinks.

Thanks to Josh Wills for feedback on an earlier version of this post!


  1. A fun thing is I randomly had lunch with the Snowflake founders in 2012 and they offered me a job the next day. The company was like… 10 people in total? ↩︎

  2. There was one major architectural difference of Snowflake vs Redshift. A very underappreciated tech shift is how much faster networks got around that time. Up until that point, the wisdom was “move compute to data”, but Snowflake bet early on a full decoupling. Redshift later launched Spectrum, but IMO they should have just gone all-in on it. ↩︎

  3. This is from DB rankings. ↩︎

  4. The margins on EC2 is reportedly about 50% and about 60% for AWS as a whole. Just comparing instance pricing for Redshift and for EC2 for equivalent instance types also gives a good idea of the Redshift markup above EC2. ↩︎

  5. Snowflake is a public company and you can compute their margins from their latest quarterly report. ↩︎

  6. From Snowflake's investor presentation ↩︎

  7. Cloudflare is actively going after the very high AWS egress fees and it's going to be interesting to see how that plays out. ↩︎