Writing code for a computer is hard enough. You take something big and fuzzy, some large vague business outcome you want to achive. Then you break it down recursively and think about all the cases until you have clear logical statements a computer can follow. Computers are very good at following logical statements.
Read more…As I am en route to see my first total solar eclipse, I was curious how hard it would be to compute eclipses in Python. It turns out, ignoring some minor coordinate system head-banging, I was able to get something half-decent working in a couple of hours.
Read more…CIA produced a fantastic book during the peak of World War 2 called Simple Sabotage. It laid out various ways for infiltrators to ruin productivity of a company. Some of the advice is timeless, for instance the section about “General interference with Organizations and Production”:
Read more…Long story short: I’m working on a super cool tool called Modal. Please check it out — it lets you run things in the cloud without having to think about infrastructure. Scaling out, scheduling, containerization, using GPUs, setting up webhooks, and all kinds of other stuff. It’s primarily meant for data teams. We aren’t quite live, but you can sign up for our waitlist.
Read more…This is is in many respects a successor to a blog post I wrote last year about what I want from software infrastructure, but the ideas morphed in my head into something sort of wider.
I encountered AWS in 2006 or 2007 and remember thinking that it’s crazy — why would anyone want to put their stuff in someone else’s data center? But only a couple of years later, I was running a bunch of stuff on top of AWS.
Read more…Hi! It’s your friendly project management theorician. You might remember me from blog posts such as Why software projects take longer than you think, which is a blog post I wrote a long time ago positing that software projects completion time follow a log-normal distribution.
Read more…
Here’s a theory I have about cloud vendors (AWS, Azure, GCP):
We currently have cloud vendors that offer end-to-end solutions from the developer experience down to the hardware:
Read more…This isn’t as much of a blog post as an elaboration of a tweet I posted the other day:
Read more…I think this specialization of data teams into 99 different roles (data scientist, data engineer, analytics engineer, ML engineer etc) is generally a bad thing driven by the fact that tools are bad and too hard to use
I guess I should really call this a parable.
The backdrop is: you have been brought in to grow a tiny data team (4 people) at a mid-stage startup ($10M annual revenue), although this story could take place at many different types of companies.
Software infrastructure (by which I include everything ending with *aaS, or anything remotely similar to it) is an exciting field, in particular because (despite what the neo-luddites may say) it keeps getting better every year! I love working with something that moves so quickly.
Read more…