Luigi conquering the world
I keep forgetting to buy a costume for Halloween every year, so this year I prepared and got myself a Luigi costume a month in advance. Only to realize I was going to be out of town the whole weekend. If anyone wants a Luigi costume, let me know!
(I'm not as big as the guy in the picture)
Anyway, that's not the Luigi this blog post is about. This is about the Python workflow manager that I've open sourced. If you're putting together batch jobs into complex pipelines, you should check it out.
There's been a couple of companies in NYC using it for a while, eg. Foursquare, Bitly, and Mortar Data. The latter company even provides a hosted Luigi solution.
Lately, there's been a surge of contributors (at the time of writing there's 78 contributors, which is pretty cool!). People at Houzz has sent tons of pull requests, and they are not the only ones.
Two blog posts just came out, both describing their use of Luigi (among other things)
- Buffer's New Data Architecture
- Asana: How to Build Stable, Accessible Data Infrastructure at a Startup
In addition, I found this interesting deck from Stripe:
There's also a couple of meetups in the past and the future. There was a dedicated Luigi meetup on July 31 at the Spotify NYC office, also sponsored by Run Ads who use Luigi a lot. Here are my slides about Luigi's future, heavily delayed:
I'm also talking about Luigi at the NYC Data Science meetup on Dec 19 (also at Spotify's NYC office). Feel free to drop by and ask me some tough questions!