I was just at the NYC Predictive Analytics meetup talking about how we build machine learning algorithms using Hadoop to power music recommendations.
Great meetup, where we had two speakers, me and Blake Shaw from Foursquare.
Read more…
I thought this article about the company culture at HubSpot is kind of funny. “HubSpot's Awesome Presentation Shows how to Create a 21st Century Culture”.
Just FYI: You're not different. You're a bunch of white hipsters aged 25-30 dressed up in the same theme.
Read more…
I was in Portland, OR for a few days hanging out at OSCON. Was fun. I also talked a bit about Luigi:
Next week I'm presenting at the NYC Predictive Analytics meetup together with Blake Shaw from Foursquare.
Read more…
Sometimes you have to maximize some function $$ f(w_1, w_2, ldots, w_n) $$ where $$ w_1 + w_2 + ldots + w_n = 1 $$ and $$ 0 le w_i le 1 $$ . Usually, $$ f $$ is concave and differentiable, so there's one unique global maximum and you can solve it by applying gradient ascent.
Read more…
Continuing in the same spirit of shameless self-promotion, here's some recent Luigi press:
Reddit thread A Guide to Python Frameworks for Hadoop (slides from the NYC Hadoop User Group) This presentation from the Open Analytics NYC meetup about how Foursquare uses Luigi Luigi is in the middle of a pretty massive refactoring of the visualizer.
Read more…
Just open sourced hdfs2cass which is a Hadoop job (written in Java) to do efficient Cassandra bulkloading. The nice thing is that it queries Cassandra for its topology and uses that to partition the data so that each reducer can upload data directly to a Cassandra node.
Read more…
We had an unconference at Spotify last Thursday and I added a semi-trolling semi-serious topic about abolishing documentation. Or NoDoc, as I'm going to call this movement. This was meant to be mostly a thought experiment, but I don't see it as complete madness.
Read more…