How to build up a data team (everything I ever learned about recruiting)
During my time at Spotify, I've reviewed thousands of resumes and interviewed hundreds of people. Lots of them were rejected but lots of them also got offers. Finally, I've also had my share of offers rejected by the candidate.
Recruiting is one of those things where the Dunning-Kruger effect is the most pronounced: the more you do it, the more you realize how bad you are at it. Every time I look back a year, I realize 10 things I did wrong. Extrapolating this, I know in another year I'll realize all the stupid mistakes I'm doing right now. Anyway, that being said, here are some things I learned from recruiting.
Getting the word out
Depending on where you work, people might have no clue about your company. Why would they work for something they have never heard of? Or alternatively – something they know of, but doesn't necessarily associate with cutting edge tech? There's a million companies out there doing cool stuff, so make sure that people know your company stands out. Blog, talk at meetups, open source stuff, go to conferences. I honestly don't know what works – I don't have any numbers. But you need to hedge your bets by attacking on all angles at the same time.
I think developers have a hard time justifying this just because success is not easily quantifiable – this is a branding exercise, and it's super hard to find out if you're doing the right thing. But over time if you do this right, you will get anecdotal feedback from candidates coming in saying they saw your presentation or read this cool story on Hacker News, or what not.
Finding the people
I don't think there's anything magic about this – just go through external recruiters, internal recruiters, job postings, connections, whatever.
Presenting the opportunity
I think most people in the industry are fed up with bad bulk messages over email/LinkedIn. Ideally, the hiring manager should introduce themselves, or for more senior roles having more senior people reaching out (all the way up to the CTO). If a recruiter is reaching out, it's super important to make sure the recruiter can reach out to people with a quick note on what's interesting about the team and why it's a good fit.
Finding the right candidates
Recruiting is some crazy type of active learning problem with this observation bias where you only see how well the people you hire are doing. In particular there was a lot of discussion a while back when Google claimed there was no correlation between test scores and GPA. I think there totally are really strong correlations on a macro scale, but if you are already filtering out people based on those criteria, obviously you will reduce the strength, or even reverse it. Not that I claim to have found any magic criteria. I do however thing the two most successful traits that I've observed are (with the risk of sounding cheesy):
- Programming fluency (10,000 hour rule or whatever) – you need to be able to visualize large codebases, and understand how things fit together. I strongly believe that data engineers need to understand the full stack from idea, to machine learning algorithm, to code running in production. I've seen other companies having a “throw it over the fence” attitude, with one team brainstorming algorithms, another team in another city implementing them. I think that's a flawed way to have a tight learning cycle. In particular, I'm hesitant to hire candidates who are strong on the theoretical side, but with little experience writing code. That's why I really avoid the “data science” label – most people within this group are generally lacking on the core programming side. I don't think this necessarily means candidates has to have a solid understanding of the CAP theorem and the linux page cache. The most important thing is they have written a lot of code, can work with nontrivial code bases, and can write clean, maintainable code. There is nothing magic to this – but a person who only has written Matlab scripts probably will have a harder time adjusting.
- Understand the big picture – go from a vision to a set of tasks, to a bunch of code being written. People have to be able to go from an idea (“analyze this data set and build an algorithm that uses it as a signal for the recommender system”) to code, without having to hand hold them throughout every single step. People who need to be given small tasks rather than the underlying problem will never understand why we're working on things, and will inevitably end up doing the wrong thing.
- Smart people are actually really impressed by good interview process. If some smart ML genius comes in and walks them through a super hard problem, the candidate will feel like they can learn from that interviewer. Conversely, giving interview problems that are not well structured, without any follow up questions, or discussion notes, will give a really bad impression. The best type of interview problems have a (b), (c) and (d) version that you can pull up in case the candidate nails (a) directly.
- Understand the level of selling you need to do. If you get a super senior person in, spend the bare minimum to establish that this person actually lives up to their rumor. Then, spend the rest of the time explaining the role and why you are so excited about it.
- Explaining the role and why it's a good fit is everything. But obviously not superficial stuff. These are engineers and you need to impress them with the stuff they will learn. Talk about why your company is working on, what challenges you have, what technologies you are using.
- It's often much easier to start by listening to the candidate. Let them ask questions and talk more than you. Try to understand what they are looking for. Then, explain how this position meets or does not meet those criteria.
- Everyone has an angle. Someone coming from finance is probably more excited to hear about your product vision. Someone from a small failed startup is probably more excited to hear about your benefits.
- Make the candidate meet the team they will be working with. There's so many companies failing this. I remember going through a series of ten Google interviews many years ago and then the recruiter wouldn't tell me what team I would work on. I immediately turned down the offer. I think on the other side of the spectrum, the best thing you can do is to bring the candidate out on a team lunch to meet as many as possible.
- Make them understand you take this role seriously. Set up a quick chat with the CTO or some senior people.
- If it's a senior person, and everyone is super impressed, hire the person. If it's a senior person, and people are on the fence, you shouldn't hire the person.
- If it's a junior person, it's sometimes hard to know. One thing I've learned is that excitement really counts. For instance the candidate kicks ass during the ML interview, and clearly has solid programming fluency, but doesn't necessarily know much about scalability and version control, then I really think it boils down to how excited this person is.
- At the end of the day, if you realize this candidate is brilliant, but not necessarily the right fit for your role, find something else. It's a sign of a great organization that you can always find a place for smart people.
- Candidates will be on a spectrum between “theoretical” (more ML stuff) or “applied” (more scalability, backend, data processing). But just for the purpose of the argument, let's assume people are on each side of the spectrum. For a high performing machine learning team, I think a ratio of 1:2 is good. But everyone should be able to work on every part of the system, if needed.
- If you're a small company, you will end up hiring lots of people who are similar to yourself. I don't mean necessarily people of the same gender or origin, but people who will all go out to the same bar getting the same beer. This might good for cohesion early on, but as soon as you grow beyond the early stage, you might end up with a destructive monoculture that is super hard to break out of.
- On the same note: if you're a big company and your team still skews a certain way, then it means you have prejudice problems you need to work on. Seriously. For instance, with the number of female software engineers being 20.2%, you can derive that 95% of all 50 people team should contain at least five women, and 70% of all teams should contain at least eight women. It's just plain math.
For more reading on the subject, I just came across this interesting blog post by Cheng-Tao Chu: Why building a data science team is deceptively hard.