Finding the right engineers for your organization isn’t easy. Here’s how using interview rubrics can help.
A few years ago, after a month in my first role as a software engineer at GitHub, I had the chance to interview engineers who wanted to join the company. At that point, I was still in the process of understanding and absorbing the company’s culture, so I felt a little uneasy about interviewing. After all, when you interview a candidate, you are looking for someone who will fit well into how the company works, and its culture.
When I started digging into the interview materials, I came across interview rubrics, which I hadn’t encountered before. For each interview question, there was a rubric that gave an idea of what kind of answers to expect from candidates, what makes an immediate no-hire, what should lead to hiring, and potential signals of a strong hire. This was the same for questions addressing both technical and soft skills.
As a brand new interviewer at the company, the rubric gave me an overview of what the company values in candidates, and helped me judge engineers more objectively and consistently during the interview process.
Here I’m going to explain what interview rubrics look like, and how to approach building them into your own hiring processes.
Interview rubric examples
Rubrics for non-technical questions:
First, let’s start with a non-technical question. Assuming that, as a company, you care about diversity, you would want to understand the candidate’s view on diversity. So, you can create a rubric that looks like this:
Question: ‘Do you consider having teams composed of diverse individuals to be an organizational advantage?’
No hire: The candidate completely dismisses the need or benefits of diversity in a company. Either they consider it a waste of time or feel that diverse teams are not needed to build world-class products.
Hire: The candidate understands diversity efforts and sees their value. They might not have had first-hand experience within diverse teams, but they can see the value they bring to products and teams.
Strong hire: The candidate sympathizes with or even advocates for diversity efforts. They might mention an experience when they saw diversity impacting products they built or how team dynamics benefited from diverse points of view.
The example rubric is a guideline to recognize what pattern to look for in the interviewee’s answer to a question. It might not apply directly, but it can serve well as a baseline.
Rubrics for technical questions:
Technical interview rubrics can help the interviewer uncover habits of thought or what the candidate values as an engineer. For example, startups tend to favor speed of delivery and pragmatism in decision-making, while larger companies favor amplifying team impact and higher quality. Now, let’s look at one example for a technical exercise:
Question: ‘You’re asked to optimize page load time on a website. It’s unclear what’s causing the load time to be sub-optimal. You can download the code with a pre-seeded database displaying the behavior. How would you approach this problem?’
No hire: The candidate doesn’t offer any possible solution to the problem or comes up with a solution without considering any trade-offs (i.e., ‘just adding a cache’).
Hire: The candidate goes directly into the code to try to understand what’s going on; they find the problematic SQL query but, for example, miss front-end optimizations available. Finally, the candidate mentions that a cache could be helpful, but they understand and explain the trade-offs.
Strong hire: The candidate analyzes the performance in rendering and backend time. They determine that most of the time is spent in the backend and fix a problematic SQL query by adding a database index. In addition, the candidate might suggest removing unused Javascript libraries.
How to approach building rubrics
Engineering managers should not build rubrics in a vacuum; they should seek alignment and consensus among team members and the broader organization. This alignment can lead to a few things: it can help everyone to better understand the organization’s values and what to look for in a candidate. In addition, it can help prevent rejection loops where all candidates get rejected. Finally, it can help you come up with a fair compensation offer by weighing the combined results of the answers to determine if a candidate is mid-level/senior/staff. It’s also possible to rank different responses based on the level of nuance you expect in the candidate’s response at each engineering grade.
Naturally, rubrics are just guidance for your team. Ultimately, you should give your team the freedom to have a say in hiring new team members and trust them to make a strong call. Rubrics should also be a work-in-progress document that transforms over time in line with company values and the changing business landscape.
If you currently don’t have rubrics for your interviews, get your team together and ask them what it is that they would expect to hear from a strong candidate in each question. This exercise can serve as a starting point to get the conversation going and build a robust rubric. Good luck!
