Despite significant investments in building DEI programs – and even greater investments in publicizing those programs – very little progress has actually been made by big tech companies in hiring diverse developer teams.
On the surface, the tech industry is the poster-child for failing fast and innovating faster. Lean methodology, rapid iteration, and identifying market inefficiencies are a few of the buzzwords that we all hear on a daily basis. So why does that appetite for change and disruption stop at hiring? Why are we (as a tech industry) still relying on the same practices that keep failing – like resume screens, intimidating technical interviews, and bias-riddled technical recruiting practices that filter out diverse developer talent instead of welcoming it in?
The only way to correct systemic failure is to change the system
Erica Stanley’s article on the topic, which featured on LeadDev earlier this year, speaks about the changes that companies need to make to build more inclusive, antiracist teams. I highly recommend reading her entire piece, but she made two points that I agree with particularly strongly:
- ‘Ambiguity is a breeding ground for bias and lack of accountability’;
- ‘Consistency is necessary to scale your efforts across your team and throughout your company.’
Both of these points are critical to creating structured hiring and interviewing solutions that break down the systemic barriers facing underrepresented minorities in tech. At Karat, our approach is to bring consistency and clarity to the technical interview process, which we accomplish with structure.
Five keys to a structured hiring process
- Assess competencies
- Avoid ambiguity
- Use a structured scoring rubric
- Train and review your interviewers
- Beware of pedigree bias
Assess competencies
The first step to structuring your hiring program is to perform a job analysis, and create a list of relevant competencies. For every job, someone needs to review the job description, responsibilities, and any existing assessments for the role to determine those competencies.
For a typical engineering role, this could include project discussions, algorithms, or talking through code reviews. Note, you are not looking for ninjas, rockstars, or whatever we’re calling great developers this week. That kind of coded language-of-the-day is an instant red flag for many candidates from underrepresented groups, because it can’t be measured objectively.
Having a formal competency framework lets you evaluate each competency intentionally and consistently across candidates from all backgrounds.
Also, be clear with the criteria and competencies you’re assessing. Are you expecting complete code in a technical interview, or an outline? Is a brute force solution good enough, or do you expect candidates to optimize? If you want to see a candidate test their code, ask them how they would test a given piece of code, don’t make them guess at what you’re looking for during an already-stressful job interview.
Avoid ambiguity
Being clear in what you’re assessing is a good way to set appropriate expectations with candidates, but ambiguity can take multiple forms. Some interview questions intentionally test the candidate’s ability to handle ambiguity by leaving out key pieces of information; but if not administered properly, these can insert bias into the process.
For too long, cultural biases have encouraged women and people of color to accommodate others. This bias often penalizes assertiveness by interpreting it as “aggression”, or “anger”. These cultural norms are often internalized, which can make some candidates less likely to ask clarifying questions unless specifically prompted to do so.
At the very least, if you are assessing someone’s ability to handle ambiguous situations, tell them that’s what you’re doing. That way the candidate knows they won’t be dinged for asking probing questions during the interview.
Ambiguity is also problematic when it comes to making hiring decisions, which is why creating a structured and consistent way to measure interview results is so crucial. As Stanley covered in her article, ambiguous feedback ‘may look like telling someone that something they did was great but not telling them how it was great.’ The exact same principle applies to sharing candidate feedback from interviews, which should be explicit.
So how do you structure interview feedback for consistency?
Use a structured scoring rubric
Consistency requires a structured scoring rubric that forces the interviewer to make observations rather than judgments. Checking a box to confirm that a candidate produced a fully working solution during a paired coding interview sends a clear and measurable hiring signal that is easy to compare across candidates.
Structured rubrics help interviewers and hiring managers by making it clear which competencies matter, and by creating an objective scale to assess those competencies.
Furthermore, the act of filling out a structured rubric will force your interviewers to objectively confront their biases because each evaluation is backed by a rigorous definition that you can audit and defend. This not only puts candidates on a level playing field, but it improves your hiring signal overall.
For anyone interested in learning more, here’s a handy guide for creating structured scoring rubrics.
Train and review your interviewers
Most people don’t intentionally plan on injecting their bias into an interview process, but it creeps in when interviewers make mistakes or don’t know what they’re doing. Luckily, interviewing is like any other skill in that people get better at it with practice and feedback. Make a point to train and review your interviewers.
A carefully-delivered hint can make the difference between a complete and functional solution, and a candidate who spins their wheels for 15 minutes formulating an approach in a technical interview. Make sure you have guardrails that define what constitutes “hand-holding” during interviews. There’s nothing wrong with interviewers who want their candidates to succeed, but if they aren’t identifying the guidance they shared in the writeup, those details won’t make it into your hiring signal. When that happens, you run the risk of hiring developers who may not be able to get the job done because the interviewer ‘really liked them and just gave a little hint’.
This may sound like a heavy lift for a time-strapped engineering team, but the shift to remote work and remote interviews makes recording and reviewing your interviewer performance easier than ever.
Have a quality control process that reviews interviewers periodically for instances of bias, undocumented hand-holding, and ambiguity. This helps ensure that your hiring program is conducted with the same attention to detail as the rest of your engineering program!
Pedigree bias and inclusive recruiting
Lastly, be intentional about your candidate intake process. Based on the industry-wide data we collect, around 10% of software engineering candidates in the tech industry who apply through company career sites are invited to interview. If you have a competency-based hiring process, make sure your recruiting and candidate intake is also tied to those competencies, and not to artificial barriers like resume screens, which can introduce pedigree bias.
If your resume screens out candidates from outside the top ten computer science schools, then you’ve created artificial scarcity, and what’s worse, you’ve automatically constrained the diversity of your pipeline to the diversity of those schools.
Coding challenges, work sample reviews, and technical interviews are all ways to assess competencies that don’t appear on a resume. This can open your recruiting up to a wider, more inclusive pool of developer candidates.
Fair interviews are more predictive and more enjoyable
Unstructured interviews and hiring processes put some candidates at a disadvantage. This isn’t only wrong from an ethical standpoint, it’s bad business too. An unfair interview is a non-predictive one that will exclude some great potential candidates from your process.
In a recent panel on inclusive interviewing, Lowe’s Corporate Technology Recruiting Manager, Tricia Lincoln, underscored the importance of having a fair interview. ‘Candidates react positively to brands where they feel like they’re being treated fairly and feel like they have the same shot as everyone else,’ said Lincoln. ‘I bring all my interviewers together and explain why it’s important to treat candidates the same and fairly, especially in the technical assessment. Once they buy into that philosophy, informed interviewers can really change the narrative and help build more diverse teams.’
Conclusion
By structuring your developer hiring around competencies, reducing ambiguity, using structured rubrics, training and reviewing interviewees, and creating an application process that welcomes diverse talent, you can help fix the systemic gaps that exist in today’s tech sector.
Not only will you have a more diverse and inclusive team, but with the improved hiring signal and candidate experience, I’d wager that your overall hiring bar rises, and you get closer to meeting hiring targets in the process.
