The Code-Free Developer Interview

by Pete Holiday

In the presentation "The Code-Free Developer Interview" at RailsConf 2018, Pete Holiday discusses innovative interview techniques aimed at evaluating software engineering candidates without the traditional stress of coding tests. Holiday, a software engineering manager at CallRail, critiques existing methods such as whiteboarding, logic puzzles, and live coding, arguing that they often fail to reflect real work and can disadvantage certain candidates. Instead, he proposes a more conversational approach.

Key Points Discussed:
- Ineffectiveness of Traditional Coding Interviews: Holiday emphasizes that coding interviews, while well-intentioned, often don't replicate real-world scenarios and can cause undue stress for candidates. This leads to uneven assessments, especially for those with demanding schedules or less access to resources.
- Alternative Interview Techniques: Instead of coding, Holiday suggests three main methods:
- Experience Digging: Candidates discuss specific projects they've worked on, detailing their roles and the challenges faced. This allows interviewers to assess their thought processes and the practical applicability of their skills.
- Code Review Role Play: Candidates review sample code, allowing interviewers to gauge their communication skills and understanding of coding standards. This method involves providing simplified code that highlights common issues without overwhelming candidates.
- Collaborative System Design: This technique engages candidates in a brainstorming session where they collaboratively design a system. Holiday highlights that this method allows the interviewer to assess a candidate's problem-solving abilities, thought process, and comfort level within a technical discussion without the pressure of programming in real-time.
- General Interviewing Tips: Key recommendations include defining what to look for in candidates beforehand, asking consistent questions, and practicing interviewing skills regularly to improve assessment quality.
- Conclusions: Holiday concludes that reverting to coding tests can be acceptable as a temporary solution but strongly proposes transitioning away from these practices for a more effective, conversation-driven interviewing process. He believes that refining interview techniques can lead to higher-quality hires and improve candidates' perceptions of the company’s interview process.

By prioritizing dialogue and practical experience over stressful coding tasks, companies can foster a more inclusive and effective hiring process, ultimately benefiting all parties involved.

00:00:10.849 All right, well I guess we'll go ahead and get started. It's 10:50, welcome to this presentation.

00:00:13.200 This is the Code-Free Developer Interview. My name is Pete Holiday, and you can find me on Twitter at @TooMuchPete. Feel free to tweet at me during the presentation; I'm sure I'll get the notifications up here and we can have a nice little conversation.

00:00:18.660 I am currently a software engineering manager at CallRail, a company based in Atlanta, Georgia. We specialize in call tracking analytics.

00:00:24.359 I've successfully implemented these interview techniques at CallRail to hire dozens of engineers over the years, all without having seen a single line of code from them before they started. My goal for this session is quite modest: to convince you that there's a better way to evaluate your potential engineering candidates than making them code for you.

00:00:40.320 So, what exactly is a coding interview? My definition of a coding interview is any practice that requires you to see code that your candidate actually wrote, such as reviewing their code samples or GitHub profiles. This is often one of the most tedious tasks for a hiring manager.

00:00:51.920 It could also include code tests, utility tasks, or coding challenges where candidates are asked to build something and submit it for review. Live coding or pair programming can be incredibly stressful, even for people who usually don’t have anxiety.

00:01:03.539 Furthermore, anything that relies on seeing code that a candidate has written to evaluate their fit for your company falls into this category. But why do we utilize these practices in the first place? They're supposedly an improvement over less effective interview techniques, such as whiteboarding riddles or language trivia. However, many of these techniques often fail to predict whether a software engineer will perform well at our company.

00:01:29.570 The theory behind coding interviews is sound: if you’re hiring a software developer, you should see them in action, right? However, the reality is that coding interviews don't replicate real work. They create a high-pressure environment that would make it difficult for many candidates to accurately demonstrate their abilities.

00:01:51.920 Additionally, these interviews disproportionately disadvantage candidates who may not have ample free time due to various circumstances, such as being a single parent or caring for a family member. In a similar vein, live coding sessions can induce significant stress, making it hard to gauge a candidate’s true capabilities.

00:02:05.310 Creating effective coding assessments is challenging, and maintaining them can be even harder as you want to avoid putting candidates in 'dependency hell.' With such high stakes, they may take twice as long completing your coding challenge due to existing commitments, leading to inefficient interviews.

00:02:28.840 In conclusion, the techniques we often rely on for coding interviews can actively undermine our goals of hiring good talent quickly. So, what should we do instead? The solution is remarkably simple: talk to the candidates.

00:02:46.150 I understand that this may sound a bit facetious, but I emphasize that the methods I’m discussing are not the only solutions. They're effective, but so is any conversation that builds rapport and engages with a candidate's interests and experiences.

00:03:01.060 Before we delve deeper, I’d like to share some general interviewing tips that apply to any interview format. First, define what you're looking for ahead of time. This will prevent ambiguity and help to counter personal biases during the selection process.

00:03:19.440 Second, ask all candidates consistent questions, as this will streamline the evaluation process. After their sessions, interviewers should document their thoughts before discussing them with anyone else.

00:03:44.290 This way, the initial impressions are preserved, which allows interviewers to maintain a clearer understanding of each candidate. Finally, remember that interviewing is a skill that needs practice. Your engineers might conduct only one interview a year, which won't hone their abilities effectively.

00:04:32.210 To ensure effective hiring, you have to give careful thought to recruiting practices. If you cultivate these interviewing skills, you’ll attract higher-quality candidates who will ultimately consider your company favorably.

00:04:45.610 This brings us to the techniques worth considering. First, dig into their experiences by reviewing their resumes. Ask them to elaborate on a project they found particularly interesting or challenging, providing them a chance to highlight their strengths.

00:05:02.060 The next technique is to ask detailed questions regarding how their projects function. Take on the role of an onboarder to better understand their work.

00:05:13.430 Another good question is to inquire about their team’s worst technical debt and what steps they’d take to address it. This reveals their understanding of the business implications of such issues.

00:05:29.200 Also, get them to discuss any outages they've faced, how they dealt with them, and what lessons they learned. This can provide insight into their resilience and problem-solving abilities.

00:05:59.260 Moving on, I want to clarify that while this could be termed a 'code-free' interview, I do suggest having a coding review as part of the process. This doesn’t have to be a long and complex assessment.

00:06:13.800 Instead, provide a small piece of code for the candidate to review, allowing you to evaluate how they communicate and engage with the code. Look for candidates who approach issues thoughtfully rather than merely pointing out mistakes.

00:06:30.060 Be sure to emphasize that the candidate should treat this as a role-play scenario with you as the original developer. This format allows you to assess not only technical skills but also how candidates express their thoughts and provide constructive feedback.

00:07:06.210 The next method is collaborative system design. In this technique, instead of writing code, engage the candidate in a discussion to build a hypothetical system. This approach can reveal their technical level while remaining enjoyable.

00:07:27.000 By asking questions and exploring various aspects of the design process, you gain valuable insight into how they think about engineering challenges.

00:07:45.670 During this discussion, it’s important to let candidates drive the conversation and to be open to different paths that could yield various results. This method is particularly effective for quickly gauging a candidate's experience level.

00:08:02.800 Through this collaborative process, you can assess whether they exhibit junior, mid-level, or senior capabilities in a short timeframe. As the conversation progresses, candidates tend to relax, focusing on the problem rather than the interview itself.

00:08:23.570 So when approaching a project, choose something understandable, preferably an application most people will recognize. For example, if you work for a well-known company, use an internal product for design discussions.

00:08:39.000 If working for a less recognized company, pick a familiar system like Facebook or Twitter to explore. Ensure the project scope is clear and manageable, as you won't be expected to build a fully functioning system in a short time.

00:09:01.120 Pre-populate irrelevant or tedious sections so candidates can focus on the key features. Allow them to lead the conversation, particularly when deciding on different design paths. Also, be ready to adapt the complexity based on how comfortable they appear.

00:09:26.780 Finally, while coding interviews may still be necessary at times—like determining the capabilities of entry-level candidates—it's important to transition away from them when moving towards these improved practices.

00:09:43.030 If you find that there's a significant need for coding interviews in your hiring process now, leverage them temporarily as you reacquaint yourself with the gravity of adopting better alternatives moving forward.

00:09:58.099 Thank you for your attention. I'm happy to answer questions and discuss some of these concepts further. I appreciate your time and interest in transforming your interviewing process.