Hiring is Not Hazing

00:00:04.940 Hello, I'm Aja Hammerly, and I'm giving a talk today entitled 'Hiring is Not Hazing.' If you have questions about the talk or you want to talk to me about it, please engage with me on Twitter.

00:00:10.080 I'm on Twitter more often than I should be, and my Twitter handle is @the_actualizer.

00:00:15.780 So, let’s get started. Interviews often suck. I've read countless blog posts, Twitter threads, Facebook rants, etc., about how awful it is to interview for a job.

00:00:20.939 I completely agree—interviews do suck, and I would argue that tech interviews are even worse.

00:00:27.900 The anger that people express about tech interviews is astounding and, quite frankly, mostly valid.

00:00:33.780 Tech interviews are awful, and we all kind of hate them, but I'm here to offer you a slightly more positive message. I believe we can do better, and this talk will cover several strategies to improve the tech interview process.

00:00:46.020 Some of the suggestions I will make are small things that you, as a single interviewer, can implement in the 45 minutes to an hour you get to spend with a candidate. Other recommendations are larger and may require buy-in from your manager, or even your legal or HR teams. Most of these ideas are my opinions, based on research I've done and the hundreds of interviews I’ve given, as well as my experiences as an interviewee.

00:01:11.939 So, let’s actually get started and see how we can improve this process.

00:01:16.979 But first, let's address the elephant in the room.

00:01:20.079 I work at Google, which is infamous for its interview processes. I know firsthand; I failed my first three rounds of interviews at Google, and they were not pleasant experiences.

00:02:05.040 I wouldn't have attempted that last fourth interview if I hadn't been in a stable mental place where I didn't really need the job.

00:02:10.679 I got lucky that fourth time to have a friend cheering me on and helping me through the process. I think I can safely say that Google’s hiring process doesn’t work for many people.

00:02:21.599 As a hiring manager at Google, this frustrates me immensely, and that frustration is part of what motivated me to delve into various methods of tech hiring that inspired this talk.

00:02:27.300 I want to make it clear upfront that I haven’t fixed Google interviews. I'm trying, and I'm collaborating with some fantastic people to explore what we can do to improve the entire process. Progress is slow, but I believe I've made the 45 minutes that candidates spend with me a bit better by implementing the tips, tricks, and philosophy I'll discuss today.

00:02:47.879 Let’s start with assessing technical skills.

00:02:52.980 Whenever I ask people about tech interviews, one of the main objections revolves around how technical skills are assessed. I embarked on my process by examining all the different methods we use to test technical skills to determine which is truly the best.

00:03:16.260 Whiteboard coding is a traditional way to assess technical interviews, and people absolutely hate it. Despite the disdain, we continue to use it, which suggests there must be some valid motives for its persistence.

00:03:34.800 First, there’s a wealth of existing preparation materials available for whiteboard coding. A lot of effort and money has been spent helping candidates improve in this area. Second, it’s relatively inexpensive for companies to execute; they just need a whiteboard or a shared coding environment.

00:03:46.680 Lastly, it doesn’t take a lot of time for the candidate to execute during a 45-minute to an hour interview, which is also convenient for the interviewer. So, what are the cons? Everyone hates it. Even I enjoy it, but I understand that I’m the exception.

00:04:09.659 It’s also not realistic; while we may draw architecture diagrams on a whiteboard and sometimes write snippets of code, we almost never implement an entire function in such a setting. It’s extremely difficult to assess objectively since different interviewers have different grading criteria, leading to stress directly related to the ambiguous nature of these assessments.

00:04:45.660 Moreover, interviewers tend to ask really poor whiteboard questions, which only exacerbates the problem.

00:05:03.300 The next most common technique for assessing technical skill is through some kind of homework challenge. Many argue that this is a much fairer way to gauge technical ability than a whiteboard interview.

00:05:21.660 Let’s consider the pros and cons here. For starters, homework challenges are more realistic, as they involve performing a task in accordance with a specification, which closely resembles what developers do daily. Candidates can also use their tools and references.

00:05:38.460 If you're asking a closed book homework question, just don't. Coding isn’t a memory test; it’s about problem-solving! Another advantage of homework is that it allows for more complex problems that take longer than 45 minutes to solve, significantly increasing the realism of the challenge.

00:05:58.740 However, there are downside to homework challenges that we often overlook. It's really difficult for anyone with caretaking responsibilities to commit to a homework challenge, whether they’re caring for a child, a relative, or managing household duties.

00:06:21.180 Finding a few hours over the weekend or in the evening can be tough, particularly for someone doing most of the chores at home.

00:06:38.220 Additionally, it’s hard to assist someone on a homework assignment when they go off track until after submission. While in whiteboard interviews, I can redirect candidates when they start going the wrong way.

00:06:54.900 Conversely, with homework, I often don’t realize until after a candidate submits how far off they went or if they misinterpreted the question.

00:07:10.800 Also, many homework questions we ask are poorly structured, often taking too long for candidates. I’ve heard from many individuals that their homework challenges have taken anywhere from 6 to 12 hours to complete. That’s far too long.

00:07:24.660 Another suggestion that folks in the community often discuss is to bring candidates on-site for a day for pairing interviews. The pros of this method lie in its realism and the ability to see how well the candidate works in a team.

00:07:53.580 Programming is fundamentally a team sport, not an individual one. There’s less prep required for interviewers if the candidate is simply coming on-site.

00:08:14.760 However, significant potential ethical issues arise when asking someone to work for free. Many homework assignments often look like actual work tasks where candidates could be working on tasks that benefit the company. If this is the case, don’t do it.

00:08:35.100 Additionally, offering to pay candidates can create complications around non-compete agreements and anti-moonlighting clauses. For instance, I had to turn down an interview that sought to pay me because my current job required me to work unpaid for all technical services.

00:08:55.859 Coming on-site for a full day can be intimidating for many candidates. Especially for those who lack experience in pairing, it can feel daunting.

00:09:11.160 Moreover, it can be difficult to pick a task that is consistent and fair for all candidates. If you change tasks based on what a candidate is doing on that day, it defeats the consistency of assessment.

00:09:33.240 I've seen many people suggest another method for evaluating technical skills: reviewing a candidate's GitHub. This can be broadened to include portfolio reviews.

00:09:54.240 Essentially, a portfolio review involves asking candidates to share examples of their best work and discussing them with the interviewer. This is a common practice in many creative fields like design.

00:10:09.780 There are pros and cons to portfolio reviews. On the pro side, it allows candidates to showcase their best work by selecting projects they are proud of, ensuring they aren't highlighting less impressive elements of their history.

00:10:40.800 Also, this technique requires minimal prep from interviewers who simply need to show up, express interest, and ask relevant questions.

00:11:04.800 However, there are significant cons. Most candidates can’t share code from past jobs due to confidentiality agreements.

00:11:21.600 In creative fields, once a product has shipped, there are no trade secrets revealed in discussing the work. But in software development, if a product has shipped, they often can't share the associated code.

00:11:41.340 Additionally, many people don’t have significant time for open-source contributions, especially those with caretaking responsibilities or are burnt out and don't want to spend their evenings coding.

00:12:05.040 Lastly, differentiating between absence of signal and negative signal is challenging. Just because someone lacks items in their portfolio doesn't mean they're a bad coder.

00:12:37.920 That absence of work doesn't mean they are not capable, which can be hard for junior interviewers to recognize.

00:12:55.320 So, with all these methods, which one is best?

00:13:09.120 The answer is none of the above.

00:13:15.300 They all have their strengths and weaknesses, and no matter which one you choose, it won’t work perfectly for everyone. We can’t keep doing the same things.

00:13:28.079 So when I got to this point, I had to ask what we should do if we can’t just switch between portfolio review or homework assessments to replace whiteboarding?

00:13:40.500 When I spoke to people about interviews, I noticed some common generalizations. I heard: 'I hate whiteboard interviews,' 'the CS 200-201 material isn’t relevant to my job,' 'I hate homework problems — the last one I did took eight hours,' or 'I hate pairing interviews because I don’t know anything about the company’s domain or code base.'

00:14:03.180 Some were frustrated with evaluations because they felt that if they missed a semicolon in a whiteboard interview, they'd be penalized severely.

00:14:14.760 The underlying issue for many was not just the methods of the interviews, but bad questions. Many complained about irrelevant questions or expectations about knowledge of a company's domain.

00:14:27.900 We frequently do a poor job of evaluating candidates and I’ve seen people deduct points for technical inaccuracies when the focus really should be on holistic understanding.

00:14:44.940 A lot of interviewers approach with a negative attitude, seeing themselves as gatekeepers instead of allies seeking the right person for their team.

00:15:04.680 The result is that tech interviews often suck because we ask bad questions, carry a poor evaluative framework, and show up with bad attitudes. But we can fix these issues!

00:15:25.440 We can ask better questions, improve how we evaluate candidates, and approach interviews with a positive attitude. The exciting part is that most of these changes can happen immediately — right after this talk!

00:15:57.720 Let’s dive into how do we ask better questions. First off, keep it simple. So many interview questions and take-home exercises end up being long and convoluted.

00:16:10.680 Fortunately, candidates aren’t at their best in interview environments, so it’s essential to cut them some slack! Your actual problem statement should consist of a single sentence or possibly two at most.

00:16:24.720 For example, a simple format you can follow would be: 'Write a function that takes input X, does Y, and returns Z.' I strive to keep questions simple enough to fall into this format.

00:16:37.380 This makes it easier for candidates to understand the expectations and allows them to be successful.

00:16:56.880 A common question formatted this way might read: 'Write a function that takes a string and checks if it’s a palindrome, returning true or false.'

00:17:10.680 Please ensure to provide clear definitions for your input. If something’s unclear, the candidate won’t know how to tackle it. It's important to be prepared to provide clarity on potential ambiguities.

00:17:25.159 It’s fine if your question seems overly simple. Remember that interviewees may not be at their peak performance.

00:17:40.620 One of my go-to questions is an elementary exercise from the first chapter of a beginner's computing book for non-computer scientists. An incredibly simple problem can yield valuable insights into a candidate’s design approaches, data structure knowledge, and code clarity.

00:18:01.740 Another common pitfall I've observed involves jargon in questions. When it comes to technical language, there is obvious slang or regionalisms that candidates from other countries may not grasp.

00:18:14.340 There’s also internal jargon that holds no meaning to outsiders. For example, let’s take a data structure like a JSON object describing my cat.

00:18:37.920 In Ruby, I would typically use a hash, in Python it would be a dictionary, while in PHP, it would be referred to as an array. Therefore, if I framed my question using terminology specific to a language, the candidate may become confused.

00:18:55.740 As such, it’s important to provide generalized input for data structures instead and keep the context relatable for all candidates.

00:19:16.080 Here’s an example: ‘Given a list of votes, determine who won the election.’ This paints a more familiar, universal scenario for candidates.

00:19:32.940 The same problem can also be posed without the scenario: 'Write a function that determines the most common character in a string.' This version is accessible and straightforward.

00:19:48.300 Making questions familiar can be particularly effective for newer or less experienced candidates.

00:20:04.680 So be cautious when choosing your context! Topics like sports, specific industries, or technical jargon can present unnecessary hurdles.

00:20:17.880 If the candidate struggles with your context, they can get lost despite having the necessary skills. Avoid any context that could alienate your candidate.

00:20:31.800 Ensure that job questions remain relevant to the role. Don’t ask front-end developers about concepts that are not pertinent to their position. Avoid asking back-end developers about JavaScript or CSS-related queries.

00:20:46.920 And for goodness’ sake, don’t ask riddles or trick questions! Questions like why manhole covers are round serve no real purpose.

00:21:03.120 Instead, shift your focus to things that matter: real technical tasks.

00:21:17.520 Another strategy involves reading aloud your questions to candidates and providing them in writing. This dual approach is particularly beneficial for candidates who may not communicate fluently in English.

00:21:33.450 It’s also essential not to over-specify in your questions. For example, asking, 'Calculate the winner of an election, handle ties by selecting the first candidate...' may seem detailed but could also hinder creativity.

00:21:48.960 Overly detailed questions may distract candidates from the actual problem-solving process, impeding their mind from exploring innovative solutions.

00:22:06.120 Encourage questions from candidates during the interview. Let them know that they may not have received all the relevant information they need and invite them to raise any uncertainties.

00:22:22.320 This keeps candidates engaged and reassures them they are a part of an interactive process, facilitating an open dialogue.

00:22:37.200 Lastly, remember to keep it concise. The most common failures I've observed are overly complex or hard questions! Limit coding assessments to 30-45 minutes for whiteboard exercises and 2-3 hours for take-home coding tasks.

00:22:50.580 If you're concerned about timing, a good rule of thumb is to provide candidates three times as long as it would take you or a friend to tackle the problem.

00:23:03.750 Now that we’ve discussed better questions, let’s delve into improving evaluations. It’s crucial to assess candidates properly, as well-structured questions are of little use without suitable evaluations.

00:23:15.840 Many candidates are worried that they’ll face harsh grading while being evaluated inconsistently. To counter this, having a rubric is vital.

00:23:36.900 A rubric is a rating guide that includes examples associated with each rating. Many big companies now use rubrics, and I’ve even created rubrics detailing coding questions, shared in a document that everyone can access.

00:24:01.740 For instance, let’s assume your job description mentions that a candidate should be proficient in Python. Your rubric could look like this.

00:24:20.880 Exceptional candidates would understand the standard library, articulate the pros and cons of various approaches, and write Pythonic code.

00:24:37.680 Candidates unfamiliar with the language will struggle with even basic syntax, losing points by not using hints effectively.

00:24:52.500 The other two categories exist between these extremes, addressing those who showcase a solid grasp of the language.

00:25:10.320 Rubrics are qualitative assessments—meaning we evaluate candidates based on their demonstration of skills, rather than a strict point-based system.

00:25:34.320 Instead of assigning points for errors, focus on whether their skills reflect a borderline, solid, or exceptional performance.

00:25:50.160 It's also wise to record the reasoning behind your evaluations. To ensure consistency across various candidates, check for biases such as recency bias or anchoring bias.

00:26:06.840 For example, if you decide a candidate is solid, ask yourself why they didn't fall into the exceptional category.

00:26:19.740 Some more questions to ask yourself when writing evaluations include: can the candidate clearly communicate their technical decisions?

00:26:35.640 Assess how the candidate managed feedback, whether they considered common errors, and how they utilized hints during the interview.

00:26:51.300 Lastly, consider whether it matters if the candidate missed a semicolon on their whiteboard code. Ask yourself if it would have impacted real-life coding instances.

00:27:03.960 Alongside that, recognize that everyone will require ramp-up time during onboarding. Ultimately, if it's something that can be taught, it may not be worth stressing over in the interview.

00:27:20.040 When I first started at Google, I switched to using gender-neutral pronouns in my evaluations. This practice aids in writing more objective feedback, avoiding gender biases.

00:27:36.000 So, we’ve explored better questions and better evaluations, now let’s discuss attitudes. A good attitude is crucial to the interview process.

00:27:50.520 People frequently approach interviews with negative attitudes, thinking they have to endure the same suffering they did.

00:28:07.920 We need to foster an environment where everyone feels their best interests are paramount, focusing on making the process positive for everyone.

00:28:23.640 I made a significant change to my interviewing mindset in the past five years by positioning myself as a member of 'team the candidate.'

00:28:36.240 If I’m interviewing you, I’m on your side! Everyone involved in the process should hope for the candidate's success.

00:28:53.160 If candidates perceive a supportive atmosphere, they are more likely to perform positively in interviews.

00:29:12.480 This mindset shift has transformed the way I conduct interviews and improved my overall perspective when hiring.

00:29:29.520 Being part of the candidate's team means being optimistic; it’s crucial not to take on the gatekeeper role, which often leads to a negative experience.

00:29:47.880 Focus on providing avenues for improvement where needed, recognizing their strengths, and addressing challenges along the way.

00:30:03.840 Additionally, respect the candidate’s boundaries during the interview. Candidates, like all of us, have basic needs regarding food, water, and breaks.

00:30:20.760 Be aware of any necessary accommodations required by your local law, as being respectful and inclusive is the right approach.

00:30:38.400 Encourage humane interactions. Offering candidates insights on the interview format can provide necessary support for them as they prepare.

00:30:54.240 Approach your interactions with authenticity, but also remember to steer clear of personal inquiry in interviews. Even seemingly innocent questions can backfire.

00:31:11.520 Questions should focus on the job: what excites them about the position or what aspects they enjoy in their current role.

00:31:29.160 In conclusion, this will help create a better atmosphere for all parties involved. Thank you for your attention.

00:31:47.460 To summarize and for a cheat sheet, keep questions simple, avoid jargon, maintain familiar contexts, discourage over-specifying, and encourage candidates to ask questions.

00:32:03.599 Use rubrics for evaluations, consistently question your evaluations, and consider if mistakes truly matter. Prioritize gender-neutral language in your write-ups.

00:32:24.480 Maintain a positive attitude, be supportive, focus on strengths, and always prioritize candidates' wellbeing. Interviewing is about finding the best fit through collaboration.

00:32:51.000 Thank you very much. Please engage with me on Twitter and elsewhere, as I'm deeply passionate about this topic!