00:00:04.940
Hello, I'm Aja Hammerly, and I'm giving a talk today entitled 'Hiring is Not Hazing.' If you have questions about the talk or you want to talk to me about it, please engage with me on Twitter.
00:00:10.080
I'm on Twitter more often than I should be, and my Twitter handle is @the_actualizer.
00:00:15.780
So, let’s get started. Interviews often suck. I've read countless blog posts, Twitter threads, Facebook rants, etc., about how awful it is to interview for a job.
00:00:20.939
I completely agree—interviews do suck, and I would argue that tech interviews are even worse.
00:00:27.900
The anger that people express about tech interviews is astounding and, quite frankly, mostly valid.
00:00:33.780
Tech interviews are awful, and we all kind of hate them, but I'm here to offer you a slightly more positive message. I believe we can do better, and this talk will cover several strategies to improve the tech interview process.
00:00:46.020
Some of the suggestions I will make are small things that you, as a single interviewer, can implement in the 45 minutes to an hour you get to spend with a candidate. Other recommendations are larger and may require buy-in from your manager, or even your legal or HR teams. Most of these ideas are my opinions, based on research I've done and the hundreds of interviews I’ve given, as well as my experiences as an interviewee.
00:01:11.939
So, let’s actually get started and see how we can improve this process.
00:01:16.979
But first, let's address the elephant in the room.
00:01:20.079
I work at Google, which is infamous for its interview processes. I know firsthand; I failed my first three rounds of interviews at Google, and they were not pleasant experiences.
00:02:05.040
I wouldn't have attempted that last fourth interview if I hadn't been in a stable mental place where I didn't really need the job.
00:02:10.679
I got lucky that fourth time to have a friend cheering me on and helping me through the process. I think I can safely say that Google’s hiring process doesn’t work for many people.
00:02:21.599
As a hiring manager at Google, this frustrates me immensely, and that frustration is part of what motivated me to delve into various methods of tech hiring that inspired this talk.
00:02:27.300
I want to make it clear upfront that I haven’t fixed Google interviews. I'm trying, and I'm collaborating with some fantastic people to explore what we can do to improve the entire process. Progress is slow, but I believe I've made the 45 minutes that candidates spend with me a bit better by implementing the tips, tricks, and philosophy I'll discuss today.
00:02:47.879
Let’s start with assessing technical skills.
00:02:52.980
Whenever I ask people about tech interviews, one of the main objections revolves around how technical skills are assessed. I embarked on my process by examining all the different methods we use to test technical skills to determine which is truly the best.
00:03:16.260
Whiteboard coding is a traditional way to assess technical interviews, and people absolutely hate it. Despite the disdain, we continue to use it, which suggests there must be some valid motives for its persistence.
00:03:34.800
First, there’s a wealth of existing preparation materials available for whiteboard coding. A lot of effort and money has been spent helping candidates improve in this area. Second, it’s relatively inexpensive for companies to execute; they just need a whiteboard or a shared coding environment.
00:03:46.680
Lastly, it doesn’t take a lot of time for the candidate to execute during a 45-minute to an hour interview, which is also convenient for the interviewer. So, what are the cons? Everyone hates it. Even I enjoy it, but I understand that I’m the exception.
00:04:09.659
It’s also not realistic; while we may draw architecture diagrams on a whiteboard and sometimes write snippets of code, we almost never implement an entire function in such a setting. It’s extremely difficult to assess objectively since different interviewers have different grading criteria, leading to stress directly related to the ambiguous nature of these assessments.
00:04:45.660
Moreover, interviewers tend to ask really poor whiteboard questions, which only exacerbates the problem.
00:05:03.300
The next most common technique for assessing technical skill is through some kind of homework challenge. Many argue that this is a much fairer way to gauge technical ability than a whiteboard interview.
00:05:21.660
Let’s consider the pros and cons here. For starters, homework challenges are more realistic, as they involve performing a task in accordance with a specification, which closely resembles what developers do daily. Candidates can also use their tools and references.
00:05:38.460
If you're asking a closed book homework question, just don't. Coding isn’t a memory test; it’s about problem-solving! Another advantage of homework is that it allows for more complex problems that take longer than 45 minutes to solve, significantly increasing the realism of the challenge.
00:05:58.740
However, there are downside to homework challenges that we often overlook. It's really difficult for anyone with caretaking responsibilities to commit to a homework challenge, whether they’re caring for a child, a relative, or managing household duties.
00:06:21.180
Finding a few hours over the weekend or in the evening can be tough, particularly for someone doing most of the chores at home.
00:06:38.220
Additionally, it’s hard to assist someone on a homework assignment when they go off track until after submission. While in whiteboard interviews, I can redirect candidates when they start going the wrong way.
00:06:54.900
Conversely, with homework, I often don’t realize until after a candidate submits how far off they went or if they misinterpreted the question.
00:07:10.800
Also, many homework questions we ask are poorly structured, often taking too long for candidates. I’ve heard from many individuals that their homework challenges have taken anywhere from 6 to 12 hours to complete. That’s far too long.
00:07:24.660
Another suggestion that folks in the community often discuss is to bring candidates on-site for a day for pairing interviews. The pros of this method lie in its realism and the ability to see how well the candidate works in a team.
00:07:53.580
Programming is fundamentally a team sport, not an individual one. There’s less prep required for interviewers if the candidate is simply coming on-site.
00:08:14.760
However, significant potential ethical issues arise when asking someone to work for free. Many homework assignments often look like actual work tasks where candidates could be working on tasks that benefit the company. If this is the case, don’t do it.
00:08:35.100
Additionally, offering to pay candidates can create complications around non-compete agreements and anti-moonlighting clauses. For instance, I had to turn down an interview that sought to pay me because my current job required me to work unpaid for all technical services.
00:08:55.859
Coming on-site for a full day can be intimidating for many candidates. Especially for those who lack experience in pairing, it can feel daunting.
00:09:11.160
Moreover, it can be difficult to pick a task that is consistent and fair for all candidates. If you change tasks based on what a candidate is doing on that day, it defeats the consistency of assessment.
00:09:33.240
I've seen many people suggest another method for evaluating technical skills: reviewing a candidate's GitHub. This can be broadened to include portfolio reviews.
00:09:54.240
Essentially, a portfolio review involves asking candidates to share examples of their best work and discussing them with the interviewer. This is a common practice in many creative fields like design.
00:10:09.780
There are pros and cons to portfolio reviews. On the pro side, it allows candidates to showcase their best work by selecting projects they are proud of, ensuring they aren't highlighting less impressive elements of their history.
00:10:40.800
Also, this technique requires minimal prep from interviewers who simply need to show up, express interest, and ask relevant questions.
00:11:04.800
However, there are significant cons. Most candidates can’t share code from past jobs due to confidentiality agreements.
00:11:21.600
In creative fields, once a product has shipped, there are no trade secrets revealed in discussing the work. But in software development, if a product has shipped, they often can't share the associated code.
00:11:41.340
Additionally, many people don’t have significant time for open-source contributions, especially those with caretaking responsibilities or are burnt out and don't want to spend their evenings coding.
00:12:05.040
Lastly, differentiating between absence of signal and negative signal is challenging. Just because someone lacks items in their portfolio doesn't mean they're a bad coder.
00:12:37.920
That absence of work doesn't mean they are not capable, which can be hard for junior interviewers to recognize.
00:12:55.320
So, with all these methods, which one is best?
00:13:09.120
The answer is none of the above.
00:13:15.300
They all have their strengths and weaknesses, and no matter which one you choose, it won’t work perfectly for everyone. We can’t keep doing the same things.
00:13:28.079
So when I got to this point, I had to ask what we should do if we can’t just switch between portfolio review or homework assessments to replace whiteboarding?
00:13:40.500
When I spoke to people about interviews, I noticed some common generalizations. I heard: 'I hate whiteboard interviews,' 'the CS 200-201 material isn’t relevant to my job,' 'I hate homework problems — the last one I did took eight hours,' or 'I hate pairing interviews because I don’t know anything about the company’s domain or code base.'
00:14:03.180
Some were frustrated with evaluations because they felt that if they missed a semicolon in a whiteboard interview, they'd be penalized severely.
00:14:14.760
The underlying issue for many was not just the methods of the interviews, but bad questions. Many complained about irrelevant questions or expectations about knowledge of a company's domain.
00:14:27.900
We frequently do a poor job of evaluating candidates and I’ve seen people deduct points for technical inaccuracies when the focus really should be on holistic understanding.
00:14:44.940
A lot of interviewers approach with a negative attitude, seeing themselves as gatekeepers instead of allies seeking the right person for their team.
00:15:04.680
The result is that tech interviews often suck because we ask bad questions, carry a poor evaluative framework, and show up with bad attitudes. But we can fix these issues!
00:15:25.440
We can ask better questions, improve how we evaluate candidates, and approach interviews with a positive attitude. The exciting part is that most of these changes can happen immediately — right after this talk!
00:15:57.720
Let’s dive into how do we ask better questions. First off, keep it simple. So many interview questions and take-home exercises end up being long and convoluted.
00:16:10.680
Fortunately, candidates aren’t at their best in interview environments, so it’s essential to cut them some slack! Your actual problem statement should consist of a single sentence or possibly two at most.
00:16:24.720
For example, a simple format you can follow would be: 'Write a function that takes input X, does Y, and returns Z.' I strive to keep questions simple enough to fall into this format.
00:16:37.380
This makes it easier for candidates to understand the expectations and allows them to be successful.
00:16:56.880
A common question formatted this way might read: 'Write a function that takes a string and checks if it’s a palindrome, returning true or false.'
00:17:10.680
Please ensure to provide clear definitions for your input. If something’s unclear, the candidate won’t know how to tackle it. It's important to be prepared to provide clarity on potential ambiguities.
00:17:25.159
It’s fine if your question seems overly simple. Remember that interviewees may not be at their peak performance.
00:17:40.620
One of my go-to questions is an elementary exercise from the first chapter of a beginner's computing book for non-computer scientists. An incredibly simple problem can yield valuable insights into a candidate’s design approaches, data structure knowledge, and code clarity.
00:18:01.740
Another common pitfall I've observed involves jargon in questions. When it comes to technical language, there is obvious slang or regionalisms that candidates from other countries may not grasp.
00:18:14.340
There’s also internal jargon that holds no meaning to outsiders. For example, let’s take a data structure like a JSON object describing my cat.
00:18:37.920
In Ruby, I would typically use a hash, in Python it would be a dictionary, while in PHP, it would be referred to as an array. Therefore, if I framed my question using terminology specific to a language, the candidate may become confused.
00:18:55.740
As such, it’s important to provide generalized input for data structures instead and keep the context relatable for all candidates.
00:19:16.080
Here’s an example: ‘Given a list of votes, determine who won the election.’ This paints a more familiar, universal scenario for candidates.
00:19:32.940
The same problem can also be posed without the scenario: 'Write a function that determines the most common character in a string.' This version is accessible and straightforward.
00:19:48.300
Making questions familiar can be particularly effective for newer or less experienced candidates.
00:20:04.680
So be cautious when choosing your context! Topics like sports, specific industries, or technical jargon can present unnecessary hurdles.
00:20:17.880
If the candidate struggles with your context, they can get lost despite having the necessary skills. Avoid any context that could alienate your candidate.
00:20:31.800
Ensure that job questions remain relevant to the role. Don’t ask front-end developers about concepts that are not pertinent to their position. Avoid asking back-end developers about JavaScript or CSS-related queries.
00:20:46.920
And for goodness’ sake, don’t ask riddles or trick questions! Questions like why manhole covers are round serve no real purpose.
00:21:03.120
Instead, shift your focus to things that matter: real technical tasks.
00:21:17.520
Another strategy involves reading aloud your questions to candidates and providing them in writing. This dual approach is particularly beneficial for candidates who may not communicate fluently in English.
00:21:33.450
It’s also essential not to over-specify in your questions. For example, asking, 'Calculate the winner of an election, handle ties by selecting the first candidate...' may seem detailed but could also hinder creativity.
00:21:48.960
Overly detailed questions may distract candidates from the actual problem-solving process, impeding their mind from exploring innovative solutions.
00:22:06.120
Encourage questions from candidates during the interview. Let them know that they may not have received all the relevant information they need and invite them to raise any uncertainties.
00:22:22.320
This keeps candidates engaged and reassures them they are a part of an interactive process, facilitating an open dialogue.
00:22:37.200
Lastly, remember to keep it concise. The most common failures I've observed are overly complex or hard questions! Limit coding assessments to 30-45 minutes for whiteboard exercises and 2-3 hours for take-home coding tasks.
00:22:50.580
If you're concerned about timing, a good rule of thumb is to provide candidates three times as long as it would take you or a friend to tackle the problem.
00:23:03.750
Now that we’ve discussed better questions, let’s delve into improving evaluations. It’s crucial to assess candidates properly, as well-structured questions are of little use without suitable evaluations.
00:23:15.840
Many candidates are worried that they’ll face harsh grading while being evaluated inconsistently. To counter this, having a rubric is vital.
00:23:36.900
A rubric is a rating guide that includes examples associated with each rating. Many big companies now use rubrics, and I’ve even created rubrics detailing coding questions, shared in a document that everyone can access.
00:24:01.740
For instance, let’s assume your job description mentions that a candidate should be proficient in Python. Your rubric could look like this.
00:24:20.880
Exceptional candidates would understand the standard library, articulate the pros and cons of various approaches, and write Pythonic code.
00:24:37.680
Candidates unfamiliar with the language will struggle with even basic syntax, losing points by not using hints effectively.
00:24:52.500
The other two categories exist between these extremes, addressing those who showcase a solid grasp of the language.
00:25:10.320
Rubrics are qualitative assessments—meaning we evaluate candidates based on their demonstration of skills, rather than a strict point-based system.
00:25:34.320
Instead of assigning points for errors, focus on whether their skills reflect a borderline, solid, or exceptional performance.
00:25:50.160
It's also wise to record the reasoning behind your evaluations. To ensure consistency across various candidates, check for biases such as recency bias or anchoring bias.
00:26:06.840
For example, if you decide a candidate is solid, ask yourself why they didn't fall into the exceptional category.
00:26:19.740
Some more questions to ask yourself when writing evaluations include: can the candidate clearly communicate their technical decisions?
00:26:35.640
Assess how the candidate managed feedback, whether they considered common errors, and how they utilized hints during the interview.
00:26:51.300
Lastly, consider whether it matters if the candidate missed a semicolon on their whiteboard code. Ask yourself if it would have impacted real-life coding instances.
00:27:03.960
Alongside that, recognize that everyone will require ramp-up time during onboarding. Ultimately, if it's something that can be taught, it may not be worth stressing over in the interview.
00:27:20.040
When I first started at Google, I switched to using gender-neutral pronouns in my evaluations. This practice aids in writing more objective feedback, avoiding gender biases.
00:27:36.000
So, we’ve explored better questions and better evaluations, now let’s discuss attitudes. A good attitude is crucial to the interview process.
00:27:50.520
People frequently approach interviews with negative attitudes, thinking they have to endure the same suffering they did.
00:28:07.920
We need to foster an environment where everyone feels their best interests are paramount, focusing on making the process positive for everyone.
00:28:23.640
I made a significant change to my interviewing mindset in the past five years by positioning myself as a member of 'team the candidate.'
00:28:36.240
If I’m interviewing you, I’m on your side! Everyone involved in the process should hope for the candidate's success.
00:28:53.160
If candidates perceive a supportive atmosphere, they are more likely to perform positively in interviews.
00:29:12.480
This mindset shift has transformed the way I conduct interviews and improved my overall perspective when hiring.
00:29:29.520
Being part of the candidate's team means being optimistic; it’s crucial not to take on the gatekeeper role, which often leads to a negative experience.
00:29:47.880
Focus on providing avenues for improvement where needed, recognizing their strengths, and addressing challenges along the way.
00:30:03.840
Additionally, respect the candidate’s boundaries during the interview. Candidates, like all of us, have basic needs regarding food, water, and breaks.
00:30:20.760
Be aware of any necessary accommodations required by your local law, as being respectful and inclusive is the right approach.
00:30:38.400
Encourage humane interactions. Offering candidates insights on the interview format can provide necessary support for them as they prepare.
00:30:54.240
Approach your interactions with authenticity, but also remember to steer clear of personal inquiry in interviews. Even seemingly innocent questions can backfire.
00:31:11.520
Questions should focus on the job: what excites them about the position or what aspects they enjoy in their current role.
00:31:29.160
In conclusion, this will help create a better atmosphere for all parties involved. Thank you for your attention.
00:31:47.460
To summarize and for a cheat sheet, keep questions simple, avoid jargon, maintain familiar contexts, discourage over-specifying, and encourage candidates to ask questions.
00:32:03.599
Use rubrics for evaluations, consistently question your evaluations, and consider if mistakes truly matter. Prioritize gender-neutral language in your write-ups.
00:32:24.480
Maintain a positive attitude, be supportive, focus on strengths, and always prioritize candidates' wellbeing. Interviewing is about finding the best fit through collaboration.
00:32:51.000
Thank you very much. Please engage with me on Twitter and elsewhere, as I'm deeply passionate about this topic!
