00:00:06.080
Well, hello my friends! We're coming up on the midday-ish, mid-afternoon-ish time, at least for those in the central time zone. For the rest of you all over the world, thank you for kicking back and staying with us for another live session. I just wanted to share that, excuse my attire, but I completed my 5K today for the RubyConf virtual 5K challenge. If you haven't seen it, check the Slack; it's there! I encourage you to get going as well; you don’t have to run a full 5K—you can also commit to 30 minutes instead. Anyway, enough of that! Let’s dive right in.
00:01:00.399
Our next speaker is Jesse. I want to welcome him to the stage. Jesse, the stage is all yours. Thank you! Hello, RubyConf! I wanted to start by giving a huge thank you to the RubyConf planning team. Abby and Marty were my primary contacts, but I know there were many others who read proposals and worked hard to put this conference together, so a big thank you to them. I also want to extend my gratitude to our sponsors. But my biggest thanks of all goes to all of you tuning in right now. I truly appreciate it! This conference is such a treat during these uncertain times, both in the United States and across the globe. I’m so grateful to be here with all of you.
00:02:00.560
My name is Jesse Spevack, and I use he/him pronouns. I'm a staff engineer at Ibotta, a cashback shopping app. Ibotta has a great engineering culture where we use some of the most cutting-edge tools in the business. The work is super interesting due to the complexity of the domain and the large scale of our operations. Most importantly, the team is filled with fantastic people who have high expectations for themselves, so I highly recommend checking us out if you’re looking for your next position. Personally, I took a non-traditional path into software. I spent 11 years in the K-12 education space and came to software through a coding school in Denver called the Turing School of Software and Design. If you have friends or family contemplating a second career, we have a great Slack channel for second-career folks at RubyConf. I definitely encourage sending them to Turing; it’s been an amazing program for me.
00:03:19.840
So, today, I want to begin by discussing how interviewing is broken for job seekers. Next, I’ll talk about how it’s broken for interviewers. I will share my experiences interviewing hundreds of candidates and being interviewed myself a few times. I hope to paint a clear picture of the current state of interviewing. Then, I’ll explain how I worked with my team at Ibotta to reimagine our interview process while drawing on the lessons I've learned from the Ruby community. Finally, I’ll present some strategies you can use to reshape hiring at your own companies, regardless of whether you’re a CTO or a junior developer just starting out.
00:04:07.840
The Ruby community has a history of discussing interviews, and I want to shout out a major inspiration for me: Eric Weinstein’s RailsConf 2019 talk, "Interview Them Where They Are." It jolted me out of my complacent acceptance of the current state of interviewing, and I hope that this talk will have a similar effect on some of you.
00:04:19.600
Let me start by sharing my own interview experience from almost four years ago. I think it effectively illustrates the typical interview process at that time, which might remind you of your last interview or resemble your current company's process. The first step was a phone call with Human Resources. The purpose, I presume, was to ensure I didn't sound too much like a crazy person and to check whether my salary expectations aligned with the company's offer. Right from the start, this process felt broken. Corporate America systematically shares salary information through consulting groups that collect and sell this data, while the workers often do not have access. Nearly every HR team has paid for this data, and even if they don’t have it, they’ve had far more salary conversations than the typical developer. Thus, the interviewee is always at a disadvantage regarding the understanding of compensation.
00:05:07.760
Let’s pause and consider how this asymmetry affects certain groups more acutely than others. If you’re a white male like me, studies show you are statistically making approximately 5.4% more than your female colleagues in similar developer roles with the same level of experience. Reports such as Glassdoor's "Progress on the Gender Pay Gap 2019" and Hired Inc.'s wage inequality report reveal similar gaps based on racial and sexual identities. For instance, a Black male colleague is paid 91 cents for every dollar a white male earns, while male LGBTQ colleagues typically make 96 cents on the dollar. The situation worsens for female LGBTQ colleagues at just 92 cents on the dollar. This is not an exhaustive list, but the takeaway is that salary transparency is crucial in ending wage discrimination. We need to press our employers to stop trying to save money off the ripple effects of systemic racism and sexism.
00:06:38.560
Returning to my hiring process: after my phone screen, I interviewed with the engineering manager for the team I was applying to join. The purpose of this conversation was to gauge my potential value to the team and to excite me about working at Ibotta enough that I'd be willing to persevere through the rigorous process ahead. After the interview, I was assigned a take-home project, which involved building a set of APIs to perform various CRUD operations on words and their anagrams. In our community, a lot has been said regarding take-home projects. DHH notably pointed out in a blog post that many companies ask candidates to complete extensive projects that require 20, 30, or even 40 hours of unpaid work, which can be challenging for candidates juggling existing jobs and life responsibilities. Personally, I spent around 16 hours on the anagram API project, including implementing bonus endpoints, writing unit and integration tests, and preparing a comprehensive README file. After completing the project, I told my wife it was the best work I had ever done.
00:10:52.640
A few days after submitting the project, I was invited to a remote project feedback session. I was excited to hear the praises the engineers would surely have for the 16 hours of work I put into my anagram API. I felt certain my wife’s two days of single-handedly caring for our then one-year-old twins while I coded would not go unrewarded. During the feedback session, I walked through my code with two engineers at Ibotta. As someone with virtually no professional coding experience at the time, I had assumed that their roles were primarily to provide feedback on the code written at Ibotta. However, they gave me considerable feedback that was surprising. They questioned why my median calculation might not be accurate or why I chose not to optimize certain queries. The session was brutal. I emerged from the bedroom, dripping with sweat and feeling totally dejected, convinced I wouldn’t get the job at Ibotta.
00:12:00.560
Let’s summarize the many ways this take-home project phase is broken. First, Ibotta isn’t an anagram company; it’s a cashback shopping app. Reflecting on my past three and a half years with Ibotta, it’s clear there’s little overlap between the code I wrote for the anagrams and the code I write to reward shoppers for their purchases. Secondly, the 16-hour investment I made for the project did not result in any compensation. I was desperate to get my foot in the door, but many others may not have the luxury of dedicating that much time due to their life circumstances. For me, the most taxing part of that timeframe was the childcare my wife provided. As a single parent, I would not have managed to complete that project at all. Finally, after the feedback session, I was left feeling I wasn’t smart enough to work at Ibotta. To my surprise, I was called in for a half-day in-person interview.
00:14:30.240
During the interview, I entered a room with three engineers waiting for me. I was asked questions about my background and then directed to a whiteboard to write a function that takes in an array of integers or arrays. This function outputs the sum of all integers. I was then asked to model a game of chess on the whiteboard. After that, a new set of engineers entered and repeated the process. I had to implement a function to reduce overlapping ranges on a number line. This question included a trick I did not know: that sorting the list of ranges simplifies the algorithm for combining the overlapping ranges. Certain issues arise from this process, particularly from a candidate’s perspective. First, I was asked to write a recursive function in front of an audience. The only recursive function I’ve seen in my time at Ibotta came up a year prior, and I had removed it because it was confusing.
00:16:21.440
Not everyone enjoys chess as much as I do. Moreover, the range reduction trick is somewhat of a gimmick—those familiar with resources like "Cracking the Coding Interview" or certain computer science classes might know this trick. I doubt there's a true correlation between knowing such tricks and being a productive software developer. Furthermore, I haven’t mentioned yet that the only non-white male I interacted with throughout my interview experience was Kayla from HR. Generalizing my experience: I completed a time-consuming unpaid project, felt inadequate because of it, was asked to perform at a whiteboard on unrelated problems, and most crucially, I had it easy. People often make assumptions about me based on my appearance and speech, especially in software—assumptions that usually work in my favor; a white male statistically has benefits that others often do not.
00:18:52.600
In summary, this process is just not nice. Next, I want to discuss how this process is broken from the interviewer’s standpoint. Many agree that the interview process is flawed for job seekers, but let me explain why it’s also detrimental for interviewers. Over the last four years, I’ve led approximately 200 technical interviews that followed this same structure. The goal of these interviews is to gather data to estimate the candidate’s potential value to the team. An ideal process would yield accurate estimations of this value. At Ibotta, we use a four-point scale: strong no, no, yes, and strong yes. I’d like to illustrate the discrepancies I’ve encountered over my history of interviews.
00:20:29.440
I've only given one or two strong no's out of my 200+ interviews. In those cases, the applicants did not get the job, so I lack insight into their true capabilities from working with them. Conversely, I’ve given fewer than 10 strong yeses, and although those candidates were hired, only a fraction accepted the offers. Among the strong yes candidates whom I did get to work alongside, I must admit that once, I was completely wrong about my assessment. This suggests that roughly 10-20% of my strong yes predictions were incorrect. Consequently, I’m left with a significant number of interviews where I lack confidence in the data I’ve gathered.
00:21:54.880
Sometimes I’ve worked with candidates I’d rated negatively, only to be pleasantly surprised by the value they brought to the team. Conversely, I’ve also encountered candidates I rated positively who didn’t live up to expectations. While it’s conceivable that I have poor judgment, and others might be more adept at reading people, I believe I possess at least an average judgment. Thus, it seems the real issue lies in the quality of data retrieved during this interview process. The process does not provide sufficient quality data to enable interviewers to make confident decisions.
00:23:48.560
Let’s revisit that process one more time: we ask our candidates to complete a take-home project with hardly anything in common with the actual job, and after they endure our harsh feedback, they must answer questions on a whiteboard—questions we already know the answers to—that also differ significantly from the work they’d be undertaking on our team. Again, the process just isn’t nice. In the Ruby community, we have an unofficial motto: "Matz is nice, and so we are nice." This slogan, which refers to Yukihiro Matsumoto, the creator of Ruby, has shortened over time into Minaswan. So, what if we conducted our interviews with kindness? After all, if Matz is nice, we should be, too.
00:25:49.200
Enter the Minaswan interview. I want to dive into the elements of our Minaswan interview, but first, I should emphasize that it’s not a strict process to follow; it’s a mindset. This is a profound concept: if we treat each other with kindness, everyone benefits. I will describe our interview process and highlight elements that reflect our Minaswan approach, as well as areas for enhancement. The Minaswan interview is significantly shorter than my previous experience. Even if the quality of our data remains the same, spending less time collecting it is still a win for both the candidate and the team. We still conduct the phone screening and the managerial interview, but we completely eliminated the take-home project.
00:27:34.560
Once the manager approves a candidate, we schedule a 90-minute technical interview with two Ibotta engineers. This interview consists of four parts. In part one, we introduce ourselves, outline the structure of the interview, and allow the candidate to introduce themselves. The goal here is to put the candidate at ease. Some candidates feel relieved knowing what to expect, which can significantly ease their stress. The next part of our technical interview requires candidates to read some code we've written. We call this our 'Code Composition Module.' We explain that this code represents the domain they would be working in if hired, and we want them to discuss what the code does at a high level, what they like about it, and what they dislike.
00:29:20.560
We don’t provide actual code from our production code base; that would be too complicated and difficult to fit into a 20-30 minute conversation. Instead, we prepare a few Ruby files and tests and give them to the candidate. We encourage them to open the files in any text editor or IDE they're comfortable with. Again, the idea is: Matz is nice, and so we are nice! Many interviewees have not previously worked with Ruby, so we assure them they can use Google for assistance and ask any questions throughout the process. This is significant because most developers will need to do this when they join our team. Overall, we aim to frame the conversation positively, suggesting the candidate will succeed.
00:30:55.840
Another part of our language assures candidates that the exercise isn't a bug hunt. Developers are often conditioned to feel they might be tricked or that there’s a gotcha in the interview. To counter this, we explicitly confirm that the code is bug-free. We also carefully explain why we want candidates to read our code: much of their initial work will involve reading and understanding code. The best candidates will be able to link individual lines of code and the broader business picture, ask insightful questions about our technical choices and user stories, and show genuine enthusiasm for the work at hand.
00:33:29.120
This part of the interview allows us to assess their skills based on how they think through the code. After the discussion, we usually spend about 30 minutes asking the candidate to write some code. This Code Composition Module involves having them manipulate JSON outputs—similar to much of the work performed daily at Ibotta. We begin by displaying an example output, moving backwards to detail the input data structure. This gives candidates clear objectives for the task and reinforces that there’s no trick involved. The process and their implementation approach matter more to us than the specific output—success lies in how thoroughly they approach the challenge.
00:35:27.440
Throughout this, we maintain a welcoming atmosphere; candidates can tap us with questions as if it were their first day on the job. The candidate’s performance in this section offers insights into their planning and coding style. The best candidates develop a plan and document their thought processes clearly. Their systematic approach reveals collaborative qualities and how they respond to feedback. As we conclude, we allow time for the candidate to ask us questions and provide immediate feedback to ensure they know their strengths during the interview.
00:37:33.600
Usually, candidates who excel in this technical interview have successfully completed our only technical hurdle. Afterward, we invite them to participate in a subsequent 2-part interview: a culture add interview evaluated by developers based on discussions about their work experience, followed by a leadership interview conducted by the hiring manager. To summarize, the old interview process was not nice. The new interview aims to be nice, striving to gather data that allows for swift and accurate assessments of a candidate. The approach shortens the overall length while directly correlating each component with skills we genuinely value.
00:39:30.080
Of course, there’s room for improvement in our methodology. For instance, our Code Composition Module tends to favor those comfortable with languages such as Python, Ruby, or JavaScript, where JSON is straightforward. We’ve developed starter files to assist our Java candidates, but it’s a notable weak spot. Furthermore, we haven’t been entirely consistent in how we execute interviews, particularly regarding the level of support we extend to candidates. Although we tend to offer more guidance to junior candidates, we recognize the risk of unconscious bias and are committed to addressing this area, ensuring equal support for all.
00:41:22.440
I'd also like to see increased salary transparency, especially when working with candidates from underrepresented groups. I hope I’ve demonstrated the effectiveness of the Minaswan interview and would like to end with some suggestions on garnering support for this process. First, many tech companies have embraced remote work since March. This transition offered a natural opportunity to reboot our process. Second, ensure HR is on board; a straightforward method is to shorten the overall interview process. HR despises it when candidates drop out because they receive offers from other companies or feel overwhelmed by a lengthy process.
00:43:50.000
Third, involve engineers; you can’t achieve this alone! I initially assembled some questions but quickly shared my efforts with engineering managers and individual contributors for feedback. This collaboration was invaluable, resulting in numerous pull requests on the repository we used for the initiative. Our Minaswan interview has been in practice for a few months, and while it’s too soon to confirm impeccable decision-making outcomes, the feedback we've received is promising. For instance, engineers appear more enthusiastic than before to conduct interviews, and HR is thrilled with this new approach.
00:46:15.440
Finally, candidates are expressing that they genuinely enjoy the interview process. I’d like to share feedback from one candidate that I’m particularly proud of. In an email, they mentioned, "I wanted to provide positive feedback regarding the interview on Thursday. I understand that you are transitioning from a take-home assignment to a live pair-style interview. I believe this style is potentially the best form of technical interview I’ve encountered so far. I sometimes feel frustrated by contrived elite coding-style interviews, but I really thought the interview segments allowed for practical assessments of a candidate’s knowledge and capacity to learn new concepts and languages. I hope you feel the same, and that other candidates do as well, as I would love to see the industry move in this direction for technical interviewing techniques. Though I don’t usually comment on my interview reflections, I wanted to give credit where it is due.”
00:48:56.960
All this to say: Matz is nice, and so we are nice. Thank you very much, RubyConf! This has been a lot of fun. I will be available on Slack to respond to questions immediately after this call. I appreciate your time, and I look forward to seeing you all at the next session!
