Human Errors

by Olivier Lacan

In the talk titled "Human Errors" by Olivier Lacan, presented at RubyConf 2017, the speaker explores the challenges programmers face with runtime errors and how Ruby's error feedback mechanisms can be improved.

Key Points:

Understanding Errors: Lacan distinguishes between general mistakes and specific technical errors, emphasizing the importance of context in understanding how to remedy them.
The Nature of Human Errors: He presents the analogy of wandering when lost and connects it to how programmers feel when they encounter errors, noting that friendly and clear feedback is crucial for recovery.
User Perspectives: Lacan introduces the concept of "middle users" (developers) versus "end users" and stresses that both groups deserve clear error messaging.
Error Handling in Ruby: He highlights improvements in Ruby's error messages over time, particularly how the language has integrated tools for better diagnostics, such as the 'Did you mean' feature to suggest corrections for typos in method calls.
Case Studies in Error Messages: Lacan discusses specific examples where Ruby error messages fell short, such as when features are refactored or how certain methods provide ambiguous error responses, which can lead to confusion.
Call to Action: He encourages the community to enhance error messaging to include information like column numbers and contextual suggestions, ultimately fostering a better developer experience.

Conclusions:

Lacan concludes that better error messages can dramatically improve the experience of developers working in Ruby and reinforces the necessity for collaboration between library maintainers and users to enhance understanding of error contexts. This focus on context-sensitive feedback can lead to healthier coding practices and encourage a culture of improvement within the developer community.

00:00:10.700 All right, what a more intimate setting! How's everybody doing? That's the only way I can tell how many people are in the room.

00:00:14.160 When you make sound, I can understand. So, I'm going to ask you to make sound for random reasons. I know Katie here is going to make sense; she's a really good heckler. Just learn from her, I guess.

00:00:21.600 So, first of all, welcome! I hope the keynote was really nice. I was frantically finishing my slides while they were speaking, so, of course, I didn’t see it. If there’s anything really serious I should know, like someone screaming at some point saying, "I don't know, type safety is a thing now" or something like that, just let me know.

00:00:39.239 The first thing I will ask of you is if there are any humans in the room. I'd like to point out that there are people not reacting at all to this thing, so there are probably aliens among us, and they're probably next to you.

00:00:47.790 I don’t speak alien, and this talk is not for aliens. You can translate for them if you want. Speaking of aliens, I'm actually an alien myself. I come from a planet of weird people who took Louisiana from its native people and then sold it back to you Americans, if there are Americans in the room.

00:01:06.689 Now, you don’t have to clap. When I say Louisiana, it’s like—that's the Louisiana that was sold, and there’s like Denver in there, which is really cool too. I feel like, damn it, I could have had an easy vacation if Napoleon hadn’t said, "I don't need this anymore."

00:01:19.140 My name is Olivier. This is how you pronounce it: you go, "Oh Lee V. Zen." At the end, you feel like that! I am obviously French, which is just like this town that you’re sitting in, by the way. So, you are sitting on French ground right now. Oh, I am so sorry.

00:01:57.750 My full name is this—I’m not going to pronounce it. You figure out how to say that; that's my internet name because that's what my mother named me. For over five years, I have been developing, maintaining, and securing codeschool.com, the place that brought you Rails for Zombies, the not-so-broken Try Ruby, and a few too many jingles.

00:02:21.540 Since I'm French, when I make mistakes, I often claim that it's on purpose. Airports are really hard with the mic, because you can't do this in a professional setting.

00:02:35.850 I remember one time I ordered the same dish as my fiancée at the Epcot Food and Wine Festival. I didn’t understand—or by which I mean I didn’t listen—that she wanted to share her food with me. So, when it was too late to cancel the order, I assured her that no, I was really hungry and wanted to order that.

00:02:46.440 I was, but not enough to waste a bunch of money on the exact same dish as her, and all of this happened because I was afraid this very non-threatening person would realize I had screwed up, just for that reason and no other.

00:03:11.250 Mistakes are made, and it’s really hard to admit them. But it's often harder to understand how you've managed to make one, especially without feedback. Let’s say you make an error, and you know that the path to recovery is this far away. Feedback can help, but there’s something missing between those two points—a cognitive step that helps you reach recovery.

00:03:39.989 If you have context as a little stepstool to reach recovery, it’s that much easier. That’s sort of what I’m going to try to talk to you about today, along with some funky animations.

00:04:04.239 The why is usually the first thing that people ask when there's an error. "Why did it happen to me?" We often do not understand why computers seem so easily displeased with us, yet we can be so easily displeased with users if they stray away from the happy path.

00:04:26.710 The most interesting questions to ask are not really why, but what, or how, or when, or where. So that’s what we’re going to talk about today. However, first, I have to do the formal thing, which is to define what I’m talking about.

00:04:47.259 We’re going to talk about definitions and words and concepts; you’ve been warned. For those who are following me on Twitter, I’m so sorry. An error is a mistake, which could be a spelling error or an error of judgment. The broadest definition is the state or condition of being wrong in conduct.

00:05:09.039 That’s something you did or a judgment you made. The specific definition for us technical folks is a measure of the estimated difference between the observed or calculated value of a quantity and its true value.

00:05:28.030 To simplify for those who don't like long definitions: the observed value does not equal the true value. I’m really excited about the origins of words, the etymology, because it helps understand how they came to be.

00:05:57.490 The word error comes from the Middle English 'erro', which I don’t know why in French we use for 'errer'. In Latin, it's 'erro', so I really want to say 'err' but that’s odd since it happens to be 'error' in Latin.

00:06:10.420 The Latin version is interesting. It’s about wandering when you're trying to go somewhere, and you lost your way. I love that metaphor—you wandering about trying to reach your destination.

00:06:43.150 Wandering can be enjoyable if you have a friendly environment, with lots of signposts to guide you back to your goal. Otherwise, you might encounter hostile people along the way.

00:06:56.170 We all want to stop making mistakes, but it’s impossible. We know it, but we often don’t internalize the fact that we're always going to make mistakes. Disregarding them won’t help.

00:07:14.410 During moments like this, I like to think about reframing at code school, for instance. A way to reframe is to do support work. If you work for a product company, talk to the humans who are using the software we create.

00:07:39.289 These individuals are our targets; the people we’re trying to build things for. Broadly speaking, for us, they are end-users, which is a really weird phrase when you think about it. It sounds like we’re hatched in secret plans to eradicate all users.

00:08:00.769 This is definitely not what we want to do. Instead, I want to discuss middle users, which is my new neologism. I’m guessing that the people in this room are end-users of many things, but there are also middle users for a lot of different things.

00:08:33.149 We’re not just the people using the finished product; we are developers. I’m going to do a terribly awkward analogy that goes like this: from left to right, we have the start user, or the robot or computer, and in the middle, we have the middle user.

00:08:52.400 We go to conferences and spend all day inside instead of eating po-boys and sipping coffee from the French coffee truck. It's awesome!

00:09:04.360 Then there’s that little rascal running away, our end user, darting across the street without looking and clicking on things too fast, without a coat on.

00:09:32.700 So occasionally, the start user—our robot, which could be the language or the operating systems that we use—decides to yell at our defenseless little end user. When that happens, I say 'decides' as if it has a mind of its own, but really, it's often because we drop the ball.

00:09:58.680 We gave our little robot way too much power over people, and if they don't understand what he’s yammering on about, they might encounter error codes like 253.

00:10:06.599 So, we intervene. We put on our fancy error handling badges, and just to be sure, maybe a helmet because those pincers look a little scary. Then, we whack the start user with our justice hammer!

00:10:33.210 It’s not just the end users who suffer from confusing error feedback. When the feedback doesn’t seem appropriate or is too difficult to understand, it can also be tough for middle users like us who build software for the end users.

00:10:54.440 Instead of handling the user experience in a friendly way, we might encounter errors from tools like Windows, iOS, or those new credit card chip readers that beep at you viciously even when the transaction succeeded.

00:11:20.000 We find ourselves receiving generic Postgres, Ruby, or SAS errors. Just a few months or maybe years ago, I started noticing strange exceptions popping up from call sites for methods that had been recently refactored.

00:11:46.890 Let’s say we have a method called 'explode' that takes a single code argument and prints the string, but when we call it without an argument, we get 'ArgumentError: wrong number of arguments (given 0, expected 1)'. This feels normal.

00:12:14.590 The backtrace that follows the exception message points to the first line of the explode method definition, and when we look at it, we see that we wanted a code argument. So, everything seems fine.

00:12:34.900 When you call the explode method again, providing the argument, it works just fine, and everybody’s happy. But what happens when our team refactors this method to make its interface more explicit, changing the code argument into a required keyword argument?

00:12:54.020 Now, when we call it with no argument, we receive a very useful error: 'ArgumentError: missing keyword: code'. Cool, I know what I’m supposed to supply. But what happens when we pass the original argument with no keyword?

00:13:08.140 A missing keyword expected one argument, got one argument. As it turns out, that was wrong. It’s hard to make sense of if you're not watching closely.

00:13:18.470 The response seemed really confusing, and I asked a bunch of people who were smarter than me for answers. Turns out it is indeed a bug! I submitted that to the Ruby bug tracker about nine months ago.

00:13:35.700 As soon as you submit an issue on the Ruby bug tracker, it's no boo time, which means it takes way too much time to get resolved. I’m not going to point out every issue.

00:14:04.700 The patch monster, Nobu, has been amazing in responding to reports. Almost immediately after submitting my issue, I received a response from him addressing the bug.

00:14:42.970 This fix was merged within four months, so now when you use the old method signature and send it, you still get a not-so-great error message, but at least you now have one more useful piece of information included.

00:15:04.970 At least now it includes the message which mentions the required keyword; this change should be included in the Ruby 2.5.0 release!

00:15:45.200 If you're curious about how to get your questions answered quickly, this talk explains how to leverage the community and the resources we have. After the conference, I encourage you to check out the Ruby bug tracker or to help improve these errors.

00:16:24.130 Now that’s out of the way, let’s talk about joy! I have a lot of joy right now. I’m too excited to be here, and speaking of excitement, Matt mentioned that for him, the purpose of life is partly to find joy.

00:16:56.110 Programmers often feel joy when they concentrate on the creative side of programming, and Ruby is designed to make programmers happy. Ruby has inspired many other languages and frameworks to focus on this side of performance.

00:17:25.700 Rust now provides error messages like, "could not find type Neuromancer, but did you mean necromancy?" That's an amazing touch. Elm also does this well, and I’ll explain later.

00:17:54.290 Ruby has also made strides in this area! Since Ruby 2.3, the 'Did you mean' gem has been integrated into Ruby. So, if you misspell a method like 'explode', you’ll receive helpful suggestions.

00:18:23.490 When you encounter an error that seems confusing, having an easy and quick recovery option is super beneficial. This is a vital aspect of language or framework design.

00:18:59.140 Let’s look at the Ruby exception classes. Currently, we get information like backtrace and cause message, but if we could also provide column number and the specific keyword causing the exception, it can help developers tremendously.

00:19:27.900 I asked Matz about this recently, and it might be possible to enhance this feature within the Ruby source code.

00:19:52.660 Speaking of which, we encountered this issue with a two-year-old proposal aiming to improve error messaging to include more context. Someone submitted a functionality that would help clarify the last method call on nil.

00:20:12.450 This would help users understand what triggered the undefined method error, thereby providing clarity on the broken chain there.

00:20:35.930 Returning to the topic of contextual errors, if we had information like the specific call that led to an error, we could improve the developer experience dramatically.

00:21:03.080 The distinction between merely showing an error message and providing a pathway to understanding the root cause is profound.

00:21:25.010 Let’s examine a not-so-great error message. For instance, with RSpec, encountering an 'undefined method shell escape for path name Lib' error can leave you puzzled.

00:21:58.270 You look at the backtrace, which first points to 'this thing runner', and if you don’t know what that means, it can be challenging to respond.

00:22:20.340 So I usually end up using tools like Pry or better errors to figure things out quickly.

00:22:42.480 While the errors indicate where the failure occurred, they don’t provide much context. It’s often difficult to discern what action might have caused the problem.

00:23:02.060 For another common error, many beginners try to install a gem using sudo, they might encounter a permissions error as they try to update it later.

00:23:22.460 Again, the error message can be clear in what happened but not in how to fix it. There’s often no useful context, leading to users rating it only a five.

00:23:42.640 This helps highlight the gaps in these error messages and indicates a need for better recovery pathways.

00:24:02.700 So, speaking with Justin Searles, we discussed how a library or application lacking a need to rescue a potential error shouldn’t be an excuse to ignore how confusing the error might be.

00:24:21.759 With Rails, the test files can lead to errors not being handled well, hence it’s essential to work collaboratively between library creators and users to foster better contextual error handling.

00:24:41.420 We need both the library maintainers and users to grasp the kinds of context-sensitive feedback they can offer to help clarify errors.

00:25:09.500 In this regard, we should make it our goal to create better error messages. The overall context of errors is critical to improving our user experience.

00:25:31.050 Let’s take pre-existing errors and enhance their clarity, especially in complex interactions.

00:25:51.770 The point is clear, and many students get frustrated when these occur because the context is so hard to decipher.

00:26:13.520 In summary, let’s work on taking better care of our exceptions because they are an integral part of our programming practices.

00:26:51.319 I think we can make language and context improves a better quality of development overall.

00:27:03.109 Thank you!