Keynote: I've made a huge mistake

Talks

Aaron Patterson

#software-development

#rails-plugins

Keynote: I've made a huge mistake

by Aaron Patterson

In his keynote speech titled "I've made a huge mistake," Aaron Patterson discusses several themes centered around failure and personal growth in the realm of software development. He combines humor and personal anecdotes while reflecting on his experiences as a developer and his role within the Ruby on Rails community at Rails Conf 2012.

Key Points Discussed:

Introduction to Personal Background:

Aaron shares his role as a Senior Software Engineer and expresses gratitude for the opportunity to speak at the conference, emphasizing the excitement and nervousness of being on stage.
Topic of Annual Reviews:

He describes the complexity of filling out an annual review form while navigating various technical challenges related to software compatibility, illustrating this with a detailed example of accessing a specific webpage through different browsers and configurations.
Focus on Personal Failures:

Title of the talk reflects Aaron's intention to address failures personally encountered.
Failure to Remember:

He highlights a project named Tokaido that aims to simplify the Ruby installation process for new developers, remarking on the hurdles often faced by beginners and educational settings.
Failure to Evolve:

Patterson discusses the significance of concurrency in applications and criticizes the Rails framework for not having a uniform queuing API, which complicates scaling email functionalities in production environments.
Failure to Lead:

He touches upon the development of features in coding, particularly how experienced developers are less tolerant of technical debt. He divides features into three categories: cosmetic, refactoring, and course-correcting, each with its implications for long-term maintenance and system architecture.
Technical Debt Dynamics:

The presentation discusses the balance between including new features and managing the associated debt, explaining how this trade-off affects overall project quality.

Important Examples and Case Studies:

Action Mailer and Queuing:

Aaron exemplifies the need for a queuing system in Rails, explaining the confusion created by different queuing systems and their interfaces, advocating for a consistent API.
Book metaphor for computation distribution:

He elaborates on the evolution of computing, indicating how control has shifted from centralized systems to distributed ones, highlighting browsers as more powerful tools capable of executing more complex computations.

Conclusion and Takeaways:

He encourages developers to recognize the importance of their time, avoid unnecessary complications in their workflows, and strive for an evolution in tech that alleviates burdens related to installation and feature adoption. Aaron emphasizes the necessity of evolving coding practices to manage technical debt effectively, aligning development efforts with future changes in technology.

00:00:17.199 I was always looking for ways to make people smile. Since we get hungry every day, we have to cook every day. Even though cooking is fun, doing it every day makes it something you have to do. We want to inspire people to enjoy cooking.

00:00:45.440 First off, I want to say that I really appreciate it. I'm super excited to be the first person to welcome you to RailsConf. Thank you! My name is Aaron Patterson. I'm a senior software engineer. If you have questions about my talk or just want to say hi, feel free to tweet at me or send me an email. Unfortunately, I won't have time for questions after this.

00:01:00.000 Now is the time when I nearly die from nervousness on stage. To help with my nervousness, I have a slide here. I think to myself, "What would Freddie Mercury do?" I just have to think about that, and hopefully, I'll get through this. I'm a member of Ruby core and Rails core, and I work for a company called AT&T Interactive. I have to thank them because if it weren't for them, I wouldn't be here today.

00:01:22.799 I am employed to do open-source software for AT&T. Thank you! Last year, I talked to all of you about how I file my expense reports, and that seemed to go over pretty well. But I figured you probably already heard that once, so today I decided to tell you about how I get my annual review.

00:01:52.920 The way that it works is that I work from home. I have a laptop, and my network setup looks kind of like this: I have a laptop that connects to the internet, which is amazing. I need to fill in my annual review, and it's on one of those websites where you have to boil your entire self down into a few meaningless categories with some unknown number in it. You know what I'm saying? Like, yeah, I'm a three or something.

00:02:25.480 That's what I have to do. The way that I access that is I have to go through an amazing VPN. So I fire up my little Cisco VPN, and it tunnels through the cloud to get me into work. I'm doing that, working on my thing, but then I finally access the website, and it says, "Hey, your browser is not supported!" I'm like, "Oh crap, okay, I'm running Chrome." So, I think, "Okay, I'll switch over to Safari." But then it tells me I still need an older version of Safari.

00:02:52.080 I think to myself, "What am I going to do now?" Then it dawns on me: I actually have a Windows VM on my laptop. I fire that up and go to Internet Explorer to hit that website because, of course, it wants Internet Explorer.

00:03:19.320 I do that, but then it says, "Your browser is not supported in IE!" I'm like, "What?" They tell me I need to run IE7, not IE9, or whatever it was that I'm running. I have no idea how to downgrade IE on my machine. So, I'm wondering what I'm going to do. Then I realize I have a Mac server at home with a Windows VM running on it. I can connect to my Mac, fire up the virtual machine, hit that, and go back out to the network using IE7.

00:04:02.720 That sounds great! Except my Mac doesn't have access to the VPN. So, I decide to do an SSH tunnel on my laptop to route IE through the SSH tunnel and back up through the VPN. Here’s the network diagram of that; this is technically known among network architects as a Rube Goldberg network architecture. Finally, I filled all of that out and got it done.

00:04:42.760 This year, I worked very hard. Last year, I was amazed when Cory Haines gave a talk, and I noticed on the program that Cory Haines's title is just "Cory Haines." So, I'm super jealous of this; it's really amazing! I worked really hard, and I actually got a promotion. I'm no longer a senior software engineer; I am now... Cory Haines!

00:05:14.720 I want to talk about failure, specifically my personal failures. I have tentatively titled this talk, "I've Made a Huge Mistake." The first thing I want to address is the failure to remember.

00:05:48.840 To explain this, I want to discuss a project called Tokaido. This project has been developed by Yehuda Katz, who received funding from Kickstarter. I'm really excited about this project. Besides having an awesome name, Tokaido has Chinese characters, which you can even get tattooed on you! Then, you might meet a Japanese person who asks why you have this tattoo, and you have to explain it.

00:06:47.560 Tokaido is a binary distribution of Ruby for OS X, which ships with all the gems required to run Rails. Ideally, you should be able to double-click it and have it installed, then fire up your terminal to use 'rails new' without needing to install any extra gems.

00:07:06.240 I'm excited about this for students—not just students in a classroom setting, but anyone trying to learn Ruby. This project can save them a lot of time because they can install Rails without worrying about extra dependencies.

00:07:43.040 I'm also excited about this from the perspective of teachers. I used to teach Ruby at the University of Washington, and I remember how our first entire class was spent installing Ruby and getting Rails running on students' laptops.

00:08:22.080 But I'm really excited about this project for all developers. I want to ask you: how important is your time? Most of you would say that your time is very important since you can't get it back. You must use your time wisely.

00:08:46.880 But when it comes to installing Ruby, many may think it's just five steps from Hacker News. Just complete those steps, and then it will all be done, and you're good to go! However, do you know what a shared object is? Do you know what shared objects your extensions load?

00:09:04.720 When you install Ruby from source, eventually something is going to go wrong, and you might link against the wrong shared object. To address these issues, you need to be able to answer questions such as: what is the 'dld Library path'? What is 'dlopen'? How do I find out how to use 'o tool'?

00:09:34.080 The truth is, when you're installing Ruby and all your gems from source, you'll eventually encounter failures, and it's going to require you to dig deep to figure out what went wrong.

00:09:59.840 Now, I'll ask you again: how important is your time? Would you rather waste your time looking up the order that 'dlopen' searches in 'dld library path' or would you rather be working on your Rails application?

00:10:38.920 The other thing that's exciting about Tokaido is the ecosystem. I feel kind of bad for Yehuda because this is a very ambitious project. He will need to do a lot of leg work to get it running.

00:11:16.000 He's going to have to work with people who develop C extensions and get them to create statically compiled extensions. He will also need to work on tools like 'rake compiler'. If you author a C extension, you should be using rake compiler. He has to improve the toolchain; otherwise, nobody will build these things, and his project will fail.

00:11:46.080 Now, if you still don't believe me that this is a good idea, I encourage you to volunteer for RailsBridge. If you're convinced that those five steps on Hacker News are the best way, go volunteer for this group.

00:12:05.720 Not only will you be doing a great service by helping others learn Ruby on Rails, but you'll also gain a firsthand understanding of the problems they encounter while trying to learn.

00:12:33.360 The next thing I want to discuss is failure to evolve. I feel I have failed to take concurrency as seriously as I should in Rails, and I want to improve this situation.

00:13:00.960 Does anyone know why we have ActionMailer? The reason we have ActionMailer is that we want to send emails easily from our applications. We might write code that looks something like this, which I'm sure many of you have done, likely in your production applications today.

00:13:30.560 The issue is that when you deploy this to production and you need to scale, you might find that processing your request can lead to sending an email, which becomes a problem because if there's an exception during the sending process, users might receive a server error.

00:14:05.920 If your SMTP server is slow or you're shelling out to send mail, you're going to experience poor response times or, in a worse-case scenario, display a 500 error.

00:14:40.720 It would be much nicer to pull sending emails out of the request-response process. We should process the request, send the response quickly, and handle the email in the background. This follows the producer-consumer pattern.

00:15:26.320 The controller produces that work (sending an email), and then we have a consumer that actually does the emailing. Now the question arises as to why we have ActionMailer but no queuing system.

00:15:54.640 In Rails, we do have a queue called ActiveQueue, but the main issue is that nobody uses it due to a lack of documentation, examples, and lots of configuration.

00:16:31.640 To use ActiveQueue, we instantiate a new ActiveQueue and enqueue our runnable. When we want the consumer to consume the emails, we create a new thread to pop off the queue and run it.

00:17:07.600 The configuration for ActiveQueue is quite simple; however, there's actually no implementation as the ActiveQueue's implementation is zero lines. This sending email issue remains real, and many developers opt for background job processors like Rescue or Sidekiq, but they all have different interfaces.

00:17:49.440 This leads to a lot of work migrating between job processors if you need to scale, which is frustrating. I feel strongly that we should demand a consistent API from these libraries. They should implement common interfaces so we can switch between them easily.

00:18:32.090 I would like Rails 4 to become the driving force of such a consistent API, providing a sample implementation based on Ruby's normal queuing system. That way, when you want queuing capability within Rails, you can conform to this API.

00:19:05.400 I want to discuss another area where I feel I've failed: leadership with respect to features. I think many older developers are afraid of features, but it's not that they are actually afraid. Older developers tend to have lower tolerance for technical debt.

00:19:48.600 When a particular feature comes along, they fear the potential debt associated with that feature. If I were to graph a developer's tolerance for debt, I would show that younger developers tend to have a high tolerance for debt while older developers have a lower tolerance.

00:20:25.880 Thus, younger developers might be quick to accept new features without realizing the maintenance costs that could arise. This correlates with a tendency to merge code that comes from forks without considering the potential drawbacks.

00:21:06.960 So, let's categorize features into three types: cosmetic features that have dubious value and unknown debt, refactoring features that have high value and potentially negative debt, and course-correcting features, which are also high value but have unknown debt.

00:21:51.520 For example, tag logging in Rails is a cosmetic feature. It was designed to log information with certain tags, but it lacked thread safety and compatibility with some logging libraries. The implementation took many commits to address the various concerns that arose.

00:22:52.640 Refactoring features tend to have high value and could even reduce debt. An example would be the notification improvements in Rails 4, which offer better event handling capabilities.

00:23:30.000 Last, course-correcting features, like the asset pipeline in Rails, help unify approaches and solve issues developers encounter. While we know we have to move in a particular direction, we might incur a lot of debt in the process.

00:24:11.680 Though, we must remember that for technical debt, there's no bailout plan. Nobody can just magically forgive or cover it up; it's a cost that someone has to pay down.

00:24:48.680 The landscape of computation is changing. We're moving from server-based processing to more computations being handled by the browser. We are attempting to distribute computational tasks more efficiently.

00:25:27.200 Thank you.

RailsConf 2012