Make Ruby Write Your Code for You

by Alex Stephen

In the talk "Make Ruby Write Your Code for You" presented at RubyConf 2018, Alex Stephen discusses the concept of code generation using Ruby, emphasizing its practical applications in software development. He begins by introducing his background at Google Cloud, where he and his team tackled repetitive coding tasks that resulted in maintenance burdens. To address this, they developed a code generator called Magic Modules, which automates the writing of code based on templates and a word bank of parameters. The talk elucidates the principles behind this code generation process, making it more accessible to developers.

Key points discussed include:

- The Problem of Code Duplication: Repeatedly writing similar code for different features can lead to excessive technical debt.

- Understanding Auto-Generation: Rather than being intimidating, auto-generation is about instructing the computer to write code based on defined parameters. This concept is likened to creating Mad Libs—using templates that get filled in to produce dynamic code.

- Templates and Word Banks: Effective code generation relies on creating templates that resemble Ruby code, utilizing a 'word bank' of key values. ERB (Embedded Ruby) is introduced as a powerful library for this purpose, allowing for simple and effective template creation.

- Implementation in Practice: The speaker provides a practical analogy of ordering pizzas through Ruby code to illustrate how auto-generation helps manage complexity and enhance code maintenance.

- When to Use Auto-Generation: Stephen advises evaluating new features for similarities to existing code to discern if auto-generation is a suitable approach.

- Structuring the Word Bank: YAML is suggested as a format for creating the word bank, facilitating easier object creation and management.

In conclusion, Stephen's presentation offers a compelling argument for utilizing code generation as a means to simplify code writing processes and reduce maintenance burdens, making it an accessible tool for Ruby developers. He encourages the audience to be innovative in their use of these techniques as a way to streamline development workflows. The talk wraps up with a call to explore the Magic Modules project on GitHub and connect with him on social platforms to further the discussion.

00:00:15.350 Hi everyone, my name is Alex Stephen. I go by aTrambleRaptor throughout the interwebs. Today, I'm going to talk to you about how you can make Ruby write your code for you. Let me give you some quick background about myself. I work at Google, specifically at Google Cloud. For those of you who don't know much about Google Cloud, it does a lot—you can create virtual machines, container clusters, load balancers, databases, and more.

00:00:34.620 My team, in particular, focuses on open-source integrations. We look at open-source tools like Puppet, Chef, Ansible, and Terraform, and we try to add features to those tools to allow them to create virtual machines and SQL databases. When I first joined the team, we added support for virtual machines in both Puppet and Chef.

00:00:47.280 Eventually, we extended our work to include load balancers. By the time we got to the third or fourth feature, we noticed that if you squinted at the code for these different features, they looked almost identical. Yes, they were a bit different as they served different purposes, but they interacted with very similar APIs and operated in very similar ways. We questioned why we were writing so much similar code repeatedly, especially when we expected to create 50 or 60 different features.

00:01:17.400 So, we decided to build a code generator called Magic Modules. It's available on GitHub as an open-source project (github.com/GoogleCloudPlatform/magic-modules). Magic Modules is a Ruby-based code generator that takes information about Google Cloud and generates actual Ruby code for the tools we develop. As we progressed, we extended the generator to output Python code for Ansible and Go code for Terraform.

00:01:54.630 Code generation, or writing code that in turn writes other code, is something I've been doing for a long time. Today, I will share the concepts we utilized to create Magic Modules and explain how approachable this whole code generation process is. I will also discuss when you should and should not use this process in your own code development.

00:02:35.819 I've used the term 'auto-generation' frequently, which might sound intimidating. The idea that I'm writing Ruby that outputs Python and Go may seem wacky. Auto-generation essentially means that computers are writing your code for you. However, this notion can feel rather black-boxed and might make you feel powerless, as if what you're doing doesn’t truly matter.

00:03:03.060 Let me reframe the idea of auto-generation. When you are auto-generating code, you are instructing the computer on precisely what you want it to write and how to write it. The computer will then execute those instructions, creating the output exactly where you want it. Hopefully, this perspective helps you see it as more approachable—like another powerful tool in your Ruby toolbox.

00:03:41.189 In tech, we love the meme that something is requested to happen everywhere, right? I think of auto-generation as writing something once, then having the computer replicate it in multiple places with various modifications. But now you might be wondering how you actually auto-generate code. That sounds complicated. Has anyone ever done a Mad Lib before? Okay, about half the room has been to middle school. Mad Libs consist of templates containing blanks that you fill in with words from a word bank, resulting in various sentences. In essence, you can create numerous sentences from one template.

00:04:51.840 Now let’s consider applying this to code. For this talk, I’ll use a food-related analogy involving different food APIs that allow adding toppings and ordering food. In my word bank, I have things like a pizza class that takes options like pepperoni and sausage. The left side shows code that resembles Ruby, but with several keywords missing. Though variations in inputs could make the code valid if you had relevant objects instantiated, it shows the concept of using templates to create dynamic code.

00:05:59.670 Instead of filling a text template, we are filling a code template with parameters that, when executed, resemble valid Ruby code. If there's one takeaway from this talk, it’s that auto-generation is similar to a Mad Lib, taking a Ruby-like file and injecting keywords to culminate in valid, runnable Ruby code through the Ruby interpreter. However, it’s also essential to understand what auto-generation isn't.

00:06:36.150 In the context of Magic Modules and my experience, auto-generation isn’t about blockchain, Bitcoin, or machine learning techniques. There are no deep learning models or abstract syntax trees involved here. The auto-generation I talk about focuses purely on the mechanics of generating code without additional complexities found in other paradigms.

00:07:03.060 Auto-generation is a simple trick where a template helps to produce code that looks adequately similar to Ruby code. Now, when should we consider this approach? Before diving in, it's beneficial to discuss the abstractions we work with daily as Ruby programmers.

00:07:12.080 I've got an example of Ruby code where a block is called twice to order a pepperoni pineapple pizza. Most of us might consider it a good practice to avoid code duplication through function reuse, letting us replace repetitive calls with a single parameterized function, such as 'order pizza.' Such functions enable code reuse and allow us to maintain single spots for block definitions, providing abstraction over actions. We can call the function without worrying about its internal workings; we just expect it to deliver a pizza.

00:08:11.890 Similarly, we also use classes in Ruby. I could create a class like 'PizzaOrder' and initialize it with toppings, which would then allow me to create different instances and call the order method. Again, this serves the exact same purpose as before—managing the complexity of pizza-related actions while abstracting away internal mechanics.

00:08:44.840 Considering our previous example, what happens if I need to manage different variations like salads or hamburgers? I would want to replicate code by copying and pasting my pizza order script to include salad orders, changing 'pizza' to 'salad.' Subsequently, I might want to add a separate function specific to breadsticks, requiring me to copy-paste and slightly modify additional code. As our codebases grow larger, maintaining multiple variations introduces challenges, especially during refactoring. If I have fifty or a hundred items, the burden of manual code alterations becomes excessive, leading to technical debt.

00:09:55.220 This is why we need to approach auto-generation as a solution for similar functionality. At a small scale, it's easy to employ simple copy-paste practices, but the challenge arises when scale increases. As we approach more repetitions, the effort required to manage the code escalates dramatically due to increased complexity.

00:10:50.550 The frequent effort required for manual repetition grows significantly after the first batch of auto-generated code. Initially, it may take substantial effort to create templates and word banks, but as more features accumulate with little variation, the cost of adding features decreases significantly. Thus, you should consider auto-generating your code when you have numerous similar code segments to manage.

00:11:25.590 It's reasonable to evaluate every new feature you embark on during the development process. Ask yourself: is this feature similar to what I’ve auto-generated, or is it too distinct? The process of auto-generating code, once initiated, becomes challenging to revert. Handwritten alterations within generated code make maintaining coherence throughout subsequent auto-generation tedious.

00:12:11.900 You might currently reflect on the title, 'Make Ruby Write Your Code for You.' The rest of my talk will explore why Ruby is a fantastic choice for building code generators and facilitating this template and word bank process.

00:12:45.610 When auto-generating code, we require three core elements: a word bank, template files, and an injection mechanism for values from the word bank into the templates. I’ll focus on templates first. Ruby has a powerful built-in library called ERB, which stands for Embedded Ruby, that many of you are likely familiar with if you’ve done Rails development.

00:13:54.680 In ERB, you write a template file combining plain text and Ruby code enclosed within specific delimiters. You pass this ERB template and an array of data to Ruby so it can generate a complete text output. This process aligns seamlessly with our mad-libs analogy since we are taking a template file that resembles Ruby code and allowing Ruby to interpret it.

00:14:57.330 Now, let’s illustrate how the template can closely resemble Ruby code. We can create a version of a sample code segment as a template that incorporates dynamic parameters. The idea is often to inject values from a given data source into your representative template, so the original layout and function still present.

00:15:43.178 Many in the room likely have backgrounds in web development with Rails, so this might seem reminiscent of how views operate. There is a crucial shift when using these auto-generating templates. You want to avoid cleverness in your code and keep your templates as explicit as possible to ensure clarity and avoid ambiguity. The product generated by this process is less essential than the templates we utilize to create it.

00:16:21.750 Next, let’s discuss your word bank. The word bank contains key pieces of information needed for your templates, and you should reverse engineer your word bank from established templates. Starting with one feature written by hand, identify vital values and their roles. By assessing the essential elements, you can mould around the pieces necessary for an effective word bank.

00:17:34.110 How you format this word bank also matters. In my work with Magic Modules, I have found YAML to be a convenient format to outline this data. Ruby has libraries that can convert Ruby objects to and from YAML, facilitating the manual creation of Ruby objects through handcrafted YAML files.

00:18:23.410 This feature became revolutionary for me—the capability to define Ruby objects in a YAML format, and then easily revert them back into a usable Ruby object. However, beware of some caveats, such as initializers not being called when retrieving objects from YAML or encountering security risks depending on the YAML source. But on the whole, defining objects through YAML exemplifies flexibility.

00:19:15.860 To summarize, auto-generation closely mirrors the experience of creating Mad Libs. It leverages a template that formats as Ruby code while drawing from a word bank that supplies vital information. The end result is operational Ruby code that can be executed by a Ruby interpreter, akin to traditionally written human code.

00:20:20.940 Thank you very much for your attention! I’m available across various platforms, including Twitter and GitHub as aTrambleRaptor. You'll find my slides published and the Magic Modules code generator we discussed, which I’ve been working on at GitHub.com/GoogleCloudPlatform/magic-modules. Good luck! I hope these ideas inspire you to identify practical applications in your code development.