00:00:15.350
Hi everyone, my name is Alex Stephen. I go by aTrambleRaptor throughout the interwebs. Today, I'm going to talk to you about how you can make Ruby write your code for you. Let me give you some quick background about myself. I work at Google, specifically at Google Cloud. For those of you who don't know much about Google Cloud, it does a lot—you can create virtual machines, container clusters, load balancers, databases, and more.
00:00:34.620
My team, in particular, focuses on open-source integrations. We look at open-source tools like Puppet, Chef, Ansible, and Terraform, and we try to add features to those tools to allow them to create virtual machines and SQL databases. When I first joined the team, we added support for virtual machines in both Puppet and Chef.
00:00:47.280
Eventually, we extended our work to include load balancers. By the time we got to the third or fourth feature, we noticed that if you squinted at the code for these different features, they looked almost identical. Yes, they were a bit different as they served different purposes, but they interacted with very similar APIs and operated in very similar ways. We questioned why we were writing so much similar code repeatedly, especially when we expected to create 50 or 60 different features.
00:01:17.400
So, we decided to build a code generator called Magic Modules. It's available on GitHub as an open-source project (github.com/GoogleCloudPlatform/magic-modules). Magic Modules is a Ruby-based code generator that takes information about Google Cloud and generates actual Ruby code for the tools we develop. As we progressed, we extended the generator to output Python code for Ansible and Go code for Terraform.
00:01:54.630
Code generation, or writing code that in turn writes other code, is something I've been doing for a long time. Today, I will share the concepts we utilized to create Magic Modules and explain how approachable this whole code generation process is. I will also discuss when you should and should not use this process in your own code development.
00:02:35.819
I've used the term 'auto-generation' frequently, which might sound intimidating. The idea that I'm writing Ruby that outputs Python and Go may seem wacky. Auto-generation essentially means that computers are writing your code for you. However, this notion can feel rather black-boxed and might make you feel powerless, as if what you're doing doesn’t truly matter.
00:03:03.060
Let me reframe the idea of auto-generation. When you are auto-generating code, you are instructing the computer on precisely what you want it to write and how to write it. The computer will then execute those instructions, creating the output exactly where you want it. Hopefully, this perspective helps you see it as more approachable—like another powerful tool in your Ruby toolbox.
00:03:41.189
In tech, we love the meme that something is requested to happen everywhere, right? I think of auto-generation as writing something once, then having the computer replicate it in multiple places with various modifications. But now you might be wondering how you actually auto-generate code. That sounds complicated. Has anyone ever done a Mad Lib before? Okay, about half the room has been to middle school. Mad Libs consist of templates containing blanks that you fill in with words from a word bank, resulting in various sentences. In essence, you can create numerous sentences from one template.
00:04:51.840
Now let’s consider applying this to code. For this talk, I’ll use a food-related analogy involving different food APIs that allow adding toppings and ordering food. In my word bank, I have things like a pizza class that takes options like pepperoni and sausage. The left side shows code that resembles Ruby, but with several keywords missing. Though variations in inputs could make the code valid if you had relevant objects instantiated, it shows the concept of using templates to create dynamic code.
00:05:59.670
Instead of filling a text template, we are filling a code template with parameters that, when executed, resemble valid Ruby code. If there's one takeaway from this talk, it’s that auto-generation is similar to a Mad Lib, taking a Ruby-like file and injecting keywords to culminate in valid, runnable Ruby code through the Ruby interpreter. However, it’s also essential to understand what auto-generation isn't.
00:06:36.150
In the context of Magic Modules and my experience, auto-generation isn’t about blockchain, Bitcoin, or machine learning techniques. There are no deep learning models or abstract syntax trees involved here. The auto-generation I talk about focuses purely on the mechanics of generating code without additional complexities found in other paradigms.
00:07:03.060
Auto-generation is a simple trick where a template helps to produce code that looks adequately similar to Ruby code. Now, when should we consider this approach? Before diving in, it's beneficial to discuss the abstractions we work with daily as Ruby programmers.
00:07:12.080
I've got an example of Ruby code where a block is called twice to order a pepperoni pineapple pizza. Most of us might consider it a good practice to avoid code duplication through function reuse, letting us replace repetitive calls with a single parameterized function, such as 'order pizza.' Such functions enable code reuse and allow us to maintain single spots for block definitions, providing abstraction over actions. We can call the function without worrying about its internal workings; we just expect it to deliver a pizza.
00:08:11.890
Similarly, we also use classes in Ruby. I could create a class like 'PizzaOrder' and initialize it with toppings, which would then allow me to create different instances and call the order method. Again, this serves the exact same purpose as before—managing the complexity of pizza-related actions while abstracting away internal mechanics.
00:08:44.840
Considering our previous example, what happens if I need to manage different variations like salads or hamburgers? I would want to replicate code by copying and pasting my pizza order script to include salad orders, changing 'pizza' to 'salad.' Subsequently, I might want to add a separate function specific to breadsticks, requiring me to copy-paste and slightly modify additional code. As our codebases grow larger, maintaining multiple variations introduces challenges, especially during refactoring. If I have fifty or a hundred items, the burden of manual code alterations becomes excessive, leading to technical debt.
00:09:55.220
This is why we need to approach auto-generation as a solution for similar functionality. At a small scale, it's easy to employ simple copy-paste practices, but the challenge arises when scale increases. As we approach more repetitions, the effort required to manage the code escalates dramatically due to increased complexity.
00:10:50.550
The frequent effort required for manual repetition grows significantly after the first batch of auto-generated code. Initially, it may take substantial effort to create templates and word banks, but as more features accumulate with little variation, the cost of adding features decreases significantly. Thus, you should consider auto-generating your code when you have numerous similar code segments to manage.
00:11:25.590
It's reasonable to evaluate every new feature you embark on during the development process. Ask yourself: is this feature similar to what I’ve auto-generated, or is it too distinct? The process of auto-generating code, once initiated, becomes challenging to revert. Handwritten alterations within generated code make maintaining coherence throughout subsequent auto-generation tedious.
00:12:11.900
You might currently reflect on the title, 'Make Ruby Write Your Code for You.' The rest of my talk will explore why Ruby is a fantastic choice for building code generators and facilitating this template and word bank process.
00:12:45.610
When auto-generating code, we require three core elements: a word bank, template files, and an injection mechanism for values from the word bank into the templates. I’ll focus on templates first. Ruby has a powerful built-in library called ERB, which stands for Embedded Ruby, that many of you are likely familiar with if you’ve done Rails development.
00:13:54.680
In ERB, you write a template file combining plain text and Ruby code enclosed within specific delimiters. You pass this ERB template and an array of data to Ruby so it can generate a complete text output. This process aligns seamlessly with our mad-libs analogy since we are taking a template file that resembles Ruby code and allowing Ruby to interpret it.
00:14:57.330
Now, let’s illustrate how the template can closely resemble Ruby code. We can create a version of a sample code segment as a template that incorporates dynamic parameters. The idea is often to inject values from a given data source into your representative template, so the original layout and function still present.
00:15:43.178
Many in the room likely have backgrounds in web development with Rails, so this might seem reminiscent of how views operate. There is a crucial shift when using these auto-generating templates. You want to avoid cleverness in your code and keep your templates as explicit as possible to ensure clarity and avoid ambiguity. The product generated by this process is less essential than the templates we utilize to create it.
00:16:21.750
Next, let’s discuss your word bank. The word bank contains key pieces of information needed for your templates, and you should reverse engineer your word bank from established templates. Starting with one feature written by hand, identify vital values and their roles. By assessing the essential elements, you can mould around the pieces necessary for an effective word bank.
00:17:34.110
How you format this word bank also matters. In my work with Magic Modules, I have found YAML to be a convenient format to outline this data. Ruby has libraries that can convert Ruby objects to and from YAML, facilitating the manual creation of Ruby objects through handcrafted YAML files.
00:18:23.410
This feature became revolutionary for me—the capability to define Ruby objects in a YAML format, and then easily revert them back into a usable Ruby object. However, beware of some caveats, such as initializers not being called when retrieving objects from YAML or encountering security risks depending on the YAML source. But on the whole, defining objects through YAML exemplifies flexibility.
00:19:15.860
To summarize, auto-generation closely mirrors the experience of creating Mad Libs. It leverages a template that formats as Ruby code while drawing from a word bank that supplies vital information. The end result is operational Ruby code that can be executed by a Ruby interpreter, akin to traditionally written human code.
00:20:20.940
Thank you very much for your attention! I’m available across various platforms, including Twitter and GitHub as aTrambleRaptor. You'll find my slides published and the Magic Modules code generator we discussed, which I’ve been working on at GitHub.com/GoogleCloudPlatform/magic-modules. Good luck! I hope these ideas inspire you to identify practical applications in your code development.
