00:00:22.279
Okay, I think we're going to get started. Thank you for coming to my talk titled "Pre-evaluation in Ruby." My name is Kevin Deisz, and you can find me on Twitter at @KatyDeisz. Please follow me! I often share my thoughts on new Ruby features, even if they are hot takes. I work at a small company called Culture HQ in Boston, where we focus on improving workplace culture. If you're interested in enhancing the culture at your workplace, feel free to talk to me afterward.
00:00:52.710
First of all, I want to start off with a warning: this is a very technical talk. However, this is not intended to scare you. My goal here is to relay this information without completely overwhelming or alienating anyone. If you are an absolute beginner, I hope you will still find value in this talk, although I understand it might be somewhat challenging. If at the end of this talk, you feel lost and gain no value, that would mean I have officially failed. So, if you have questions, please feel free to talk to me afterward.
00:01:31.560
Let's begin by discussing compilers, specifically Ruby's compiler, and the various steps it goes through. Then we'll explore how we can extend that compilation process and the value it can bring. When we talk about compilers, we typically consider several steps: lexical analysis, semantic analysis, instruction generation for the virtual machine, various optimization passes, and that's what we'll focus on.
00:02:06.840
To illustrate lexical analysis, let's look at an example sentence: "Matz is nice, so we are nice." Here, lexical analysis involves breaking this sentence into individual tokens that we can then analyze with grammar. We must identify the parts of speech, which is similar to what other programming languages accomplish during their compilation process.
00:02:33.180
For instance, we recognize that 'Matz' is a noun, 'is' is a linking verb, 'nice' is an adjective, and so forth. By segmenting the sentence into these tokens, we create a structure akin to a syntax tree, which we can call an abstract syntax tree (AST). We've managed to derive this abstract representation from plain text.
00:02:57.780
Continuing from the concept of an abstract syntax tree, we can see how this structure plays out in practical examples within Ruby code. For instance, consider an expression parser used in Ruby. This parser generates the necessary structure to facilitate semantic analysis, which further assigns meaning to the derived tokens.
00:03:30.450
To create our semantic tree, we can define rules, such as identifying a verb phrase. Once identified, we can extend this grammar definition and recognize a full subject phrase. In practical terms, we can modify our grammar to encompass more complex sentence structures. But, I must admit, my understanding of grammar terms is limited.
00:04:03.090
Nevertheless, as we build this tree, we integrate new elements, such as the conjunction 'so,' known as a subordinating conjunction. To finalize our sentence representation, we need to incorporate a terminal punctuation mark, like a period. With that, we have successfully built a syntax tree that models our initial sentence structure.
00:04:44.890
Now, moving forward, after constructing our abstract syntax tree (AST), we proceed to generate instructions for the virtual machine (VM). Each node in our tree corresponds to actions performed within the VM, manipulating the machine's state. As we traverse the AST, we generate these instructions. We will push attributes to the stack and pop them based on our grammar rules.
00:05:51.300
In our example of the verb phrase, we push an attribute representing the second part of our action, and subsequently, we address our subject phrase by linking the actions correctly. We essentially determine that 'Matz is nice' translates to the appropriate instructions for our virtual machine. We also handle conditional aspects introduced through subordinating conjunctions, implementing these as if statements during instruction generation.
00:06:10.050
In essence, we are able to walk the tree step-by-step to generate the machine instructions we need. After traversing our tree and generating the machine instructions, we can apply optimizations. For instance, while our grammar permits various forms, we can simplify expressions where constants are involved, eliminating unnecessary checks and streamlining our code execution.
00:06:40.680
One common optimization in Ruby can be demonstrated through a simple expression like '5 + 3.' At face value, one might assume this always evaluates to 8. However, if we delve into Ruby's compilation approach, we discover it should revert this evaluation if someone unintentionally overrides the addition method. This touches on a critical aspect of Ruby's flexibility and the potential for unintentional interference with method handling.
00:07:16.580
The key takeaway here is the need for consideration around monkey patching, especially concerning arithmetic methods on core Ruby classes. The broader implication is that our community is inadvertently compromising compiler optimizations to account for these edge cases when flexibility doesn't always yield better performance.
00:08:25.060
Faced with this dilemma, I created a gem named 'pre_val.' This gem assumes developers are less likely to engage in unfavorable practices like monkey patching core methods unnecessarily, thus maintaining the promise of cleaner optimizations within our code. Through this approach, we parse Ruby code, derive its semantic structure, and conduct our optimizations before feeding it back into Ruby.
00:09:20.160
To help with this, Ruby provides the Ripper library, which allows us to generate the abstract syntax tree. We then build our own custom node structures based on these results. Following this, we include a method for converting our modified tree back into valid Ruby source code. Much of this process isn’t fundamentally new; techniques like this are already utilized across various other projects and tools.
00:10:34.240
The essence of using this transformational approach is to enhance the way Ruby understands and processes expressions without constraining developers to a specific coding style or structure. However, it bears mentioning that while we can reformat this compiled representation back to source, doing so could limit the gem's intended purpose of enabling diverse expressions.
00:11:24.760
This leads us to the implementation details where our pre_val gem takes a string input, processes it, and applies semantic analysis to generate the abstract syntax tree. This tree is then manipulated through various optimizations before returning to Ruby. Developers can utilize the public API, utilizing Preval's processing capabilities to streamline their code effectively.
00:13:26.050
The heart of the gem lies in the visitor pattern applied to optimize the AST. It grants developers the flexibility to derive properties from their Ruby code and optimize them without needing to boil down to minor differences in their code styles. There's a public-facing portion of the API that handles this processing, allowing users to reap the benefits of optimizations.
00:14:27.020
Finally, public API exposes functionality that enables developers to define their nodes and customize their optimization strategies. This means extending the system to accommodate anything beyond the provided defaults, thus resulting in individual capabilities bolstered by the foundation of the gem.
00:15:21.370
If you consider integrating Preval into your existing applications, the transition can be seamless. The gem enables improvements without incurring runtime performance penalties. With that being said, I'd like to offer a demonstration of its capabilities shortly.
00:16:10.260
As we navigate the inner workings of our tool, you will discover that using Preval involves minimal configuration. The beauty of this gem is evident as it process underlying Ruby structures, optimizes them thoroughly, and enhances runtime performance. Never does this requirement dictate how one should express their Ruby code.
00:17:36.930
In conclusion, Preval allows you to explore Ruby's freedom of expression while efficiently optimizing your code behind the scenes. You can expect substantial improvements to code processing, potential for less code review friction, and ultimately a better developer experience.
00:19:51.490
I had like three cups of coffee so I went really fast. If anyone has any questions, I have plenty of time.
00:20:08.000
In response to a question about the gem's capability to validate its transformations, I must mention that while I was halfway through building a validation feature, I realized integer classes have been monkey patched by Rails. Therefore, I opted to allow the gem to run without validations for now.
00:20:53.000
As for queries regarding formatting the compiled code back to the source, it is indeed possible but not without its drawbacks. The essence of the gem is rooted in allowing multiple expressions in Ruby and does not require forcing a single way of expression.
00:21:04.800
Regarding user customization, yes, you can undeniably add your extensions according to the documentation. Just define a class which has a special method that corresponds to the node type. So if you'd like to expand your functionality further, you can readily do that.
00:21:48.930
If there are no more questions, I will be available afterward. Thank you so much for attending this session!