Search
Talks
Pre-evaluation in Ruby
Edit
#ruby-optimization
#jit-compiler
#abstract-syntax-tree-ast
#compiler-techniques
#performance
#ruby-vm
Summarized using AI

Pre-evaluation in Ruby

Kevin Newton • April 18, 2019 • Fukuoka, Japan • Talk

In the talk "Pre-evaluation in Ruby" presented by Kevin Newton at RubyKaigi 2019, the speaker explores an optimization technique in Ruby called pre-evaluation. This approach aims to enhance Ruby's performance, acknowledging the language’s historical difficulties with optimizations due to its flexibility and productivity features that often come at the expense of performance. The talk covers several important aspects related to how pre-evaluation works and its implications for Ruby programming.

Key points discussed include:
- Programming Compiler Steps: The presentation begins with an overview of the process that Ruby compilers follow, including stages like lexical analysis, semantic analysis, and instruction generation for the virtual machine.
- Lexical and Semantic Analysis: Deisz explains lexical analysis as the breakdown of sentences into tokens, followed by creating an abstract syntax tree (AST) through semantic analysis to understand the relationships between those tokens.
- Performance Limitations: He addresses Ruby's limitations in executing optimizations at runtime and the inherent constraints of Ruby's virtual machine. Notably, he states that certain decisions need to be made at compile time rather than runtime due to these limitations.
- User Participation in Optimizations: Users can opt into performance improvements by limiting their usage of Ruby features. This concept allows for better compiler optimizations, although complex Ruby expressions may pose challenges.
- Introduction of pre_val: The introduction of the pre_val gem enables developers to analyze their Ruby code and identify potential compile-time optimizations. The gem utilizes Ripper to generate the AST and thus automates the source code adjustment for better optimization.
- Demonstration of Sinatra App: Deisz provides a demonstration of a Sinatra application leveraging the pre_val gem to showcase how real-time optimization can be achieved in Ruby code.
- Enhancements through Collaboration: The talk emphasizes the importance of community feedback for building efficient optimizations and maintaining usability without sacrificing performance. Deisz encourages ongoing collaboration within the Ruby community.

In conclusion, Deisz reinforces that while Ruby's flexibility can complicate optimizations, tools like pre_val aim to bridge this gap, making Ruby code clearer and more performant. These efforts ultimately contribute to the ongoing evolution and robustness of the Ruby programming language.

Pre-evaluation in Ruby
Kevin Newton • April 18, 2019 • Fukuoka, Japan • Talk

Ruby is historically difficult to optimize due to features that improve flexibility and productivity at the cost of performance. Techniques like Ruby's new JIT compiler and deoptimization code help, but still are limited by techniques like monkey-patching and binding inspection.

Pre-evaluation is another optimization technique that works based on user-defined contracts and assumptions. Users can opt in to optimizations by limiting their use of Ruby's features and thereby allowing further compiler work.

In this talk we'll look at how pre-evaluation works, and what benefits it enables.

RubyKaigi 2019 https://rubykaigi.org/2019/presentations/kddeisz.html#apr20

RubyKaigi 2019

00:00:00 Hi, thanks for coming. This talk is about pre-evaluation in Ruby.
00:00:05 While it is not largely recognized now, it will be after you see this talk. My name is Kevin Deisz.
00:00:12 You can follow me on the Internet; I work at a company called Culture HQ.
00:00:23 We are based out of Boston, Massachusetts, and our mission is to improve workplace culture.
00:00:30 If you're interested in that kind of work, you should check us out.
00:00:37 Additionally, I'm from the US, and if anyone is interested in trading Pokémon in Pokémon Go, come see me after.
00:00:46 I'm still looking for region-exclusives from here.
00:01:03 So anyway, let’s start with the review of our process. In programming, particularly in Ruby, we write compilers that follow standard steps.
00:01:21 These steps include lexical analysis to identify tokens.
00:01:27 Then we perform semantic analysis to generate an abstract syntax tree (AST).
00:01:38 Next, we generate instructions for the virtual machine, run optimization passes, and finally execute the instructions.
00:01:51 To understand these processes, we can start with lexical analysis.
00:01:56 Lexical analysis involves breaking a sentence into individual tokens and interpreting those tokens.
00:02:02 Typically, we start with nouns, followed by verbs, adjectives, conjunctions, adverbs, and so on until we reach a period.
00:02:11 Once we have our list of tokens, we will then move on to semantic analysis.
00:02:19 This process applies grammatical rules to create trees that represent the relationships between those tokens.
00:02:31 For example, we might define a subject phrase as a combination of a noun and a verb phrase.
00:02:50 Next, we can look at grammar rules: we will create a rule for sentences, which includes moving back to the tree.
00:03:02 Finally, we arrive at our completed abstract syntax tree for our unique English grammar.
00:03:10 While these rules are limited, this grammar can still understand the provided sentence given a good lexer.
00:03:29 This idea is similar to how libraries like Rack handle tokenizing, which leads to generating instruction sequences.
00:03:41 We need to traverse down the tree to understand what our operations will accomplish; each segment corresponds to an action or attribute we want to process.
00:04:07 Going down to the leaves of this tree structure, we will be able to apply expressions to achieve our goals.
00:04:26 After generating instructions from the AST, we executed these statements at runtime.
00:04:55 During execution, we can identify certain constants and make decisions accordingly.
00:05:14 In the context of Ruby, we know Matz is nice, which makes it a constant value.
00:05:30 This pre-evaluation allows for optimizations such as constant folding.”
00:05:58 However, from a performance standpoint, we have to acknowledge that Ruby often has limitations with optimizations.
00:06:13 Let’s consider what optimizations Ruby can manage versus what can’t be done.
00:06:32 The execution of instruction sequences happens at runtime, regardless of the circumstances.
00:06:56 Comprehensively, we need to assess how much we can do at compile time versus runtime.
00:07:13 Ruby’s virtual machine has certain inherent constraints in terms of the optimizations it can execute.”
00:07:44 A significant benefactor in this optimization process involves users opting into these performance improvements.
00:08:02 However, complex expressions in Ruby can complicate whether assertions will be true at runtime.
00:08:12 Some things, like constant assignments, can change during execution, which can break our assumptions.
00:08:33 Nonetheless, opting into optimizations is crucial to leverage compile-time aspects effectively.
00:08:51 Using pre_val, my gem, we can analyze our code to determine which optimization can occur at compile time.
00:09:16 pre_val uses Ripper, which generates the AST, combining source structure evaluations.
00:09:41 This approach allows us to automate the process of adjusting source code for better optimization.
00:10:04 Moreover, it supports various programming methods, such as the visitor pattern.
00:10:31 We can also enhance this through creating nodes to traverse our abstract syntax tree and apply optimizations where applicable.
00:11:04 For instance, if we encounter trivial expressions, we can opt not to evaluate them.
00:11:28 The goal is to execute clearer code that is more intuitive in Ruby while still harnessing its power.
00:11:52 The beauty of pre_val rests in how it enables the language to be as user-friendly as desired while preserving performance.
00:12:21 For a demonstration, I organized a Sinatra app within the gem.
00:12:50 This app showcases how pre_val can operate to execute its processes in real-time.
00:13:08 With this demo, one can define methods seamlessly while maintaining optimized evaluation under the hood.
00:13:48 Where possible, we can minimize the need for additional syntax, making Ruby more enjoyable.
00:14:00 Moving forward, pre_val aims to streamline the coding process without impeding the robustness of Ruby.
00:14:37 It serves as an extremely effective tool that can smooth the complexities of Ruby's optimizations.
00:15:14 Always, my goal is to focus on enhancing the positive characteristics of Ruby.
00:15:53 Before wrapping up, I want to point out that Ruby isn’t as static as it sometimes appears, since it allows incredible flexibility.
00:16:15 There's quite a bit of technical depth to explore that can be fun as we continue working toward robustness.
00:16:32 Questions? I would love to take any inquiries regarding pre_val or anything else.
00:17:02 Knowing that I am asking about an extreme case, have you added enough optimizations to break Rails yet?
00:17:49 I have added enough optimizations to break my own app. I did want to be able to say that I was running this in production.
00:18:08 It's important to be cautious with certain implementations, as they can drastically influence results.
00:18:45 Rails can often introduce complexities, especially around method overwrites.
00:19:23 These optimizations are essential and will require thorough documentation around good and bad practices.
00:20:08 Overall, pre_val aims for greater usability in production systems without risking performance.
00:20:51 Your concerns about certain aspects of method behavior are valid, specific implementations give rise to interesting discussions.
00:21:31 Collaboration is always key, and community feedback is vital for optimizing future updates.
00:21:45 I’m hoping to build on pre_val in the best productive way possible.
00:22:34 Lastly, remember that community engagement constitutes the heart of Ruby development.
00:23:15 Collaboration with the Ruby community can lead to incredible advancements and expansions.
00:23:39 Thank you very much for your time, and feel free to reach out to me for further conversations.
00:24:20 I'm on the Internet as kddeisz, and I'm happy to discuss any questions you have.
Kevin Newton
@kddnewton
Explore all talks recorded at RubyKaigi 2019
+73