Simple Ruby DSL Techniques: Big Project Impact!

Talks

Aman King

1 talk

#ruby

#domain-specific-language-dsl

#metaprogramming

#activerecord

#software-development

#productivity

#domain-driven-design

Simple Ruby DSL Techniques: Big Project Impact!

by Aman King

In the video titled "Simple Ruby DSL Techniques: Big Project Impact!" presented by Aman King at the Garden City Ruby 2014, the speaker discusses the significance of Domain-Specific Languages (DSLs) within Ruby projects, emphasizing their contribution to improving team productivity and delivering project success.

Key points covered in the presentation include:
- Understanding DSLs: DSLs are specialized programming languages tailored to specific problems or industries, aiming to enhance communication and productivity by utilizing familiar domain terminology.
- Importance of Custom DSLs: Focusing on writing DSLs pertinent to specific business domains can help address unique project requirements that generalized frameworks like Rails may overlook.
- Fluent Interfaces and Other Techniques: King outlines multiple methods for implementing DSLs, starting with fluent interfaces that utilize method chaining, and progressing through single line declarations to more complex block declarations that require deeper metaprogramming techniques.
- Practical Examples: The speaker provides tangible examples from real-world projects. For instance, he describes using a DSL in a ticketing system to report incidents with defined severities and priorities through expressive and clear API lending itself to better communication between developers and stakeholders.
- Processing User Input: Another example involves sanitizing user-generated content in a forum to filter out profanity while maintaining original entries, demonstrating how DSLs can encapsulate and clarify logic.
- Handling Complex Relationships: King mentions the creation of methods to manage social relationships in applications, illustrating how they can articulate specific functionalities and interactions effectively.
- Cautions with Metaprogramming: The presentation addresses potential pitfalls associated with metaprogramming, which, while powerful, can complicate code readability if used excessively.

The conclusion encourages teams to adopt custom DSLs in their Ruby projects to harness their benefits, such as increased coding clarity, maintainability, and overall productivity. By framing the programming environment in terms familiar to domain experts, teams can improve efficiency and foster deeper technical engagement among team members.

00:00:25.240 Okay, let's focus on simple Ruby DSL techniques. My apologies if this talk bores you because it will be a pure Ruby discussion. I won't be referencing C, Java, or Clojure, nor will I bring up architectural concepts. That said, I consider myself a polyglot programmer, but I've spent a lot of time working on Ruby projects. While Ruby may not be my favorite language—ask me offline about that—I do appreciate it. It has helped me think differently.

00:01:07.960 So, when my talk got selected, the organizers, who are doing a wonderful job, reached out to me with a set of guidelines for speakers. One of the guidelines suggested starting with a joke or, at the very least, ending with one. So, here's my obligatory humor to kick things off.

00:01:26.880 However, a disclaimer: jokes can offend people, especially the funniest ones, which often target someone. This joke might offend certain communities, but I assure you, no offense is intended. You might sense a hint of satire in it. Also, this isn't an original joke; you may have heard it before, but please make an effort to laugh at the end— it will make me feel good. So, here’s the joke: What do beggars and developers have in common? Don’t answer—I know you already know. It’s a very poor joke that’s been around for a long time.

00:01:54.119 So, when two beggars meet at a railway station, or when two developers meet at a tech conference, they often introduce themselves with the question, 'What platform do you work on?' Thanks for that laughter; I appreciate it! There’s a bit of satire in that too, and it’s a topic for philosophical discussion later. The difference is, developers typically continue the conversation with another question: 'What domain do you work in?' This presents an opportunity to gather some examples of domains. Now, just shout out—what domains are you currently working in? It could be finance, retail, social networking, education, or health.

00:02:30.239 Now, it’s quite interesting that nobody mentioned domains like logistics or warehousing. Maybe it’s because we often think of these as implementation concerns. While they may not seem directly related to the business domains we work in, they are still significant and deserve consideration. I like to think of them as technical domains. But, what exactly do we mean by 'domain'? There are certainly various answers out there. Here's my perspective: a domain is a specialized field with a shared context and vocabulary among its specialists.

00:03:03.480 This talk will actually focus on that last point: the common vocabulary among specialists. Let me clarify with some examples. How many of you remember the grandmother test? It’s when your grandmother can understand the sentences we use. I’m curious, how many of you understand these terms? Go ahead and raise your hands; don't be shy! It seems a few hands went up.

00:03:44.080 We clearly know what these terms mean, and Ruby understands them too. We don't need to explain concepts like private methods or delegation, as Ruby supports these terms natively. This is essentially a DSL (Domain-Specific Language) in core Ruby. Many of us have used these methods before—actually, they are Ruby methods. However, when we specifically talk about DSLs, these methods might not be apparent as clear examples. More commonly, we refer to frameworks or libraries such as Active Record or Rails controllers, which use DSLs.

00:04:02.880 One of my favorite libraries is Sunspot, as it simplifies working with Solr. I remember working on a Java project using Lucene—it was a challenging experience, and I wouldn't want to return to it. All these examples represent Ruby DSLs relevant to the technical domain.

00:04:47.920 So, where are the DSLs for the business domains we discussed earlier, like health, education, retail, finance, and logistics? The framework writers typically don't create these DSLs. Why? Because they can't predict what we, the consumers of those frameworks, actually need in our specific business contexts. They might assume that every domain will require authentication or caching, and that data persistence is essential, but they can’t guess the specific business problems we face. Only our clients know those details.

00:05:16.960 Thus, it falls to you to write those DSLs for your business domain. Why write them, you may ask? A simple answer is: why not? After years of fruitful experiences with Rails and Active Record, we love how clean and easy they make implementations. Since a lot of our productivity gains stem from these DSL examples, it follows that we should strive to harness similar gains in our own business domains.

00:05:32.840 Now, let’s transition to the second part of my talk, which focuses more on the 'stuff.' I believe in a balance of fluff and substance; some prefer no fluff at all, just the substance. However, if we want to write a DSL, we first need to understand what DSLs are about. This understanding is akin to recognizing code smells to facilitate refactoring. Just as you can’t perform effective refactoring until you identify the code issues and relate them to refactoring techniques, similarly, you need to recognize DSLs.

00:06:22.640 This is my attempt at classifying DSLs to help you recognize them better. For instance, consider this kind of DSL, which I would classify as a fluent interface. This term is not original to me; Martin Fowler wrote about it many years ago and has recently published a book on DSLs where he relates fluent interfaces to DSL implementations.

00:06:55.679 Let’s look at another kind of DSL that I would call a single line declaration. There’s nothing unintuitive about it; it’s simply a single line of Ruby code that serves as a declaration. I’d like to emphasize that many DSLs are declarative in nature rather than imperative. If you’re unfamiliar with these concepts, I recommend reading up on them. SQL is a great example of a declarative language; you use statements like 'SELECT * FROM this_table' without telling the database how to fetch the records.

00:07:41.560 Similarly, when using Rails, a command like 'before_update' is a declaration. It doesn’t expose the implementation details about how the records are updated. Moving on, you can also declare things using blocks, breaking them into multiple lines by passing them as blocks, which gives rise to three basic categories into which most DSLs can be classified. Now, I want to discuss how you can implement these DSLs yourself.

00:08:30.200 This brings us to the next stage: recognizing the secret sauce, or ingredients, that will help you create such DSLs. For a fluent interface, you’ll need method chaining, and possibly the Builder pattern or a variation of it. I’ll go into more detail about this later. For single line declarations, you typically need class methods along with some basic metaprogramming. Most of the metaprogramming involved will be manageable.

00:09:14.920 Moving to block declarations, they can also use class methods or instance methods, combined with more intricate metaprogramming that, while potentially complex, is still approachable. I’ll now share some examples—specifically, code that emulates the kind of concepts we’ve implemented in real client projects. I’ve uploaded some of these examples to GitHub, so you'll be able to review them later.

00:09:59.600 Let’s start with the implementation of a fluent interface. The domain we’re looking at is a production support ticketing system. Perhaps you’re dealing with incidents, and let’s say we want to create a simple report with defined severity—such as critical or major—and priority levels—like high or medium—from a specified date range. The structure might be something a business user can comprehend, although the symbols and punctuations might cause confusion.

00:10:55.360 The essential point is that, from an implementation perspective, you might guess that there’s nothing overly complicated here. You might be asking, 'What makes it a DSL?' The key difference lies in the API. What sets this apart is that the API embodies the domain, rather than being a generic persistence-related API.

00:11:13.200 Let’s take a moment to look behind the scenes. Hopefully, this text is readable for everyone at the back? Can everyone see? Well, let's assume we have a class called Incident that wraps our relevant data. Here’s an interesting piece: the class feature may look unimpressive as it merely delegates to another class-level method called 'new.' We are familiar that this method invokes the instance method 'initialize.' Essentially, we start by capturing a list of the systems in the class with an instance variable.

00:11:50.200 The methods will be similar; they accept arguments and store them in instance variables. What’s critical is that they return the object itself to facilitate method chaining. So, you could do something like 'incident.report.do.for,' which ultimately returns an incident report object. You can continue chaining with methods like 'severity' and 'priority', and so forth—it's fairly straightforward. Are you all following along?

00:12:32.640 So far, nothing complicated. This design resembles a simplified version of the Builder pattern. But once you've collected all the options, you’ll want to utilize them effectively. You typically have a method at the end that performs the heavy lifting. For instance, in Active Record, a method takes all your 'where' arguments and processes them to make a database query.

00:13:03.360 This isn't your final method; you can say something like, 'class_name.where.limit.order' and then ultimately invoke the query. Just as in Active Record, you can encapsulate this logic in a simple DSL to make persistent interactions more meaningful.

00:13:51.760 Let's consider using default values carefully. For instance, when a user doesn’t call a method, treating 'nil' as a viable default ensures we don’t assume that each builder method will be applied to all attributes. This carefully crafted approach allows us to filter and utilize our collected data efficiently.

00:14:31.399 So, let's address some examples based on the simple implementation I described. These examples have been tweaked from codebases, showing how worker jobs manage errors in the background in production environments. This is a combination of Active Record and the Builder pattern DSL.

00:15:13.920 In another project, we worked with Neo4j, a graph database, where we used a DSL to generate Cypher queries sent over a RESTful API. The key here is writing these DSLs to simplify the complexity of interaction with various systems.

00:15:38.080 Now, switching gears to single line declarations—with an example from a community forum. Posts have attributes that may include titles and descriptions. Users can insert profanity into these fields. Hence, we aim to sanitize any profane terms, contained within those class declarations. The user might create a post with inappropriate language, but you should return the sanitized version when accessing the value. However, it's crucial to keep the original text visible for specific scenarios.

00:16:31.120 In this case, using a `Struct` can provide a method title that processes the text differently than a standard `Struct` implementation. This introduces a more complex metaprogramming technique. A separate module can be created to integrate with the class to facilitate defining multiple methods based on the sanitized and unsanitized conditions. We generate method names related to attributes that we specified, and we apply the necessary transformations.

00:17:17.560 By using 'alias_method_chain' in Ruby, we can determine how these methods will be synchronized with code execution. As a result, we define our title and its sanitized counterpart within the one created object.

00:18:01.360 Next, let’s move into using our DSLs more effectively—say you want filtering profanities in certain action parameters during the creation of posts. Your code implementation gets even more nuanced by checking strict user specifications before allowing actions like creating or destroying a post.

00:18:48.600 Exploring further, we can dive into active involvement with an activity feed—wondering how a comment generates a content feed item following its creation. You might come across SQS-based AWS implementations that efficiently handle background job operations; you can simply specify image workers that read from a designated queue.

00:19:32.800 The declaration can get quite complex. For instance, in the realm of social networking, a social graph denotes relationships between followers and celebrities. We can declare methods that explicitly handle these varying relationship types. When fans participate as followers of celebrities, specific functionalities must articulate this relationship correctly.

00:20:12.320 Implementing this involves utilizing a class method, potentially in conjunction with blocks, to encapsulate more complex metaprogramming initiatives. We will leverage specifications that accumulate all relationship definitions, thereby applying relationships while managing code behaviors accurately.

00:20:57.839 As we delve deeper into the structure of complex DSLs, we continue to adapt to the specific demands of the business context. Here’s an example where class methods delegate tasks suitably. You can create intricate methods while seamlessly interacting with the inherited class structure.

00:21:48.000 Quickly recapping: we can pinpoint fluent interfaces as the most basic and simple DSLs to implement. Moving onto single line declarations, you will commonly incorporate modules, mixins, and basic metaprogramming techniques to establish clarity in user applications. The complexity may increase with block declarations due to potential advanced metaprogramming practices.

00:22:24.560 When using the Builder pattern, it’s important to exercise restraint. Not every class construction process needs one. It’s essential to keep the class responsibilities distinct from the Builder. Always approach default values with meticulous care; there’s no guarantee that every method will be invoked for each attribute.

00:23:18.720 At this point, it’s worth noting the potential pitfalls associated with metaprogramming. While it can streamline processes, excessive use may complicate code readability. Furthermore, as you adapt your DSL to include variations, remember that clarity of understanding is critical, ensuring that others can grasp complex DSLs as easily as you do.

00:24:05.960 DSL usage, especially when tailored to specific business domains, can significantly improve productivity. By conducting a quick analysis of developer and QA interactions, it's clear that tasks can be completed in a fraction of the time, thanks to ready-made reusable code structures. You’ll find that maintenance improves with clarity, preventing over-reliance on extensive inheritance and promoting consistent coding.

00:25:00.560 Considering productivity and team development, a custom DSL can significantly raise your Ruby skill levels as team members engage more deeply with metaprogramming techniques and complex design patterns. There’s an overarching benefit in comprehensively modeling business concepts within these DSLs, leading to enhanced understanding for everyone involved.

00:25:51.480 Lastly, I recommend giving the implementation of custom DSLs for business domains a try. You might initially experience some resistance, but even if members don't fully embrace it in the beginning, the practice can yield significant long-term advantages. You already utilize DSL structures daily, whether you’re working with Rails or RSpec. Explore creating tailored DSLs for your business domain; the investment in time and energy could yield surprising results.

00:30:01.600 Thank you all!

Garden City Ruby 2014