The Missing System

Ruby

The Missing System

Jim Gay

@saturnflyer

#ruby

#domain-specific-language-dsl

#clean-code

#software-architecture

#microservices

#code-organization

#design-patterns

The Missing System

by Jim Gay

In the presentation titled "The Missing System" at the wroc_love.rb 2015 event, Jim Gay shares insights on organizing code to facilitate growth within software systems. Emphasizing the importance of structuring applications properly before transitioning to microservices, Gay discusses various strategies and tools that can enhance the software building experience.

Key Points Discussed:
- Organizing Code: Gay highlights the challenge of managing complex systems and the necessity of communicating system behavior, especially when dealing with legacy code or onboarding new developers.
- Essential Responsibilities: He examines how to assign responsibilities effectively in code to improve maintainability and reduce frustration.
- Barriers in Code: Introducing barriers can foster a sense of freedom in modifying code. Good code should be 'habitable', meaning developers can confidently make changes without dread.
- 'Worse is Better' Philosophy: Referencing Richard Gabriel's ideas, Gay promotes simplicity in design, advocating for minimalistic systems that avoid over-architecting.
- Visualization of Code: He showcases examples of code that lacks clarity and how restructuring can create better visualization and understanding of functionality.
- DCI Pattern: Gay discusses the Data-Context-Interaction (DCI) pattern, illustrating how to manage object responsibilities coherently to evolve software over time. He explains that software should be adaptable, allowing features to grow organically through piecemeal development.
- Behavior Localizing and Encapsulation: He emphasizes the importance of keeping business logic localized, managing how objects interact without exposing unnecessary implementation details.
- Testing and Context Flexibility: Towards the end, Gay touches on the ease of testing within his framework and how contexts can remain flexible to avoid redundancy.

Conclusions and Takeaways:
- Effective software design requires foresight and simplicity, enabling ease of modification and growth.
- Structuring code with a focus on responsibilities not only aids current development but also sets a solid foundation for future expansions.
- Approach each problem with a mindset of creating systems that are easy to navigate and extend, allowing room for organic growth.
- Tools and methodologies, such as the Surrounded gem, help developers manage their code structures efficiently and maintain clear behavioral mapping among objects.

00:00:15.400 I want to talk a bit about how I try to organize code in my system to plan for growth. A lot of the discussions here focus on how to deal with systems that grow large. Sebastian made a good point earlier: if you want to move to something like microservices, don't do it until you actually know how to structure an application well.

00:00:21.640 These are some of the things I've done to help manage the behavior in my systems. I'm Jim Gay, also known as Saturn Flyer on Twitter, GitHub, and other platforms.

00:00:26.760 I wrote a book called Clean Ruby, and I'm nearly finished with the Ruby DSL Handbook. I’ll be discussing some topics from both of these books today.

00:00:39.640 I'll also talk about a gem called Surrounded, which I developed. Here is the URL for it. I like to visualize my code, and this to me is kind of what a lot of systems look like.

00:00:52.879 There are objects that perform various functions; they share similar attributes or may sometimes be completely different. There are lots of different components in a system, and especially when you encounter a legacy system or onboard a new developer, it can be challenging to communicate what's going on. When they open it up and see something messy and confusing, it can be overwhelming.

00:01:07.680 However, everything begins with one small feature, and at that initial stage, it's straightforward. I want to discuss responsibilities and how we handle them since doing so can significantly impact the software building experience.

00:01:18.799 You could break the software itself, but the experience of managing responsibilities can be fantastic if you're doing it well—easy to work with and change—or it can simply drain your energy.

00:01:36.880 One of the key points I want to emphasize is creating barriers in your code that allow you to make changes. These barriers, in my opinion, can actually provide a lot of freedom.

00:01:49.080 Has anyone read this book or heard of it? Some have, great! This book is by Richard Gabriel, who was influential in the Lisp programming community.

00:01:56.760 If you’ve ever heard the phrase 'worse is better', it originates from him. It conveys the idea that you should create very simple systems and avoid over-architecting your code.

00:02:08.960 You can buy this book; I think it’s out of print, but there's a free PDF available on his site, so I suggest you look for it. I haven’t read the entire thing, but there’s a lot of valuable content.

00:02:24.400 Here's a relevant quote: 'Habitability is the characteristic of source code that enables us to dive in and feel confident about making changes.'

00:02:35.640 This is a bit of sample code I used in a presentation last year at Rubicon. It’s somewhat messy, and if you don’t know what's going on, that’s okay—I’m not going to walk through the entire thing.

00:02:54.560 It grabs values from objects and attempts to create a string representation of an address. It includes elements like address line, city, province, and postal code, and tries to omit certain values if they aren't present.

00:03:06.880 For example, if you don’t have your street address, the output will only show your city and postal code. Looking at this code, it doesn’t feel hospitable; I wouldn’t expect anyone to dive in quickly and see what's happening.

00:03:25.440 In my RubyConf presentation, I demonstrated how I solved this issue. I decided to push all the formatting responsibilities into a template object, allowing the template to manage the formatting of the values.

00:03:41.799 This template takes care of understanding how to format the provided address, and the address simply sends itself to the template method. This person class we discussed earlier receives significant responsibilities pushed down into this object.

00:04:05.760 This approach is easier to manage; if we want to change the format, we know exactly where to make adjustments, without having to search high up in the hierarchy. It essentially operates on the principle of 'tell, don’t ask'—continuously informing objects to perform tasks and pushing responsibilities down rather than querying information externally.

00:04:21.839 This results in clearer responsibilities. Has anyone heard of Christopher Alexander? He’s an architect who significantly influenced many thoughts on software design and architecture.

00:04:41.600 A relevant quote from him is that making buildings for use in a hundred years is quite challenging. Buildings are more permanent than software, yet even he considers how they can evolve over time.

00:04:51.679 Later in the Patterns of Software book, after referencing Christopher Alexander and discussing his ideas, Richard Gabriel suggests that bugs reflect our inability to produce a comprehensive master plan.

00:05:04.800 Much like we can't pinpoint what will be needed in the next two years, or even six months, it’s extremely challenging to create a perfect waterfall plan.

00:05:20.640 Most of the bugs we encounter do not stem from miscalculations but from a lack of understanding of the problem at hand.

00:05:37.600 I'll introduce the tool I use to manage piecemeal growth. Here are the goals I aimed to achieve while working with this code; much of what I wrote stems from research and experiments I conducted while writing Clean Ruby.

00:05:52.600 I aimed to understand DCI—how I can manage responsibilities in my system. The goal is to preserve existing object behavior, particularly concerning the DSLs within.

00:06:06.240 I wanted to localize behavior and apply it only when necessary. I aimed for this to be as non-invasive as possible. For instance, here’s a simple initialization pattern you may come across in your code.

00:06:23.200 You might frequently find yourself doing something akin to this, whether you’re using modules or classes. I often found myself repeating this pattern, which I wanted to simplify.

00:06:39.840 Consequently, I devised a way to keep track of the objects in this class and their roles. I created a map to reference them later.

00:06:53.440 The ZIP here is indicative of the implementation—it looks more simplified when executed. I track the admin and user objects passed in.

00:07:05.840 To streamline my code, I wanted to avoid unnecessary repetition of the same pattern. This is how I handled it while developing the DSL.

00:07:20.560 It’s just a basic class method named 'initialize', which takes several arguments concerning the objects being passed in.

00:07:36.760 It has other methods too, such as setting up reader access and marking certain parts as private.

00:07:50.720 In certain situations, especially when I'm not implementing the same action consistently, I want to retain the established algorithm without repeating the special behavior.

00:08:06.080 I designed my approach to allow modifications without disrupting the existing procedure. Therefore, I created a new 'initialize' method for the point of demonstration.

00:08:21.040 When I call 'super' from this method, it accurately points to the original implementation. My implementation retains whatever structures I previously declared.

00:08:38.600 The functionality rests in that initialized class method. It constructs a module, defines methods on it, and integrates it into the hierarchy.

00:08:55.760 Essentially, this is what is happening here.

00:09:01.480 This is a simplified DSL addressing around 80% of cases. Yet, I enabled the functionality to accommodate additional unique cases.

00:09:20.080 It’s essential to consider coupling, especially in code. We frequently discuss managing dependencies, but cohesion is another important aspect to consider.

00:09:36.720 Cohesion entails how parts of your system relate to one another. When multiple objects share behaviors related to the same functionalities, they are highly cohesive.

00:09:53.360 Conversely, if behaviors are unrelated, cohesion is low. Hence, managing cohesion is critical, and this quote exemplifies how to approach changes in your system.

00:10:10.960 When diving into your code, you don’t want to spend excessive time making sense of it all to find a solution.

00:10:28.000 Consider how much information you need to juggle before addressing the problem you set out to solve.

00:10:43.440 Programs can be complex, with various objects performing different tasks. However, it's difficult to discern which objects relate to each other.

00:10:56.720 To remediate this, I want to build cohesive sets of behaviors amongst the objects within my system.

00:11:10.560 I desire features that feel habitable, allowing me to easily navigate through highly cohesive parts that I can compose and think of in smaller terms.

00:11:20.440 In the Surrounded gem, this mapping occurs automatically. I devised a pattern where I create a module or class, allowing for flexible implementations.

00:11:31.200 Yet, there's no reason not to follow this guideline. You can opt for any implementation you like, whether you place methods in modules or define them in a class.

00:11:43.760 A particular irritation I encountered was the difficulty when changing implementations—especially when many modules or classes were defined.

00:12:00.480 So, I created a DSL that enabled easier selection. By specifying changes within the code, I could adapt to implementations without significant overhead.

00:12:14.960 The system still functions even without the DSL; it handles cases effectively. You maintain the flexibility to create your own classes, provided they map correctly.

00:12:28.000 What does this tool accomplish? The context serves as more than a namespace; it protects access while localizing the behavior.

00:12:36.000 This reputation is vital because I do not want external access to implementation details of certain business processes.

00:12:49.440 I want everything concerning a particular problem managed internally. If we need some implementation, we can utilize that specific object.

00:13:04.080 It's not essential to eliminate all duplication; sometimes, it’s acceptable to repeat behavior if it isn’t identical.

00:13:15.760 Using this approach ensures that business logic remains localized, underscoring the strict limitations imposed by my DSL.

00:13:29.360 Should you venture outside those limits, you can freely share behavior; however, using that roll method guarantees all elements are private.

00:13:38.960 Given everything localized, what should I do with it? My third goal is to apply behavior only as needed, similar to any other class.

00:13:50.800 This class has a range of features, and methods can be defined just like any other object. Here's how I use them: when working with an object derived from the class, all designated behaviors will apply.

00:14:04.880 Thus, each role can wrap or extend its behavior as outlined, and when the actions are performed, the behaviors are unwound in an insure block.

00:14:18.000 This ensures there are no lingering behaviors post-execution, particularly important if an exception arises.

00:14:28.720 I also devised a method to manage behavior wrapping, ensuring the appropriate object behaviors are consistently applied.

00:14:39.920 When I initialize an object like a user activation class, I can assess the triggers within the same context and summon all pertinent methods.

00:14:55.679 If I'm looking for the specifics regarding when these behaviors will apply, I can retrieve all available triggers. I've even used this to build user interfaces.

00:15:09.440 If a user interacts with a portion of my system that has several actions available depending on their role, I can display buttons accordingly.

00:15:27.760 Only the actions relevant to a user are displayed based on predefined behaviors within that context, refining user interactions.

00:15:41.840 Now, if the systems I make are growing through piecemeal growth, how can I effectively manage that evolution? I want the objects passed into my context to communicate with one another.

00:16:00.560 I am treating them as distinct systems, which can be a challenge in any programming language. This can often feel counterintuitive, but with Ruby, it posed specific challenges.

00:16:15.760 Upon starting my research on DCI, I initially dismissed it but later realized its significance through my pain points.

00:16:31.640 In contexts, should I decide to append an object later, it quickly transforms into a complex network, necessitating method signature adjustments, which is cumbersome.

00:16:48.680 Consequently, I revisited the DCI concepts and realized that these objects should act as a unified system that recognizes one another.

00:17:05.040 This technique made it significantly easier to add objects to the system, allowing them to autonomously interact with one another.

00:17:18.160 This module enhances object functionality, granting them access to their current context—here, it's the user activation.

00:17:31.760 Utilizing method missing, objects can examine their context to discover who is present.

00:17:48.000 I want to prevent method signatures from becoming unwieldy by ensuring that objects can access necessary methods without excessive parameters.

00:18:05.160 The design sometimes appears unconventional; predicate methods should ideally return Boolean values. However, the context method allows each object to ask, 'Is this named role within?'

00:18:21.760 If the role isn’t present, it short circuits, continuing on. Should the object exist within the assigned player role, access is granted.

00:18:38.560 I keep track of roles, the objects embodying them, and their behavior constants. I can inquire if a particular name exists and connect it accordingly.

00:18:51.680 An empty block is passed during method missing; this aids in identifying which object is attempting to access other roles.

00:19:07.680 To establish relationships between multiple objects within the same context, these elements can communicate privately, allowing for complex interactions.

00:19:22.720 If I add any behavior to one role, it can command the other roles accordingly.

00:19:36.960 By evaluating the string self from the method binding, I establish which object is sending the message, thus preserving context.

00:19:52.560 Every role retains localized communications, ensuring external objects do not call methods or induce side effects without my logical intervention.

00:20:07.920 Allowing only objects contained within the context to access other objects defines how implementation details are managed.

00:20:22.560 It's necessary to reflect on how accessible the implementation details should be, ensuring private constants and methods are preserved, contrary to popular Ruby practices.

00:20:37.760 Many Rubyists tend to publicly declare methods, but I encourage trying the opposite while recognizing how effective it can be.

00:20:50.880 Richard Gabriel emphasizes in his book that piecemeal growth is a reality that we often overlook. We don't start by constructing vast, intricate systems.

00:21:07.040 While we might understand the complexity of certain domains, they tend to evolve over time, highlighting the learning process inherent in software development.

00:21:21.760 Utilizing this tool allows for gradual growth; what begins with two interacting objects can expand naturally as additional elements emerge.

00:21:38.800 Should you decide to integrate this approach? While this may not perfectly align with everyone’s needs, especially if components seem overly complex, it grew on me.

00:21:55.760 Initially, I hesitated to implement certain features, but over time, I recognized their necessity, reflecting how I cultivated both my systems and the Surrounded project.

00:22:12.160 Thank you.

00:22:18.560 Thanks, Jim. Questions, anyone?

00:22:34.640 Do you feel like it's somewhat similar to the concept of bounded context? Before you define bounded context, you could apply it and later extract it as a microservice?

00:22:47.440 Yes, while I can’t claim extensive knowledge about DDD, listening to the panel made me realize that people's view of DCI may miss the detail that a specific role in one context doesn't need to share code with roles in another.

00:22:58.360 Exactly!

00:23:05.840 How easy is it to write tests using this gem? What does it take to test the implementations?

00:23:18.080 The testing process chiefly concerns how you compose objects. Your goal is to facilitate task allocation within a system.

00:23:32.960 Thus, while testing, you want the external world to perceive a change in state after object interaction, employing a black box approach.

00:23:47.760 Are the contexts and use cases composable? If they aren't, how do you avoid redundancy during implementation?

00:24:01.680 I often find myself needing to reuse objects in other contexts, and this model accommodates that without forcing continued use of the same context.

00:24:14.000 One of the remarkable features here is the ability to compactly map behavior, enabling fast construction of contexts.

00:24:27.280 The ease of creating multiple contexts permits me to experiment with distinct implementations quickly, free from the burden of permanently altering larger structures.

00:24:42.160 You might discover many advantages to building these rapidly since your approach remains compartmentalized, ultimately simplifying conceptualization.

00:24:56.720 What is 'respond_to_missing'? Is it a Ruby feature or part of your code?

00:25:09.720 That's actually part of Ruby's functionality. If you define 'method_missing', you must also set up 'respond_to_missing' to signal if the object responds to specific messages.

00:25:20.640 What is your new book about? Is it focused on using DSL for specific purposes?

00:25:33.560 I’m writing the Ruby DSL Handbook, which stems from my client work focused on QA automation and making declarative representations for web pages.

00:25:48.240 While teaching developers about the structure of designs, I aimed to maintain flexibility, ensuring that dynamically-defined methods remain localized.

00:26:03.040 It’s a short book sharing simple ideas to reduce complexities during the development phase.

00:26:11.680 Any more questions?

00:26:22.600 Thank you very much, Jim.

Jim Gay

@saturnflyer

wroclove.rb 2015