Ruby Video
Talks
Micro Libraries FTW
PS
Piotr Solnica
5 talks
#ruby
#dependency-management
#rubygems
#frameworks
#open-source
Summarized using AI

Micro Libraries FTW

by Piotr Solnica

In the video titled "Micro Libraries FTW," Piotr Solnica discusses the concept of micro libraries within the Ruby programming ecosystem. The speaker expresses a vision for a Ruby ecosystem composed of numerous small, focused, and well-documented libraries that can work together seamlessly. This talk, presented at wroc_love.rb 2014, aims to explain the benefits of micro libraries in contrast to larger libraries, as well as introduce a new initiative called micro.rb.

Key Points Discussed:

- Differences between Big Libraries and Micro Libraries:

- Big libraries address multiple problems and have complex, broad interfaces, while micro libraries focus on solving specific problems with narrow, intuitive interfaces.

- Micro libraries generally have less code and less frequent changes compared to their larger counterparts.

  • Impact of Ruby on Rails:

    • Rails has popularized larger libraries and extensive monkey patching, which can create a tendency for developers to build complex libraries rather than focused, small ones.

  • Advocacy for the UNIX Philosophy:

    • Solnica encourages adopting the UNIX philosophy, which promotes simple tools that can be combined to create powerful solutions, highlighting a missed opportunity for creating more micro libraries.

  • Use Cases for Micro Libraries:

    • Micro libraries are beneficial when extracting complex functionality for reuse, generalizing repeated patterns, or addressing very specific problems in a focused manner.

  • Frameworks Built on Micro Libraries:

    • Solnica asserts that frameworks should be constructed using micro libraries, emphasizing a top-down approach to maintain clean architecture while avoiding dependency conflicts.

  • Challenges with Dependencies:

    • Issues can arise with dependency management in Ruby due to slow installation and conflicts when using multiple libraries. Proper semantic versioning and documentation are crucial to mitigate these challenges.

  • Examples of Micro Libraries:

    • The talk references several existing micro libraries, such as Equalizer and Anima, showcasing how they work together smoothly to provide functionality without added complexity.

  • Introduction of Micro.rb:

    • The new project micro.rb aims to compile focused libraries meeting specific criteria, facilitating users in finding stable, production-ready libraries and connecting with maintainers.

Conclusion:

- Piotr Solnica concludes that the development of micro libraries is essential for establishing a rich Ruby ecosystem. He believes that frameworks built upon these libraries can lead to simpler, more efficient, and maintainable codebases, urging developers to embrace this approach for enhanced clarity and functionality in their projects.

00:00:12.879 Hello.
00:00:19.060 Thank you for joining me. I'm going to talk about micro libraries, but first, let's start with a quick note.
00:00:25.390 I want to thank Matt. My GPU in my laptop broke yesterday, so I bought a MacBook from Matt. Thank you! He saved my life.
00:00:41.210 So, who am I? Unsurprisingly, I'm a Ruby developer actively working on various open source Ruby libraries. You can find me on GitHub or follow me on Twitter.
00:00:51.980 This is one of the biggest open source projects I was working on recently, and I want to mention it because this talk is heavily based on my experience with this particular project.
00:01:07.369 Now, what's on the agenda? I'll start by explaining some major differences between big libraries and micro libraries. Then I'll talk a little about micro frameworks and demonstrate some example micro libraries that we already have. At the end, I want to introduce you to a new project called micro.rb, which I launched last weekend.
00:01:30.440 So, what are big libraries versus micro libraries? Here's a simple table showing four major differences.
00:01:46.729 First, big libraries solve many problems, whereas micro libraries focus on solving just one specific problem. This leads to the next point: big libraries are well-baked and have lots of lines of code, while micro libraries tend to have a very small amount of code.
00:02:35.240 The next difference is in their interfaces. Big libraries, since they support a lot of functionality, often have very wide interfaces that might even be confusing to use. In contrast, micro libraries have very focused, narrow interfaces that are easy to use and quite obvious.
00:02:50.420 Lastly, let's discuss the concept of churn. Big libraries tend to change a lot—churn represents how often code within a given library is modified. In contrast, small libraries don’t undergo frequent changes, especially after reaching a major version.
00:03:09.860 I want to talk a little bit about Ruby on Rails' influence on this topic, as I believe it’s closely related. First of all, Ruby developers love Rails, primarily because many of us began our journey with Ruby through Rails, myself included.
00:03:39.530 We learned Ruby via Rails, which is somewhat peculiar because we started with a very specific mindset—the Rails mindset. I think Rails has a tremendous impact on how we write Ruby code. There are three aspects of Rails and the styles it promotes, implicitly or explicitly.
00:04:07.100 First, Rails is in favor of larger libraries, consisting of just five libraries, all of which are gigantic. If you’ve ever looked into ActiveRecord or ActiveSupport, you’ll see they encompass thousands of lines of code, adding serious complexity.
00:04:51.830 Second, ActiveSupport promoted monkey patching in an extreme way, leading developers to monkey patch everything. Personally, I consider monkey patching an anti-pattern, and it should generally be avoided in libraries.
00:05:12.290 Finally, I’ve noticed a recent trend where people are hesitant to introduce new classes and objects, possibly because they’re accustomed to ActiveRecord, which handles too many responsibilities at once.
00:05:39.120 I've received numerous feature requests in my projects where users ask, 'How about adding this?' or 'What about this feature?' My response is often, 'No, it’s good as it is. We don't need extra features; they should be handled by a separate layer.' Users, however, are often confused, as Rails offers it all.
00:06:02.629 Lately, I’ve been thinking about our dependence on Rails. For the last few years, I’ve been discussing with people writing about the issues with Rails and proposing alternative methods of utilizing it.
00:06:34.410 I've concluded that it’s better to embrace the UNIX philosophy—a fundamental concept that encourages building simple programs that perform trivial tasks but combined can create powerful, useful tools.
00:06:48.810 I think we are missing many of these small, focused tools that we could use in conjunction to build high-level frameworks. This is why I believe micro libraries are invaluable.
00:07:14.810 Now, why would you consider building a micro library? There are three scenarios where building a micro library makes sense. The first is when you have complex functionality in an existing project that you anticipate needing in another project or multiple projects.
00:07:34.327 In such cases, it’s an excellent opportunity to extract, decouple, refactor, and clearly specify interfaces to create a micro library that can be reused.
00:07:59.819 The second use case is when you notice a pattern that you keep repeating across different parts of your codebase. This is a good opportunity to think about how to generalize, extract, and encapsulate that pattern into a small library.
00:08:18.580 The last use case, which is probably rare, is when you have a very specific problem you want to solve that can be addressed by a very focused library.
00:08:30.360 For example, if you're developing an ORM and need to generate SQL, you would want an SQL generator.
00:08:51.850 Now, let's shift focus to frameworks. I believe frameworks should be built on top of micro libraries. While building frameworks, I've noticed two main approaches: bottom-up and top-down.
00:09:06.690 Both methods have their challenges. Bottom-up is only beneficial when you truly understand the complexities involved, which doesn’t often happen.
00:09:26.040 Usually, there’s this experimental phase where you need to play with things and learn. This leads me to believe that the top-down approach is often more effective.
00:09:48.700 In the top-down approach, you start with a higher-level concept and work your way down, identifying what you need along the way.
00:09:59.450 However, I find that the top-down approach can often be misleading; it’s great in theory but has flaws in practice, especially in Ruby projects.
00:10:16.710 Many developers begin building immense frameworks while mixing together high-level concepts and core domain logic, which often results in disaster.
00:10:32.150 A prime example is Rails itself; it experienced considerable hurdles that required a complete rewrite.
00:10:50.180 So, when building a framework, you ought to focus on the high-level concerns, leveraging the underlying micro libraries while keeping the code simple and clean.
00:11:04.480 Obviously, there are challenges involved; I could give a whole talk just on that. One of the primary issues is managing dependencies.
00:11:23.040 In Ruby, we have RubyGems and Bundler, but they can be slow. From a developer's viewpoint, installing something or dealing with gem files and version specifications can get tedious.
00:11:50.150 When you have many dependencies and start encountering conflicts, you could end up in a situation where you are completely locked out because of a version conflict.
00:12:10.390 While conflicts can be resolved by adjusting version specifications, they are still incredibly annoying.
00:12:28.046 We also have something called meta gems, which are gems that depend on multiple other gems. When people install such a gem, they often see many other dependencies being installed.
00:12:49.230 I've seen users get overwhelmed, thinking they picked a lightweight library only to discover it installs several other libraries.
00:13:05.220 While dependencies can be problematic, I believe that having genuinely focused libraries that can work together is a better approach, even if it introduces some dependency challenges.
00:13:27.650 The biggest issue arises when frameworks depend on the same gem; when dependency hell strikes, it can cause significant complications.
00:13:44.880 I was considering how to tackle this; the most crucial step is being strict with semantic versioning. It's critical that conflicts between versions are managed accurately.
00:14:06.860 It's preferable to have a conflict rather than releasing library versions that install but do not function properly. Maintain your version numbers diligently.
00:14:29.639 Documentation is vital; please maintain proper changelogs. A simple note about breaking changes in changelogs can save a lot of developers time.
00:14:50.779 There are online services that can assist with this. For instance, Ruby Toolbox lets you see what other gems depend on your gem.
00:15:02.710 I’ve found it helpful to identify which of my projects are being depended on by others and engage with their maintainers.
00:15:16.509 Whenever I'm about to make breaking changes, I inform affected maintainers, which facilitates collaboration.
00:15:29.740 Another useful tool is Gemnasium, which shows the dependencies of your gem, making it easier to remain current with updates.
00:15:42.290 Let me share some example micro libraries that we have. Most of these were created while we worked on Ruby Object Mapper or were inspired by that project.
00:16:00.660 The first is 'Equalizer', a tiny gem that enables you to include a module, pass in the desired attributes, and gain equality methods along with a nice inspect method for free.
00:16:18.950 Another example is 'Anima', which is developed by Marcus. It leverages Equalizer internally, allowing you to create constructors that accept hashes while also providing equality methods.
00:16:30.919 This also includes the inspect method from Equalizer, showcasing how several libraries can work seamlessly together.
00:16:44.489 Objects in Virtus will utilize Equalizer under the hood, providing additional functionality for free, like equality methods and inspection.
00:16:56.910 So, concerning frameworks, I believe they should be constructed using small, focused components. For instance, Ruby Object Mapper is evolving into a framework.
00:17:14.140 Internally, it is shaping up to resemble a domain-specific language (DSL), and I've been emphasizing higher-level interfaces.
00:17:31.230 One significant challenge in data mapper libraries is managing data as it flows between the database and your objects.
00:17:48.130 When extracting data from the database, you must load it correctly into objects, then, when saving, it must be transformed back into a database-compatible representation.
00:18:07.080 Searching for a pure mapper library was essential, and I discovered that Morpher, developed by Marcus, has become the effective mapper for Ruby Object Mapper.
00:18:23.100 This is fantastic because Morpher elegantly handles data loading and dumping, allowing you to call just one method to execute these transformations.
00:18:41.080 If we were to resolve this within ROM, we would end up writing countless lines of code; instead, by relying on Morpher, we alleviate that complexity.
00:19:00.490 Morhp's transformers offer a clean solution to this, and with ease, we can call to reverse the transformation for data dumping.
00:19:18.510 This is a huge problem, and Morpher addresses it effectively, eliminating the need for excess complexity within my codebase.
00:19:35.890 Initially, while working on Ruby Object Mapper, we aspired to promote a development style that favors small, simplistically focused libraries.
00:19:50.340 I realized that using a data mapper implementation might not align with that ideal.
00:20:06.550 I wanted to establish a separate project to more directly promote those principles.
00:20:25.450 Hence, I stumbled across a website called microjas.com, and it inspired me to start micro.rb.
00:20:41.290 Micro.rb will eventually compile libraries that adhere to specific criteria which we are still defining.
00:20:58.880 The differences between big libraries and micro libraries that I previously mentioned will likely become foundational criteria for this site.
00:21:16.540 I want micro.rb to serve as a tool where everyone can find libraries that solve specific problems efficiently.
00:21:34.280 It should be easy to identify libraries that are stable and used in production, ensuring confidence in their selection.
00:21:50.470 Additionally, I want micro.rb to serve as a platform connecting maintainers with potential contributors, ensuring libraries stay well-maintained.
00:22:07.740 This initiative aims to promote the UNIX philosophy, supporting a development style centered around small, effective libraries.
00:22:23.200 I envision creating a new ecosystem filled with distinct, remarkable libraries that can seamlessly combine to create higher-level tools.
00:22:39.020 I want to advocate for frameworks built atop these libraries, like Ruby Object Mapper, as they represent a much better direction.
00:22:57.760 And that’s it! Thank you.
00:23:42.500 Thanks for the talk! Are there any questions?
00:23:52.710 Is there ever a situation where you would consider constructing a big library?
00:24:09.120 No, not at all.
00:24:26.100 So, you believe that all libraries can and should be split into small, focused libraries?
00:24:44.150 Yes, that's my current belief.
00:25:12.630 Do you believe Virtus is still a micro library, considering it initially aimed for single-focus attributes but has become more complex?
00:25:37.270 I think it has evolved into something larger; while it's still under 2,000 lines, its complexity has increased significantly.
00:25:54.650 Now, I would characterize it as a micro framework—tiny but still a framework.
00:26:16.670 Have you ever faced issues with this approach of separating into smaller concerns?
00:26:36.840 Yes, particularly in the early stages, while building projects simultaneously, we experienced pain during upgrades.
00:26:53.130 However, over time, we have started to benefit from that groundwork.
00:27:05.900 Do you think typically the UNIX philosophy works effectively within micro libraries in Ruby, considering the increased learning curve?
00:27:16.000 The costs are indeed higher, but I perceive frameworks as comprising higher-level libraries.
00:27:39.620 Micro libraries generally handle simpler tasks, so their use may not be apparent unless you dig into how a framework utilizes them.
00:27:59.030 Is ROM an essential part of your experience, or are you needing assistance in your projects?
00:28:10.870 How do you work when you invite help?
00:28:31.200 The process can be tricky. We operate ad hoc; there's no precisely organized roadmap.
00:28:55.080 If anyone would like to help, they should reach out on IRC or contact me through Twitter or email.
00:29:20.010 What are your thoughts on using micro libraries together to create your framework? How do you best make them cooperate?
00:29:57.080 I create wrappers particularly in Rails apps to interact with them, and I conduct integration tests to ensure everything works.
00:30:14.080 Do you believe we should use micro libraries directly or assemble them into a larger library?
00:30:30.440 It depends. If you need to use them directly, then do so, but it may be more convenient to build on existing frameworks too.
00:30:50.100 It essentially comes down to preference; you can write more code if you need to, but you have more control over what you’re implementing.
00:31:23.800 Does anyone have further questions?
PS
Piotr Solnica
5 talks
Edit
Explore all talks recorded at wroclove.rb 2014
+13