Microservices

Summarized using AI

Dependencies – an asset and a curse

Joakim Antman • May 10, 2023 • Helsinki, Finland

In the video "Dependencies – an asset and a curse," Joakim Antman speaks at the Helsinki Ruby Brigade event, focusing on the complexities and challenges of managing software dependencies, particularly in the context of web development. He begins with a brief introduction of his background in software development, including experiences with various programming languages and frameworks, transitioning through Perl, C++, .NET, and eventually joining the Ruby community during the Rails 4 era.

Key Points Discussed:

  • Introduction to Dependencies: Antman starts by inviting attendees to share their thoughts on dependencies, humorously mentioning his own reliance on coffee while illustrating that dependencies in software can lead to both benefits and complications.
  • Real-World Challenges: The speaker shares a true story involving a service initially built on Ruby 1.8 and Rails 2.0. After being acquired by a larger corporation, higher expectations led to significant challenges in upgrading outdated dependencies.
  • Transition to Microservices: With increased pressure for modern technologies, the organization sought to break the monolithic architecture into smaller microservices; however, poor decisions exacerbated the challenges they faced.
  • Legacy Software Issues: Antman highlights the difficulties faced with outdated versions, particularly with SSL/TLS issues on Ruby 1.8, leading to integration failures.
  • The 'Scale or Die' Document: This document outlined critical maintenance issues, marking the point where they accepted the old codebase was unmanageable. It also indicated a necessity to move away from the old Ruby code base, though existing products impeded new developments.
  • Successful Upgrade Journey: The speaker discusses the eventual successful upgrade from Rails 2.3 to Rails 7, emphasizing the importance of strategic planning and team collaboration during this transition.
  • Establishing a Culture of Ownership: Teams were encouraged to maintain their services by addressing security concerns quickly. A dashboard was created to monitor open security issues and the freshness of dependencies, introducing the concept of 'lib years' to quantify outdated dependencies.
  • Collaboration and Common Goals: Antman's experience leads to the conclusion that while dependencies can be beneficial, they require careful management. Promoting a culture of ownership and setting visible goals can foster collaboration among development teams.

Conclusions and Takeaways:
- Dependencies should be viewed as valuable yet risky; careful management is crucial to avoid potential pitfalls.
- The importance of maintaining hope and not rushing to implement microservices without proper strategy.
- The necessity of establishing a culture within teams to be responsible for the security and freshness of their projects to prevent overwhelming technical debt.
Antman ends on a humorous note, wishing the audience not to encounter legacy software challenges again, particularly referring to Ruby Rails 2.3.

Dependencies – an asset and a curse
Joakim Antman • May 10, 2023 • Helsinki, Finland

Helsinki Ruby Brigade, 2024-02-21.

Helsinki Ruby Brigade

00:00:08.360 Hi everyone. Since most of us have never met each other, I thought I would start by introducing myself. I'm a web developer and have been developing various projects for the internet for some time. I got into this field many years ago while using Perl, which I believe was mentioned earlier today. Back then, it was all about building homepages and collecting data from HTML forms.
00:00:23.160 However, my paid career didn't start in web development. I began with C and C++ developing Windows applications during the Windows Vista era. Yes, that's a time we may all prefer to forget!
00:00:28.600 Eventually, I found my way back to the web through .NET, which made sense given my background in Windows development. During this time, I was also introduced to open source. Although I knew what it was, I started contributing to projects. My first contribution was a jQuery multi-select plugin, which is a bit amusing considering that jQuery seems to be outdated now. But to my surprise, there's a new version of jQuery set to release soon!
00:00:40.960 As times changed, I became involved in the Ruby world during the Rails 4 era, which was a significant turning point for me. Today, I want to discuss dependencies. If you're working with Rails projects, you might find yourself relying on my contributions, as I'm the maintainer of the Ruby JWT gem. Dependency management is a crucial topic we will cover today.
00:01:02.480 To kick things off, I want to hear your thoughts. When you hear the term 'dependencies,' what comes to mind? Coffee, right? I also depend on coffee! Dependencies can be essential, but if you're heavily dependent on them, it can lead to complications. For instance, consider the 'DLL hell' that Windows users faced, which arose from various dependencies. But today, we're not going to focus on our coffee dependency. Instead, we will explore software dependencies and possibly how they can refer to that sense of hell we sometimes experience.
00:01:56.600 I will discuss this from the perspective of a service that operates on the internet, equipped with a user interface and APIs that fulfill a specific purpose and solve a problem. This service has been around for quite some time, enabling businesses to interact with each other. I'll share a true story, albeit with some artistic license, as there may be details that aren't entirely factual.
00:02:27.680 In the early days, this service was built on Ruby 1.8 and Rails 2.0 a very long time ago. Initially, new features were added, and it grew into something valuable for its users. During this time, some dependencies were updated, such as the transition from Rails 2.0 to 2.3. However, one day, this service was acquired by a large corporation, which marked a critical turning point in its development.
00:03:02.920 With new expectations from the new owners, it seemed they believed they were purchasing the best technology in the world. But we were still running Ruby 1.8 and Rails 2.3. This presented a challenge as new stakeholders entered the organization, bringing fresh ideas on how they wished to work with modern technologies. Given the pressure of needing new services and building microservices with happier developers, the idea of separating our existing framework into smaller services became appealing.
00:04:07.440 Back then, we started making various decisions about how to move forward. Unfortunately, several poor choices were made during this period which created an unsustainable workload. We were stuck with an outdated Ruby version, and while new services emerged, the core monolithic application still needed maintenance and updates. Daily developer tasks, such as running tests needed older Ruby and Rails versions, were a constant headache.
00:04:46.320 One of the most persistent issues with these outdated versions was handling SSL/TLS. SSL on Ruby 1.8 was a nightmare. I vividly remember how integrations with external services would break suddenly due to random SSL errors stemming from the outdated OpenSSL library in use. However, there was one instance where we were lucky, happening upon an alternative URL that worked with the Ruby version we had at the time, which allowed us to bypass the problem temporarily.
00:05:49.280 The ongoing challenge of maintaining outdated software mirrored the difficulties one might face when upgrading macOS; management was a complete nightmare, and it felt like we were racing against time. Over time, however, we became desperate. The situation with maintaining Ruby 1.8 and Rails 2.x was becoming unsustainable, and it was necessary to abandon it altogether. Despite the growing success we appeared to achieve, cracks were forming throughout our system.
00:07:14.960 We had several problems, including old database issues, load balancer complications, and performance concerns. Documentation developed during this period contained a checklist titled 'Scale or Die,' which encapsulated the things we needed to tackle to continue to thrive. Looking back, it was a grim indication that we had given up hope on the old code base. In that document, one statement stood out: 'No new features into the old Ruby code base.' This demonstrated our acceptance that the old system had become unmanageable.
00:08:18.000 Despite this, we recognized that not adding new features would not work, especially as our revenue was tied to the existing product. It is crucial to understand that rewrites rarely succeed. Thus, we continued to attempt to update our legacy systems and the frustrations associated with running outdated Ruby versions grew. Yet, despite the lengthy struggle, a glimmer of hope appeared on the horizon.
00:09:42.920 Fortunately, we didn't run a standard Rails 2.3 version; instead, we utilized an LTS version that a group of dedicated developers was maintaining. They assured us that it was possible to run the application using Ruby 2.x. The first major milestone for us was to get the application to run on Ruby 2.3. Against the odds and with plenty of hard work, we accomplished what many thought was impossible.
00:11:06.160 As we updated Ruby versions, we found ourselves celebrating small victories while still limping along with the Rails 2.3 framework. Our continued use of legacy systems remained unsustainable, emphasizing a need to face reality. A solid decision was made in 2023 to finally upgrade from Rails 2.3. We were aware that running on such old software not only posed risks but also was impractical.
00:12:01.720 Before the upgrade, we made a plan that revolved around the 'Rails Upgrade' project. With allocated time to focus on this work, we convened a team, and after three months, we emerged with Rails 7. Although I cannot dive into the intricate details of that process yet, I can confirm that we are still addressing some unresolved issues. This upgrade was a culmination of years of gradual neglect, resulting in various complexities.
00:13:42.360 Additionally, the number of years we had spent maintaining outdated systems meant that various services we created during the years were now older than our oldest application. This realization was disheartening. However, not every service created was useless or needed to be discarded. Some decisions still made sense and aided our functionality.
00:15:07.960 A significant issue we faced was that upgrades were handled by various individuals who had varying levels of commitment. Some people were motivated to update certain parts because they had originally worked on them, while other services fell into neglect. The long-running use of Ruby 1.8 made developers paranoid about upgrades, as they were never straightforward and often came with considerable risks.
00:16:20.920 Thus, we established a culture of ownership amongst the developers responsible for maintaining our services. For starters, this ownership began by requiring teams to ensure their services' dependencies were kept secure. Everyone now had to quickly address security concerns flagged by our internal security tools. In doing so, we created a dashboard that highlighted when any service had an open security issue, prompting quick corrective action.
00:18:09.320 After growing this sense of ownership, we expanded the scope to measure not only security issues but also dependency freshness. While it was easy to manage security issues—either you had them or you didn’t—it became trickier to monitor how outdated our dependencies were. For instance, being two versions behind a specific dependency might not always be alarming, but the urgency increased for Rails versions, where being several versions behind could indicate a significant problem. Hence, we introduced a concept called 'lib years,' which allowed us to quantify how long we had maintained outdated dependencies.
00:19:45.920 With this measure in place, we could articulate our application's state more effectively. Consequently, we committed to a promise that all our Ruby versions would remain on the latest patch version while maintaining a maximum of a 20-year gap with no open security issues. This goal became a shared responsibility among all teams managing various services.
00:21:55.280 Reflecting on this journey, it is clear that dependencies are both valuable and risky. They can propel your development forward, but if not managed properly, they can weigh you down. It's essential to maintain hope, even when the situation appears dire. Splitting into microservices just out of desperation is rarely a sound strategy. Making promises and setting visible goals fosters collaboration, motivating everyone to work towards a common aim.
00:23:10.880 I sincerely hope none of you ever have to deal with Ruby Rails 2.3 again. Thank you!
Explore all talks recorded at Helsinki Ruby Brigade
+1