Talks
Keynote: Second system syndrome
Summarized using AI
Keynote: Second system syndrome
by Yukihiro "Matz" MatsumotoIn this keynote presentation delivered at BalticRuby 2024, Yukihiro "Matz" Matsumoto, the creator of Ruby, discusses the concept of 'Second System Syndrome' — a phenomenon whereby engineers, after achieving success with a smaller system, become overly ambitious when designing a successor. Matz reflects on his own experiences and the pitfalls of this syndrome, sharing insights on software development and community dynamics that can lead to failure. He highlights key points including:
- Definition of Second System Syndrome: The syndrome describes how engineers, after initial success, tend to complicate their designs, leading to failure.
- Historical Examples: Matz cites examples from programming languages like PL/I, Perl 6 (now Raku), Python 3000, and PHP 6, illustrating how ambitious redesigns often result in projects that are overly complex and not well-adopted by users.
- Impact of Boredom in the Community: Matz talks about how boredom can drive members of an open-source community away, stressing the importance of keeping projects engaging and continually evolving.
- Ruby's Evolution: He discusses the transition from Ruby 1.8 to Ruby 1.9 and the more incremental changes leading to Ruby 3.0. He emphasizes learning from past mistakes to avoid the traps associated with Second System Syndrome.
- Performance Improvements: Looking toward future updates (like Ruby 3.4), Matz outlines Ruby's goals for enhancing performance without introducing major incompatibilities.
- Community Engagement: Matz underscores the necessity of an engaged community in maintaining a programming language, where feedback and incremental improvements drive evolution and relevance.
Matz concludes with a call to action for developers to work gradually, learn from historical pitfalls, and maintain an active role within the open-source community for Ruby's future success. He emphasizes the importance of balancing innovation with stability to keep Ruby competitive and appealing to new talent and ideas.
00:00:07
Please give a warm welcome to our keynote speaker, Yukihiro "Matz" Matsumoto, the creator of Ruby.
00:00:23
Good morning! I'm a little bit nervous. The last time I visited Europe was in early 2020 at RubyConf in Paris.
00:00:30
Shortly after that, the COVID-19 pandemic happened, and we were all forced to have online conferences.
00:00:39
However, the biggest value of a conference is the face-to-face networking, meeting new people, and taking photos.
00:00:50
So, it’s great to be back! Today, I'm going to talk about my failure almost 20 years ago, and I will explain the concept of 'Second System Syndrome.'
00:01:03
Have you ever heard of the Second System Syndrome? Okay, some of you have. It first appeared in the legendary book titled "The Mythical Man-Month."
00:01:13
Its definition states that when someone is designing a successor to a relatively small, elegant, and successful system, there is a tendency to become emboldened by that success.
00:01:27
This can lead to designing a system that is overly complex or a feature-laden monstrosity.
00:01:35
It describes the jump from the simple operating system SY on the IBM 7000 series to the OS 360.
00:01:43
A similar effect can happen in an evolving system. It's somewhat confusing.
00:01:51
When we create a nice system, for example, a programming language or any software, we know that it won’t be perfect. It may be good, but not perfect.
00:02:06
So, we are tempted to create a second system because of our historical mistakes or design flaws.
00:02:13
We attempt to throw away the first system and rebuild everything from scratch, trying to solve every imperfection all at once.
00:02:27
Have you ever failed to resist that kind of temptation? I have.
00:02:34
This is part of human nature; every engineer is human after all.
00:02:43
We are often driven to create a new, perfect system by abandoning the first system.
00:02:51
At that time, we envision a second perfect system that fixes all the drawbacks and mistakes, hoping to create a perfect system without any flaws.
00:03:05
But as always, our estimations tend to be overly optimistic.
00:03:18
For instance, we often think a system will be finished before the deadline, but usually, that doesn’t happen.
00:03:27
This tendency is part of human nature; our estimations are typically too optimistic.
00:03:34
The second system tends to be overly complex and time-consuming, consuming too much money, and ultimately, it is prone to failure.
00:04:02
This pattern of failure is repeated frequently. In "The Mythical Man-Month," the second system is described as the biggest mistake or obstacle in software projects.
00:04:11
For example, we once had a language called PL/I, which was the second system. It was an ambitious language created in the early 1960s.
00:04:24
At that time, we had existing languages like COBOL and FORTRAN.
00:04:30
COBOL was designed for business applications, while FORTRAN was developed for scientific computing. Both are still in use today, although their popularity has diminished.
00:04:51
Some banking systems are still developed in COBOL, and FORTRAN is also used for various scientific computations.
00:05:03
However, the attempt to create the second system, which was supposed to be a perfect language for both business and scientific use, did not succeed.
00:05:10
When I was in school, I encountered PL/I. It was a nice programming language, but it wasn't as attractive as C, C++, or even COBOL.
00:05:20
The Second System Syndrome occurs everywhere.
00:05:34
Around 2000 to 2004, the programming language community failed and fell into the pitfall of the Second System Syndrome.
00:05:44
I called this phenomenon 'language lence.' It started with the Perl community.
00:05:50
This was the beginning of web applications. The Perl community was very popular in programming domains. But in 2000, at a Perl conference in the USA, someone shouted that we needed something new.
00:06:07
They expressed that everyone was getting bored with Perl and that they were leaving the community.
00:06:20
In an open source community, members are free to leave if they find something more exciting, like JavaScript or PHP.
00:06:30
To combat this boredom, they began a new project to remove historical baggage and collected proposals from the community.
00:06:38
They referred to this collection as RFCs, requests for comments, and the authors compiled these requests into a document called Apocalypse.
00:06:53
Eventually, they created 12 books that described the future of a new programming language called Perl 6.
00:07:03
However, Perl 6 was a huge challenge that ended up embodying the typical Second System Syndrome.
00:07:13
It became overly ambitious and ultimately too optimistic in its goals.
00:07:24
Boredom is a significant enemy of open-source software. When the community gets bored, they tend to leave.
00:07:32
The open source software must continue to move forward to keep the community engaged.
00:07:45
We can’t let boredom take over; we should avoid falling into the Second System Syndrome.
00:07:54
What happened to Perl? The first version of Perl was released in 1986, Perl 5.0 followed in 1993, and we finally saw the release of Perl 5.38 last year.
00:08:06
But what about Perl 6? How many of you are currently using Perl 6?
00:08:14
Very few, right? Perl 6 was renamed Raku in 2019, yet it is still not widely used.
00:08:27
Despite Raku having wonderful features and advanced capabilities, very few people are using it.
00:08:39
I can attest that 'scrap and build' is generally a bad idea for larger systems.
00:08:46
If your system can be rebuilt in a day, a week, or so forth, it might be feasible.
00:08:55
But if your system is complex and large, rebuilding can take years.
00:09:07
That’s what we learned too.
00:09:11
In the early 2000s, the Python community made a similar mistake with Python 3000.
00:09:26
They tried to remove all historical baggage, which ultimately resulted in a split community.
00:09:38
Python 3000 transformed into Python 3, but many developers continued to use Python 2 for years.
00:09:51
More than 15 years later, many in the community still used Python 2, which was a tragedy.
00:10:03
In 2020, they finally stopped maintaining Python 2, and the community began migrating to Python 3.
00:10:15
It took over 15 years for the Python community to entirely migrate because of this Second System Syndrome.
00:10:25
Similarly, the PHP community faced issues with the development of PHP 6, which was eventually cancelled.
00:10:37
They started PHP 7 from PHP 5 and learned from their mistakes.
00:10:50
In the JavaScript world, a similar situation occurred with JavaScript 4, which was ultimately cancelled.
00:11:00
The drastic changes introduced were too many at once, which the community ultimately rejected.
00:11:13
They returned to JavaScript 5, going back to the older version.
00:11:27
During my keynote presentation at the Ruby conference in 2003, I declared a project for Ruby 2.
00:11:40
Ruby 2 was supposed to be a version different from Ruby 2.0, which we finally saw in 2013.
00:11:53
Ruby 2 was meant to be similar to Perl 6, where we would attempt to refresh Ruby by removing historical baggage.
00:12:03
Before we fell into the trap of past failures, we gathered Ruby Change Requests (RC) and many ideas for the future of Ruby.
00:12:12
However, I realized I was too lazy to handle all the paperwork and eventually gave up on the project.
00:12:24
Fortunately, we had incremental progress and introduced Ruby 1.9, which made significant improvements after Ruby 1.8.
00:12:36
We added multilingual support which included UTF-8 compatibility.
00:12:47
Although Ruby 1.9 led to some incompatibilities, it represented a far shorter split than Python experienced.
00:12:58
The transition period was around five years, which is much better than the 15 years seen in Python.
00:13:09
Many developers still used Ruby 1.8, however, they gradually migrated to newer versions.
00:13:18
There were also enhancements from Ruby Enterprise Edition, which aimed at bolstering Ruby 1.8.
00:13:30
Around that time, we started observing a quicker migration towards newer versions.
00:13:41
Ruby 3.0 kept evolving and now our focus is on making the language faster.
00:13:55
We aim to increase performance yearly, slowly but surely.
00:14:00
With the recent addition of the JIT compiler, we've seen impressive performance improvements.
00:14:09
A Jet compiler was introduced to Ruby 2.6, demonstrating significant speed benefits.
00:14:22
In a CPU-intensive benchmark, it was reported that Ruby now processes tasks up to three times faster than before.
00:14:34
However, we found out that in Ruby on Rails, the applications showed about a 5% slower response.
00:14:48
This happened because Rails applications consist of multiple methods requiring compilation by the JIT compiler.
00:15:06
Nonetheless, the new technology promises an exciting and significant boost to overall speed in future applications.
00:15:17
Scrap and build is indeed a poor approach. We should evolve gradually.
00:15:31
With Ruby, past features like keyword arguments and method combinations were all reintroduced progressively.
00:15:43
Most of the ideas I had for Ruby 2 have found their way into Ruby 3.
00:15:54
Instead of introducing them all at once, we paced ourselves.
00:16:02
The performance improvements we are looking at will be evident in future Ruby releases.
00:16:16
As we move forward, we wish to incorporate new features but need to be cautious about compatibility.
00:16:30
In 2024, we are looking closely at new is implementations to avoid any mistakes made in the past.
00:16:44
Aiming for the Ruby 3.4 update, we are determined to uphold community interest.
00:16:56
With this update, we plan to integrate more features while maintaining compatibility.
00:17:07
The improvements will include enhancements to method inlining and memory management.
00:17:22
Our goal is to make it significantly easier for developers to run their applications.
00:17:38
We anticipate that future versions will build upon the lessons we have learned over the years.
00:17:51
Wouldn’t it be nice to see quicker garbage collection distributions to boost performance?
00:18:05
Additionally, we are also focusing on adaptive garbage collection methods.
00:18:17
While existing systems can already manage memory efficiently, we look to optimize that further.
00:18:30
If we can adjust our garbage collection based on access patterns, we can improve responsiveness immensely.
00:18:43
So, we are exploring all these avenues to ensure Ruby stays competitive in the current landscape.
00:18:54
I believe that in the long run, performance is paramount.
00:19:04
I will also be referring to efforts in application servers like Falcon.
00:19:16
These will help better optimize our web applications through innovative design.
00:19:29
With integration of features such as Fibers yielding faster context switching, we gain traction for a better model.
00:19:44
The goal is to make these integrating features feel natural for developers implementing them within their projects.
00:19:59
I hope to expand on these ideas further as part of our planned roadmap.
00:20:11
An open-source community is reliant on active participation; it’s critical to make Ruby appealing.
00:20:22
Strong tools and libraries are essential for a solid Ruby ecosystem.
00:20:35
We have seen numerous applications in various industries thrive thanks to Ruby.
00:20:48
It’s vital that we continuously engage with the community for feedback.
00:20:58
The open environment in Ruby is part of what makes it a wonderful language; we must continue to expand and improve.
00:21:11
Let’s foster an atmosphere that inspires new talent to emerge within the Ruby community.
00:21:25
We’re working very hard to keep Ruby relevant in the years to come.
00:21:39
Together, we can build a more robust future for Ruby and its community.
00:21:49
Thank you for your time today. I appreciate everyone's engagement and support!
00:22:02
Special thanks to the sponsors who made this event possible.
00:22:09
Thank you to the organizers for putting together such a fantastic conference.
Explore all talks recorded at BalticRuby 2024
+4