Metaprogramming
Ruby Arrays on Steroids
Summarized using AI
Ruby Arrays on Steroids
by Michael FeathersIn the talk "Ruby Arrays on Steroids," Michael Feathers explores the limitations that conventional programming paradigms impose on developers and highlights how adopting forgotten or less mainstream paradigms, particularly those derived from APL, can enhance problem-solving capabilities in Ruby. The presentation revolves around the theme of empowering programmers to utilize higher levels of abstraction to produce more concise and elegant code.
Key Points Discussed:
- Programming Assumptions: The discussion begins with common misconceptions in the programming community regarding the capabilities of mainstream programmers, highlighting how these assumptions can lead to less powerful and verbose solutions.
- Comparison of Languages: Feathers contrasts the programming experience of languages like C and Ruby with Lisp languages, noting that many find Lisp's functional programming capabilities superior, yet it remains less popular largely due to its complexity.
- Functional Programming Concepts: The importance of higher-order functions, closures, and lambdas, which are now being integrated into languages that previously lacked them, is underscored as essential tools that empower programmers.
- Conciseness in Code: Feathers emphasizes that utilizing concise code through powerful abstractions leads to improved software quality and fewer bugs, demonstrating how complex tasks in Ruby require significantly fewer lines of code than in languages like Java.
- APL and Its Derivatives: The talk introduces APL, a powerful but often misunderstood language, and then discusses J, another derivative, showcasing their capabilities in array manipulation and concise programming.
- Introducing Jop: Feathers mentions a Ruby gem called Jop that aims to extend Ruby's array functions by incorporating operations from J, thereby improving Ruby's overall functionality.
- Learning New Paradigms: The talk ends on the note that programmers must embrace learning new languages and paradigms to enhance problem-solving skills and that concepts like APL's abstractions could eventually become mainstream.
Conclusions and Takeaways:
- Programmers frequently underestimate their ability to learn and apply new concepts, which limits their growth.
- The coding community should explore new paradigms for problem-solving, as they facilitate easier coding and lead to higher-quality solutions.
- Embracing diverse programming languages early can demystify complex concepts and foster broader adoption over time.
This presentation serves as an informative rallying cry for developers to embrace higher abstraction and more powerful programming languages, particularly those that might currently seem daunting or niche.
00:00:13.280
Nice being here with you guys today! I come from the U.S., and it's the middle of the night for me, so it's kind of like I'm awake.
00:00:18.830
But I wanted to talk to you today about something a little bit different. I'm going to have some Ruby code inside this presentation, but it's going to take me a little while to get there.
00:00:30.650
How many people here have ever programmed in the C programming language? Wow, almost everybody! That's incredible.
00:00:36.620
It's funny to me that in the industry, we often make assumptions about what people can and cannot do in terms of programming.
00:00:42.590
We assume that some things are extremely difficult while others are fairly easy, and I think this inhibits us greatly within the industry. As a result, we end up not writing programs that are as powerful as they could be.
00:01:00.080
Let's go ahead and take a look at something here. Do you guys understand what this code does? It's the Fibonacci sequence.
00:01:05.930
Now, everybody here who programs in C knows what this is, right? Even people who don't program in C have probably seen enough curly brace languages to get a sense of how this works.
00:01:19.219
What's funny to me about this is that if you try to show this code to a non-programmer, it looks like complete gibberish to them. They might look at you like you're a rocket scientist for being able to do this fascinating thing with computers.
00:01:26.119
If you try to explain to non-programmers how this works, you have to explain all these concepts, like a big sheet of memory that is broken down into cells.
00:01:34.310
Each variable has a certain number of blocks of memory associated with it, and they all have addresses.
00:01:39.469
You have to explain why there’s a flow of control from top to bottom, except when there are loops that make you jump back to the top. There are an incredible number of concepts you have to explain to non-programmers.
00:01:50.060
We kind of forget how much we really know about programming. All the sub-bases we've picked up, and we push ourselves to a point where we often feel stupid until we learn something we didn't know before.
00:02:01.130
This creates a shock for non-programmers when you're explaining these things to them. We forget how much abstraction we can currently handle, and I think this is a shame.
00:02:07.130
There are far more powerful modes of abstraction within the industry that people are not fully comfortable with, but we can definitely move towards them.
00:02:13.130
What about Lisp? Has anyone played with Clojure or Common Lisp? There’s been a resurgence of Lisp recently thanks to Clojure.
00:02:25.280
Over and over again, people in C, Ruby, and other languages decide to take the plunge into Lisp and find that it's way more powerful. They can articulate their solutions extremely concisely.
00:02:38.060
However, Lisp never became a mainstream language. Why not? Generally, two things deterred people: the parentheses and the belief that mainstream programmers couldn't handle it.
00:02:44.870
The key idea in functional programming is that you can pass functions to other functions, but people often thought that was too much for the average programmer to handle.
00:02:57.600
Look around at how many languages are just now adding closures and lambdas to them. C is adding closures, Java and C# have had them for a while, and these concepts are from the 1940s and 1950s.
00:03:06.100
We have this notion that these things are too powerful for the mainstream programmer to handle. In the 90s, there was an assumption that there was only so much that programmers could handle.
00:03:18.840
Alan Kay, who was the impetus behind the Smalltalk programming language, which is entirely object-oriented, came up with a lot of foundational concepts of object orientation.
00:03:29.690
Now we see objects everywhere. People in Java use objects all the time, even for mundane tasks. Back in the 90s, many procedural programmers thought transitioning to object orientation was painful.
00:03:38.060
In fairness, a number of older programmers struggled with this transition. However, the vast majority of people made the shift and now we’re seeing the same thing happening with functional programming.
00:03:44.870
Newer programmers are exploring languages like Haskell, Scala, and looking into functional programming constructs in Ruby. Some find it hard.
00:03:53.130
I wrote an observation once about what I called the era of patronizing language design in the 90s. You saw it everywhere, especially with Java.
00:04:02.000
Java could have been more powerful if not for the fear that mainstream programmers can’t handle certain things. Look at Ruby and all the metaprogramming we do; it’s taken for granted that we can modify classes at runtime.
00:04:14.240
In Java, reflection was created, but it was barely read-only. Programmers were terrified that allowing modifications at runtime would lead to an ugly mess.
00:04:24.240
We know that sort of thing happens, but it’s also self-correcting. Programmers do push the boundaries, but we have also learned to temper this in the Ruby community.
00:04:36.420
Gregor Kiczales came up with aspect-oriented programming with AspectJ, attempting to protect programmers from modifying classes. This was patronizing; the average programmer can handle these concepts.
00:04:50.560
Matz gave us all this capability in Ruby, and people started to play and experiment. Some overdid it, but most benefited and created powerful work that would have been impossible otherwise.
00:05:03.270
We shortchange ourselves when we restrict the learning and adopting of new paradigms. Studies of programming languages show that if you can make a program smaller, its quality improves.
00:05:17.160
Higher-level abstraction languages correlate with fewer bugs due to fewer lines of code. Working in Ruby means achieving what could take many lines in C or Java in far fewer lines.
00:05:29.170
This is a small program in Haskell that takes a set of strings and processes them. The powerful abstractions allow for concise and effective programming.
00:05:42.000
In Ruby, we can also accomplish tasks concisely. Here’s a Ruby program that retrieves specific data from an array and analyzes it.
00:05:54.560
When I look at this code, I realize if I were doing it in Java, it would take dozens of lines. I won't explain it now, but it's a functional style. Enumerable offers powerful functional abstractions.
00:06:06.780
Understanding methods like group_by will significantly change your programming life. We can program at much higher levels of abstraction in Ruby, leading to higher quality software.
00:06:14.440
Now let’s discuss APL. Has anyone ever heard of APL? It's one of the most maligned programming languages, designed by a mathematician in the '60s and '70s.
00:06:24.200
APL was originally a mathematical notation and many see its character set as unreadable. It's not entirely different from Perl in terms of readability.
00:06:42.400
However, I believe we’re seeing a trend in the industry toward more concise programming at higher abstraction levels. Raising the bar requires us to embrace conciseness.
00:06:55.900
APL, devised by Ken Iverson, works with elegant solutions. Here’s how concise a quicksort can be implemented.
00:07:10.080
Interestingly, some APL programs use array operations without needing to create variables. In functional programming, you often deal with actions without needing to define variables explicitly.
00:07:23.210
Now let's dive into J, another APL derivative. Here’s a factorial function in J that applies to a single number or an array.
00:07:36.580
This operational flexibility illustrates how everything can function in multidimensional arrays. Operators apply regardless of the data's dimensionality.
00:07:43.420
Here’s how you create and manipulate data in J. I can fill a 2x2 matrix, a 2x3 matrix, or even a 2x2x2 cube effortlessly.
00:08:00.180
J allows for such operations flexibly; you can generate sequences with ease and create rich structures in memory.
00:08:12.200
Creating a 12-dimensional matrix in J? It allows that too! The language is powerful in its flexibility, enabling matrix summation seamlessly.
00:08:26.900
J integrates summation and array manipulation effortlessly. When summing through a 3x3 matrix, it reduces dimensions naturally.
00:08:39.240
The idea of composing simple operations into more complex structures is a powerful one, enabling programmers to work efficiently.
00:08:55.060
An operation in J can give you the position of each element if sorted, which is a bit complicated, but allows for meaningful abstraction.
00:09:19.750
This compositionality facilitates the writing of concise programs to solve complex problems effectively, just like how it is done with Conway’s Life.
00:09:31.250
This demonstrates the beauty of writing concise code. It fosters a deeper understanding of programming concepts.
00:09:47.100
I’m working on a little gem called Jop, which extends Ruby's array capabilities by incorporating some operations from J.
00:10:03.570
This gem aims to provide powerful functionalities like selecting elements based on given positions, enhancing the overall experience.
00:10:19.580
There are several powerful operations within Ruby, often taken for granted, and the goal is to simplify the coding process.
00:10:33.020
Remember earlier the construction of multi-dimensional arrays? In Ruby, one can cycle through sequences automatically.
00:10:44.290
If the sequence is less than the number of elements, it automatically repeats; this demonstrates Ruby's functionalities beautifully.
00:10:59.910
There are many resources for learning J. It’s a unique toolset that can make you a powerful programmer when utilized effectively.
00:11:12.820
I believe there's a legacy among APL derivatives, and using different characters can be off-putting.
00:11:25.180
Just like we see in programming today, where languages become approachable, it's important to utilize powerful array operations.
00:11:38.350
Learning and contributing to these languages can truly enhance your skillset. In this programming landscape, being polyglot is increasingly valuable.
00:12:00.980
In the industry, people have used high-level languages for a variety of tasks. The fear surrounding new languages often dissipates over time.
00:12:12.440
To expand our cognitive toolkit, it's crucial to learn new languages, creating a robust skill set that allows creative problem solving.
00:12:26.960
Just like learning calculus at an early age, programming can be made less intimidating through foundational knowledge.
00:12:39.600
If kids learn programming paradigms like Haskell or APL early, they won't find them scary, thus paving the way for broader adoption.
00:12:54.950
I wouldn't be surprised to see APL derivatives and similar paradigms become more mainstream as time goes on.
00:13:07.200
In conclusion, we often underestimate our capabilities to learn new things and think abstractly, which is vital for growth.
00:13:21.650
Thank you for your attention!
00:13:27.740
Questions?
00:13:34.090
It seems everyone is a bit stunned by the information.
00:13:39.809
Thank you for the presentation. My first thought is about whether these operations can be done in parallel.
00:13:48.549
There's a lot of opportunities for parallelization in the context of financial computations and heavy mathematics.
00:14:04.989
Processing large arrays on graphics cards is possible and has been utilized in various niche areas, optimizing rotation and index manipulations.
00:14:20.969
These languages have been tweaked to work efficiently without the need for frequent copying of large data sets. It’s about how we think about these operations.
00:14:36.180
As programming evolves, we develop better ways to handle data without stifling performance. The ease of learning remains paramount.
00:14:52.400
Debugging concise code raises concern about maintainability, especially if it's all folded into a single line.
00:15:03.310
However, with a solid understanding of how functions interact and compose, testing can be effectively managed.
00:15:17.450
My experience shows that because of the increasing familiarity with various programming paradigms, debugging may not be as cumbersome as it seems.
00:15:33.780
The industry is getting used to diverse programming styles, much like we adapted to object-oriented design principles.
00:15:47.200
Each language has a unique learning curve, but as programmers, mastery of multiple syntaxes and abstractions makes us versatile and valuable.
00:16:02.450
Writing concise code can help simplify complex tasks. Domain-specific languages have emerged as a solution that may help with various concerns.
00:16:17.250
However, care must be taken in ensuring DSLs genuinely add value over traditional languages and are not a source of unnecessary complexity.
00:16:34.360
The use of a DSL in stable domains proves beneficial; we can fine-tune solutions that promote reusability and understanding.
00:16:48.680
Creating specific languages suited for unique situations can lead to efficiency.
00:17:03.600
Programming languages often reflect the underlying methodologies with which they were designed. A good design can foster a successful DSL.
00:17:18.500
While languages like Lisp or J deal with numbers, exploring new paradigms like set or tree structures could provide fresh insights.
00:17:34.510
These discussions elevate the understanding of programming concepts, pushing the boundaries of what languages can or cannot do.
00:17:48.020
Thank you for your engagement, and I look forward to further discussions in the future.
Explore all talks recorded at wroc_love.rb 2014
+13