Elixir: Power of Erlang, Joy of Ruby

LoneStarRuby Conf 2013

Dave Thomas

@pragdave

#elixir

#functional-programming

#software-development

Elixir: Power of Erlang, Joy of Ruby

by Dave Thomas

The video titled "Elixir: Power of Erlang, Joy of Ruby" features Dave Thomas, a prominent figure in the Ruby programming community, discussing the Elixir programming language. Thomas starts by expressing his long-standing love for Ruby, which he discovered in 1998, and emphasizes the importance of staying updated with evolving programming trends. He introduces Elixir as a modern alternative that combines elements of functional programming and Ruby-like syntax while running on the Erlang VM.

The key points of the presentation include:

- Evolution of Programming: Thomas discusses the shift towards functional and concurrent programming paradigms, highlighting the limitations of traditional software development methods.

- Moore's Law: He explains Moore's Law and its relevance to the increasing complexity of software due to the growing number of transistors in modern processors, necessitating new programming approaches.
- Functional Programming: Elixir is showcased as a functional programming language that offers robust concurrency. Thomas illustrates core concepts such as pattern matching, recursion, and declarative definitions, showcasing their efficiency in managing data and eliminating explicit loops.
- Hands-On Coding: Throughout the presentation, Thomas provides live coding examples to demonstrate Elixir's syntax and features, comparing them with Ruby. He writes functions for common tasks like calculating the length of lists and generating Fibonacci numbers using recursion and pattern matching.
- Concurrency in Elixir: Thomas highlights Elixir's ability to handle multiple processes concurrently, sharing a coding exercise that implements a parallel map function (pmap) that allows the execution of functions across multiple processes efficiently.
- Testing: He touches on Elixir’s ExUnit testing framework, showcasing how to write tests that confirm the correctness of functions, further emphasizing Elixir's structured and organized approach to code development.

In conclusion, Thomas advocates for adopting Elixir into developers' toolkits, encouraging Ruby developers to explore its functionalities alongside Ruby rather than as a replacement. He emphasizes the language's potential for writing highly reliable, scalable, and performant server applications while being enjoyable to use. Thomas concludes with an enthusiastic endorsement, expressing his excitement about the future possibilities with Elixir.

00:00:24.880 If you could everyone, come up here and move your arms around a bit. It's great!

00:00:30.560 Let me start off by explaining something: I love Ruby. I've been working with Ruby since 1998. I've revised the Pickaxe book four times, and in fact, the latest revision came off the presses just last week. There are actually four copies lying around somewhere. So, I have a commitment to Ruby—don't get me wrong.

00:00:45.520 I'm not here to say that all you Ruby programmers shouldn’t be here or that it's not worthwhile. That's not the case.

00:00:58.480 However, I also think that we owe it to ourselves, all of us, to keep an eye on what's going on in the programming landscape. If we don't, as Sandy and I were discussing over lunch, we're eventually going to be out of date.

00:01:10.560 How many people here used to go, or still go, to the No Fluff Just Stuff conferences? A few? Not many? I'm actually kind of proud of the fact that I've been banned from speaking at those, because I used to attend a set of Java conferences and talk about Ruby. They said, "Oh, you can't do that anymore; go talk about Java." But I told them, "No, I'm not going to talk about Java," so here I am at a Ruby conference talking about Elixir.

00:01:33.040 Why am I talking about Elixir? To me, Elixir is like a gateway from what I used to do to what I will be doing. I've been looking for this for 10 years, and I finally found it.

00:01:50.560 The reality is that the future is functional, and the future is concurrent. These two technologies combined are pretty much the only way we can continue to develop software effectively. We're running into all sorts of obstacles in traditional software development.

00:02:05.280 How many times have you heard someone say, "Oh, that’s not going to work"? We want to achieve 100% reliability.

00:02:21.440 For the longest time, I couldn't figure out what he had to do with it, but you know the saying that every year you get twice as ugly? I don't know why that came to mind, but does he—or does he not—look like the Wendy’s Dave Thomas guy?

00:02:44.320 Anyway, the reason he's cheery is because he's worth four billion dollars! He came up with the infamous Moore's Law. Now, Moore's Law is not actually what most people claim it to be. Moore's Law says that the most cost-effective density for putting transistors onto a silicon chip doubles every two years.

00:03:01.600 It's nothing to do with performance or anything else; it just states that Moore believed he could pack twice as many transistors onto a chip and hit the optimal point for cost-effectiveness.

00:03:17.040 I was trying to remember the first microprocessor I ever used, and it was definitely the 8008. I also used the 6502, which is on the lower end of the graph here, both of which had maybe 3,000 transistors.

00:03:38.720 The y-axis on this graph is exponential, just as it was in Sandy's graph on books.

00:03:54.560 Nowadays, my laptop has a Core i7 processor in it, which is unbelievable. Just last night, when I got these numbers, I couldn't believe that my laptop has roughly a billion transistors in this processor.

00:04:11.360 That’s a billion transistors! Some guys actually measured the size of the processor chip, because Intel doesn’t release those specs, and worked out that it’s about 121 square millimeters.

00:04:27.520 That means I have roughly eight and a quarter million transistors per square millimeter in my processor. That’s almost unimaginable! If you want to know how big a square millimeter is, I found a use for this: If you take the cross-section of a typical paper clip, it’s about a square millimeter.

00:04:43.840 That means in that little space, there are 8.25 million transistors.

00:05:06.720 But what that really means in practice is that you can't use a billion transistors the same way you would use ten thousand. With ten thousand transistors, you can coordinate them and have them all working on the same problem at the same time. However, with a billion transistors, that's not possible.

00:05:23.040 The speed of light gets in the way of coordinating all of them. This is why we're moving more towards multi-core and hyper-threading inside the cores to give us parallelism in our processes.

00:05:40.960 We've kind of hoped the problem would go away because right now we can sort of get away with writing our Rails code on our servers and running multiple processes, and it kind of works. But it kind of doesn't.

00:05:58.960 You all know the issues we face when we try to run that way.

00:06:10.240 So for those reasons, I think we're looking at a sea change. In order to continue to exploit the growing curve of Moore's Law, we're going to have to find new ways of writing software. I believe that functional programming is one of those new ways.

00:06:27.520 Elixir, I think, is functional, concurrent, pragmatic, and fun. Let's spend a bit of time looking at the functional aspects.

00:06:37.600 Initially, I had, like, I don't know, a hundred slides or something, but I thought, screw that; let me just code instead. So I'm just going to code some stuff up, and you will get to see how bad a typist I am. Together, we may learn something.

00:06:54.720 The first thing I'm going to do is actually go into the shell, which in Elixir is just like IRB. It’s called IEx. Well, I say it’s like IRB, but it’s nothing like IRB.

00:07:11.680 Right now, I am orchestrating roughly 25 independent processes in my Elixir shell here. The compiler is running in a separate process, and the IEx command line is running in another process. They’re all communicating back and forth, which is kind of cool.

00:07:30.080 It’s totally transparent. Just like with IRB, I can do things like type an expression, and it works. If I assign a value to a variable, for example, when I say a = 1, I'm not really assigning to a variable at all.

00:07:47.760 In a functional programming language, all I'm doing is asserting that a and 1 both have the same value. Because of this, I can also say 1 = a, and it will respond with 'Yep, no problem at all.'

00:08:02.560 However, if I say 2 = a, that now gives me something called a 'no match error.' The reason for this is that Elixir will only bind new values to variables if they’re on the left-hand side of an equal sign.

00:08:20.960 Since a already has the value of 1, Elixir states that whatever's over on the left must also equal 1.

00:08:38.320 Now, I’ve talked about pattern matching. This is an absolutely critical aspect of any functional programming language. Let's have a quick look at pattern matching; we’ve already seen the case of a = 1.

00:08:58.320 Here we have a pattern where some variable is equated with a scalar, which will assign the variable to have the same value as that scalar.

00:09:17.520 I could also use an Elixir list instead. Now my variable a has been associated with that list [1, 2, 3]. But pattern matching goes quite a bit deeper than that. I can say, define d, e, f equal to a. What this does is say: on both sides, I'm expecting to see a list of three elements.

00:09:37.760 As luck would have it, a is indeed a list of three elements, so this will match and assign 1 to d, 2 to e, and 3 to f.

00:09:58.720 We can take this further; I can take a list like a and split it into its head and its tail. I could say: head followed by tail in a list equals a, and if I do that, then the head will be 1, and the tail will contain the rest of the list.

00:10:14.560 So, pattern matching allows me to construct and deconstruct my data, and as you will see later, it also allows me to write larger programs without any conditional logic.

00:10:34.560 Typically, that's also done without any loops. All I'm doing in my code is simply declaring stuff.

00:10:50.160 Let's start off by writing some Elixir code. I'm going to use Sublime Text and write a quick bit of Elixir, defining a module called 'sequence,' because eventually it will be... oops!

00:11:06.240 So all Elixir code goes into a module. This isn't strictly true; however, when you want to compile Elixir code, you do need to put it into a module.

00:11:18.480 That's because Elixir runs on the Erlang Virtual Machine, and that’s how the Erlang VM manages its code. You may have heard that Erlang allows you to do hot swapping of code; well, so does Elixir.

00:11:37.920 In a running application, I can replace it at the module level, which is why I need to put my code in a module.

00:11:53.360 So, I can write a method called 'hello' and just say hi. Later down here, I can call it by saying sequence.hello.

00:12:06.640 If I run that code, wow! You would be hard-pressed to think that that was Ruby, right? It’s very, very similar.

00:12:21.440 In fact, Elixir takes a bunch of things from Ruby and runs with them. For example, the 'def' keyword looks like a function definition, which it kind of is, but actually it's also a function call.

00:12:38.720 In Elixir, 'def' is a function call, just like 'defmodule.' This function takes two parameters: the name of the method and the body of the method.

00:12:53.760 It just so happens that in Elixir, there’s a bit of syntactic sugar that lets you write the body as 'do ... end.' Underneath, that actually gets translated into something else.

00:13:06.960 The 'do' in the function is just like a Ruby hash, a named parameter that I'm passing in. For example, an if statement in Elixir—if I write 'if a equals b, do something, else do something else'—is also just syntactic sugar.

00:13:25.760 Because 'if' is also just a method—actually, it's a macro—but effectively, it’s a method that takes three parameters: the expression, a 'do' block, and an 'else' block.

00:13:42.080 And that means I can rewrite the entire language if I want to. Everything is soft; when you talk about meta programming, this is serious meta programming.

00:13:59.280 Unlike other languages you might name, it's done in a very controlled style. First of all, meta programming in Elixir is done by manipulating the parse tree, which happens to be an Elixir data structure.

00:14:19.200 It's done in a hygienic way, so it will not overwrite any of your local values accidentally, and it’s done in a scoped way.

00:14:36.160 In Elixir, every change you make to the environment, whether defining macros or including a module, is lexically scoped.

00:14:51.200 So for example, I can say, "Apply this different version of a particular operator only to the parameters of this method call." That can come in very handy.

00:15:05.680 However, that's not why we're here. Let's have a look at something slightly different—let's explore lists.

00:15:20.800 I want to be able to say something like: IO.puts sequence.length of some list.

00:15:35.520 There are built-in ways of doing this, but let’s do it the hard way. I need to define a method called 'length' that’s going to give me the length of the list.

00:15:51.520 In a functional programming language, I'm going to use pattern matching to do most of the work. So, the length of an empty list can be defined using pattern matching: an empty list will equal 0.

00:16:06.720 That takes care of one case. The other case is for a non-empty list with a head and a tail, where the tail could be empty. We know the head contributes 1, and the length of the tail is simply the length of the tail.

00:16:24.239 Does this make sense? It’s a nice little recursive definition.

00:16:36.960 If I run that, it blows up, but that’s okay. Oh, kernel.length conflicts with a local variable. Look at that; although it’s untyped, it does a fair amount of work to ensure that, for example, I haven't misspelled function names or accidentally overwritten function names.

00:16:53.440 So, let’s call that 'len' instead of 'length.' There we go.

00:17:09.440 A couple of things here: first of all, there's my result: 5. Notice the variable head is unused; because when I did the pattern match, I set head to be the head of the list and tail to be the tail of the list, but in my code, I only ever use tail.

00:17:23.520 Elixir has a convention that if I put an underscore in front of a variable name, it says, 'I don't plan to use this variable.' By doing that, my warning goes away.

00:17:38.880 I could, in fact, just say underscore here, and that just works, but I personally prefer to leave the variable name because it documents what I’m doing.

00:17:55.440 So, that’s a typical recursive definition. You don’t find yourself doing those too often. One thing that I like about this from an aesthetic point of view is that my definitions are pretty much the specification of what I want to write.

00:18:12.960 For example, 'The length of an empty list is zero.' Someone could tell you that, right? And 'The length of any other list is one plus the length of the rest of the list.' That’s a specification.

00:18:27.200 Let me show you another specification that’s useful for my purposes: Fibonacci numbers—1, 1, 2, 3, 5, 8, etc. Each number is the sum of the previous two numbers.

00:18:43.760 If you go to a math book or Wikipedia, they will tell you the definition of the Fibonacci function: the Fibonacci of 0 is 1, the Fibonacci of 1 is 1, and the Fibonacci of any number n is the Fibonacci of n minus 1 plus the Fibonacci of n minus 2.

00:19:02.080 That is the mathematical definition of Fibonacci.

00:19:18.560 But to convert that into running Elixir code, all I have to do is write it like this. Now, if I say sequence.fib of 20... there it is! Is that cool?

00:19:34.080 Now at this point, you're probably saying, 'But Dave, that’s all very well, but how many times a day do you actually calculate the Fibonacci series?' Okay, no more than five or six times, I have to tell you.

00:19:48.160 But that's okay, because the reality is that this form of expressing code I do all the time. In fact, when I'm writing Ruby code, I typically work out the various conditions and write them down; not as tidily as this, but I'll write things like '0 is 1' or 'an empty request means I send back a four.'

00:20:05.920 I do all those kinds of things in Ruby, and I do the same in Elixir. But it just turns out that coincidentally, I'm writing code. That's really cool!

00:20:21.840 Let’s look at a few more examples. First of all, let’s define a map function, which is similar to the Ruby map function: I want to take a collection and apply a function to each element in that collection.

00:20:38.720 So I'm going to start off with the use, because that's always a good way to do things. So in this case, I will do IO.puts sequence.map.

00:20:53.440 Okay, so I’m going to map through a list: let’s say [4, 5, 6, 7], and I'm going to map it through my Fibonacci function.

00:21:09.440 My Fibonacci sequence is going to be... Oh, I should do it this way.

00:21:27.920 Let's say I want to define a function that takes a number n, and then its body is to calculate sequence.fib of n.

00:21:46.480 So my map function needs a collection and a function. What are my special cases here? I think it’s the same as the 'length' example we did: if I want to map over an empty collection, then the result will also be an empty collection.

00:22:02.560 Now, we’re dealing with the case of mapping over a non-empty collection, so we’ll do that head-tail thing again.

00:22:19.440 We get the function, apply it to the head of the list because that's part of what I need to do, and then I have to generate the rest of the list that has been mapped.

00:22:36.880 I can express that by saying 'fun.head,' so that’s calling the function on the head, then I'm going to build the rest of the list by mapping on the tail.

00:22:55.120 That map function is the same map function that I defined earlier, and what's going to happen is that each time around this recursion, it's going to take one extra element off the list.

00:23:09.920 It will convert that element by running it through the function and build a new list where that element is included. Eventually, it will come to this clause and terminate the recursion.

00:23:27.440 In theory, if I run this code, it should output correctly. Okay, let me change this to be an inspect.

00:23:43.040 Alright, I just fell foul of some Erlang quirks. But anyway, there’s my mapped list. It’s really quite straightforward and very understandable code. It is recursive, but so what?

00:24:00.000 Here, I've written a map function and a Fibonacci function, and in both cases, I have no explicit loops and no explicit conditional logic. That’s fantastic!

00:24:18.720 So, you might say, 'But Dave, you’re doing testing by inspection. Can we do better than that?' You bet we can! So let's do something like test basic functions.

00:24:34.560 I need to use ExUnit, the testing framework, and I actually have to start the testing framework because remember about the multiple processes: it runs in its own separate process.

00:24:49.040 Now I can write some tests. Test basic Fibonacci function works. Alright, I should start my assertion.

00:25:05.840 Sequence.fib(10) should equal 10946.

00:25:20.720 And I’ll test basic map function to see if it works.

00:25:35.840 Now let's showcase a few examples here.

00:25:44.640 Now, you’ll see what happens when I run that.

00:26:00.920 Oh dear, I have made a typo; bad me.

00:26:09.840 So those are basic unit tests written using the ExUnit framework. I will see what happens when I run that.

00:26:25.920 It says function test2 undefined. I’m sorry, where do I get the function done?

00:26:36.640 Oh, I see; it’s not 'use ExUnit,' it’s 'use ExUnit.Case.' That’s why! Let’s run that.

00:26:53.600 Oh, it seems my test failed, and I'm expecting 5, 8, 13, but got 5, 8, 13, 21.

00:27:01.440 In most testing frameworks, I'd have to say assert equals, just to compare results, but in Elixir, here my test function intercepts calls to equals and inside that block.

00:27:16.320 It runs the original equals, but it still has access to the two operands as a code fragment. It can take that stuff, convert it back into text, and use it here.

00:27:30.560 That is pretty cool! Just think of the cool things you can do with that.

00:27:48.080 Now, let's push the boat out. I talked about it being a parallel language, a concurrent language. Let’s come back to our sequence map and see if we can make it parallel.

00:28:02.720 I have five minutes left, is that right? God help me!

00:28:20.040 I’m going to write a parallel version of map. Let’s call it pmap, which will take a collection and a function.

00:28:38.560 What it will do is this: I'll show you how I think about functional programming. It will take the collection, run each of the elements of this collection in a separate subprocess.

00:28:54.560 So, we're going to call it spawn children instead of function just to save myself a little typing. After we spawn each process, we’re going to collect the results.

00:29:10.560 What’s this pipe operator here? It’s like a pipeline, just like in a UNIX shell. I can pipe the output of one thing to the next. Here, I've taken my collection and I'm piping it into spawn children, and the results that come back I'm going to pipe to collect results.

00:29:25.920 This is significant not just for the saving in typing; it changes the way I think about programming. I think about programming as mapping from A to B rather than doing something.

00:29:43.120 So, I have a collection, and I'm going to map it into a set of processes, and then I’m going to map those processes into some results. It’s a transformation, not an explicit imperative way of coding.

00:30:00.080 How do we code those things up? I know I’m running out of time, so I'll do this rapidly. I have my spawn children function, and it will take a collection and a function.

00:30:17.120 It will take the collection and map it through a function called spawn a child, spawner for each child.

00:30:33.360 Clearly, I need to pass it the element from the collection and the function. So, I could do something like this: element, fun, and then wrap this in an anonymous function.

00:30:50.280 But because that’s such a common pattern, Elixir has a shortcut for this. I can map it through spawn children, where the first element is the first parameter passed in, and Elixir automatically wraps that into an anonymous function.

00:31:06.760 This anonymous function is nothing more than a call to a named function, and Elixir rewrites that into a direct function call.

00:31:21.840 What's my spawn child function going to look like? It receives a value and a function. This gets a little tricky because we're going to spawn ourselves and make ourselves run in a separate process.

00:31:39.960 To run a separate process in Elixir, I'll call spawn and choose the most verbose method, for clarity. I’m going to spawn a function in my current module called child.

00:31:55.520 I’ll pass three parameters: the value, the function, and myself, where 'self' refers to the current process.

00:32:11.920 The child will send me a message back to say, here’s the value. Now I need to write the child function. I actually like writing one-liners in Elixir.

00:32:27.840 My child method will receive a value, a function, and a pointer to self, which is actually the parent of the child.

00:32:43.600 It will send a message back to the parent using the Elixir arrow notation, indicating which message to send, which will include the PID and the function, as in: value = fun.(value).

00:33:00.120 Now back to the original pmap at the top: I've done the first part of spawning a separate process for each value in my collection, and it’s running independently.

00:33:16.880 Now I just have to collect the results back. I will define a collect results method that takes a set of process IDs. I need to receive the results in order.

00:33:31.520 I'm using a receive block. This means I will receive a message and use pattern matching to control what message I receive.

00:33:54.800 In this case, I want to receive the message from a specific PID. I haven’t defined that yet, but I will shortly.

00:34:07.280 My collect results method will take a PID as a parameter. I’m mapping through the PIDs and defining 'collect results for PID,' which takes a PID as a parameter.

00:34:22.440 I’ll run into any issues when I run this. If I run out of time or it's running for the first time, that seems unlikely.

00:34:35.920 Let’s run this and see what happens with any luck. So far, there are no errors, and pmap still functions correctly.

00:34:50.320 It seems to be working! Let’s change this to pmap instead of map.

00:35:07.520 So now it works cool just to ensure you see that clearly, and I know I'm going to get the hook.

00:35:24.320 I'm going to show you one more cool thing—if I take that file called, what an imaginative name that is, I have loaded it into my Elixir.

00:35:41.040 I can call sequence. I should import sequence to save myself typing.

00:35:56.560 So now I can say fib(30) and get a result. Now, if I run fib through a regular map, say for 30, let's see what's that?

00:36:12.560 It's giving me trouble now with larger numbers. When I run this through pmap, let’s see...

00:36:27.760 Now I’m using all of my CPU cores, and all of them are busy.

00:36:42.960 That’s cool! At this point, I could go on for days, but then they won’t let me come back.

00:37:02.080 So, I want to thank you all. I’m done! Thank you!

LoneStarRuby Conf 2013