00:00:20.480
Thanks, Mike, thank you for the invitation. I want to start with a picture from this building, which I have to say I love this venue, and especially this building.
00:00:25.720
Especially when it's not on fire! I think we were really dumb yesterday, so if it starts beeping, I will be leaving this time. If you have not done it, I suggest you walk up this round arch; you can walk all the way up to the top.
00:00:39.920
And on that walk, you will see a stair somewhere where it says 'the beginning'. I want the beginning of my talk to be the end of my last talk on a very similar topic.
00:01:01.640
That was an idea that I was proposing and that was the realization that every gem you start begins with a namespace, while when you develop a Rails app, you never start with a namespace.
00:01:15.400
So, if you compare the two, a gem is like a box with a little label on it, while a Rails app is, let's say, a gorgeous infinite nothingness, complete with all the bells and whistles of Active Everything.
00:01:28.520
To gauge how relevant that is, who writes Rails apps on a daily basis? That's about two-thirds, maybe three-quarters. And who writes Ruby but never touches Rails? That's way less, although there are still a few.
00:01:41.200
I hope that for the second group, what I'm going to say will apply a little bit less, possibly because of Rails not being around. The idea from my last talk was that the next time you start an app, just put everything in a namespace.
00:02:05.000
I ended with what I found to be an awesome finishing slide: 'Give yourself a box so you can start thinking outside of it.' The idea here is that if you have a gem and you give something a name, putting things inside of it, you can open that box.
00:02:24.239
You might go, 'What the heck?' when you find something that obviously doesn't fit in the box. If you try that with a Rails app, you don't have a box; you don't have a name, so everything's going to fit.
00:02:31.200
The best game you can play is, 'What doesn't belong with the others?' For example, I don't know, red, green, loud; what doesn't belong with the others? You're going to know it's loud.
00:02:48.000
But if you look at your typical Rails app, what are you going to have? You're going to have a user, I don’t know, company roles, maybe the pricing engine that you wrote, and then the marketing part for that. You still think you can play that game and find out what may or may not belong to this app? It’s very hard.
00:03:13.320
This situation pushes us toward just putting all of our app in there.
00:03:25.040
What I want to talk about today is component-based Ruby and Rails architectures. I promise you, I put this slide in before that happened.
00:03:31.040
If I may want this button more desperately than you, trust me. Please stop me if I'm going too fast.
00:03:37.360
I want to talk about large applications, and one thing I heard is that you just never build them.
00:03:50.040
I work for Pivotal Labs, and we may start a new project every few weeks. Many of them are greenfield, so many of them are tiny when we start. However, you heard from Matt yesterday that we have pretty big apps as well.
00:04:03.720
For every app, our definition of success is that a client can stay successful after we leave. Thus, we have to strive for some sort of architecture that allows for application development over time.
00:04:18.560
We need to think about the app being larger than it is while we develop it. Despite TDD and all, Sandy Metz wrote her awesome book last year and gave a few talks.
00:04:36.000
One thing she mentioned was that your app is out to kill you; the growing complexity is going to chase you. If the app were a bull, you might feel like that guy.
00:04:54.080
This is exciting for a second, but then it becomes painful. Who has felt like their app was behaving like that bull? Okay, I have felt that way as well.
00:05:02.760
They’re typically not fast; quite the opposite, but they behave like something that is too big to handle.
00:05:11.759
So what I want to get to is for us to be more like this guy: calm, in control, with sunglasses and a hat.
00:05:17.080
But I don't really like bullfighting, so I actually want us to be this guy, just hanging out. I have to admit, I'm somewhat of a consultant, so I'm a little bit mercenary.
00:05:30.080
I want to go home at five on a Friday night. I think this guy is probably the one who's going to do that most successfully.
00:05:42.319
This never-build-large-apps is part of a larger quote from Justin Meyer, who stated the secret to building large apps is to never build a large app. Instead, break your application into small pieces.
00:05:55.560
Then assemble those testable bite-sized pieces into your big application.
00:06:06.919
I think he was talking about JavaScript; I'm talking about Ruby and Rails. You can tell from the font that you're supposed to read this slide.
00:06:20.360
So I'm going to use a tiny sample application throughout this talk.
00:06:27.800
If you cannot read this after all, just search for 'GitHub and the next big thing' and I think it comes up first. I promise this thing is tiny and it does only one thing.
00:06:40.840
It's an announcement page: you can announce whatever you have to announce and people can sign up for updates.
00:06:54.400
I actually don't know if I spell with two N's, and I don't know if my Twitter handle is only one. We'll see how often I tweet.
00:07:00.919
So the server thanks you for signing up. I just hit return, and when I made this sample app, I had to come up with a bit more than just that to actually have something to work with.
00:07:11.160
I realized that pressing return makes it really easy for me to do that many times, but of course I only want my email address registered once.
00:07:17.639
So I thought I'd give the server the ability to feedback the level of annoyance it feels while you're continuously pressing return.
00:07:24.560
At some point, it'll just freak out.
00:07:30.879
Ah, there's the freak-out picture. Let’s try again. The wireless is gone.
00:07:36.240
It's an awesome picture of a guy with a fantastic hairdo.
00:07:44.039
Okay, I need your help though. I need you to imagine something really big, because this app is tiny.
00:08:01.560
I'm going to make examples of refactorings or re-architecting that don’t make sense in that app. So bear with me and always imagine something very large.
00:08:09.720
I'm going to go through eight ways of architecting this application and I have roughly two minutes for each.
00:08:14.800
So let’s start with number one: a normal Rails application, where everything is in one place.
00:08:23.720
We have a controller here with two actions: 'new' and 'create'. Let's briefly see what happens.
00:08:30.800
It's no surprise; it's not a well-written method. John would refactor this rightly so. We’re trying to find the entry that you gave in this form.
00:08:43.760
If we find it, we update the tries on that entry and we put it into this annoyance meter, which will tell us how annoyed we are.
00:08:50.760
If we don’t find the entry, we're just going to say 'thanks for signing up', and there's some catching of unexpected cases, but that's pretty much what's happening in this controller.
00:09:01.200
Now, there’s a model backing this, called 'Entry', which has the email address and the number of tries. There's also the annoyance meter, which isn't really important, it just derives a new string from these counts.
00:09:17.320
So I suspect everyone here has seen this, and I assure you that in every one of these stages, you can run the tests, and if I didn't pick the wrong commit, they will pass.
00:09:36.000
I have tests, but I won't show them. I'll just be moving them around for everyone.
00:09:48.040
Now remember, I said that at the beginning of every Rails app, it’s kind of this infinite void. A good picture for that is probably a big dumb site.
00:10:02.880
I've already asked if you ever felt like the guy in front of the bull; maybe you also felt like the driver in that bulldozer.
00:10:09.399
The reason I'm saying this about such a tiny app is that there are two things in there that are now indistinguishable from 30,000 feet.
00:10:22.440
If you look at it, there's no real structure; it pretty much doesn't exist.
00:10:35.040
So for all intents and purposes, if you extrapolate these two classes to say 50 or 100, you have no clue who’s interacting with whom, what’s going on, and why.
00:10:49.680
Unfortunately, that might not look chaotic right now, but if I were to write 100 class names here, it would be chaotic—you wouldn't be able to assess any sort of structure.
00:11:08.160
So, a first way of getting structure into this application is through modules. The benefit we get from that is a higher-level structure.
00:11:19.600
Not much changes: the controller doesn’t change at all except it now references the 'Entry' and the 'AnnoyanceMeter' within their namespaces.
00:11:33.600
The annoyance meter class is now under a module. I've moved some other stuff around, mainly for testing.
00:11:49.920
So now we have this structure. If we think about the metaphor, we've moved from a dump site to a recycling yard.
00:12:02.079
Recycling yards are awesome; I should know because I remodeled my house and visited those places very often.
00:12:13.520
You can point to a corner of the recycling yard and say, 'That's where the scrap metal is.'
00:12:20.640
However, if you've ever gone to such a place, you've also seen people stick the hard-to-recycle stuff under their car.
00:12:35.360
They try to pawn it off to the recycling yard because they just want to get rid of it.
00:12:41.360
That’s indicative of a lack of guarantees regarding the independence of these classes and whether they're doing what they say.
00:12:56.360
So, at least I didn’t have to write 'entry' anymore because I now have a higher-level concept of 'email signup'.
00:13:09.600
The third step is what I want to call or improve on this: I already hinted at it by wanting to prove that these two pieces are independent.
00:13:21.720
I call it the gem component app. Now I may need to explain a bit more about this step.
00:13:31.680
First off, you're not seeing that engine's folder right there; we’re looking at the annoyance gem instead.
00:13:46.840
I made a folder called 'gems' and put an 'annoyance' folder there. This is a complete folder structure of a gem.
00:14:00.640
We have a gem spec that defines our annoyance gem.
00:14:11.120
There’s a gem file that tells us we’re running tests with RSpec, and magically, we don’t need to look at the tests now.
00:14:21.800
Now that I have tests in this gem, if I go into that subfolder, I can now run them independently.
00:14:33.760
Because of the way RSpec loads these files, I can prove to you that these tests for levels and for the meter of annoyance will pass without needing to know about email signup.
00:14:50.160
That matters a lot if you're dealing with tens or hundreds of files. I can prove that this thing is independent.
00:15:05.440
Now you might ask me two questions: one, why am I not using Git submodules? The answer is that I'm not smart enough. Two, why am I not using a different repository? The answer is I'm too lazy.
00:15:24.160
You can just do this.
00:15:32.720
How do I use it in the application? The gem file of the main application now points to the annoyance gem—it's referring to it by its path.
00:15:42.560
That’s all there is to it. This will be loaded and I can access it like before.
00:15:54.440
If I look into the teaser controller, we are still down here just loading that annoyance meter; nothing else special has happened.
00:16:01.640
The only thing that happened is that we now have provably independent tests, which is of great value.
00:16:08.840
In the words of Eric Evans, choose modules that tell the story of the system and contain a cohesive set of concepts.
00:16:21.679
We are now able to prove that a set of concepts, namely within that gem, is independent of the others.
00:16:32.000
I found something funny to put in this slide: it's good for push and pull to be so explicit, but if you look at the doors, it's not that explicit as to how to open them.
00:16:45.360
Now look at Icelandic signs; you know how to open these doors—they scream at you how to open them!
00:16:58.360
For the structure of the app, now there’s a second component, and it’s getting to be bold because it is provably independent.
00:17:09.999
We still have that email sign-up around, so we got a little more structure. Is that cool so far?
00:17:22.800
Let’s continue.
00:17:31.040
Now, gems make up a portion of apps, but if I have dependencies toward anything Rails, I have to do a lot of homework myself.
00:17:49.440
In the next step, I want to call the Rails component app. I’m still going to do that for a Rails component.
00:18:02.360
So, I want provable structure for Rails.
00:18:05.920
The way to do this—or more readily—is with Rails engines. If a Rails engine were actually a train engine, then Ruby on Rails applications would be trains.
00:18:22.760
I should clarify that engines still have this wrong perception of being for pagination, generic administration, and authentication.
00:18:38.200
They are not. If you look at the docs, it says Rails engines allow you to wrap a specific Rails application or subset of functionality and share it with other applications or within a larger packaged application.
00:19:01.080
I should know because I pushed that change last time I gave a talk like this. It’s great that it says that now.
00:19:15.320
So now, we have an engines folder in addition to the gems folder.
00:19:30.480
And you may have noticed that the app folder went away. Since we only had one controller, I just pulled something out.
00:19:41.120
There’s no longer a necessity for this app to contain code itself; it only contains gems.
00:19:55.320
Let's start with the more self-contained one: email signup. If we go in here, there is still that 'Entry', and it's now within a module.
00:20:05.280
This ‘Entry’ is now also an engine, which you can check out in the source code or find out more about it.
00:20:18.240
I can’t go into more detail now, but I’m happy to discuss that in the questions.
00:20:30.560
So essentially, this means I have my database migrations in here, I have tests for it, and it runs independently.
00:20:39.440
The main app, as before, just includes those gems and references them.
00:20:48.720
Let's quickly review the teaser, though, because the teaser now contains the controller that did before.
00:20:56.960
The teaser also contains asset controllers and views in my example; however, it does not contain models because it doesn't have any data itself.
00:21:04.960
The email sign up only has the model; it has only this entry.
00:21:14.160
The teaser must be getting that data from somewhere else, and indeed it requires email sign-up to be present.
00:21:25.040
It references that dependency in the gem spec so that it's a valid gem definition.
00:21:32.000
The controller hasn't changed much, though I made some modifications to the entry management.
00:21:39.640
You can find more about that in the code, but essentially, we're still just accessing the entry.
00:21:49.640
That's cool. I should probably look at the build scripts at this point or should have done that a while ago.
00:21:58.440
Since I have all these dependent components, I no longer just say 'rspec spec'. The top block just runs everything in the main app.
00:22:07.520
I think it just loads up the whole thing and sees that nothing breaks.
00:22:18.480
Then we go into the email signup engine and have model specs there, and then we go into the teaser engine.
00:22:39.880
We test the controllers, then have requests in the teaser request specs, Jasmine specs in the teaser, and still the test for the annoyance.
00:22:50.400
It's all in one. You could split this up; at this point, you can have about six parallel builds on Jenkins, all just a few seconds long.
00:23:00.560
The new structure is a shrinking Rails up top, a big teaser engine that controls what the app can do, and the annoyance gem.
00:23:10.640
Did anyone see a code smell around single responsibility in any of this?
00:23:20.000
I’ll give a hint: did anyone see a code smell around single responsibility? There is a thing called 'email signup entry'.
00:23:30.640
It contains the number of tries you took to enter your email. It’s not very visible, but if you think about what the output of such a table is when you finally publish your announcement.
00:23:46.920
What do you want to do? You want to send out emails to all the people. Why would this table need to know how many times someone tried to do that? It doesn't.
00:23:57.600
So this thing knows too much. Please bear with me and extrapolate to 100 files, and this issue will be everywhere. You will have a couple of god classes, and it will be hard to refactor them.
00:24:06.960
So we might as well do it now. Shoot, what was my number? Ah, number 5.
00:24:16.240
I want to get to the next point and make that problem go away. I want clearer responsibilities.
00:24:27.040
I will introduce what I’ve dubbed the event counter. Every time I try to sign up, it’s kind of an event.
00:24:38.200
However, because of my app's requirements, I'm not interested in seeing every event; I just want to count them.
00:24:46.160
So, I introduced an event counter engine in this case. It comes with a new class, event logger. It can log, using a generic object identifier and event identifier.
00:25:05.720
What you get back is a count of the number of tries you've already logged.
00:25:15.760
The teaser controller has changed a little bit; it injects another dependency down here. It now uses this event logger.
00:25:33.440
But other than that, you still see the code's essentially the same.
00:25:41.680
So I got rid of these two roles that the entry had. It didn’t make sense at this point, but it does make sense.
00:25:51.040
And if you think 'event counter' is kind of a stupid engine, imagine how Facebook is doing their likes: the event is just a status post.
00:26:09.360
The like is the action. Why not separate that into its own engine?
00:26:16.360
You can potentially use Redis within Postgres to make the storage and queries faster. If it doesn't need to be together, it shouldn’t be together.
00:26:28.480
So we might move toward a more loosely coupled architecture instead of the lousy coupling that was there before.
00:26:39.040
Now I want to come back to this logger class. I want to call this my sixth iteration on the architecture.
00:26:55.440
This iteration revolves around the service-oriented app.
00:27:03.840
I will not be talking about HTTP SOA; that’s point 8. This is just the service pattern.
00:27:14.840
You can find this in Eric Evans' book. The first thing you might notice is that there’s this active record class.
00:27:23.799
It's private, no it's not. You cannot make a class private like this. But, right now, that doesn’t matter.
00:27:36.160
What we're really offering here is what I want to call a service class: the logger that gives me one method.
00:27:46.040
You can log an event, and here’s the identifier. I want to do this to reduce surface area.
00:27:56.320
If you have not seen Justin C’s talk on mocking and testing, he makes good points about that.
00:28:06.920
Just some quick numbers: if you look at how many methods are on these two classes, the count and the logger, it’s way less on the logger.
00:28:17.440
If you subtract what an object has, then the logger only has one method; it screams at you what you should do with it.
00:28:30.560
I think this is especially important when extracting components and making them live on their own—they should state what they are about.
00:28:40.720
An active record class, I would argue, never states what it is about because you can always do so much stuff.
00:28:54.560
It’s very hard to prevent it. The structure doesn’t change, but we have just this event counter.
00:29:05.120
What I want to talk about is rather the next number: number 7.
00:29:13.440
You may have been wondering if I’m finally going to take one of these gems and rip it out.
00:29:24.640
In practice, I have done that but only after months on a project, when pretty much one project was done.
00:29:34.480
We realized, oh, that gem is actually one where we're interfacing with the main architecture, and it’s all repetitive stuff.
00:29:46.720
Then you're two steps away from that gem: build, gem, push, and you have a gem.
00:29:54.760
So it’s just the engine vanishing from the source code; it now lives in another directory.
00:30:05.680
I also pushed it to GitHub, and nothing else changed. Actually, one thing does change: the reference to this gem in the gem spec of the teaser now has a version.
00:30:13.440
In the gem file, there's no longer the reference to that path.
00:30:22.880
Again, there’s no change in structure, only that the middle point—the event counter—is now somewhere else.
00:30:31.640
This brings me to my last point: the HTTP SOA.
00:30:45.440
Marty Hot, who now organizes RailsConf, had this tweet about many talk submissions on how to split your mega Rails app into services.
00:31:00.200
Since my talk didn’t get accepted, I'm not going to talk you through it.
00:31:07.960
However, it's pretty clear what the next steps are. We already extracted an engine to handle the event counter.
00:31:15.720
What are you going to do next? You might take a tiny app like Sinatra, require that gem, and make one endpoint for logging.
00:31:27.480
Instead of in the teaser directly going to that event counter, call out to the service.
00:31:36.480
If it gets more complicated, you might want to revisit earlier talks to get your testing right.
00:31:47.600
This pattern, among all the structural architectural patterns that I've discussed today, I've only used once, and that was for auditing.
00:31:57.960
We knew this thing was always going to take all this data; it's never going to change.
00:32:10.360
We’re not interested in it while developing an app; you’ll be stuck maintaining multiple servers.
00:32:24.360
You will have to version stuff, but in the grand scheme of how to structure an application, this isn’t as special as many make it sound.
00:32:34.360
It’s just one more step that you can take, but by no means, you have to.
00:32:46.840
I've been running you through these eight different architectures for this tiny app.
00:32:59.680
And I've posed the HTTP SOA at the top of a staircase. What’s funny about reaching the top when it isn’t raining is that it looks pretty up there.
00:33:10.760
So again, you should definitely go up there. But the steps at the beginning and the end face in different directions.
00:33:18.800
I think this metaphor of a circle is more appropriate: as soon as I hit the next block and start a new app, I can start asking all these questions again.
00:33:28.800
I bet every one of you has touched one or pretty much all of these at some point in some application.
00:33:37.560
That's all I got. Thank you!
00:34:02.440
n
