00:00:07.359
Good afternoon, everyone! Is everyone awake? Getting your coffee in? Haven't crashed yet? Um, like Josh was saying, I'm Ben Smith, and I work for Pivotal Labs out of Boulder, Colorado.
00:00:15.200
Before I start this talk, I want to share a few observations I've made on this trip, especially regarding the differences I've seen between the US and Australia.
00:00:23.000
So, potato gems—does everyone know what this is? I don't know if we have these in the States. Well, we do, but they're called tater tots—those little fried bits of potatoes you eat in the morning.
00:00:34.559
It's a little weird that they're called gems, though, right? Because in order to consume a gem, you have to gem install it first.
00:00:41.800
And that's a little weird. We all know that you have to be careful when you gem install stuff because you could be running malicious code. Everyone knows that, right?
00:00:49.160
I think, in general, tater tots in America are a little bit more secure than potato gems, but Australia definitely is ahead of the US and other areas of security.
00:00:56.800
For example, your ATMs here actually have guards. I saw that at Manly Wharf yesterday, and I was worried he was going to steal my PIN.
00:01:06.160
But it turns out he's just protecting me from my ser... something. I don't know. But I'm not here to talk about security, Ruby gems, or even potato gems.
00:01:15.200
So, who here saw my talk last year? Nice! Good!
00:01:20.118
Last year, I talked about all that good stuff—security and gem stuff. I'm super happy to be back this year, and I'd like to thank everyone for having me back.
00:01:28.080
I feel great to be here again. I especially want to thank the organizers for making it possible for me to make the trip over. Thank you so much!
00:01:36.040
This year, I'm going to talk about architecture and Rails. You might say that Rails has its own architecture and that we don't have to worry about it too much.
00:01:42.200
Rails does use a design pattern that we all know and love—MVC. But once your app gets big enough, relying on just the Rails way of doing things might get you in trouble.
00:01:50.880
So I'm going to tell you a story about a Rails app that we tried our best to architect for success.
00:02:00.479
And what does success mean? What does it mean for a Rails app to be successful for a product or for a client?
00:02:08.000
Well, in this case, we knew our project would be a long one—it would last eight months or longer, which for Pivotal Labs standards is a big chunk of time.
00:02:17.400
The majority of our projects last about three to six months. We knew we'd have a large development team that would start off with just two Rails developers and then ramp up to ten, along with four iOS developers.
00:02:25.920
We anticipated that in eight months we would end up with a ton of code, given the number of developers we had.
00:02:35.600
This particular project was a from-scratch rebuild. The client had an existing codebase and toolset, and their goal was to replace all of that with a new one.
00:02:41.080
They also wanted to add some new features on top of it. We knew to get that existing feature set parity, we would end up with a considerable amount of code.
00:02:50.960
The client also emphasized the importance of scalability—not that it needed to handle thousands of requests per second on day one, but that we had it in mind.
00:02:58.480
At some point in the future, it could handle load like that. Lastly, the client wanted to replace their entire old codebase all at once—a big bang release.
00:03:05.640
This isn't something we normally do, but we worked with them hoping we could release a bit sooner than when it's all done.
00:03:13.560
So what does success look like in these terms? Normally, I'm all about doing the simplest thing possible, so I'd suggest you take this talk with a grain of salt.
00:03:21.919
What I'm going to show you is not a silver bullet; it will help you in some cases and only complicate things in others.
00:03:30.279
I'm also not an expert on this subject; I'm just some guy who spent some time trying some things, and I'll talk about them today.
00:03:38.480
So success with these constraints meant we wanted to think about architecture from day one, and our first decision was to use Rails engines.
00:03:44.840
We decided to use Rails engines, but not in the normal way they're usually used.
00:03:51.560
Who knows what a Rails engine is to start? Okay, a good number of people. A quick intro to engines.
00:03:58.639
Engines allow you to take an entire Rails app—so models, controllers, CSS, JavaScript—everything, bundle it up, and you can require it as a dependency.
00:04:05.840
You can potentially turn it into a gem that you can reuse in other apps. For example, Devise is a great example of an engine.
00:04:12.679
Who knows what Devise is? Has everyone heard of that great authentication engine?
00:04:18.679
The engines we created for this project were a little different; they were never built into gems and were never pushed to Ruby Gems.
00:04:27.159
Not only that, they never even had their own Git repository. All the engines we created went under our normal Rails sorcery.
00:04:34.360
This meant that for all you crazy SOA advocates, we didn't have to worry about versioning interfaces or separate deployments.
00:04:40.760
All the engines I'm going to show you today lived inside of a single Rails app, and they were deployed and versioned together.
00:04:47.240
Of course, you might ask why we used engines in the first place. We decided early on that there were a few distinct components we could separate out into their own engines.
00:04:55.680
But let me give you a little background first, so you can understand.
00:05:02.000
The product itself was a content publishing platform for things like video, audio, blog posts, tweets, Facebook posts, SMS—everything you can think of.
00:05:08.440
On top of that, it included a full social network with all the bells and whistles.
00:05:14.479
The content was consumed via an iOS app fed by a JSON API, and the content was created and moderated via a web interface.
00:05:20.020
So that doesn't sound too complex, but we're focused on architecture.
00:05:26.279
We took our simple Rails app and started to break it apart, and in this diagram, you can see we have our Rails app.
00:05:32.320
Inside it, we have four engines, and the arrows point in the direction of the dependencies.
00:05:40.240
I'll refer to this main Rails app that includes the engines as the wrapper Rails app from here on out.
00:05:46.439
On the left there, we have an admin engine—that's where the content was created, such as videos uploaded and blog posts written.
00:05:53.760
On the right side, we have the social network. This is the user-facing content in the form of a JSON API consumed by our iOS app.
00:06:01.160
We also created a common engine, where we put things like users and roles required by both the admin and social network engines.
00:06:08.240
At the top, we have another engine called 'Theer,' responsible for publishing this content.
00:06:15.400
So the admin could create content, and it would get copied over to the social network or posted out to a platform like Twitter whenever it was scheduled to go live.
00:06:22.240
Going forward, I'm not going to draw this box around all the engines; just assume that all the engines I show you are within a single Rails app.
00:06:31.640
Remember, the arrows point in the direction of the dependencies.
00:06:38.920
Theer depends upon the admin engine, the common engine, and the social network, while the social network just depends upon the common engine.
00:06:46.600
And common doesn't depend upon anything. Does this make sense to everyone?
00:06:54.600
It's about to get more complicated.
00:07:02.400
So the next thing we did was write a gem—not an engine, but a gem—to wrap an SMS service called Mobile Compass.
00:07:10.960
We chose to make this a gem instead of an engine because it didn't need a database connection or the asset pipeline.
00:07:18.560
I made Mobile Compass here red so you can see it's not an engine like the rest, just a gem.
00:07:27.040
In this architecture, we wrote quite a bit of code, but as we got further into building this application, we identified specific users.
00:07:34.400
We realized we could create their own engines for them. Specifically, we found there were more than one type of admin.
00:07:42.480
There was a global admin who would do things like create users and a content admin whose sole purpose was to create content, like writing blog posts.
00:07:51.040
So we broke these two out into separate engines, and they both depended on an admin assets engine.
00:07:58.480
Admin assets contained things like layouts and styles that both admin engines used.
00:08:05.920
Once we realized it made sense to split out engines based on roles in our system, we created another admin engine for social admins.
00:08:12.640
Social admins were moderators of wall posts, status updates, or any other user-generated content.
00:08:19.120
We felt pretty good about this; we were identifying engines and keeping our app broken up into smaller pieces.
00:08:26.239
Then we had a breakthrough. It actually wasn't us; it was our office director who walked by.
00:08:33.560
He said, 'Your social admin engine depends upon too much.' It doesn't need a dependency on things like controllers and the social network engine.
00:08:40.720
All it needs are the models. Now remember, the social network engine is this user-facing JSON API.
00:08:48.760
The social admin is an admin web interface to moderate things like wall posts and status updates.
00:08:56.160
The models for wall posts and status updates live inside of the social network. It took me a while to come around to this idea, but we did have a point.
00:09:04.960
The social admin engine depended upon all of the social network, meaning it had references to things like controllers or views, which just aren't needed.
00:09:12.560
And keeping your dependencies to a minimum is always a good idea. What we really wanted was to just depend on the models of the social network.
00:09:20.160
This would allow us to create controllers and views to moderate things like wall posts and status updates.
00:09:27.680
So we went back to our engines and did a refactoring.
00:09:32.480
We pulled the models out of the social network and put them into an engine called social network content.
00:09:40.120
We renamed social network to social network API because now it truly was just an API.
00:09:46.920
It contained controllers and presenters, but it didn't contain any models or domain objects.
00:09:54.640
Finally, we added a dependency from the social admin to social network content, which going back to the original problem allows the social admin to only require what it needs.
00:10:01.440
Instead of having this, we now have something that looks like this, addressing our original problem of requiring too much.
00:10:09.840
We thought, 'Why stop there?' We also had the same problem in our content admin engine.
00:10:17.280
It was requiring all of the content admin engine, but all it really needed was the models inside of it.
00:10:24.080
So we split out the content admin engine into content admin elements.
00:10:31.040
We updated our dependencies, and at this point, we started to notice a pattern.
00:10:38.080
Here we had some engines that could respond to HTTP requests—engines with routes and controllers.
00:10:45.400
And then we had other engines that only contained models or business objects.
00:10:52.680
So we started calling them web engines and domain engines.
00:10:59.680
Here, I've color-coded the web engines in blue and the domain engines in brown.
00:11:06.760
As a team, we found that identifying these patterns and putting names to them was really helpful.
00:11:12.640
Once you identify a pattern and name it, you can think in terms of what makes sense for that given pattern.
00:11:19.680
For example, if someone told me to add a users controller to our common engine, I could say no; that doesn't make sense.
00:11:26.040
This is a domain engine, so it shouldn't have controllers.
00:11:32.760
Speaking of an engine called common and patterns, we realized that having something called common is a bad idea.
00:11:39.040
Common means you can put whatever you want in there, like the lib folder in Rails.
00:11:46.080
So we renamed it to what we wanted it to be called—Users.
00:11:53.520
Once we renamed it, we realized there was stuff in there that shouldn't be in there—specifically, a profanity filter.
00:12:00.880
So we pulled that out into an engine. From here, we did some more work and more work.
00:12:06.880
This is where we ended up after about six months of development.
00:12:12.720
We kept on working on the app, but it just keeps getting more complex and crazier.
00:12:19.440
So I'm just going to stop here. How does this look? The first word that comes to mind when drawing this diagram is 'hell.'
00:12:27.200
Drawing a diagram like this makes the app seem crazy, right? It feels like it would be impossible to work in.
00:12:33.440
But here's the hard part to understand: the complexity is not because of the engines themselves.
00:12:39.960
The complexity is part of the domain of this application. We would have the same dependencies and the same crazy mess if we didn't have engines.
00:12:47.480
But without engines, we wouldn't be able to talk about them or see them visually.
00:12:57.560
In reality, you really don't need to know that entire graph by heart; you only need to focus and understand a subset of it.
00:13:05.720
There's also a general pattern to these engines, and that general pattern is that web engines depend on domain engines, which depend on the database.
00:13:14.280
If you think about it, that's not far off from your standard Rails stack that has controllers and views depending on models, which depend on the database.
00:13:20.520
So as crazy and scary as this looks, it's not far off from your standard Rails stack.
00:13:27.840
If you don't believe me, here's my Twitter handle in the bottom left corner. Go ahead and tweet at me, echo me on the internet.
00:13:34.520
So here’s the craziness of engines. Now you're probably asking yourself, 'How do I get there? This looks great!'
00:13:43.239
I'm going to show you how to get started with engines really quickly. Don't worry, all these slides will be posted online, so you don't have to write this down.
00:13:51.280
This is how you create a Rails engine: you run this command from your Rails app. This is how you require it in your Gem file.
00:14:00.520
You can require one engine from another engine, or you can require an engine in your wrapper app.
00:14:07.120
Then, if it has routes and controllers, this is how you mount it.
00:14:13.000
If you want to create a gem instead of an engine, it's as easy as this—same thing to require it.
00:14:19.800
The next step after you start creating these engines and gems is to run this once and create an engine template.
00:14:27.440
Once you create an engine template, all you need to do is copy and paste that engine template anytime you want to create a new engine.
00:14:33.920
Do a global find and replace on a couple of strings, rename a few files—ideally in some sort of script that you write.
00:14:40.680
This gives you a standard engine template to build off of, so if you want some standard basic dependencies or setup in your template, you can do that.
00:14:48.680
For example, if you want to standardize on using RSpec or HAML, you could do that in your template.
00:14:55.520
So that any engine you create from here on out will have these libraries included.
00:15:02.920
The same thing applies to plain old gems—you want to create a template to build off of.
00:15:10.560
The point here is to really lower the bar to create a new engine or a new gem.
00:15:17.680
You want to get to the point where it only takes a couple of minutes to create a new engine, and it cannot be screwed up.
00:15:25.000
We found that it was easier to take two pieces of code and merge them together later rather than to have one big chunk.
00:15:33.160
If you start with code separately and then have to merge it together later, you're going to be in a better spot in the long haul.
00:15:41.080
So, got all that? Makes sense? All the slides will be posted online. Let's get back to the interesting stuff—doing engines the right way.
00:15:56.720
From the beginning, we decided that engines should be completely self-contained.
00:16:02.840
For us, this meant database table namespacing, putting the migrations inside the engines themselves, and having all the tests contained within each respective engine.
00:16:09.000
Let’s talk about database table namespacing.
00:16:15.440
With this architecture, we wanted clear and real boundaries—almost as real as if you were doing full SOA.
00:16:23.079
So if you took our simple Week One architecture and converted it to SOA, it might look something like this.
00:16:30.520
This is what we wanted to mimic: messages over HTTP, separate databases, different Rails apps.
00:16:36.320
But we wanted things simple and easy, so we wanted one database, messages in Ruby, and engines instead of apps.
00:16:44.440
We took our engines with their single database and chose to namespace all the tables in that database.
00:16:52.000
So each table is prepended with the name of the engine it belongs to.
00:17:00.479
Moving from this to this would mean putting each engine in its own wrapper Rails app, breaking out the database tables by namespace.
00:17:08.479
And we added a small wrapper so you could send HTTP messages rather than Ruby messages.
00:17:16.880
The other cool thing about being able to pull out an engine into its own app like this is that you could actually scale that app on its own.
00:17:25.440
For example, the social network might need to handle a lot of requests compared to the admin, which may not need to handle very many.
00:17:32.960
Putting migrations inside each engine was another decision we made. The interesting thing is Rails doesn't fully support this out of the box.
00:17:40.000
It sort of supports it, but not for the use cases that we wanted and were using.
00:17:47.440
In cases where you run something like this—Rails install migrations—it copies your migrations out of the engine into your wrapper app where they can be run.
00:17:55.520
This works fine if you never add new migrations to your engines, but if you're adding migrations and changing things, this becomes a pain.
00:18:02.640
I don’t want to spend too much time on this, but here's a monkey patch that allows you to leave your migrations inside your engines.
00:18:09.520
The last thing I want to mention about migrations and engines is it's best to have each migration only touch one database table at a time.
00:18:17.520
The reason for this is that if your migrations only touch one database table, it becomes easy to move tables between engines.
00:18:24.560
It basically boils down to cutting and pasting all the migrations from one engine to another, rebuilding your test database, and you're good to go.
00:18:31.920
There's more info here. This is a crazy idea, though, because if you think about having a full service,
00:18:37.920
and you want to move a database table from one service to another, that could be very difficult.
00:18:43.920
In this case, it just becomes trivial.
00:18:50.800
The final self-imposed constraint we put on engines was testing them in isolation.
00:19:00.800
This means the tests for the engine's functionality actually live inside of the engine.
00:19:06.840
When the tests run, they only require the one engine they're testing, proving that engine is self-contained.
00:19:12.040
For example, if you put your tests for the admin engine inside the engine itself, you can prove there isn't any code that references things in the social network.
00:19:18.000
If you put all your tests in your wrapper app at the top level, you can't prove that.
00:19:26.760
The wrapper app loads all of these engines together.
00:19:32.880
Going back to doing engines the right way, these were the things we felt were necessary for keeping everything inside of each engine clean.
00:19:40.040
But outside each individual engine, we also concluded how engines should interact with each other.
00:19:47.600
Not having circular dependencies was key for us. Circular dependencies make it much harder to refactor later.
00:19:56.000
If you look at our dependency diagram here, there's no circular dependencies.
00:20:02.800
This is a directed acyclic graph, and if there were a circular dependency, it's just because I drew it wrong.
00:20:09.000
As a general practice, we decided whenever it seemed impossible to avoid a circular dependency, it meant we were trying to split things apart too much.
00:20:15.640
So if you have two engines, A and B, and it seems like there's a circular dependency, what you probably want is an engine that is both A and B combined.
00:20:21.520
Whenever you can avoid doing this, you should.
00:20:28.480
In Rails, it's super easy to create circular dependencies. Something like this is common: `User has many posts, post belongs to User`.
00:20:35.680
It's easy to do but makes your application harder to refactor when it gets big.
00:20:42.520
This type of code results in a circular dependency between users and posts, each referencing the other.
00:20:48.400
So how do you avoid falling into the circular dependency trap? Well, you could stop using ActiveRecord associations.
00:20:55.520
And how do you stop using ActiveRecord associations? Well, you could stop using ActiveRecord.
00:21:02.720
We didn't quite do that, but we did something along those lines.
00:21:09.640
The approach we took was to create a domain API layer or a service layer—a layer between our controllers and our models.
00:21:16.240
An interface to our domain logic.
00:21:23.680
All our domain API was comprised of simple Ruby classes that wrapped ActiveRecord calls.
00:21:30.560
The arguments to the methods were just simple IDs or params, and the output was always plain old Ruby objects.
00:21:37.480
Let's take a look at an example.
00:21:43.480
Here's a simple example of a domain API class called UserManager.
00:21:49.320
We have a method here called `find_user_by_id` that takes an ID, finds a user, and calls `UserRecord.find`.
00:21:57.520
The UserRecord, if you look below, is an ActiveRecord class.
00:22:04.640
You'll notice that the classes are defined below the private line, meaning they should only be used within the UserManager.
00:22:11.520
The ActiveRecord user is passed off to `User.new`, which copies the attributes off the ActiveRecord class and onto itself.
00:22:19.680
We could actually just return the ActiveRecord object if we wanted to, but we wanted to deter people from using ActiveRecord as much as possible.
00:22:25.040
By returning just a plain old Ruby object like this User object, the caller of this method can't do things like call `update_attributes` or `destroy`.
00:22:32.080
We want to avoid exposing any ActiveRecord calls that we don't want to expose.
00:22:39.120
This is a simple example. You'd probably want to meta-program this a bit.
00:22:46.960
Meta-program the copying over of the attributes using things like ActiveModel or ActiveNaming.
00:22:53.760
You can get form and URL helpers from Rails, but in general, this is an example of the pattern we used.
00:22:59.920
This is a brand new design pattern that I call Ben's Awesome Pattern, so please tell all your friends about it.
00:23:05.920
How does having this domain API affect this situation?
00:23:12.240
Well, in this case, there are two possible things you might want to do.
00:23:19.240
Given a user, we might want to know all the posts they've written, or given a post, we might want to know who authored that post.
00:23:26.760
So if we were to write a domain API to handle both these cases, what would it look like?
00:23:33.600
For the first case, we want to know the posts by a user. It would be as easy as creating a `find_all_by_user_id` on a post manager.
00:23:40.320
We could do something like this: `new up a post manager` and call `PostManager.find_all_by_user_id`, passing it the user ID.
00:23:47.960
This gives us back the posts, handling the case of knowing the user and wanting the posts.
00:23:54.560
So we don't need this line; the user no longer has a dependency on posts.
00:24:02.720
When we want to know the post for the user, we're going to the post manager.
00:24:11.200
Now, if we start using our user manager, we should be able to do something like this.
00:24:17.680
To find the author of a post, we create a new user manager given a post that has a user ID attribute.
00:24:24.560
We can now find the author of that post.
00:24:31.200
The crazy thing here is that using these domain API objects, there are no real code dependencies between users and posts anymore.
00:24:38.920
There is a dependency because the post has a user ID, but no circular dependency exists.
00:24:46.800
Another great thing is that the user doesn't have to change when we add posts to our application.
00:24:54.600
The same applies when you add other things; for instance, when you add comments and videos, the user still doesn't need to change.
00:25:02.880
You'd be creating small domain API objects for each of those things.
00:25:10.560
So you'd have a comment manager and a videos manager, and all those things would depend on the user, not the other way around.
00:25:18.760
Now, who's seen user models in Rails apps that are hundreds or even thousands of lines long? Just gigantic mess, right?
00:25:26.520
This pattern completely avoids having those huge classes or those God objects.
00:25:34.080
Instead of having a few huge classes in your application, you end up with a bunch of small ones that are loosely coupled and easy to refactor.
00:25:41.640
So how does the lack of circular dependencies affect our engines?
00:25:49.520
Reducing the number of necessary circular dependencies lets us go from a single Rails app where everything references everything else.
00:25:56.320
To a Rails app where things are all broken apart into engines.
00:26:03.160
If you have a circular dependency, then you should group those two things together.
00:26:09.760
If you have a lot of circular dependencies, you're going to have a much harder time breaking things out.
00:26:17.200
The other great thing about having engines and an acyclic dependency graph is that you can create a smart and fast test suite.
00:26:24.640
Imagine you're working within an architecture like this, and you're making changes to the admin engine.
00:26:32.640
When you run your tests, you actually don't need to run them for all the engines.
00:26:39.840
All you need to do is run the tests within the admin engine and the tests within Theer.
00:26:47.120
The Schuler engine depends upon the admin engine, so changes made in the admin engine could break things in Theer, but you don’t need to run tests for the social network or common engines.
00:26:54.600
We didn't make any changes to those engines or any engines they depend upon.
00:27:00.680
So you can write a smart test suite using something like Git to get what changed, which lists what files have changed since your last push.
00:27:08.480
You can determine what engines have changed and what engines need their tests run based on your dependency graph.
00:27:16.480
What we found is that normally, you only need to run a fraction of your tests.
00:27:23.560
We have our one-way dependencies to thank for that because, like I mentioned, the user engine, which is heavily depended on by the rest of the system, rarely changes.
00:27:30.200
For example, when we add posts to the system, the user doesn't actually need to change.
00:27:36.760
So you don't have to run the majority of the tests in your app.
00:27:44.960
How did using engines, domain APIs, and all these conventions work out for us?
00:27:51.440
Well, engines are a pain in the beginning. We had super low velocity for the first month and a half.
00:27:57.840
The client was actually getting worried, and we really couldn't bring on more developers.
00:28:04.560
We were still developing these patterns and best practices for engines in our application. Constant refactoring was a must.
00:28:11.480
At one point, we were spending about a third of our time just refactoring.
00:28:17.920
There were a few technical hurdles we had to overcome—the monkey patching of Rails being one of them.
00:28:25.520
But in the long term, engines definitely won out. At the end of those months, we saw no slowdown in development time.
00:28:34.480
There was no slowdown in the time it took to do fairly major refactorings.
00:28:40.440
Engines aged really well—oftentimes we'd finish working on an engine and ignore it for months at a time.
00:28:46.480
It makes for easy parallel development, so you have these little boxes that certain teams can work within.
00:28:55.320
You can talk in terms of which team will be working within which engine.
00:29:01.720
The potential for scaling is great—if you need to scale a certain area of your app, all you have to do is pull that engine out into its own app.
00:29:07.440
Lastly, the ability to write better tests was a big advantage. So overall, I give this architecture a thumbs up.
00:29:14.640
But it probably took me two months to really come around to it. I really hated it for the first month.
00:29:21.520
If this stuff seems interesting to you, or if you think it might help one of your current apps, give it a try.
00:29:27.120
Try creating a new engine for the next major feature you add, or try pulling out some of your models into their own engine.
00:29:34.720
Try avoiding circular dependencies and creating a service layer.
00:29:41.920
Most importantly, try to think outside of the Rails way of doing things.
00:29:48.960
As Cam said this morning, build the right thing the right way.
00:29:56.040
Start thinking in those terms.
00:30:00.000
Thanks! If you want to learn more about engines, my coworkers and I have been blogging about engines.
00:30:06.560
Here's a link. There's also a link to my SpeakerDeck. I'll post these specific slides in a few minutes.
00:30:14.000
That's it. Thank you!
00:30:20.800
Awesome, thank you very much, Ben! Do we have any questions?
00:30:26.440
We do have a whiteboard and a dry-erase marker, but we went low tech on that.
00:30:32.960
It was actually a good exercise in our process—anytime someone added a new engine, they would erase the whole thing and redraw it.
00:30:40.560
This made sure that everyone on the team knew how everything was interacting.
00:30:47.440
When you redrew it, you would look through the gem files and ensure there were no circular dependencies.
00:30:54.800
That’s the approach we took; we kind of took the process approach on that.
00:30:58.000
We've got another question over here.
00:31:04.560
Sure! I really like the idea of splitting out the tests into different engines.
00:31:11.400
Was there a test for the overall application to ensure everything tied together?
00:31:18.040
Yes, we did have some tests, and those tests lived at the highest level.
00:31:24.960
They lived in the wrapper app and tested to make sure that data was moving correctly between each of these engines.
00:31:31.800
There were a few high-level smoke tests, so to speak.
00:31:39.840
Thank you! Yep, here.
00:31:45.840
Thanks for the good talk! You mentioned at the beginning trying to split the app into engines early.
00:31:54.080
What are your general rules or advice regarding how to split the app into engines at the very beginning, because it feels like premature optimization sometimes?
00:32:02.080
That's a hard question—trying to decide where the seams are to break your app apart.
00:32:09.480
We started by looking at a high-level overview of the big areas of functionality or the users of the system.
00:32:16.200
That's how we began breaking things out.
00:32:23.880
I've also explored breaking things out at the lowest possible level.
00:32:29.680
For example, I created a couple of engines where the engine only contained a single model and a single database table.
00:32:36.800
It really just depends on your application and your team—the comfort level with it.
00:32:43.880
So it’s not really a good answer, but it’s the best I can give you—it depends.
00:32:50.000
I'm happy to talk more with you about it after.
00:32:54.560
I've got one over here.
00:32:57.600
Was each engine in its own codebase included as a gem, or how were you structured?
00:33:03.600
They were all under the same codebase—so we had a single Rails app, a single sorcery.
00:33:10.160
We had a subdirectory for engines next to the app directory or the lib directory.
00:33:16.960
That's where we had all of our engines underneath.
00:33:24.320
I was just wondering how you got your tests to trigger for a single engine on a commit?
00:33:31.160
Rails gives you a 'dummy app,' which is a little, full Rails app inside of your engine that requires that engine.
00:33:38.120
You boot up this little dummy app that requires the engine inside of it, and then you can run your tests just within that context.
00:33:45.720
That's basically how it works.
00:33:52.440
Okay, just, if you stop exposing ActiveRecord objects in your Rails application, don’t you find it hard to work with forms?
00:33:57.600
Even optimizations, like including relationships, is challenging?
00:34:04.160
It is harder to not use ActiveRecord; it does make things really easy.
00:34:10.760
It also makes your code really tangled, so it's a trade-off.
00:34:16.960
People who have written code before there was Rails know this is stuff we had to do.
00:34:24.560
On the other hand, there is a balance.
00:34:30.960
Things like form helpers are essential to getting everything to work well in your views.
00:34:38.160
You can get some of that using parts of Rails, so using Active Naming and Active Model, you can include them in the ruby objects.
00:34:48.160
It doesn’t expose all the ActiveRecord stuff, but it still gives you a lot of those helpers.
00:34:55.120
So it is a little bit of a balance.
00:35:02.720
Awesome, we've got time for one more question.
00:35:07.320
A comment and a question.
00:35:13.600
The wonderful thing about new frameworks is getting to discover the same problems again and invent the same solutions.
00:35:21.440
Large software systems have had to deal with these problems obviously for a long time.
00:35:29.440
What I wanted to ask was, were you using foreign keys? If so, how are you dealing with foreign key relationships between different engines?
00:35:37.120
Foreign keys can be interesting because you have tables living in different engines.
00:35:42.040
They don't necessarily know about each other, but there are two solutions.
00:35:49.640
If there's a need for performance, one option would be to merge them together.
00:35:57.760
We did that in one case where we need two tables to be performant, so we pulled those together.
00:36:05.440
The other option is to maintain two separate things and create something above it that depends on them.
00:36:12.560
That thing above it creates the join between the two.
00:36:18.960
It's a little bit of gray area that we toyed with.
00:36:26.600
Awesome, thank you very much, Ben. Everyone, a big round of applause for Ben!