Summarized using AI

The Rails Boot Process

Xavier Noria • September 27, 2024 • Toronto, Canada • Talk

Introduction

The video titled "The Rails Boot Process" features Xavier Noria, a Rails Core member, who delves into the mechanics of how a Rails application boots. It addresses critical aspects such as when the logger is ready, when the $LOAD_PATH is set, the execution of initializers, and the configuration of autoloaders. This talk was presented at the Rails World 2024 event.

Key Points Discussed

  • Booting Process Overview: Noria explains that booting a Rails application involves more than just launching the web server; it primarily means preparing the application for use, making all models and configurations available for tasks, including console operations.
  • Understanding Boot Steps: The booting of a Rails application can be manually initiated through commands like require config/environment.rb, setting the stage for all components to work seamlessly.
  • Lazy Load Hooks: Noria introduces lazy load hooks that allow initializers to execute when a class is loaded, rather than at boot time, enhancing efficiency and performance in application lifecycle.
  • Initializer Mechanism: He elaborates on the use of initializers, allowing users to register blocks for configurations that will run during the boot process, thus providing the flexibility to customize components according to user needs.
  • Rails Application Structure: The primary bootstrapping files that are crucial in the generated Rails application, like config.ru, config/environment.rb, and config/application.rb, are explained in detail, clarifying their roles in the boot process.
  • Railties and Engines: Noria distinguishes between Railties and engines, showcasing how both facilitate the booting process and allow applications to inherit specific functionalities.
  • Execution Order of Initializers: The video emphasizes topological sorting to manage how initializers execute, permitting dependent initializers to run in the correct order based on configuration requirements.

Examples & Illustrations

  • ActiveRecord Configuration: Noria provides practical examples, such as how ActiveRecord utilizes lazy load hooks to set its logger dynamically based upon the application’s configurations, demonstrating the principles discussed.
  • Real World Application: He references over 300 initializers available, providing insights into the internal workings of a Rails application and its customizable architecture.

Conclusion

In summary, the boot process of a Rails application is an intricate but organized sequence involving multiple stages and hooks. This foundational knowledge enables Rails developers to optimize and customize their applications effectively. Understanding these mechanics not only aids in development but also enhances performance, making it vital for developers working with Rails frameworks to grasp the boot process intricately.

The Rails Boot Process
Xavier Noria • September 27, 2024 • Toronto, Canada • Talk

What happens when a Rails application boots? When is the logger ready? When is $LOAD_PATH set? When do initializers run or when are the autoloaders are set up? Rails Core member Xavier Noria covered all this and more in his talk at #RailsWorld.

#Rails #Rails8 #internals #boot #railties #engines #initialization

Thank you Shopify for sponsoring the editing and post-production of these videos. Check out insights from the Engineering team at: https://shopify.engineering/

Stay tuned: all 2024 Rails World videos will be subtitled in Japanese and Brazilian Portuguese soon thanks to our sponsor Happy Scribe, a transcription service built on Rails. https://www.happyscribe.com/

Rails World 2024

00:00:11.719 how ra applications boot okay um as you know the ra components do not need to
00:00:19.039 live in a ra application so for instance you can use active record outside of a ra
00:00:24.720 application however somehow when you when you are in the context of a ra application
00:00:31.359 things get coordinated you know somehow and you have everything available you have active job available you have you
00:00:37.960 can talk to the database you don't have to do anything right so the talk is
00:00:43.239 about how this works and also a little bit of what happens when okay all right
00:00:51.199 so the first thing that we are going to see is how you boot a ra application because for many people booting a ra
00:00:58.640 application means launching the server okay but this is actually two things is
00:01:04.239 booting which is smaller than launching the server is booting first and when you
00:01:09.240 are you have the application app then you launch a web server but if you think
00:01:14.320 about that booting application is this is having all configure ination and
00:01:20.600 ready to be used so for instance you can do that in in a console right so we
00:01:26.119 launch a console and we have all models available we can talk with the DAT database we can schedule jobs and we
00:01:33.159 have the entire application at our disposal okay so the application has been boot there's no web server running
00:01:40.000 okay but the application has been boot everything is ready okay the majority of RA
00:01:45.439 commands uh boot the application there are a few that that that don't for
00:01:51.040 instance if you're R being ra stats it does un boot the application but the majority of them uh do rate tasks also
00:01:59.920 Al boot the application for instance if you have a migration the models are
00:02:05.560 available right so how do you do that I mean uh we
00:02:11.400 are booting the application in these examples from the console from the shell but if you want to do that
00:02:17.400 programmatically how do you do it so if you are in a ruby script uh that's the
00:02:22.959 public interface you require config environment RV okay that's that you you do that thing and that Boots the
00:02:29.400 application next line you you you have everything ready for you okay in particular case of a rate task you have
00:02:36.599 to depend on the environment task which is a task provided by rails and that task if you do the walkth
00:02:43.800 through ends up actually calling required config environment.
00:02:49.800 RB so we are on the same page of what does it mean to boot the the application now
00:02:57.239 before we get into the actual topic there's a a couple of preliminary
00:03:02.760 sections uh uh a couple of concept that we need to understand to go through the
00:03:08.560 actual boot process the first is lazy load hooks right so let's imagine we
00:03:15.799 have this initializer that wants to extend active record with a module all
00:03:22.480 right one way to do that is to you know you require and then you open the the
00:03:29.040 class in check your thing done okay makes sense but there is a a a
00:03:35.519 convention in in applications in raise applications that says the framework is
00:03:42.879 responsible for loading things whenever whenever they have to be loaded and you
00:03:49.200 just hook into this process so that when the framework decides that it has that
00:03:55.720 it has to load then you are going to be able to do your thing and you do do that with lazy hooks so this is the this is
00:04:02.840 the thing all right you say unload active record there should be a do there
00:04:17.120 reopening the class you say Hey whenever the class is ready and I defer that to you please
00:04:24.560 call me and then I am going to do my thing okay that that's the way to do it
00:04:30.520 um there are more than 40 uh load hooks like this one I show it active record
00:04:36.479 there are there are uh many more all right now the the second preliminary
00:04:41.960 concept that I want to explain is initialization hooks that maybe maybe you have seen in your own applications
00:04:49.960 which is things like this okay they are behind the scenes implemented with the previous uh
00:04:56.840 technique but they have a special syntax let's say special API all right so config after initilize means there's
00:05:04.720 there's a there's a checkpoint during booting uh that means this that things
00:05:10.759 have initialized and it says please call me when Once once the analization has
00:05:16.520 been finished please call me all right and there are a bunch of them okay
00:05:21.759 so all right so with this with these two things now we can start
00:05:29.840 uh understanding how things Boot and we we are going to start with the main
00:05:35.400 bootstrapping files that we have in a generated new uh R application which are
00:05:41.039 these three okay config Bo RV config environment V config application Dov we
00:05:48.039 saw before that config environment. RV is is the canonical entry point if you
00:05:53.199 want to boot but we have these three all right okay so let's start with config
00:06:00.919 environment V config config butb and config environment dob normally you do
00:06:05.960 not even open them okay they are there they are generated they they have their
00:06:11.280 function let's say but normally you do not modify them or even open them we
00:06:17.240 normally work more with config application. B right but let's see what
00:06:22.479 config environmental th be the entry point to the boot process does it does
00:06:28.039 two things very simple file all right it requires config application. RV and then
00:06:33.840 it calls this method on the application Singleton right okay so let's see what config
00:06:42.880 application Dov looks like it is this okay we have seen this many times but
00:06:49.199 let's let's just let's study this in detail the first thing that it does is to actually load config
00:06:57.440 butb so let's jump to config for a moment it is this also simple file okay
00:07:06.000 it sets a environment variable for bundler to find gem file and then it
00:07:11.879 performs bundle setup Bund bundle setup basically it it it is saying okay um I I
00:07:19.560 you the application is only going to be able to load the gems that are specified
00:07:25.479 in the gem file and also from the standard Library okay so any other gem
00:07:30.680 that was not declared there or um any other gem with uh declared but with a
00:07:36.319 different version or anything like that is just not going to be available in your environment okay that's B setup and
00:07:42.720 that boots net setup does a few things to speed up your required um your
00:07:49.759 required calls basically that's boots snap set up okay so we have those things
00:07:55.000 in root uh dot orb and now once loaded we have required
00:08:02.639 rails all okay we are going to see require uh rails all
00:08:10.159 later but uh at this point we can say that we have rails M available after
00:08:16.560 this line we have rails M available because one question is when are things
00:08:21.680 available when can when can when can I refer to rails M when can I refer to rails logger okay those are questions
00:08:29.039 that that uh I believe uh there's lacking documentation about that it would be cool that we uh Define a clear
00:08:36.599 contract and clear documentation and a test Suite that uh you know verifies all
00:08:42.039 all that contract in any case at this point we have rails M we do not we do
00:08:47.040 not have rails root for instance at this point then bundle require is going to
00:08:53.680 require the gems in your gem file unless you opt out basically right okay
00:09:00.240 now we arrive to this line uh we are defining here an
00:09:07.480 application that inherits from graes application okay a class that inherits from graes application this light these
00:09:14.880 lines looks very Innocent but it is not and the reason is that uh in Ruby uh
00:09:23.440 we have a a u method that is optional in classes call it inhad
00:09:30.120 okay so if a class implements uh the inherited method and the class is subclassed your inherited Hook is called
00:09:38.000 there's an inherited H hook uh configured here that is called right in
00:09:43.560 that line okay so just as a side effect of
00:09:48.920 subclassing a number of things are going to happen these things okay so by
00:09:54.200 subclassing now we we get access to the application Singleton
00:09:59.519 we get ra root so that is why ra you can use ra root in in the in the class body
00:10:06.200 you couldn't do that before but at this point you can why because the inherited hook you know magically is making these
00:10:13.920 things available for you the autoloaders are available for
00:10:19.279 configuration you cannot still autoload but you can access them and make them uh
00:10:24.720 print locks or you know or or ignore things whatever you know at this point
00:10:30.120 load path gets leap prepended so I don't know if you have
00:10:35.600 ever needed to load something from liap if you do require something from
00:10:41.480 liap um uh before we Define the application is not does not work you can
00:10:47.279 do require relative but require does not work why because lip is still not there but after you know in the in the class
00:10:54.399 body it is there you could do a require there at this points and then there's a before configuration hooks executed uh
00:11:01.240 as a last step in the herited hook um that is not uh particularly useful for
00:11:07.680 applications perhaps because at this point application still hasn't had the opportunity to do anything but for gems
00:11:14.880 it is it is practical all right so once in the next
00:11:21.040 line we have all of this at our disposal and then you execute the class body
00:11:29.360 so that is uh that is what required relative application does in config
00:11:35.839 environment ofv now the second line is calling initialize uh exclamation mark on the
00:11:42.720 application singl ton to understand this line we need the
00:11:48.079 rest of the presentation so let's go but let's do a recap before all right
00:11:54.320 so uh let's order the ideas we set up bundler and Boots snap we require rails
00:12:00.839 all and the gem dependencies we have these things available at this point before configuration hooks get
00:12:08.360 called we execute the class body and then the mysterious line that comes now
00:12:13.800 all right okay we have barely done anything at
00:12:19.519 this point but a number of things have happened okay all right so to understand
00:12:24.920 ination ex exclamation mark we need to introduce a few Concepts
00:12:30.480 okay these Concepts were not present in the early days of ruon rails the boot
00:12:36.000 process uh at the beginning was uh quite proced and these things were um were
00:12:43.079 designed with ra three ra 3 had like a big big uh redesign of the boot process
00:12:49.639 so uh this has been here since that version the first concept is rail ties
00:12:55.680 okay so a rail tie technically is a subclass of RA rail Ty that's it a
00:13:02.000 subass but by subclassing you get some things okay some features like you get
00:13:08.680 config and points for instance we are going to see examples now uh you get
00:13:14.120 something that we call initializers that is fundamental for this talk and uh also you can configure
00:13:20.959 callbacks for ra commands so super important for this
00:13:26.360 talk the most important method of this talk is ra rail TI initializer okay it
00:13:33.079 does a number of things it registers a block to be execute during initialization okay so you can imagine
00:13:39.040 we are going to see examples in initializer so in the body of a rail TI which is a class okay class body you can
00:13:47.360 say initializer you are going to give it a name I'm going to mention that now you
00:13:52.480 pass a block and it says basically I I want to register this block to be called by who by rails okay
00:14:00.519 so we register this block and blocks these blocks are going
00:14:05.800 to be execut in the order they have been registered unless you pass one of these
00:14:12.920 options before and after and after uh allow you to alter this order okay so by
00:14:18.320 default it goes like linear um but you can say please uh invoke this block
00:14:25.480 before that other initializer runs okay this is possible with these two things
00:14:31.959 uh we give them a descripted name and the name is the one that allows you to say please execute me before or after
00:14:39.240 that other one you can refer to other initializers and if you want to list them just for curiosity uh you can you
00:14:47.160 can use this this uh common be rails initializers uh there's more than 300 so
00:14:52.839 you're not going to get bored um and it has to be said that in in reality most
00:14:58.440 of them are private interface okay so for curiosity or if you want to debu something you know you can list them but
00:15:05.199 you cannot rely on most of them being you know present always or maybe they
00:15:10.440 you know some time ago there were 200 no there's 300 and some can be delet some
00:15:16.199 can be refactored it's not part of that missing documentation is is being clear
00:15:22.199 about what is public interface indeed okay however you can list that you know
00:15:27.240 and see uh how things get executed so an example real example from active record
00:15:34.160 so active record is a library that can be used outside of rails but it defines a rail Tie by defining a rail tie the
00:15:42.560 the the com the component is able to hook into rails that's the design so
00:15:48.800 this one for instance simple one uh sets the logger for active record right so we
00:15:56.440 you see we are subclassing rails rail TI uh conventionally the name of the
00:16:03.360 class is going to be name space rail ti so that but that technically is not
00:16:09.199 required okay uh you subclassing is what defines a rail type but the convention
00:16:14.360 the nameing convention is that one now you see in the class body that's a class method so in the class body we are able
00:16:20.680 to register an initializer that is going to set the logger uh for active record
00:16:27.639 and it goes like this look it it is using a a a lazy load hook so it says uh
00:16:35.519 please whenever active record base is loaded call me okay and so when when I
00:16:43.199 get called Simple block I am going to take the ra
00:16:48.360 logger uh by default all right the ra logger and assign so active record has
00:16:53.480 its own logger but which logger is the natural one if you are in the context of a r application the same log that the
00:17:00.000 application is using right so that is encode in this block basically so call
00:17:06.360 me and I unless I have already a value I am going to set the logger for myself to
00:17:13.120 the one the raise application is um replication half okay has so this
00:17:21.319 is the basic idea of this design we can see it here in this in this simple Slide the basic idea is I Define
00:17:30.520 metalizers the ownership of the configuration belongs to the user and
00:17:35.600 the ra application so we have config do active record. something for instance okay um in this case it is not using the
00:17:43.760 configuration you know but in general the idea is that you get the configuration endpoints the ownership of
00:17:51.080 them is is for the Raz for the raise application and the user and then the
00:17:56.919 initializer that you that you Reg register are going to use the values of
00:18:03.360 those configuration and points to set the component itself the component sets
00:18:09.440 itself it does its own configuration its own initialization by reading what the
00:18:15.000 user configured in the race application that's the basic idea of of this design
00:18:20.480 right another example uh the the actual name so this initializer is active
00:18:27.039 record uh the the real name is initialized database but it was too long so I edited just a little bit uh same
00:18:36.080 idea the LA see hook okay unload active record base uh now look I am going now to set
00:18:45.200 the configurations on myself according to the ones defined in the application and after that I going
00:18:51.919 to establish a connection with the database this is done in this
00:18:57.240 initializer okay all right an example also from active
00:19:02.520 record for a callback uh attached to a ra command in
00:19:09.200 this case the runner command okay you know the runner command allows you to run either a program that you pass in
00:19:15.919 the Shell as an argument or either a a um file okay if the argument is a file
00:19:23.480 is going to interpret the file after booting the application but in addition to boot the application
00:19:29.520 uh it is able to also uh run code like
00:19:34.679 this okay so basically this is saying in the case of uh executing the runner command I am going to load so now active
00:19:43.520 record decides that is the time to load active record base so by loading active
00:19:48.760 record base for instance as we saw the connection to the database is going to
00:19:53.960 be established all right otherwise it's going to be made maybe uh later in other
00:20:00.440 commands or in other scenarios all right so there's a bunch of rail ties in a
00:20:06.440 rails eight application all these ones and now we go for the
00:20:12.440 second Concept in this in the second part of the presentation which is engines so rails engine is a subclass of
00:20:21.159 rails rail type and an engine is a subclass of rail
00:20:26.480 engine all right so ra engines is a big topic because an
00:20:33.720 engine is basically like a small race application so to speak you can have models you can have views controllers
00:20:39.520 routes uh and everything okay uh but but uh only you know in the context of this
00:20:46.200 presentation what is relevant for instance is that you since you inherit from rails rail TI everything that we
00:20:52.840 have seen before applies to engines okay you have the ability to register
00:20:58.000 initialize to have config endpoints Etc now uh engines can have config SL
00:21:06.039 initializers as well and you can have roads okay that's
00:21:11.200 more or less relevant to this presentation uh if you want to dig uh deeper into engines I recommend that you
00:21:18.520 uh watch this this uh presentation From Grace Oliver crafting rail plugins that
00:21:23.559 that he did in rails com Detroit this year all
00:21:29.360 right so a rails eight application ships with a number of of engines okay these
00:21:36.840 ones and now now we know we know enough to
00:21:42.240 understand this line of code because if we unfold what is happening in this
00:21:48.240 required call it is loading the rail Di and the engines so rails all translates
00:21:56.960 to do the is required rails and then the rail Di and the engines okay so at this
00:22:04.120 point in config application. RB we all loading the rail ties and the engines and as a side effect the initializers
00:22:11.440 are being registered they they are doing nothing but they are all loaded and registered okay so we know
00:22:18.720 them now the last concept is applications ra application is a
00:22:23.760 subclass of rails engine and again an application is subclass of RA
00:22:30.559 application and as a consequence a ra application is an engine which is an
00:22:36.760 which is a rail TI okay so we have this inherit uh this um inheritance going on
00:22:43.159 with these three classes so this is what we are doing here now uh we are subclassing ra
00:22:51.120 application so no we can understand in a deeper sense what is going on here we're
00:22:57.159 defining an engine we are inheriting all this stuff okay they are not the same thing for instance a Rel application has
00:23:03.320 outo loaders and engine does not have outo loaders okay is the ra application that has the autoloaders that are going
00:23:09.880 to manage the code of the application itself and the code of the engines as well okay but so there are some
00:23:16.640 differences okay but they are engines and they are uh rail di as a consequence
00:23:22.880 of that so finally we have all set up to understand this line of
00:23:30.279 code um initialize is very simple so what is a little bit more complicated is
00:23:37.039 the setup that I have in explained but once you have all this in place this is
00:23:42.159 quite simple you get the initializers you do a topological sort on them topological sort means uh so we
00:23:50.120 said they are registered they they run in the order they they have been registered but you can uh you say uh you
00:23:57.919 can you can say please execute me before this after that whatever you know so a
00:24:04.080 topological sord basically gives you uh a linear uh collection that respects
00:24:09.880 these relative orders right this this is this is in this comes with Ruby okay you can do that
00:24:15.360 okay so we we get like we get an array where all these constraints are in place
00:24:23.080 and we we do not have to think about hey I have to run this is done by this sort okay
00:24:29.159 simple we get this uh collection and we uh execute the nizers that's the idea
00:24:34.960 right this aners are not just the one the ones that that
00:24:40.240 engines uh Define there are there are two sets of initializers that are
00:24:45.640 special and they are like her code we could say okay which are so we have three sets here okay
00:24:52.039 bootstrap rail ties which are the ones that we have been talking about mainly and then the finisher set okay let's see
00:24:59.200 what the bootstrap set does at this point we load uh the um
00:25:06.640 configuration for the environment we are running um uh we have set up okay so if
00:25:12.320 you are in development mode now is the time that we are going to load config
00:25:17.360 environment development dob okay why does development B take precedence of of
00:25:25.600 config application RB now we can understanding simply because it runs after the after it so if you if you say
00:25:33.000 config Fu equals 1 in application V and then config Fu equals two in
00:25:39.520 development. RV well the the second one runs after the first one so is going to
00:25:44.760 be overwritten that's as simple as that okay now we we load active support all
00:25:51.960 there's a way to opt out of this but it's very rarily used I would say uh so
00:25:57.440 in particular you don't have the whole active support in in config application RV but you have it when these things so
00:26:04.640 when the initialize um exclamation mark call uh is invoked at this point we
00:26:11.480 Define the logger as well we set up the once autoloader that that is why you are
00:26:17.799 able in modern versions of rails to autoload uh the constants managed by the
00:26:24.919 ones outo loaders so application has two out loaders ones and main not the topic of this presentation but um uh the ones
00:26:32.520 out to loaded is ready so that that is why in config iniz you can autoload
00:26:38.520 constants managed by this one not the other one and then we have at this point
00:26:43.760 we have this hook so this is the bootstrap now all you know the the all the
00:26:51.440 initializers defined by uh rail ties and engines are the ones that going are
00:26:57.399 going to be ex executed at this point okay and in particular at this point we
00:27:02.919 get config enzers the ones of your application and also the ones of engines
00:27:08.720 at this at this time they have been executed uh there's two mat here is 300
00:27:14.960 of them and they are like very particular not I I wouldn't say they are especially interesting most of them okay
00:27:21.000 so we only have 30 minutes I'm not going to enter into these ones but this is the
00:27:26.320 bul of this uh ad hoc initializers happen here and in particular config initializers uh from
00:27:33.960 your application and engines finally The Finisher initializer the third set now
00:27:40.640 we set up the main autoloader okay so we have the bootstrap set we have config um
00:27:47.760 initializers executed and after that there's like the last step the last mile of the boot process you you get the main
00:27:54.880 AOL loader now we build the middleware stock and this two prepare Hook is a
00:28:00.679 special hook that uh is executed when you boot and also when you reload then at this point we eager load
00:28:09.000 if uh if enabled and load the routes okay the load the routes are loaded
00:28:16.039 unconditionally uh um normally but in rails 8 they are going to be lazy loaded
00:28:22.600 unless you are eager loading the application okay so if you are running something that doesn't need the r routs
00:28:28.559 they are not going to be loaded they are going to be loaded on demand okay but if you areer loading they are going to be
00:28:34.039 loaded and then at this point we have after initialize and finally we enable uh yit
00:28:42.840 why so long is more performant so it is not it is not of particular interest a waste of time that Y is monitoring what
00:28:49.600 is happening in the boot process because in generally speaking the boot process you you get things that happen once okay
00:28:57.200 and what you want to when what you want to um uh work on is code that is uh
00:29:04.919 executed multiple times that's the interest of the jit so that is why uh please do not monitor anything now that
00:29:11.880 we have finished the boot process uh you can start uh monitoring
00:29:23.960 bit of stuff here uh but let's do a summary of the summaries like you know
00:29:30.559 uh let's recap a little bit bird's eye of the whole process we load config
00:29:37.640 boter B this is mainly bundler setup boot snap setup we load the the config
00:29:43.760 application RV that defines the rail ties the engines as as a side effect of that we get all the initializers set
00:29:50.960 up uh then we subass rails application by subclassing a number of things are
00:29:58.600 happening after that we get all of this setup done we prepare uh the bootstrap
00:30:04.799 in utilizers we append the finishers that we saw and then we run them all in
00:30:10.600 topological order and while that is happening we trigger initalization Hooks
00:30:16.399 and we are ready to rock all right thank you
Explore all talks recorded at Rails World 2024
+33