00:00:16.789
Okay, thank you. So first of all, I'd like to say that I really like being here. For me, it's the first time visiting Poland as well as attending a Ruby language conference.
00:00:23.130
I'm quite familiar with a dozen programming languages, but Ruby is not one of them. However, that doesn't really matter for this talk. I basically have two objectives.
00:00:34.350
The first objective is to give you a brief introduction to the NixOS project, tell you what it's about, and highlight its advantages.
00:00:47.070
The other objective is to explain the underlying vision of NixOS. With the tools in the NixOS project, we aim to deploy systems declaratively. This declarative approach can be found in any tool that's part of the NixOS project, but it's not limited to tools from the NixOS project.
00:01:03.390
So, what does it mean to be declarative? If I say I want to do something declaratively, what am I implying? I searched the internet and found a nice presentation about linguistics.
00:01:11.100
In natural languages, there are basically four kinds of sentences, one of which is declarative sentences. A declarative sentence states facts, for instance, 'the dog in the neighbor's yard is barking.' Another class of sentences is imperative sentences.
00:01:38.250
This XKCD comic illustrates an example of imperative sentences; they are essentially commands or polite instructions. In programming, we find similar concepts. You can also be declarative in programming.
00:01:57.299
One definition I found is that in declarative languages, you express the logic of a computation without describing its control flow. This vision mainly applies to languages like Prolog. In practice, people tend to use a more pragmatic definition.
00:02:25.049
For example, a programming language researcher stated that in a declarative language, you describe what is to be computed rather than how to compute the result or behavior. In contrast, imperative languages involve describing computations, typically modifying mutable state and performing input/output operations.
00:02:53.400
Interestingly enough, it's challenging to draw a clear line between how and what. In some contexts, like Prolog, you don't need to describe the control flow to evaluate a predicate; the language runtime takes care of that. It acts more like a what specification rather than how.
00:03:20.880
Familiar examples of declarative languages are HTML and CSS. With HTML, you express the structure of the page by defining individual elements such as paragraphs, divs, etc., while in CSS, you describe the styles for those elements.
00:04:02.190
For instance, you might say you want the outer div to be centered or have a border around it. You can specify that an inner div should have a certain width without needing to explain how to render it, as the browser's layout engine handles that.
00:04:26.370
Ultimately, with just two files, you can render a functioning page. However, this example is quite simplistic. As many of you know, in web application development, once your application is complete, you want to make it available to end users.
00:04:39.930
This means deploying it to a production environment. Typically, development occurs on a local machine, and it's essential that the application runs on a server or a collection of servers, possibly in a cloud environment. Deploying a system generally involves executing several activities.
00:05:52.560
Depending on the programming language you use, you may need to compile your software. For Ruby, this typically isn't required, but if you're using Java, for example, you need to compile class files.
00:06:10.350
You also need to package your software, often building it into a JAR file for Java. Afterward, you must transfer these packages from your development machine to the target machines on the network, which may also require activating components and modifying configuration files.
00:06:45.300
Moreover, if you have an existing version of your web application that needs an upgrade, this process often entails removing obsolete parts and replacing them with newer versions.
00:07:06.570
Deploying systems turns out to be far more complicated than many people assume. One reason for the complexity of deployment is the choice of technology.
00:07:20.880
When developing a service-oriented system or one composed of microservices, you may want to utilize various technologies. This could include different databases or programming languages for implementing individual services and diverse operating systems as needed.
00:07:54.360
As a result, having diverse technologies also leads to many types of deployment procedures, turning deployment into a potential nightmare. Similarly, deploying in production is typically done on a much larger scale.
00:08:13.140
While you develop on a single machine, if you've built a successful application, more than one server will likely be required to handle the traffic from millions of users.
00:08:29.940
Additional complexities arise, including connectivity between machines, which must also be accounted for. If you upgrade the system, things might go wrong, such as breaking your configuration.
00:08:48.600
In such cases, you'd prefer to revert to a previously working configuration, but this can be challenging. At some point, you might find yourself unsure about what changes made your configuration fail.
00:09:01.290
This usually leads to a frustrating cycle of attempting fixes without ever being able to discern what went wrong or how to revert to the previous state, which can result in downtime for your application and upset clients.
00:09:38.370
To deploy a system—particularly a large web application, or a collection of services—you need automation. There are many automated deployment solutions available; some are designed for specific technologies while others are more general-purpose.
00:09:55.530
An example tool is Chef, which implements Ruby. Is anyone using Chef in production? A few hands went up. Great! Is anyone using Puppet? Any users of Nix? I see just one. Cool.
00:10:32.700
Among these tools, some utilize declarative deployment specifications. What does that mean? Essentially, when I develop a system on my machine, I want that same system to run in the production environment.
00:10:51.080
But I don't want to have to explicitly dictate the activities required for deployment. I expect the system to determine what needs to be done.
00:11:15.960
For instance, I took an example from a Chef tutorial showing how to deploy a WordPress application. In this configuration file, you describe the outcome of changes.
00:11:40.410
For example, you might want to obtain the latest WordPress from the WordPress website and create the target folder on the remote machine to extract the WordPress tarball. Chef tries to achieve that outcome.
00:12:11.580
Instead of specifying step-by-step tasks, such as creating a directory, you express that you want this directory to exist, with specific attributes like ownership and permissions. The deployment system determines the necessary steps.
00:12:38.320
For instance, if the directory is already present, Chef won't recreate it. That's an optimization to avoid unnecessary actions.
00:12:58.730
However, Chef has limitations. It captures only a subset of changes, such as deploying WordPress, but deploying it also requires an Apache web server and PHP, which may not be included in the configuration.
00:13:13.960
If you try deploying this configuration to a machine that doesn't have Apache installed, it won't work. Therefore, Chef specifications aren't guaranteed to be complete.
00:13:27.830
Additionally, upgrading WordPress involves overwriting files, which could lead to downtime for the site, much to the annoyance of users.
00:13:52.300
Now, let's discuss the NixOS project. I am a contributor to NixOS, and its central component is the NixOS Linux distribution, built on the Nix package manager. In NixOS, the deployment of machine configurations is entirely automated from declarative specifications.
00:14:24.290
As an example, I can describe a specification that captures the configuration of my notebook. This includes specifying the bootloader on a partition, defining the root and swap partitions, and indicating which end-user packages, such as Firefox, to run.
00:15:00.450
You can also specify that you want to use the KDE desktop and run certain system services, like OpenSSH. With one simple command, you run 'nixos-rebuild switch', and the entire configuration is deployed.
00:15:12.590
The system figures out the necessary steps to reproduce that configuration, downloading all needed dependencies, building everything required, and composing configuration files accordingly.
00:15:43.650
Additionally, if you upgrade an existing configuration, NixOS always stores multiple versions of packages alongside the older versions. This means that if an upgrade fails for any reason, you can instantly revert to the previous version.
00:16:11.790
The bootloader in NixOS allows you to select any previously deployed configuration that hasn't been garbage collected. For example, if an NVIDIA display driver fails and prevents you from booting into your desktop, you can easily revert to an older, stable configuration.
00:16:45.260
This is one of the most powerful features of the NixOS Linux distribution compared to traditional distributions like Debian or Fedora.
00:17:10.090
So, how does NixOS accomplish this? NixOS uses the Nix package manager to carry out deployments. The Nix package manager isolates packages, storing them in a unique manner.
00:17:35.450
In NixOS, we have a special directory called the Nix store, where packages are stored. As you can see, the file names of each package are prefixed by a hash code, derived from all build-time dependencies involved in compiling the package from source.
00:18:06.060
This includes the sources, libraries, compilers, and build scripts. For instance, if you build Firefox with a newer version of GCC, it will produce a path in the Nix store with a different hash code.
00:18:40.250
This allows you to store it alongside existing variants without conflict. NixOS can deploy entire machine configurations from declarative specifications, and this methodology extends to individual packages.
00:19:02.720
For example, the expression shown describes how to build OpenSSH from source. In this function definition, the parameters correspond to the dependencies required to build the package.
00:19:36.950
To build OpenSSH, you need OpenSSL and certain utilities. The function fetches the OpenSSH source tarball from the official website, and in the remainder of the function, you execute build commands.
00:20:06.490
While building the package, we always create isolated environments, clearing all environment variables and restricting access to the file system. If you forget to specify an OpenSSL dependency, the build will fail.
00:20:32.500
This strict mechanism ensures that if the build succeeds, it will also succeed elsewhere. The hash codes associated with packages also ensure that they will be identical regardless of when or where they were built.
00:20:57.360
If you were to build the same package twice and its hash code already exists, NixOS will skip rebuilding it by recognizing that the result will always be the same, improving deployment efficiency.
00:21:19.470
In NixOS, you can compose expressions to create package sets, including all dependencies. By importing the expression that specifies how to build OpenSSH, you can declare its dependencies while building it.
00:21:38.359
As mentioned, in NixOS, the entire package set is bootstrapped, compiling essential packages like GCC, make, etc., isolating them within the Nix store.
00:21:59.750
To build OpenSSH, you would provide a composition expression as a parameter, producing the necessary path in the Nix store, with the hash code reflecting dependencies.
00:22:20.650
The isolated Nix store can be inconvenient for end users, as every package resides in its unique directory with a hash code prefix. For instance, if you want to launch Firefox, you shouldn't have to provide the full path.
00:22:55.420
To address this, NixOS employs a mechanism known as user environments. Users can select their desired packages, and the Nix package manager constructs a unified view, so all packages appear in a single location.
00:23:10.380
When user environments are created, the system generates a symlink that points to the currently used version. Adding this symlink to the path allows users to easily execute programs.
00:23:30.480
For instance, when upgrading OpenSSH, the system builds the new version and stores it alongside existing ones. After the upgrade, the symlink pointing to the current version is updated to the new one.
00:23:49.360
By doing so, if a user runs SSH, it resolves to the new version. The advantage of this method is that it allows atomic upgrades, as the old and new versions exist side by side until the symlink is flipped.
00:24:17.920
At any time, you can remove older generations of packages by executing a command to trigger garbage collection, ensuring that old packages are marked obsolete, allowing safe removal.
00:24:49.280
The architectural organization of NixOS allows for comprehensive management of everything, including kernel modules and configuration files. For instance, if you deploy Apache with PHP, the configuration file becomes a package that depends on the PHP module.
00:25:14.050
NixOS doesn't use traditional directory structures, like /bin or /usr. Instead, everything is contained within the Nix store, although minimal directories exist in the file hierarchy.
00:25:39.290
NixOS's deployment approach can extend to distributed systems. An example of such a system is Trac, which integrates a subversion server with a ticketing system, and can host components across multiple machines.
00:26:04.830
With NixOS, you can run the application components on separate machines and link them over the network. This is possible through a network configuration, allowing multiple machines to communicate effectively.
00:26:40.480
The NixOS tools can facilitate this whole process. By executing just a couple of commands, NixOS can build the configurations for all machines and transfer dependencies and configurations to the remote machines.
00:27:06.700
Once completed, NixOS can switch the machines to their new configurations seamlessly by flipping symlinks, ensuring that if any step fails during the process, it can revert quickly.
00:27:34.580
The NixOS project encompasses more than just the Nix package manager, and NixOS is more than just a Linux distribution. There are additional tools to support this ecosystem.
00:28:04.720
Hydra is a continuous integration server built around the Nix package manager. Additionally, NixOps is an ingenious tool for deploying microservices across networks of machines.
00:28:28.150
While I won't delve into details about these tools today, I wanted to make it clear that NixOS offers a comprehensive range of capabilities.
00:28:42.479
These tools all share the commonality of being driven by automated deployment specifications. They support a variety of programming languages, enabling reproducible deployments and reliable deployments.
00:29:10.409
Because we can store multiple versions of packages and switch almost atomically, the NixOS approach is less destructive compared to traditional methods, while also being efficient.
00:29:35.170
To summarize my earlier question about being declarative, the tools in the NixOS project solve technical problems. For example, the Nix package manager provides a declarative specification of a package.
00:30:06.480
In my company, we are developing a service that allows conference attendees to use an app to explore the conference program, speaker details, and venue maps.
00:30:34.540
We create customizable apps for clients, enabling them to incorporate their branding, logos, and references to their homepages. Instead of dictating technical requirements, we developed a custom tool leveraging the Nix utilities.
00:31:09.700
This allows us to express specifications, such as the conference name, home page, icon set, and background images. The deployment system takes care of producing the necessary iOS and Android apps and other configurations.
00:31:39.540
This philosophy applies to our backend services as well. We have a configurator backend, allowing users to customize content. For larger clients, we also support integrating third-party information systems with our conference app.
00:32:08.520
Instead of manually entering conference details, the system can automatically extract data from clients' existing systems, facilitating a smooth deployment process.
00:32:32.370
In conclusion, I've covered three key aspects: the vision behind declarative deployment, the workings of Nix OS and the Nix package manager, and the possibility of creating domain-specific solutions on top of these systems.
00:32:55.550
For those interested in learning more about Nix OS, everything is available for free and open source on our website, so feel free to experiment with it.
00:33:37.910
To clarify, the Nix package manager can be used on other systems apart from Nix OS, meaning if you’re interested in trying these concepts without switching, you can utilize it on various Linux distributions.
00:33:48.700
You can also implement it on macOS and to some extent on FreeBSD, though support may not be as robust. That's it for my presentation, and now I welcome any questions.
00:34:11.220
I heard about NixOS years ago, and I'm curious if there are any barriers preventing wider adoption of Nix and what challenges the project faces.
00:34:49.510
As for wider adoption, one challenge I've noticed is that many people find the concepts quite unconventional. Several system administrators may not have programming knowledge and may prefer manual control over configuration.
00:35:22.780
Additionally, the Nix package manager relies heavily on functional programming concepts, which can be challenging for some users to grasp. Over the past three years, contributor growth has substantially increased, leading to additional practical challenges.
00:35:43.450
For instance, our build server cluster is overwhelmed with requests, delaying our ability to address bugs effectively.
00:36:08.060
You mentioned Chef being imperative, but the examples provided may illustrate a misunderstanding of Chef's approach. In Chef, configurations are more declarative by declaring the desired end states.
00:36:45.610
The explicit transformation of past states into desired states might give an impression of imperativity, but fundamentally, Chef aims for a balanced method blending both paradigms.
00:37:20.560
Speaking of NixOS, it configures services statelessly, meaning it produces new configurations from declarations, irrespective of past conditions.
00:37:42.680
So, when configuring something like Xorg, it generates the configuration without consideration for the existing state, guaranteeing a clean output based on specified needs.
00:37:58.250
While companies may struggle with the integration of declarative principles like Nix, distinctive strengths lie within how effectively it maintains unique versions of libraries.
00:38:09.890
One of the advantages of using NixOS is the power of package sharing and how package management eliminates duplication. Sharing packages can minimize storage requirements remarkably.
00:38:33.840
However, the need for expressions for deployments could be restrictive. The advantages are compelling for those who are willing to adjust their approaches.
00:39:00.760
It's difficult to generalize a speed difference between Nix and other solutions like Chef for installs or upgrades, as it varies by the nature of the task.
00:39:23.830
Typically, Nix may operate slower due to its unique build and deployment methods compared to the relative speed of Chef.
00:39:43.890
Furthermore, if your systems require high-frequency interactions and constant rebuilding, traditional imperative approaches like Chef may prove invaluable.
00:40:15.700
Nevertheless, when applying Nix principles, the resultant deployments draw from a more assured state of stability ultimately ensuring user satisfaction.
00:40:40.270
NixOS is not strictly limited to managing states on a single machine; concepts extend to facilitate deployments in dynamic cloud environments.
00:40:55.640
Even if deploying starts anew, the benefits of isolated environments and dependency management remain salient long-term.
00:41:27.749
As for the building of packages, no, Nix does not mandate building each package from source; it consults binary caches to provide optimized interactions.
00:41:58.260
Thus, situations where built versions exist in the caches can expedite installations considerably without including unnecessary overhead.
00:42:29.870
The concept of atomic upgrades refers to the atomized handling of package operations, meaning that while individual package management occurs discreetly, a holistic approach must still contend with system-wide characteristics.
00:43:05.100
Consequently, upgrading vital components such as kernels may invite practical complexities alongside the need for reboots, yet NixOS strives to mitigate these concerns to optimize user interaction.
00:43:59.720
Thank you, Sander.