Nix: Container maintenance and management

Video transcript

We utilized ChatGPT to enhance the grammar and syntax of the transcript.

Hello, hello everyone! I hope you had a good coffee break. Thank you for attending this talk. Today, I'm going to talk about Nix and a little bit about taming container maintenance.

Who am I? I’m Jérôme Vieilledent, a Product Manager at Platform.sh. Yes, I know there are a lot of Platform.sh speakers, but feel free to reach out to me anytime if you want to discuss product, Platform.sh, or just chat. I’m here at the booth.

To introduce this talk, I need to explain why we came to this technology, which some might consider awkward, but it’s actually awesome. We’ll be discussing Nix and its role in managing containers at Platform.sh.

First, let's talk about containers at Platform.sh. We are using containers, but we're not using Docker; instead, we use LXC (Linux containers). This decision was made for historical reasons. When Platform.sh started, Docker was barely experimental, but LXC was more stable and well-known, similar to BSD jails or Solaris zones.

Yes, we use containers, but we also have an industrialized orchestration system. We don't use Kubernetes; instead, we have a piece of software called 'Manufacturer,' which I hope we’ll open source someday. It orchestrates everything on our platform.

Another fact is that we operate a horizontal platform-as-a-service (PaaS). Our goal is to be a platform that can host any type of application, regardless of the runtime. This talk will mainly focus on PHP, but our platform supports many other runtimes. Maintaining these containers is a challenge. For PHP alone, we have 11 different images, each with varying versions of Debian and their dependencies. It’s even possible (though not recommended) to host a PHP 5.4 application.

In total, Platform.sh supports many runtimes—71 images just for applications, and 95 more for services such as databases (Redis, Memcached), search engines (Solr, Elasticsearch), and more. Altogether, that’s 166 images to maintain. We have three people who maintain all of these full-time, and they’re very good at what they do.

But that's not all. Your applications need more than just PHP. For instance, you might need tools like ImageMagick for image manipulation or something to convert files to PDFs. Let’s talk about the additional packages you may need in your containers.

Here’s my Christmas wish list: I want PHP 8.3, which I always run because I like being on the bleeding edge. But I also need PHP 7.2 for an old build script I’ve been using for a decade. I want ImageMagick, but I want version 6, not version 7, because I’m too lazy to deal with the breaking changes. And I need LibreOffice to generate PDFs.

But there’s a problem. These different dependencies don’t always work well together. What you end up with is something we call "dependency hell." Imagine trying to compile all of these different versions of PHP and their extensions—along with older Python versions—all at once. It would likely break, resulting in what we call an XKCD moment, where you realize you're building an enormous chain of dependencies just to support itself.

So, what's the solution? Well, there is one: it's called Nix.

Okay, so what is Nix, and why is it so awesome? Nix is a functional package manager. What does that mean? First, it’s supported by a functional programming language. This allows packages to be treated like values, and since it’s functional, all functions used to generate packages (which we call “derivations” in Nix) cannot have side effects. Everything is built in total isolation.

How do we achieve that? First, it’s not possible to have undeclared dependencies. If you’ve used .deb or .rpm files before, you might have run into this issue where dependencies aren’t properly declared. That can also happen with package managers like Homebrew. But in Nix, it’s impossible to build something without declaring it explicitly.

The good thing about Nix is that you can finally say, “It works on my machine,” and mean it. If it works on your machine, it will work on another machine too. It provides consistency across different environments.

Another key aspect is that all packages are built in total isolation. How? Nix uses something called the Nix store, typically located at /nix/store. All packages are stored there, and each package is associated with a hash that uniquely identifies it and its dependencies. This hash is based on a SHA-256 checksum, and it captures the package and all its dependencies, ensuring isolation.

This approach also allows for multiple versions of the same package to coexist. You could have PHP 5.4 running alongside PHP 8.3 without conflict. That’s incredibly difficult to achieve with traditional package managers.

Nix is supported by a functional language, but you don’t need to learn much about it to use it. The more you dig into Nix, however, the more you’ll encounter “Nix expressions,” which are files written in this functional language. It’s a simple language, somewhat similar to JSON but more powerful. These Nix expressions are evaluated to create derivations. Each time you build a Nix package, the Nix expression you’ve written (specifying your packages and any shell hooks you need) becomes a new derivation in the Nix store.

Nix is also backed by Nixpkgs, the largest package repository in the world, with over 80,000 packages. It’s essentially a massive GitHub repository that Nix queries whenever you want to install something. By default, everything is built from the ground up (compiled), but there’s also a cache of pre-built binaries for most platforms, so you won’t always need to compile things yourself.

To give you an example, here’s an old picture showing Subversion and its dependencies. On the left, you see how it’s supposed to work. You want the “hello” package, which depends on libc (because everything depends on libc), and then there’s another dependency for a specific version of libgmp. The problem is that if you rely on the libc from your operating system, you might run into issues when trying to compile something that expects a different version.

This kind of dependency conflict happens often, especially with packages like Subversion, which may need a specific version of OpenSSL or Berkeley DB. But with Nix, everything is isolated and versioned, so you avoid this type of conflict entirely. Each package gets its own isolated environment, and thanks to the hash-based system, Nix can handle multiple versions of the same package effortlessly.

All right, so now let’s try something practical, a little demo. I still want my PHP 8.3 because it rocks, and I want my "eerie" extension. I chose one called SOAP—nobody uses SOAP anymore, right? Except in the shower, of course. But it’s a good example.

Since I’m running on a Mac, I still want Python 2.7 (even though it’s outdated). Spoiler alert: if you try to install it with Nix, it will warn you that it’s not safe, and you’ll need to override it. So, I’ve added a specific environment variable for that. I also want ImageMagick, but I’ve decided to drop LibreOffice for this demo because it takes too long to build.

Let’s give it a try. For this, I’m going to use nix-shell. Nix-shell is a command used to create a temporary environment in your terminal with the packages you want. I’ll put it in verbose mode so you can see what’s happening. I’ve pre-downloaded everything so we don’t have to wait 10 minutes for it to install.

I’ve said I want PHP 8.3. So, I need the PHP 8.3 package along with the SOAP extension. I also want Composer because I’m going to need that, and I’ll add Python 2.7 and ImageMagick 6.

What Nix does here is evaluate all the Nix expressions for every dependency of the few packages I’m installing. As you can see, there are a lot of dependencies. This is what’s happening behind the scenes in your operating system when you try to build something.

Let’s check what I have on my system right now without Nix. I have PHP 8.2 installed, but I only have Python 3, and no python command is available. I also don’t have ImageMagick, which we’ll need later.

Now, after using Nix, I have PHP 8.3, ImageMagick 6.7 (which took a long time to compile), and Python 2. I’ve also installed the SOAP extension, so we can take a metaphorical “shower” now.

That’s basically what Nix does—it sets up a working environment with all the packages you need. But specifying each package manually for every shell session can be tedious. It’s great for testing new software, but you’ll want to automate it at some point. There are several ways to do this, but one of the simplest methods is to create a script that loads everything for you.

I’ve created a very simple Nix shell script that declares all my inputs and dependencies. It even includes a shell hook that prints “Hello Symfony” when it’s run. Now, instead of typing all the options manually, I can just run nix-shell, and everything is set up for me automatically.

That was the fancy part of the demo. We could try it with LibreOffice, but we don’t have time today. What this all means is that with Nix, for our own systems, we can reduce the number of images we need to maintain from 166 to just one composable image. That’s a huge win for us.

Of course, we might still maintain a few different images for specific services, but the point is that Nix allows us to manage everything with a single image that can be customized for various needs. And if you're using Docker, you can even use Nix to generate proper Docker images, built from your packages rather than starting with a Docker image and adding things later. There’s also a Nix image for Docker itself, making it much easier to add packages and tools to your stack.

This is not only useful for us in maintaining our infrastructure with all those containers, but it’s also useful for everyone who encounters dependency hell. I'm sure you've all experienced that issue where you're trying to compile something, but it needs a specific library version, which depends on another library version, and so on, until you finally give up. That's why Docker was invented, right? To deal with dependency hell. But it can still happen, even in Docker.

So, that’s basically it. We’ve covered how Nix can simplify dependency management and container maintenance. Well, I’m done a little early, so I’d be happy to take any questions.

We’ve got some time for questions. Oh, we have the cube for passing the microphone around.

Q&A

Question: What base image are you using for your LXC containers? Is there a specific base image?
Answer: Great question! We’re using Debian as the base image. We’re running LXC, and the base image is pretty much raw Debian, so it’s built from scratch.

Question: You mentioned that you can create a Nix configuration from a machine. Does that mean it will fix the versions of every package you have installed on that machine indefinitely?
Answer: Yes, that’s exactly right—until you decide to upgrade. Nix allows you to manage your packages in a way that ensures consistency. And that brings me to something I didn’t mention earlier: Nix can also be used as a full operating system. You can use it to manage all your packages and upgrade whenever you want.

Your question touches on version pinning. If you specify PHP 8.3, for example, it will continue to use PHP 8.3 until the end of its life cycle. At that point, you’d need to upgrade to PHP 8.4 or whatever the next version is. The same applies to other packages. For instance, LibreOffice has two channels: the fresh channel (with the latest features) and the stable channel. You can choose which channel you want to follow.

And even if you’re following the main Nix packages channel, you can always pin a specific package version. To do that, you just reference the commit in Nixpkgs that corresponds to the version you want, and you’re done. The package won’t upgrade unless you manually change the pinned version.

Question: How do you share your configurations between developers? Do you commit a Nix file with everything included, or do you build something and then share it as a package?
Answer: The easiest way is to write a Nix expression, like I did in the demo. But instead of doing it just for nix-shell, you’d create a more comprehensive file, probably called default.nix. This file would define your entire stack. Then, anyone on your team can simply run nix-build or whatever command you set up, and it will build everything according to that default.nix file.

You can even make it easier by creating a Makefile that runs the Nix command. That way, anyone can just type make up or something similar, and they’ll have the same stack as you. It’s very simple to share across teams.

Question: Is there any compatibility process with Docker images or Docker files?
Answer: Not directly, as far as I know. But Nix has tools that can help you generate Docker images, so all the layers of the image will be built for you by Nix. That’s one way of doing it. So, while there’s no direct conversion from Docker files, you can still use Nix to build Docker images.

Question: How do you deploy Nix? Is there a platform as a service for Nix?
Answer: For Platform.sh, we’re about to launch Nix as an experimental feature. In a few weeks, you’ll be able to define your stack in your YAML file using Nix. Instead of specifying all your dependencies manually, you’ll just say, “I want PHP 8.3 and these other packages,” and it will automatically use the Nix image.

We’re still working on it, but in the future, you might even be able to write custom Nix expressions directly in your YAML file. And yes, there’s also the potential for using Docker alongside Nix. It’s still evolving, but Nix is coming to Platform.sh very soon.

Question: If I want to use Nix in both my development and production environments, what’s the best way to deploy it right now? Should I just push my default.nix file, or is there another approach?
Answer: At the moment, it’s not fully supported for production deployments. However, I’d love to discuss this further with you because we’re very interested in enabling Nix-based deployments in production. We’d like to make it possible to evaluate Nix expressions for your containers at build time, so let’s talk afterward about how we can make that happen.

Question: How does Nix work under the hood? Is it using some sort of preload magic or static binaries for dependencies?
Answer: Good question! Nix uses the cache from cache.nixos.org. There’s a huge platform that pre-builds all the common packages and dependencies. If the package you need is not available in the cache, it will just compile it for you. And if you want more control, you can always set up your own cache. This allows you to pre-build all your dependencies and have your local infrastructure pick them up first before NixOS’s cache.

Jérôme: All right, thank you for all the great questions, folks. That’s a wrap for today’s talk. Thank you again for attending, and remember, if you want to dive deeper into Nix, come find me or the other Platform.sh team members here. And don’t forget, the PHP stack maintainer for Nix is also in the room—he’s hiding, but he’s here, and he has some great stories to share!

Thank you, and I hope you enjoy the rest of the event!