NixOS, if you haven’t heard of it, is a Linux distribution that does things differently from other major distributions. Instead of a traditional package manager and dotfiles (Linux user shorthand for configuration files) scattered everywhere, NixOS centralises all this into configuration-as-code (CasC) in its declarative config/programming language hybrid, Nix. By default, packages can only be installed from the central Nixpkgs repository. With Nix’s “flakes,” you can define custom packages and modules in the Nix language, or import predefined Nix code from sources like GitHub. NixOS may not have a massive market share among Linux distributions (Linux has ~5% desktop share depending on the source; Nix has 0.01%, truly a niche within a niche). Still, the community is passionate and often shares flakes they’ve made. For most applications, you can usually find a starting point from someone else.

As a long-time Linux user interested in declarative systems and configuration-as-code, I’ve followed NixOS since it appeared. Its promise is a centralised, stable, reproducible configuration repository, so I can set up my computer to my preferences with a single command. This also allows for quick re-installs with minimal downtime, or an easy way to remember and install those rarely used tools.

It is more resistant to the “time bloat” I experienced with long-running Linux installations. Sometimes, packages I installed are no longer used, but I don’t notice or know about lingering files. My root partition would slowly bloat with useless data. In NixOS, installed packages are well-defined in a config file that I can commit to Git with comments and messages. This lets me audit what is installed and why. I can also regularly reinstall my system to clear any bloat not defined in the Nix config.

I could probably ramble for another 500 words about why I wanted to switch to Nix. However, that’s not what this post is about. Until now, I hadn’t found time to learn Nix or risk my main system staying broken for several days while I set up my config. That changed this week. I had more annual leave left than I could carry over, so I booked a week off and challenged myself to switch to Nix.

The Switch

On Saturday evening, I took the plunge. Using an ISO, a USB drive, a basic understanding of Nix, and probably too much confidence, I wiped my previous Arch installation, though my home folders were on a separate partition, so it wasn’t a total start-from-scratch. Then I began installing Nix.

What follows is a list of issues I encountered while setting up my Nix install. I could have discovered most of these issues beforehand with more research or by testing in a VM. But I was eager for the switch. Learning by trial by fire works best for me. I should also note that for several of these issues, I am operating in a highly specific niche. Issues are often inevitable, and finding fixes can be difficult.

Packages vs Sub-Modules

The first tripping point I encountered was my understanding of packages vs sub-modules, which is that you can install packages via an array in NixOS, e.g:

environment.systemPackages = with pkgs; [
  pkgs.zsh
  # This will install the zsh shell package
  # but this is not the correct way if you want to configure it at an individual user level 
];

Instead, if you want to configure at the level of individual users, you should install the Home Manager module, which allows for configuring user-level packages and configuration files in their home directories. To configure zsh, I have:

  programs = {
    zsh = {
      enable = true;
      enableCompletion = true;
      syntaxHighlighting.enable = true;
      oh-my-zsh = {
        enable = true;
        theme = "agnoster";
        plugins = [
          "sudo"
          "git"
          "aws"
          "tmuxinator"
        ];
      };
      shellAliases = {
        rm = "rm -I";
      };
      setOptions = [
        "HIST_FIND_NO_DUPS"
      ];
      sessionVariables = {
        EDITOR = "vim";
      };
    };
  };

The programs.zsh.enable attribute enables the zsh package and module in the context of home-manager, so it will not only install zsh, but it will also configure my .zshrc with the settings I’ve configured above. Now, this package/sub-modules issue didn’t confuse me with zsh, as I well knew there was a config file for it, so I knew I’d need to do more than just install it.

It did, however, trip me up with 1Password, as I didn’t consider it having config in the Linux definition. If you use 1Password on Linux, you’ll know you can enable a setting that lets you unlock your vault (if it has already been unlocked since it started running) using system authentication. If you install 1Password via the pkgs method, you will not be able to unlock it via system authentication, even though 1Password gives no indication of that. The setting can be enabled, just when you click the fingerprint button to use system auth, nothing happens. It turned out I had to configure 1Password as follows:

programs._1password-gui = {
  enable = true;
  polkitPolicyOwners = [ "jonsey" ];
  # This setting enables a polkit policy (which is a Linux authentication thing),
  # which in turn enables the system authentication
};

N.B For anyone else making the switch, MyNixOS.com is an excellent resource for finding the attributes for basically everything.

Flakes & Overlays

Now, some preamble for the next tripping point: I use a window manager called Hyprland. It’s a tiling window manager. If you aren’t a Linux user, the details here aren’t too important. Like most Linux window managers and desktop environments, it uses Workspaces (also called Virtual Desktops). After switching from KDE Plasma to Hyprland, I found workspaces worked differently. On KDE, each monitor had its own set of workspaces. On Hyprland, workspaces across all monitors are tied together. Luckily, I found a Hyprland plugin called split-monitor-workspaces. On my old Arch setup, installing this was simple: just run hyprpm add and hyprpm enable. This wouldn’t work on Nix, since the hyprpm tool wasn’t shipped with Hyprland. Also, installing via another package manager isn’t the “Nix way.” Even though several plugins are defined in the Nixpkgs repository, split-monitor-workspaces is not included.

This required me to use what NixOS defines as a Flake, so I could add an Overlay, which is a fancy way of saying add custom packages, as you “overlay” it onto your system. The Flake allows you to define it in a structured way and also generates a lock file (to ensure repeatable builds). Luckily for me, the developer of split-monitor-workspaces not only defined a package for the plugin, but also added a section to their README explaining how to use it in a flake. This did take a few more attempts to get right than I’d have liked, partly due to my lack of understanding of flakes and my system still pulling in a version of Hyprland that was not compatible with the plugin (you need to define Hyprland as a flake input like the plugin, not just as a follows for the plugin).

Honourable Mention Incidental Issue (Which would also have been easier to resolve on Arch)

I’ve long used a YubiKey, a physical security device that provides an additional verification step, as a second factor for authentication on my system. I use it to log in and to escalate permissions using either sudo or Polkit (a toolkit for defining and handling authorisations). I found great documentation in the Nix wiki for setting up YubiKey second-factor authentication. After installing the needed packages and configuring PAM (Pluggable Authentication Modules for Linux, which control authentication), things worked great for login and sudo. But with Polkit, it would silently ignore the YubiKey requirement and continue.

For a while, I thought this was a NixOS issue. Then I found This GitHub Issue, which shows that the latest version of polkit makes a change that stops it from using external devices for authentication. After some digging, I found a Nix configuration I could change to override this and get Polkit working properly. This wasn’t a NixOS issue. However, on Arch, I would have identified it as Polkit much more easily and could have downgraded and pinned an earlier version until a fix was available. In Arch, the package maintainers eventually fixed it.

Streamdeckd and Golang vendoring

I have long been the maintainer of Streamdeckd, a plug-and-play Linux driver for the Elgato Streamdeck that also features a plugin system to allow developers to create their own “handlers”. For anyone unfamiliar with Golang’s plugin system, it is famously flaky, and if plugins aren’t built with the exact same version of Go, with the exact same package versions, and the exact same build flags, then Golang will throw (well, return) an error when it tries to load them. Now, hold that thought of plugins being flaky, to get Streamdeckd setup as a package on NixOS, I defined my own package (or Derivation as they’re called in Nix), using the buildGoModule builder function, which takes a pointer to a go module (either a direct path, or a GitHub repository with the fetchFromGithub fetcher), and builds it, the package setup I had looked something like this:

{ lib, buildGoModule, fetchFromGitHub }:
let
    pkgs = import <nixpkgs> {};

in buildGoModule rec {
  pname = "streamdeckd";
  version = "master";

  buildInputs = [ pkgs.libudev-zero ];

  nativeBuildInputs = [ pkgs.pkg-config ];

  src = fetchFromGitHub {
    owner = "unix-streamdeck";
    repo = "streamdeckd";
    rev = "master";
    hash = lib.fakeHash;
  };

  vendorHash = lib.fakeHash;

  meta = with lib; {
    description = "Elgato Streamdeck Driver for Linux";
    license = licenses.bsd3;
    platforms = platforms.linux;
  };
}

buildGoModule requires the go module to be vendorable (or already vendored), which is Golang terminology for package source code being stored locally to the go module it is imported to, a bit like node_modules for npm, but, as Golang has no central package store like npmjs, and instead links to git repositories are used for importing modules, where developers could easily unpublish, or alter tags of modules. This means it can be recommended that the vendor directory be stored in version control to reduce the risk of changes in external repositories breaking the module.

As NixOS’s whole ethos is reproducible systems, vendoring is required for building Go modules to ensure no 3rd-party module changes the outcome. This is all perfectly reasonable; however, with streamdeckd, somehow vendoring the main streamdeckd repository would break loading any plugins, resulting in some errors. Over the years, I’ve become quite dependent on my streamdeckd plugins, so this was almost a breaking point for me. I did, however, after a lot of trial and error, and reaching out to some far smarter Golang developers than myself, discover that having the plugins requiring streamdeckd as a dependency probably was causing the issue, and instead having the plugins and streamdeckd depend on a 3rd module that defined the plugin interface fixed things.

Now, this issue reads as one with my bad Golang code, instead of one with NixOS, and that is entirely correct, but it’s an issue I wouldn’t have encountered on any system but NixOS, and as such, it frustrated the process of migrating quite a bit, especially when combined with the next issue. Luckily, I was able to resolve the issue and, in the process, make a fair few improvements to Streamdeckd that I otherwise wouldn’t have.

Closures

Nix packages are built and run in what are called closures, which are, to the best of my current understanding, akin to a virtual environment, where the package has visibility of only the packages it depends on to work, and no others, as this helps prevent side effects and aids Nix’s stability. Again, very good idea, part of the reason I wanted to switch to Nix. This, however, came back to bite me when I tried to fix streamdeckd. I use JetBrains Goland to work on Go, and its closure does not include libudev, which streamdeckd depends on. This meant I could not run or debug streamdeckd via Goland, and instead wrote the code in Goland, then would go build and run the code from my terminal, not a great experience.

Based on the initial research I did, it’s not exactly easy to inject additional libraries into a closure, nor is it entirely recommended, and I still haven’t found a solution to this issue I’m entirely happy with. What I have resorted to is a “Before Launch” script that runs nix-shell, which lets you run a shell with additional packages you can specify; e.g., nix-shell -p libudev-zero runs a shell session with libudev. This does work, for the most part, but feels more like a hack than I’d like.

Where I’m at now

All-in-all, this whole migration experience did turn out to be a bit more “the grass is always greener” than I anticipated, while I did manage to fix most of the issues I listed above, and none were really Nix’s fault, and could likely have been discovered before I attempted the migration, it does leave a concern in my mind that there’s more blockers in my workflow I am yet to discover, and unlike other Linux distributions, where these kind of issues are usually quite quick to fix, on Nix, at least at my current level of knowledge, tend to open into a rabbit hole, when I might not be able to afford the time investment to fix it. I still believe in Nix’s vision, and on balance, there are more pros than cons, so I’m going to stick with Nix for the moment, but time will tell whether it is my next long-term Linux distribution of choice.

#Ramblings#Linux