Published at 2021-11-27
NixOS is amazing because you can re-use your configuration across multiple systems. Put it in a git repo and you got yourself a nice sync mechanism across all of your systems.
However, you still have to manually deal with setting up drive encryption and partitioning every time you set up a new device.
After doing this manually every time and figuring it out again every time from scratch, I decided to write my process of doing this down.
The first step is to get a nixos iso image to boot from and install nixos.
Download the iso from the nixos home page and “burn” it to a usb drive (remember the days when burning dvds with the latest debian was the thing?). I went with the gnome image here but the other ones are completely fine too.
dd makes this very easy (assuming /dev/sde is the usb drive you want to use as your install medium):
This will completely wipe your usb drive.
dd if=nixos-gnome-21.05.4381.3bea86e918d-x86_64-linux.iso of=/dev/sde status=progress
Put the usb drive in the pc you want to install nixos on, go in the bios and choose the usb device as a boot medium. If everything works as expected, you should land on the gnome (or other) home screen.
Open up a terminal. Figure out what drive you want to use with fdisk -l. You’ll need to use the entire disk, not single partitions.
We’ll use gdisk to create the partitions. gdisk is an interactive utility, meaning you get a kind of prompt to do the partitioning.
Run
gdisk <drive>
to start it.
We’ll create two physical partitions with gdisk:
The last one will be a container which will hold nixos and the swap partition.
Quick primer on gdisk:
p to show the current statusd to delete a partitionn to create a new partitionw to write everything once doneFirst, remove all existing partitions from the drive with d. It will ask you every time about the partition you want to delete.
Then create the following partitions:
| Number | Type | Size | What |
|---|---|---|---|
| 1 | ef00 |
+500M | The (u)efi partition |
| 2 | 8300 |
The one partition that’s going to hold the os and swap |
It should look something like this at the end (output from p):
Disk /dev/nvme0n1: 1953525168 sectors, 931.5 GiB
Model: Samsung SSD 980 1TB
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 58075B91-4D6F-4DA0-8BC2-CA7243F1ADB0
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)
Number Start (sector) End (sector) Size Code Name
1 2048 1026047 500.0 MiB EF00 EFI system partition
2 1026048 1953525134 931.0 GiB 8300 Linux filesystem
Command (? for help):
Next, we’re going to create an encrypted root container for the os/everything else partition with cryptsetup.
If your output looks like mine from above, this is the third partition.
Create the encrypted container:
cryptsetup luksFormat <device>2
And open it:
cryptsetup open <device>2 nixenc
cryptsetup will ask you for a password on both commands.
You will need to enter this after your system is installed on every boot.
Once the container is open, you have a /dev/mapper/nixenc device available as if it was a normal disk.
Note that we specified the last part of that in the cryptsetup open command.
We’ll use a volume group to hold the swap and root partition. We could encrypt them individually, but using a volume group won’t require us to enter the password multiple times when booting the computer.
First, we’ll tell lvm to handle the luks device we just formatted as if it was a physical partition:
pvcreate /dev/mapper/nixenc
Then we’ll create the actual volume group and call it vg:
vgcreate vg /dev/mapper/nixenc
Now that we have a volume group, we can finally create the new volumes:
lvcreate -n swap -L 8GB vg # the swap partition
lvcreate -n root -l +100%FREE vg # root partition with the os and everything else
Both of these new volumes will appear at /dev/mapper/vg-swap and /dev/mapper/vg-root to format and use them.
To actually use the volumes, you need to format them.
First, set up the boot partition on the first device:
mkfs.vfat -n boot <device>1
Then create and enable the swap partition:
mkswap /dev/mapper/vg-swap
swapon /dev/mapper/vg-swap
Enabling it will make nixos-generate-config detect it and put it in your hardware-configuration.nix. And you’ll be able to use it during the installation.
Lastly, create the actual btrfs root partition:
mkfs.btrfs -L root /dev/mapper/vg-root
If you want to set up brtfs subvolumes, now is a good time for that.
Mount the new btrfs partition to /mnt:
mount /dev/mapper/vg-root /mnt
And mount the uefi partition to /mnt/boot:
mkdir /mnt/boot
mount <device 1> /mnt/boot
Then run
nixos-generate-config --root /mnt
to generate a new nixos config.
If you already have a nixos config in a git repo somewhere (like I do), you should clone it now.
I clone my nix config to /var/nix and then symlink it to /etc/nixos/configuration.nix so that nixos will pick it up and use it.
Note that you need to clone the repo to /mnt because that’s where we the root os partition is mounted:
mkdir /mnt/var
cd /mnt/var
git clone <path to the repo>
To create the symlink, it’s important to create one with a relative path - nixos is not yet installed in / but in /mnt.
I usually do something like this:
cd /mnt/etc/nixos
mv configuration.nix configuration.generated.nix
ln -s ../../var/nix/configuration.nix configuration.nix
Usually, it’s a good idea to take a look at the auto generated hardware-configuration.nix and add it to the already existing config because
it has all disks and everything else detected by nixos-generate-config.
You might have references to packages from the nixos unstable channel in your config.
I usually add the unstable channel to my nix channels as nixos-unstable.
If you don’t have that channel available in nix channels, the installation will fail. To add it:
nix-channel --add https://nixos.org/channels/nixos-unstable nixos-unstable
Refresh the channels so they are actually usable:
nix-channel --update
This is something that I ran into with my specific setup - if you’re starting from scratch you shouldn’t need it. The generated
hardware-configuration.nixshould contain the correct settings.
To tell grub the device it should boot from we need to tell it the root device. In order to do that, first figure out which uuid is has.
We’re going to use lsblk for that:
$ lsblk -o name,type,mountpoint,uuid
NAME TYPE MOUNTPOINT UUID
loop0 loop /nix/.ro-store
sda disk 1980-01-01-00-00-00-00
├─sda1 part /iso 1980-01-01-00-00-00-00
└─sda2 part 1234-5678
nvme0n1 disk
├─nvme0n1p1 part /mnt/boot 8C6D-DD63
└─nvme0n1p2 part d6f3e071-f449-4aab-87f4-93ee3a3fbab1 # This is the uuid we're looking for
└─nixenc crypt qtCMVj-QKcW-0rcm-Pyud-Fqzc-tA8f-inZp3M
├─vg-swap lvm [SWAP] a7208e31-c1e7-44b8-895c-d01d0b930508
└─vg-root lvm /mnt
Add the following entry to a boot.nix or hardware-configuration.nix file:
boot.initrd.luks.devices = {
root = {
device = "/dev/disk/by-uuid/<the uuid of the root partition from above>";
preLVM = true;
allowDiscards = true;
};
};
Now that everything is set up, we can actually install the nixos system with
nixos-install
Depending on your configuration, internet speed and hardware, this will take a while.
It may look like no progress is made right after starting the installation. In my case, this was due to my installation referencing the nixpkgs git repo somewhere. With over 300k commits and at 2GB that takes a while to clone.
(I really should optimize this)
Once nixos-install has finished, reboot your system.
If everything went well, it should greet you with a login screen.
You probably have users created in your nix config. To be able to log in with them, they need a password.
After booting, when greeted with the login screen, switch to a terminal with ctrl + alt + some F key.
Log in as root with the password you specified during nixos-install.
Change the password of your daily user with
passwd <username>
You should now be able to use that user to log in with the login screen.
There’s two additional things you might want to consider when mounting the btrfs root partition: compression and noatime.
Every time you access a file linux will save the current time as the last time you accessed that file (that’s called atime).
Because btrfs is a Copy-on-Write filesystem it copies the full metadata of a file every time you access it in order to save the atime.
If you visit a lot of files, this might lead to bad performance.
To prevent linux from saving the atime you can set the noatime option when mounting the file.
In general, I didn’t notice this to be a problem in day-to-day use, especially with a ssd.
Check out the BTRFS-Wiki for more information about this.
BTRFS supports compression with a few different algorithms. If enabled, it will transparently compress everything on the drive which will give you some additional space for storing files. This might come in handy with nixos in particular which generally stores a bit of files after a while.
Note that if you enable it, it will only compress newly saved files so you will only notice it after using it a while. It won’t help you if you’re already running low on disk space.
I am currently not using compression in my daily driver computers but have it enabled for my nas (which is also running nixos) and had a good experience with it so far. I can’t say much about the performance but I think it will have a performance impact depending on the type of compression used.
If you have numbers or experiences about this, please hit me up.
If you’re not using home-manager as part of your system configuration, you’ll need to set it up once you’re booted and logged in to your new system. I also have an config for that.
First, we need to add the home manager channel:
nix-channel --add https://github.com/nix-community/home-manager/archive/release-21.11.tar.gz home-manager
nix-channel --update
Then create a symlink from the existing config to ~/.config/nixpkgs/home.nix:
mkdir ~/.config/nixpkgs -p
ln -s /var/nix/home-manager/home.nix ~/.config/nixpkgs/home.nix
Install home manager:
nix-shell '<home-manager>' -A install
If that fails, you need to log out and in again so that nix correctly finds the home-manager nixos channel.
Run home-manager switch to install the new config.
If something doesn’t work and your new installation won’t boot, grab that usb key from earlier and boot in the desktop again. Open up a terminal.
To be able to modify your config inside of the encrypted luks container we need to open, mount it and mount the boot partition:
cryptsetup luksOpen <device 2> nixenc
mount /dev/mapper/vg-root /mnt
mount /dev/<device 1> /mnt/boot
swapon /dev/mapper/vg-swap
Now you can modify your config and run nixos-install when done to reinstall everything.
Nixos is smart enough to figure out what changed and will only reinstall those parts - the new reinstallation shouldn’t take that long.
And there you have it. A working nixos system, complete with encryption and home-manager.
I’m far from being a nixos expert. If you have any suggestions about this post or setup in general, please send them to me!