extern/nixos-and-flakes-book
1
0
Fork 0
mirror of https://github.com/ryan4yin/nixos-and-flakes-book.git synced 2025-08-18 00:19:43 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat: add new chapter - development

Browse Source
This commit is contained in:
Ryan Yin
2023-06-27 10:53:11 +08:00
parent 1b5937ec65
commit 5d0a512393
15 changed files with 1491 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/development/cross-platform-compilation.md Normal file
View File

@@ -0,0 +1,6 @@
## Cross-platform compilation
First of all, on any Linux platform, there are two ways to do cross-platform compilation.
Taking the building of an`aarch64-linux` program on a `x86_64-linux` host as an example, the two methods are described as follows:
TODO

11
docs/development/dev-environments.md Normal file
View File

@@ -0,0 +1,11 @@
## Dev Environments
We have learned how to build development environments, but it's a bit tedious to write `flake.nix` for each project.
Luckily, some people in the community have done this for us. The following repository contains development environment templates for most programming languages. Just copy and paste them:
- [dev-templates](https://github.com/the-nix-way/dev-templates)
If you think the structure of `flake.nix` is still too complicated and want a simpler way, you can also consider using the following project, which encapsulates Nix more thoroughly and provides users with a simpler definition:
- [cachix/devenv](https://github.com/cachix/devenv)

150
docs/development/distributed-building.md Normal file
View File

@@ -0,0 +1,150 @@
## Distributed Building
Distributed building can speed up the build process by make use of multiple machines.
NixOS's official cache.nixos.org provides the vast majority of caches for the X86_64 architecture, so distributed builds are generally not very useful for ordinary NixOS users.
Distributed building is of great application value only in scenarios where there is no cache available:
1. Users of RISC-V or ARM64 architectures (especially RISC-V), because there are very few caches for these two architectures in the official cache repository, which often requires a lot of local compilation.
2. Users who customize the system a lot, because the packages in the official cache repository are all default configurations, if you change the build parameters, then the official cache is not applicable, then you need to compile locally.
2. For instance, in the embedded scenario, there is often a need for customization of the underlying kernel, drivers, etc., which leads to the need for local compilation.
### Configure distributed building
Currently, there is no official documentation for this, and I have listed some recommended reference documents at the end of this chapter, along with my distributed build configuration (a NixOS Module):
```nix
{ ... }: {
####################################################################
#
# NixOS's Configuration for Remote Building / Distributed Building
#
####################################################################
# set local's max-job to 0 to force remote building(disable local building)
# nix.settings.max-jobs = 0;
nix.distributedBuilds = true;
nix.buildMachines =
let
sshUser = "ryan";
# ssh key's path on local machine
sshKey = "/home/ryan/.ssh/ai-idols";
systems = [
# native arch
"x86_64-linux"
# emulated arch using binfmt_misc and qemu-user
"aarch64-linux"
"riscv64-linux"
];
# all available system features are poorly documentd here:
# https://github.com/NixOS/nix/blob/e503ead/src/libstore/globals.hh#L673-L687
supportedFeatures = [
"benchmark"
"big-parallel"
"kvm"
];
in
[
# Nix seems always give priority to trying to build remotely
# to make use of the local machine's high-performance CPU, do not set remote builder's maxJobs too high.
{
# some of my remote builders are running NixOS
# and has the same sshUser, sshKey, systems, etc.
inherit sshUser sshKey systems supportedFeatures;
# the hostName should be:
# 1. a hostname that can be resolved by DNS
# 2. the ip address of the remote builder
# 3. a host alias defined globally in /etc/ssh/ssh_config
hostName = "aquamarine";
# remote builder's max-job
maxJobs = 3;
# speedFactor's a signed integer
# but it seems that it's not used by Nix, takes no effect
speedFactor = 1;
}
{
inherit sshUser sshKey systems supportedFeatures;
hostName = "ruby";
maxJobs = 2;
speedFactor = 1;
}
{
inherit sshUser sshKey systems supportedFeatures;
hostName = "kana";
maxJobs = 2;
speedFactor = 1;
}
];
# optional, useful when the builder has a faster internet connection than yours
nix.extraOptions = ''
builders-use-substitutes = true
'';
# define the host alias for remote builders
# this config will be written to /etc/ssh/ssh_config
programs.ssh.extraConfig = ''
Host ai
HostName 192.168.5.100
Port 22
Host aquamarine
HostName 192.168.5.101
Port 22
Host ruby
HostName 192.168.5.102
Port 22
Host kana
HostName 192.168.5.103
Port 22
'';
# define the host key for remote builders so that nix can verify all the remote builders
# this config will be written to /etc/ssh/ssh_known_hosts
programs.ssh.knownHosts = {
# 星野 愛久愛海, Hoshino Aquamarine
aquamarine = {
hostNames = [ "aquamarine" "192.168.5.101" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDnCQXlllHoLX5EvU+t6yP/npsmuxKt0skHVeJashizE";
};
# 星野 瑠美衣, Hoshino Rubii
ruby = {
hostNames = [ "ruby" "192.168.5.102" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE7n11XxB8B3HjdyAsL3PuLVDZxWCzEOUTJAY8+goQmW";
};
# 有馬 かな, Arima Kana
kana = {
hostNames = [ "kana" "192.168.5.103" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ3dDLOZERP1nZfRz3zIeVDm1q2Trer+fWFVvVXrgXM1";
};
};
}
```
## Defects
The problems I have observed so far are:
1. You cannot specify which hosts to use at build time, you can only specify a list of hosts in the configuration file, and nix automatically selects available hosts.
two。
2. When choosing a host, I found that Nix always preferred the remote host, while my local host had the best performance, which caused the local host's CPU to be underutilized.
3. The smallest unit of distributed building is Derivation, so when building some big packages, other machines may be idle for a long time, waiting for the big package to be built, which leads to a waste of resources.
## References
- [Distributed build - NixOS Wiki](https://nixos.wiki/wiki/Distributed_build)
- [Document available system features - nix#7380](https://github.com/NixOS/nix/issues/7380)
- [Distributed builds seem to disable local builds nix#2589](https://github.com/NixOS/nix/issues/2589)
- [Offloading NixOS builds to a faster machine](https://sgt.hootr.club/molten-matter/nix-distributed-builds/)
- [tests/nixos/remote-builds.nix - Nix Source Code](https://github.com/NixOS/nix/blob/713836112/tests/nixos/remote-builds.nix#L46)

291
docs/development/intro.md Normal file
View File

@@ -0,0 +1,291 @@
## Development on NixOS
Due to the reproducibility of NixOS itself, it is very suitable for building development environments.
But if you want to migrate the experience on other distros directly to NixOS, you may encounter problems, because NixOS has its own set of logic, we will explain this briefly below.
On NixOS, it is recommended to install only common tools in the global environment, such as `git`, `vim`, `emacs`, `tmux`, `zsh`, etc., while the development environment of each language should be an independent environment for each project.
You should NOT install the development environment of each language in the global environment, and the project environment should be completely isolated from each other, and will not affect each other.
We will introduce how the development environment works in NixOS in the following sections.
### Create a development environment
We can create a development environment via `pkgs.mkShell { ... }`, and open an interactive Bash Shell of this development environment via `nix develop`.
Let's take a look at [the source code of `pkgs.mkShell`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/mkshell/default.nix) to see how it works:
```nix
{ lib, stdenv, buildEnv }:
# A special kind of derivation that is only meant to be consumed by the
# nix-shell.
{ name ? "nix-shell"
, # a list of packages to add to the shell environment
packages ? [ ]
, # propagate all the inputs from the given derivations
inputsFrom ? [ ]
, buildInputs ? [ ]
, nativeBuildInputs ? [ ]
, propagatedBuildInputs ? [ ]
, propagatedNativeBuildInputs ? [ ]
, ...
}@attrs:
let
mergeInputs = name:
(attrs.${name} or [ ]) ++
(lib.subtractLists inputsFrom (lib.flatten (lib.catAttrs name inputsFrom)));
rest = builtins.removeAttrs attrs [
"name"
"packages"
"inputsFrom"
"buildInputs"
"nativeBuildInputs"
"propagatedBuildInputs"
"propagatedNativeBuildInputs"
"shellHook"
];
in
stdenv.mkDerivation ({
inherit name;
buildInputs = mergeInputs "buildInputs";
nativeBuildInputs = packages ++ (mergeInputs "nativeBuildInputs");
propagatedBuildInputs = mergeInputs "propagatedBuildInputs";
propagatedNativeBuildInputs = mergeInputs "propagatedNativeBuildInputs";
shellHook = lib.concatStringsSep "\n" (lib.catAttrs "shellHook"
(lib.reverseList inputsFrom ++ [ attrs ]));
phases = [ "buildPhase" ];
# ......
# when distributed building is enabled, prefer to build locally
preferLocalBuild = true;
} // rest)
```
`pkgs.mkShell { ... }` is a special Derivation (Nix package). Its `name` `buildInputs` and other parameters are customizable, and `shellHook` is a special parameter that will be executed when `nix develop` enters the environment.
Here is a `flake.nix` that defined a development environment with nodejs 18 installed:
```nix
{
description = "A Nix-flake-based Node.js development environment";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05";
};
outputs = { self , nixpkgs ,... }: let
# system should match the system you are running on
# system = "x86_64-linux";
system = "x86_64-darwin";
in {
devShells."${system}".default = let
pkgs = import nixpkgs {
inherit system;
overlays = [
(self: super: rec {
nodejs = super.nodejs-18_x;
pnpm = super.nodePackages.pnpm;
yarn = (super.yarn.override { inherit nodejs; });
})
];
};
in pkgs.mkShell {
# create an environment with nodejs-18_x, pnpm, and yarn
packages = with pkgs; [
node2nix
nodejs
pnpm
yarn
];
shellHook = ''
echo "node `${pkgs.nodejs}/bin/node --version`"
'';
};
};
}
```
Create an empty folder, save the above configuration as `flake.nix`, and then execute `nix develop` (or more precisely, you can use `nix develop .#default`), you will find that you have entered a nodejs 18 development environment, you can use `node` `npm` `pnpm` `yarn` and other commands. And when you just entered, `shellHook` was also executed, outputting the current version of nodejs.
### Enter the build environment of any Nix package
Now let's take a look at `nix develop`, first read the help document output by `nix develop --help`:
```
Name
nix develop - run a bash shell that provides the build environment of a derivation
Synopsis
nix develop [option...] installable
# ......
```
It tells us that `nix develop` accepts a parameter `installable`, which means that we can enter the development environment of any installable Nix package through it, not just the environment created by `pkgs.mkShell`.
By default, `nix develop` will try to use the following attributes in the flake outputs:
- `devShells.<system>.default`
- `packages.<system>.default`
If we use `nix develop /path/to/flake#<name>` to specify the flake package address and flake output name, then `nix develop` will try the following attributes in the flake outputs:
- `devShells.<system>.<name>`
- `packages.<system>.<name>`
- `legacyPackages.<system>.<name>`
Now let's try it out. First, test it to confirm that We don't have `c++` `g++` and other compilation-related commands in the current environment:
```shell
ryan in 🌐 aquamarine in ~
› c++
c++: command not found
ryan in 🌐 aquamarine in ~
› g++
g++: command not found
```
Then use `nix develop` to enter the build environment of the `hello` package in `nixpkgs`:
```shell
# login to the build environment of the package `hello`
ryan in 🌐 aquamarine in ~
› nix develop nixpkgs#hello
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› env | grep CXX
CXX=g++
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› c++ --version
g++ (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› g++ --version
g++ (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
We can see that the `CXX` environment variable have been set, and the `c++` `g++` and other commands can be used normally now.
In addition, we can also call every build phase of the `hello` package normally:
> The default execution order of all build phases of a Nix package is: `$prePhases unpackPhase patchPhase $preConfigurePhases configurePhase $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase fixupPhase installCheckPhase $preDistPhases distPhase $postPhases`
```shell
# unpack source code
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› unpackPhase
unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
source root is hello-2.12.1
setting SOURCE_DATE_EPOCH to timestamp 1653865426 of file hello-2.12.1/ChangeLog
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› ls
hello-2.12.1
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› cd hello-2.12.1/
# generate Makefile
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via ❄️ impure (hello-2.12.1-env)
› configurePhase
configure flags: --prefix=/tmp/xxx/outputs/out --prefix=/tmp/xxx/outputs/out
checking for a BSD-compatible install... /nix/store/02dr9ymdqpkb75vf0v1z2l91z2q3izy9-coreutils-9.3/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /nix/store/02dr9ymdqpkb75vf0v1z2l91z2q3izy9-coreutils-9.3/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
# ......
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating po/Makefile.in
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
# build the package
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via C v12.3.0-gcc via ❄️ impure (hello-2.12.1-env) took 2s
› buildPhase
build flags: SHELL=/run/current-system/sw/bin/bash
make all-recursive
make[1]: Entering directory '/tmp/xxx/hello-2.12.1'
# ......
ranlib lib/libhello.a
gcc -g -O2 -o hello src/hello.o ./lib/libhello.a
make[2]: Leaving directory '/tmp/xxx/hello-2.12.1'
make[1]: Leaving directory '/tmp/xxx/hello-2.12.1'
# run the built program
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via C v12.3.0-gcc via ❄️ impure (hello-2.12.1-env)
› ./hello
Hello, world!
```
This usage is mainly used to debug the build process of a Nix package, or to execute some commands in the build environment of a Nix package.
### `nix shell` & `nix run`
Compare to `nix develop`, these two commands are much simpler and easier to understand.
`nix shell` is used to enter an environment containing the specified Nix package and open an interactive shell for it:
```shell
# hello not exists
› hello
hello: command not found
# enter an environment containing hello
› nix shell nixpkgs#hello
# now hello exists
› hello
Hello, world!
```
`nix run` is used to create an environment containing the specified installable and run the installable in it:
```shell
# hello not exists
› hello
hello: command not found
# enter an environment containing hello and run it
› nix run nixpkgs#hello
Hello, world!
```
Because `nix run` will directly run the Nix package as an installable, the Nix package used as its parameter must be able to generate an executable program.
According to the description of `nix run --help`, `nix run` will execute `<out>/bin/<name>`, where `<out>` is the root directory of a Derivation, and `<name>` is selected in the following order:
- The meta.mainProgram attribute of the derivation.
- The pname attribute of the derivation.
- The name part of the value of the name attribute of the derivation.
For instance, if name is set to `hello-1.10`, nix run will run $out/bin/hello.
## References
- [pkgs.mkShell - nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell)
- [A minimal nix-shell](https://fzakaria.com/2021/08/02/a-minimal-nix-shell.html)
- [One too many shell, Clearing up with nix' shells nix shell and nix-shell - Yannik Sander](https://blog.ysndr.de/posts/guides/2021-12-01-nix-shells/)

115
docs/development/kernel-development.md Normal file
View File

@@ -0,0 +1,115 @@
# Kernel Development
> WIP work in progress
An example of kernel development with `flake.nix`.
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small";
# custom kernel's source
thead-kernel = {
url = "github:revyos/thead-kernel/lpi4a";
flake = false;
};
};
outputs = inputs@{
self
,nixpkgs
,thead-kernel
,... }:
let
pkgsKernel = import nixpkgs {
localSystem = "x86_64-linux";
crossSystem = {
config = "riscv64-unknown-linux-gnu";
};
overlays = [
(self: super: {
# use gcc 13 to compile this custom kernel
linuxPackages_thead = super.linuxPackagesFor (super.callPackage ./pkgs/kernel {
src = thead-kernel;
stdenv = super.gcc13Stdenv;
kernelPatches = with super.kernelPatches; [
bridge_stp_helper
request_key_helper
];
});
})
];
};
in
{
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
specialArgs = {
inherit nixpkgs pkgsKernel;
};
modules = [
{
# cross-compile this flake.
nixpkgs.crossSystem = {
system = "riscv64-linux";
};
}
./modules/licheepi4a.nix
./modules/sd-image-lp4a.nix
];
};
# use `nix develop .#kernel` to enter the environment with the custom kernel build environment available.
# and then use `unpackPhase` to unpack the kernel source code and cd into it.
# then you can use `make menuconfig` to configure the kernel.
#
# problem
# - using `make menuconfig` - Unable to find the ncurses package.
devShells.x86_64-linux.kernel = pkgsKernel.linuxPackages_thead.kernel.dev;
# use `nix develop .#fhs` to enter the fhs test environment defined here.
devShells.x86_64-linux.fhs = let
pkgs = import nixpkgs {
system = "x86_64-linux";
};
in
# the code here is mainly copied from:
# https://nixos.wiki/wiki/Linux_kernel#Embedded_Linux_Cross-compile_xconfig_and_menuconfig
(pkgs.buildFHSUserEnv {
name = "kernel-build-env";
targetPkgs = pkgs_: (with pkgs_;
[
# we need theses packages to run `make menuconfig` successfully.
pkgconfig
ncurses
pkgsKernel.gcc13Stdenv.cc
gcc
]
++ pkgs.linux.nativeBuildInputs);
runScript = pkgs.writeScript "init.sh" ''
# set the cross-compilation environment variables.
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
export ARCH=riscv
export PKG_CONFIG_PATH="${pkgs.ncurses.dev}/lib/pkgconfig:"
exec bash
'';
}).env;
};
}
```
With the above `flake.nix`, I can enter the kernel build environment with `nix develop .#kernel`, and then use `unpackPhase` to unpack the kernel source code and cd into it.
But I can't use `make menuconfig` to configure the kernel, because the `ncurses` package is missing in this environment.
To solve this problem, I add a `fhs` environment to install the `ncurses` package and other necessary packages, and then I can use `nix develop .#fhs` to enter this environment and use `make menuconfig` to configure the kernel.
## References
- https://github.com/jordanisaacs/kernel-module-flake

22
docs/development/packaging-101.md Normal file
View File

@@ -0,0 +1,22 @@
# Packging 101
> WIP work in progress
TODO
### 1. stdenv
TODO
### 2. language specific frameworks
TODO
## References
- [NixOS Series 3: Software Packaging 101](https://lantian.pub/en/article/modify-computer/nixos-packaging.lantian/)
- [stdenv - Nixpkgs Manual](https://github.com/NixOS/nixpkgs/tree/nixos-unstable/doc/languages-frameworks)
- [languages-frameworks - Nixpkgs Manual](https://github.com/NixOS/nixpkgs/tree/nixos-unstable/doc/stdenv)