mirror of
https://github.com/ryan4yin/nixos-and-flakes-book.git
synced 2024-11-22 16:13:26 +01:00
commit
6927d32565
@ -1,23 +1,21 @@
|
||||
## Cross-platform compilation
|
||||
## Cross-platform Compilation
|
||||
|
||||
First of all, on any Linux platform, there are two ways to do cross-platform compilation.
|
||||
Taking the building of an`aarch64-linux` program on a `x86_64-linux` host as an example, the two methods are described as follows:
|
||||
On any Linux platform, there are two ways to do cross-platform compilation. For example, to build an `aarch64-linux` program on an `x86_64-linux` host, you can use the following methods:
|
||||
|
||||
1. Use the cross-compilation toolchain to compile the aarch64 program
|
||||
1. The disadvantage is that you cannot use the NixOS binary cache, and you need to compile everything yourself (cross-compilation also has cache, but there is basically nothing in it)
|
||||
2. The advantages are that you don't need to emulate the instruction set, and the performance is high
|
||||
2. Use QEMU to emulate the aarch64 architecture, and then compile the program in the emulator
|
||||
1. The disadvantage is that the instruction set is emulated and the performance is low
|
||||
2. The advantage is that you can use the NixOS binary cache, and you don't need to compile everything yourself
|
||||
1. Use the cross-compilation toolchain to compile the `aarch64` program.
|
||||
- The disadvantage is that you cannot use the NixOS binary cache, and you need to compile everything yourself (cross-compilation also has a cache, but there is basically nothing in it).
|
||||
- The advantages are that you don't need to emulate the instruction set, and the performance is high.
|
||||
2. Use QEMU to emulate the `aarch64` architecture and then compile the program in the emulator.
|
||||
- The disadvantage is that the instruction set is emulated, and the performance is low.
|
||||
- The advantage is that you can use the NixOS binary cache, and you don't need to compile everything yourself.
|
||||
|
||||
by using method one, you don't need to enable binfmt_misc, but you need to execute the compilation through the cross-compilation toolchain.
|
||||
|
||||
If you use method two, you need to enable the binfmt_misc of the aarch64 architecture in the NixOS configuration of the building machine.
|
||||
If you use method one, you don't need to enable `binfmt_misc`, but you need to execute the compilation through the cross-compilation toolchain.
|
||||
|
||||
If you use method two, you need to enable the `binfmt_misc` of the `aarch64` architecture in the NixOS configuration of the building machine.
|
||||
|
||||
### Cross Compilation
|
||||
|
||||
nixpkgs comes with a set of predefined host platforms for cross compilation called pkgsCross, let's explore them in `nix repl`:
|
||||
`nixpkgs` provides a set of predefined host platforms for cross-compilation called `pkgsCross`. You can explore them in `nix repl`.
|
||||
|
||||
```shell
|
||||
› nix repl '<nixpkgs>'
|
||||
@ -95,7 +93,6 @@ If you want to set `pkgs` to a cross-compilation toolchain globally in a flake,
|
||||
|
||||
The `nixpkgs.crossSystem` option is used to set `pkgs` to a cross-compilation toolchain, so that all the contents built will be `riscv64-linux` architecture.
|
||||
|
||||
|
||||
## Compile through emulated system
|
||||
|
||||
The second method is to cross-compile through the emulated system. This method does not require a cross-compilation toolchain.
|
||||
@ -109,7 +106,7 @@ To use this method, first your building machine needs to enable the binfmt_misc
|
||||
|
||||
# Enable binfmt emulation.
|
||||
boot.binfmt.emulatedSystems = [ "aarch64-linux" "riscv64-linux" ];
|
||||
|
||||
|
||||
# ......
|
||||
}
|
||||
```
|
||||
@ -145,7 +142,7 @@ Sometimes we may need to use a custom toolchain for building, such as using our
|
||||
|
||||
For example, let's try to use a different version of gcc, and test it through `nix repl`:
|
||||
|
||||
```shell
|
||||
````shell
|
||||
|
||||
```shell
|
||||
› nix repl -f '<nixpkgs>'
|
||||
@ -164,7 +161,7 @@ nix-repl> a.pkgsCross.riscv64.stdenv.cc
|
||||
# take a look at the default pkgs, it is still 11.3
|
||||
nix-repl> pkgs.pkgsCross.riscv64.stdenv.cc
|
||||
«derivation /nix/store/pq3g0wq3yfc4hqrikr03ixmhqxbh35q7-riscv64-unknown-linux-gnu-stage-final-gcc-wrapper-11.3.0.drv»
|
||||
```
|
||||
````
|
||||
|
||||
So how to use this method in Flakes? The example `flake.nix` is as follows:
|
||||
|
||||
|
@ -1,20 +1,17 @@
|
||||
## Distributed Building
|
||||
|
||||
Distributed building can speed up the build process by make use of multiple machines.
|
||||
Distributed building can speed up the build process by utilizing multiple machines.
|
||||
|
||||
NixOS's official cache.nixos.org provides the vast majority of caches for the X86_64 architecture, so distributed builds are generally not very useful for ordinary NixOS users.
|
||||
For ordinary NixOS users, distributed building is generally not very useful because NixOS's official `cache.nixos.org` provides the vast majority of caches for the `x86_64` architecture.
|
||||
|
||||
Distributed building is of great application value only in scenarios where there is no cache available:
|
||||
Distributed building is of great value in scenarios where there is no cache available, such as:
|
||||
|
||||
1. Users of `RISC-V` or `ARM64` architectures (especially `RISC-V`), because there are very few caches for these two architectures in the official cache repository, which often requires a lot of local compilation.
|
||||
2. Users who customize the system a lot, because the packages in the official cache repository are all default configurations. If you change the build parameters, then the official cache is not applicable, and you need to compile locally. For instance, in the embedded scenario, there is often a need for customization of the underlying kernel, drivers, etc., which leads to the need for local compilation.
|
||||
|
||||
1. Users of RISC-V or ARM64 architectures (especially RISC-V), because there are very few caches for these two architectures in the official cache repository, which often requires a lot of local compilation.
|
||||
2. Users who customize the system a lot, because the packages in the official cache repository are all default configurations, if you change the build parameters, then the official cache is not applicable, then you need to compile locally.
|
||||
2. For instance, in the embedded scenario, there is often a need for customization of the underlying kernel, drivers, etc., which leads to the need for local compilation.
|
||||
### Configure Distributed Building
|
||||
|
||||
|
||||
### Configure distributed building
|
||||
|
||||
Currently, there is no official documentation for this, and I have listed some recommended reference documents at the end of this chapter, along with my distributed build configuration (a NixOS Module):
|
||||
Currently, there is no official documentation for distributed building. However, I have listed some recommended reference documents at the end of this chapter, along with my distributed build configuration (a NixOS Module).
|
||||
|
||||
```nix
|
||||
{ ... }: {
|
||||
@ -28,7 +25,7 @@ Currently, there is no official documentation for this, and I have listed some r
|
||||
# set local's max-job to 0 to force remote building(disable local building)
|
||||
# nix.settings.max-jobs = 0;
|
||||
nix.distributedBuilds = true;
|
||||
nix.buildMachines =
|
||||
nix.buildMachines =
|
||||
let
|
||||
sshUser = "ryan";
|
||||
# ssh key's path on local machine
|
||||
@ -92,11 +89,11 @@ Currently, there is no official documentation for this, and I have listed some r
|
||||
Host ai
|
||||
HostName 192.168.5.100
|
||||
Port 22
|
||||
|
||||
|
||||
Host aquamarine
|
||||
HostName 192.168.5.101
|
||||
Port 22
|
||||
|
||||
|
||||
Host ruby
|
||||
HostName 192.168.5.102
|
||||
Port 22
|
||||
@ -130,17 +127,15 @@ Currently, there is no official documentation for this, and I have listed some r
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Defects
|
||||
|
||||
The problems I have observed so far are:
|
||||
|
||||
1. You cannot specify which hosts to use at build time, you can only specify a list of hosts in the configuration file, and nix automatically selects available hosts.
|
||||
two。
|
||||
two。
|
||||
2. When choosing a host, I found that Nix always preferred the remote host, while my local host had the best performance, which caused the local host's CPU to be underutilized.
|
||||
3. The smallest unit of distributed building is Derivation, so when building some big packages, other machines may be idle for a long time, waiting for the big package to be built, which leads to a waste of resources.
|
||||
|
||||
|
||||
## References
|
||||
|
||||
- [Distributed build - NixOS Wiki](https://nixos.wiki/wiki/Distributed_build)
|
||||
|
@ -1,19 +1,18 @@
|
||||
## Dev Environemnts on NixOS
|
||||
## Development Environments on NixOS
|
||||
|
||||
Due to the reproducibility of NixOS itself, it is very suitable for building development environments.
|
||||
But if you want to migrate the experience on other distros directly to NixOS, you may encounter problems, because NixOS has its own set of logic, we will explain this briefly below.
|
||||
NixOS's reproducibility makes it ideal for building development environments. However, if you're used to other distros, you may encounter problems because NixOS has its own logic. We'll explain this briefly below.
|
||||
|
||||
On NixOS, it is recommended to install only common tools in the global environment, such as `git`, `vim`, `emacs`, `tmux`, `zsh`, etc., while the development environment of each language should be an independent environment for each project.
|
||||
On NixOS, it's recommended to only install common tools in the global environment, such as `git`, `vim`, `emacs`, `tmux`, `zsh`, etc. The development environment of each language should be an independent environment for each project.
|
||||
|
||||
You should NOT install the development environment of each language in the global environment, and the project environment should be completely isolated from each other, and will not affect each other.
|
||||
You should NOT install the development environment of each language in the global environment. The project environment should be completely isolated from each other and will not affect each other.
|
||||
|
||||
We will introduce how the development environment works in NixOS in the following sections.
|
||||
In the following sections, we'll introduce how the development environment works in NixOS.
|
||||
|
||||
### Create a development environment
|
||||
### Creating a Development Environment
|
||||
|
||||
We can create a development environment via `pkgs.mkShell { ... }`, and open an interactive Bash Shell of this development environment via `nix develop`.
|
||||
We can create a development environment using `pkgs.mkShell { ... }` and open an interactive Bash shell of this development environment using `nix develop`.
|
||||
|
||||
Let's take a look at [the source code of `pkgs.mkShell`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/mkshell/default.nix) to see how it works:
|
||||
To see how `pkgs.mkShell` works, let's take a look at [its source code](https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/mkshell/default.nix).
|
||||
|
||||
```nix
|
||||
{ lib, stdenv, buildEnv }:
|
||||
@ -288,4 +287,4 @@ For instance, if name is set to `hello-1.10`, nix run will run $out/bin/hello.
|
||||
|
||||
- [pkgs.mkShell - nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell)
|
||||
- [A minimal nix-shell](https://fzakaria.com/2021/08/02/a-minimal-nix-shell.html)
|
||||
- [One too many shell, Clearing up with nix' shells nix shell and nix-shell - Yannik Sander](https://blog.ysndr.de/posts/guides/2021-12-01-nix-shells/)
|
||||
- [One too many shell, Clearing up with nix' shells nix shell and nix-shell - Yannik Sander](https://blog.ysndr.de/posts/guides/2021-12-01-nix-shells/)
|
||||
|
@ -1,11 +1,10 @@
|
||||
## Downgrade or upgrade packages
|
||||
## Downgrading or Upgrading Packages
|
||||
|
||||
After using Flakes, most people are currently using the `nixos-unstable` branch of nixpkgs. Sometimes you will encounter some bugs, such as the [chrome/vscode crash problem](https://github.com/swaywm/sway/issues/7562)
|
||||
After using Flakes, most people are currently using the `nixos-unstable` branch of nixpkgs. However, sometimes you may encounter bugs, such as the [chrome/vscode crash problem](https://github.com/swaywm/sway/issues/7562).
|
||||
|
||||
To resolve problems, we may need to downgrade or upgrade some packages. In Flakes, all package versions and hash values are one-to-one corresponding to the git commit of their flake input.
|
||||
Therefore, to downgrade or upgrade a package, we need to lock the git commit of its flake input.
|
||||
To resolve these issues, you may need to downgrade or upgrade some packages. In Flakes, all package versions and hash values correspond one-to-one to the git commit of their flake input. Therefore, to downgrade or upgrade a package, you need to lock the git commit of its flake input.
|
||||
|
||||
For example, let's add multiple nixpkgs, each using a different git commit or branch:
|
||||
For example, you can add multiple nixpkgs, each using a different git commit or branch:
|
||||
|
||||
```nix
|
||||
{
|
||||
@ -85,6 +84,8 @@ And then refer to the packages from `pkgs-stable` or `pkgs-fd40cef8d` in your su
|
||||
}
|
||||
```
|
||||
|
||||
After adjusting the configuration, deploy it with `sudo nixos-rebuild switch`, then your firefox/chrome/vscode will be downgraded to the version corresponding to `nixpkgs-stable` or `nixpkgs-fd40cef8d`.
|
||||
After adjusting the configuration, deploy it with `sudo nixos-rebuild switch`. This will downgrade your Firefox/Chrome/VSCode to the version corresponding to `nixpkgs-stable` or `nixpkgs-fd40cef8d`.
|
||||
|
||||
According to [1000 instances of nixpkgs](https://discourse.nixos.org/t/1000-instances-of-nixpkgs/17347), it's not a good practice to use `import` in submodules to customize `nixpkgs`. Each `import` creates a new instance of nixpkgs, which increases build time and memory usage as the configuration grows. To avoid this problem, we create all nixpkgs instances in `flake.nix`.
|
||||
|
||||
> according to [1000 instances of nixpkgs](https://discourse.nixos.org/t/1000-instances-of-nixpkgs/17347), it's not a good practice to use `import` in sub modules to customize `nixpkgs`, because each `import` will create a new instance of nixpkgs, which will increase the build time and memory usage as the configuration grows. So here we create all nixpkgs instances in `flake.nix` to avoid this problem.
|
||||
|
@ -1,8 +1,8 @@
|
||||
## Start using home-manager
|
||||
## Getting Started with Home Manager
|
||||
|
||||
As I mentioned earlier, NixOS can only manage system-level configuration, to manage the Home directory(user-level configuration), we need to install home-manager.
|
||||
As I mentioned earlier, NixOS can only manage system-level configuration. To manage user-level configuration in the Home directory, we need to install Home Manager.
|
||||
|
||||
According to the official document [Home Manager Manual](https://nix-community.github.io/home-manager/index.html), in order to install home-manager as a module of NixOS, we need to create `/etc/nixos/home.nix` first, an example content shown below:
|
||||
According to the official [Home Manager Manual](https://nix-community.github.io/home-manager/index.html), to install Home Manager as a module of NixOS, we first need to create `/etc/nixos/home.nix`. Here's an example of its contents:
|
||||
|
||||
```nix
|
||||
{ config, pkgs, ... }:
|
||||
|
@ -1,13 +1,12 @@
|
||||
## Updating the System
|
||||
|
||||
## Update the system
|
||||
|
||||
With Flakes, it is also very simple to update the system. Just run the following commands in `/etc/nixos`:
|
||||
With Flakes, updating the system is simple. Just run the following commands in `/etc/nixos`:
|
||||
|
||||
```shell
|
||||
# update flake.lock
|
||||
# Update flake.lock
|
||||
nix flake update
|
||||
# apply the updates
|
||||
# Apply the updates
|
||||
sudo nixos-rebuild switch
|
||||
```
|
||||
|
||||
Sometimes you may encounter some error of sha256 mismatch when running `nixos-rebuild switch`, which may be solved by updating `flake.lock` through `nix flake update`.
|
||||
Sometimes, you may encounter a sha256 mismatch error when running `nixos-rebuild switch`. This can be resolved by updating `flake.lock` through `nix flake update`.
|
||||
|
@ -1,21 +1,21 @@
|
||||
## pkgs.callPackage
|
||||
|
||||
In the previous content, We have used `import xxx.nix` to import Nix files many times, this syntax simply returns the execution result of the file, without any further processing of it.
|
||||
In the previous section, we used `import xxx.nix` to import Nix files multiple times. This syntax simply returns the execution result of the file without any further processing.
|
||||
|
||||
`pkgs.callPackage` is also used to import Nix files, its syntax is `pkgs.callPackage xxx.nix { ... }`, but unlike `import`, the Nix file imported by it must be a Derivation or a function that returns a Derivation. Its result is a Derivation(a software package) too.
|
||||
`pkgs.callPackage` is also used to import Nix files. Its syntax is `pkgs.callPackage xxx.nix { ... }`. However, unlike `import`, the Nix file imported by it must be a Derivation or a function that returns a Derivation. Its result is also a Derivation (a software package).
|
||||
|
||||
So what does the Nix file that can be used as a parameter of `pkgs.callPackge` look like? You can take a look at the `hello.nix` `fcitx5-rime.nix` `vscode/with-extensions.nix` `firefox/common.nix` we mentioned earlier, they can all be imported by `pkgs.callPackage`.
|
||||
So, what does the Nix file that can be used as a parameter of `pkgs.callPackge` look like? You can take a look at the `hello.nix`, `fcitx5-rime.nix`, `vscode/with-extensions.nix`, and `firefox/common.nix` files we mentioned earlier. They can all be imported by `pkgs.callPackage`.
|
||||
|
||||
When the `xxx.nix` used in `pkgs.callPackge xxx.nix {...}` is a function (most Nix packages are like this), the execution flow is as follows:
|
||||
|
||||
1. `pkgs.callPackge xxx.nix {...}` will first `import xxx.nix` to get the function defined in it. The parameters of this function usually have `lib`, `stdenv`, `fetchurl` and other parameters, as well as some custom parameters, which usually have default values.
|
||||
2. Then `pkgs.callPackge` will first look up the value matching the name from the current environment as the parameter to be passed to the function. parameters like `lib` `stdenv` `fetchurl` are defined in nixpkgs, and they will be found in this step.
|
||||
3. Then `pkgs.callPackge` will merge its second parameter `{...}` with the attribute set obtained in the previous step, and then pass it to the function imported from `xxx.nix` and execute it.
|
||||
4. Finally we get a Derivation as the result of the function execution.
|
||||
1. `pkgs.callPackge xxx.nix {...}` first imports `xxx.nix` to get the function defined in it. The parameters of this function usually have `lib`, `stdenv`, `fetchurl`, and other parameters, as well as some custom parameters that usually have default values.
|
||||
2. Then, `pkgs.callPackge` looks up the value matching the name from the current environment as the parameter to be passed to the function. Parameters like `lib`, `stdenv`, and `fetchurl` are defined in nixpkgs and will be found in this step.
|
||||
3. Next, `pkgs.callPackge` merges its second parameter `{...}` with the attribute set obtained in the previous step and passes it to the function imported from `xxx.nix` to execute it.
|
||||
4. Finally, we get a Derivation as the result of the function execution.
|
||||
|
||||
The common usage of `pkgs.callPackage` is to import customized Nix packages and used them in Nix Module.
|
||||
The common usage of `pkgs.callPackage` is to import customized Nix packages and use them in Nix Modules.
|
||||
|
||||
For example, we have customized a NixOS kernel configuration `kernel.nix`, and it uses the SBC's name and kernel source as its variable parameters:
|
||||
For example, suppose we have customized a NixOS kernel configuration `kernel.nix`, which uses the SBC's name and kernel source as its variable parameters:
|
||||
|
||||
```nix
|
||||
{
|
||||
@ -32,7 +32,7 @@ For example, we have customized a NixOS kernel configuration `kernel.nix`, and i
|
||||
modDirVersion = "5.10.113";
|
||||
|
||||
inherit src lib stdenv;
|
||||
|
||||
|
||||
# file path to the generated kernel config file(the `.config` generated by make menuconfig)
|
||||
#
|
||||
# here is a special usage to generate a file path from a string
|
||||
@ -45,7 +45,7 @@ For example, we have customized a NixOS kernel configuration `kernel.nix`, and i
|
||||
Then we can use `pkgs.callPackage ./kernel.nix {}` in any Nix Module to import and replace any of its parameters:
|
||||
|
||||
```nix
|
||||
{ lib, pkgs, pkgsKernel, kernel-src, ... }:
|
||||
{ lib, pkgs, pkgsKernel, kernel-src, ... }:
|
||||
|
||||
{
|
||||
# ......
|
||||
@ -61,9 +61,8 @@ Then we can use `pkgs.callPackage ./kernel.nix {}` in any Nix Module to import a
|
||||
}
|
||||
```
|
||||
|
||||
Just like the example above, we can use `pkgs.callPackage` to pass different `src` and `boardName` to the function defined in `kernel.nix` through `pkgs.callPackage`, so that different kernel packages can be generated.
|
||||
The `kernel.nix` can be used to adapt to different kernel sources and different development boards by changing the parameters passed to it.
|
||||
Similarly to the example above, we can use `pkgs.callPackage` to pass different `src` and `boardName` parameters to the function defined in `kernel.nix`. This allows us to generate different kernel packages. By changing the parameters passed to it, `kernel.nix` can be used to adapt to different kernel sources and development boards.
|
||||
|
||||
## References
|
||||
|
||||
- [Chapter 13. Callpackage Design Pattern - Nix Pills](https://nixos.org/guides/nix-pills/callpackage-design-pattern.html)
|
||||
- [Chapter 13. Callpackage Design Pattern - Nix Pills](https://nixos.org/guides/nix-pills/callpackage-design-pattern.html)
|
||||
|
@ -1,21 +1,19 @@
|
||||
## Overlays
|
||||
|
||||
The `override` we introduced previously will generate a new Derivation, which does not affect the original Derivation in `pkgs`, and is only suitable for use as a local parameter,
|
||||
if you need to override a Derivation that is also dependent on other Nix packages, then other Nix packages will still use the original Derivation.
|
||||
The `override` we introduced earlier generates a new Derivation that does not affect the original Derivation in `pkgs`. It's only suitable for use as a local parameter. If you need to override a Derivation that is also dependent on other Nix packages, then other Nix packages will still use the original Derivation.
|
||||
|
||||
To solve this problem, Nix provides the ability to use `overlays`. Simply put, `overlays` can globally modify the Derivation in `pkgs`.
|
||||
To solve this problem, Nix provides the ability to use `overlays`. Simply put, `overlays` can globally modify the Derivations in `pkgs`.
|
||||
|
||||
In the classic Nix environment, Nix automatically applies all `overlays` configuration under the paths `~/.config/nixpkgs/overlays.nix` `~/.config/nixpkgs/overlays/*.nix`,
|
||||
but in Flakes, in order to ensure the reproducibility of the system, it cannot depend on any configuration outside the Git repository, so this classic method cannot be used now.
|
||||
In the classic Nix environment, Nix automatically applies all `overlays` configurations under the paths `~/.config/nixpkgs/overlays.nix` and `~/.config/nixpkgs/overlays/*.nix`. However, in Flakes, to ensure the reproducibility of the system, it cannot depend on any configuration outside the Git repository. Therefore, this classic method cannot be used now.
|
||||
|
||||
When using Flakes to write configuration for NixOS, home Manager and NixOS both provide the `nixpkgs.overlays` option to define `overlays`. Related documentation:
|
||||
When using Flakes to write configuration for NixOS, both home Manager and NixOS provide the `nixpkgs.overlays` option to define `overlays`. For more information, refer to the following documentation:
|
||||
|
||||
- [home-manager docs - `nixpkgs.overlays`](https://nix-community.github.io/home-manager/options.html#opt-nixpkgs.overlays)
|
||||
- [nixpkgs source code - `nixpkgs.overlays`](https://github.com/NixOS/nixpkgs/blob/30d7dd7e7f2cba9c105a6906ae2c9ed419e02f17/nixos/modules/misc/nixpkgs.nix#L169)
|
||||
|
||||
For example, the following content is a Module that loads Overlays, which can be used as either a home Manager Module or a NixOS Module, because the two definitions are exactly the same:
|
||||
For example, the following content is a Module that loads overlays, which can be used as either a home Manager Module or a NixOS Module because the two definitions are exactly the same:
|
||||
|
||||
> home Manager is an external component after all, and most people use the unstable branch of home Manager & nixpkgs, which sometimes causes problems with home Manager Module, so it is recommended to import `overlays` in a NixOS Module.
|
||||
> Home Manager is an external component, and most people use the unstable branch of Home Manager and nixpkgs, which sometimes causes problems with Home Manager Module. Therefore, it's recommended to import `overlays` in a NixOS Module.
|
||||
|
||||
```nix
|
||||
{ config, pkgs, lib, ... }:
|
||||
|
Loading…
Reference in New Issue
Block a user