From 9fb4014c227780ea74fcfbf2135a62d6a540913d Mon Sep 17 00:00:00 2001 From: Ryan Yin Date: Tue, 27 Jun 2023 18:26:13 +0800 Subject: [PATCH] feat: corss-platform compilation --- docs/advanced-topics/index.md | 2 - .../development/cross-platform-compilation.md | 250 +++++++++++++++++- docs/zh/advanced-topics/index.md | 2 - .../development/cross-platform-compilation.md | 2 +- 4 files changed, 250 insertions(+), 6 deletions(-) diff --git a/docs/advanced-topics/index.md b/docs/advanced-topics/index.md index 9e57659..8e648d1 100644 --- a/docs/advanced-topics/index.md +++ b/docs/advanced-topics/index.md @@ -17,8 +17,6 @@ After becoming familiar with Flakes, you may want to try some advanced technique And many other useful community projects to explore, here are some of them: -- [dev-templates](https://github.com/the-nix-way/dev-templates): Dev environments for numerous languages based on Nix flakes. -- [devenv](https://github.com/cachix/devenv): development environment management - [agenix](https://github.com/ryantm/agenix): secrets management - [colmena](https://github.com/zhaofengli/colmena): NixOS deployment tools - [nixos-generator](https://github.com/nix-community/nixos-generators): generate iso/qcow2/... from nixos configuration diff --git a/docs/development/cross-platform-compilation.md b/docs/development/cross-platform-compilation.md index 10c35ae..545250b 100644 --- a/docs/development/cross-platform-compilation.md +++ b/docs/development/cross-platform-compilation.md @@ -3,4 +3,252 @@ First of all, on any Linux platform, there are two ways to do cross-platform compilation. Taking the building of an`aarch64-linux` program on a `x86_64-linux` host as an example, the two methods are described as follows: -TODO \ No newline at end of file +1. Use the cross-compilation toolchain to compile the aarch64 program + 1. The disadvantage is that you cannot use the NixOS binary cache, and you need to compile everything yourself (cross-compilation also has cache, but there is basically nothing in it) + 2. The advantages are that you don't need to emulate the instruction set, and the performance is high +2. Use QEMU to emulate the aarch64 architecture, and then compile the program in the emulator + 1. The disadvantage is that the instruction set is emulated and the performance is low + 2. The advantage is that you can use the NixOS binary cache, and you don't need to compile everything yourself + +by using method one, you don't need to enable binfmt_misc, but you need to execute the compilation through the cross-compilation toolchain. + +If you use method two, you need to enable the binfmt_misc of the aarch64 architecture in the NixOS configuration of the building machine. + + +### Cross Compilation + +nixpkgs comes with a set of predefined host platforms for cross compilation called pkgsCross, let's explore them in `nix repl`: + +```shell +› nix repl '' +warning: future versions of Nix will require using `--file` to load a file +Welcome to Nix 2.13.3. Type :? for help. + +Loading installable ''... +Added 19273 variables. +nix-repl> pkgsCross. +pkgsCross.aarch64-android pkgsCross.msp430 +pkgsCross.aarch64-android-prebuilt pkgsCross.musl-power +pkgsCross.aarch64-darwin pkgsCross.musl32 +pkgsCross.aarch64-embedded pkgsCross.musl64 +pkgsCross.aarch64-multiplatform pkgsCross.muslpi +pkgsCross.aarch64-multiplatform-musl pkgsCross.or1k +pkgsCross.aarch64be-embedded pkgsCross.pogoplug4 +pkgsCross.arm-embedded pkgsCross.powernv +pkgsCross.armhf-embedded pkgsCross.ppc-embedded +pkgsCross.armv7a-android-prebuilt pkgsCross.ppc64 +pkgsCross.armv7l-hf-multiplatform pkgsCross.ppc64-musl +pkgsCross.avr pkgsCross.ppcle-embedded +pkgsCross.ben-nanonote pkgsCross.raspberryPi +pkgsCross.fuloongminipc pkgsCross.remarkable1 +pkgsCross.ghcjs pkgsCross.remarkable2 +pkgsCross.gnu32 pkgsCross.riscv32 +pkgsCross.gnu64 pkgsCross.riscv32-embedded +pkgsCross.i686-embedded pkgsCross.riscv64 +pkgsCross.iphone32 pkgsCross.riscv64-embedded +pkgsCross.iphone32-simulator pkgsCross.rx-embedded +pkgsCross.iphone64 pkgsCross.s390 +pkgsCross.iphone64-simulator pkgsCross.s390x +pkgsCross.loongarch64-linux pkgsCross.sheevaplug +pkgsCross.m68k pkgsCross.vc4 +pkgsCross.mingw32 pkgsCross.wasi32 +pkgsCross.mingwW64 pkgsCross.x86_64-darwin +pkgsCross.mips-linux-gnu pkgsCross.x86_64-embedded +pkgsCross.mips64-linux-gnuabi64 pkgsCross.x86_64-freebsd +pkgsCross.mips64-linux-gnuabin32 pkgsCross.x86_64-netbsd +pkgsCross.mips64el-linux-gnuabi64 pkgsCross.x86_64-netbsd-llvm +pkgsCross.mips64el-linux-gnuabin32 pkgsCross.x86_64-unknown-redox +pkgsCross.mipsel-linux-gnu +pkgsCross.mmix +``` + +If you want to set `pkgs` to a cross-compilation toolchain globally in a flake, you only need to add a Module in `flake.nix`, as shown below: + +```nix +{ + description = "NixOS running on LicheePi 4A"; + + inputs = { + nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05"; + }; + + outputs = inputs@{ self, nixpkgs, ... }: { + nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem { + # native platform + system = "x86_64-linux"; + modules = [ + + # add this module, to enable cross-compilation. + { + nixpkgs.crossSystem = { + # target platform + system = "riscv64-linux"; + }; + } + + # ...... other modules + ]; + }; + }; +} +``` + +The `nixpkgs.crossSystem` option is used to set `pkgs` to a cross-compilation toolchain, so that all the contents built will be `riscv64-linux` architecture. + + +## Compile through emulated system + +The second method is to cross-compile through the emulated system. This method does not require a cross-compilation toolchain. + +To use this method, first your building machine needs to enable the binfmt_misc module in the configuration. If your building machine is NixOS, add the following configuration to your NixOS Module to enable the simulated build system of `aarch64-linux` and `riscv64-linux` architectures: + +```nix +{ ... }: +{ + # ...... + + # Enable binfmt emulation. + boot.binfmt.emulatedSystems = [ "aarch64-linux" "riscv64-linux" ]; + + # ...... +} +``` + +As for `flake.nix`, its setting method is very simple, even simpler than the setting of cross-compilation, as shown below: + +```nix +{ + description = "NixOS running on LicheePi 4A"; + + inputs = { + nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05"; + }; + + outputs = inputs@{ self, nixpkgs, ... }: { + nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem { + # native platform + system = "riscv64-linux"; + modules = [ + # ...... other modules + ]; + }; + }; +} +``` + +You do not need to add any additional modules, just specify `system` as `riscv64-linux`. +Nix will automatically detect whether the current system is `riscv64-linux` during the build. If not, it will automatically build through the emulated system(QEMU). For users, these underlying operations are completely transparent. + +### Custom build toolchain + +Sometimes we may need to use a custom toolchain for building, such as using our own gcc, or using our own musl libc, etc. This modification can be achieved through overlays. + +For example, let's try to use a different version of gcc, and test it through `nix repl`: + +```shell + +```shell +› nix repl -f '' +Welcome to Nix 2.13.3. Type :? for help. + +Loading installable ''... +Added 17755 variables. + +# replace gcc through overlays, this will create a new instance of nixpkgs +nix-repl> a = import { crossSystem = { config = "riscv64-unknown-linux-gnu"; }; overlays = [ (self: super: { gcc = self.gcc12; }) ]; } + +# check the gcc version, it is indeed changed to 12.2 +nix-repl> a.pkgsCross.riscv64.stdenv.cc +«derivation /nix/store/jjvvwnf3hzk71p65x1n8bah3hrs08bpf-riscv64-unknown-linux-gnu-stage-final-gcc-wrapper-12.2.0.drv» + +# take a look at the default pkgs, it is still 11.3 +nix-repl> pkgs.pkgsCross.riscv64.stdenv.cc +«derivation /nix/store/pq3g0wq3yfc4hqrikr03ixmhqxbh35q7-riscv64-unknown-linux-gnu-stage-final-gcc-wrapper-11.3.0.drv» +``` + +So how to use this method in Flakes? The example `flake.nix` is as follows: + +```nix +{ + description = "NixOS running on LicheePi 4A"; + + inputs = { + nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small"; + }; + + outputs = { self, nixpkgs, ... }: + { + nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem { + system = "x86_64-linux"; + modules = [ + { + nigpkgs.crossSystem = { + config = "riscv64-unknown-linux-gnu"; + }; + + # replace gcc with gcc12 through overlays + nixpkgs.overlays = [ (self: super: { gcc = self.gcc12; }) ]; + } + + # other moduels ...... + ]; + }; + }; +} +``` + +`nixpkgs.overlays` is used to modify the `pkgs` instance globally, and the modified `pkgs` instance will take effect to the whole flake. It will likely cause a large number of cache missing, and thus require building a large number of Nix packages locally. + +To avoid this problem, a better way is to create a new `pkgs` instance, and only use this instance when building the packages we want to modify. The example `flake.nix` is as follows: + +```nix +{ + description = "NixOS running on LicheePi 4A"; + + inputs = { + nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small"; + }; + + outputs = { self, nixpkgs, ... }: let + # create a new pkgs instance with overlays + pkgs-gcc12 = import nixpkgs { + localSystem = "x86_64-linux"; + crossSystem = { + config = "riscv64-unknown-linux-gnu"; + }; + + overlays = [ + (self: super: { gcc = self.gcc12; }) + ]; + }; + in { + nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem { + system = "x86_64-linux"; + specialArgs = { + # pass the new pkgs instance to the module + inherit pkgs-gcc12; + }; + modules = [ + { + nigpkgs.crossSystem = { + config = "riscv64-unknown-linux-gnu"; + }; + } + + ({pkgs-gcc12, ...}: { + # use the custom pkgs instance to build the package hello + environment.systemPackages = [ pkgs-gcc12.hello ]; + }) + + # other moduels ...... + ]; + }; + }; +} +``` + +Through the above method, we can easily customize the build toolchain of some packages without affecting the build of other packages. + +## References + +- [Cross compilation - nix.dev](https://nix.dev/tutorials/cross-compilation) diff --git a/docs/zh/advanced-topics/index.md b/docs/zh/advanced-topics/index.md index ebfeba6..075a871 100644 --- a/docs/zh/advanced-topics/index.md +++ b/docs/zh/advanced-topics/index.md @@ -17,8 +17,6 @@ 以及其他许多实用的社区项目可探索，我比较关注的有这几个： -- [dev-templates](https://github.com/the-nix-way/dev-templates): 原汁原味的 devShell 模板，可以用于快速搭建开发环境。 -- [devenv](https://github.com/cachix/devenv): 开发环境管理 - [agenix](https://github.com/ryantm/agenix): secrets 管理 - [nixos-generator](https://github.com/nix-community/nixos-generators): 镜像生成工具，从 nixos 配置生成 iso/qcow2 等格式的镜像 - [lanzaboote](https://github.com/nix-community/lanzaboote): 启用 secure boot diff --git a/docs/zh/development/cross-platform-compilation.md b/docs/zh/development/cross-platform-compilation.md index 129e716..9065c83 100644 --- a/docs/zh/development/cross-platform-compilation.md +++ b/docs/zh/development/cross-platform-compilation.md @@ -97,7 +97,7 @@ pkgsCross.mmix ### 通过模拟系统进行跨平台编译 -第二种方法是通过模拟系统进行跨平台编译，这种方法的好处是不需要交叉编译工具链，但是需要往 binfmt_misc 模块中注册 qemu 的二进制文件。 +第二种方法是通过模拟系统进行跨平台编译，这种方法不需要交叉编译工具链，但是需要往 binfmt_misc 模块中注册 qemu 的二进制文件。 要使用这种方法，首先你的构建机需要在配置中启用 binfmt_misc 模块，如果你的构建机是 NixOS，将如下配置添加到你的 NixOS Module 即可启用 `aarch64-linux` 与 `riscv64-linux` 两种架构的模拟构建系统：