extern/nixos-and-flakes-book
1
0
Fork 0
mirror of https://github.com/ryan4yin/nixos-and-flakes-book.git synced 2024-11-22 16:13:26 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #38 from ryan4yin/fix-callpackage

Browse Source 
fix: pkgs.callPackage
This commit is contained in:
Ryan Yin 2023-10-04 14:27:56 +08:00 committed by GitHub
commit 159750f3f4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 187 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

92
docs/nixpkgs/callpackage.md
View File

@ -1,24 +1,73 @@
# `pkgs.callPackage`
In the previous section, we used the `import xxx.nix` syntax to import Nix files. This syntax simply returns the execution result of the file without any further processing.
`pkgs.callPackage` is used to parameterize the construction of Nix Derivation. To understand its purpose, let's first consider how we would define a Nix package (also known as a Derivation) without using `pkgs.callPackage`.
Another method to import Nix files is `pkgs.callPackage`. Its syntax is `pkgs.callPackage xxx.nix { ... }`. However, unlike `import`, the Nix file imported by `pkgs.callPackage` must be a Derivation or a function that returns a Derivation. The result of `pkgs.callPackage` is also a Derivation, which represents a software package.
## 1. Without `pkgs.callPackage`
So, what does a Nix file that can be used as a parameter for `pkgs.callPackage` look like? You can refer to the `hello.nix`, `fcitx5-rime.nix`, `vscode/with-extensions.nix`, and `firefox/common.nix` files mentioned earlier. All of these files can be imported using `pkgs.callPackage`.
We can define a Nix package using code like this:
When the `xxx.nix` file used in `pkgs.callPackage xxx.nix {...}` is a function (most Nix packages follow this pattern), the execution flow is as follows:
```nix
pkgs.writeShellScriptBin "hello" ''echo "hello, ryan!"''
```
1. `pkgs.callPackage xxx.nix {...}` first imports `xxx.nix` to obtain the function defined within it. This function usually has parameters like `lib`, `stdenv`, `fetchurl`, and sometimes additional custom parameters with default values.
To verify this, you can use `nix repl`, and you'll see that the result is indeed a Derivation:
2. Then, `pkgs.callPackage` searches for a value matching the parameter names from the current environment. Parameters like `lib`, `stdenv`, and `fetchurl` are defined in nixpkgs and will be found in this step.
```shell
› nix repl -f '<nixpkgs>'
Welcome to Nix 2.13.5. Type :? for help.
3. Next, `pkgs.callPackage` merges its second parameter, `{...}`, with the attribute set obtained in the previous step. It then passes this merged set as the parameter to the function imported from `xxx.nix` for execution.
Loading installable ''...
Added 19203 variables.
4. Finally, the result of the function execution is a Derivation.
nix-repl> pkgs.writeShellScriptBin "hello" '' echo "hello, xxx!" ''
«derivation /nix/store/zhgar12vfhbajbchj36vbbl3mg6762s8-hello.drv»
```
The common use case for `pkgs.callPackage` is to import customized Nix packages and use them in Nix Modules.
While the definition of this Derivation is quite concise, most Derivations in nixpkgs are much more complex. In previous sections, we introduced and extensively used the `import xxx.nix` method to import Nix expressions from other Nix files, which can enhance code maintainability.
For example, let's say we have a customized NixOS kernel configuration file named `kernel.nix`, which uses the SBC's name and kernel source as its variable parameters:
1. To enhance maintainability, you can store the definition of the Derivation in a separate file, e.g., `hello.nix`.
1. However, the context within `hello.nix` itself doesn't include the `pkgs` variable, so you'll need to modify its content to pass `pkgs` as a parameter to `hello.nix`.
2. In places where you need to use this Derivation, you can import it using `import ./hello.nix pkgs` and use `pkgs` as a parameter to execute the function defined within.
Let's continue to verify this using `nix repl`, and you'll see that the result is still a Derivation:
```shell
› cat hello.nix
pkgs:
pkgs.writeShellScriptBin "hello" '' echo "hello, xxx!" ''
› nix repl -f '<nixpkgs>'
Welcome to Nix 2.13.5. Type :? for help.
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
Loading installable ''...
Added 19203 variables.
nix-repl> import ./hello.nix pkgs
«derivation /nix/store/zhgar12vfhbajbchj36vbbl3mg6762s8-hello.drv»
```
## 2. Using `pkgs.callPackage`
In the previous example without `pkgs.callPackage`, we directly passed `pkgs` as a parameter to `hello.nix`. However, this approach has some drawbacks:
1. All other dependencies of the `hello` Derivation are tightly coupled with `pkgs`.
1. If we need custom dependencies, we have to modify either `pkgs` or the content of `hello.nix`, which can be cumbersome.
2. In cases where `hello.nix` becomes complex, it's challenging to determine which Derivations from `pkgs` it relies on, making it difficult to analyze the dependencies between Derivations.
`pkgs.callPackage`, as a tool for parameterizing the construction of Derivations, addresses these issues. Let's take a look at its source code and comments [nixpkgs/lib/customisation.nix#L101-L121](https://github.com/NixOS/nixpkgs/blob/fe138d3/lib/customisation.nix#L101-L121):
In essence, `pkgs.callPackage` is used as `pkgs.callPackage fn args`, where `fn` is a Nix file or function, and `args` is an attribute set. Here's how it works:
1. `pkgs.callPackge fn args` first checks if `fn` is a function or a file. If it's a file, it imports the function defined within.
1. After this step, you have a function, typically with parameters like `lib`, `stdenv`, `fetchurl`, and possibly some custom parameters.
2. Next, `pkgs.callPackge fn args` merges `args` with the `pkgs` attribute set. If there are conflicts, the parameters in `args` will override those in `pkgs`.
3. Then, `pkgs.callPackge fn args` extracts the parameters of the `fn` function from the merged attribute set and uses them to execute the function.
4. The result of the function execution is a Derivation, which is a Nix package.
What can a Nix file or function, used as an argument to `pkgs.callPackge`, look like? You can examine examples we've used before: `hello.nix`, `fcitx5-rime.nix`, `vscode/with-extensions.nix`, and `firefox/common.nix`. All of them can be imported using `pkgs.callPackage`.
For instance, if you've defined a custom NixOS kernel configuration in `kernel.nix` and made the development branch name and kernel source code configurable:
```nix
{
@ -36,16 +85,16 @@ For example, let's say we have a customized NixOS kernel configuration file name
inherit src lib stdenv;
# File path to the generated kernel config file (`.config` generated by make menuconfig)
# file path to the generated kernel config file(the `.config` generated by make menuconfig)
#
# Here, we use a special usage to generate a file path from a string.
# here is a special usage to generate a file path from a string
configfile = ./. + "${boardName}_config";
allowImportFromDerivation = true;
})
```
We can use `pkgs.callPackage ./kernel.nix {}` in any Nix Module to import and replace any of its parameters:
You can use `pkgs.callPackage ./hello.nix {}` in any Nix module to import and use it, replacing any of its parameters as needed:
```nix
{ lib, pkgs, pkgsKernel, kernel-src, ... }:
@ -56,16 +105,27 @@ We can use `pkgs.callPackage ./kernel.nix {}` in any Nix Module to import and re
boot = {
# ......
kernelPackages = pkgs.linuxPackagesFor (pkgs.callPackage ./pkgs/kernel {
src = kernel-src; # The kernel source is passed as a `specialArgs` and injected into this module.
boardName = "licheepi4a"; # The board name, used to generate the kernel config file path.
src = kernel-src; # kernel source is passed as a `specialArgs` and injected into this module.
boardName = "licheepi4a"; # the board name, used to generate the kernel config file path.
});
# ......
}
```
In the example above, we use `pkgs.callPackage` to pass different `src` and `boardName` parameters to the function defined in `kernel.nix`. This allows us to generate different kernel packages. By changing the parameters passed to `pkgs.callPackage`, `kernel.nix` can adapt to different kernel sources and development boards.
As shown above, by using `pkgs.callPackage`, you can pass different `src` and `boardName` values to the `kernel.nix` function to generate different kernel packages. This allows you to adapt the same `kernel.nix` to different kernel source code and development boards.
The advantages of `pkgs.callPackage` are:
1. Derivation definitions are parameterized, and all dependencies of the Derivation are the function parameters in its definition. This makes it easy to analyze dependencies between Derivations.
2. All dependencies and other custom parameters of the Derivation can be easily replaced by using the second parameter of `pkgs.callPackage`, greatly enhancing Derivation reusability.
3. While achieving the above two functionalities, it does not increase code complexity, as all dependencies in `pkgs` can be automatically injected.
So it's always recommended to use `pkgs.callPackage` to define Derivations.
## References
- [Chapter 13. Callpackage Design Pattern - Nix Pills](https://nixos.org/guides/nix-pills/callpackage-design-pattern.html)
- [callPackage, a tool for the lazy - The Summer of Nix](https://summer.nixos.org/blog/callpackage-a-tool-for-the-lazy/)
- [Document what callPackage does and its preconditions - Nixpkgs Issues](https://github.com/NixOS/nixpkgs/issues/36354)

122
docs/zh/nixpkgs/callpackage.md
View File

@ -1,21 +1,109 @@
# pkgs.callPackage
前面我们介绍并大量使用了 `import xxx.nix` 来导入 Nix 文件，这种语法只是单纯地返回该文件的执行结果，不会对该结果进行进一步处理。
比如说 `xxx.nix` 的内容是形如 `{...}: {...}`，那么 `import xxx.nix` 的结果就是该文件中定义的这个函数。
`pkgs.callPackage` 被用于参数化构建 Nix 包，为了理解它的用处，我们首先考虑下不使用 `pkgs.callPakcage` 的情况下，我们要如何定义一个 Nix 包（也就是 Derivation）。
`pkgs.callPackage` 也是用来导入 Nix 文件的，它的语法是 `pkgs.callPackage xxx.nix { ... }`. 但跟 `import` 不同的是，它导入的 nix 文件内容必须是一个 Derivation 或者返回 Derivation 的函数，它的执行结果一定是一个 Derivation，也就是一个软件包。
## 1. 不使用 `pkgs.callPackage` 的情况
我们可以使用如下代码来定义一个 Nix 包：
```nix
pkgs.writeShellScriptBin "hello" ''echo "hello, ryan!"''
```
使用 `nix repl` 来验证一下，能看到它的执行结果确实是一个 Derivation：
```shell
› nix repl -f '<nixpkgs>'
Welcome to Nix 2.13.5. Type :? for help.
Loading installable ''...
Added 19203 variables.
nix-repl> pkgs.writeShellScriptBin "hello" '' echo "hello, xxx!" ''
«derivation /nix/store/zhgar12vfhbajbchj36vbbl3mg6762s8-hello.drv»
```
上面这个 Derivation 的定义很短，就一行，但 nixpkgs 中大部分的 Derivation 的定义都要比这复杂很多。
前面我们介绍并大量使用了 `import xxx.nix` 来从其他 Nix 文件中导入 Nix 表达式，我们可以在这里也使用这种方法来提升代码的可维护性：
1. 将上面这一行 Derivation 的定义存放到单独的文件 `hello.nix` 中。
1. 但 `hello.nix` 自身的上下文中不包含 `pkgs` 这个变量，所以需要修改下其内容，将 `pkgs` 作为参数传递给 `hello.nix`。
1. 在需要使用这个 Derivation 的地方，使用 `import ./hello.nix pkgs` 来导入它并使用 `pkgs` 作为参数来执行其中定义的函数。
仍然使用 `nix repl` 来验证一下，能看到它的执行结果仍然是一个 Derivation：
```shell
› cat hello.nix
pkgs:
pkgs.writeShellScriptBin "hello" '' echo "hello, xxx!" ''
› nix repl -f '<nixpkgs>'
Welcome to Nix 2.13.5. Type :? for help.
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
Loading installable ''...
Added 19203 variables.
nix-repl> import ./hello.nix pkgs
«derivation /nix/store/zhgar12vfhbajbchj36vbbl3mg6762s8-hello.drv»
```
## 2. 使用 `pkgs.callPackage` 的情况
在前面不使用 `pkgs.callPackage` 的例子中，我们直接将 `pkgs` 作为参数传到了 `hello.nix` 中，这样做的缺点有：
1. `hello` 这个 derivation 的所有其他依赖项都只能从 `pkgs` 中获取，耦合度太高。
1. 比如说我们如果需要其他自定义依赖项，就必须修改 `pkgs` 或者修改 `hello.nix` 的内容，而这两个都很麻烦。
1. 在 `hello.nix` 变复杂的情况下，很难判断 `hello.nix` 到底依赖了 `pkgs` 中的哪些 Derivation，很难分析 Derivation 之间的依赖关系。
而 `pkgs.callPackage` 作为一个参数化构建 Derivation 的工具函数，可解决上述两个问题。
首先看看源码中此函数的定义与注释 [nixpkgs/lib/customisation.nix#L101-L121](https://github.com/NixOS/nixpkgs/blob/fe138d3/lib/customisation.nix#L101-L121
)：
```nix
/* Call the package function in the file `fn` with the required
arguments automatically. The function is called with the
arguments `args`, but any missing arguments are obtained from
`autoArgs`. This function is intended to be partially
parameterised, e.g.,
callPackage = callPackageWith pkgs;
pkgs = {
libfoo = callPackage ./foo.nix { };
libbar = callPackage ./bar.nix { };
};
If the `libbar` function expects an argument named `libfoo`, it is
automatically passed as an argument. Overrides or missing
arguments can be supplied in `args`, e.g.
libbar = callPackage ./bar.nix {
libfoo = null;
enableX11 = true;
};
*/
callPackageWith = autoArgs: fn: args:
let
f = if lib.isFunction fn then fn else import fn;
fargs = lib.functionArgs f;
# All arguments that will be passed to the function
# This includes automatic ones and ones passed explicitly
allArgs = builtins.intersectAttrs fargs autoArgs // args;
# ...... 省略后面的内容 ......
```
简单的说，它的使用格式是 `pkgs.callPackage fn args`，其中 `fn` 是一个 nix 文件或者函数，`args` 是一个 attribute set，它的工作流程是：
1. `pkgs.callPackge fn args` 会先判断 `fn` 是一个函数还是一个文件，如果是文件就先通过 `import xxx.nix` 导入其中定义的函数。
1. 第一步执行完毕得到的是一个函数，其参数通常会有 `lib`, `stdenv`, `fetchurl` 等参数，可能还会带有一些自定义参数。
2. 之后，`pkgs.callPackge fn args` 会将 `args` 与 `pkgs` 这个 attribute set 合并。如果存在冲突，`args` 中的参数会覆盖 `pkgs` 中的参数。
3. 再之后，`pkgs.callPackge fn args` 会从上一步得到的 attribute set 中提取出 `fn` 函数的参数，并使用它们来执行 `fn` 函数。
4. 函数执行结果是一个 Derivation，也就是一个 Nix 包。
那可以作为 `pkgs.callPackge` 参数的 nix 文件具体长啥样呢，可以去看看我们前面举例过的 `hello.nix` `fcitx5-rime.nix` `vscode/with-extensions.nix` `firefox/common.nix`，它们都可以被 `pkgs.callPackage` 导入。
当 `pkgs.callPackge xxx.nix {...}` 中的 `xxx.nix`，其内容为一个函数时（绝大多数 nix 包都是如此），执行流程如下：
1. `pkgs.callPackge xxx.nix {...}` 会先 `import xxx.nix`，得到其中定义的函数，该函数的参数通常会有 `lib`, `stdenv`, `fetchurl` 等参数，以及一些自定义参数，自定义参数通常都有默认值。
2. 接着 `pkgs.callPackge` 会首先从当前环境中查找名称匹配的值，作为将要传递给前述函数的参数。像 `lib` `stdenv` `fetchurl` 这些都是 nixpkgs 中的函数，在这一步就会查找到它们。
3. 接着 `pkgs.callPackge` 会将其第二个参数 `{...}` 与前一步得到的参数集（attribute set）进行合并，得到一个新的参数列表，然后将其传递给该函数并执行。
4. 函数执行结果是一个 Derivation，也就是一个软件包。
这个函数比较常见的用途是用来导入一些自定义的 Nix 包。
比如说我们自定义了一个 NixOS 内核配置 `kernel.nix`，并且将开发版名称与内核源码作为了可变更参数：
```nix
@ -64,6 +152,16 @@
就如上面所展示的，通过 `pkgs.callPackage` 我们可以给 `kernel.nix` 定义的函数传入不同的 `src` 与 `boardName`，来生成不同的内核包，这样就可以使用同一份 `kernel.nix` 来适配不同的内核源码与不同的开发板了。
`pkgs.callPackage` 的优势在于：
1. Derivation 的定义被参数化，定义中的所有函数参数就是 Derivation 的所有依赖项，这样就可以很方便的分析 Derivation 之间的依赖关系。
2. Derivation 的所有依赖项与其他自定义参数都可以很方便地被替换（通过使用 `pkgs.callPackage` 的第二个参数），Derivation 定义的可复用性大大提升。
3. 在实现了前两条功能的情况下，并未增加代码的复杂度，所有 `pkgs` 中的依赖项都可以被自动注入，不需要手动传递。
因此我们总是推荐使用 `pkgs.callPackage` 来定义 Derivation。
## 参考
- [Chapter 13. Callpackage Design Pattern - Nix Pills](https://nixos.org/guides/nix-pills/callpackage-design-pattern.html)
- [callPackage, a tool for the lazy - The Summer of Nix](https://summer.nixos.org/blog/callpackage-a-tool-for-the-lazy/)
- [Document what callPackage does and its preconditions - Nixpkgs Issues](https://github.com/NixOS/nixpkgs/issues/36354)