diff --git a/docs/development/dev-environments.md b/docs/development/dev-environments.md index cf873e0..3c1f4c4 100644 --- a/docs/development/dev-environments.md +++ b/docs/development/dev-environments.md @@ -9,3 +9,65 @@ Luckily, some people in the community have done this for us. The following repos If you think the structure of `flake.nix` is still too complicated and want a simpler way, you can also consider using the following project, which encapsulates Nix more thoroughly and provides users with a simpler definition: - [cachix/devenv](https://github.com/cachix/devenv) + +## Dev Environment for Python + +The development environment for Python is much more cumbersome compared to languages like Java or Go because it defaults to installing software in the global environment. +To install software for the current project, you must create a virtual environment first (unlike in languages such as JavaScript or Go, +where virtual environments are not necessary). This behavior is very unfriendly for Nix. + +By default, when using pip in Python, it installs software globally. On NixOS, running `pip install` directly will result in an error: + +```bash +› pip install -r requirements.txt +error: externally-managed-environment + +× This environment is externally managed +╰─> This command has been disabled as it tries to modify the immutable + `/nix/store` filesystem. + + To use Python with Nix and nixpkgs, have a look at the online documentation: + . + +note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. +hint: See PEP 668 for the detailed specification. +``` + +Based on the error message, `pip install` is directly disabled by NixOS. Even when attempting `pip install --user`, it is similarly disabled. +To improve the reproducibility of the environment, Nix eliminates these commands altogether. +Even if we create a new environment using methods like `mkShell`, +these commands still result in errors (presumably because the pip command in Nixpkgs itself has +been modified to prevent any modification instructions like `install` from running). + +However, many project installation scripts are based on pip, which means these scripts cannot be used directly. +Additionally, the content in nixpkgs is limited, and many packages from PyPI are missing. +This requires users to package them themselves, adding a lot of complexity and mental burden. + +One solution is to use the `venv` virtual environment. Within a virtual environment, you can use commands like pip normally: + +```shell +python -m venv ./env +source ./env/bin/activate +``` + +Alternatively, you can use a third-party tool called `virtualenv`, but this requires additional installation. + +For those who still lack confidence in the venv created directly with Python, they may prefer to include the virtual environment in `/nix/store` to make it immutable. +This can be achieved by directly installing the dependencies from `requirements.txt` or `poetry.toml` using Nix. +There are existing Nix packaging tools available to assist with this: + +> Note that even in these environments, running commands like `pip install` directly will still fail. +Python dependencies must be installed through `flake.nix` because the data is located in the `/nix/store` directory, +and these modification commands can only be executed during the Nix build phase. + +- [DavHau/mach-nix](https://github.com/DavHau/mach-nix) +- [poetry2nix](https://github.com/nix-community/poetry2nix) + +The advantage of these tools is that they utilize the lock mechanism of Nix Flakes to improve reproducibility. +However, the downside is that they add an extra layer of abstraction, making the underlying system more complex. + + +Finally, in some more complex projects, neither of the above solutions may be feasible. +In such cases, the best solution is to use containers such as Docker or Podman. Containers have fewer restrictions compared to Nix and can provide the best compatibility. + + diff --git a/docs/zh/development/dev-environments.md b/docs/zh/development/dev-environments.md index cdb444a..80025a6 100644 --- a/docs/zh/development/dev-environments.md +++ b/docs/zh/development/dev-environments.md @@ -9,3 +9,54 @@ 如果你觉得 `flake.nix` 的结构还是太复杂了,希望能有更简单的方法,也可以考虑使用下面这个项目,它对 Nix 做了更彻底的封装,对用户提供了更简单的定义: - [cachix/devenv](https://github.com/cachix/devenv) + + +## Python 开发环境 + +Python 的开发环境比 Java/Go 等语言要麻烦许多,因为它默认就往全局环境装软件,要往当前项目装,还必须得先创建虚拟环境(JS/Go 等语言里可没虚拟环境这种幺蛾子)。 +这对 Nix 而言是非常不友好的行为。 + +Python 的 pip 默认会将软件安装到全局,在 NixOS 中 `pip install` 会直接报错: + +```shell +› pip install -r requirements.txt +error: externally-managed-environment + +× This environment is externally managed +╰─> This command has been disabled as it tries to modify the immutable + `/nix/store` filesystem. + + To use Python with Nix and nixpkgs, have a look at the online documentation: + . + +note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages. +hint: See PEP 668 for the detailed specification. +``` + +根据错误信息,`pip install` 直接被 NixOS 禁用掉了,测试了 `pip install --user` 也同样被禁用。为了提升环境的可复现能力,Nix 把它们全部废掉了。 +即使我们通过 `mkShell` 等方式创建一个新环境,这些命令照样会报错(猜测是 Nixpkgs 中的 pip 命令本身就被魔改了,只要是跑 `install` 等修改指令就直接嘎掉)。 + +但是很多项目的安装脚本都是基于 pip 的,这导致这些脚本都不能直接使用,而且另一方面 nixpkgs 中的内容有限,很多 pypi 中的包里边都没有,还得自己打包,相对麻烦很多,也加重了用户的心智负担。 + +解决方案之一是改用 `venv` 虚拟环境,在虚拟环境里当然就能正常使用 pip 等命令了: + +```shell +python -m venv ./env +source ./env/bin/activate +``` + +或者使用第三方工具 `virtualenv`,缺点是这个需要额外安装。 + +这样用 python 直接创建的 venv,对一些人而言可能还是没有安全感,仍然希望将这个虚拟环境也弄进 `/nix/store` 里使其不可变,通过 nix 直接安装 `requirements.txt` 或者 `poetry.toml` 中的依赖项。 +这当然是可行的,有现成的 Nix 封装工具帮我们干这个活: + +> 注意即使是在这俩环境中,直接跑 `pip install` 之类的安装命令仍然是会失败的,必须通过 `flake.nix` 来安装 Python 依赖! +因为数据还是在 `/nix/store` 中,这类修改命令必须在 Nix 的构建阶段才能执行... + +- [DavHau/mach-nix](https://github.com/DavHau/mach-nix) +- [poetry2nix](https://github.com/nix-community/poetry2nix) + +这俩工具的好处是,能利用上 Nix Flakes 的锁机制来提升可复现能力,缺点是多了一层封装,底层变得更复杂了。 + + +最后,在一些更复杂的项目上,上述两种方案可能都行不通,这时候最佳的解决方案,就是改用容器了,比如 Docker、Podman 等,容器的限制没 Nix 这么严格,能提供最佳的兼容性。