From 8d4710b1325712ef91159e7ac4291911ef1c8fd2 Mon Sep 17 00:00:00 2001
From: Ryan Yin <xiaoyin_c@qq.com>
Date: Tue, 18 Jul 2023 13:13:24 +0800
Subject: [PATCH] feat: dev-environments - python

---
 docs/development/dev-environments.md    | 62 +++++++++++++++++++++++++
 docs/zh/development/dev-environments.md | 51 ++++++++++++++++++++
 2 files changed, 113 insertions(+)

diff --git a/docs/development/dev-environments.md b/docs/development/dev-environments.md
index cf873e0..3c1f4c4 100644
--- a/docs/development/dev-environments.md
+++ b/docs/development/dev-environments.md
@@ -9,3 +9,65 @@ Luckily, some people in the community have done this for us. The following repos
 If you think the structure of `flake.nix` is still too complicated and want a simpler way, you can also consider using the following project, which encapsulates Nix more thoroughly and provides users with a simpler definition:
 
 - [cachix/devenv](https://github.com/cachix/devenv)
+
+## Dev Environment for Python
+
+The development environment for Python is much more cumbersome compared to languages like Java or Go because it defaults to installing software in the global environment. 
+To install software for the current project, you must create a virtual environment first (unlike in languages such as JavaScript or Go,
+where virtual environments are not necessary). This behavior is very unfriendly for Nix.
+
+By default, when using pip in Python, it installs software globally. On NixOS, running `pip install` directly will result in an error:
+
+```bash
+› pip install -r requirements.txt
+error: externally-managed-environment
+
+× This environment is externally managed
+╰─> This command has been disabled as it tries to modify the immutable
+    `/nix/store` filesystem.
+
+    To use Python with Nix and nixpkgs, have a look at the online documentation:
+    <https://nixos.org/manual/nixpkgs/stable/#python>.
+
+note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
+hint: See PEP 668 for the detailed specification.
+```
+
+Based on the error message, `pip install` is directly disabled by NixOS. Even when attempting `pip install --user`, it is similarly disabled. 
+To improve the reproducibility of the environment, Nix eliminates these commands altogether. 
+Even if we create a new environment using methods like `mkShell`, 
+these commands still result in errors (presumably because the pip command in Nixpkgs itself has 
+been modified to prevent any modification instructions like `install` from running).
+
+However, many project installation scripts are based on pip, which means these scripts cannot be used directly.
+Additionally, the content in nixpkgs is limited, and many packages from PyPI are missing.
+This requires users to package them themselves, adding a lot of complexity and mental burden.
+
+One solution is to use the `venv` virtual environment. Within a virtual environment, you can use commands like pip normally:
+
+```shell
+python -m venv ./env
+source ./env/bin/activate
+```
+
+Alternatively, you can use a third-party tool called `virtualenv`, but this requires additional installation.
+
+For those who still lack confidence in the venv created directly with Python, they may prefer to include the virtual environment in `/nix/store` to make it immutable.
+This can be achieved by directly installing the dependencies from `requirements.txt` or `poetry.toml` using Nix.
+There are existing Nix packaging tools available to assist with this:
+
+> Note that even in these environments, running commands like `pip install` directly will still fail.
+Python dependencies must be installed through `flake.nix` because the data is located in the `/nix/store` directory,
+and these modification commands can only be executed during the Nix build phase.
+
+- [DavHau/mach-nix](https://github.com/DavHau/mach-nix)
+- [poetry2nix](https://github.com/nix-community/poetry2nix)
+
+The advantage of these tools is that they utilize the lock mechanism of Nix Flakes to improve reproducibility.
+However, the downside is that they add an extra layer of abstraction, making the underlying system more complex.
+
+
+Finally, in some more complex projects, neither of the above solutions may be feasible.
+In such cases, the best solution is to use containers such as Docker or Podman. Containers have fewer restrictions compared to Nix and can provide the best compatibility.
+
+
diff --git a/docs/zh/development/dev-environments.md b/docs/zh/development/dev-environments.md
index cdb444a..80025a6 100644
--- a/docs/zh/development/dev-environments.md
+++ b/docs/zh/development/dev-environments.md
@@ -9,3 +9,54 @@
 如果你觉得 `flake.nix` 的结构还是太复杂了，希望能有更简单的方法，也可以考虑使用下面这个项目，它对 Nix 做了更彻底的封装，对用户提供了更简单的定义：
 
 - [cachix/devenv](https://github.com/cachix/devenv)
+
+
+## Python 开发环境
+
+Python 的开发环境比 Java/Go 等语言要麻烦许多，因为它默认就往全局环境装软件，要往当前项目装，还必须得先创建虚拟环境（JS/Go 等语言里可没虚拟环境这种幺蛾子）。
+这对 Nix 而言是非常不友好的行为。
+
+Python 的 pip 默认会将软件安装到全局，在 NixOS 中 `pip install` 会直接报错：
+
+```shell
+› pip install -r requirements.txt
+error: externally-managed-environment
+
+× This environment is externally managed
+╰─> This command has been disabled as it tries to modify the immutable
+    `/nix/store` filesystem.
+
+    To use Python with Nix and nixpkgs, have a look at the online documentation:
+    <https://nixos.org/manual/nixpkgs/stable/#python>.
+
+note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
+hint: See PEP 668 for the detailed specification.
+```
+
+根据错误信息，`pip install` 直接被 NixOS 禁用掉了，测试了 `pip install --user` 也同样被禁用。为了提升环境的可复现能力，Nix 把它们全部废掉了。
+即使我们通过 `mkShell` 等方式创建一个新环境，这些命令照样会报错（猜测是 Nixpkgs 中的 pip 命令本身就被魔改了，只要是跑 `install` 等修改指令就直接嘎掉）。
+
+但是很多项目的安装脚本都是基于 pip 的，这导致这些脚本都不能直接使用，而且另一方面 nixpkgs 中的内容有限，很多 pypi 中的包里边都没有，还得自己打包，相对麻烦很多，也加重了用户的心智负担。
+
+解决方案之一是改用 `venv` 虚拟环境，在虚拟环境里当然就能正常使用 pip 等命令了：
+
+```shell
+python -m venv ./env
+source ./env/bin/activate
+```
+
+或者使用第三方工具 `virtualenv`，缺点是这个需要额外安装。
+
+这样用 python 直接创建的 venv，对一些人而言可能还是没有安全感，仍然希望将这个虚拟环境也弄进 `/nix/store` 里使其不可变，通过 nix 直接安装 `requirements.txt` 或者 `poetry.toml` 中的依赖项。
+这当然是可行的，有现成的 Nix 封装工具帮我们干这个活：
+
+> 注意即使是在这俩环境中，直接跑 `pip install` 之类的安装命令仍然是会失败的，必须通过 `flake.nix` 来安装 Python 依赖！
+因为数据还是在 `/nix/store` 中，这类修改命令必须在 Nix 的构建阶段才能执行...
+
+- [DavHau/mach-nix](https://github.com/DavHau/mach-nix)
+- [poetry2nix](https://github.com/nix-community/poetry2nix)
+
+这俩工具的好处是，能利用上 Nix Flakes 的锁机制来提升可复现能力，缺点是多了一层封装，底层变得更复杂了。
+
+
+最后，在一些更复杂的项目上，上述两种方案可能都行不通，这时候最佳的解决方案，就是改用容器了，比如 Docker、Podman 等，容器的限制没 Nix 这么严格，能提供最佳的兼容性。