nushell/crates/nu-cmd-lang
Devyn Cairns cf321ab510
Make EngineState clone cheaper with Arc on all of the heavy objects (#12229)
# Description
This makes many of the larger objects in `EngineState` into `Arc`, and
uses `Arc::make_mut` to do clone-on-write if the reference is not
unique. This is generally very cheap, giving us the best of both worlds
- allowing us to mutate without cloning if we have an exclusive
reference, and cloning if we don't.

This started as more of a curiosity for me after remembering that
`Arc::make_mut` exists and can make using `Arc` for mostly immutable
data that sometimes needs to be changed very convenient, and also after
hearing someone complain about memory usage on Discord - this is a
somewhat significant win for that.

The exact objects that were wrapped in `Arc`:

- `files`, `file_contents` - the strings and byte buffers
- `decls` - the whole `Vec`, but mostly to avoid lots of individual
`malloc()` calls on Clone rather than for memory usage
- `blocks` - the blocks themselves, rather than the outer Vec
- `modules` - the modules themselves, rather than the outer Vec
- `env_vars`, `previous_env_vars` - the entire maps
- `config`

The changes required were relatively minimal, but this is a breaking API
change. In particular, blocks are added as Arcs, to allow the parser
cache functionality to work.

With my normal nu config, running on Linux, this saves me about 15 MiB
of process memory usage when running interactively (65 MiB → 50 MiB).

This also makes quick command executions cheaper, particularly since
every REPL loop now involves a clone of the engine state so that we can
recover from a panic. It also reduces memory usage where engine state
needs to be cloned and sent to another thread or kept within an
iterator.

# User-Facing Changes
Shouldn't be any, since it's all internal stuff, but it does change some
public interfaces so it's a breaking change
2024-03-19 19:07:00 +01:00
..
src Make EngineState clone cheaper with Arc on all of the heavy objects (#12229) 2024-03-19 19:07:00 +01:00
build.rs Cratification: Break out nu_cmd_lang into a separate crate (#8181) 2023-02-24 09:54:42 -06:00
Cargo.toml Remove feat extra and include in default (#12140) 2024-03-10 17:29:02 +01:00
LICENSE Fix rest of license year ranges (#8727) 2023-04-04 09:03:29 +12:00
README.md nu-cmd-lang readme (#9446) 2023-06-15 08:45:23 -07:00

nu-cmd-lang

the base language and command crate of nu

The commands in this crate are the core commands of the nu language. It is also the base crate upon which all other command crates sit on top of including:

  • nu-command
  • nu-cli
  • nu-cmd-dataframe
  • nu-cmd-extra

As time goes on and the nu language develops further in parallel with nushell we will be adding other command crates to the system.

What does it mean to be a base crate ?

A base crate is one with minimal dependencies in our system so that other developers can come along and use this crate without having a lot of baggage in terms of other crates which will bloat their underlying application.

Background on nu-cmd-lang

This crate was designed to be a small, concise set of tools or commands that serve as the foundation layer of both nu and nushell. These are the core commands needed to have a nice working version of the nu language without all of the support that the other commands provide inside nushell. Prior to the launch of this crate all of our commands were housed in the crate nu-command. Moving forward we would like to slowly break out the commands in nu-command into different crates; the naming and how this will work and where all the commands will be located is a "work in progress" especially now that the standard library is starting to become more popular as a location for commands. As time goes on some of our commands written in rust will be migrated to nu and when this happens they will be moved into the standard library.