forked from extern/nushell
Make EngineState clone cheaper with Arc on all of the heavy objects (#12229)
# Description This makes many of the larger objects in `EngineState` into `Arc`, and uses `Arc::make_mut` to do clone-on-write if the reference is not unique. This is generally very cheap, giving us the best of both worlds - allowing us to mutate without cloning if we have an exclusive reference, and cloning if we don't. This started as more of a curiosity for me after remembering that `Arc::make_mut` exists and can make using `Arc` for mostly immutable data that sometimes needs to be changed very convenient, and also after hearing someone complain about memory usage on Discord - this is a somewhat significant win for that. The exact objects that were wrapped in `Arc`: - `files`, `file_contents` - the strings and byte buffers - `decls` - the whole `Vec`, but mostly to avoid lots of individual `malloc()` calls on Clone rather than for memory usage - `blocks` - the blocks themselves, rather than the outer Vec - `modules` - the modules themselves, rather than the outer Vec - `env_vars`, `previous_env_vars` - the entire maps - `config` The changes required were relatively minimal, but this is a breaking API change. In particular, blocks are added as Arcs, to allow the parser cache functionality to work. With my normal nu config, running on Linux, this saves me about 15 MiB of process memory usage when running interactively (65 MiB → 50 MiB). This also makes quick command executions cheaper, particularly since every REPL loop now involves a clone of the engine state so that we can recover from a panic. It also reduces memory usage where engine state needs to be cloned and sent to another thread or kept within an iterator. # User-Facing Changes Shouldn't be any, since it's all internal stuff, but it does change some public interfaces so it's a breaking change
This commit is contained in:
@ -229,7 +229,7 @@ pub fn find_non_whitespace_index(contents: &[u8], start: usize) -> usize {
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_passthrough_command(working_set_file_contents: &[(Vec<u8>, usize, usize)]) -> bool {
|
||||
pub fn is_passthrough_command(working_set_file_contents: &[(Arc<Vec<u8>>, usize, usize)]) -> bool {
|
||||
for (contents, _, _) in working_set_file_contents {
|
||||
let last_pipe_pos_rev = contents.iter().rev().position(|x| x == &b'|');
|
||||
let last_pipe_pos = last_pipe_pos_rev.map(|x| contents.len() - x).unwrap_or(0);
|
||||
|
@ -124,9 +124,9 @@ impl NuCompleter {
|
||||
|
||||
let output = parse(&mut working_set, Some("completer"), line.as_bytes(), false);
|
||||
|
||||
for pipeline in output.pipelines.into_iter() {
|
||||
for pipeline_element in pipeline.elements {
|
||||
let flattened = flatten_pipeline_element(&working_set, &pipeline_element);
|
||||
for pipeline in output.pipelines.iter() {
|
||||
for pipeline_element in &pipeline.elements {
|
||||
let flattened = flatten_pipeline_element(&working_set, pipeline_element);
|
||||
let mut spans: Vec<String> = vec![];
|
||||
|
||||
for (flat_idx, flat) in flattened.iter().enumerate() {
|
||||
|
@ -1,3 +1,5 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::util::eval_source;
|
||||
use log::info;
|
||||
use log::trace;
|
||||
@ -117,7 +119,7 @@ pub fn evaluate_file(
|
||||
std::process::exit(1);
|
||||
}
|
||||
|
||||
for block in &mut working_set.delta.blocks {
|
||||
for block in working_set.delta.blocks.iter_mut().map(Arc::make_mut) {
|
||||
if block.signature.name == "main" {
|
||||
block.signature.name = source_filename.to_string_lossy().to_string();
|
||||
} else if block.signature.name.starts_with("main ") {
|
||||
|
Reference in New Issue
Block a user