forked from extern/nushell
3b5172a8fa
This is an attempt to implement a new `Value::LazyRecord` variant for performance reasons. `LazyRecord` is like a regular `Record`, but it's possible to access individual columns without evaluating other columns. I've implemented `LazyRecord` for the special `$nu` variable; accessing `$nu` is relatively slow because of all the information in `scope`, and [`$nu` accounts for about 2/3 of Nu's startup time on Linux](https://github.com/nushell/nushell/issues/6677#issuecomment-1364618122). ### Benchmarks I ran some benchmarks on my desktop (Linux, 12900K) and the results are very pleasing. Nu's time to start up and run a command (`cargo build --release; hyperfine 'target/release/nu -c "echo \"Hello, world!\""' --shell=none --warmup 10`) goes from **8.8ms to 3.2ms, about 2.8x faster**. Tests are also much faster! Running `cargo nextest` (with our very slow `proptest` tests disabled) goes from **7.2s to 4.4s (1.6x faster)**, because most tests involve launching a new instance of Nu. ### Design (updated) I've added a new `LazyRecord` trait and added a `Value` variant wrapping those trait objects, much like `CustomValue`. `LazyRecord` implementations must implement these 2 functions: ```rust // All column names fn column_names(&self) -> Vec<&'static str>; // Get 1 specific column value fn get_column_value(&self, column: &str) -> Result<Value, ShellError>; ``` ### Serializability `Value` variants must implement `Serializable` and `Deserializable`, which poses some problems because I want to use unserializable things like `EngineState` in `LazyRecord`s. To work around this, I basically lie to the type system: 1. Add `#[typetag::serde(tag = "type")]` to `LazyRecord` to make it serializable 2. Any unserializable fields in `LazyRecord` implementations get marked with `#[serde(skip)]` 3. At the point where a `LazyRecord` normally would get serialized and sent to a plugin, I instead collect it into a regular `Value::Record` (which can be serialized)
35 lines
1.1 KiB
Rust
35 lines
1.1 KiB
Rust
use crate::{ShellError, Span, Value};
|
|
use std::fmt;
|
|
|
|
// Trait definition for a lazy record (where columns are evaluated on-demand)
|
|
// typetag is needed to make this implement Serialize+Deserialize... even though we should never actually serialize a LazyRecord.
|
|
// To serialize a LazyRecord, collect it into a Value::Record with collect() first.
|
|
#[typetag::serde(tag = "type")]
|
|
pub trait LazyRecord: fmt::Debug + Send + Sync {
|
|
// All column names
|
|
fn column_names(&self) -> Vec<&'static str>;
|
|
|
|
// Get 1 specific column value
|
|
fn get_column_value(&self, column: &str) -> Result<Value, ShellError>;
|
|
|
|
fn span(&self) -> Span;
|
|
|
|
// Convert the lazy record into a regular Value::Record by collecting all its columns
|
|
fn collect(&self) -> Result<Value, ShellError> {
|
|
let mut cols = vec![];
|
|
let mut vals = vec![];
|
|
|
|
for column in self.column_names() {
|
|
cols.push(column.into());
|
|
let val = self.get_column_value(column)?;
|
|
vals.push(val);
|
|
}
|
|
|
|
Ok(Value::Record {
|
|
cols,
|
|
vals,
|
|
span: self.span(),
|
|
})
|
|
}
|
|
}
|