@jntrnr and I discussed the fact that we can now *graduate* nuon to be a
first class citizen...
This PR moves
* from nuon
* to nuon
out of the *experimental* stage and into *formats*
# Description
`Span` is `Copy`, so we probably should not be passing references of
`Span` around. This PR replaces all instances of `&Span` with `Span`,
copying spans where necessary.
# User-Facing Changes
This alters some public functions to take `Span` instead of `&Span` as
input. Namely, `EngineState::get_span_contents`,
`nu_protocol::extract_value`, a bunch of the math commands, and
`Gstat::gstat`.
# Description
- A new one is the removal of unnecessary `#` in raw strings without `"`
inside.
-
https://rust-lang.github.io/rust-clippy/master/index.html#/needless_raw_string_hashes
- The automatically applied removal of `.into_iter()` touched several
places where #9648 will change to the use of the record API. If
necessary I can remove them @IanManske to avoid churn with this PR.
- Manually applied `.try_fold` in two places
- Removed a dead `if`
- Manual: Combat rightward-drift with early return
requires
- https://github.com/nushell/nushell/pull/9455
# ⚙️ Description
in this PR i move the commands we've all agreed, in the core team, to
move out of the core Nushell to the `extra` feature.
> **Warning**
> in the first commits here, i've
> - moved the implementations to `nu-cmd-extra`
> - removed the declaration of all the commands below from `nu-command`
> - made sure the commands were not available anymore with `cargo run --
-n`
## the list of commands to move
with the current command table downloaded as `commands.csv`, i've run
```bash
let commands = (
open commands.csv
| where is_plugin == "FALSE" and category != "deprecated"
| select name category "approv. %"
| rename name category approval
| insert treated {|it| (
($it.approval == 100) or # all the core team agreed on them
($it.name | str starts-with "bits") or # see https://github.com/nushell/nushell/pull/9241
($it.name | str starts-with "dfr") # see https://github.com/nushell/nushell/pull/9327
)}
)
```
to preprocess them and then
```bash
$commands | where {|it| (not $it.treated) and ($it.approval == 0)}
```
to get all untreated commands with no approval, which gives
```
╭────┬───────────────┬─────────┬─────────────┬──────────╮
│ # │ name │ treated │ category │ approval │
├────┼───────────────┼─────────┼─────────────┼──────────┤
│ 0 │ fmt │ false │ conversions │ 0 │
│ 1 │ each while │ false │ filters │ 0 │
│ 2 │ roll │ false │ filters │ 0 │
│ 3 │ roll down │ false │ filters │ 0 │
│ 4 │ roll left │ false │ filters │ 0 │
│ 5 │ roll right │ false │ filters │ 0 │
│ 6 │ roll up │ false │ filters │ 0 │
│ 7 │ rotate │ false │ filters │ 0 │
│ 8 │ update cells │ false │ filters │ 0 │
│ 9 │ decode hex │ false │ formats │ 0 │
│ 10 │ encode hex │ false │ formats │ 0 │
│ 11 │ from url │ false │ formats │ 0 │
│ 12 │ to html │ false │ formats │ 0 │
│ 13 │ ansi gradient │ false │ platform │ 0 │
│ 14 │ ansi link │ false │ platform │ 0 │
│ 15 │ format │ false │ strings │ 0 │
╰────┴───────────────┴─────────┴─────────────┴──────────╯
```
# 🖌️ User-Facing Changes
```
$nothing
```
# 🧪 Tests + Formatting
- ⚫ `toolkit fmt`
- ⚫ `toolkit clippy`
- ⚫ `toolkit test`
- ⚫ `toolkit test stdlib`
# 📖 After Submitting
```
$nothing
```
# 🔍 For reviewers
```bash
$commands | where {|it| (not $it.treated) and ($it.approval == 0)} | each {|command|
try {
help $command.name | ignore
} catch {|e|
$"($command.name): ($e.msg)"
}
}
```
should give no output in `cargo run --features extra -- -n` and a table
with 16 lines in `cargo run -- -n`
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
This PR fixes the following nightly clippy warnings.
```
warning: you should consider adding a `Default` implementation for `HjsonFormatter<'a>`
--> crates/nu-json/src/ser.rs:700:5
|
700 | / pub fn new() -> Self {
701 | | HjsonFormatter::with_indent(b" ")
702 | | }
| |_____^
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#new_without_default
= note: `#[warn(clippy::new_without_default)]` on by default
help: try adding this
|
698 + impl<'a> Default for HjsonFormatter<'a> {
699 + fn default() -> Self {
700 + Self::new()
701 + }
702 + }
|
warning: `nu-json` (lib) generated 1 warning
warning: private item shadows public glob re-export
--> crates/nu-command/src/strings/mod.rs:8:1
|
8 | mod str_;
| ^^^^^^^^^
|
note: the name `str_` in the type namespace is supposed to be publicly re-exported here
--> crates/nu-command/src/strings/mod.rs:17:9
|
17 | pub use str_::*;
| ^^^^^^^
note: but the private item here shadows it
--> crates/nu-command/src/strings/mod.rs:8:1
|
8 | mod str_;
| ^^^^^^^^^
= note: `#[warn(hidden_glob_reexports)]` on by default
warning: incorrect NaN comparison, NaN cannot be directly compared to itself
--> crates/nu-command/src/formats/to/nuon.rs:186:20
|
186 | && val != &f64::NAN
| ^^^^^^^^^^^^^^^^
|
= note: `#[warn(invalid_nan_comparisons)]` on by default
help: use `f32::is_nan()` or `f64::is_nan()` instead
|
186 - && val != &f64::NAN
186 + && !val.is_nan()
|
warning: `nu-command` (lib) generated 2 warnings (run `cargo clippy --fix --lib -p nu-command` to apply 1 suggestion)
Compiling nu v0.81.1 (/data/source/nushell)
warning: this expression creates a reference which is immediately dereferenced by the compiler
--> crates/nu-command/tests/commands/rm.rs:392:27
|
392 | dir_to_clean: &test_dir,
| ^^^^^^^^^ help: change this to: `test_dir`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
= note: `#[warn(clippy::needless_borrow)]` on by default
warning: `nu-command` (test "main") generated 1 warning (run `cargo clippy --fix --test "main"` to apply 1 suggestion)
warning: `nu-command` (lib test) generated 2 warnings (2 duplicates)
warning: `nu-json` (lib test) generated 1 warning (1 duplicate)
Finished dev [unoptimized + debuginfo] target(s) in 3.89s
```
# User-Facing Changes
N/A
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the
standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This PR reverts https://github.com/nushell/nushell/pull/9391
We try not to revert PRs like this, though after discussion with the
Nushell team, we decided to revert this one.
The main reason is that Nushell, as a codebase, isn't ready for these
kinds of optimisations. It's in the part of the development cycle where
our main focus should be on improving the algorithms inside of Nushell
itself. Once we have matured our algorithms, then we can look for
opportunities to switch out technologies we're using for alternate
forms.
Much of Nushell still has lots of opportunities for tuning the codebase,
paying down technical debt, and making the codebase generally cleaner
and more robust. This should be the focus. Performance improvements
should flow out of that work.
Said another, optimisation that isn't part of tuning the codebase is
premature at this stage. We need to focus on doing the hard work of
making the engine, parser, etc better.
# User-Facing Changes
Reverts the HashMap -> ahash change.
cc @FilipAndersson245
# Description
see https://github.com/nushell/nushell/issues/9390
using `ahash` instead of the default hasher. this will not affect
compile time as we where already building `ahash`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the
standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This adds `match` and basic pattern matching.
An example:
```
match $x {
1..10 => { print "Value is between 1 and 10" }
{ foo: $bar } => { print $"Value has a 'foo' field with value ($bar)" }
[$a, $b] => { print $"Value is a list with two items: ($a) and ($b)" }
_ => { print "Value is none of the above" }
}
```
Like the recent changes to `if` to allow it to be used as an expression,
`match` can also be used as an expression. This allows you to assign the
result to a variable, eg) `let xyz = match ...`
I've also included a short-hand pattern for matching records, as I think
it might help when doing a lot of record patterns: `{$foo}` which is
equivalent to `{foo: $foo}`.
There are still missing components, so consider this the first step in
full pattern matching support. Currently missing:
* Patterns for strings
* Or-patterns (like the `|` in Rust)
* Patterns for tables (unclear how we want to match a table, so it'll
need some design)
* Patterns for binary values
* And much more
# User-Facing Changes
[see above]
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
Should close#7255.
# Description
**TL;DR**: this PR adds `--indent <int>`, `--tabs <int>` and `--raw` to
control a bit more the `string` output of `to nuon`, as done in `to
json` already, the goal being to promote the `NUON` format through easy
to read and formatted output `.nuon` files 😋
### outside of `crates/nu-command/src/formats/to/nuon.rs`
as the signature of `value_to_string` has changed, the single call to it
outside of its module definition has been changed to use default values
=> `value_to_string(&value, Span::unknown(), 0, &None)` in
`crates/nu-command/src/filters/uniq.rs`
### changes to `ToNuon` in `crates/nu-command/src/formats/to/nuon.rs`
- the signature now features `--raw`, `--indent <int>` and `--tabs
<int>`
- the structure of the `run` method is inspired from the one in `to
json`
- we get the values of the arguments
- we convert the input to a usable `Value`
- depending on whether the user raised `--raw`, `--indent` or `--tabs`,
we call the conversion to `string` with different values of the
indentation, starting at depth 0
- finally, we return `Ok` or a `ShellError::CantConvert` depending on
the conversion result
- some tool functions
- `get_true_indentation` gives the full indentation => `indent` repeated
`depth` times
- `get_true_separators` gives the line and field separators => a `("\n",
"")` when using some formatting or `("", " ")` when converting as pure
string on a single line
the meat of `nuon.rs` is now the `value_to_string` recursive function:
- takes the depth and the indent string
- adds correct newlines, space separators and indentation to the output
- calls itself with the same indent string but `depth + 1` to increase
the indentation by one level
- i used the `nl`, `idt`, `idt_po` (**i**n**d**en**t** **p**lus **o**ne)
and `idt_pt` (**i**n**d**en**t** **p**lus **t**wo) to make the
`format!`s easier to read
# User-Facing Changes
users can now
- control the amount and nature of NUON string output indentation with
- `--indent <number of " " per level>`
- `--tabs <number of "\t" per level>`
- use the previous behaviour of `to nuon` with the `--raw` option
- have new examples with `help to nuon`
> **Note**
> the priority order of the options is the following
> 1. `--raw`
> 2. `--tabs`
> 3. `--indent`
>
> the default is `--indent 2`
# Tests + Formatting
### new tests
- tests involving the string output of `to nuon`, i.e. tests not of the
form `... | to nuon | from nuon ...`, now use the `to nuon --raw`
command => this is the smallest change to have the tests pass, as the
new `to nuon --raw` is equivalent to the old `to nuon`
- in `crates/nu-command/src/formats/to/nuon.rs`, the previous example
has been replaced with three examples
- `[1 2 3] | to nuon` to show the default behaviour
- `[1 2 3] | to nuon --raw` to show the not-formatted output
- a more complex example with `{date: 2000-01-01, data: [1 [2 3] 4.56]}
| to nuon`
- the result values have been defined and the `examples` tests pass
### dev
- 🟢 `cargo fmt --all`
- 🟢 `cargo clippy --workspace -- -D warnings -D
clippy::unwrap_used -A clippy::needless_collect`
- 🟢 `cargo test --workspace` ~~passes but without
`to_nuon_errs_on_closure`~~ fixed in
0b4fad7eff
# After Submitting
the `to nuon` page would have to be regenerated at some point due to the
new tests
# Description
Our `ShellError` at the moment has a `std::mem::size_of<ShellError>` of
136 bytes (on AMD64). As a result `Value` directly storing the struct
also required 136 bytes (thanks to alignment requirements).
This change stores the `Value::Error` `ShellError` on the heap.
Pro:
- Value now needs just 80 bytes
- Should be 1 cacheline less (still at least 2 cachelines)
Con:
- More small heap allocations when dealing with `Value::Error`
- More heap fragmentation
- Potential for additional required memcopies
# Further code changes
Includes a small refactor of `try` due to a type mismatch in its large
match.
# User-Facing Changes
None for regular users.
Plugin authors may have to update their matches on `Value` if they use
`nu-protocol`
Needs benchmarking to see if there is a benefit in real world workloads.
**Update** small improvements in runtime for workloads with high volume
of values. Significant reduction in maximum resident set size, when many
values are held in memory.
# Tests + Formatting
# Description
Changes old `from xml` `to xml` data formats. See #7682 for reasoning
behind this change.
Output is now a series of records with `tag`, `attributes` and `content`
fields.
Old:
New:
# User-Facing Changes
New output/input format, better error handling for `from xml` and `to
xml` commands.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
Fixes#8341.
The `CustomValue::to_json()` function is an odd duck; it defaults to
returning `null`, and no `CustomValue` implementations override it to do
anything useful. I forgot to implement `to_json()` for `SQLiteDatabase`,
so `open foo.db | to json` was returning `null`.
To fix this, I've removed `CustomValue::to_json()` and now `to json`
will collect a `CustomValue` into a regular `Value` before doing a JSON
conversion.
Continuation of #8229 and #8326
# Description
The `ShellError` enum at the moment is kind of messy.
Many variants are basic tuple structs where you always have to reference
the implementation with its macro invocation to know which field serves
which purpose.
Furthermore we have both variants that are kind of redundant or either
overly broad to be useful for the user to match on or overly specific
with few uses.
So I set out to start fixing the lacking documentation and naming to
make it feasible to critically review the individual usages and fix
those.
Furthermore we can decide to join or split up variants that don't seem
to be fit for purpose.
# Call to action
**Everyone:** Feel free to add review comments if you spot inconsistent
use of `ShellError` variants.
# User-Facing Changes
(None now, end goal more explicit and consistent error messages)
# Tests + Formatting
(No additional tests needed so far)
# Commits (so far)
- Remove `ShellError::FeatureNotEnabled`
- Name fields on `SE::ExternalNotSupported`
- Name field on `SE::InvalidProbability`
- Name fields on `SE::NushellFailed` variants
- Remove unused `SE::NushellFailedSpannedHelp`
- Name field on `SE::VariableNotFoundAtRuntime`
- Name fields on `SE::EnvVarNotFoundAtRuntime`
- Name fields on `SE::ModuleNotFoundAtRuntime`
- Remove usused `ModuleOrOverlayNotFoundAtRuntime`
- Name fields on `SE::OverlayNotFoundAtRuntime`
- Name field on `SE::NotFound`
Continuation of #8229
# Description
The `ShellError` enum at the moment is kind of messy.
Many variants are basic tuple structs where you always have to reference
the implementation with its macro invocation to know which field serves
which purpose.
Furthermore we have both variants that are kind of redundant or either
overly broad to be useful for the user to match on or overly specific
with few uses.
So I set out to start fixing the lacking documentation and naming to
make it feasible to critically review the individual usages and fix
those.
Furthermore we can decide to join or split up variants that don't seem
to be fit for purpose.
**Everyone:** Feel free to add review comments if you spot inconsistent
use of `ShellError` variants.
- Name fields of `SE::IncorrectValue`
- Merge and name fields on `SE::TypeMismatch`
- Name fields on `SE::UnsupportedOperator`
- Name fields on `AssignmentRequires*` and fix doc
- Name fields on `SE::UnknownOperator`
- Name fields on `SE::MissingParameter`
- Name fields on `SE::DelimiterError`
- Name fields on `SE::IncompatibleParametersSingle`
# User-Facing Changes
(None now, end goal more explicit and consistent error messages)
# Tests + Formatting
(No additional tests needed so far)
# Description
Working on uniformizing the ending messages regarding methods usage()
and extra_usage(). This is related to the issue
https://github.com/nushell/nushell/issues/5066 after discussing it with
@jntrnr
# User-Facing Changes
None.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
# Description
The code to generate the nuon format supports writing range literals,
which obviates the need to expand the range as added in #8047
# User-Facing Changes
`to nuon` will still output ranges as literals
# Tests + Formatting
- Add test for `to nuon` range output
- Add `from nuon` test for range
# Description
Fixes#8002, which expands ranges `1..3` to expand to array-like when
saving and converting to json. Now,
```
> 1..3 | save foo.json
# foo.json
[
1,
2,
3
]
> 1..3 | to json
[
1,
2,
3
]
```
# User-Facing Changes
_(List of all changes that impact the user experience here. This helps
us keep track of breaking changes.)_
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- [X] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [X] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- [X] `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
# Description
Adds a `profile` command that profiles each pipeline element of a block
and can also recursively step into child blocks.
# Limitations
* It is implemented using pipeline metadata which currently get lost in
some circumstances (e.g.,
https://github.com/nushell/nushell/issues/4501). This means that the
profiler will lose data coming from subexpressions. This issue will
hopefully be solved in the future.
* It also does not step into individual loop iteration which I'm not
sure why but maybe that's a good thing.
# User-Facing Changes
Shouldn't change any existing behavior.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
_(Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.)_
I opened this PR to unify the run command method. It's mainly to improve
consistency across the tree.
# User-Facing Changes
None.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
# Description
Lint: `clippy::uninlined_format_args`
More readable in most situations.
(May be slightly confusing for modifier format strings
https://doc.rust-lang.org/std/fmt/index.html#formatting-parameters)
Alternative to #7865
# User-Facing Changes
None intended
# Tests + Formatting
(Ran `cargo +stable clippy --fix --workspace -- -A clippy::all -D
clippy::uninlined_format_args` to achieve this. Depends on Rust `1.67`)
# Description
Fixes#7800 .
`to csv` and `to tsv` no longer:
- accept anything but records and tables as input,
- accept lists that are not tables,
- accept tables and records with values that are not primitives (other
lists, tables and records).
# User-Facing Changes
Using `to csv` and `to tsv` on any of inputs mentioned above will result
in `cant_convert` error.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
This is an attempt to implement a new `Value::LazyRecord` variant for
performance reasons.
`LazyRecord` is like a regular `Record`, but it's possible to access
individual columns without evaluating other columns. I've implemented
`LazyRecord` for the special `$nu` variable; accessing `$nu` is
relatively slow because of all the information in `scope`, and [`$nu`
accounts for about 2/3 of Nu's startup time on
Linux](https://github.com/nushell/nushell/issues/6677#issuecomment-1364618122).
### Benchmarks
I ran some benchmarks on my desktop (Linux, 12900K) and the results are
very pleasing.
Nu's time to start up and run a command (`cargo build --release;
hyperfine 'target/release/nu -c "echo \"Hello, world!\""' --shell=none
--warmup 10`) goes from **8.8ms to 3.2ms, about 2.8x faster**.
Tests are also much faster! Running `cargo nextest` (with our very slow
`proptest` tests disabled) goes from **7.2s to 4.4s (1.6x faster)**,
because most tests involve launching a new instance of Nu.
### Design (updated)
I've added a new `LazyRecord` trait and added a `Value` variant wrapping
those trait objects, much like `CustomValue`. `LazyRecord`
implementations must implement these 2 functions:
```rust
// All column names
fn column_names(&self) -> Vec<&'static str>;
// Get 1 specific column value
fn get_column_value(&self, column: &str) -> Result<Value, ShellError>;
```
### Serializability
`Value` variants must implement `Serializable` and `Deserializable`, which poses some problems because I want to use unserializable things like `EngineState` in `LazyRecord`s. To work around this, I basically lie to the type system:
1. Add `#[typetag::serde(tag = "type")]` to `LazyRecord` to make it serializable
2. Any unserializable fields in `LazyRecord` implementations get marked with `#[serde(skip)]`
3. At the point where a `LazyRecord` normally would get serialized and sent to a plugin, I instead collect it into a regular `Value::Record` (which can be serialized)
# Description
Refactor command: "to url" in: "to url query". Changed usage sentence.
Closes: #7495
# User-Facing Changes
Now we get a query string from a record or table by using command: "to
url query".
```
> help to url query
Convert record or table into query string applying percent-encoding.
Usage:
> to url query
Flags:
-h, --help - Display the help message for this command
Signatures:
<record> | to url query -> <string>
<table> | to url query -> <string>
Examples:
Outputs a query string representing the contents of this record
> { mode:normal userid:31415 } | to url query
Outputs a query string representing the contents of this 1-row table
> [[foo bar]; ["1" "2"]] | to url query
Outputs a query string representing the contents of this record
> {a:"AT&T", b: "AT T"} | to url query
```
# Tests + Formatting
Added this test:
```
Example {
description: "Outputs a query string representing the contents of this record",
example: r#"{a:"AT&T", b: "AT T"} | to url query"#,
result: Some(Value::test_string("a=AT%26T&b=AT+T")),
},
```
to ensure percent-encoding.
# After Submitting
If PR is accepted I'll open another PR on documentation to notify
changes on
[this.](https://github.com/nushell/nushell.github.io/blob/main/book/commands/to_url.md)
# Description
_(Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.)_
I implemented the status bar we talk about yesterday. The idea was
inspired by the progress bar of `wget`.
I decided to go for the second suggestion by `@Reilly`
> 2. add an Option<usize> or whatever to RawStream (and ListStream?) for
situations where you do know the length ahead of time
For now only works with the command `save` but after the approve of this
PR we can see how we can implement it on commands like `cp` and `mv`
When using `fetch` nushell will check if there is any `content-length`
attribute in the request header. If so, then `fetch` will send it
through the new `Option` variable in the `RawStream` to the `save`.
If we know the total size we show the progress bar
but if we don't then we just show the stats like: data already saved,
bytes per second, and time lapse.
Please let me know If I need to make any changes and I will be happy to
do it.
# User-Facing Changes
A new flag (`--progress` `-p`) was added to the `save` command
Examples:
```nu
fetch https://github.com/torvalds/linux/archive/refs/heads/master.zip | save --progress -f main.zip
fetch https://releases.ubuntu.com/22.04.1/ubuntu-22.04.1-desktop-amd64.iso | save --progress -f main.zip
open main.zip --raw | save --progress main.copy
```
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
-
I am getting some errors and its weird because the errors are showing up
in files i haven't touch. Is this normal?
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
Co-authored-by: Reilly Wood <reilly.wood@icloud.com>
# Description
Refactored the quadratic complexity on `uniq` to use a HashMap, as key I
converted the Value to string.
I tried to use the HashableValue, but it looks it is not very developed
yet and it was getting more complex and difficult.
This improves performance on large data sets.
Fixes https://github.com/nushell/nushell/issues/7477
# Tests + Formatting
```
> let data = fetch "https://home.treasury.gov/system/files/276/yield-curve-rates-1990-2021.csv"
> $data | uniq
```
it keeps original attribute order in Records:
```
> [ {b:2, a:1} {a:1, b:2} ] | uniq
╭───┬───┬───╮
│ # │ b │ a │
├───┼───┼───┤
│ 0 │ 2 │ 1 │
╰───┴───┴───╯
```
# Description
The message reads "You must use one of the following subcommands. Using
this command as-is will only produce this help message." and is added to
commands like `into`, `bytes`, `str`, etc.
# User-Facing Changes
See above.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
# Description
Certain commands that operate on tables also work on bare records, but
their type sig didn't reflect that. This corrects this.
I did not fix certain commands which, I feel, currently give unintended
behaviour when given plain records. These are `sort-by` and `uniq-by`.
Also corrected the wording of some stuff in headers.rs, and removed a
wrong comment in insert.rs.
# User-Facing Changes
See above.
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
# Description
* I was dismayed to discover recently that UnsupportedInput and
TypeMismatch are used *extremely* inconsistently across the codebase.
UnsupportedInput is sometimes used for input type-checks (as per the
name!!), but *also* used for argument type-checks. TypeMismatch is also
used for both.
I thus devised the following standard: input type-checking *only* uses
UnsupportedInput, and argument type-checking *only* uses TypeMismatch.
Moreover, to differentiate them, UnsupportedInput now has *two* error
arrows (spans), one pointing at the command and the other at the input
origin, while TypeMismatch only has the one (because the command should
always be nearby)
* In order to apply that standard, a very large number of
UnsupportedInput uses were changed so that the input's span could be
retrieved and delivered to it.
* Additionally, I noticed many places where **errors are not propagated
correctly**: there are lots of `match` sites which take a Value::Error,
then throw it away and replace it with a new Value::Error with
less/misleading information (such as reporting the error as an
"incorrect type"). I believe that the earliest errors are the most
important, and should always be propagated where possible.
* Also, to standardise one broad subset of UnsupportedInput error
messages, who all used slightly different wordings of "expected
`<type>`, got `<type>`", I created OnlySupportsThisInputType as a
variant of it.
* Finally, a bunch of error sites that had "repeated spans" - i.e. where
an error expected two spans, but `call.head` was given for both - were
fixed to use different spans.
# Example
BEFORE
```
〉20b | str starts-with 'a'
Error: nu:🐚:unsupported_input (link)
× Unsupported input
╭─[entry #31:1:1]
1 │ 20b | str starts-with 'a'
· ┬
· ╰── Input's type is filesize. This command only works with strings.
╰────
〉'a' | math cos
Error: nu:🐚:unsupported_input (link)
× Unsupported input
╭─[entry #33:1:1]
1 │ 'a' | math cos
· ─┬─
· ╰── Only numerical values are supported, input type: String
╰────
〉0x[12] | encode utf8
Error: nu:🐚:unsupported_input (link)
× Unsupported input
╭─[entry #38:1:1]
1 │ 0x[12] | encode utf8
· ───┬──
· ╰── non-string input
╰────
```
AFTER
```
〉20b | str starts-with 'a'
Error: nu:🐚:pipeline_mismatch (link)
× Pipeline mismatch.
╭─[entry #1:1:1]
1 │ 20b | str starts-with 'a'
· ┬ ───────┬───────
· │ ╰── only string input data is supported
· ╰── input type: filesize
╰────
〉'a' | math cos
Error: nu:🐚:pipeline_mismatch (link)
× Pipeline mismatch.
╭─[entry #2:1:1]
1 │ 'a' | math cos
· ─┬─ ────┬───
· │ ╰── only numeric input data is supported
· ╰── input type: string
╰────
〉0x[12] | encode utf8
Error: nu:🐚:pipeline_mismatch (link)
× Pipeline mismatch.
╭─[entry #3:1:1]
1 │ 0x[12] | encode utf8
· ───┬── ───┬──
· │ ╰── only string input data is supported
· ╰── input type: binary
╰────
```
# User-Facing Changes
Various error messages suddenly make more sense (i.e. have two arrows
instead of one).
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
This PR changes `to text` so that when given a `ListStream`, it streams
the incoming values instead of collecting them all first.
The easiest way to observe/verify this PR is to convert a list to a very
slow `ListStream` with `each`:
```bash
ls | get name | each {|n| sleep 1sec; $n} | to text
```
The `to text` output will appear 1 item at a time.
`proptest` caught a failing test condition for `&&` as a literal string. It requires a quotation to be parsed correctly by current `from nuon`
https://github.com/nushell/nushell/actions/runs/3753242377/jobs/6376308675
The change in the parser that now returns an error was introduced by https://github.com/nushell/nushell/pull/7241
This in theory doesn't have to be an error (it is a diagnostic for nushell code) but it is probably better safe than sorry to require quotation here.
- Add a test for `&&` in `to nuon` from proptest fail
- Fix `to nuon` generating invalid `&&` literal
- Add a test for `,` in `to nuon`/`from nuon` cycle
- Bonus: should already be properly quoted
# Description
Just some minor updates to xml deps
# User-Facing Changes
_(List of all changes that impact the user experience here. This helps
us keep track of breaking changes.)_
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
chrono version update
# Description
upgrade chrono to 0.4.23
# Major Changes
If you're considering making any major change to nushell, before
starting work on it, seek feedback from regular contributors and get
approval for the idea from the core team either on
[Discord](https://discordapp.com/invite/NtAbbGn) or [GitHub
issue](https://github.com/nushell/nushell/issues/new/choose).
Making sure we're all on board with the change saves everybody's time.
Thanks!
# Tests + Formatting
Make sure you've done the following, if applicable:
- Add tests that cover your changes (either in the command examples, the
crate/tests folder, or in the /tests folder)
- Try to think about corner cases and various ways how your changes
could break. Cover those in the tests
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace --features=extra -- -D warnings -D
clippy::unwrap_used -A clippy::needless_collect` to check that you're
using the standard code style
- `cargo test --workspace --features=extra` to check that all tests pass
# After Submitting
* Help us keep the docs up to date: If your PR affects the user
experience of Nushell (adding/removing a command, changing an
input/output type, etc.), make sure the changes are reflected in the
documentation (https://github.com/nushell/nushell.github.io) after the
PR is merged.
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
* add signature information when help on one command
* tell user that one command support operated on cell paths
Also, make type output to be more friendly, like `record<>` should just be `record`
And the same to `table<>`, which should be `table`
* simplify code
* don't show signatures for parser keyword
* update comment
* output arg syntax shape as type, so it's the same as describe command
* fix string when no positional args
* update signature body
* update
* add help signature test
* fix arg output format for composed data type like list or record
* fix clippy
* add comment
* Add failing test that list of ints and floats is List<Number>
* Start defining subtype relation
* Make it possible to declare input and output types for commands
- Enforce them in tests
* Declare input and output types of commands
* Add formatted signatures to `help commands` table
* Revert SyntaxShape::Table -> Type::Table change
* Revert unnecessary derive(Hash) on SyntaxShape
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Remove unnecessary `#[allow]` annots
Reduce the number of lint exceptions that are not necessary with the
current state of the code (or more recent toolchain)
* Remove dead code from `FileStructure` in nu-command
* Replace `allow(unused)` with relevant feature switch
* Deal with `needless_collect` with annotations
* Change hack for needless_collect in `from json`
This change obviates the need for `allow(needless_collect)`
Removes a pessimistic allocation for empty strings, but increases
allocation size to `Value`
Probably not really worth it.
* Revert "Deal with `needless_collect` with annotations"
This reverts commit 05aca98445.
The previous state seems to better from a performance perspective as a
`Vec<String>` is lighter weight than `Vec<Value>`
* Add search terms for uppercase
* Add search terms
* Add search terms
* Change to parse
* Add search terms for from
* Add search terms for to
* Remove duplicate function
* Remove duplication of search terms
* Remove search term
* remove unnecessary FlatShape
* add proptest
* remove files that belonged in another PR
* more tests, more chars
* add exception for parser error unrelated ot PR
* Test commands for proper names and search terms
Assert that the `Command.name()` is equal to `Signature.name`
Check that search terms are not just substrings of the command name as
they would not help finding the command.
* Clean up search terms
Remove redundant terms that just replicate the command name.
Try to eliminate substring between search terms, clean up where
necessary.
