# Description
This PR just tweaks the `parse` command's usage and examples to make it
clearer what's going on "under the hood".
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This improves the error when the determined output of a custom command
doesn't match the specified output type by adding the actual determined
output type.
# User-Facing Changes
Previous: `command doesn't output {0}`
New: `expected {0}, but command outputs {1}`
# Tests + Formatting
Passing.
# After Submitting
- [ ] release notes? (minor change, but helpful)
# Description
Just a quick one, but `List(Any)` has to come before `Table`, because
`List(Any)` is a valid match for `Table`, so it will choose `Table`
output even if the input was actually `List(Any)`. I ended up removing
`Table` because it's just not needed at all anyway.
Though, I'm not really totally sure this is correct - I think the parser
should probably actually just have some idea of what the more specific
type is, and choose the most specific type match, rather than just doing
it in order. I guess this will result in the output just always being
`List(Any)` for now. Still better than a bad typecheck error
# User-Facing Changes
Fixes the following contrived example:
```nushell
def foo []: nothing -> list<int> {
seq 1 10 | # list<int>
each { |n| $n * 20 } | # this causes the type to become list<any>
take until { |x| $x < 10 } } # table is first, so now this is type table
# ...but table is not compatible with list<int>
}
```
# After Submitting
- [ ] make typechecker type choice more robust
- [ ] release notes
Somehow this logic was missed on my end. ( I mean I was not even
thinking about it in original patch 😄 )
Please recheck
Added a regression test too.
close#13336
cc: @fdncred
# Description
Follow up fix to #13332, so that changes to config when running under IR
actually happen as well. Since I merged them around the same time, I
forgot about this.
# Description
Allows `Stack` to have a modified local `Config`, which is updated
immediately when `$env.config` is assigned to. This means that even
within a script, commands that come after `$env.config` changes will
always see those changes in `Stack::get_config()`.
Also fixed a lot of cases where `engine_state.get_config()` was used
even when `Stack` was available.
Closes#13324.
# User-Facing Changes
- Config changes apply immediately after the assignment is executed,
rather than whenever config is read by a command that needs it.
- Potentially slower performance when executing a lot of lines that
change `$env.config` one after another. Recommended to get `$env.config`
into a `mut` variable first and do modifications, then assign it back.
- Much faster performance when executing a script that made
modifications to `$env.config`, as the changes are only parsed once.
# Tests + Formatting
All passing.
# After Submitting
- [ ] release notes
# Description
Add a few more options to `view ir` for finding blocks, which I found
myself wanting while trying to trace through the generated code.
If we end up adding support for plugins to call commands that are in
scope by name, this will also make it possible for
`nu_plugin_explore_ir` to just step through IR automatically (by passing
the block/decl ids) without exposing too many internals. With that I
could potentially add keys that allow you to step in to closures or
decls with the press of a button, just by calling `view ir --json`
appropriately.
# User-Facing Changes
- `view ir` can now take names of custom commands that are in scope.
- integer arguments are treated as block IDs, which sometimes show up in
IR (closure, block, row condition literals).
- `--decl-id` provided to treat the argument as a decl ID instead, which
is also sometimes necessary to access something that isn't in scope.
# Description
Fix `view ir` to use `Signature::build()` rather than `new()`, which is
required for `--help` to work. Also add `Category::Debug`, as that's
most appropriate.
# Description
`Signature::get_positional()` was returning an owned `PositionalArg`,
which contains a bunch of strings. `ClosureEval` uses this in
`try_add_arg`, making all of that unnecessary cloning a little bit hot.
# User-Facing Changes
Slightly better performance
# Description
Just more efficient allocation during `Stack::gather_captures()` so that
we don't have to grow the `Vec` needlessly.
# User-Facing Changes
Slightly better performance.
# Description
Was having an issue compiling main after the IR pr. Talked to devyn and
he led me to change a couple things real quick and we're compiling once
again.
# Description
This is another easy performance lift that just changes `env_vars` and
`env_hidden` on `Stack` to use `Arc`. I noticed that these were being
cloned on essentially every closure invocation during captures
gathering, so we're paying the cost for all of that even when we don't
change anything. On top of that, for `env_vars`, there's actually an
entirely fresh `HashMap` created for each child scope, so it's highly
unlikely that we'll modify the parent ones.
Uses `Arc::make_mut` instead to take care of things when we need to
mutate something, and most of the time nothing has to be cloned at all.
# Benchmarks
The benefits are greater the more calls there are to env-cloning
functions like `captures_to_stack()`. Calling custom commands in a loop
is basically best case for a performance improvement. Plain `each` with
a literal block isn't so badly affected because the stack is set up
once.
## random_bytes.nu
```nushell
use std bench
do {
const SCRIPT = ../nu_scripts/benchmarks/random-bytes.nu
let before_change = bench { nu $SCRIPT }
let after_change = bench { target/release/nu $SCRIPT }
{
before: ($before_change | reject times),
after: ($after_change | reject times)
}
}
```
```
╭────────┬──────────────────────────────╮
│ │ ╭──────┬───────────────────╮ │
│ before │ │ mean │ 603ms 759µs 727ns │ │
│ │ │ min │ 593ms 298µs 167ns │ │
│ │ │ max │ 648ms 612µs 291ns │ │
│ │ │ std │ 9ms 335µs 251ns │ │
│ │ ╰──────┴───────────────────╯ │
│ │ ╭──────┬───────────────────╮ │
│ after │ │ mean │ 518ms 400µs 557ns │ │
│ │ │ min │ 507ms 762µs 583ns │ │
│ │ │ max │ 566ms 695µs 166ns │ │
│ │ │ std │ 9ms 554µs 767ns │ │
│ │ ╰──────┴───────────────────╯ │
╰────────┴──────────────────────────────╯
```
## gradient_benchmark_no_check.nu
```nushell
use std bench
do {
const SCRIPT = ../nu_scripts/benchmarks/gradient_benchmark_no_check.nu
let before_change = bench { nu $SCRIPT }
let after_change = bench { target/release/nu $SCRIPT }
{
before: ($before_change | reject times),
after: ($after_change | reject times)
}
}
```
```
╭────────┬──────────────────────────────╮
│ │ ╭──────┬───────────────────╮ │
│ before │ │ mean │ 146ms 543µs 380ns │ │
│ │ │ min │ 142ms 416µs 166ns │ │
│ │ │ max │ 189ms 595µs │ │
│ │ │ std │ 7ms 140µs 342ns │ │
│ │ ╰──────┴───────────────────╯ │
│ │ ╭──────┬───────────────────╮ │
│ after │ │ mean │ 134ms 211µs 678ns │ │
│ │ │ min │ 132ms 433µs 125ns │ │
│ │ │ max │ 135ms 722µs 583ns │ │
│ │ │ std │ 793µs 134ns │ │
│ │ ╰──────┴───────────────────╯ │
╰────────┴──────────────────────────────╯
```
# User-Facing Changes
Better performance, particularly for custom commands, especially if
there are a lot of environment variables. Nothing else.
# Tests + Formatting
All passing.
# Description
This PR adds an internal representation language to Nushell, offering an
alternative evaluator based on simple instructions, stream-containing
registers, and indexed control flow. The number of registers required is
determined statically at compile-time, and the fixed size required is
allocated upon entering the block.
Each instruction is associated with a span, which makes going backwards
from IR instructions to source code very easy.
Motivations for IR:
1. **Performance.** By simplifying the evaluation path and making it
more cache-friendly and branch predictor-friendly, code that does a lot
of computation in Nushell itself can be sped up a decent bit. Because
the IR is fairly easy to reason about, we can also implement
optimization passes in the future to eliminate and simplify code.
2. **Correctness.** The instructions mostly have very simple and
easily-specified behavior, so hopefully engine changes are a little bit
easier to reason about, and they can be specified in a more formal way
at some point. I have made an effort to document each of the
instructions in the docs for the enum itself in a reasonably specific
way. Some of the errors that would have happened during evaluation
before are now moved to the compilation step instead, because they don't
make sense to check during evaluation.
3. **As an intermediate target.** This is a good step for us to bring
the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at
some point, as code generated from new AST can be directly compared to
code generated from old AST. If the IR code is functionally equivalent,
it will behave the exact same way.
4. **Debugging.** With a little bit more work, we can probably give
control over advancing the virtual machine that `IrBlock`s run on to
some sort of external driver, making things like breakpoints and single
stepping possible. Tools like `view ir` and [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than
before to see what exactly is going on with your Nushell code.
The goal is to eventually replace the AST evaluator entirely, once we're
sure it's working just as well. You can help dogfood this by running
Nushell with `$env.NU_USE_IR` set to some value. The environment
variable is checked when Nushell starts, so config runs with IR, or it
can also be set on a line at the REPL to change it dynamically. It is
also checked when running `do` in case within a script you want to just
run a specific piece of code with or without IR.
# Example
```nushell
view ir { |data|
mut sum = 0
for n in $data {
$sum += $n
}
$sum
}
```
```gas
# 3 registers, 19 instructions, 0 bytes of data
0: load-literal %0, int(0)
1: store-variable var 904, %0 # let
2: drain %0
3: drop %0
4: load-variable %1, var 903
5: iterate %0, %1, end 15 # for, label(1), from(14:)
6: store-variable var 905, %0
7: load-variable %0, var 904
8: load-variable %2, var 905
9: binary-op %0, Math(Plus), %2
10: span %0
11: store-variable var 904, %0
12: load-literal %0, nothing
13: drain %0
14: jump 5
15: drop %0 # label(0), from(5:)
16: drain %0
17: load-variable %0, var 904
18: return %0
```
# Benchmarks
All benchmarks run on a base model Mac Mini M1.
## Iterative Fibonacci sequence
This is about as best case as possible, making use of the much faster
control flow. Most code will not experience a speed improvement nearly
this large.
```nushell
def fib [n: int] {
mut a = 0
mut b = 1
for _ in 2..=$n {
let c = $a + $b
$a = $b
$b = $c
}
$b
}
use std bench
bench { 0..50 | each { |n| fib $n } }
```
IR disabled:
```
╭───────┬─────────────────╮
│ mean │ 1ms 924µs 665ns │
│ min │ 1ms 700µs 83ns │
│ max │ 3ms 450µs 125ns │
│ std │ 395µs 759ns │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```
IR enabled:
```
╭───────┬─────────────────╮
│ mean │ 452µs 820ns │
│ min │ 427µs 417ns │
│ max │ 540µs 167ns │
│ std │ 17µs 158ns │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```
##
[gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu)
IR disabled:
```
╭───┬──────────────────╮
│ 0 │ 27ms 929µs 958ns │
│ 1 │ 21ms 153µs 459ns │
│ 2 │ 18ms 639µs 666ns │
│ 3 │ 19ms 554µs 583ns │
│ 4 │ 13ms 383µs 375ns │
│ 5 │ 11ms 328µs 208ns │
│ 6 │ 5ms 659µs 542ns │
╰───┴──────────────────╯
```
IR enabled:
```
╭───┬──────────────────╮
│ 0 │ 22ms 662µs │
│ 1 │ 17ms 221µs 792ns │
│ 2 │ 14ms 786µs 708ns │
│ 3 │ 13ms 876µs 834ns │
│ 4 │ 13ms 52µs 875ns │
│ 5 │ 11ms 269µs 666ns │
│ 6 │ 6ms 942µs 500ns │
╰───┴──────────────────╯
```
##
[random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu)
I got pretty random results out of this benchmark so I decided not to
include it. Not clear why.
# User-Facing Changes
- IR compilation errors may appear even if the user isn't evaluating
with IR.
- IR evaluation can be enabled by setting the `NU_USE_IR` environment
variable to any value.
- New command `view ir` pretty-prints the IR for a block, and `view ir
--json` can be piped into an external tool like [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir).
# Tests + Formatting
All tests are passing with `NU_USE_IR=1`, and I've added some more eval
tests to compare the results for some very core operations. I will
probably want to add some more so we don't have to always check
`NU_USE_IR=1 toolkit test --workspace` on a regular basis.
# After Submitting
- [ ] release notes
- [ ] further documentation of instructions?
- [ ] post-release: publish `nu_plugin_explore_ir`
# Description
Fixes the lexer to recognize `out>|`, `err>|`, `out+err>|`, etc.
Previously only the short-style forms were recognized, which was
inconsistent with normal file redirections.
I also integrated it all more into the normal lex path by checking `|`
in a special way, which should be more performant and consistent, and
cleans up the code a bunch.
Closes#13331.
# User-Facing Changes
- Adds `out>|` (error), `err>|`, `out+err>|`, `err+out>|` as recognized
forms of the pipe redirection.
# Tests + Formatting
All passing. Added tests for the new forms.
# After Submitting
- [ ] release notes
# Description
Makes `polars unpivot` use the same arguments as `polars pivot` and
makes it consistent with the polars' rust api. Additionally, support for
the polar's streaming engine has been exposed on eager dataframes.
Previously, it would only work with lazy dataframes.
# User-Facing Changes
* `polars unpivot` argument `--columns`|`-c` has been renamed to
`--index`|`-i`
* `polars unpivot` argument `--values`|`-v` has been renamed to
`--on`|`-o`
* `polars unpivot` short argument for `--streamable` is now `-t` to make
it consistent with `polars pivot`. It was made `-t` for `polars pivot`
because `-s` is short for `--short`
GOOD CATCH.............................................................
SORRY
I've added a test to catch regression just in case.
close#13319
cc: @fdncred
# Description
From the feedbacks from @amtoine , it's good to make nushell shows error
for `o>|` syntax.
# User-Facing Changes
## Before
```nushell
'foo' o>| print 07/09/2024 06:44:23 AM
Error: nu::parser::parse_mismatch
× Parse mismatch during operation.
╭─[entry #6:1:9]
1 │ 'foo' o>| print
· ┬
· ╰── expected redirection target
```
## After
```nushell
'foo' o>| print 07/09/2024 06:47:26 AM
Error: nu::parser::parse_mismatch
× Parse mismatch during operation.
╭─[entry #1:1:7]
1 │ 'foo' o>| print
· ─┬─
· ╰── expected `|`. Redirection stdout to pipe is the same as piping directly.
╰────
```
# Tests + Formatting
Added one test
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
This PR introduces a new `Signals` struct to replace our adhoc passing
around of `ctrlc: Option<Arc<AtomicBool>>`. Doing so has a few benefits:
- We can better enforce when/where resetting or triggering an interrupt
is allowed.
- Consolidates `nu_utils::ctrl_c::was_pressed` and other ad-hoc
re-implementations into a single place: `Signals::check`.
- This allows us to add other types of signals later if we want. E.g.,
exiting or suspension.
- Similarly, we can more easily change the underlying implementation if
we need to in the future.
- Places that used to have a `ctrlc` of `None` now use
`Signals::empty()`, so we can double check these usages for correctness
in the future.
cc: @zhiburt
This is an internal refactoring for `explore`.
Previously, views inside `explore` were created with default/incorrect
configuration and then the correct configuration was passed to them
using a function called `setup()`. I believe this was because
configuration was dynamic and could change while `explore` was running.
After https://github.com/nushell/nushell/pull/10259, configuration can
no longer be changed on the fly. So we can clean this up by removing
`setup()` and passing configuration to views when they are created.
# Description
Fix `into bits` to have consistent behavior when passed a byte stream.
# User-Facing Changes
Previously, it was returning a binary on stream, even though its
input/output types don't describe this possibility. We don't need this
since we have `into binary` anyway.
# Tests + Formatting
Tests added
# Description
fixed#12699
When bare dates or naive times are specified in toml files, `from toml`
returns invalid dates or times.
This PR fixes the problem to correctly handle toml datetime.
The current version command returns the default datetime
(`chrono::DateTime::default()`) if the datetime parse fails. However, I
felt that this behavior was a bit unfriendly, so I changed it to return
`Value::string`.
# User-Facing Changes
The command returns a date with default time and timezone if a bare date
is specified.
```
~/Development/nushell> "dob = 2023-05-27" | from toml
╭─────┬────────────╮
│ dob │ a year ago │
╰─────┴────────────╯
~/Development/nushell> "dob = 2023-05-27" | from toml |
Sat, 27 May 2023 00:00:00 +0000 (a year ago)
~/Development/nushell>
```
If a bare time is given, a time string is returned.
```
~/Development/nushell> "tm = 11:00:00" | from toml
╭────┬──────────╮
│ tm │ 11:00:00 │
╰────┴──────────╯
~/Development/nushell> "tm = 11:00:00" | from toml | get tm
11:00:00
~/Development/nushell>
```
# Tests + Formatting
When I ran tests, `commands::touch::change_file_mtime_to_reference`
failed with the following error.
The error also occurs in the master branch, so it's probably unrelated
to these changes.
(maybe a problem with my dev environment)
```
$ ~/Development/nushell> toolkit check pr
~~~~~~~~
test usage_start_uppercase ... ok
test format_conversions::yaml::convert_dict_to_yaml_with_integer_floats_key ... ok
test format_conversions::yaml::convert_dict_to_yaml_with_boolean_key ... ok
test format_conversions::yaml::table_to_yaml_text_and_from_yaml_text_back_into_table ... ok
test quickcheck_parse ... ok
test format_conversions::yaml::convert_dict_to_yaml_with_integer_key ... ok
failures:
---- commands::touch::change_file_mtime_to_reference stdout ----
=== stderr
thread 'commands::touch::change_file_mtime_to_reference' panicked at crates/nu-command/tests/commands/touch.rs:298:9:
assertion `left == right` failed
left: SystemTime { tv_sec: 1720344745, tv_nsec: 862392750 }
right: SystemTime { tv_sec: 1720344745, tv_nsec: 887670417 }
failures:
commands::touch::change_file_mtime_to_reference
test result: FAILED. 1542 passed; 1 failed; 32 ignored; 0 measured; 0 filtered out; finished in 12.04s
error: test failed, to rerun pass `-p nu-command --test main`
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
- ⚫ `toolkit test stdlib`
~/Development/nushell> toolkit test stdlib
Compiling nu v0.95.1 (/Users/hiroki/Development/nushell)
Compiling nu-cmd-lang v0.95.1 (/Users/hiroki/Development/nushell/crates/nu-cmd-lang)
Finished dev [unoptimized + debuginfo] target(s) in 6.64s
Running `target/debug/nu --no-config-file -c '
use crates/nu-std/testing.nu
testing run-tests --path crates/nu-std
'`
2024-07-07T19:00:20.423|INF|Running from_jsonl_invalid_object in module test_formats
2024-07-07T19:00:20.436|INF|Running env_log-prefix in module test_logger_env
~~~~~~~~~~~
2024-07-07T19:00:22.196|INF|Running debug_short in module test_basic_commands
~/Development/nushell>
```
# After Submitting
nothing
# Description
Refactors `help operators` so that its output is always up to date with
the parser.
# User-Facing Changes
- The order of output rows for `help operators` was changed.
- `not` is now listed as a boolean operator instead of a comparison
operator.
- Edited some of the descriptions for the operators.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
My last PR (https://github.com/nushell/nushell/pull/13242) made it so
that the last branch in the variable completer doesn't sort suggestions.
Sorry about that. This should fix it.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Variables will now be sorted properly.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Added one test case to verify this won't happen again.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Bug fix: `PipelineData::check_external_failed()` was not preserving the
original `type_` and `known_size` attributes of the stream passed in for
streams that come from children, so `external-command | into binary` did
not work properly and always ended up still being unknown type.
# User-Facing Changes
The following test case now works as expected:
```nushell
> head -c 2 /dev/urandom | into binary
# Expected: pretty hex dump of binary
# Previous behavior: just raw binary in the terminal
```
# Tests + Formatting
Added a test to cover this to `into binary`
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
Related to #13298
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Exotic types like `Duration` and `Filesize` return a float on division
by the same type, i.e., the unit is gone since division results in a
scalar. When using the modulo operator, the output type has the same
unit.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Division results in a float like the following:
```sh
~/Public/nushell> 512sec / 3sec
170.66666666666666
```
Modulo results in an output with the same unit:
```sh
~/Public/nushell> 512sec mod 3sec
2sec
```
Type checking isn't confused with output types:
```sh
~/Public/nushell> (512sec mod 3sec) / 0.5sec
4
```
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Existing tests are passing.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This PR fixes an issue with `explore` where you can "drill down" into
the same value forever. For example:
1. Run `ls | explore`
2. Press Enter to enter cursor mode
3. Press Enter again to open the selected string in a new layer
4. Press Enter again to open that string in a new layer
5. Press Enter again to open that string in a new layer
6. Repeat and eventually you have a bajillion layers open with the same
string
IMO it only makes sense to "drill down" into lists and records.
In a separate commit I also did a little refactoring, cleaning up naming
and comments.
# Description
Fixes: #13189
The issue is caused `error make` returns a `Value::Errror`, and when
nushell pass it to `table -e` in `std help`, it directly stop and render
the error message.
To solve it, I think it's safe to make these examples return None
directly, it doesn't change the reult of `help error make`.
# User-Facing Changes
## Before
```nushell
~> help "error make"
Error: nu:🐚:eval_block_with_input
× Eval block failed with pipeline input
╭─[NU_STDLIB_VIRTUAL_DIR/std/help.nu:692:21]
691 │ ] {
692 │ let commands = (scope commands | sort-by name)
· ───────┬──────
· ╰── source value
693 │
╰────
Error: × my custom error message
```
## After
```nushell
Create an error.
Search terms: panic, crash, throw
Category: core
This command:
- does not create a scope.
- is a built-in command.
- is a subcommand.
- is not part of a plugin.
- is not a custom command.
- is not a keyword.
Usage:
> error make {flags} <error_struct>
Flags:
-u, --unspanned - remove the origin label from the error
-h, --help - Display the help message for this command
Signatures:
<nothing> | error make[ <record>] -> <any>
Parameters:
error_struct: <record> The error to create.
Examples:
Create a simple custom error
> error make {msg: "my custom error message"}
Create a more complex custom error
> error make {
msg: "my custom error message"
label: {
text: "my custom label text" # not mandatory unless $.label exists
# optional
span: {
# if $.label.span exists, both start and end must be present
start: 123
end: 456
}
}
help: "A help string, suggesting a fix to the user" # optional
}
Create a custom error for a custom command that shows the span of the argument
> def foo [x] {
error make {
msg: "this is fishy"
label: {
text: "fish right here"
span: (metadata $x).span
}
}
}
```
# Tests + Formatting
Added 1 test
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
you can also mention related issues, PRs or discussions!
-->
This PR should close#13247
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
- The deprecated `itertools::unfold` function is replaced with
`std::iter::from_fn` for the generate command.
- The mutable iterator state is no longer passed as an argument to
`from_fn` but it gets captured with the closure's `move`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
No user facing changes
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests for the generate command are passing locally.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
closes#13298 so that duration mod duration / duration = duration
### Before
```nushell
(92sec mod 1min) / 1sec
Error: nu::parser::unsupported_operation
× division is not supported between float and duration.
╭─[entry #5:1:1]
1 │ (92sec mod 1min) / 1sec
· ────────┬─────── ┬ ──┬─
· │ │ ╰── duration
· │ ╰── doesn't support these values
· ╰── float
╰────
```
### After
```nushell
❯ (92sec mod 1min) / 1sec
32
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes#13280. After apply this patch, we can use non-timezone string +
format option `into datetime` cmd
# User-Facing Changes
AS-IS (before fixing)
```
$ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T'
Error: nu:🐚:cant_convert
× Can't convert to could not parse as datetime using format '%m.%d.%Y %T'.
╭─[entry #1:1:25]
1 │ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T'
· ──────┬──────
· ╰── can't convert input is not enough for unique date and time to could not parse as datetime using format '%m.%d.%Y %T'
╰────
help: you can use `into datetime` without a format string to enable flexible parsing
$ "09.02.2024 11:06:11" | into datetime
Mon, 2 Sep 2024 11:06:11 +0900 (in 2 months)
```
TO-BE(After fixing)
```
$ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T'
Mon, 2 Sep 2024 20:06:11 +0900 (in 2 months)
$ "09.02.2024 11:06:11" | into datetime
Mon, 2 Sep 2024 11:06:11 +0900 (in 2 months)
```
# Tests + Formatting
If there is agreement on the direction, I will add a test.
# After Submitting
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
This reverts commit 0cfd5fbece.
The original PR messed up syntax higlighting of aliases and causes
panics of completion in the presence of alias.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Complete the `--numbered` removal that was started with the deprecation
in #13112.
# User-Facing Changes
Breaking change - Use `| enumerate` in place of `--numbered` as shown in
the help example
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
Searched online doc for `--numbered` to ensure no other usage needed to
be updated.
# Description
This PR just creates a better error message when the `save` command
fails.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR fixes the problem pointed out in
https://github.com/nushell/nushell/issues/13204, where the Fish-like
completions aren't sorted properly (this PR doesn't close that issue
because the author there wants more than just fixed sort order).
The cause is all of the file/directory completions being fetched first
and then sorted all together while being treated as strings. Instead,
this PR sorts completions within each individual directory, avoiding
treating `/` as part of the path.
To do this, I removed the `sort` method from the completer trait (as
well as `get_sort_by`) and made all completers sort within the `fetch`
method itself. A generic `sort_completions` helper has been added to
sort lists of completions, and a more specific `sort_suggestions` helper
has been added to sort `Vec<Suggestion>`s.
As for the actual change that fixes the sort order for file/directory
completions, the `complete_rec` helper now sorts the children of each
directory before visiting their children. The file and directory
completers don't bother sorting at the end (except to move hidden files
down).
To reviewers: don't let the 29 changed files scare you, most of those
are just the test fixtures :)
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This is the current behavior with prefix matching:
And with fuzzy matching:
Notice how `partial/hello.txt` is the last suggestion, even though it
should come before `partial-a`. This is because the ASCII code for `/`
is greater than that of `-`, so `partial-` is put before `partial/`.
This is this PR's behavior with prefix matching (`partial/hello.txt` is
at the start):
And with fuzzy matching:
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- Modified the partial completions test fixture to test whether this PR
even fixed anything
- Modified fixture to test sort order of .nu completions (a previous
version of my changes didn't sort all the completions at the end but
there were no tests catching that)
- Added a test for making sure subcommand completions are sorted by
Levenshtein distance (a previous version of my changes sorted in
alphabetical order but there were no tests catching that)
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
There was a bug where anytime the plugin cache remove was called, the
plugin gc was turned back on. This probably happened when I added the
reference counter logic.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Part of https://github.com/nushell/nushell/issues/12963, step 2.
This PR refactors Call and related argument structures to remove their
dependency on `Expression::span` which will be removed in the future.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Should be none. If you see some error messages that look broken, please
report.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
With #13254, the content-type pipeline metadata field was added. This
pull request allows it to be manipulated with `metadata set`
# User-Facing Changes
* `metadata set` now has a `--content-type` flag
# Description
Provides the ability to use http commands as part of a pipeline.
Additionally, this pull requests extends the pipeline metadata to add a
content_type field. The content_type metadata field allows commands such
as `to json` to set the metadata in the pipeline allowing the http
commands to use it when making requests.
This pull request also introduces the ability to directly stream http
requests from streaming pipelines.
One other small change is that Content-Type will always be set if it is
passed in to the http commands, either indirectly or throw the content
type flag. Previously it was not preserved with requests that were not
of type json or form data.
# User-Facing Changes
* `http post`, `http put`, `http patch`, `http delete` can be used as
part of a pipeline
* `to text`, `to json`, `from json` all set the content_type metadata
field and the http commands will utilize them when making requests.
# Description
I introduced a regression in #13272 that resulted in `detect columns
--guess` to panic whenever it had to handle empty, whitespace-only, or
non-whitespace-only lines that go all the way to the last column (and as
such, cannot be considered to be lines that only have entries for the
first colum). I fix this by detecting these cases and skipping them,
since these are usually decoration lines. An example is the second line
output by `winget list`:
What we don't want to skip, however, is lines that contain no
whitespace, and fit into the detected first column, since these lines
represent cases where data is only available for the first column, and
are not just decoration lines. For example (made up example, there are
no such entries in `winget lits`'s output), in this output we would not
want to skip the `Docker Desktop` line :
```
Name Id Version Available Source
-------------------------------------------------------------------------------------------------------------------------------------
AMD Software ARPMachineX64AMD Catalyst Install Manager 24.4.1
AMD Ryzen Master ARPMachineX64AMD Ryzen Master 2.13.0.2908
Docker Desktop
Mozilla Firefox (x64 en-US) Mozilla.Firefox 127.0.2 winget
```
NOTE: `winget list | detect columns --guess` does not panic, but sadly
still does not work as expected. I believe this is not a nushell issue
anymore, but a `winget` one. When being piped, `winget` seems to add
extra whitespace and random `\r` symbols at the beginning of the text.
This messes with the column detection, of course.
# User-Facing Changes
`detect columns --guess` should not panic when receiving output from
`winget list` at all anymore.
A breaking change is the skipping of decoration lines, especially since
scripts probably were doing something like
`winget list | lines | reject 1 | str join "\n" | detect columns
--guess`. This will now cause them to reject a line with valid data.
# Tests + Formatting
Added tests that exercise these edge cases, as well as a single-column
test to make sure that trivial cases keep working.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
