# Description
Fixes#14401 where expressions passed to `timeit` will execute twice.
This PR removes the expression support for `timeit`, as this behavior is
almost exclusive to `timeit` and can hinder migration to the IR
evaluator in the future. Additionally, `timeit` used to be able to take
a `block` as an argument. Blocks should probably only be allowed for
parser keywords, so this PR changes `timeit` to instead only take
closures as an argument. This also fixes an issue where environment
updates inside the `timeit` block would affect the parent scope and all
commands later in the pipeline.
```nu
> timeit { $env.FOO = 'bar' }; print $env.FOO
bar
```
# User-Facing Changes
`timeit` now only takes a closure as the first argument.
# After Submitting
Update examples in the book/docs if necessary.
- fixes flakey tests from solving #14241
# Description
This is a preliminary fix for the flaky tests and also
shortened the `--max-time` in the tests.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
---------
Signed-off-by: Alex Kattathra Johnson <alex.kattathra.johnson@gmail.com>
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
Fixes#14515
Also tweaks the fix from #11261 _just in case_ someone has a `foo`
executable
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
N/A
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
closes https://github.com/nushell/nushell/issues/14487
This PR tries to allow the `select` to stream better by changing the for
loops that collected the output into a `Vec<Value>` prior to returning
it into a map that returns the data as it is processed.
One curiosity, `select` transforms the input into a `PipelineIterator`.
If I remove this code, it still passes all tests. I'm not sure all this
`PipelineIterator` code is even needed. I left it for someone to tell me
if it's necessary.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR enables some tests that were disabled on macos.
We shall see if the CI passes. (Update: CI has passed.)
# User-Facing Changes
Should be no user-facing changes as only a test-file is modified.
# Tests + Formatting
Test coverage should increase
Co-authored-by: Jasha <jsimpson@hiddenroad.com>
# Description
While reviewing #14388, I think we can make some improvement on parser.
For the following code:
```nushell
let a = 3
a = 10 # should be error
$a = 10 # another error
```
I think they can raise `ParseError`, so nushell doesn't need to move
forward compiling IR block.
# User-Facing Changes
```nushell
let a = 3
a = 10
```
Will raise parse error instead of compile error.
# Tests + Formatting
Added 1 test.
# Description
Adds a new `Filesize` type so that `FromValue` can be used to convert a
`Value::Filesize` to a `Filesize`. Currently, to extract a filesize from
a `Value` using `FromValue`, you have to extract an `i64` which coerces
`Value::Int`, `Value::Duration`, and `Value::Filesize` to an `i64`.
Having a separate type also allows us to enforce checked math to catch
overflows. Similarly, it allows us to specify other trait
implementations like `Display` in a common place.
# User-Facing Changes
Multiplication with filesizes now error on overflow. Should not be a
breaking change for plugins (i.e., serialization) since `Filesize` is
marked with `serde(transparent)`.
# Tests + Formatting
Updated some tests.
# Description
As title, this pr is going to deprecate `--ignore-shell-errors` and
`--ignore-program-errors`.
Because I think these two flags makes `do` command complicate, and it
should be easy to use `-i` instead.
# User-Facing Changes
After the pr, using these two flags will raise deprecated warning.
```nushell
> do --ignore-program-errors { ^pwd }
Error: × Deprecated option
╭─[entry #2:1:1]
1 │ do --ignore-program-errors { ^pwd }
· ─┬
· ╰── `--ignore-program-errors` is deprecated and will be removed in 0.102.0.
╰────
help: Please use the `--ignore-errors(-i)`
/home/windsoilder/projects/nushell
> do --ignore-shell-errors { ^pwd }
Error: × Deprecated option
╭─[entry #3:1:1]
1 │ do --ignore-shell-errors { ^pwd }
· ─┬
· ╰── `--ignore-shell-errors` is deprecated and will be removed in 0.102.0.
╰────
help: Please use the `--ignore-errors(-i)`
/home/windsoilder/projects/nushell
```
# Tests + Formatting
NaN
# Description
The "append" operator currently serves as both the append operator and
the concatenation operator. This dual role creates ambiguity when
operating on nested lists.
```nu
[1 2] ++ 3 # appends a value to a list [1 2 3]
[1 2] ++ [3 4] # concatenates two lists [1 2 3 4]
[[1 2] [3 4]] ++ [5 6]
# does this give [[1 2] [3 4] [5 6]]
# or [[1 2] [3 4] 5 6]
```
Another problem is that `++=` can change the type of a variable:
```nu
mut str = 'hello '
$str ++= ['world']
($str | describe) == list<string>
```
Note that appending is only relevant for lists, but concatenation is
relevant for lists, strings, and binary values. Additionally, appending
can be expressed in terms of concatenation (see example below). So, this
PR changes the `++` operator to only perform concatenation.
# User-Facing Changes
Using the `++` operator with a list and a non-list value will now be a
compile time or runtime error.
```nu
mut list = []
$list ++= 1 # error
```
Instead, concatenate a list with one element:
```nu
$list ++= [1]
```
Or use `append`:
```nu
$list = $list | append 1
```
# After Submitting
Update book and docs.
---------
Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com>
- fixes#14398
I will properly fill out this PR and fix any tests that might break when
I have the time, this was a quick fix.
# Description
This PR makes `from csv` and `from tsv`, with the `--flexible` flag,
stop dropping extra/unexpected columns.
# User-Facing Changes
`$text`'s contents
```csv
value
1,aaa
2,bbb
3
4,ddd
5,eee,extra
```
Old behavior
```nushell
> $text | from csv --flexible --noheaders
╭─#─┬─column0─╮
│ 0 │ value │
│ 1 │ 1 │
│ 2 │ 2 │
│ 3 │ 3 │
│ 4 │ 4 │
│ 5 │ 5 │
╰─#─┴─column0─╯
```
New behavior
```nushell
> $text | from csv --flexible --noheaders
╭─#─┬─column0─┬─column1─┬─column2─╮
│ 0 │ value │ ❎ │ ❎ │
│ 1 │ 1 │ aaa │ ❎ │
│ 2 │ 2 │ bbb │ ❎ │
│ 3 │ 3 │ ❎ │ ❎ │
│ 4 │ 4 │ ddd │ ❎ │
│ 5 │ 5 │ eee │ extra │
╰─#─┴─column0─┴─column1─┴─column2─╯
```
- The first line in a csv (or tsv) document no longer limits the number
of columns
- Missing values in columns are longer automatically filled with `null`
with this change, as a later row can introduce new columns. **BREAKING
CHANGE**
Because missing columns are different from empty columns, operations on
possibly missing columns will have to use optional access syntax e.g.
`get foo` => `get foo?`
# Tests + Formatting
Added examples that run as tests and adjusted existing tests to confirm
the new behavior.
# After Submitting
Update the workaround with fish completer mentioned
[here](https://www.nushell.sh/cookbook/external_completers.html#fish-completer)
This commit upgrades calamine in order to benefit from recent
developments, e.g. ignore annotations in column headers (see
https://github.com/tafia/calamine/pull/467 for reference).
# Description
This PR allows nushell to run powershell scripts easier. You can already
do `powershell -c script.ps1` but this PR takes it a step further by
doing the `powershell -c` part for you. So, if you have script.ps1 you
can execute it by running it in the command position of the repl.
or once it's in json, just consume it with nushell.
# User-Facing Changes
Easier to run powershell scripts. It should work on Windows with
powershell.exe.
# Tests + Formatting
Added 1 test
# After Submitting
---------
Co-authored-by: Wind <WindSoilder@outlook.com>
# Description
This PR updates the uutils/coreutils crates to the latest version. I
hard-coded debug to false, a new uu_mv parameter. It may be interesting
to add that but I just wanted to get all the uu crates on the same
version.
I had to update the tests because --no-clobber works but doesn't say
anything when it's not clobbering and previously we were checking for an
error message.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
Part of https://github.com/nushell/nushell/issues/11549
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR adds a `utouch` command that uses the `touch` command from
https://github.com/uutils/coreutils. Eventually, `utouch` may be able to
replace `touch`.
The conflicts in Cargo.lock and Cargo.toml are because I'm using the
uutils/coreutils main rather than the latest release, since the changes
that expose `uu_touch`'s internal functionality aren't available in the
latest release.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Users will have access to a new `utouch` command with the following
flags:
todo
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# User-Facing Changes
The parser now errors on more invalid command signatures:
```nushell
# expected parameter or flag
def foo [ bar: int: ] {}
# expected type
def foo [ bar: = ] {}
def foo [ bar: ] {}
# expected default value
def foo [ bar = ] {}
```
# Description - fixes#14174
This PR addresses a bug in the `seq char` command where the command's
behavior did not align with its help description, which stated that it
prints a sequence of ASCII characters. The initial implementation only
allowed alphabetic characters, leading to user confusion when
non-alphabetic characters (e.g., digits, punctuation) were rejected or
when unexpected behavior occurred for certain input ranges.
### Changes Made:
- **Updated the input validation**: Modified the `is_single_character`
function to accept any ASCII character instead of restricting to
alphabetic characters.
- **Enhanced error messages**: Clarified error messages to specify that
any single ASCII character is acceptable.
- **Expanded functionality**: Ensured that the command can now generate
sequences that include non-alphabetic ASCII characters.
- **Updated tests**: Added tests to cover new use cases involving
non-alphabetic characters and improved validation.
### Examples After Fix:
- `seq char '0' '9'` now outputs `['0', '1', '2', '3', '4', '5', '6',
'7', '8', '9']`
- `seq char ' ' '/'` outputs a list of characters from space to `/`
- `seq char 'A' 'z'` correctly includes alphabetic and non-alphabetic
characters between `A` and `z`
# User-Facing Changes
- Users can now input any single ASCII character for the `start` and
`end` parameters of `seq char`.
- The output will accurately include all characters within the specified
ASCII range, including digits and punctuation.
# Tests + Formatting
- Added new tests to ensure the `seq char` command supports sequences
including non-alphabetic ASCII characters.
Fixes#14265
# User-Facing Changes
`ls` without a path argument now errors when the current working
directory is unreadable due to missing permissions:
```diff
mkdir foo
chmod 100 foo
cd foo
ls | to nuon
-[]
+Error: × Permission denied
```
Fixes#13267
As we can see from the bisect done in the comments.
Bisected to https://github.com/nushell/nushell/pull/12625 /
460a1c8f87
We can see that this update brought the use of `read_dir` and for it, it
is mentioned in the [rust
docs](https://doc.rust-lang.org/std/fs/fn.read_dir.html#platform-specific-behavior)
that it does **not** provide any specific order of files.
As was the advice there, I went and applied a manual `sort` to the
entries and tested it manually on my local machine.
If required I could probably try and add tests for the order
consistency, would need some time to find my way around them, so I'm
sending the PR first.
@sholderbach suggested that we need to have a test for a function can't
use mutable variable.
https://github.com/nushell/nushell/pull/14311#issuecomment-2470035194
So this pr is going to add a case for it.
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
Fixes: #14110Fixes: #14087
I think it's ok to not generating instruction to `def` and `export def`
call. Because they just return `PipelineData::Empty` without doing
anything.
If nushell generates instructions for `def` and `export def`, nushell
will try to capture variables for these block. It's not the time to do
this.
# User-Facing Changes
```
nu -c "
def bar [] {
let x = 1
($x | foo)
}
def foo [] {
foo
}
"
```
Will no longer raise error.
# Tests + Formatting
Added 4 tests
Addresses the following points from #14162
> - There is no built-in counterpart to url build-query for splitting a
query string
There is `from url`, which, due to naming, is a little hard to discover
and suffers from the following point
> - url parse can create records with duplicate keys
> - url parse's params should either:
> - ~group the same keys into a list.~
> - instead of a record, be a key-value table. (table<key: string,
value: string>)
# Description
## `url split-query`
Counterpart to `url build-query`, splits a url encoded query string to
key value pairs, represented as `table<key: string, value: string>`
```
> "a=one&a=two&b=three" | url split-query
╭───┬─────┬───────╮
│ # │ key │ value │
├───┼─────┼───────┤
│ 0 │ a │ one │
│ 1 │ a │ two │
│ 2 │ b │ three │
╰───┴─────┴───────╯
```
## `url parse`
The output's `param` field is now a table as well, mirroring the new
`url split-query`
```
> 'http://localhost?a=one&a=two&b=three' | url parse
╭──────────┬─────────────────────╮
│ scheme │ http │
│ username │ │
│ password │ │
│ host │ localhost │
│ port │ │
│ path │ / │
│ query │ a=one&a=two&b=three │
│ fragment │ │
│ │ ╭───┬─────┬───────╮ │
│ params │ │ # │ key │ value │ │
│ │ ├───┼─────┼───────┤ │
│ │ │ 0 │ a │ one │ │
│ │ │ 1 │ a │ two │ │
│ │ │ 2 │ b │ three │ │
│ │ ╰───┴─────┴───────╯ │
╰──────────┴─────────────────────╯
```
# User-Facing Changes
- `url parse`'s output has the mentioned change, which is backwards
incompatible.
# Description
Fixes#14151 where `to text` treats list streams and lists values
differently.
# User-Facing Changes
New line is always added after items in a list or record except for the
last item if the `--no-newline` flag is provided.
# Description
Fixes: https://github.com/nushell/nushell/issues/13425
It's just a follow up to #13958.
User input can be a directory, in this case, we need to use the return
value of `find_in_dirs_env` carefully, so in case, I renamed
maybe_file_path to maybe_file_path_or_dir to emphasize it.
# User-Facing Changes
`$env.FILE_PWD` and `$env.CURRENT_FILE` will be more reliable to use.
# Tests + Formatting
Added 2 tests
# Description
Dividing two ints can currently return either an int or a float. Not
having a single return type for an operation between two types seems
problematic. Additionally, the type signature for division says that
dividing two ints returns only an int which does not match the current
implementation (it can also return a float). This PR changes division
between almost all types to return a float (except for `filesize /
number` or `duration / number`, since there are no float representations
for these types).
Currently, floor division between certain types is not implemented even
though the type signature allows it. Also, the current implementation of
floor division uses a combination of clamping and flooring rather than
simply performing floor division which this PR fixes. Additionally, the
signature was changed so that `int // float`, `float // int`, and `float
// float` now return float instead of int. This matches the automatic
float promotion in the rest of the operators (as well as how Python does
floor division which I think is the original inspiration).
Since regular division has always returned fractional values (and now
returns a float to reflect that), `mod` is now defined in terms of floor
division. That is, `D // d = q`, `D mod d = r`, and `D = d * q + r `.
This is just like the `%` operator in Python, which is also based off
floor division (at least for ints and floats). Additionally,
implementations missing from `mod`'s current type signature have been
added (`duration mod int` and `duration mod float`).
This PR also overhauls the overflow checking and errors for div, mod,
and floor div. If an operation overflows, it will now cause an error.
# User-Facing Changes
- Div now returns a float in most cases.
- Floor division now actually does floor division.
- Floor division now does automatic float promotion, returning a float
in more instances.
- Floor division now actually allows division with filesize and
durations as its type signature claimed.
- Mod is now defined and implemented in terms of floor division rather
than truncating division.
- Mod now actually allows filesize and durations as its type signature
claimed.
- Div, mod, and floor div now all have proper overflow checks.
## Examples
When the divisor and the dividend have the same sign, the quotient and
remainder will be the same as before. (Except that this PR will give
more accurate results, since it does not do an intermediate float
conversion). If the signs of the divisor and dividend are different,
then the results will be different, or rather actually correct.
Before:
```nu
let q = 8 // -3 # -3
let r = 8 mod -3 # 2
8 == $q * -3 + $r # false
```
After:
```nu
let q = 8 // -3 # -3
let r = 8 mod -3 # -1
8 == $q * -3 + $r # true
```
Before:
```nu
let q = -8 // 3 # -3
let r = -8 mod 3 # -2
-8 == $q * 3 + $r # false
```
After:
```nu
let q = -8 // 3 # -3
let r = -8 mod 3 # 1
-8 == $q * 3 + $r # true
```
# Tests + Formatting
Added a few tests.
# After Submitting
Probably update the docs.
Fixes#14145
# User-Facing Changes
An empty rest match would be `null` previously. Now it will be an empty
list.
This is a breaking change for any scripts relying on the old behavior.
Example script:
```nu
match [1] {
[_ ..$rest] => {
match $rest {
null => { "old" }
[] => { "new" }
}
}
}
```
This expression would evaluate to "old" on current nu versions and "new"
with this patch.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Adds --no-deref flag to `touch`. Nice and backwards compatible, and I
get to touch symlinks. I still don't get to set their dates directly,
but maybe that'll come with utouch.
Some sadness in the implementation, since `set_symlink_file_times`
doesn't take Option values and we call it twice with the old "read"
values from reference (or now, if missing). This shouldn't be a big
concern since `touch` already did two calls if you set both mtime and
atime. Also, `--no-deref` applies both to the reference file, and to the
target file. No splitting them up, because that's silly.
Can always bikeshed. I nicked `--no-deref` from the uutils flag, and
made the short flag `-d` because it obviously can't be `-h`. I thought
of `-S` like in `glob`, for the "negative/filter out" uppercase short
letters. Ultimately I don't think it matters much.
Should fix#14212 since it's not really tied to uutils, besides the
comment about setting a `datetime` value directly.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
New flag.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Maybe.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Fixes: #13362
This PR fixes the `Display` impl for `CellPath`, as laid out in #13362
and #14090:
```nushell
> $.0."0"
$.0."0"
> $."foo.bar".baz
$."foo.bar".baz
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Cell-paths are now printed using the same `$.` notation that is used to
create them, and ambiguous column names are properly quoted.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR makes the `Display` implementation for `CellPath` show a `?`
suffix on every optional entry, which makes the output consistent with
the language syntax.
Before this PR, the printing of cell paths was confusing, e.g. `$.x` and
`$.x?` were both printed as `x`. Now, the second one is printed as `x?`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
The formatting of cell paths now matches the syntax used to create them,
reducing confusion.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
All tests pass, including `stdlib` tests.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Related to #14181
# Description
Our understanding of `ESC[3J` has apparently been wrong. And I say "our"
because I posted a [Super User
answer](https://superuser.com/a/1738611/1210833) a couple of years ago
with the same misconception (now fixed). In addition, the [crossterm
crate
doc](https://docs.rs/crossterm/latest/crossterm/terminal/enum.ClearType.html)
is wrong on the topic.
`ESC[3J` doesn't clear the screen plus the scrollback; it *only* clears
the scrollback. Reference the official [Xterm Control Sequences
doc](https://www.xfree86.org/4.8.0/ctlseqs.html).
> CSI P s J
>
> Erase in Display (ED)
>
> P s = 0 → Erase Below (default)
> P s = 1 → Erase Above
> P s = 2 → Erase All
> P s = 3 → Erase Saved Lines (xterm)
This also means that:
```nu
$"(ansi clear_entire_screen_plus_buffer)"
```
... doesn't.
This PR updates it to `ansi clear_scrollback_buffer` (short-code remains
the same).
# User-Facing Changes
Breaking-change: `ansi clear_entire_screen_plus_buffer` is renamed `ansi
clear_scrollback_buffer`
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
Self-documenting command via `ansi -l`
# Description
Makes `join` `right-table` support table literal notation instead of
parsing the column list (treated as empty data):
```diff
[{a: 1}] | join [[a]; [1]] a | to nuon
-[]
+[[a]; [1]]
```
Fixes#13537, fixes#14134
# Description
This PR closes#14137 and allows the display hook to be set on byte
streams. So, with a hook like this below.
```nushell
display_output: {
metadata access {|meta| match $meta.content_type? {
"application/x-nuscript" | "application/x-nuon" | "text/x-nushell" => { nu-highlight },
"application/json" => { ^bat --language=json --color=always --style=plain --paging=never },
_ => {},
}
} | table
}
```
You could type `open toolkit.nu` and the text of toolkit.nu would be
highlighted by nu-highlight. This PR also changes the way content-type
is assigned with `open`. Previously it would only assign it if `--raw`
was specified.
Lastly, it changes the `is_external()` function to only say
`ByteStreamSource::Child`'s are external instead of both Child and
`ByteStreamSource::File`. Again, this was to allow the hook to function
properly. I'm not sure what negative ramifications changing
`is_external()` could have, but there may be some?
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR tries to make `to text` more consistent with how it adds
newlines and also gives you an opt-out --no-newline option.
I wasn't sure how to change the `PipelineData::ByteStream` match arm. I
figure something needs to be done there but I'm not sure how to do it.
# User-Facing Changes
newlines are more consistent.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR aims to close#14027, in which it was noticed that the transpose
command "swallows" error messages.
*Note that in exploring the linked issue, [other situations were
identified](https://github.com/nushell/nushell/issues/14027#issuecomment-2414602880)
which also produce inconsistent behaviour. These have knowingly been
omitted from this PR, to minimize its scope, and since they seem to have
a different cause. It's probably best to make a separate issue/PR in
which to tackle a broader scan of error handling, with a suspected
relation to streams.*
# User-Facing Changes
The user will see errors from deeper in the pipeline, in case the errors
originated there.
# Tests + Formatting
Toolkit PR check was run successfully.
One test was added, covering this exact situation, in order to prevent
regressions.
The bug is relatively obscure, so it may be prone to reappear during
refactorings.
# Description
This PR adds a couple more options for dealing with try/catch errors. It
adds a `json` version of the error and a `rendered` version of the
error. It also respects the error_style configuration point.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR updates `group-by` and `split-by` to allow other nushell Values
to be used, namely bools.
### Before
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
Error: nu:🐚:cant_convert
× Can't convert to string.
╭─[entry #1:1:2]
1 │ [false, false, true, false, true, false] | group-by | table -e
· ──┬──
· ╰── can't convert bool to string
╰────
```
### After
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
╭───────┬───────────────╮
│ │ ╭───┬───────╮ │
│ false │ │ 0 │ false │ │
│ │ │ 1 │ false │ │
│ │ │ 2 │ false │ │
│ │ │ 3 │ false │ │
│ │ ╰───┴───────╯ │
│ │ ╭───┬──────╮ │
│ true │ │ 0 │ true │ │
│ │ │ 1 │ true │ │
│ │ ╰───┴──────╯ │
╰───────┴───────────────╯
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR adds a couple more options for dealing with try/catch errors. It
adds a `json` version of the error and a `rendered` version of the
error. It also respects the error_style configuration point.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR updates `group-by` and `split-by` to allow other nushell Values
to be used, namely bools.
### Before
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
Error: nu:🐚:cant_convert
× Can't convert to string.
╭─[entry #1:1:2]
1 │ [false, false, true, false, true, false] | group-by | table -e
· ──┬──
· ╰── can't convert bool to string
╰────
```
### After
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
╭───────┬───────────────╮
│ │ ╭───┬───────╮ │
│ false │ │ 0 │ false │ │
│ │ │ 1 │ false │ │
│ │ │ 2 │ false │ │
│ │ │ 3 │ false │ │
│ │ ╰───┴───────╯ │
│ │ ╭───┬──────╮ │
│ true │ │ 0 │ true │ │
│ │ │ 1 │ true │ │
│ │ ╰───┴──────╯ │
╰───────┴───────────────╯
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
The idea comes from @amtoine, I think it would be good to keey
`display_error.exit_code` same value, if user is using default config or
using no config file at all.
Closes#13920
# User-Facing Changes
`random binary` and `random chars` now support filesize arguments:
```nushell
random binary 1kb
random chars --length 1kb
```
# Description
Fixes#13991. This was done by more clearly separating the case when a
pipeline is drained vs when it is being written (to a file).
I also added an `OutDest::Print` case which might not be strictly
necessary, but is a helpful addition.
# User-Facing Changes
Bug fix.
# Tests + Formatting
Added a test.
# After Submitting
There are still a few redirection bugs that I found, but they require
larger code changes, so I'll leave them until after the release.
# Description
Contributors to this projects will have a test failure if their `umask`
is not set to `0022`.
Apparently on Debian (at least on my install), it is set to `0002` which
makes my test fail. While `0022` is safer than the value I have, I want
to reduce the amount if issue new contributors could have.
I am making this test not assuming anything and instead, reading the
user umask.
# Related discussion
I see that the `umask` command implementation has been discussed in
#12256 . We could use this and enforce a umask for tests who rely on
this. I believe however (let me know what you think) that hard coded
values are harder to read in the test.
# User-Facing Changes
N/A
# Tests + Formatting
All green on my side after this MR 👍
# After Submitting
Documentation is not impacted
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
Removes the `group` command that was deprecated back in 0.96.0 with
#13377.
# User-Facing Changes
Breaking change, removed `group` command.
# Description
Fixes: #13425
Similar to `source-env`, `use` command should also remove `FILE_PWD` and
`CURRENT_FILE` after evaluating code block in the module file.
And user input can be a directory, in this case, we need to use the
return value of `find_in_dirs_env` carefully, so in case, I renamed
`maybe_file_path` to `maybe_file_path_or_dir` to emphasize it.
# User-Facing Changes
`$env.FILE_PWD` and `$env.CURRENT_FILE` will be more reliable to use.
# Tests + Formatting
Added 2 test cases.
# After Submitting
NaN
# Description
Closes#12535
Implements sort-by functionality of #8322
Fixes sort-by part of #8667
This PR does two main things: add a new cell path and closure parameter
to `sort-by`, and attempt to make Nushell's sorting behavior
well-defined.
## `sort-by` features
The `columns` parameter is replaced with a `comparator` parameter, which
can be a cell path or a closure. Examples are from docs PR.
1. Cell paths
The basic interactive usage of `sort-by` is the same. For example, `ls |
sort-by modified` still works the same as before. It is not quite a
drop-in replacement, see [behavior changes](#behavior-changes).
Here's an example of how the cell path comparator might be useful:
```nu
> let cities = [
{name: 'New York', info: { established: 1624, population: 18_819_000 } }
{name: 'Kyoto', info: { established: 794, population: 37_468_000 } }
{name: 'São Paulo', info: { established: 1554, population: 21_650_000 }
}
]
> $cities | sort-by info.established
╭───┬───────────┬────────────────────────────╮
│ # │ name │ info │
├───┼───────────┼────────────────────────────┤
│ 0 │ Kyoto │ ╭─────────────┬──────────╮ │
│ │ │ │ established │ 794 │ │
│ │ │ │ population │ 37468000 │ │
│ │ │ ╰─────────────┴──────────╯ │
│ 1 │ São Paulo │ ╭─────────────┬──────────╮ │
│ │ │ │ established │ 1554 │ │
│ │ │ │ population │ 21650000 │ │
│ │ │ ╰─────────────┴──────────╯ │
│ 2 │ New York │ ╭─────────────┬──────────╮ │
│ │ │ │ established │ 1624 │ │
│ │ │ │ population │ 18819000 │ │
│ │ │ ╰─────────────┴──────────╯ │
╰───┴───────────┴────────────────────────────╯
```
2. Key closures
You can supply a closure which will transform each value into a sorting
key (without changing the underlying data). Here's an example of a key
closure, where we want to sort a list of assignments by their average
grade:
```nu
> let assignments = [
{name: 'Homework 1', grades: [97 89 86 92 89] }
{name: 'Homework 2', grades: [91 100 60 82 91] }
{name: 'Exam 1', grades: [78 88 78 53 90] }
{name: 'Project', grades: [92 81 82 84 83] }
]
> $assignments | sort-by { get grades | math avg }
╭───┬────────────┬───────────────────────╮
│ # │ name │ grades │
├───┼────────────┼───────────────────────┤
│ 0 │ Exam 1 │ [78, 88, 78, 53, 90] │
│ 1 │ Project │ [92, 81, 82, 84, 83] │
│ 2 │ Homework 2 │ [91, 100, 60, 82, 91] │
│ 3 │ Homework 1 │ [97, 89, 86, 92, 89] │
╰───┴────────────┴───────────────────────╯
```
3. Custom sort closure
The `--custom`, or `-c`, flag will tell `sort-by` to interpret closures
as custom sort closures. A custom sort closure has two parameters, and
returns a boolean. The closure should return `true` if the first
parameter comes _before_ the second parameter in the sort order.
For a simple example, we could rewrite a cell path sort as a custom sort
(see
[here](https://github.com/nushell/nushell.github.io/pull/1568/files#diff-a7a233e66a361d8665caf3887eb71d4288000001f401670c72b95cc23a948e86R231)
for a more complex example):
```nu
> ls | sort-by -c {|a, b| $a.size < $b.size }
╭───┬─────────────────────┬──────┬──────────┬────────────────╮
│ # │ name │ type │ size │ modified │
├───┼─────────────────────┼──────┼──────────┼────────────────┤
│ 0 │ my-secret-plans.txt │ file │ 100 B │ 10 minutes ago │
│ 1 │ shopping_list.txt │ file │ 100 B │ 2 months ago │
│ 2 │ myscript.nu │ file │ 1.1 KiB │ 2 weeks ago │
│ 3 │ bigfile.img │ file │ 10.0 MiB │ 3 weeks ago │
╰───┴─────────────────────┴──────┴──────────┴────────────────╯
```
## Making sort more consistent
I think it's important for something as essential as `sort` to have
well-defined semantics. This PR contains some changes to try to make the
behavior of `sort` and `sort-by` consistent. In addition, after working
with the internals of sorting code, I have a much deeper understanding
of all of the edge cases. Here is my attempt to try to better define
some of the semantics of sorting (if you are just interested in changes,
skip to "User-Facing changes")
- `sort`, `sort -v`, and `sort-by` now all work the same. Each
individual sort implementation has been refactored into two functions in
`sort_utils.rs`: `sort`, and `sort_by`. These can also be used in other
parts of Nushell where values need to be sorted.
- `sort` and `sort-by` used to handle `-i` and `-n` differently.
- `sort -n` would consider all values which can't be coerced into a
string to be equal
- `sort-by -i` and `sort-by -n` would only work if all values were
strings
- In this PR, insensitive sort only affects comparison between strings,
and natural sort only applies to numbers and strings (see below).
- (not a change) Before and after this PR, `sort` and `sort-by` support
sorting mixed types. There was a lot of discussion about potentially
making `sort` and `sort-by` only work on lists of homogeneous types, but
the general consensus was that `sort` should not error just because its
input contains incompatible types.
- In order to try to make working with data containing `null` values
easier, I changed the PartialOrd order to sort `Nothing` values to the
end of a list, regardless of what other types the list contains. Before,
`null` would be sorted before `Binary`, `CellPath`, and `Custom` values.
- (not a change) When sorted, lists of mixed types will contain sorted
values of each type in order, for the most part
- (not a change) For example, `[0x[1] (date now) "a" ("yesterday" | into
datetime) "b" 0x[0]]` will be sorted as `["a", "b", a day ago, now, [0],
[1]]`, where sorted strings appear first, then sorted datetimes, etc.
- (not a change) The exception to this is `Int`s and `Float`s, which
will intermix, `Strings` and `Glob`s, which will intermix, and `None` as
described above. Additionally, natural sort will intermix strings with
ints and floats (see below).
- Natural sort no longer coerce all inputs to strings.
- I did originally make natural only apply to strings, but @fdncred
pointed out that the previous behavior also allowed you to sort numeric
strings with numbers. This seems like a useful feature if we are trying
to support sorting with mixed types, so I settled on coercing only
numbers (int, float). This can be reverted if people don't like it.
- Here is an example of this behavior in action, which is the same
before and after this PR:
```nushell
$ [1 "4" 3 "2"] | sort --natural
╭───┬───╮
│ 0 │ 1 │
│ 1 │ 2 │
│ 2 │ 3 │
│ 3 │ 4 │
╰───┴───╯
```
# User-Facing Changes
## New features
- Replaces the `columns` string parameter of `sort-by` with a cell path
or a closure.
- The cell path parameter works exactly as you would expect
- By default, the `closure` parameter acts as a "key sort"; that is,
each element is transformed by the closure into a sorting key
- With the `--custom` (`-c`) parameter, you can define a comparison
function for completely custom sorting order.
## Behavior changes
<details>
<summary><code>sort -v</code> does not coerce record values to
strings</summary>
This was a bit of a surprising behavior, and is now unified with the
behavior of `sort` and `sort-by`. Here's an example where you can
observe the values being implicitly coerced into strings for sorting, as
they are sorted like strings rather than numbers:
Old behavior:
```nushell
$ {foo: 9 bar: 10} | sort -v
╭─────┬────╮
│ bar │ 10 │
│ foo │ 9 │
╰─────┴────╯
```
New behavior:
```nushell
$ {foo: 9 bar: 10} | sort -v
╭─────┬────╮
│ foo │ 9 │
│ bar │ 10 │
╰─────┴────╯
```
</details>
<details>
<summary>Changed <code>sort-by</code> parameters from
<code>string</code> to <code>cell-path</code> or <code>closure</code>.
Typical interactive usage is the same as before, but if passing a
variable to <code>sort-by</code> it must be a cell path (or closure),
not a string</summary>
Old behavior:
```nushell
$ let sort = "modified"
$ ls | sort-by $sort
╭───┬──────┬──────┬──────┬────────────────╮
│ # │ name │ type │ size │ modified │
├───┼──────┼──────┼──────┼────────────────┤
│ 0 │ foo │ file │ 0 B │ 10 hours ago │
│ 1 │ bar │ file │ 0 B │ 35 seconds ago │
╰───┴──────┴──────┴──────┴────────────────╯
```
New behavior:
```nushell
$ let sort = "modified"
$ ls | sort-by $sort
Error: nu:🐚:type_mismatch
× Type mismatch.
╭─[entry #10:1:14]
1 │ ls | sort-by $sort
· ──┬──
· ╰── Cannot sort using a value which is not a cell path or closure
╰────
$ let sort = $."modified"
$ ls | sort-by $sort
╭───┬──────┬──────┬──────┬───────────────╮
│ # │ name │ type │ size │ modified │
├───┼──────┼──────┼──────┼───────────────┤
│ 0 │ foo │ file │ 0 B │ 10 hours ago │
│ 1 │ bar │ file │ 0 B │ 2 minutes ago │
╰───┴──────┴──────┴──────┴───────────────╯
```
</details>
<details>
<summary>Insensitve and natural sorting behavior reworked</summary>
Previously, the `-i` and `-n` worked differently for `sort` and
`sort-by` (see "Making sort more consistent"). Here are examples of how
these options result in different sorts now:
1. `sort -n`
- Old behavior (types other than numbers, strings, dates, and binary
sorted incorrectly)
```nushell
$ [2sec 1sec] | sort -n
╭───┬──────╮
│ 0 │ 2sec │
│ 1 │ 1sec │
╰───┴──────╯
```
- New behavior
```nushell
$ [2sec 1sec] | sort -n
╭───┬──────╮
│ 0 │ 1sec │
│ 1 │ 2sec │
╰───┴──────╯
```
2. `sort-by -i`
- Old behavior (uppercase words appear before lowercase words as they
would in a typical sort, indicating this is not actually an insensitive
sort)
```nushell
$ ["BAR" "bar" "foo" 2 "FOO" 1] | wrap a | sort-by -i a
╭───┬─────╮
│ # │ a │
├───┼─────┤
│ 0 │ 1 │
│ 1 │ 2 │
│ 2 │ BAR │
│ 3 │ FOO │
│ 4 │ bar │
│ 5 │ foo │
╰───┴─────╯
```
- New behavior (strings are sorted stably, indicating this is an
insensitive sort)
```nushell
$ ["BAR" "bar" "foo" 2 "FOO" 1] | wrap a | sort-by -i a
╭───┬─────╮
│ # │ a │
├───┼─────┤
│ 0 │ 1 │
│ 1 │ 2 │
│ 2 │ BAR │
│ 3 │ bar │
│ 4 │ foo │
│ 5 │ FOO │
╰───┴─────╯
```
3. `sort-by -n`
- Old behavior (natural sort does not work when data contains non-string
values)
```nushell
$ ["10" 8 "9"] | wrap a | sort-by -n a
╭───┬────╮
│ # │ a │
├───┼────┤
│ 0 │ 8 │
│ 1 │ 10 │
│ 2 │ 9 │
╰───┴────╯
```
- New behavior
```nushell
$ ["10" 8 "9"] | wrap a | sort-by -n a
╭───┬────╮
│ # │ a │
├───┼────┤
│ 0 │ 8 │
│ 1 │ 9 │
│ 2 │ 10 │
╰───┴────╯
```
</details>
<details>
<summary>
Sorting a list of non-record values with a non-existent column/path now
errors instead of sorting the values directly (<code>sort</code> should
be used for this, not <code>sort-by</code>)
</summary>
Old behavior:
```nushell
$ [2 1] | sort-by foo
╭───┬───╮
│ 0 │ 1 │
│ 1 │ 2 │
╰───┴───╯
```
New behavior:
```nushell
$ [2 1] | sort-by foo
Error: nu:🐚:incompatible_path_access
× Data cannot be accessed with a cell path
╭─[entry #29:1:17]
1 │ [2 1] | sort-by foo
· ─┬─
· ╰── int doesn't support cell paths
╰────
```
</details>
<details>
<summary><code>sort</code> and <code>sort-by</code> output
<code>List</code> instead of <code>ListStream</code> </summary>
This isn't a meaningful change (unless I misunderstand the purpose of
ListStream), since `sort` and `sort-by` both need to collect in order to
do the sorting anyway, but is user observable.
Old behavior:
```nushell
$ ls | sort | describe -d
╭──────────┬───────────────────╮
│ type │ stream │
│ origin │ nushell │
│ subtype │ {record 3 fields} │
│ metadata │ {record 1 field} │
╰──────────┴───────────────────╯
```
```nushell
$ ls | sort-by name | describe -d
╭──────────┬───────────────────╮
│ type │ stream │
│ origin │ nushell │
│ subtype │ {record 3 fields} │
│ metadata │ {record 1 field} │
╰──────────┴───────────────────╯
```
New behavior:
```nushell
ls | sort | describe -d
╭────────┬─────────────────╮
│ type │ list │
│ length │ 22 │
│ values │ [table 22 rows] │
╰────────┴─────────────────╯
```
```nushell
$ ls | sort-by name | describe -d
╭────────┬─────────────────╮
│ type │ list │
│ length │ 22 │
│ values │ [table 22 rows] │
╰────────┴─────────────────╯
```
</details>
- `sort` now errors when nothing is piped in (`sort-by` already did
this)
# Tests + Formatting
I added lots of unit tests on the new sort implementation to enforce new
sort behaviors and prevent regressions.
# After Submitting
See [docs PR](https://github.com/nushell/nushell.github.io/pull/1568),
which is ~2/3 finished.
---------
Co-authored-by: NotTheDr01ds <32344964+NotTheDr01ds@users.noreply.github.com>
Co-authored-by: Ian Manske <ian.manske@pm.me>
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Maybe we can deprecate `encode new-base64` and `decode new-base64`
first, to make the code clean and simple I'd rather remove the old
`encode base64` and `decode base64` and replace them with the `*
new-base64` commands.
Related PR: https://github.com/nushell/nushell/pull/13428
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
- `encode new-base64` --> `encode base64`
- `decode new-base64` --> `decode base64`
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
It's a breaking change
# Description
Fixes a couple panics:
```
> {} | inspect
Error: x Main thread panicked.
|-> at crates/nu-command/src/debug/inspect_table.rs:87:15
`-> attempt to divide by zero
```
```
> {} | explore
# see an empty column, press Down
Error: x Main thread panicked.
|-> at crates/nu-explore/src/views/cursor/mod.rs:39:9
`-> attempt to subtract with overflow
```
# User-Facing Changes
`{} | inspect` now outputs an empty table:
```
╭─────────────┬────────╮
│ description │ record │
├─────────────┴────────┤
│ │
├──────────────────────┤
```
`{} | explore` opens the help menu.
Both match the empty list behavior.
# Tests
I'm not sure how to test `explore`, as it waits for interaction.
# Description
Partialy addresses #13868. `try` does not catch non-zero exit code
errors from the last command in a pipeline if the result is assigned to
a variable using `let` (or `mut`).
This was fixed by adding a new `OutDest::Value` case. This is used when
the pipeline is in a "value" position. I.e., it will be collected into a
value. This ended up replacing most of the usages of `OutDest::Capture`.
So, this PR also renames `OutDest::Capture` to `OutDest::PipeSeparate`
to better fit the few remaining use cases for it.
# User-Facing Changes
Bug fix.
# Tests + Formatting
Added two tests.
# Description
Similar to #13870 (thanks @WindSoilder), this PR adds a boolean which
determines whether to ignore any errors from an external command. This
is in order to fix#13876. I.e., `do -p` does not wait for externals to
complete before continuing.
# User-Facing Changes
Bug fix.
# Tests + Formatting
Added a test.
This allows parsing of data (e.g. key-value pairs) where the last column
may contain the delimiter.
- this PR should close#13742
# Description
Adds a `--number (-n)` flag to `split column`, analogous to `split row
--number`.
```
~> ['author: Salina Yoon' r#'title: Where's Ellie?: A Hide-and-Seek Book'#] | split column --number 2 ': ' key value
╭───┬────────┬──────────────────────────────────────╮
│ # │ key │ value │
├───┼────────┼──────────────────────────────────────┤
│ 0 │ author │ Salina Yoon │
│ 1 │ title │ Where's Ellie?: A Hide-and-Seek Book │
╰───┴────────┴──────────────────────────────────────╯
```
# User-Facing Changes
* `split column` gains a `--number` option
# Tests + Formatting
Tests included in strings::split::column::test::test_examples and
commands::split_column::to_column.
# After Submitting
Reference documentation is auto-generated from code. No other
documentation updates necessary.
# Description
Fixes: #13460
The issue is caused by `try_exists` method on path, it will return
`Err(NotADirectory)` if user tried to check for a path under a regular
file.
To fix it, I think it's ok to use `exists` rather than `try_exists`,
although
[Path::exists()](https://doc.rust-lang.org/std/path/struct.Path.html#method.exists)
only checks whether or not a path was both found and readable. I think
it's ok, and we can add this information under `extra_description`.
# User-Facing Changes
The following code will no longer raise error:
```
touch a; 'a/b' | path exists
```
# Tests + Formatting
Added 1 test.
# Description
Fixes a bug in the IR for `try` to match that of the regular evaluator
(continuing from #13515):
```nushell
# without IR:
try { ^false } catch { 'caught' } # == 'caught'
# with IR:
try { ^false } catch { 'caught' } # error, non-zero exit code
```
In this PR, both now evaluate to `caught`. For the implementation, I had
to add another instruction, and feel free to suggest better
alternatives. In the future, it might be possible to get rid of this
extra instruction.
# User-Facing Changes
Bug fix, `try { ^false } catch { 'caught' }` now works in IR.
# Description
This PR makes it so that non-zero exit codes and termination by signal
are treated as a normal `ShellError`. Currently, these are silent
errors. That is, if an external command fails, then it's code block is
aborted, but the parent block can sometimes continue execution. E.g.,
see #8569 and this example:
```nushell
[1 2] | each { ^false }
```
Before this would give:
```
╭───┬──╮
│ 0 │ │
│ 1 │ │
╰───┴──╯
```
Now, this shows an error:
```
Error: nu:🐚:eval_block_with_input
× Eval block failed with pipeline input
╭─[entry #1:1:2]
1 │ [1 2] | each { ^false }
· ┬
· ╰── source value
╰────
Error: nu:🐚:non_zero_exit_code
× External command had a non-zero exit code
╭─[entry #1:1:17]
1 │ [1 2] | each { ^false }
· ──┬──
· ╰── exited with code 1
╰────
```
This PR fixes#12874, fixes#5960, fixes#10856, and fixes#5347. This
PR also partially addresses #10633 and #10624 (only the last command of
a pipeline is currently checked). It looks like #8569 is already fixed,
but this PR will make sure it is definitely fixed (fixes#8569).
# User-Facing Changes
- Non-zero exit codes and termination by signal now cause an error to be
thrown.
- The error record value passed to a `catch` block may now have an
`exit_code` column containing the integer exit code if the error was due
to an external command.
- Adds new config values, `display_errors.exit_code` and
`display_errors.termination_signal`, which determine whether an error
message should be printed in the respective error cases. For
non-interactive sessions, these are set to `true`, and for interactive
sessions `display_errors.exit_code` is false (via the default config).
# Tests
Added a few tests.
# After Submitting
- Update docs and book.
- Future work:
- Error if other external commands besides the last in a pipeline exit
with a non-zero exit code. Then, deprecate `do -c` since this will be
the default behavior everywhere.
- Add a better mechanism for exit codes and deprecate
`$env.LAST_EXIT_CODE` (it's buggy).
# Description
The content-type was not being handled appropriately when sending
requests with a string value.
# User-Facing Changes
- Passing a string value through a pipeline with a content-type set as
metadata is handled correctly
- Passing a string value as a parameter with a content-type set as a
flag is handled correctly.
# Description
After looking at #13751 I noticed that the config setting
`use_grid_icons` seems out of place. So, I removed it from the config
and made it a parameter to the `grid` command.
Closes#13687Closes#13686
# Description
Light refactoring of `send_request `in `client.rs`. In the end there are
more lines but now the logic is more concise and facilitates adding new
conditions in the future. Unit tests ran fine and I tested a few cases
manually.
Cool project btw, I'll be using nushell from now on.
# Description
This changes the behavior of `tee` to be more transparent when given a
value that isn't a list or range. Previously, anything that wasn't a
byte stream would converted to a list stream using the iterator
implementation, which led to some surprising results. Instead, now, if
the value is a string or binary, it will be treated the same way a byte
stream is, and the output of `tee` is a byte stream instead of the
original value. This is done so that we can synchronize with the other
thread on collect, and potentially capture any error produced by the
closure.
For values that can't be converted to streams, the closure is just run
with a clone of the value instead on another thread. Because we can't
wait for the other thread, there is no way to send an error back to the
original thread, so instead it's just written to stderr using
`report_error_new()`.
There are a couple of follow up edge cases I see where byte streams
aren't necessarily treated exactly the same way strings are, but this
should mostly be a good experience.
Fixes#13489.
# User-Facing Changes
Breaking change.
- `tee` now outputs and sends string/binary stream for string/binary
input.
- `tee` now outputs and sends the original value for any other input
other than lists/ranges.
# Tests + Formatting
Added for new behavior.
# After Submitting
- [ ] release notes: breaking change, command change
Mistakes have been made. I forgot about a bunch of `todo`s in the helper
functions. So, this PR replaces them with proper errors. It also adds
tests for parse-time evaluation, because one `todo` I missed was in a
`run_const` function.
Based on the discussion in #13419.
## Description
Reworks the `decode`/`encode` commands by adding/changing the following
bases:
- `base32`
- `base32hex`
- `hex`
- `new-base64`
The `hex` base is compatible with the previous version of `hex` out of
the box (it only adds more flags). `base64` isn't, so the PR adds a new
version and deprecates the old one.
All commands have `string -> binary` signature for decoding and `string
| binary -> string` signature for encoding. A few `base64` encodings,
which are not a part of the
[RFC4648](https://datatracker.ietf.org/doc/html/rfc4648#section-6), have
been dropped.
## Example usage
```Nushell
~/fork/nushell> "string" | encode base32 | decode base32 | decode
string
```
```Nushell
~/fork/nushell> "ORSXG5A=" | decode base32
# `decode` always returns a binary value
Length: 4 (0x4) bytes | printable whitespace ascii_other non_ascii
00000000: 74 65 73 74 test
```
## User-Facing Changes
- New commands: `encode/decode base32/base32hex`.
- `encode hex` gets a `--lower` flag.
- `encode/decode base64` deprecated in favor of `encode/decode
new-base64`.
# Description
The meaning of the word usage is specific to describing how a command
function is *used* and not a synonym for general description. Usage can
be used to describe the SYNOPSIS or EXAMPLES sections of a man page
where the permitted argument combinations are shown or example *uses*
are given.
Let's not confuse people and call it what it is a description.
Our `help` command already creates its own *Usage* section based on the
available arguments and doesn't refer to the description with usage.
# User-Facing Changes
`help commands` and `scope commands` will now use `description` or
`extra_description`
`usage`-> `description`
`extra_usage` -> `extra_description`
Breaking change in the plugin protocol:
In the signature record communicated with the engine.
`usage`-> `description`
`extra_usage` -> `extra_description`
The same rename also takes place for the methods on
`SimplePluginCommand` and `PluginCommand`
# Tests + Formatting
- Updated plugin protocol specific changes
# After Submitting
- [ ] update plugin protocol doc
# Description
The previous behaviour of `into record` on lists was to create a new
record with each list index as the key. This was not very useful for
creating meaningful records, though, and most people would end up using
commands like `headers` or `transpose` to turn a list of keys and values
into a record.
This PR changes that instead to do what I think the most ergonomic thing
is, and instead:
- A list of records is merged into one record.
- A list of pairs (two element lists) is folded into a record with the
first element of each pair being the key, and the second being the
value.
The former is just generally more useful than having to use `reduce`
with `merge` for such a common operation, and the latter is useful
because it means that `$a | zip $b | into record` *just works* in the
way that seems most obvious.
Example:
```nushell
[[foo bar] [baz quux]] | into record # => {foo: bar, baz: quux}
[{foo: bar} {baz: quux}] | into record # => {foo: bar, baz: quux}
[foo baz] | zip [bar quux] | into record # => {foo: bar, baz: quux}
```
The support for range input has been removed, as it would no longer
reflect the treatment of an equivalent list.
The following is equivalent to the old behavior, in case that's desired:
```
0.. | zip [a b c] | into record # => {0: a, 1: b, 2: c}
```
# User-Facing Changes
- `into record` changed as described above (breaking)
- `into record` no longer supports range input (breaking)
# Tests + Formatting
Examples changed to match, everything works. Some usage in stdlib and
`nu_plugin_nu_example` had to be changed.
# After Submitting
- [ ] release notes (commands, breaking change)
# Description
Previously when nushell failed to parse the content type header, it
would emit an error instead of returning the response. Now it will fall
back to `text/plain` (which, in turn, will trigger type detection based
on file extension).
May fix (potentially) nushell/nushell#11927
Refs:
https://discord.com/channels/601130461678272522/614593951969574961/1272895236489613366
Supercedes: #13609
# User-Facing Changes
It's now possible to fetch content even if the server returns an invalid
content type header. Users may need to parse the response manually, but
it's still better than not getting the response at all.
# Tests + Formatting
Added a test for the new behaviour.
# After Submitting
# Description
Fixes: #13479
# User-Facing Changes
Given the following setup:
```
cd /tmp
touch src_file.txt
ln -s src_file.txt link1
```
### Before
```
ls -lf link1 | get target.0 # It outputs src_file.txt
```
### After
```
ls -lf link1 | get target.0 # It outputs /tmp/src_file.txt
```
# Tests + Formatting
Added a test for the change
# Description
Something I meant to add a long time ago. We currently don't have a
convenient way to print raw binary data intentionally. You can pipe it
through `cat` to turn it into an unknown stream, or write it to a file
and read it again, but we can't really just e.g. generate msgpack and
write it to stdout without this. For example:
```nushell
[abc def] | to msgpack | print --raw
```
This is useful for nushell scripts that will be piped into something
else. It also means that `nu_plugin_nu_example` probably doesn't need to
do this anymore, but I haven't adjusted it yet:
```nushell
def tell_nushell_encoding [] {
print -n "\u{0004}json"
}
```
This happens to work because 0x04 is a valid UTF-8 character, but it
wouldn't be possible if it were something above 0x80.
`--raw` also formats other things without `table`, I figured the two
things kind of go together. The output is kind of like `to text`.
Debatable whether that should share the same flag, but it was easier
that way and seemed reasonable.
# User-Facing Changes
- `print` new flag: `--raw`
# Tests + Formatting
Added tests.
# After Submitting
- [ ] release notes (command modified)
This PR closes [Issue
#13482](https://github.com/nushell/nushell/issues/13482)
# Description
This PR tend to make all math function to be constant.
# User-Facing Changes
The math commands now can be used as constant methods.
### Some Example
```
> const MODE = [3 3 9 12 12 15] | math mode
> $MODE
╭───┬────╮
│ 0 │ 3 │
│ 1 │ 12 │
╰───┴────╯
> const LOG = [16 8 4] | math log 2
> $LOG
╭───┬──────╮
│ 0 │ 4.00 │
│ 1 │ 3.00 │
│ 2 │ 2.00 │
╰───┴──────╯
> const VAR = [1 3 5] | math variance
> $VAR
2.6666666666666665
```
# Tests + Formatting
Tests are added for all of the math command to test there constant
behavior.
I mostly focused on the actual user experience, not the correctness of
the methods and algorithms.
# After Submitting
I think this change don't require any additional documentation. Feel
free to correct me in this topic please.
# Description
Attempt to guess the content type of a file when opening with --raw and
set it in the pipeline metadata.
<img width="644" alt="Screenshot 2024-08-02 at 11 30 10"
src="https://github.com/user-attachments/assets/071f0967-c4dd-405a-b8c8-f7aa073efa98">
# User-Facing Changes
- Content of files can be directly piped into commands like `http post`
with the content type set appropriately when using `--raw`.
# Description
Part 4 of replacing std::path types with nu_path types added in
https://github.com/nushell/nushell/pull/13115. This PR migrates various
tests throughout the code base.
- **Doccomment style fixes**
- **Forgotten stuff in `nu-pretty-hex`**
- **Don't `for` around an `Option`**
- and more
I think the suggestions here are a net positive, some of the suggestions
moved into #13498 feel somewhat arbitrary, I also raised
https://github.com/rust-lang/rust-clippy/issues/13188 as the nightly
`byte_char_slices` would require either a global allow or otherwise a
ton of granular allows or possibly confusing bytestring literals.
# Description
Fixes: #13260
When user run a command like this:
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```
`save` command sinks the output of previous commands' stderr output. I
think it should be `stderr`.
# User-Facing Changes
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```
The command will output ` New Err` to stderr.
# Tests + Formatting
Added 2 cases.
- **Suggested default impl for the new `*Stack`s**
- **Change a hashmap to make clippy happy**
- **Clone from fix**
- **Fix conditional unused in test**
- then **Bump rust toolchain**
# Description
This makes assignment operations and `const` behave the same way `let`
and `mut` do, absorbing the rest of the pipeline.
Changes the lexer to be able to recognize assignment operators as a
separate token, and then makes the lite parser continue to push spans
into the same command regardless of any redirections or pipes if an
assignment operator is encountered. Because the pipeline is no longer
split up by the lite parser at this point, it's trivial to just parse
the right hand side as if it were a subexpression not contained within
parentheses.
# User-Facing Changes
Big breaking change. These are all now possible:
```nushell
const path = 'a' | path join 'b'
mut x = 2
$x = random int
$x = [1 2 3] | math sum
$env.FOO = random chars
```
In the past, these would have led to (an attempt at) bare word string
parsing. So while `$env.FOO = bar` would have previously set the
environment variable `FOO` to the string `"bar"`, it now tries to run
the command named `bar`, hence the major breaking change.
However, this is desirable because it is very consistent - if you see
the `=`, you can just assume it absorbs everything else to the right of
it.
# Tests + Formatting
Added tests for the new behaviour. Adjusted some existing tests that
depended on the right hand side of assignments being parsed as
barewords.
# After Submitting
- [ ] release notes (breaking change!)
# Description
Close: #12083Close: #12084
# User-Facing Changes
It's a breaking change because we have switched the position of
`<initial>` and `<closure>`, after the change, initial value will be
optional. So it's possible to do something like this:
```nushell
> let f = {|fib = [0, 1]| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} }
> generate $f | first 5
╭───┬───╮
│ 0 │ 0 │
│ 1 │ 1 │
│ 2 │ 1 │
│ 3 │ 2 │
│ 4 │ 3 │
╰───┴───╯
```
It will also raise error if user don't give initial value, and the
closure don't have default parameter.
```nushell
❯ let f = {|fib| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} }
❯ generate $f
Error: × The initial value is missing
╭─[entry #5:1:1]
1 │ generate $f
· ────┬───
· ╰── Missing intial value
╰────
help: Provide <initial> value in generate, or assigning default value to closure parameter
```
# Tests + Formatting
Added some test cases.
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
Following from #13377, this PR refactors the related `window` command.
In the case where `window` should behave exactly like `chunks`, it now
reuses the same code that `chunks` does. Otherwise, the other cases have
been rewritten and have resulted in a performance increase:
| window size | stride | old time (ms) | new time (ms) |
| -----------:| ------:| -------------:| -------------:|
| 20 | 1 | 757 | 722 |
| 2 | 1 | 364 | 333 |
| 1 | 1 | 343 | 293 |
| 20 | 20 | 90 | 63 |
| 2 | 2 | 215 | 175 |
| 20 | 30 | 74 | 60 |
| 2 | 4 | 141 | 110 |
# User-Facing Changes
`window` will now error if the window size or stride is 0, which is
technically a breaking change.
# Description
This grew quite a bit beyond its original scope, but I've tried to make
`$in` a bit more consistent and easier to work with.
Instead of the parser generating calls to `collect` and creating
closures, this adds `Expr::Collect` which just evaluates in the same
scope and doesn't require any closure.
When `$in` is detected in an expression, it is replaced with a new
variable (also called `$in`) and wrapped in `Expr::Collect`. During
eval, this expression is evaluated directly, with the input and with
that new variable set to the collected value.
Other than being faster and less prone to gotchas, it also makes it
possible to typecheck the output of an expression containing `$in`,
which is nice. This is a breaking change though, because of the lack of
the closure and because now typechecking will actually happen. Also, I
haven't attempted to typecheck the input yet.
The IR generated now just looks like this:
```gas
collect %in
clone %tmp, %in
store-variable $in, %tmp
# %out <- ...expression... <- %in
drop-variable $in
```
(where `$in` is the local variable created for this collection, and not
`IN_VARIABLE_ID`)
which is a lot better than having to create a closure and call `collect
--keep-env`, dealing with all of the capture gathering and allocation
that entails. Ideally we can also detect whether that input is actually
needed, so maybe we don't have to clone, but I haven't tried to do that
yet. Theoretically now that the variable is a unique one every time, it
should be possible to give it a type - I just don't know how to
determine that yet.
On top of that, I've also reworked how `$in` works in pipeline-initial
position. Previously, it was a little bit inconsistent. For example,
this worked:
```nushell
> 3 | do { let x = $in; let y = $in; print $x $y }
3
3
```
However, this causes a runtime variable not found error on the second
`$in`:
```nushell
> def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
Error: nu:🐚:variable_not_found
× Variable not found
╭─[entry #115:1:35]
1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
· ─┬─
· ╰── variable not found
╰────
```
I've fixed this by making the first element `$in` detection *always*
happen at the block level, so if you use `$in` in pipeline-initial
position anywhere in a block, it will collect with an implicit
subexpression around the whole thing, and you can then use that `$in`
more than once. In doing this I also rewrote `parse_pipeline()` and
hopefully it's a bit more straightforward and possibly more efficient
too now.
Finally, I've tried to make `let` and `mut` a lot more straightforward
with how they handle the rest of the pipeline, and using a redirection
with `let`/`mut` now does what you'd expect if you assume that they
consume the whole pipeline - the redirection is just processed as
normal. These both work now:
```nushell
let x = ^foo err> err.txt
let y = ^foo out+err>| str length
```
It was previously possible to accomplish this with a subexpression, but
it just seemed like a weird gotcha that you couldn't do it. Intuitively,
`let` and `mut` just seem to take the whole line.
- closes#13137
# User-Facing Changes
- `$in` will behave more consistently with blocks and closures, since
the entire block is now just wrapped to handle it if it appears in the
first pipeline element
- `$in` no longer creates a closure, so what can be done within an
expression containing `$in` is less restrictive
- `$in` containing expressions are now type checked, rather than just
resulting in `any`. However, `$in` itself is still `any`, so this isn't
quite perfect yet
- Redirections are now allowed in `let` and `mut` and behave pretty much
how you'd expect
# Tests + Formatting
Added tests to cover the new behaviour.
# After Submitting
- [ ] release notes (definitely breaking change)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Replaces the `dirs_next` family of crates with `dirs`. `dirs_next` was
born when the `dirs` crates were abandoned three years ago, but they're
being maintained again and most projects depend on `dirs` nowadays.
`dirs_next` has been abandoned since.
This came up while working on
https://github.com/nushell/nushell/pull/13382.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
None.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests and formatter have been run.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Before this change `default` would dive into lists and replace `null`
elements with the default value.
Therefore there was no easy way to process `null|list<null|string>`
input and receive `list<null|string>`
(IOW to apply the default on the top level only).
However it's trivially easy to apply default values to list elements
with the `each` command:
```nushell
[null, "a", null] | each { default "b" }
```
So there's no need to have this behavior in `default` command.
# User-Facing Changes
* `default` no longer dives into lists to replace `null` elements with
the default value.
# Tests + Formatting
Added a couple of tests for the new (and old) behavior.
# After Submitting
* Update docs.
# Description
The name of the `group` command is a little unclear/ambiguous.
Everything I look at it, I think of `group-by`. I think `chunks` more
clearly conveys what the `group` command does. Namely, it divides the
input list into chunks of a certain size. For example,
[`slice::chunks`](https://doc.rust-lang.org/std/primitive.slice.html#method.chunks)
has the same name. So, this PR adds a new `chunks` command to replace
the now deprecated `group` command.
The `chunks` command is a refactored version of `group`. As such, there
is a small performance improvement:
```nushell
# $data is a very large list
> bench { $data | chunks 2 } --rounds 30 | get mean
474ms 921µs 190ns
# deprecation warning was disabled here for fairness
> bench { $data | group 2 } --rounds 30 | get mean
592ms 702µs 440ns
> bench { $data | chunks 200 } --rounds 30 | get mean
374ms 188µs 318ns
> bench { $data | group 200 } --rounds 30 | get mean
481ms 264µs 869ns
> bench { $data | chunks 1 } --rounds 30 | get mean
642ms 574µs 42ns
> bench { $data | group 1 } --rounds 30 | get mean
981ms 602µs 513ns
```
# User-Facing Changes
- `group` command has been deprecated in favor of new `chunks` command.
- `chunks` errors when given a chunk size of `0` whereas `group` returns
chunks with one element.
# Tests + Formatting
Added tests for `chunks`, since `group` did not have any tests.
# After Submitting
Update book if necessary.
# Description
Fixes#13359
In an attempt to generate names for flat columns resulting from a nested
accesses #3016 generated new column names on nested selection, out of
convenience, that composed the cell path as a string (including `.`) and
then simply replaced all `.` with `_`. As we permit `.` in column names
as long as you quote this surprisingly alters `select`ed columns.
# User-Facing Changes
New columns generated by selection with nested cell paths will for now
be named with a string containing the keys separated by `.` instead of
`_`. We may want to reconsider the semantics for nested access.
# Tests + Formatting
- Alter test to breaking change on nested `select`
Somehow this logic was missed on my end. ( I mean I was not even
thinking about it in original patch 😄 )
Please recheck
Added a regression test too.
close#13336
cc: @fdncred
GOOD CATCH.............................................................
SORRY
I've added a test to catch regression just in case.
close#13319
cc: @fdncred
# Description
Bug fix: `PipelineData::check_external_failed()` was not preserving the
original `type_` and `known_size` attributes of the stream passed in for
streams that come from children, so `external-command | into binary` did
not work properly and always ended up still being unknown type.
# User-Facing Changes
The following test case now works as expected:
```nushell
> head -c 2 /dev/urandom | into binary
# Expected: pretty hex dump of binary
# Previous behavior: just raw binary in the terminal
```
# Tests + Formatting
Added a test to cover this to `into binary`
# Description
Provides the ability to use http commands as part of a pipeline.
Additionally, this pull requests extends the pipeline metadata to add a
content_type field. The content_type metadata field allows commands such
as `to json` to set the metadata in the pipeline allowing the http
commands to use it when making requests.
This pull request also introduces the ability to directly stream http
requests from streaming pipelines.
One other small change is that Content-Type will always be set if it is
passed in to the http commands, either indirectly or throw the content
type flag. Previously it was not preserved with requests that were not
of type json or form data.
# User-Facing Changes
* `http post`, `http put`, `http patch`, `http delete` can be used as
part of a pipeline
* `to text`, `to json`, `from json` all set the content_type metadata
field and the http commands will utilize them when making requests.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
fixes#13245
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
In addition to addressing #13245, this PR also updated one of the doc
example to the `find` command to document that non-regex mode is case
insensitive, which may surprise some users.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Ben Yang <ben@ya.ng>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
fixed#11678
The sub-commands of from command (`from {csv, tsv, ssv}`) name columns
starting from index 0.
This behaviour is inconsistent with other commands such as `detect
columns`.
This PR makes the subcommands index 0-based.
# User-Facing Changes
The subcommands (`from {csv, tsv, ssv}`) return a table with the columns
starting at index 0 if no header data is passed.
```
~/Development/nushell> "foo bar baz" | from ssv -n -m 1
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo │ bar │ baz │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo,bar,baz" | from csv -n
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo │ bar │ baz │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo\tbar\tbaz" | from tsv -n
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo │ bar │ baz │
╰───┴─────────┴─────────┴─────────╯
```
# Tests + Formatting
When I ran tests, `commands::touch::change_file_mtime_to_reference`
failed with the following error.
The error also occurs in the master branch, so it's probably unrelated
to these changes.
(maybe a problem with my dev environment)
```
$ toolkit check pr
~~~~~~~~
failures:
---- commands::touch::change_file_mtime_to_reference stdout ----
=== stderr
thread 'commands::touch::change_file_mtime_to_reference' panicked at crates/nu-command/tests/commands/touch.rs:298:9:
assertion `left == right` failed
left: SystemTime { tv_sec: 1719149697, tv_nsec: 57576929 }
right: SystemTime { tv_sec: 1719149697, tv_nsec: 78219489 }
failures:
commands::touch::change_file_mtime_to_reference
test result: FAILED. 1533 passed; 1 failed; 32 ignored; 0 measured; 0 filtered out; finished in 10.87s
error: test failed, to rerun pass `-p nu-command --test main`
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
- ⚫ `toolkit test stdlib`
```
# After Submitting
nothing
# Description
@hustcer reported that slashes were disappearing from external args
since #13089:
```
$> ossutil ls oss://abc/b/c
Error: invalid cloud url: "oss:/abc/b/c", please make sure the url starts with: "oss://"
$> ossutil ls 'oss://abc/b/c'
Error: oss: service returned error: StatusCode=403, ErrorCode=UserDisable, ErrorMessage="UserDisable", RequestId=66791EDEFE87B73537120838, Ec=0003-00000801, Bucket=abc, Object=
```
I narrowed this down to the ndots handling, since that does path parsing
and path reconstruction in every case. I decided to change that so that
it only activates if the string contains at least `...`, since that
would be the minimum trigger for ndots, and also to not activate it if
the string contains `://`, since it's probably undesirable for a URL.
Kind of a hack, but I'm not really sure how else we decide whether
someone wants ndots or not.
# User-Facing Changes
- bare strings not containing ndots are not modified
- bare strings containing `://` are not modified
# Tests + Formatting
Added tests to prevent regression.
# Description
* As discussed in the comments in #11954, this suppresses the index
column on `cal` output. It does that by running `table -i false` on the
results by default.
* Added new `--as-table/-t` flag to revert to the old behavior and
output the calendar as structured data
* Updated existing tests to use `--as-table`
* Added new tests against the string output
* Updated `length` test which also used `cal`
* Added new example for `--as-table`, with result
# User-Facing Changes
## Breaking change
The *default* `cal` output has changed from a `list` to a `string`. To
obtain structured data from `cal`, use the new `--as-table/-t` flag.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
