mirror of
https://github.com/nushell/nushell.git
synced 2025-05-17 16:30:47 +02:00
43f9ec295f
31 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Solomon
|
e76586ede4
|
reset argument/redirection state after eval_call errors (#15400)
Closes #15395 # User-Facing Changes Certain errors no longer leave the argument stack in an unexpected state: ```diff let x: any = 1; try { $x | get path } catch { print caught } -$.path # extra `print` argument from the failed `get` call caught ``` # Description If `eval_call` fails in `check_input_types` or `gather_arguments`, the cleanup code is still executed. |
||
|
Solomon
|
dd56c813f9
|
preserve variable capture spans in blocks (#15334)
Closes #15160 # User-Facing Changes Certain "variable not found" errors no longer highlight the surrounding block. Before: ```nushell do { match foo { _ => $in } } Error: nu:🐚:variable_not_found × Variable not found ╭─[entry #1:1:1] 1 │ ╭─▶ do { 2 │ │ match foo { 3 │ │ _ => $in 4 │ │ } 5 │ ├─▶ } · ╰──── variable not found ``` After: ```nushell Error: nu:🐚:variable_not_found × Variable not found ╭─[entry #1:3:10] 2 │ match foo { 3 │ _ => $in · ─┬─ · ╰── variable not found ``` |
||
132ikl
|
78c93e5ae0
|
Run-time pipeline input type checking performance optimizations (#15192)
# Description Avoids cloning custom command signatures during run-time pipeline input type checking # User-Facing Changes N/A # Tests + Formatting N/A |
||
Ian Manske
|
62e56d3581
|
Rework operator type errors (#14429)
# Description This PR adds two new `ParseError` and `ShellError` cases for type errors relating to operators. - `OperatorUnsupportedType` is used when a type is not supported by an operator in any way, shape, or form. E.g., `+` does not support `bool`. - `OperatorIncompatibleTypes` is used when a operator is used with types it supports, but the combination of types provided cannot be used together. E.g., `filesize + duration` is not a valid combination. The other preexisting error cases related to operators have been removed and replaced with the new ones above. Namely: - `ShellError::OperatorMismatch` - `ShellError::UnsupportedOperator` - `ParseError::UnsupportedOperationLHS` - `ParseError::UnsupportedOperationRHS` - `ParseError::UnsupportedOperationTernary` # User-Facing Changes - `help operators` now lists the precedence of `not` as 55 instead of 0 (above the other boolean operators). Fixes #13675. - `math median` and `math mode` now ignore NaN values so that `[NaN NaN] | math median` and `[NaN NaN] | math mode` no longer trigger a type error. Instead, it's now an empty input error. Fixing this in earnest can be left for a future PR. - Comparisons with `nan` now return false instead of causing an error. E.g., `1 == nan` is now `false`. - All the operator type errors have been standardized and reworked. In particular, they can now have a help message, which is currently used for types errors relating to `++`. ```nu [1] ++ 2 ``` ``` Error: nu::parser::operator_unsupported_type × The '++' operator does not work on values of type 'int'. ╭─[entry #1:1:5] 1 │ [1] ++ 2 · ─┬ ┬ · │ ╰── int · ╰── does not support 'int' ╰──── help: if you meant to append a value to a list or a record to a table, use the `append` command or wrap the value in a list. For example: `$list ++ $value` should be `$list ++ [$value]` or `$list | append $value`. ``` |
||
Wind
|
2f18b9c856
|
Enable nushell error with backtrace (#14945)
# Description After this pr, nushell is able to raise errors with a backtrace, which should make users easier to debug. To enable the feature, users need to set env variable via `$env.NU_BACKTRACE = 1`. But yeah it might not work perfectly, there are some corner cases which might not be handled. I think it should close #13379 in another way. ### About the change The implementation mostly contained with 2 parts: 1. introduce a new `ChainedError` struct as well as a new `ShellError::ChainedError` variant. If `eval_instruction` returned an error, it converts the error to `ShellError::ChainedError`. `ChainedError` struct is responsable to display errors properly. It needs to handle the following 2 cases: - if we run a function which runs `error make` internally, it needs to display the error itself along with caller span. - if we run a `error make` directly, or some commands directly returns an error, we just want nushell raise an error about `error make`. 2. Attach caller spans to `ListStream` and `ByteStream`, because they are lazy streams, and *only* contains the span that runs it directly(like `^false`, for example), so nushell needs to add all caller spans to the stream. For example: in `def a [] { ^false }; def b [] { a; 33 }; b`, when we run `b`, which runs `a`, which runs `^false`, the `ByteStream` only contains the span of `^false`, we need to make it contains the span of `a`, so nushell is able to get all spans if something bad happened. This behavior is happened after running `Instruction::Call`, if it returns a `ByteStream` and `ListStream`, it will call `push_caller_span` method to attach call spans. # User-Facing Changes It's better to demostrate how it works by examples, given the following definition: ```nushell > $env.NU_BACKTRACE = 1 > def a [x] { if $x == 3 { error make {msg: 'a custom error'}}} > def a_2 [x] { if $x == 3 { ^false } else { $x } } > def a_3 [x] { if $x == 3 { [1 2 3] | each {error make {msg: 'a custom error inside list stream'} } } } > def b [--list-stream --external] { if $external == true { # error with non-zero exit code, which is generated from external command. a_2 1; a_2 3; a_2 2 } else if $list_stream == true { # error generated by list-stream a_3 1; a_3 3; a_3 2 } else { # error generated by command directly a 1; a 2; a 3 } } ``` Run `b` directly shows the following error: <details> ```nushell Error: chained_error × oops ╭─[entry #27:1:1] 1 │ b · ┬ · ╰── error happened when running this ╰──── Error: chained_error × oops ╭─[entry #26:10:19] 9 │ # error generated by command directly 10 │ a 1; a 2; a 3 · ┬ · ╰── error happened when running this 11 │ } ╰──── Error: × a custom error ╭─[entry #6:1:26] 1 │ def a [x] { if $x == 3 { error make {msg: 'a custom error'}}} · ─────┬──── · ╰── originates from here ╰──── ``` </details> Run `b --list-stream` shows the following error <details> ```nushell Error: chained_error × oops ╭─[entry #28:1:1] 1 │ b --list-stream · ┬ · ╰── error happened when running this ╰──── Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #26:7:16] 6 │ # error generated by list-stream 7 │ a_3 1; a_3 3; a_3 2 · ─┬─ · ╰── source value 8 │ } else { ╰──── Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #23:1:29] 1 │ def a_3 [x] { if $x == 3 { [1 2 3] | each {error make {msg: 'a custom error inside list stream'} } } } · ┬ · ╰── source value ╰──── Error: × a custom error inside list stream ╭─[entry #23:1:44] 1 │ def a_3 [x] { if $x == 3 { [1 2 3] | each {error make {msg: 'a custom error inside list stream'} } } } · ─────┬──── · ╰── originates from here ╰──── ``` </details> Run `b --external` shows the following error: <details> ```nushell Error: chained_error × oops ╭─[entry #29:1:1] 1 │ b --external · ┬ · ╰── error happened when running this ╰──── Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #26:4:16] 3 │ # error with non-zero exit code, which is generated from external command. 4 │ a_2 1; a_2 3; a_2 2 · ─┬─ · ╰── source value 5 │ } else if $list_stream == true { ╰──── Error: nu:🐚:non_zero_exit_code × External command had a non-zero exit code ╭─[entry #7:1:29] 1 │ def a_2 [x] { if $x == 3 { ^false } else { $x } } · ──┬── · ╰── exited with code 1 ╰──── ``` </details> It also added a message to guide the usage of NU_BACKTRACE, see the last line in the following example: ```shell ls asdfasd Error: nu:🐚:io::not_found × I/O error ╰─▶ × Entity not found ╭─[entry #17:1:4] 1 │ ls asdfasd · ───┬─── · ╰── Entity not found ╰──── help: The error occurred at '/home/windsoilder/projects/nushell/asdfasd' set the `NU_BACKTRACE=1` environment variable to display a backtrace. ``` # Tests + Formatting Added some tests for the behavior. # After Submitting |
||
|
132ikl
|
13d5a15f75
|
Run-time pipeline input typechecking tweaks (#14922)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR makes two changes related to [run-time pipeline input type checking](https://github.com/nushell/nushell/pull/14741): 1. The check which bypasses type checking for commands with only `Type::Nothing` input types has been expanded to work with commands with multiple `Type::Nothing` inputs for different outputs. For example, `ast` has three input/output type pairs, but all of the inputs are `Type::Nothing`: ``` ╭───┬─────────┬────────╮ │ # │ input │ output │ ├───┼─────────┼────────┤ │ 0 │ nothing │ table │ │ 1 │ nothing │ record │ │ 2 │ nothing │ string │ ╰───┴─────────┴────────╯ ``` Before this PR, passing a value (which would otherwise be ignored) to `ast` caused a run-time type error: ``` Error: nu:🐚:only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:6] 1 │ echo 123 | ast -j -f "hi" · ─┬─ ─┬─ · │ ╰── only nothing, nothing, and nothing input data is supported · ╰── input type: int ╰──── ``` After this PR, no error is raised. This doesn't really matter for `ast` (the only other built-in command with a similar input/output type signature is `cal`), but it's more logically consistent. 2. Bypasses input type-checking (parse-time ***and*** run-time) for some (not all, see below) commands which have both a `Type::Nothing` input and some other non-nothing `Type` input. This is accomplished by adding a `Type::Any` input with the same output as the corresponding `Type::Nothing` input/output pair. This is necessary because some commands are intended to operate on an argument with empty pipeline input, or operate on an empty pipeline input with no argument. This causes issues when a value is implicitly passed to one of these commands. I [discovered this issue](https://discord.com/channels/601130461678272522/615962413203718156/1329945784346611712) when working with an example where the `open` command is used in `sort-by` closure: ```nushell ls | sort-by { open -r $in.name | lines | length } ``` Before this PR (but after the run-time input type checking PR), this error is raised: ``` Error: nu:🐚:only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:1] 1 │ ls | sort-by { open -r $in.name | lines | length } · ─┬ ──┬─ · │ ╰── only nothing and string input data is supported · ╰── input type: record<name: string, type: string, size: filesize, modified: date> ╰──── ``` While this error is technically correct, we don't actually want to return an error here since `open` ignores its pipeline input when an argument is passed. This would be a parse-time error as well if the parser was able to infer that the closure input type was a record, but our type inference isn't that robust currently, so this technically incorrect form snuck by type checking until #14741. However, there are some commands with the same kind of type signature where this behavior is actually desirable. This means we can't just bypass type-checking for any command with a `Type::Nothing` input. These commands operate on true `null` values, rather than ignoring their input. For example, `length` returns `0` when passed a `null` value. It's correct, and even desirable, to throw a run-time error when `length` is passed an unexpected type. For example, a string, which should instead be measured with `str length`: ```nushell ["hello" "world"] | sort-by { length } # => Error: nu:🐚:only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry #32:1:10] # => 1 │ ["hello" "world"] | sort-by { length } # => · ───┬─── ───┬── # => · │ ╰── only list<any>, binary, and nothing input data is supported # => · ╰── input type: string # => ╰──── ``` We need a more robust way for commands to express how they handle the `Type::Nothing` input case. I think a possible solution here is to allow commands to express that they operate on `PipelineData::Empty`, rather than `Value::Nothing`. Then, a command like `open` could have an empty pipeline input type rather than a `Type::Nothing`, and the parse-time and run-time pipeline input type checks know that `open` will safely ignore an incorrectly typed input. That being said, we have a release coming up and the above solution might take a while to implement, so while unfortunate, bypassing input type-checking for these problematic commands serves as a workaround to avoid breaking changes in the release until a more robust solution is implemented. This PR bypasses input type-checking for the following commands: * `load-env`: can take record of envvars as input or argument * `nu-check`: checks input string or filename argument * `open`: can take filename as input or argument * `polars when`: can be used with input, or can be chained with another `polars when` * `stor insert`: data record can be passed as input or argument * `stor update`: data record can be passed as input or argument * `format date`: `--list` ignores input value * `into datetime`: `--list` ignores input value (also added a `Type::Nothing` input which was missing from this command) These commands have a similar input/output signature to the above commands, but are working as intended: * `cd`: The input/output signature was actually incorrect, `cd` always ignores its input. I fixed this in this PR. * `generate` * `get` * `history import` * `interleave` * `into bool` * `length` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> As a temporary workaround, pipeline input type-checking for the following commands has been bypassed to avoid undesirable run-time input type checking errors which were previously not caught at parse-time: * `open` * `load-env` * `format date` * `into datetime` * `nu-check` * `stor insert` * `stor update` * `polars when` # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> CI became green in the time it took me to type the description 😄 # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> N/A |
||
|
132ikl
|
948965d42f
|
Immediately return error if detected as pipeline input or positional argument (#14874)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR returns error values while checking pipeline input types and positional argument types. This should help return non-nested errors earlier and prevent confusing errors. The positional argument change is directly related to an example given on Discord. Before this PR, this is the error shown: ``` Error: nu:🐚:cant_convert × Can't convert to record. ╭─[/home/rose/tmp/script.nu:23:5] 22 │ let entry = $in 23 │ ╭─▶ { 24 │ │ name: $entry, 25 │ │ details: { 26 │ │ context: $context 27 │ │ } 28 │ ├─▶ } · ╰──── can't convert error to record 29 │ } ╰──── ``` After this PR, this is the error shown: ``` Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[/home/rose/tmp/script.nu:23:5] 22 │ let entry = $in 23 │ ╭─▶ { 24 │ │ name: $entry, 25 │ │ details: { 26 │ │ context: $context 27 │ │ } 28 │ ├─▶ } · ╰──── source value 29 │ } ╰──── Error: nu:🐚:type_mismatch × Type mismatch. ╭─[/home/rose/tmp/much.nu:3:38] 2 │ $in | each { |elem| 3 │ print $elem.details.context.yaml.0 · ┬ · ╰── Can't access record values with a row index. Try specifying a column name instead 4 │ } | each { |elem| ╰──── ``` I'm not certain if the pipeline input error check actually can ever be triggered, but it seems to be a good defensive error handling strategy regardless. My addition of the `Value::Error` case in the first place would suggest it can be, but after looking at it more closely the error that caused me to add the case in the first place was actually unrelated to input typechecking. Additionally, this PR does not affect the handling of nested errors, so something like: ```nushell try { ... } catch {|e| $e | reject raw | to nuon } ``` works the same before and after this PR. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Errors values detected as arguments to commands or as pipeline input to commands are immediately thrown, rather than passed to the command. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> N/A |
||
Piepmatz
|
66bc0542e0
|
Refactor I/O Errors (#14927)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
As mentioned in #10698, we have too many `ShellError` variants, with
some even overlapping in meaning. This PR simplifies and improves I/O
error handling by restructuring `ShellError` related to I/O issues.
Previously, `ShellError::IOError` only contained a message string,
making it convenient but overly generic. It was widely used without
providing spans (#4323).
This PR introduces a new `ShellError::Io` variant that consolidates
multiple I/O-related errors (except for `ShellError::NetworkFailure`,
which remains distinct for now). The new `ShellError::Io` variant
replaces the following:
- `FileNotFound`
- `FileNotFoundCustom`
- `IOInterrupted`
- `IOError`
- `IOErrorSpanned`
- `NotADirectory`
- `DirectoryNotFound`
- `MoveNotPossible`
- `CreateNotPossible`
- `ChangeAccessTimeNotPossible`
- `ChangeModifiedTimeNotPossible`
- `RemoveNotPossible`
- `ReadingFile`
## The `IoError`
`IoError` includes the following fields:
1. **`kind`**: Extends `std::io::ErrorKind` to specify the type of I/O
error without needing new `ShellError` variants. This aligns with the
approach used in `std::io::Error`. This adds a second dimension to error
reporting by combining the `kind` field with `ShellError` variants,
making it easier to describe errors in more detail. As proposed by
@kubouch in [#design-discussion on
Discord](https://discord.com/channels/601130461678272522/615329862395101194/1323699197165178930),
this helps reduce the number of `ShellError` variants. In the error
report, the `kind` field is displayed as the "source" of the error,
e.g., "I/O error," followed by the specific kind of I/O error.
2. **`span`**: A non-optional field to encourage providing spans for
better error reporting (#4323).
3. **`path`**: Optional `PathBuf` to give context about the file or
directory involved in the error (#7695). If provided, it’s shown as a
help entry in error reports.
4. **`additional_context`**: Allows adding custom messages when the
span, kind, and path are insufficient. This is rendered in the error
report at the labeled span.
5. **`location`**: Sometimes, I/O errors occur in the engine itself and
are not caused directly by user input. In such cases, if we don’t have a
span and must set it to `Span::unknown()`, we need another way to
reference the error. For this, the `location` field uses the new
`Location` struct, which records the Rust file and line number where the
error occurred. This ensures that we at least know the Rust code
location that failed, helping with debugging. To make this work, a new
`location!` macro was added, which retrieves `file!`, `line!`, and
`column!` values accurately. If `Location::new` is used directly, it
issues a warning to remind developers to use the macro instead, ensuring
consistent and correct usage.
### Constructor Behavior
`IoError` provides five constructor methods:
- `new` and `new_with_additional_context`: Used for errors caused by
user input and require a valid (non-unknown) span to ensure precise
error reporting.
- `new_internal` and `new_internal_with_path`: Used for internal errors
where a span is not available. These methods require additional context
and the `Location` struct to pinpoint the source of the error in the
engine code.
- `factory`: Returns a closure that maps an `std::io::Error` to an
`IoError`. This is useful for handling multiple I/O errors that share
the same span and path, streamlining error handling in such cases.
## New Report Look
This is simulation how the I/O errors look like (the `open crates` is
simulated to show how internal errors are referenced now):
## `Span::test_data()`
To enable better testing, `Span::test_data()` now returns a value
distinct from `Span::unknown()`. Both `Span::test_data()` and
`Span::unknown()` refer to invalid source code, but having a separate
value for test data helps identify issues during testing while keeping
spans unique.
## Cursed Sneaky Error Transfers
I removed the conversions between `std::io::Error` and `ShellError` as
they often removed important information and were used too broadly to
handle I/O errors. This also removed the problematic implementation
found here:
|
||
Bahex
|
089c5221cc
|
Add new operators has and not-has (#14841)
# Description
This PR add 2 new operators, `has` and `not-has`. They are basically
`in` and `not-in` with the order of operands swapped.
Motivation for this was the awkward way of searching for rows that
contain an item using `where`
```nushell
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]]
| where ("e" in $it.children)
```
vs
```nushell
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]]
| where children has "e"
```
# User-Facing Changes
Added `has` and `not-has` operators, mirroring `in` and `not-in`.
# Tests + Formatting
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
|
||
|
132ikl
|
214714e0ab
|
Add run-time type checking for command pipeline input (#14741)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds type checking of all command input types at run-time. Generally, these errors should be caught by the parser, but sometimes we can't know the type of a value at parse-time. The simplest example is using the `echo` command, which has an output type of `any`, so prefixing a literal with `echo` will bypass parse-time type checking. Before this PR, each command has to individually check its input types. This can result in scenarios where the input/output types don't match the actual command behavior. This can cause valid usage with an non-`any` type to become a parse-time error if a command is missing that type in its pipeline input/output (`drop nth` and `history import` do this before this PR). Alternatively, a command may not list a type in its input/output types, but doesn't actually reject that type in its code, which can have unintended side effects (`get` does this on an empty pipeline input, and `sort` used to before #13154). After this PR, the type of the pipeline input is checked to ensure it matches one of the input types listed in the proceeding command's input/output types. While each of the issues in the "before this PR" section could be addressed with each command individually, this PR solves this issue for _all_ commands. **This will likely cause some breakage**, as some commands have incorrect input/output types, and should be adjusted. Also, some scripts may have erroneous usage of commands. In writing this PR, I discovered that `toolkit.nu` was passing `null` values to `str join`, which doesn't accept nothing types (if folks think it should, we can adjust it in this PR or in a different PR). I found some issues in the standard library and its tests. I also found that carapace's vendor script had an incorrect chaining of `get -i`: ```nushell let expanded_alias = (scope aliases | where name == $spans.0 | get -i 0 | get -i expansion) ``` Before this PR, if the `get -i 0` ever actually did evaluate to `null`, the second `get` invocation would error since `get` doesn't operate on `null` values. After this PR, this is immediately a run-time error, alerting the user to the problematic code. As a side note, we'll need to PR this fix (`get -i 0 | get -i expansion` -> `get -i 0.expansion`) to carapace. A notable exception to the type checking is commands with input type of `nothing -> <type>`. In this case, any input type is allowed. This allows piping values into the command without an error being thrown. For example, `123 | echo $in` would be an error without this exception. Additionally, custom types bypass type checking (I believe this also happens during parsing, but not certain) I added a `is_subtype` method to `Value` and `PipelineData`. It functions slightly differently than `get_type().is_subtype()`, as noted in the doccomments. Notably, it respects structural typing of lists and tables. For example, the type of a value `[{a: 123} {a: 456, b: 789}]` is a subtype of `table<a: int>`, whereas the type returned by `Value::get_type` is a `list<any>`. Similarly, `PipelineData` has some special handling for `ListStream`s and `ByteStream`s. The latter was needed for this PR to work properly with external commands. Here's some examples. Before: ```nu 1..2 | drop nth 1 Error: nu::parser::input_type_mismatch × Command does not support range input. ╭─[entry #9:1:8] 1 │ 1..2 | drop nth 1 · ────┬─── · ╰── command doesn't support range input ╰──── echo 1..2 | drop nth 1 # => ╭───┬───╮ # => │ 0 │ 1 │ # => ╰───┴───╯ ``` After this PR, I've adjusted `drop nth`'s input/output types to accept range input. Before this PR, zip accepted any value despite not being listed in its input/output types. This caused different behavior depending on if you triggered a parse error or not: ```nushell 1 | zip [2] # => Error: nu::parser::input_type_mismatch # => # => × Command does not support int input. # => ╭─[entry #3:1:5] # => 1 │ 1 | zip [2] # => · ─┬─ # => · ╰── command doesn't support int input # => ╰──── echo 1 | zip [2] # => ╭───┬───────────╮ # => │ 0 │ ╭───┬───╮ │ # => │ │ │ 0 │ 1 │ │ # => │ │ │ 1 │ 2 │ │ # => │ │ ╰───┴───╯ │ # => ╰───┴───────────╯ ``` After this PR, it works the same in both cases. For cases like this, if we do decide we want `zip` or other commands to accept any input value, then we should explicitly add that to the input types. ```nushell 1 | zip [2] # => Error: nu::parser::input_type_mismatch # => # => × Command does not support int input. # => ╭─[entry #3:1:5] # => 1 │ 1 | zip [2] # => · ─┬─ # => · ╰── command doesn't support int input # => ╰──── echo 1 | zip [2] # => Error: nu:🐚:only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry #14:2:6] # => 2 │ echo 1 | zip [2] # => · ┬ ─┬─ # => · │ ╰── only list<any> and range input data is supported # => · ╰── input type: int # => ╰──── ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> **Breaking change**: The type of a command's input is now checked against the input/output types of that command at run-time. While these errors should mostly be caught at parse-time, in cases where they can't be detected at parse-time they will be caught at run-time instead. This applies to both internal commands and custom commands. Example function and corresponding parse-time error (same before and after PR): ```nushell def foo []: int -> nothing { print $"my cool int is ($in)" } 1 | foo # => my cool int is 1 "evil string" | foo # => Error: nu::parser::input_type_mismatch # => # => × Command does not support string input. # => ╭─[entry #16:1:17] # => 1 │ "evil string" | foo # => · ─┬─ # => · ╰── command doesn't support string input # => ╰──── # => ``` Before: ```nu echo "evil string" | foo # => my cool int is evil string ``` After: ```nu echo "evil string" | foo # => Error: nu:🐚:only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry #17:1:6] # => 1 │ echo "evil string" | foo # => · ──────┬────── ─┬─ # => · │ ╰── only int input data is supported # => · ╰── input type: string # => ╰──── ``` Known affected internal commands which erroneously accepted any type: * `str join` * `zip` * `reduce` # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> * Play whack-a-mole with the commands and scripts this will inevitably break |
||
|
Bahex
|
1b01598840
|
Run ENV_CONVERSIONS whenever it's modified (#14591)
- this PR should close #14514 # Description Makes updates to `$env.ENV_CONVERSIONS` take effect immediately. # User-Facing Changes No breaking change, `$env.ENV_CONVERSIONS` can be set and its effect used in the same file. # Tests + Formatting - 🟢 toolkit fmt - 🟢 toolkit clippy - 🟢 toolkit test - 🟢 toolkit test stdlib # After Submitting N/A |
||
Darren Schroeder
|
e2c4ff8180
|
Revert "Feature: PWD-per-drive to facilitate working on multiple drives at Windows" (#14598)
Reverts nushell/nushell#14411 |
||
PegasusPlusUS
|
c8b5909ee8
|
Feature: PWD-per-drive to facilitate working on multiple drives at Windows (#14411)
This PR implements PWD-per-drive as described in discussion #14355 # Description On Windows, CMD or PowerShell assigns each drive its own current directory. For example, if you are in 'C:\Windows', switch to 'D:', and navigate to 'D:\Game', you can return to 'C:\Windows' by simply typing 'C:'. This PR enables Nushell on Windows to have the same capability, allowing each drive to maintain its own PWD (Present Working Directory). # User-Facing Changes Currently, 'cd' or 'ls' only accept absolute paths if the path starts with 'C:' or another drive letter. With PWD-per-drive, users can use 'cd' (or auto cd) and 'ls' in the same way as 'cd' and 'dir' in PowerShell, or similarly to 'cd' and 'dir' in CMD (noting that cd in CMD has slightly different behavior, 'cd' for another drive only changes current directory of that drive, but does not switch there). Interaction example on switching between drives: ```Nushell ~>D: D:\>cd Test D:\Test\>C: ~>D: D:\Test\>C: ~>cd D:.. D:\>C:x/../y/../z/.. ~>cd D:Test\Test D:\Test\Test>C: ~>D:... D:\> ``` Interaction example on auto-completion at cmd line: ```Nushell ~>cd D:\test[Enter] D:\test>~[Enter] ~>D:[TAB] ~>D:\test[Enter] D:\test>c:.c[TAB] c:\users\nushell\.cargo\ c:\users\nushell\.config\ ``` Interaction example on pass PWD-per-drive to child process: (Note CMD will use it, but PowerShell will ignore it though it still prepares such info for child process) ```Nushell ~>cd D:\Test D:\Test>cd E:\Test E:\Test\>~ ~>CMD Microsoft Windows [Version 10.0.22631.4460] (c) Microsoft Corporation. All rights reserved. C:\Users\Nushell>d: D:\Test>e: E:\Test> ``` # Brief Change Description 1.Added 'crates/nu-path/src/pwd_per_drive.rs' to implement a 26-slot array mapping drive letters to PWDs. Test cases are included in the same file, along with a doctest for the usage of PWD-per-drive. 2. Modified 'crates/nu-path/src/lib.rs' to declare module of pwd_per_drive and export struct for PWD-per-drive. 3. Modified 'crates/nu-protocol/src/engine/stack.rs' to sync PWD when set_cwd() is called. Add PWD-per-drive map as member. Clone between parent and child. Stub/proxy for nu_path::expand_path_with() to facilitate filesystem commands using PWD-per-drive. 4. Modified 'crates/nu-cli/src/repl.rs' auto_cd uses PWD-per-drive to expand path. 5. Modified 'crates/nu-cli/src/completions/completion_common.rs' to expand relative path when press [TAB] at command line. 6. Modified 'crates/nu-engine/src/env.rs' to collect PWD-per-drive info as env vars for child process as CMD or PowerShell do, this can let child process inherit PWD-per-drive info. 7. Modified 'crates/nu-engine/src/eval.rs', caller clone callee's PWD-per-drive info, supporting 'def --env' 8. Modified 'crates/nu-engine/src/eval_ir.rs', 'def --env' support. Remove duplicated fn redirect_env() 9. Modified 'src/run.rs', to init PWD-per-drive when startup. filesystem commands that modified: 1. Modified 'crates/nu-command/src/filesystem/cd.rs', 1 line change to use stackscoped PWD-per-drive. Other commands, commit pending.... Local test def --env OK: ```nushell E:\study\nushell> def --env env_cd_demo [] { ::: cd ~ ::: cd D:\Project ::: cd E:Crates ::: } E:\study\nushell> E:\study\nushell> def cd_no_demo [] { ::: cd ~ ::: cd D:\Project ::: cd E:Crates ::: } E:\study\nushell> cd_no_demo E:\study\nushell> C: C:\>D: D:\>E: E:\study\nushell>env_cd_demo E:\study\nushell\crates> C: ~>D: D:\Project>E: E:\study\nushell\crates> ``` # Tests + Formatting - `cargo fmt --all -- --check` passed. - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` passed. - `cargo test --workspace` passed on Windows developer mode and Ubuntu. - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` passed. - nushell: ``` > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` passed --------- Co-authored-by: pegasus.cadence@gmail.com <pegasus.cadence@gmail.com> |
||
|
Piepmatz
|
3d5f853b03
|
Start to Add WASM Support Again (#14418)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> The [nushell/demo](https://github.com/nushell/demo) project successfully demonstrated running Nushell in the browser using WASM. However, the current version of Nushell cannot be easily built for the `wasm32-unknown-unknown` target, the default for `wasm-bindgen`. This PR introduces initial support for the `wasm32-unknown-unknown` target by disabling OS-dependent features such as filesystem access, IO, and platform/system-specific functionality. This separation is achieved using a new `os` feature in the following crates: - `nu-cmd-lang` - `nu-command` - `nu-engine` - `nu-protocol` The `os` feature includes all functionality that interacts with an operating system. It is enabled by default, but can be disabled using `--no-default-features`. All crates that depend on these core crates now use `--no-default-features` to allow compilation for WASM. To demonstrate compatibility, the following script builds all crates expected to work with WASM. Direct user interaction, running external commands, working with plugins, and features requiring `openssl` are out of scope for now due to their complexity or reliance on C libraries, which are difficult to compile and link in a WASM environment. ```nushell [ # compatible crates "nu-cmd-base", "nu-cmd-extra", "nu-cmd-lang", "nu-color-config", "nu-command", "nu-derive-value", "nu-engine", "nu-glob", "nu-json", "nu-parser", "nu-path", "nu-pretty-hex", "nu-protocol", "nu-std", "nu-system", "nu-table", "nu-term-grid", "nu-utils", "nuon" ] | each {cargo build -p $in --target wasm32-unknown-unknown --no-default-features} ``` ## Caveats This PR has a few caveats: 1. **`miette` and `terminal-size` Dependency Issue** `miette` depends on `terminal-size`, which uses `rustix` when the target is not Windows. However, `rustix` requires `std::os::unix`, which is unavailable in WASM. To address this, I opened a [PR](https://github.com/eminence/terminal-size/pull/68) for `terminal-size` to conditionally compile `rustix` only when the target is Unix. For now, the `Cargo.toml` includes patches to: - Use my forked version of `terminal-size`. - ~~Use an unreleased version of `miette` that depends on `terminal-size@0.4`.~~ These patches are temporary and can be removed once the upstream changes are merged and released. 2. **Test Output Adjustments** Due to the slight bump in the `miette` version, one test required adjustments to accommodate minor formatting changes in the error output, such as shifted newlines. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> This shouldn't break anything but allows using some crates for targeting `wasm32-unknown-unknown` to revive the demo page eventually. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` I did not add any extra tests, I just checked that compiling works, also when using the host target but unselecting the `os` feature. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> ~~Breaking the wasm support can be easily done by adding some `use`s or by adding a new dependency, we should definitely add some CI that also at least builds against wasm to make sure that building for it keep working.~~ I added a job to build wasm. --------- Co-authored-by: Ian Manske <ian.manske@pm.me> |
||
|
132ikl
|
e1f74a6d57
|
Add label rendering to try/catch rendered errors (#14477)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Before this PR, you can access rendered error values that are raised in a `try/catch` block by accessing the `rendered` element of the catch error value: ``` $ try { ls nonexist.txt } catch {|e| print "my cool error:" $e.rendered } my cool error: nu:🐚:directory_not_found × Directory not found help: /home/rose/nonexist.txt does not exist ``` However, the rendered errors don't include the labels present in the real rendered error, which would look like this: ``` $ ls nonexist.txt Error: nu:🐚:directory_not_found × Directory not found ╭─[entry #46:1:4] 1 │ ls nonexist.txt · ──────┬───── · ╰── directory not found ╰──── help: /home/rose/nonexist.txt does not exist ``` After this PR, the rendered error includes the labels: ``` $ try { ls nonexist.txt } catch {|e| print "my cool error:" $e.rendered } my cool error: Error: nu:🐚:directory_not_found × Directory not found ╭─[entry #4:1:10] 1 │ try { ls nonexist.txt } catch {|e| print "my cool error:" $e.rendered } · ──────┬───── · ╰── directory not found ╰──── help: /home/rose/nonexist.txt does not exist ``` This change is accomplished by using the standard error formatting code to render an error. This respects the error theme as before without any extra scaffolding, but it means that e.g., the terminal size is also respected. I think this is fine because the way the error is rendered already changed based on config, and I think that a "rendered" error should give back _exactly_ what would be shown to the user anyway. @fdncred, let me know if you have any concerns with the way this is handled since you were the one who implemented this feature in the first place. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> The `rendered` element of the `try`/`catch` error record now includes labels in the error output. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> N/A |
||
|
Ian Manske
|
4d3283e235
|
Change append operator to concatenation operator (#14344)
# Description The "append" operator currently serves as both the append operator and the concatenation operator. This dual role creates ambiguity when operating on nested lists. ```nu [1 2] ++ 3 # appends a value to a list [1 2 3] [1 2] ++ [3 4] # concatenates two lists [1 2 3 4] [[1 2] [3 4]] ++ [5 6] # does this give [[1 2] [3 4] [5 6]] # or [[1 2] [3 4] 5 6] ``` Another problem is that `++=` can change the type of a variable: ```nu mut str = 'hello ' $str ++= ['world'] ($str | describe) == list<string> ``` Note that appending is only relevant for lists, but concatenation is relevant for lists, strings, and binary values. Additionally, appending can be expressed in terms of concatenation (see example below). So, this PR changes the `++` operator to only perform concatenation. # User-Facing Changes Using the `++` operator with a list and a non-list value will now be a compile time or runtime error. ```nu mut list = [] $list ++= 1 # error ``` Instead, concatenate a list with one element: ```nu $list ++= [1] ``` Or use `append`: ```nu $list = $list | append 1 ``` # After Submitting Update book and docs. --------- Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com> |
||
|
Darren Schroeder
|
8c8f795e9e |
add rendered and json error messages in try/catch (#14082)
# Description This PR adds a couple more options for dealing with try/catch errors. It adds a `json` version of the error and a `rendered` version of the error. It also respects the error_style configuration point.  # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
||
|
Ian Manske
|
28b6db115a
|
Revert PRs for 0.99.1 patch (#14119)
# Description Temporarily reverts PRs merged after the 0.99.1 bump. |
||
|
Darren Schroeder
|
e735bd475f
|
add rendered and json error messages in try/catch (#14082)
# Description This PR adds a couple more options for dealing with try/catch errors. It adds a `json` version of the error and a `rendered` version of the error. It also respects the error_style configuration point.  # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
||
|
Ian Manske
|
de08b68ba8
|
Fix try printing when it is not the last pipeline element (#13992)
# Description Fixes #13991. This was done by more clearly separating the case when a pipeline is drained vs when it is being written (to a file). I also added an `OutDest::Print` case which might not be strictly necessary, but is a helpful addition. # User-Facing Changes Bug fix. # Tests + Formatting Added a test. # After Submitting There are still a few redirection bugs that I found, but they require larger code changes, so I'll leave them until after the release. |
||
|
Piepmatz
|
b2d0d9cf13
|
Make SpanId and RegId also use new ID struct (#13963)
# Description In the PR #13832 I used some newtypes for the old IDs. `SpanId` and `RegId` already used newtypes, to streamline the code, I made them into the same style as the other marker-based IDs. Since `RegId` should be a bit smaller (it uses a `u32` instead of `usize`) according to @devyn, I made the `Id` type generic with `usize` as the default inner value. The question still stands how `Display` should be implemented if even. # User-Facing Changes Users of the internal values of `RegId` or `SpanId` have breaking changes but who outside nushell itself even uses these? # After Submitting The IDs will be streamlined and all type-safe. |
||
|
Ian Manske
|
03ee54a4df
|
Fix try not working with let, etc. (#13885)
# Description Partialy addresses #13868. `try` does not catch non-zero exit code errors from the last command in a pipeline if the result is assigned to a variable using `let` (or `mut`). This was fixed by adding a new `OutDest::Value` case. This is used when the pipeline is in a "value" position. I.e., it will be collected into a value. This ended up replacing most of the usages of `OutDest::Capture`. So, this PR also renames `OutDest::Capture` to `OutDest::PipeSeparate` to better fit the few remaining use cases for it. # User-Facing Changes Bug fix. # Tests + Formatting Added two tests. |
||
Devyn Cairns
|
a59477205d
|
Fix try: Add set_last_error() to prepare_error_handler() for IR eval (#13838)
# Description Fixes a bug with `set_last_error()` introduced by @IanManske not being called during the jump to an error handler in IR eval. Without this, `$env.LAST_EXIT_CODE` wasn't getting set in the `catch` block for an external. # Tests + Formatting Added a `tests/eval` test to cover this in both IR and non-IR eval |
||
|
Devyn Cairns
|
6b5906613c
|
Fix remaining mismatch for env handling in IR (#13796)
# Description This fixes a couple of remaining differences between the IR evaluator's handling of env vars and the AST evaluator's handling of env vars. Blocker for #13718 (this is why those tests failed) # User-Facing Changes 1. Handles checking overlays for hidden env vars properly, when getting an env var from IR instruction 2. Updates config properly when doing `redirect_env()` (these probably shouldn't be separate functions anyway, though, they're basically the same. I did this because I intended to remove one, but now it's just like that) # Tests + Formatting The `nu_repl` testbin now handles `NU_USE_IR` properly, so these tests now work as expected. # After Submitting - [ ] check in on #13718 again |
||
|
Ian Manske
|
493850b1bf
|
Fix IR for try (#13811)
# Description Fixes a bug in the IR for `try` to match that of the regular evaluator (continuing from #13515): ```nushell # without IR: try { ^false } catch { 'caught' } # == 'caught' # with IR: try { ^false } catch { 'caught' } # error, non-zero exit code ``` In this PR, both now evaluate to `caught`. For the implementation, I had to add another instruction, and feel free to suggest better alternatives. In the future, it might be possible to get rid of this extra instruction. # User-Facing Changes Bug fix, `try { ^false } catch { 'caught' }` now works in IR. |
||
|
Ian Manske
|
3d008e2c4e
|
Error on non-zero exit statuses (#13515)
# Description This PR makes it so that non-zero exit codes and termination by signal are treated as a normal `ShellError`. Currently, these are silent errors. That is, if an external command fails, then it's code block is aborted, but the parent block can sometimes continue execution. E.g., see #8569 and this example: ```nushell [1 2] | each { ^false } ``` Before this would give: ``` ╭───┬──╮ │ 0 │ │ │ 1 │ │ ╰───┴──╯ ``` Now, this shows an error: ``` Error: nu:🐚:eval_block_with_input × Eval block failed with pipeline input ╭─[entry #1:1:2] 1 │ [1 2] | each { ^false } · ┬ · ╰── source value ╰──── Error: nu:🐚:non_zero_exit_code × External command had a non-zero exit code ╭─[entry #1:1:17] 1 │ [1 2] | each { ^false } · ──┬── · ╰── exited with code 1 ╰──── ``` This PR fixes #12874, fixes #5960, fixes #10856, and fixes #5347. This PR also partially addresses #10633 and #10624 (only the last command of a pipeline is currently checked). It looks like #8569 is already fixed, but this PR will make sure it is definitely fixed (fixes #8569). # User-Facing Changes - Non-zero exit codes and termination by signal now cause an error to be thrown. - The error record value passed to a `catch` block may now have an `exit_code` column containing the integer exit code if the error was due to an external command. - Adds new config values, `display_errors.exit_code` and `display_errors.termination_signal`, which determine whether an error message should be printed in the respective error cases. For non-interactive sessions, these are set to `true`, and for interactive sessions `display_errors.exit_code` is false (via the default config). # Tests Added a few tests. # After Submitting - Update docs and book. - Future work: - Error if other external commands besides the last in a pipeline exit with a non-zero exit code. Then, deprecate `do -c` since this will be the default behavior everywhere. - Add a better mechanism for exit codes and deprecate `$env.LAST_EXIT_CODE` (it's buggy). |
||
|
Devyn Cairns
|
aa7d7d0cc3
|
Overhaul $in expressions (#13357)
# Description
This grew quite a bit beyond its original scope, but I've tried to make
`$in` a bit more consistent and easier to work with.
Instead of the parser generating calls to `collect` and creating
closures, this adds `Expr::Collect` which just evaluates in the same
scope and doesn't require any closure.
When `$in` is detected in an expression, it is replaced with a new
variable (also called `$in`) and wrapped in `Expr::Collect`. During
eval, this expression is evaluated directly, with the input and with
that new variable set to the collected value.
Other than being faster and less prone to gotchas, it also makes it
possible to typecheck the output of an expression containing `$in`,
which is nice. This is a breaking change though, because of the lack of
the closure and because now typechecking will actually happen. Also, I
haven't attempted to typecheck the input yet.
The IR generated now just looks like this:
```gas
collect %in
clone %tmp, %in
store-variable $in, %tmp
# %out <- ...expression... <- %in
drop-variable $in
```
(where `$in` is the local variable created for this collection, and not
`IN_VARIABLE_ID`)
which is a lot better than having to create a closure and call `collect
--keep-env`, dealing with all of the capture gathering and allocation
that entails. Ideally we can also detect whether that input is actually
needed, so maybe we don't have to clone, but I haven't tried to do that
yet. Theoretically now that the variable is a unique one every time, it
should be possible to give it a type - I just don't know how to
determine that yet.
On top of that, I've also reworked how `$in` works in pipeline-initial
position. Previously, it was a little bit inconsistent. For example,
this worked:
```nushell
> 3 | do { let x = $in; let y = $in; print $x $y }
3
3
```
However, this causes a runtime variable not found error on the second
`$in`:
```nushell
> def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
Error: nu:🐚:variable_not_found
× Variable not found
╭─[entry #115:1:35]
1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
· ─┬─
· ╰── variable not found
╰────
```
I've fixed this by making the first element `$in` detection *always*
happen at the block level, so if you use `$in` in pipeline-initial
position anywhere in a block, it will collect with an implicit
subexpression around the whole thing, and you can then use that `$in`
more than once. In doing this I also rewrote `parse_pipeline()` and
hopefully it's a bit more straightforward and possibly more efficient
too now.
Finally, I've tried to make `let` and `mut` a lot more straightforward
with how they handle the rest of the pipeline, and using a redirection
with `let`/`mut` now does what you'd expect if you assume that they
consume the whole pipeline - the redirection is just processed as
normal. These both work now:
```nushell
let x = ^foo err> err.txt
let y = ^foo out+err>| str length
```
It was previously possible to accomplish this with a subexpression, but
it just seemed like a weird gotcha that you couldn't do it. Intuitively,
`let` and `mut` just seem to take the whole line.
- closes #13137
# User-Facing Changes
- `$in` will behave more consistently with blocks and closures, since
the entire block is now just wrapped to handle it if it appears in the
first pipeline element
- `$in` no longer creates a closure, so what can be done within an
expression containing `$in` is less restrictive
- `$in` containing expressions are now type checked, rather than just
resulting in `any`. However, `$in` itself is still `any`, so this isn't
quite perfect yet
- Redirections are now allowed in `let` and `mut` and behave pretty much
how you'd expect
# Tests + Formatting
Added tests to cover the new behaviour.
# After Submitting
- [ ] release notes (definitely breaking change)
|
||
|
Devyn Cairns
|
a2758e6c40
|
Add IR support to the debugger (#13345)
# Description This adds tracing for each individual instruction to the `Debugger` trait. Register contents can be inspected both when entering and leaving an instruction, and if an instruction produced an error, a reference to the error is also available. It's not the full `EvalContext` but it's most of the important parts for getting an idea of what's going on. Added support for all of this to the `Profiler` / `debug profile` as well, and the output is quite incredible - super verbose, but you can see every instruction that's executed and also what the result was if it's an instruction that has a clearly defined output (many do). # User-Facing Changes - Added `--instructions` to `debug profile`, which adds the `pc` and `instruction` columns to the output. - `--expr` only works in AST mode, and `--instructions` only works in IR mode. In the wrong mode, the output for those columns is just blank. # Tests + Formatting All passing. # After Submitting - [ ] release notes |
||
|
Devyn Cairns
|
ccd0160c32
|
Make the store-env IR instruction also update config (#13351)
# Description Follow up fix to #13332, so that changes to config when running under IR actually happen as well. Since I merged them around the same time, I forgot about this. |
||
|
Darren Schroeder
|
ac561b1b0e
|
quick fix up for ir pr as_refs (#13340)
# Description Was having an issue compiling main after the IR pr. Talked to devyn and he led me to change a couple things real quick and we're compiling once again. |
||
|
Devyn Cairns
|
d7392f1f3b
|
Internal representation (IR) compiler and evaluator (#13330)
# Description This PR adds an internal representation language to Nushell, offering an alternative evaluator based on simple instructions, stream-containing registers, and indexed control flow. The number of registers required is determined statically at compile-time, and the fixed size required is allocated upon entering the block. Each instruction is associated with a span, which makes going backwards from IR instructions to source code very easy. Motivations for IR: 1. **Performance.** By simplifying the evaluation path and making it more cache-friendly and branch predictor-friendly, code that does a lot of computation in Nushell itself can be sped up a decent bit. Because the IR is fairly easy to reason about, we can also implement optimization passes in the future to eliminate and simplify code. 2. **Correctness.** The instructions mostly have very simple and easily-specified behavior, so hopefully engine changes are a little bit easier to reason about, and they can be specified in a more formal way at some point. I have made an effort to document each of the instructions in the docs for the enum itself in a reasonably specific way. Some of the errors that would have happened during evaluation before are now moved to the compilation step instead, because they don't make sense to check during evaluation. 3. **As an intermediate target.** This is a good step for us to bring the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at some point, as code generated from new AST can be directly compared to code generated from old AST. If the IR code is functionally equivalent, it will behave the exact same way. 4. **Debugging.** With a little bit more work, we can probably give control over advancing the virtual machine that `IrBlock`s run on to some sort of external driver, making things like breakpoints and single stepping possible. Tools like `view ir` and [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than before to see what exactly is going on with your Nushell code. The goal is to eventually replace the AST evaluator entirely, once we're sure it's working just as well. You can help dogfood this by running Nushell with `$env.NU_USE_IR` set to some value. The environment variable is checked when Nushell starts, so config runs with IR, or it can also be set on a line at the REPL to change it dynamically. It is also checked when running `do` in case within a script you want to just run a specific piece of code with or without IR. # Example ```nushell view ir { |data| mut sum = 0 for n in $data { $sum += $n } $sum } ``` ```gas # 3 registers, 19 instructions, 0 bytes of data 0: load-literal %0, int(0) 1: store-variable var 904, %0 # let 2: drain %0 3: drop %0 4: load-variable %1, var 903 5: iterate %0, %1, end 15 # for, label(1), from(14:) 6: store-variable var 905, %0 7: load-variable %0, var 904 8: load-variable %2, var 905 9: binary-op %0, Math(Plus), %2 10: span %0 11: store-variable var 904, %0 12: load-literal %0, nothing 13: drain %0 14: jump 5 15: drop %0 # label(0), from(5:) 16: drain %0 17: load-variable %0, var 904 18: return %0 ``` # Benchmarks All benchmarks run on a base model Mac Mini M1. ## Iterative Fibonacci sequence This is about as best case as possible, making use of the much faster control flow. Most code will not experience a speed improvement nearly this large. ```nushell def fib [n: int] { mut a = 0 mut b = 1 for _ in 2..=$n { let c = $a + $b $a = $b $b = $c } $b } use std bench bench { 0..50 | each { |n| fib $n } } ``` IR disabled: ``` ╭───────┬─────────────────╮ │ mean │ 1ms 924µs 665ns │ │ min │ 1ms 700µs 83ns │ │ max │ 3ms 450µs 125ns │ │ std │ 395µs 759ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ``` IR enabled: ``` ╭───────┬─────────────────╮ │ mean │ 452µs 820ns │ │ min │ 427µs 417ns │ │ max │ 540µs 167ns │ │ std │ 17µs 158ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ```  ## [gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu) IR disabled: ``` ╭───┬──────────────────╮ │ 0 │ 27ms 929µs 958ns │ │ 1 │ 21ms 153µs 459ns │ │ 2 │ 18ms 639µs 666ns │ │ 3 │ 19ms 554µs 583ns │ │ 4 │ 13ms 383µs 375ns │ │ 5 │ 11ms 328µs 208ns │ │ 6 │ 5ms 659µs 542ns │ ╰───┴──────────────────╯ ``` IR enabled: ``` ╭───┬──────────────────╮ │ 0 │ 22ms 662µs │ │ 1 │ 17ms 221µs 792ns │ │ 2 │ 14ms 786µs 708ns │ │ 3 │ 13ms 876µs 834ns │ │ 4 │ 13ms 52µs 875ns │ │ 5 │ 11ms 269µs 666ns │ │ 6 │ 6ms 942µs 500ns │ ╰───┴──────────────────╯ ``` ## [random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu) I got pretty random results out of this benchmark so I decided not to include it. Not clear why. # User-Facing Changes - IR compilation errors may appear even if the user isn't evaluating with IR. - IR evaluation can be enabled by setting the `NU_USE_IR` environment variable to any value. - New command `view ir` pretty-prints the IR for a block, and `view ir --json` can be piped into an external tool like [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir). # Tests + Formatting All tests are passing with `NU_USE_IR=1`, and I've added some more eval tests to compare the results for some very core operations. I will probably want to add some more so we don't have to always check `NU_USE_IR=1 toolkit test --workspace` on a regular basis. # After Submitting - [ ] release notes - [ ] further documentation of instructions? - [ ] post-release: publish `nu_plugin_explore_ir` |