Hello!
This is my 1st contribution and an attempt at fixing #15417.
# Description
When parsing a brace expression, check if the shape is a block or match
block before attempting to parse it as a closure.
Results:
- `if true {|| print hi}` now produces a `nu::parser` error instead of
executing and outputting `hi`. The `nu::parser` error is the same one
produced by running `|| print hi` (`nu::parser::shell_oror`)
- `match true {|| print hi}` now fails with a `nu::parser` error instead
of passing parsing and failing with `nu::compile::invalid_keyword_call`
My understanding reading the code/docs is that the shape is a contextual
constraint that needs to be satisfied for parsing to succeed, in this
case the `if` placing a `Block` shape constraint on next tokens. So it
would need to be checked in priority (if not `Any`) to understand how
the next tokens should be parsed. Is that correct? Or is there a reason
I'm not aware of to ignore the shape and attempt to parse as closure
like it's currently the case when the parser sees `|` or `||` as next
tokens?
# User-Facing Changes
No change in behaviour, but this PR fixes parsing to fail on some
incorrect syntax which could be considered a breaking change.
# Tests + Formatting
- Added corresponding tests
- `toolkit check pr` passed
# Description
Fixes: https://github.com/nushell/nushell/issues/15749
When sourcing a file, the ir block might be empty because it has been
used before, this pr is going to make sure that the ir block is
compiled.
# User-Facing Changes
```
touch aaa.nu
use aaa.nu
source aaa.nu
```
Will no longer raise an error.
# Tests + Formatting
Added 1 test
# After Submitting
NaN
Related:
- #15683
- #14551
- #849
- #12701
- #11527
# Description
Currently various commands have differing behavior regarding cell-paths
```nushell
{a: 1, A: 2} | get a A
# => ╭───┬───╮
# => │ 0 │ 2 │
# => │ 1 │ 2 │
# => ╰───┴───╯
{a: 1, A: 2} | select a A
# => ╭───┬───╮
# => │ a │ 1 │
# => │ A │ 2 │
# => ╰───┴───╯
{A: 1} | update a 2
# => Error: nu:🐚:column_not_found
# =>
# => × Cannot find column 'a'
# => ╭─[entry #62:1:1]
# => 1 │ {A: 1} | update a 2
# => · ───┬── ┬
# => · │ ╰── cannot find column 'a'
# => · ╰── value originates here
# => ╰────
```
Proposal: making cell-path access case-sensitive by default and adding
new syntax for case-insensitive parts, similar to optional (?) parts.
```nushell
{FOO: BAR}.foo
# => Error: nu:🐚:name_not_found
# =>
# => × Name not found
# => ╭─[entry #60:1:21]
# => 1 │ {FOO: BAR}.foo
# => · ─┬─
# => · ╰── did you mean 'FOO'?
# => ╰────
{FOO: BAR}.foo!
# => BAR
```
This would solve the problem of case sensitivity for all commands
without causing an explosion of flags _and_ make it more granular
Assigning to a field using a case-insensitive path is case-preserving.
```nushell
mut val = {FOO: "I'm FOO"}; $val
# => ╭─────┬─────────╮
# => │ FOO │ I'm FOO │
# => ╰─────┴─────────╯
$val.foo! = "I'm still FOO"; $val
# => ╭─────┬───────────────╮
# => │ FOO │ I'm still FOO │
# => ╰─────┴───────────────╯
```
For `update`, case-insensitive is case-preserving.
```nushell
{FOO: 1} | update foo! { $in + 1 }
# => ╭─────┬───╮
# => │ FOO │ 2 │
# => ╰─────┴───╯
```
`insert` can insert values into nested values so accessing into existing
columns is case-insensitive, but creating new columns uses the cell-path
as it is.
So `insert foo! ...` and `insert FOO! ...` would work exactly as they do
without `!`
```nushell
{FOO: {quox: 0}}
# => ╭─────┬──────────────╮
# => │ │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │ │ ╰──────┴───╯ │
# => ╰─────┴──────────────╯
{FOO: {quox: 0}} | insert foo.bar 1
# => ╭─────┬──────────────╮
# => │ │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │ │ ╰──────┴───╯ │
# => │ │ ╭─────┬───╮ │
# => │ foo │ │ bar │ 1 │ │
# => │ │ ╰─────┴───╯ │
# => ╰─────┴──────────────╯
{FOO: {quox: 0}} | insert foo!.bar 1
# => ╭─────┬──────────────╮
# => │ │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │ │ │ bar │ 1 │ │
# => │ │ ╰──────┴───╯ │
# => ╰─────┴──────────────╯
```
`upsert` is tricky, depending on the input, the data might end up with
different column names in rows. We can either forbid case-insensitive
cell-paths for `upsert` or trust the user to keep their data in a
sensible shape.
This would be a breaking change as it would make existing cell-path
accesses case-sensitive, however the case-sensitivity is already
inconsistent and any attempt at making it consistent would be a breaking
change.
> What about `$env`?
1. Initially special case it so it keeps its current behavior.
2. Accessing environment variables with non-matching paths gives a
deprecation warning urging users to either use exact casing or use the
new explicit case-sensitivity syntax
3. Eventuall remove `$env`'s special case, making `$env` accesses
case-sensitive by default as well.
> `$env.ENV_CONVERSIONS`?
In addition to `from_string` and `to_string` add an optional field to
opt into case insensitive/preserving behavior.
# User-Facing Changes
- `get`, `where` and other previously case-insensitive commands are now
case-sensitive by default.
- `get`'s `--sensitive` flag removed, similar to `--ignore-errors` there
is now an `--ignore-case` flag that treats all parts of the cell-path as
case-insensitive.
- Users can explicitly choose the case case-sensitivity of cell-path
accesses or commands.
# Tests + Formatting
Existing tests required minimal modification. ***However, new tests are
not yet added***.
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
- Update the website to include the new syntax
- Update [tree-sitter-nu](https://github.com/nushell/tree-sitter-nu)
---------
Co-authored-by: Bahex <17417311+Bahex@users.noreply.github.com>
- fixes#15731
# Description
Existing bare word string interpolation only works if the string doesn't
start with a subxpression.
```nushell
echo fork(2)
# => fork2
echo (2)fork
# => Error: nu::parser::unclosed_delimiter
# =>
# => × Unclosed delimiter.
# => ╭─[entry #25:1:13]
# => 1 │ echo (2)fork
# => ╰────
```
This PR lifts that restriction.
```nushell
echo fork(2)
# => fork2
echo (2)fork
# => 2fork
```
This was first brought to my attention on discord with the following
command failing to parse.
```nushell
docker run -u (id -u):(id -g)
```
It now works.
# User-Facing Changes
# Tests + Formatting
No existing test broke or required tweaking. Additional tests covering
this case was added.
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
---------
Co-authored-by: Bahex <17417311+Bahex@users.noreply.github.com>
Closes#12858
# Description
As explained in the ticket, easy to reproduce. Example: 1.07 minute is
1.07*60=64.2 secondes
```nushell
# before - wrong
> 1.07min
1min 4sec
# now - right
> 1.07min
1min 4sec 200ms
```
# User-Facing Changes
Bug is fixed when using ``into duration``.
# Tests + Formatting
Added a test for ``into duration``
Fixed ``parse_long_duration`` test: we gained precision 😄
# After Submitting
Release notes? Or blog is enough? Let me know
Closes#15502
# Description
The parsing of Exbibytes used the wrong base unit before converting.
# User-Facing Changes
`1EiB` etc. will now be parsed correctly
# Tests + Formatting
(-)
# Description
There are some clippy(version 0.1.86) errors on nushell repo. This pr is
trying to fix it.
# User-Facing Changes
Hopefully none.
# Tests + Formatting
NaN
# After Submitting
NaN
Closes#15305
# Description
Basically turns off `skip_comments` of the lex function for right hand
side expressions of `let`/`mut`, just as in `parse_const`.
# User-Facing Changes
Should be none.
# Tests + Formatting
+1
# After Submitting
Fixes#15243
# Description
As noted in #15243, a record with more characters after it (e.g.,
`{a:b}/`) will cause an OOM due to an infinite loop, introduced by
#15023. This happens because the entire string `{a:b}/` is lexed as one
token and passed to `parse_record`, where it repeatedly lexes until it
hits the closing `}`. This PR detects such extra characters and reports
an error.
# User-Facing Changes
`{a:b}/` and other such constructions will no longer cause infinite
loops. Before #15023, you would've seen an "Unclosed delimiter" error
message, but this PR changes that to "Invalid characters."
```
Error: nu::parser::extra_token_after_closing_delimiter
× Invalid characters after closing delimiter
╭─[entry #5:1:7]
1 │ {a:b}/
· ┬
· ╰── invalid characters
╰────
help: Try removing them.
```
# Tests + Formatting
# After Submitting
Fixes#14971, fixes#15229
# User-Facing Changes
Fixes a panic when variable data is accessed after invalid usage of the
`|` separator, which made it impossible to type certain match arms:
```nushell
> match $in { 1 |
Error: x Main thread panicked.
|-> at crates/nu-protocol/src/engine/state_delta.rs💯14
`-> internal error: missing required scope frame
```
# Description
Removes duplicative calls to `exit_scope` from an inner loop when `|`
parse errors are encountered. The outer loop creates and exits scopes
for each match arm.
# Description
This PR adds two new `ParseError` and `ShellError` cases for type errors
relating to operators.
- `OperatorUnsupportedType` is used when a type is not supported by an
operator in any way, shape, or form. E.g., `+` does not support `bool`.
- `OperatorIncompatibleTypes` is used when a operator is used with types
it supports, but the combination of types provided cannot be used
together. E.g., `filesize + duration` is not a valid combination.
The other preexisting error cases related to operators have been removed
and replaced with the new ones above. Namely:
- `ShellError::OperatorMismatch`
- `ShellError::UnsupportedOperator`
- `ParseError::UnsupportedOperationLHS`
- `ParseError::UnsupportedOperationRHS`
- `ParseError::UnsupportedOperationTernary`
# User-Facing Changes
- `help operators` now lists the precedence of `not` as 55 instead of 0
(above the other boolean operators). Fixes#13675.
- `math median` and `math mode` now ignore NaN values so that `[NaN NaN]
| math median` and `[NaN NaN] | math mode` no longer trigger a type
error. Instead, it's now an empty input error. Fixing this in earnest
can be left for a future PR.
- Comparisons with `nan` now return false instead of causing an error.
E.g., `1 == nan` is now `false`.
- All the operator type errors have been standardized and reworked. In
particular, they can now have a help message, which is currently used
for types errors relating to `++`.
```nu
[1] ++ 2
```
```
Error: nu::parser::operator_unsupported_type
× The '++' operator does not work on values of type 'int'.
╭─[entry #1:1:5]
1 │ [1] ++ 2
· ─┬ ┬
· │ ╰── int
· ╰── does not support 'int'
╰────
help: if you meant to append a value to a list or a record to a table, use the `append` command or wrap the value in a list. For example: `$list ++ $value` should be
`$list ++ [$value]` or `$list | append $value`.
```
# Description
Zyphys found that when parsing `{...{}, ...{}, a: 1}`, the `a:` would be
considered one token, leading to a parse error ([Discord
message](https://discord.com/channels/601130461678272522/614593951969574961/1336762075535511573)).
This PR fixes that.
What would happen is that while getting tokens, the following would
happen in a loop:
1. Get the next two tokens while treating `:` as a special character (so
we get the next field key and a colon token)
2. Get the next token while not treating `:` as a special character (so
we get the next value)
I didn't update this when I added the spread operator. With `{...{},
...{}, a: 1}`, the first two tokens would be `...{}` and `...{}`, and
the next token would be `a:`. This PR changes this loop to first get a
single token, check if it's spreading a record, and move on if so.
Alternatives considered:
- Treat `:` as a special character when getting the value too. This
would simplify the loop greatly, but would mean you can't use colons in
values.
- Merge the loop for getting tokens and the loop for parsing those
tokens. I tried this, but it complicates things if you run into a syntax
error and want to create a garbage span going to the end of the record.
# User-Facing Changes
Nothing new
# Description
Add custom command attributes.
- Attributes are placed before a command definition and start with a `@`
character.
- Attribute invocations consist of const command call. The command's
name must start with "attr ", but this prefix is not used in the
invocation.
- A command named `attr example` is invoked as an attribute as
`@example`
- Several built-in attribute commands are provided as part of this PR
- `attr example`: Attaches an example to the commands help text
```nushell
# Double numbers
@example "double an int" { 5 | double } --result 10
@example "double a float" { 0.5 | double } --result 1.0
def double []: [number -> number] {
$in * 2
}
```
- `attr search-terms`: Adds search terms to a command
- ~`attr env`: Equivalent to using `def --env`~
- ~`attr wrapped`: Equivalent to using `def --wrapped`~ shelved for
later discussion
- several testing related attributes in `std/testing`
- If an attribute has no internal/special purpose, it's stored as
command metadata that can be obtained with `scope commands`.
- This allows having attributes like `@test` which can be used by test
runners.
- Used the `@example` attribute for `std` examples.
- Updated the std tests and test runner to use `@test` attributes
- Added completions for attributes
# User-Facing Changes
Users can add examples to their own command definitions, and add other
arbitrary attributes.
# Tests + Formatting
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
- Add documentation about the attribute syntax and built-in attributes
- `help attributes`
---------
Co-authored-by: 132ikl <132@ikl.sh>
# Description
Fixes: #15049
The error occurs when using an alias with a module prefix, it can
initially pass through alias checking, but if the alias leads to
commands which have side effects, it doesn't call these functions to
apply side effects.
This pr ensure that in such cases, nushell still calls
`parse_overlay_xxx` functions to apply the side effects.
I want to make my test easier to write, so this pr depends on
https://github.com/nushell/nushell/pull/15054.
# User-Facing Changes
The following code will no longer raise an error:
```
module inner {}
module spam { export alias b = overlay use inner }
use spam
spam b
```
# Tests + Formatting
Added 2 tests.
# After Submitting
NaN
# Description
This PR fixes#14784.
<img width="384" alt="image"
src="https://github.com/user-attachments/assets/aac063a0-645d-4adb-a399-525bdb004999"
/>
Also fixes the related behavior of lsp:
completion won't work in match/else blocks, because:
1. truncation in completion causes unmatched `{`, thus a parse error.
2. the parse error further leads to a state where the whole block
expression marked as garbage
<img width="453" alt="image"
src="https://github.com/user-attachments/assets/aaf86ccc-646e-4b91-bb27-4b1737100ff2"
/>
Related PR: #14856, @tmillr
I don't have any background knowledge of those `propagate_error`,
@sgvictorino you may want to review this.
# User-Facing Changes
# Tests + Formatting
# After Submitting
# Description
This PR fixes#14816 , so that expression-contains-position check won't
need special treatment for keyword expressions.
e.g.
```nushell
overlay use foo as bar
# |_______ cursor here
if true { } else { }
# |_______ here
```
as mentioned in #14924
# User-Facing Changes
# Tests + Formatting
# After Submitting
# Description
Fixes: #14844
The issue occurs when nushell is parsing a value with
`SyntaxShape::Any`, it checks `Duration` and `Filesize` first, then
`Range`. Nushell raises errors too early while parsing
`Duration/Filesize`.
This pr changes the order of parsing to fix the issue.
# User-Facing Changes
The following code should be able to run after this pr
```nushell
let runs = 10;
1..$runs
```
# Tests + Formatting
Added 2 tests, one for filesize, one for duration.
# After Submitting
NaN
# Description
This PR add 2 new operators, `has` and `not-has`. They are basically
`in` and `not-in` with the order of operands swapped.
Motivation for this was the awkward way of searching for rows that
contain an item using `where`
```nushell
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]]
| where ("e" in $it.children)
```
vs
```nushell
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]]
| where children has "e"
```
# User-Facing Changes
Added `has` and `not-has` operators, mirroring `in` and `not-in`.
# Tests + Formatting
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Follow-up PR of #14789
The span of `$it/$in` is set to a 0-width one with start/end pointing to
the start of its scope, mainly for error messages positioning.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR addresses the issue of inconsistent spans of special variables
of `$in/$in`, as discussed in
https://github.com/nushell/nushell/pull/14770#discussion_r1908729364.
Instead of making the `declaration_span` to be Option, which will cause
too many changes that we may want to avoid, this PR set the spans to be
`unknown`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
No
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Related:
- #14329
- #13872
- #8214
# Description & User-Facing Changes
This PR allows enables the following uses, which are all no-op.
```nushell
source null
source-env null
use null
overlay use null
```
The motivation for this change is conditional sourcing of files. For
example, with this change `login.nu` may be deprecated and replaced with
the following code in `config.nu`
```nushell
const login_module = if $nu.is-login { "login.nu" } else { null }
source $login_module
```
# Tests + Formatting
I'm hoping for CI to pass 😄
# After Submitting
Add a part about the conditional sourcing pattern to the website.
# Description
Because `and` and `or` are short-circuiting operations in Nushell, they
must be compiled to a sequence that avoids evaluating the RHS if the LHS
is already sufficient to determine the output - i.e., `false` for `and`
and `true` for `or`. I initially implemented this with `branch-if`
instructions, simply returning the RHS if it needed to be evaluated, and
returning the short-circuited boolean value if it did not.
Example for `$a and $b`:
```
0: load-variable %0, var 999 "$a"
1: branch-if %0, 3
2: jump 5
3: load-variable %0, var 1000 "$b" # label(0), from(1:)
4: jump 6
5: load-literal %0, bool(false) # label(1), from(2:)
6: span %0 # label(2), from(4:)
7: return %0
```
Unfortunately, this broke polars, because using `and`/`or` on custom
values is perfectly valid and they're allowed to define that behavior
differently, and the polars plugin uses this for boolean masks. But
without using the `binary-op` instruction, that custom behavior is never
invoked. Additionally, `branch-if` requires a boolean, and custom values
are not booleans. This changes the IR to the following, using the
`match` instruction to check for the specific short-circuit value
instead, and still invoking `binary-op` otherwise:
```
0: load-variable %0, var 125 "$a"
1: match (false), %0, 4
2: load-variable %1, var 124 "$b"
3: binary-op %0, Boolean(And), %1
4: span %0 # label(0), from(1:)
5: return %0
```
I've also renamed `Pattern::Value` to `Pattern::Expression` and added a
proper `Pattern::Value` variant that actually contains a `Value`
instead. I'm still hoping to remove `Pattern::Expression` eventually,
because it's kind of a hack - we don't actually evaluate the expression,
we just match it against a few cases specifically for pattern matching,
and it's one of the cases where AST leaks into IR and I want to remove
all of those cases, because AST should not leak into IR.
Fixes#14518
# User-Facing Changes
- `and` and `or` now support custom values again.
- the IR is actually a little bit cleaner, though it may be a bit
slower; `match` is more complex.
# Tests + Formatting
The existing tests pass, but I didn't add anything new. Unfortunately I
don't think there's anything built-in to trigger this, but maybe some
testcases could be added to polars to test it.
# Description
Closes#14521
This PR tweaks the way 64-bit hex numbers are parsed.
### Before
```nushell
❯ 0xffffffffffffffef
Error: nu:🐚:external_command
× External command failed
╭─[entry #1:1:1]
1 │ 0xffffffffffffffef
· ─────────┬────────
· ╰── Command `0xffffffffffffffef` not found
╰────
help: `0xffffffffffffffef` is neither a Nushell built-in or a known external command
```
### After
```nushell
❯ 0xffffffffffffffef
-17
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
- This PR should fix/close:
- #11266
- #12893
- #13736
- #13748
- #14170
- It doesn't fix#13736 though unfortunately. The issue there is at a
different level to this fix (I think probably in the lexing somewhere,
which I haven't touched).
# The Problem
The linked issues have many examples of the problem and the related
confusion it causes, but I'll give some more examples here for
illustration. It boils down to the following:
This doesn't type check (good):
```nu
def foo []: string -> int { false }
```
This does (bad):
```nu
def foo [] : string -> int { false }
```
Because the parser is completely ignoring all the characters. This also
compiles in 0.100.0:
```nu
def blue [] Da ba Dee da Ba da { false }
```
And this also means commands which have a completely fine type, but an
extra space before `:`, lose that type information and end up as `any ->
any`, e.g.
```nu
def foo [] : int -> int {$in + 3}
```
```bash
$ foo --help
Input/output types:
╭───┬───────┬────────╮
│ # │ input │ output │
├───┼───────┼────────┤
│ 0 │ any │ any │
╰───┴───────┴────────╯
```
# The Fix
Special thank you to @texastoland whose draft PR (#12358) I referenced
heavily while making this fix.
That PR seeks to fix the invalid parsing by disallowing whitespace
between `[]` and `:` in declarations, e.g. `def foo [] : int -> any {}`
This PR instead allows the whitespace while properly parsing the type
signature. I think this is the better choice for a few reasons:
- The parsing is still straightforward and the information is all there
anyway,
- It's more consistent with type annotations in other places, e.g. `do
{|nums : list<int>| $nums | describe} [ 1 2 3 ]` from the [Type
Signatures doc
page](https://www.nushell.sh/lang-guide/chapters/types/type_signatures.html)
- It's more consistent with the new nu parser, which allows `let x :
bool = false` (current nu doesn't, but this PR doesn't change that)
- It will be less disruptive and should only break code where the types
are actually wrong (if your types were correct, but you had a space
before the `:`, those declarations will still compile and now have more
type information vs. throwing an error in all cases and requiring spaces
to be deleted)
- It's the more intuitive syntax for most functional programmers like
myself (haskell/lean/coq/agda and many more either allow or require
whitespace for type annotations)
I don't use Rust a lot, so I tried to keep most things the same and the
rest I wrote as if it was Haskell (if you squint a bit). Code
review/suggestions very welcome. I added all the tests I could think of
and `toolkit check pr` gives it the all-clear.
# User-Facing Changes
This PR meets part of the goal of #13849, but doesn't do anything about
parsing signatures twice and doesn't do much to improve error messages,
it just enforces the existing errors and error messages.
This will no doubt be a breaking change, mostly because the code is
already broken and users don't realise yet (one of my personal scripts
stopped compiling after this fix because I thought `def foo [] -> string
{}` was valid syntax). It shouldn't break any type-correct code though.
# Description
While reviewing #14388, I think we can make some improvement on parser.
For the following code:
```nushell
let a = 3
a = 10 # should be error
$a = 10 # another error
```
I think they can raise `ParseError`, so nushell doesn't need to move
forward compiling IR block.
# User-Facing Changes
```nushell
let a = 3
a = 10
```
Will raise parse error instead of compile error.
# Tests + Formatting
Added 1 test.
# Description
Adds a new `Filesize` type so that `FromValue` can be used to convert a
`Value::Filesize` to a `Filesize`. Currently, to extract a filesize from
a `Value` using `FromValue`, you have to extract an `i64` which coerces
`Value::Int`, `Value::Duration`, and `Value::Filesize` to an `i64`.
Having a separate type also allows us to enforce checked math to catch
overflows. Similarly, it allows us to specify other trait
implementations like `Display` in a common place.
# User-Facing Changes
Multiplication with filesizes now error on overflow. Should not be a
breaking change for plugins (i.e., serialization) since `Filesize` is
marked with `serde(transparent)`.
# Tests + Formatting
Updated some tests.
# Description
The "append" operator currently serves as both the append operator and
the concatenation operator. This dual role creates ambiguity when
operating on nested lists.
```nu
[1 2] ++ 3 # appends a value to a list [1 2 3]
[1 2] ++ [3 4] # concatenates two lists [1 2 3 4]
[[1 2] [3 4]] ++ [5 6]
# does this give [[1 2] [3 4] [5 6]]
# or [[1 2] [3 4] 5 6]
```
Another problem is that `++=` can change the type of a variable:
```nu
mut str = 'hello '
$str ++= ['world']
($str | describe) == list<string>
```
Note that appending is only relevant for lists, but concatenation is
relevant for lists, strings, and binary values. Additionally, appending
can be expressed in terms of concatenation (see example below). So, this
PR changes the `++` operator to only perform concatenation.
# User-Facing Changes
Using the `++` operator with a list and a non-list value will now be a
compile time or runtime error.
```nu
mut list = []
$list ++= 1 # error
```
Instead, concatenate a list with one element:
```nu
$list ++= [1]
```
Or use `append`:
```nu
$list = $list | append 1
```
# After Submitting
Update book and docs.
---------
Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com>
# User-Facing Changes
- `expected <type>` errors are now propagated from
`Closure | Block | Expression` instead of falling back to
"expected one of..." for the block:
Before:
```nushell
def foo [bar: bool] {}
if true {} else { foo 1 }
────┬────
╰── expected one of a list of accepted shapes: [Block, Expression]
```
After:
```nushell
if true {} else { foo 1 }
┬
╰── expected bool
```
# User-Facing Changes
The parser now errors on more invalid command signatures:
```nushell
# expected parameter or flag
def foo [ bar: int: ] {}
# expected type
def foo [ bar: = ] {}
def foo [ bar: ] {}
# expected default value
def foo [ bar = ] {}
```
Fixes#13776
# User-Facing Changes
Arguments to aliased externals no longer include nested import paths:
```diff
module foo { export alias bar = ^echo }
use foo
foo bar baz
-bar baz
+baz
```
# User-Facing Changes
Table literal arguments to list parameters are now correctly parsed:
```diff
def a [l: list<any>] { $l | to nuon }; a [[a]; [2]]
-[[a]]
+[[a]; [2]]
```
# Description
Fixes: #14202
After looking into the issue, I think #13910 it's not good to cut the
span if it's in external argument.
This pr is somehow revert the change, and fix
https://github.com/nushell/nushell/issues/13431 in another way.
It introduce a new state named `State::BackTickQuote`, so if an external
arg include backtick quote, it enters the state, so backtick quote won't
be the body of a string.
# User-Facing Changes
### Before
```nushell
> ^echo `(echo aa)`
aa
> ^echo `"aa"` # maybe it's not right to remove the inner quote.
aa
```
### After
```nushell
> ^echo `(echo aa)`
(echo aa)
> ^echo `"aa"` # inner quote is keeped if there are backtick quote outside.
"aa"
```
# Tests + Formatting
Added 3 tests.
Fixes#13757, fixes#9562
# User-Facing Changes
- `unclosed |` is returned for malformed closure parameters:
```
{ |a }
```
- Parameter list closing pipes are highlighted as part of the closure
# Description
This PR adds `like` as a synonym for `=~` and `not-like` as a synonym
for `!~`. This is mainly a quality-of-life change to help those people
who think in sql.
closes#13261
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Closes#13654
# User-Facing Changes
- Short flags are now fully type-checked,
including null and record signatures for literal arguments:
```nushell
def test [-v: record<l: int>] {};
test -v null # error
test -v {l: ""} # error
def test2 [-v: int] {};
let v = ""
test2 -v $v # error
```
- `polars unpivot` `--index`/`--on` and `into value --columns`
now accept `list` values
# Description
This PR adds `like` as a synonym for `=~` and `not-like` as a synonym
for `!~`. This is mainly a quality-of-life change to help those people
who think in sql.
closes#13261
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Closes#13654
# User-Facing Changes
- Short flags are now fully type-checked,
including null and record signatures for literal arguments:
```nushell
def test [-v: record<l: int>] {};
test -v null # error
test -v {l: ""} # error
def test2 [-v: int] {};
let v = ""
test2 -v $v # error
```
- `polars unpivot` `--index`/`--on` and `into value --columns`
now accept `list` values
# Description
Fixes: #13431Fixes: #13578
The issue happened because nushell thinks external program name and
external arg with totally same rule. But actually they are a little bit
different.
When parsing external program name, backtick is a thing and it should be
keeped.
But when parsing external args, backtick is just a mark that it's a
**bareword which may contain space**. So in this context, it's already
useless.
# User-Facing Changes
After the pr, the following command will work as intended.
```nushell
> ^echo `"hello"`
hello
```
# Tests + Formatting
Added 3 test cases.
# Description
In this PR I replaced most of the raw usize IDs with
[newtypes](https://doc.rust-lang.org/rust-by-example/generics/new_types.html).
Some other IDs already started using new types and in this PR I did not
want to touch them. To make the implementation less repetitive, I made
use of a generic `Id<T>` with marker structs. If this lands I would try
to move make other IDs also in this pattern.
Also at some places I needed to use `cast`, I'm not sure if the type was
incorrect and therefore casting not needed or if actually different ID
types intermingle sometimes.
# User-Facing Changes
Probably few, if you got a `DeclId` via a function and placed it later
again it will still work.
