Fixes#14972#15321#14706
# Description
Early returns `NotAConstant` if parsing errors exist in the
subexpression.
I'm not sure when the span of a block will be None, and whether there're
better ways to handle none block spans, like a more suitable ShellError
type.
# User-Facing Changes
# Tests + Formatting
+1, but possibly not the easiest way to do it.
# After Submitting
Closes#15305
# Description
Basically turns off `skip_comments` of the lex function for right hand
side expressions of `let`/`mut`, just as in `parse_const`.
# User-Facing Changes
Should be none.
# Tests + Formatting
+1
# After Submitting
Closes#15373
# Description
Now `ast -f "{||}"` will return
```
╭─content─┬─────shape─────┬─────span──────╮
│ {||} │ shape_closure │ ╭───────┬───╮ │
│ │ │ │ start │ 0 │ │
│ │ │ │ end │ 4 │ │
│ │ │ ╰───────┴───╯ │
╰─────────┴───────────────┴───────────────╯
```
Similar to those of `ast -f "[]"`/`ast -f "{}"`
# User-Facing Changes
# Tests + Formatting
I didn't find the right place to do the test, except for the examples of
`ast` command.
# After Submitting
Fixes#15243
# Description
As noted in #15243, a record with more characters after it (e.g.,
`{a:b}/`) will cause an OOM due to an infinite loop, introduced by
#15023. This happens because the entire string `{a:b}/` is lexed as one
token and passed to `parse_record`, where it repeatedly lexes until it
hits the closing `}`. This PR detects such extra characters and reports
an error.
# User-Facing Changes
`{a:b}/` and other such constructions will no longer cause infinite
loops. Before #15023, you would've seen an "Unclosed delimiter" error
message, but this PR changes that to "Invalid characters."
```
Error: nu::parser::extra_token_after_closing_delimiter
× Invalid characters after closing delimiter
╭─[entry #5:1:7]
1 │ {a:b}/
· ┬
· ╰── invalid characters
╰────
help: Try removing them.
```
# Tests + Formatting
# After Submitting
Fixes#14971, fixes#15229
# User-Facing Changes
Fixes a panic when variable data is accessed after invalid usage of the
`|` separator, which made it impossible to type certain match arms:
```nushell
> match $in { 1 |
Error: x Main thread panicked.
|-> at crates/nu-protocol/src/engine/state_delta.rs💯14
`-> internal error: missing required scope frame
```
# Description
Removes duplicative calls to `exit_scope` from an inner loop when `|`
parse errors are encountered. The outer loop creates and exits scopes
for each match arm.
# Description
This PR adds the `@category` attribute to nushell for use with custom
commands.
### Example Code
```nushell
# Some example with category
@category "math"
@search-terms "addition"
@example "add two numbers together" {
blah 5 6
} --result 11
def blah [
a: int # First number to add
b: int # Second number to add
] {
$a + $b
}
```
#### Source & Help
```nushell
❯ source blah.nu
❯ help blah
Some example with category
Search terms: addition
Usage:
> blah <a> <b>
Flags:
-h, --help: Display the help message for this command
Parameters:
a <int>: First number to add
b <int>: Second number to add
Input/output types:
╭─#─┬─input─┬─output─╮
│ 0 │ any │ any │
╰───┴───────┴────────╯
Examples:
add two numbers together
> blah 5 6
11
```
#### Show the category
```nushell
❯ help commands | where name == blah
╭─#─┬─name─┬─category─┬─command_type─┬────────description─────────┬─────params─────┬──input_output──┬─search_terms─┬─is_const─╮
│ 0 │ blah │ math │ custom │ Some example with category │ [table 3 rows] │ [list 0 items] │ addition │ false │
╰───┴──────┴──────────┴──────────────┴────────────────────────────┴────────────────┴────────────────┴──────────────┴──────────╯
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
/cc @Bahex
# Description
`overlay use` now imports constants exported from modules, just like
`use`.
```nushell
# foo.nu
export const a = 1
export const b = 2
```
- `overlay use foo.nu` being equivalent to `use foo.nu *` and exposing
constants `$a = 1` and `$b = 2`
- `overlay use foo.nu -p` being equivalent to `use foo.nu` and exposing
the constant `$foo = {a: 1, b: 2}`
# User-Facing Changes
`overlay use` now imports constants just like `use`.
# Tests + Formatting
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
N/A
# Description
This PR adds two new `ParseError` and `ShellError` cases for type errors
relating to operators.
- `OperatorUnsupportedType` is used when a type is not supported by an
operator in any way, shape, or form. E.g., `+` does not support `bool`.
- `OperatorIncompatibleTypes` is used when a operator is used with types
it supports, but the combination of types provided cannot be used
together. E.g., `filesize + duration` is not a valid combination.
The other preexisting error cases related to operators have been removed
and replaced with the new ones above. Namely:
- `ShellError::OperatorMismatch`
- `ShellError::UnsupportedOperator`
- `ParseError::UnsupportedOperationLHS`
- `ParseError::UnsupportedOperationRHS`
- `ParseError::UnsupportedOperationTernary`
# User-Facing Changes
- `help operators` now lists the precedence of `not` as 55 instead of 0
(above the other boolean operators). Fixes#13675.
- `math median` and `math mode` now ignore NaN values so that `[NaN NaN]
| math median` and `[NaN NaN] | math mode` no longer trigger a type
error. Instead, it's now an empty input error. Fixing this in earnest
can be left for a future PR.
- Comparisons with `nan` now return false instead of causing an error.
E.g., `1 == nan` is now `false`.
- All the operator type errors have been standardized and reworked. In
particular, they can now have a help message, which is currently used
for types errors relating to `++`.
```nu
[1] ++ 2
```
```
Error: nu::parser::operator_unsupported_type
× The '++' operator does not work on values of type 'int'.
╭─[entry #1:1:5]
1 │ [1] ++ 2
· ─┬ ┬
· │ ╰── int
· ╰── does not support 'int'
╰────
help: if you meant to append a value to a list or a record to a table, use the `append` command or wrap the value in a list. For example: `$list ++ $value` should be
`$list ++ [$value]` or `$list | append $value`.
```
# Description
Zyphys found that when parsing `{...{}, ...{}, a: 1}`, the `a:` would be
considered one token, leading to a parse error ([Discord
message](https://discord.com/channels/601130461678272522/614593951969574961/1336762075535511573)).
This PR fixes that.
What would happen is that while getting tokens, the following would
happen in a loop:
1. Get the next two tokens while treating `:` as a special character (so
we get the next field key and a colon token)
2. Get the next token while not treating `:` as a special character (so
we get the next value)
I didn't update this when I added the spread operator. With `{...{},
...{}, a: 1}`, the first two tokens would be `...{}` and `...{}`, and
the next token would be `a:`. This PR changes this loop to first get a
single token, check if it's spreading a record, and move on if so.
Alternatives considered:
- Treat `:` as a special character when getting the value too. This
would simplify the loop greatly, but would mean you can't use colons in
values.
- Merge the loop for getting tokens and the loop for parsing those
tokens. I tried this, but it complicates things if you run into a syntax
error and want to create a garbage span going to the end of the record.
# User-Facing Changes
Nothing new
# Description
Add custom command attributes.
- Attributes are placed before a command definition and start with a `@`
character.
- Attribute invocations consist of const command call. The command's
name must start with "attr ", but this prefix is not used in the
invocation.
- A command named `attr example` is invoked as an attribute as
`@example`
- Several built-in attribute commands are provided as part of this PR
- `attr example`: Attaches an example to the commands help text
```nushell
# Double numbers
@example "double an int" { 5 | double } --result 10
@example "double a float" { 0.5 | double } --result 1.0
def double []: [number -> number] {
$in * 2
}
```
- `attr search-terms`: Adds search terms to a command
- ~`attr env`: Equivalent to using `def --env`~
- ~`attr wrapped`: Equivalent to using `def --wrapped`~ shelved for
later discussion
- several testing related attributes in `std/testing`
- If an attribute has no internal/special purpose, it's stored as
command metadata that can be obtained with `scope commands`.
- This allows having attributes like `@test` which can be used by test
runners.
- Used the `@example` attribute for `std` examples.
- Updated the std tests and test runner to use `@test` attributes
- Added completions for attributes
# User-Facing Changes
Users can add examples to their own command definitions, and add other
arbitrary attributes.
# Tests + Formatting
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
- Add documentation about the attribute syntax and built-in attributes
- `help attributes`
---------
Co-authored-by: 132ikl <132@ikl.sh>
# Description
Fixes: #15049
The error occurs when using an alias with a module prefix, it can
initially pass through alias checking, but if the alias leads to
commands which have side effects, it doesn't call these functions to
apply side effects.
This pr ensure that in such cases, nushell still calls
`parse_overlay_xxx` functions to apply the side effects.
I want to make my test easier to write, so this pr depends on
https://github.com/nushell/nushell/pull/15054.
# User-Facing Changes
The following code will no longer raise an error:
```
module inner {}
module spam { export alias b = overlay use inner }
use spam
spam b
```
# Tests + Formatting
Added 2 tests.
# After Submitting
NaN
# Description
Fixes: #15048
The issue is happened while `parse_export_in_block`, it makes a call to
`parse_internal_call`, which may be an error.
But in reality, these errors are not useful, all useful errors will be
generated by `parse_xxx` at the end of the function.
# User-Facing Changes
The following code should no longer raise error:
```
export alias a = overlay use
```
# Tests + Formatting
Added 1 test.
# After Submitting
NaN
# Description
The parsing logic for several of our keywords is conditional on the
particular commands for those keywords being in scope:
942030199d/crates/nu-parser/src/parse_keywords.rs (L272-L279)
Thus the following involved parsing logic was not fuzzed by the existing
`parse` fuzz target so far.
This adds an additional fuzz target `parse_with_keywords` that loads the
commands from `nu-cmd-lang`. Those are primarily the keyword
implementations, thus the relevant code paths in the parser that depend
on those `DeclId`s and the potential const eval of `if` etc. get
unlocked.
The existing `parse` target is preserved if you have concerns about the
fuzzing breaking containment in some form due to those commands.
# Tests + Formatting
Found https://github.com/nushell/nushell/issues/14972 with this target
# Description
- Remove redundant fields from KnownExternal
- Command::extra_description and Command::search_terms using the
signature field
# User-Facing Changes
`extern` commands extra description is now shown in help text.
# Tests + Formatting
# After Submitting
# Description
This PR fixes#14784.
<img width="384" alt="image"
src="https://github.com/user-attachments/assets/aac063a0-645d-4adb-a399-525bdb004999"
/>
Also fixes the related behavior of lsp:
completion won't work in match/else blocks, because:
1. truncation in completion causes unmatched `{`, thus a parse error.
2. the parse error further leads to a state where the whole block
expression marked as garbage
<img width="453" alt="image"
src="https://github.com/user-attachments/assets/aaf86ccc-646e-4b91-bb27-4b1737100ff2"
/>
Related PR: #14856, @tmillr
I don't have any background knowledge of those `propagate_error`,
@sgvictorino you may want to review this.
# User-Facing Changes
# Tests + Formatting
# After Submitting
# Description
This PR fixes#14816 , so that expression-contains-position check won't
need special treatment for keyword expressions.
e.g.
```nushell
overlay use foo as bar
# |_______ cursor here
if true { } else { }
# |_______ here
```
as mentioned in #14924
# User-Facing Changes
# Tests + Formatting
# After Submitting
# Description
This PR replaces most of the constants in `ResolvedImportPattern` from
values to VarIds, this has benefits of:
1. less duplicated variables in state
2. precise span of variable, for example when calling `goto def` on a
const imported by the `use` command, this allows it to find the original
definition, instead of where the `use` command is.
Note that the logic is different here for nested submodules, not all
values are flattened and propagated to the outmost record variable, but
I didn't find any differences in real world usage.
I noticed that it was changed from `VarId` to `Value` in #10049.
Maybe @kubouch can find some edge cases where this PR fails to work as
expected.
In my view, the record constants for `ResolvedImportPattern` should even
reduced to single entry, if not able to get rid of.
# User-Facing Changes
# Tests + Formatting
# After Submitting
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
A simple method found by @tmillr to solve [this
issue](https://github.com/nushell/nushell/discussions/14854)
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I didn't find a suitable place in `nu-parser` to add the test case,
placed in `nu-lsp` instead.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes: #14844
The issue occurs when nushell is parsing a value with
`SyntaxShape::Any`, it checks `Duration` and `Filesize` first, then
`Range`. Nushell raises errors too early while parsing
`Duration/Filesize`.
This pr changes the order of parsing to fix the issue.
# User-Facing Changes
The following code should be able to run after this pr
```nushell
let runs = 10;
1..$runs
```
# Tests + Formatting
Added 2 tests, one for filesize, one for duration.
# After Submitting
NaN
# Description
This PR add 2 new operators, `has` and `not-has`. They are basically
`in` and `not-in` with the order of operands swapped.
Motivation for this was the awkward way of searching for rows that
contain an item using `where`
```nushell
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]]
| where ("e" in $it.children)
```
vs
```nushell
[[name, children]; [foo, [a, b, c]], [bar [d, e, f]]]
| where children has "e"
```
# User-Facing Changes
Added `has` and `not-has` operators, mirroring `in` and `not-in`.
# Tests + Formatting
- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib
# After Submitting
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Follow-up PR of #14789
The span of `$it/$in` is set to a 0-width one with start/end pointing to
the start of its scope, mainly for error messages positioning.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR addresses the issue of inconsistent spans of special variables
of `$in/$in`, as discussed in
https://github.com/nushell/nushell/pull/14770#discussion_r1908729364.
Instead of making the `declaration_span` to be Option, which will cause
too many changes that we may want to avoid, this PR set the spans to be
`unknown`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
No
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Related:
- #14329
- #13872
- #8214
# Description & User-Facing Changes
This PR allows enables the following uses, which are all no-op.
```nushell
source null
source-env null
use null
overlay use null
```
The motivation for this change is conditional sourcing of files. For
example, with this change `login.nu` may be deprecated and replaced with
the following code in `config.nu`
```nushell
const login_module = if $nu.is-login { "login.nu" } else { null }
source $login_module
```
# Tests + Formatting
I'm hoping for CI to pass 😄
# After Submitting
Add a part about the conditional sourcing pattern to the website.
This PR should close
1. #10327
1. #13667
1. #13810
1. #14129
# Description
This got reverted https://github.com/nushell/nushell/pull/14606 because
the previous changes only considered space a whitespace and forgot about
tabs. I now added a check for any whitespace, even if it is only those
two that would be relevant.
The added test failed before the changes.
For `#` to start a comment, then it either need to be the first
character of the token or prefixed with ` ` (space).
So now you can do this:
```
~/Projects/nushell> 1..10 | each {echo test#testing } 12/05/2024 05:37:19 PM
╭───┬──────────────╮
│ 0 │ test#testing │
│ 1 │ test#testing │
│ 2 │ test#testing │
│ 3 │ test#testing │
│ 4 │ test#testing │
│ 5 │ test#testing │
│ 6 │ test#testing │
│ 7 │ test#testing │
│ 8 │ test#testing │
│ 9 │ test#testing │
╰───┴──────────────╯
```
# User-Facing Changes
It is a breaking change if anyone expected comments to start in the
middle of a string without any prefixing ` ` (space).
# Tests + Formatting
Did all:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
# After Submitting
I cant see that I need to update anything in [the
documentation](https://github.com/nushell/nushell.github.io) but please
point me in the direction if there is anything.
# Description
Because `and` and `or` are short-circuiting operations in Nushell, they
must be compiled to a sequence that avoids evaluating the RHS if the LHS
is already sufficient to determine the output - i.e., `false` for `and`
and `true` for `or`. I initially implemented this with `branch-if`
instructions, simply returning the RHS if it needed to be evaluated, and
returning the short-circuited boolean value if it did not.
Example for `$a and $b`:
```
0: load-variable %0, var 999 "$a"
1: branch-if %0, 3
2: jump 5
3: load-variable %0, var 1000 "$b" # label(0), from(1:)
4: jump 6
5: load-literal %0, bool(false) # label(1), from(2:)
6: span %0 # label(2), from(4:)
7: return %0
```
Unfortunately, this broke polars, because using `and`/`or` on custom
values is perfectly valid and they're allowed to define that behavior
differently, and the polars plugin uses this for boolean masks. But
without using the `binary-op` instruction, that custom behavior is never
invoked. Additionally, `branch-if` requires a boolean, and custom values
are not booleans. This changes the IR to the following, using the
`match` instruction to check for the specific short-circuit value
instead, and still invoking `binary-op` otherwise:
```
0: load-variable %0, var 125 "$a"
1: match (false), %0, 4
2: load-variable %1, var 124 "$b"
3: binary-op %0, Boolean(And), %1
4: span %0 # label(0), from(1:)
5: return %0
```
I've also renamed `Pattern::Value` to `Pattern::Expression` and added a
proper `Pattern::Value` variant that actually contains a `Value`
instead. I'm still hoping to remove `Pattern::Expression` eventually,
because it's kind of a hack - we don't actually evaluate the expression,
we just match it against a few cases specifically for pattern matching,
and it's one of the cases where AST leaks into IR and I want to remove
all of those cases, because AST should not leak into IR.
Fixes#14518
# User-Facing Changes
- `and` and `or` now support custom values again.
- the IR is actually a little bit cleaner, though it may be a bit
slower; `match` is more complex.
# Tests + Formatting
The existing tests pass, but I didn't add anything new. Unfortunately I
don't think there's anything built-in to trigger this, but maybe some
testcases could be added to polars to test it.
This PR should close
1. #10327
1. #13667
1. #13810
1. #14129
# Description
For `#` to start a comment, then it either need to be the first
character of the token or prefixed with ` ` (space).
So now you can do this:
```
~/Projects/nushell> 1..10 | each {echo test#testing } 12/05/2024 05:37:19 PM
╭───┬──────────────╮
│ 0 │ test#testing │
│ 1 │ test#testing │
│ 2 │ test#testing │
│ 3 │ test#testing │
│ 4 │ test#testing │
│ 5 │ test#testing │
│ 6 │ test#testing │
│ 7 │ test#testing │
│ 8 │ test#testing │
│ 9 │ test#testing │
╰───┴──────────────╯
```
# User-Facing Changes
It is a breaking change if anyone expected comments to start in the
middle of a string without any prefixing ` ` (space).
# Tests + Formatting
Did all:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
# After Submitting
I cant see that I need to update anything in [the
documentation](https://github.com/nushell/nushell.github.io) but please
point me in the direction if there is anything.
# Description
Closes#14521
This PR tweaks the way 64-bit hex numbers are parsed.
### Before
```nushell
❯ 0xffffffffffffffef
Error: nu:🐚:external_command
× External command failed
╭─[entry #1:1:1]
1 │ 0xffffffffffffffef
· ─────────┬────────
· ╰── Command `0xffffffffffffffef` not found
╰────
help: `0xffffffffffffffef` is neither a Nushell built-in or a known external command
```
### After
```nushell
❯ 0xffffffffffffffef
-17
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This PR should close
1. #10327
1. #13667
1. #13810
1. #14129
# Description
For `#` to start a comment, then it either need to be the first
character of the token or prefixed with ` ` (space).
So now you can do this:
```
~/Projects/nushell> 1..10 | each {echo test#testing } 12/05/2024 05:37:19 PM
╭───┬──────────────╮
│ 0 │ test#testing │
│ 1 │ test#testing │
│ 2 │ test#testing │
│ 3 │ test#testing │
│ 4 │ test#testing │
│ 5 │ test#testing │
│ 6 │ test#testing │
│ 7 │ test#testing │
│ 8 │ test#testing │
│ 9 │ test#testing │
╰───┴──────────────╯
```
# User-Facing Changes
It is a breaking change if anyone expected comments to start in the
middle of a string without any prefixing ` ` (space).
# Tests + Formatting
Did all:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
# After Submitting
I cant see that I need to update anything in [the
documentation](https://github.com/nushell/nushell.github.io) but please
point me in the direction if there is anything.
---------
Co-authored-by: Wind <WindSoilder@outlook.com>
- This PR should fix/close:
- #11266
- #12893
- #13736
- #13748
- #14170
- It doesn't fix#13736 though unfortunately. The issue there is at a
different level to this fix (I think probably in the lexing somewhere,
which I haven't touched).
# The Problem
The linked issues have many examples of the problem and the related
confusion it causes, but I'll give some more examples here for
illustration. It boils down to the following:
This doesn't type check (good):
```nu
def foo []: string -> int { false }
```
This does (bad):
```nu
def foo [] : string -> int { false }
```
Because the parser is completely ignoring all the characters. This also
compiles in 0.100.0:
```nu
def blue [] Da ba Dee da Ba da { false }
```
And this also means commands which have a completely fine type, but an
extra space before `:`, lose that type information and end up as `any ->
any`, e.g.
```nu
def foo [] : int -> int {$in + 3}
```
```bash
$ foo --help
Input/output types:
╭───┬───────┬────────╮
│ # │ input │ output │
├───┼───────┼────────┤
│ 0 │ any │ any │
╰───┴───────┴────────╯
```
# The Fix
Special thank you to @texastoland whose draft PR (#12358) I referenced
heavily while making this fix.
That PR seeks to fix the invalid parsing by disallowing whitespace
between `[]` and `:` in declarations, e.g. `def foo [] : int -> any {}`
This PR instead allows the whitespace while properly parsing the type
signature. I think this is the better choice for a few reasons:
- The parsing is still straightforward and the information is all there
anyway,
- It's more consistent with type annotations in other places, e.g. `do
{|nums : list<int>| $nums | describe} [ 1 2 3 ]` from the [Type
Signatures doc
page](https://www.nushell.sh/lang-guide/chapters/types/type_signatures.html)
- It's more consistent with the new nu parser, which allows `let x :
bool = false` (current nu doesn't, but this PR doesn't change that)
- It will be less disruptive and should only break code where the types
are actually wrong (if your types were correct, but you had a space
before the `:`, those declarations will still compile and now have more
type information vs. throwing an error in all cases and requiring spaces
to be deleted)
- It's the more intuitive syntax for most functional programmers like
myself (haskell/lean/coq/agda and many more either allow or require
whitespace for type annotations)
I don't use Rust a lot, so I tried to keep most things the same and the
rest I wrote as if it was Haskell (if you squint a bit). Code
review/suggestions very welcome. I added all the tests I could think of
and `toolkit check pr` gives it the all-clear.
# User-Facing Changes
This PR meets part of the goal of #13849, but doesn't do anything about
parsing signatures twice and doesn't do much to improve error messages,
it just enforces the existing errors and error messages.
This will no doubt be a breaking change, mostly because the code is
already broken and users don't realise yet (one of my personal scripts
stopped compiling after this fix because I thought `def foo [] -> string
{}` was valid syntax). It shouldn't break any type-correct code though.
# Description
I always wondered why the module env vars `CURRENT_FILE`, `FILE_PWD`,
`PROCESS_PATH` weren't available in the source command. I tried to add
them here. I think it could be helpful but I'm not sure. I'm also not
sure this hack is what we should do but I thought I'd put it out there
for fun.
Thoughts?
### Run Module (works as it did before)
```nushell
❯ open test_module.nu
def main [] {
print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
print $"$env.FILE_PWD = ($env.FILE_PWD?)"
print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"
}
❯ nu test_module.nu
$env.CURRENT_FILE = /Users/fdncred/src/nushell/test_module.nu
$env.FILE_PWD = /Users/fdncred/src/nushell
$env.PROCESS_PATH = test_module.nu
```
### Use Module (works as it did before)
```nushell
❯ open test_module2.nu
export-env {
print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
print $"$env.FILE_PWD = ($env.FILE_PWD?)"
print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"
}
❯ use test_module2.nu
$env.CURRENT_FILE = /Users/fdncred/src/nushell/test_module.nu
$env.FILE_PWD = /Users/fdncred/src/nushell
$env.PROCESS_PATH =
```
### Sourced non-module script (this is the new part)
> [!NOTE]
> Note: We intentionally left out PROCESS_PATH since it's supposed to
> to work like argv[0] in C, which is the name of the program being
executed.
> Since we're not executing a program, we don't need to set it.
```nushell
❯ open test_source.nu
print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
print $"$env.FILE_PWD = ($env.FILE_PWD?)"
print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"
❯ source test_source.nu
$env.CURRENT_FILE = /Users/fdncred/src/nushell/test_source.nu
$env.FILE_PWD = /Users/fdncred/src/nushell
$env.PROCESS_PATH =
```
Also, what is PROCESS_PATH even supposed to be?
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The [nushell/demo](https://github.com/nushell/demo) project successfully
demonstrated running Nushell in the browser using WASM. However, the
current version of Nushell cannot be easily built for the
`wasm32-unknown-unknown` target, the default for `wasm-bindgen`.
This PR introduces initial support for the `wasm32-unknown-unknown`
target by disabling OS-dependent features such as filesystem access, IO,
and platform/system-specific functionality. This separation is achieved
using a new `os` feature in the following crates:
- `nu-cmd-lang`
- `nu-command`
- `nu-engine`
- `nu-protocol`
The `os` feature includes all functionality that interacts with an
operating system. It is enabled by default, but can be disabled using
`--no-default-features`. All crates that depend on these core crates now
use `--no-default-features` to allow compilation for WASM.
To demonstrate compatibility, the following script builds all crates
expected to work with WASM. Direct user interaction, running external
commands, working with plugins, and features requiring `openssl` are out
of scope for now due to their complexity or reliance on C libraries,
which are difficult to compile and link in a WASM environment.
```nushell
[ # compatible crates
"nu-cmd-base",
"nu-cmd-extra",
"nu-cmd-lang",
"nu-color-config",
"nu-command",
"nu-derive-value",
"nu-engine",
"nu-glob",
"nu-json",
"nu-parser",
"nu-path",
"nu-pretty-hex",
"nu-protocol",
"nu-std",
"nu-system",
"nu-table",
"nu-term-grid",
"nu-utils",
"nuon"
] | each {cargo build -p $in --target wasm32-unknown-unknown --no-default-features}
```
## Caveats
This PR has a few caveats:
1. **`miette` and `terminal-size` Dependency Issue**
`miette` depends on `terminal-size`, which uses `rustix` when the target
is not Windows. However, `rustix` requires `std::os::unix`, which is
unavailable in WASM. To address this, I opened a
[PR](https://github.com/eminence/terminal-size/pull/68) for
`terminal-size` to conditionally compile `rustix` only when the target
is Unix. For now, the `Cargo.toml` includes patches to:
- Use my forked version of `terminal-size`.
- ~~Use an unreleased version of `miette` that depends on
`terminal-size@0.4`.~~
These patches are temporary and can be removed once the upstream changes
are merged and released.
2. **Test Output Adjustments**
Due to the slight bump in the `miette` version, one test required
adjustments to accommodate minor formatting changes in the error output,
such as shifted newlines.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This shouldn't break anything but allows using some crates for targeting
`wasm32-unknown-unknown` to revive the demo page eventually.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
I did not add any extra tests, I just checked that compiling works, also
when using the host target but unselecting the `os` feature.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
~~Breaking the wasm support can be easily done by adding some `use`s or
by adding a new dependency, we should definitely add some CI that also
at least builds against wasm to make sure that building for it keep
working.~~
I added a job to build wasm.
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
# Description
While reviewing #14388, I think we can make some improvement on parser.
For the following code:
```nushell
let a = 3
a = 10 # should be error
$a = 10 # another error
```
I think they can raise `ParseError`, so nushell doesn't need to move
forward compiling IR block.
# User-Facing Changes
```nushell
let a = 3
a = 10
```
Will raise parse error instead of compile error.
# Tests + Formatting
Added 1 test.
# Description
Adds a new `Filesize` type so that `FromValue` can be used to convert a
`Value::Filesize` to a `Filesize`. Currently, to extract a filesize from
a `Value` using `FromValue`, you have to extract an `i64` which coerces
`Value::Int`, `Value::Duration`, and `Value::Filesize` to an `i64`.
Having a separate type also allows us to enforce checked math to catch
overflows. Similarly, it allows us to specify other trait
implementations like `Display` in a common place.
# User-Facing Changes
Multiplication with filesizes now error on overflow. Should not be a
breaking change for plugins (i.e., serialization) since `Filesize` is
marked with `serde(transparent)`.
# Tests + Formatting
Updated some tests.
# Description
This removes the need for the `shape_and` and `shape_or` entries in the
themes. We did not color those underlying FlatShapes or operators
differently.
Closes#14372
# User-Facing Changes
Our theme handling currently doesn't reject invalid entries so should
not cause an error. The non-functional nature was already documented.
# Description
List values and list streams have the same type (`list<>`). Rather,
streaming is a separate property of the pipeline/command output. This PR
removes the unnecessary `ListStream` type.
# User-Facing Changes
Should be none, except `random dice` now has a more specific output
type.
# Description
The "append" operator currently serves as both the append operator and
the concatenation operator. This dual role creates ambiguity when
operating on nested lists.
```nu
[1 2] ++ 3 # appends a value to a list [1 2 3]
[1 2] ++ [3 4] # concatenates two lists [1 2 3 4]
[[1 2] [3 4]] ++ [5 6]
# does this give [[1 2] [3 4] [5 6]]
# or [[1 2] [3 4] 5 6]
```
Another problem is that `++=` can change the type of a variable:
```nu
mut str = 'hello '
$str ++= ['world']
($str | describe) == list<string>
```
Note that appending is only relevant for lists, but concatenation is
relevant for lists, strings, and binary values. Additionally, appending
can be expressed in terms of concatenation (see example below). So, this
PR changes the `++` operator to only perform concatenation.
# User-Facing Changes
Using the `++` operator with a list and a non-list value will now be a
compile time or runtime error.
```nu
mut list = []
$list ++= 1 # error
```
Instead, concatenate a list with one element:
```nu
$list ++= [1]
```
Or use `append`:
```nu
$list = $list | append 1
```
# After Submitting
Update book and docs.
---------
Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com>
