Commit Graph

6326 Commits

Author SHA1 Message Date
Yash Thakur
6d36941e55
Add completions.sort option (#13311) 2024-08-05 20:30:10 -04:00
Jack Wright
2f44801414
Adding plist support (#13545)
# Description
Provides the ability convert from and to plist format.

<img width="1250" alt="Screenshot 2024-08-05 at 10 21 26"
src="https://github.com/user-attachments/assets/970f3366-eb70-4d74-a396-649374556f66">

<img width="730" alt="Screenshot 2024-08-05 at 10 22 38"
src="https://github.com/user-attachments/assets/6ec317d0-686e-47c6-bf35-8ab6e5d802db">

# User-Facing Changes
- Introduction of `from plist` command
- Introduction of `to plist`command

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-08-05 14:07:15 -07:00
Stefan Holderbach
9172b22985
Rework help generation internals (#13531)
Reworking some of the sprawling code we use to generate the `help cmd`
or `cmd --help` output.

This touches mainly the rendering and not the gathering of the necessary
data (see open bugs under
[label:help-system](https://github.com/nushell/nushell/issues?q=sort%3Aupdated-desc+is%3Aopen+label%3Ahelp-system))

Fixes #9076 
Fixes the syntax shape output on flags to be consistent.

## Example
```nushell
def test [
  positional: int,
  documented: float, # this has documentation
  default = 50,
  optional?,
  --a = "bla",
  --bla (-b) = "bla", # named with default
] {}
```

### before

![grafik](https://github.com/user-attachments/assets/1867984f-1289-4ad0-bdf5-c49ec56dfddb)


### after


![grafik](https://github.com/user-attachments/assets/8fca526f-d878-4d52-b970-fc41c7e8859c)
2024-08-05 22:44:24 +02:00
Andrej Kolchin
1c37f4b958
Create random binary command (#13542)
# Description/User-Facing Changes

Creates a new `random binary <LENGTH>` command, which returns random
bytes.

Resolve #13500
2024-08-05 21:07:12 +02:00
James Chen-Smith
b974f8f7e3
Include empty table data cells in query web tables (#13538)
# Description

Empty cells were being skipped, causing data to appear in the wrong
columns. By including the cells, data should appear in the correct
columns now. Fixes #10194.

Before:

```
$ [[a b c]; [1 null 3] [4 5 6]] | to html --partial | query web --as-table [a b c]
╭───┬───┬───┬─────────────────────╮
│ # │ a │ b │          c          │
├───┼───┼───┼─────────────────────┤
│ 0 │ 1 │ 3 │ Missing column: 'c' │
│ 1 │ 4 │ 5 │ 6                   │
╰───┴───┴───┴─────────────────────╯
```

After:

```
$ [[a b c]; [1 null 3] [4 5 6]] | to html --partial | query web --as-table [a b c]
╭───┬───┬───┬───╮
│ # │ a │ b │ c │
├───┼───┼───┼───┤
│ 0 │ 1 │   │ 3 │
│ 1 │ 4 │ 5 │ 6 │
╰───┴───┴───┴───╯
```

Co-authored-by: James Chen-Smith <jameschensmith@gmail.com>
2024-08-05 06:20:14 -05:00
Himadri Bhattacharjee
802bfed173
feat: prefer exact match when completion mode is prefix (#13302)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Fixes #13204

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

When the completion mode is set to `prefix`, path completions explicitly
check for and prefer an exact match for a basename instead of longer or
similar names.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Exact match is inactive since there's no trailing slash

```
~/Public/nushell| ls crates/nu-plugin<tab>
crates/nu-plugin/               crates/nu-plugin-core/          crates/nu-plugin-engine/        crates/nu-plugin-protocol/      
crates/nu-plugin-test-support/
```

Exact match is active

```
~/Public/nushell| ls crates/nu-plugin/<tab>
crates/nu-plugin/Cargo.toml  crates/nu-plugin/LICENSE     crates/nu-plugin/README.md   crates/nu-plugin/src/
```

Fuzzy matching persists its behavior

```
~/Public/nushell> $env.config.completions.algorithm = "fuzzy";
~/Public/nushell| ls crates/nu-plugin/car
crates/nu-cmd-plugin/Cargo.toml           crates/nu-plugin/Cargo.toml               crates/nu-plugin-core/Cargo.toml          crates/nu-plugin-engine/Cargo.toml        
crates/nu-plugin-protocol/Cargo.toml      crates/nu-plugin-test-support/Cargo.toml
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-04 06:06:10 -05:00
Stefan Holderbach
07e7c8c81f
Fixup #13526 width flag for example (#13529)
# Description
This seems to be a minor copy paste mistake. cc @Embers-of-the-Fire

Followup to #13526

# User-Facing Changes
(-)

# Tests + Formatting
(-)
2024-08-03 16:42:30 +02:00
Embers-of-the-Fire
20b53067cd
Fix overflow table display in command documentation (#13526)
# Description

Check and set table width beforehand.

Closes #13520.

# User-Facing Changes
Before:
```plain
❯ help net cmd1
network

Usage:
  > net cmd1

Flags:
  -h, --help - Display the help message for this command

Input/output types:
  ╭───┬─────────┬─────────────────────────────────────────────────────────╮
  │ # │  input  │                         output                          │
  ├───┼─────────┼─────────────────────────────────────────────────────────┤
  │ 0 │ nothing │ table<name: string, description: string, mac: string,   │
  │   │         │ ips: table<type: string, addr: string, prefix: int>,
 │
  │   │         │ flags: record<is_up: bool, is_broadcast: bool,
 │
  │   │         │ is_loopback: bool, is_point_to_point: bool,
 │
  │   │         │ is_multicast: bool>>
 │
  ╰───┴─────────┴─────────────────────────────────────────────────────────╯
```

After:
```plain
❯ help net cmd1
network

Usage:
  > net cmd1

Flags:
  -h, --help - Display the help message for this command

Input/output types:
  ╭───┬─────────┬───────────────────────────────────────────────────────╮
  │ # │  input  │                        output                         │
  ├───┼─────────┼───────────────────────────────────────────────────────┤
  │ 0 │ nothing │ table<name: string, description: string, mac: string, │
  │   │         │  ips: table<type: string, addr: string, prefix: int>, │
  │   │         │  flags: record<is_up: bool, is_broadcast: bool,       │
  │   │         │ is_loopback: bool, is_point_to_point: bool,           │
  │   │         │ is_multicast: bool>>                                  │
  ╰───┴─────────┴───────────────────────────────────────────────────────╯
```

# Tests + Formatting

- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [x] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [x] `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library


# After Submitting
- [x] Bug fix, no doc update.
2024-08-03 10:55:35 +02:00
Ian Manske
f4c0d9d45b
Path migration part 4: various tests (#13373)
# Description
Part 4 of replacing std::path types with nu_path types added in
https://github.com/nushell/nushell/pull/13115. This PR migrates various
tests throughout the code base.
2024-08-03 10:09:13 +02:00
Stefan Holderbach
85b06b22d9
Replace manual Record::get implementation (#13525)
Let's simplify here
2024-08-03 01:14:44 +02:00
Stefan Holderbach
63f00e78d1
Lift SharedCow::to_mut out of if let branches (#13524)
In some `if let`s we ran the `SharedCow::to_mut` for the test and to get
access to a mutable reference in the happy path. Internally
`Arc::into_mut` has to read atomics and if necessary clone.
For else branches, where we still want to modify the record we
previously called this again (not just in rust, confirmed in the asm).

This would have introduced a `call` instruction and its cost (even if it
would be guaranteed to take the short path in `Arc::into_mut`).
Lifting it get's rid of this.
2024-08-03 00:26:48 +02:00
Stefan Holderbach
ff1ad77130
Simplify column look-up in default (#13522)
# Description
Since we make the promise that record keys/columns are exclusice we
don't have to go through all columns after we have found the first one.
Should permit some short-circuiting if the column is found early.

# User-Facing Changes
(-)

# Tests + Formatting
(-)
2024-08-03 00:26:35 +02:00
Embers-of-the-Fire
af34d5c062
Fix internal panic for query web (#13507)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Original implementation contains multiple `expect` which can cause
internal runtime panic.

This pr forks the `css` selector impl and make it return an error that
nushell can recognize.

**Note:** The original impl is still used in pre-defined selector
implementations, but they
should never fail, so the `css` fn is preserved.

Closes #13496.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Now `query web` will not panic when the `query` parameter is not given
or has syntax error.

```plain
❯ .\target\debug\nu.exe -c "http get https://www.rust-lang.org | query web"
Error:   × CSS query parse error
   ╭─[source:1:38]
 1 │ http get https://www.rust-lang.org | query web
   ·                                      ────┬────
   ·                                          ╰─┤ Unexpected error occurred. Please report this to the developer
   ·                                            │ EmptySelector
   ╰────
  help: cannot parse query as a valid CSS selector
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:
-->
- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [x] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [x] `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
<!--
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
- [x] Impl change, no doc update.

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-08-02 21:47:18 +02:00
NotTheDr01ds
ed82f9ee18
Clarify default command help (#13519)
# Description

Updates `default` command description to be more clear and adds an
example for a missing values in a list-of-records.

# User-Facing Changes

Help/doc only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- Update `nothing` doc in Book to reference `default` per
https://github.com/nushell/nushell.github.io/issues/1073 - This was a
bit of a rabbit trail on the path to that update. ;-)

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-08-02 21:23:25 +02:00
Jack Wright
d081e3386f
Make pipeline metadata available to plugins (#13495)
# Description
Fixes an issue with pipeline metadata not being passed to plugins.
2024-08-02 11:01:20 -07:00
NotTheDr01ds
ca8eb856e8
Doc and examples for multi-dot directory traversal (#13513)
# Description

With this PR, we should be able to close
https://github.com/nushell/nushell.github.io/issues/1225

Help/doc/examples updated for:

* `cd` to show multi-dot traversal
* `cd` to show implicit `cd` with bare directory path
* Fixed/clarified another example that mentioned `$OLDPATH` while I was
in there
* `mv` and `cp` examples for multi-dot traversal
* Updated `cp` examples to use more consistent (and clear) filenames

# User-Facing Changes

Help/doc only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-08-01 15:22:25 -05:00
NotTheDr01ds
168835ecd2
random chars doc clarifications (#13511)
# Description

Clarified `random chars` help/doc:

* Default string length in absence of a `--length` arg is 25
* Characters are *"uniformly distributed over ASCII letters and numbers:
a-z, A-Z and 0-9"* (copied from the [`rand` crate
doc](https://docs.rs/rand/latest/rand/distributions/struct.Alphanumeric.html).

# User-Facing Changes

Help/Doc only
2024-08-01 21:21:39 +02:00
Piotr Kufel
4157ca711d
Factor out style-setting code (#13406)
# Description
This is mainly cleanup, but introduces a slight (positive, if anything)
behavior change:

Some menu layouts support only a subset of styles, but with this change
the user will still be able to configure them. This seems strictly
better - if reedline starts supporting one of the existing styles for a
particular layout, there won't be any need to update nushell code.


# User-Facing Changes
None

# Tests + Formatting
This is simply factoring out existing code, old tests should still cover
it.

---------

Co-authored-by: sholderbach <sholderbach@users.noreply.github.com>
2024-08-01 11:17:58 +02:00
Jack Wright
d34a24db33
setting content type metadata on all core to * commands (#13506)
# Description

All core `to *` set content type pipeline metadata. 

# User-Facing Changes
- For consistency, `from json` no longer sets the content type metadata

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib
2024-08-01 11:10:52 +02:00
Stefan Holderbach
2c6b1471e1
Contentious clippy fixes (#13498)
Lints from stable or nightly toolchain that may have questionable added
value.

- **Contentious lint to contract into single `if let`**
- **Potential false positive around `AsRef`/`Deref` fun**
2024-08-01 11:02:55 +02:00
Ian Manske
ae5fed41ed
Path migration part 3: $nu paths (#13368)
# Description
Part 3 of replacing `std::path` types with `nu_path` types added in
#13115. This PR targets the paths listed in `$nu`. That is, the home,
config, data, and cache directories.
2024-08-01 10:16:31 +02:00
Embers-of-the-Fire
ca73d85c09
Add --upgrade switch for mv command (#13505)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Add `--upgrade, -u` switch for `mv` command, corresponding to `cp`.

Closes #13458.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
```plain
❯ help mv | find update
╭──────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│        0 │   -u, --update - move and overwite only when the SOURCE file is newer than the destination file or when the destination file is missing       │
╰──────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:
-->
- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [x] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [x] `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
<!--
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

P.S.  
The standard test kit (`nu-test-support`) doesn't provide utility to
create file with modification timestamp, and I didn't find any test for
this in `cp` command. I had tested on my local machine but I'm not sure
how to integrate it into ci. If unit testing is required, I may need
your guidance.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
- [x] Command docs are auto generated.
2024-07-31 20:43:06 -05:00
Bruce Weirdan
f82c43f850
Consider numbers to be part of a word in split words (#13502)
# Description

Before this change, `"hash sha256 123 ok" | split words` would return
`[hash sha ok]` - which is surprising to say the least.

Now it will return `[hash sha256 123 ok]`.

Refs:
https://discord.com/channels/601130461678272522/615253963645911060/1268151658572025856

# User-Facing Changes
`split words` will no longer remove digits.

# Tests + Formatting
Added a test for this specific case.

# After Submitting
2024-07-31 16:35:41 -05:00
NotTheDr01ds
3dc9691aaa
Clarify random int help (#13503)
# Description

Minor nitpicks, but the `random int` help and examples were a bit
ambiguous in several aspects. Updated to clarify that:

* An unconstrained `random int` returns a non-negative integer. That is,
the smallest potential value is 0. Technically integers include negative
numbers as well, and `random int` will never return one unless you pass
it a range with a negative lower-bound.

* To that end, changed the final example to demonstrate a negative
lower-bound.

* The range is inclusive. While most people would probably assume this,
the changes make this explicit in the examples and argument description.

# User-Facing Changes

Help only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib

# After Submitting

N/A
2024-07-31 16:34:38 -05:00
Stefan Holderbach
42531e017c
Clippy fixes from stable and nightly (#13455)
- **Doccomment style fixes**
- **Forgotten stuff in `nu-pretty-hex`**
- **Don't `for` around an `Option`**
- and more

I think the suggestions here are a net positive, some of the suggestions
moved into #13498 feel somewhat arbitrary, I also raised
https://github.com/rust-lang/rust-clippy/issues/13188 as the nightly
`byte_char_slices` would require either a global allow or otherwise a
ton of granular allows or possibly confusing bytestring literals.
2024-07-31 20:37:40 +02:00
Wind
928c57db41
save: print to stderr for bytestream (#13422)
# Description
Fixes: #13260 

When user run a command like this:
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```

`save` command sinks the output of previous commands' stderr output. I
think it should be `stderr`.

# User-Facing Changes
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```
The command will output ` New Err` to stderr.

# Tests + Formatting
Added 2 cases.
2024-07-31 18:19:35 +02:00
Stefan Holderbach
813aac89bd
Clippy fixes for toolchain bump (#13497)
- **Suggested default impl for the new `*Stack`s**
- **Change a hashmap to make clippy happy**
- **Clone from fix**
- **Fix conditional unused in test**
- then **Bump rust toolchain**
2024-07-31 14:49:22 +02:00
dependabot[bot]
d2bf82d22b
Bump similar from 2.5.0 to 2.6.0 (#13492) 2024-07-31 08:10:33 +00:00
Devyn Cairns
8e2917b9ae
Make assignment and const consistent with let/mut (#13385)
# Description

This makes assignment operations and `const` behave the same way `let`
and `mut` do, absorbing the rest of the pipeline.

Changes the lexer to be able to recognize assignment operators as a
separate token, and then makes the lite parser continue to push spans
into the same command regardless of any redirections or pipes if an
assignment operator is encountered. Because the pipeline is no longer
split up by the lite parser at this point, it's trivial to just parse
the right hand side as if it were a subexpression not contained within
parentheses.

# User-Facing Changes
Big breaking change. These are all now possible:

```nushell
const path = 'a' | path join 'b'

mut x = 2
$x = random int
$x = [1 2 3] | math sum

$env.FOO = random chars
```

In the past, these would have led to (an attempt at) bare word string
parsing. So while `$env.FOO = bar` would have previously set the
environment variable `FOO` to the string `"bar"`, it now tries to run
the command named `bar`, hence the major breaking change.

However, this is desirable because it is very consistent - if you see
the `=`, you can just assume it absorbs everything else to the right of
it.

# Tests + Formatting
Added tests for the new behaviour. Adjusted some existing tests that
depended on the right hand side of assignments being parsed as
barewords.

# After Submitting
- [ ] release notes (breaking change!)
2024-07-30 18:55:22 -05:00
Darren Schroeder
ea22c319b6
make math sqrt const (#13487)
# Description

Just a quick PR to demonstrate one way to make commands const.
related to https://github.com/nushell/nushell/issues/13482

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-30 08:53:41 -05:00
Maxim Uvarov
7432e67da1
don't force stripping ansi codes from strings in stor (#13464)
# Description

The current version of `stor` forces stripping ansi code coloring from
all the strings.
In this PR, I propose to keep strings unchanged.

The logic behind the proposed changes was best described in the
[discord](https://discord.com/channels/601130461678272522/601130461678272524/1266387441074442272):
<img width="773" alt="image"
src="https://github.com/user-attachments/assets/063cdebd-684f-46f1-aca1-faeb4827723d">

with proposed changes we can store colored output:
```
stor reset; stor create --table-name test --columns {a: str};
ls | table | {a: $in} | stor insert --table-name test | null;
stor open | query db 'select a from test' | get a.0 
```

<img width="704" alt="image"
src="https://github.com/user-attachments/assets/8f062808-18fc-498b-a77e-a118f6b9953a">


# User-Facing Changes

If one was using `stor` together with ansi colored input, and then was
querying `stor` with search operations, they might break. But I don't
think that it is a big problem, as one will just need to use `ansi
reset` before storing data.

# Tests + Formatting

Tests are okay.

Thanks to @fdncred for spending time to help me write these changes 🙏
2024-07-30 08:52:28 -05:00
Bahex
0576794e74
reduce: supply <acc> to the closure as pipeline input as well (#13461)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

resolve #13459

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Make `reduce` supply the accumulator value to closure as pipeline input.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Mostly described in #13459

- Should not be a breaking change.
- Allows cleaner patterns like `... | reduce {|it| merge $it}`

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
`cargo test --package nu-cli --test main -- commands::reduce` and
`toolkit test stdlib` report no issues
2024-07-30 08:51:51 -05:00
suimong
12f57dbc62
Add "--as-columns" flag to polars into-df (#13449)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Per discussion on
[Discord](https://discord.com/channels/601130461678272522/864228801851949077/1265718178927870045)
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

To facilitate column-oriented dataframe construction, this PR added a
`--as-columns` flag to `polars into-df` command so that when specified,
and when input shape is record of lists, each list will be treated as a
column rather than a cell value, i.e. `{a: [1 3], b: [2 4]} | polars
into-df --as-columns` returns the same dataframe as `[[a b];[1 2] [3 4]]
| polars into-df`


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

A new flag `--as-columns`, no change of semantics if this flag is
unspecified.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Ben Yang <ben@ya.ng>
2024-07-30 08:50:50 -05:00
Bark
fe57c5c22e
fix: Make log respect use_ansi_coloring setting. (#13442)
# Very briefly
Fixes: #13317 
- Ignore ansi coloring on logs if this setting is true.
- Add a reset after the default left prompt (before prompt character)
which fixes all-red text when `use_ansi_coloring` is false.

# Description

## Firstly,
argumentation about the changes to `crates/nu-std/std/log.nu`

Previous behavior colored the output of all log, even when the setting
`use_ansi_coloring` was false.

![image](https://github.com/user-attachments/assets/a82991c4-ff46-455d-8dac-248de2456d78)

Current behavior honors the setting.

![image](https://github.com/user-attachments/assets/6d5365db-e05d-4d2a-8981-f22303dff081)

## Second,
While testing different scenarios, I found out that the default setting
on both (`0.95`, arch linux) and the source (`0.96`) all text was
displayed in red (the color used for the present-working-directory part
of the prompt) after setting `use_ansi_coloring` to `false` ([comment
with picture of the issue and reproduction
steps](https://github.com/nushell/nushell/issues/13317#issuecomment-2247439894)).
To which my response was adding a `(ansi reset)` at the end of the
directory part of the prompt in the default config
(`crates/nu-utils/src/sample_config/default_env.nu`) file. All later
parts follow the `use_ansi_coloring` setting and their assigned colors.

# User-Facing Changes
I would say the color, but don't know if that counts as "user-facing".

# Tests + Formatting
- Formatting was applied as advised.
- 1314 tests passed and 24 ignored, none failed.
- Clippy  did not pass due to an error on the following files:
`crates/nu-protocol/src/engine/argument.rs:81:5` and
`crates/nu-protocol/src/engine/error_handler.rs:19:5`
throwing the error `you should consider adding a 'Default'
implementation for 'ErrorHandlerStack'`.
As those files are out of the scope of the current issue, they have
**not** been changed.
2024-07-30 08:34:11 -05:00
dependabot[bot]
18161e5707
Bump shadow-rs from 0.29.0 to 0.30.0 (#13436) 2024-07-30 13:33:21 +00:00
Piotr Kufel
466b3899e0
Use the Default implementation of Suggestion (#13409)
# Description

Take advantage of the `Default` implementation of `Suggestion`. This in
particular should make code compatible forward-compatible with
https://github.com/nushell/reedline/pull/798.

# User-Facing Changes
None

# Tests + Formatting

Existing coverage.
2024-07-30 08:32:40 -05:00
Andy Gayton
7b82c6b482
feat: make ctrlc available to plugins (#13181)
# Description

This PR adds a new method to `EngineInterface`: `register_ctrlc_handler`
which takes a closure to run when the plugin's driving engine receives a
ctrlc-signal. It also adds a mirror of the `signals` attribute from the
main shell `EngineState`.

This is an example of how a plugin which makes a long poll http request
can end the request on ctrlc:
https://github.com/cablehead/nu_plugin_http/blob/main/src/commands/request.rs#L68-L77

To facilitate the feature, a new attribute has been added to
`EngineState`: `ctrlc_handlers`. This is a Vec of closures that will be
run when the engine's process receives a ctrlc signal.

When plugins are added to an `engine_state` during a `merge_delta`, the
engine passes the ctrlc_handlers to the plugin's
`.configure_ctrlc_handler` method, which gives the plugin a chance to
register a handler that sends a ctrlc packet through the
`PluginInterface`, if an instance of the plugin is currently running.

On the plugin side: `EngineInterface` also has a ctrlc_handlers Vec of
closures. Plugin calls can use `register_ctrlc_handler` to register a
closure that will be called in the plugin process when the
PluginInput::Ctrlc command is received.

For future reference these are some alternate places that were
investigated for tying the ctrlc trigger to transmitting a Ctrlc packet
through the `PluginInterface`:

- Directly from `src/signals.rs`: the handler there would need a
reference to the Vec<Arc<RegisteredPlugins>>, which would require us to
wrap the plugins in a Mutex, which we don't want to do.

- have `PersistentPlugin.get_plugin` pass down the engine's
CtrlcHandlers to .get and then to .spawn (if the plugin isn't already
running). Once we have CtrlcHandlers in spawn, we can register a handler
to write directly to PluginInterface. We don't want to double down on
passing engine_state to spawn this way though, as it's unpredictable
because it would depend on whether the plugin has already been spawned
or not.

- pass `ctrlc_handlers` to PersistentPlugin::new so it can store it on
itself so it's available to spawn.

- in `PersistentPlugin.spawn`, create a handler that sends to a clone of
the GC event loop's tx. this has the same issues with regards to how to
get CtrlcHandlers to the spawn method, and is more complicated than a
handler that writes directly to PluginInterface

# User-Facing Changes

No breaking changes

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-07-30 08:29:18 -05:00
Lee Wonjoon
e3f78b8793
Keep forward slash when autocomplete on Windows (#13321)
Related #7044 

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

When autocomplete path with `/` on Windows, paths keep with slash
instead of backslash(`\`).

If mixed both, path completion uses a last path seperator.


![image](https://github.com/nushell/nushell/assets/4014016/b09633fd-0e4a-4cd9-9c14-2bca30625804)


![image](https://github.com/nushell/nushell/assets/4014016/e1f228f8-4cce-43eb-a34a-dfa54efd2ebb)


![image](https://github.com/nushell/nushell/assets/4014016/0694443a-3017-4828-be60-5f39ffd96440)

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->


![completion](https://github.com/nushell/nushell/assets/4014016/03626544-6a14-4d8b-a607-21a4472f8037)

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-30 08:28:41 -05:00
Devyn Cairns
c31291753c
Bump version to 0.96.2 (#13485)
This should be the new development version. We most likely don't need a
0.96.2 patch release. Should be free to merge PRs after this.
2024-07-29 17:20:55 -07:00
Devyn Cairns
d618fd0527
Fix bad method links in docstrings (#13471)
# Description

Seems like I developed a bit of a bad habit of trying to link

```rust
/// [`.foo()`]
```

in docstrings, and this just doesn't work automatically; you have to do 

```rust
/// [`.foo()`](Self::foo)
```

if you want it to actually link. I think I found and replaced all of
these.

# User-Facing Changes

Just docs.
2024-07-27 19:39:29 -07:00
Devyn Cairns
d80de68665
Clean up arguments added to stack after CallDecl engine call (#13469)
# Description

Just realized I hadn't been cleaning up the arguments added to the
`Stack` after the `CallDecl` engine call was finished, so there could be
a bit of a memory leak if a plugin made many calls during the duration
of a single plugin call. This is a quick patch to that.

I'm probably going to revise how this all works at some point soon
because I think it is a bit of a pitfall. It would be good to make it
much more difficult to make a mistake with it, perhaps with a guard like
Ian did for the redirection stuff.

# After Submitting
- [ ] release with 0.96.1
2024-07-27 19:39:17 -07:00
Devyn Cairns
5f7afafe51
IR: fix incorrect capturing of subexpressions (#13467)
# Description


[Discovered](https://discord.com/channels/601130461678272522/614593951969574961/1266503282554179604)
by `@warp` on Discord.

The IR compiler was not properly setting redirect modes for
subexpressions because `FullCellPath` was always being compiled with
capture-out redirection. This is the correct behavior if there is a tail
to the `FullCellPath`, as we need the value in order to try to extract
anything from it (although this is unlikely to work) - however, the
parser also generates `FullCellPath`s with an empty tail quite often,
including for bare subexpressions.

Because of this, the following did not behave as expected:

```nushell
(docker run -it --rm alpine)
```

Capturing the output meant that `docker` didn't have direct access to
the terminal as a TTY.

As this is a minor bug fix, it should be okay to include in the 0.96.1
patch release.

# User-Facing Changes

- Fixes the bug as described when running with IR evaluation enabled.

# Tests + Formatting

I added a test for this, though we're not currently running all tests
with IR on the CI, but it should ensure this behaviour is consistent.
The equivalent minimum repro I could find was:

```nushell
(nu --testbin cococo); null
```

as this should cause the `cococo` message to appear on stdout, and if
Nushell is capturing the output, it would be discarded instead.
2024-07-27 19:38:57 -07:00
Devyn Cairns
53fbf62493
Fix keybindings list being empty by default (#13456)
# Description

Made a mistake when fixing this for IR. The default behavior with no
options set is to list everything. Restored that.

This should go in the 0.96.1 patch release.

# Tests + Formatting
Added regression test.
2024-07-26 16:03:05 +08:00
NotTheDr01ds
e68f744dda
Update query-web example to use new chunks (#13429)
# Description

A `query web` example uses the (soon to be deprecated) `group` command.
Updated it to use `chunks` replacement.
2024-07-25 22:01:46 +02:00
Devyn Cairns
6446f26283
Fix $in in range expressions (#13447)
# Description

Fixes #13441.

I must have forgotten that `Expr::Range` can contain other expressions,
so I wasn't searching for `$in` to replace within it. Easy fix.

# User-Facing Changes
Bug fix, ranges like `6 | 3..$in` work as expected now.

# Tests + Formatting
Added regression test.
2024-07-25 18:28:44 +08:00
Devyn Cairns
9f90d611e1
Bump version to 0.96.1 (#13439)
(Post-release bump.)
2024-07-25 18:28:18 +08:00
Devyn Cairns
a80dfe8e80
Bump version to 0.96.0 (#13433) 2024-07-23 16:10:35 -07:00
Darren Schroeder
366e52b76d
Update query web example since wikipedia keeps changing (#13421)
# Description

Every so ofter wikipedia changes the column names which breaks the query
example. It would be good to make query web's table extraction to be
smart enough to find tables that are close. This PR fixes the example.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-21 18:42:11 -05:00
Ian Manske
6fcd09682c
Fix setting metadata on byte streams (#13416)
# Description
Setting metadata on a byte stream converts it to a list stream. This PR
makes it so that `metadata set` does not alter its input besides the
metadata.

```nushell
open --raw test.json
| metadata set --content-type application/json
| http post https://httpbin.org/post 
```
2024-07-21 21:15:36 +00:00
Wenbo Chen
9ab706db62
Fix output format borken in char --list (#13417)
# Description

Resolve #13395 

When we use the `char --list` command, it also outputs the characters
*bel* and *backspace*, which breaks the table's alignment.
![Screenshot from 2024-07-21
16-10-03](https://github.com/user-attachments/assets/54561cbc-f1a1-4ccd-9561-65dccc939280)
I also found that the behaviour at the beginning of the table is not
correct because of some line change characters, as I mentioned in the
issue discussion.

So, the solution is to create a list containing the characters that do
not need to be output. When the name is in the list, we output the
character as an empty string. Here is the behaviour after fixing.
![Screenshot from 2024-07-21
16-16-08](https://github.com/user-attachments/assets/dd3345a3-6a72-4c90-b331-bc95dc6db66a)
![Screenshot from 2024-07-21
16-16-39](https://github.com/user-attachments/assets/57dc5073-7f8d-40c4-9830-36eba23075e6)
2024-07-21 06:30:14 -05:00
Devyn Cairns
01891d637d
Make parsing for unknown args in known externals like normal external calls (#13414)
# Description

This corrects the parsing of unknown arguments provided to known
externals to behave exactly like external arguments passed to normal
external calls.

I've done this by adding a `SyntaxShape::ExternalArgument` which
triggers the same parsing rules.

Because I didn't like how the highlighting looked, I modified the
flattener to emit `ExternalArg` flat shapes for arguments that have that
syntax shape and are plain strings/globs. This is the same behavior that
external calls have.

Aside from passing the tests, I've also checked manually that the
completer seems to work adequately. I can confirm that specified
positional arguments get completion according to their specified type
(including custom completions), and then anything remaining gets
filepath style completion, as you'd expect from an external command.

Thanks to @OJarrisonn for originally finding this issue.

# User-Facing Changes

- Unknown args are now parsed according to their specified syntax shape,
rather than `Any`. This may be a breaking change, though I think it's
extremely unlikely in practice.
- The unspecified arguments of known externals are now highlighted /
flattened identically to normal external arguments, which makes it more
clear how they're being interpreted, and should help the completer
function properly.
- Known externals now have an implicit rest arg if not specified named
`args`, with a syntax shape of `ExternalArgument`.

# Tests + Formatting
Tests added for the new behaviour. Some old tests had to be corrected to
match.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] release notes (bugfix, and debatable whether it's a breaking
change)
2024-07-21 01:32:36 -07:00
Zoey Hewll
5db57abc7d
add --quiet flag to watch command (#13415)
# Description
Adds a `--quiet` flag to the `watch` command, silencing the message
usually shown when invoking `watch`.

Resolves #13411

As implemented, `--quiet` is orthogonal to `--verbose`. I'm open to
improving the flag's name, behaviour and/or documentation to make it
more user-friendly, as I realise this may be surprising as-is.

```
> watch path {|a b c| echo $a $b $c}
Now watching files at "/home/user/path". Press ctrl+c to abort.
───┬───────────────────────
 0 │ Remove                
 1 │ /home/user/path 
 2 │                       
───┴───────────────────────
^C
> watch --quiet path {|a b c| echo $a $b $c}
───┬───────────────────────
 0 │ Remove                
 1 │ /home/user/path 
 2 │                       
───┴───────────────────────
^C
```

# User-Facing Changes

Adds `--quiet`/`-q` flag to `watch`, which removes the initial status
message.
2024-07-20 12:34:27 +02:00
suimong
dbd60ed4f4
Tiny make up to the documentation of reduce (#13408)
This tiny PR improves the documentation of the `reduce` command by
explicitly stating the direction of reduction, i.e. from left to right,
and adds an example for demonstration.


---------

Co-authored-by: Ben Yang <ben@ya.ng>
2024-07-19 19:55:38 +02:00
Devyn Cairns
aa9a42776b
Make ast::Call::span() and arguments_span() more robust (#13412)
# Description

Trying to help @amtoine out here - the spans calculated by `ast::Call`
by `span()` and `argument_span()` are suspicious and are not properly
guarding against `end` that might be before `start`. Just using
`Span::merge()` / `merge_many()` instead to try to make the behaviour as
simple and consistent as possible. Hopefully then even if the arguments
have some crazy spans, we don't have spans that are just totally
invalid.

# Tests + Formatting
I did check that everything passes with this.
2024-07-19 05:12:19 -07:00
Jan Christian Grünhage
4665323bb4
Use directories for autoloading (#13382)
fixes https://github.com/nushell/nushell/issues/13378

# Description

This PR tries to improve usage of system APIs to determine the location
of vendored autoload files.

# User-Facing Changes
The paths listed in #13180 and #13217 are changing. This has not been
part of a release yet, so arguably the user facing changes are only to
unreleased features anyway.

# Tests + Formatting
Haven't done, but if someone wants to help me here, I'm open to doing
it. I just don't know how to properly test this.

# After Submitting
2024-07-19 03:47:07 -07:00
Wind
e281c03403
generate: switch the position of <initial> and <closure>, so the closure can have default parameters (#13393)
# Description
Close: #12083 
Close: #12084 

# User-Facing Changes
It's a breaking change because we have switched the position of
`<initial>` and `<closure>`, after the change, initial value will be
optional. So it's possible to do something like this:

```nushell
> let f = {|fib = [0, 1]| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} }
> generate $f | first 5
╭───┬───╮
│ 0 │ 0 │
│ 1 │ 1 │
│ 2 │ 1 │
│ 3 │ 2 │
│ 4 │ 3 │
╰───┴───╯
```

It will also raise error if user don't give initial value, and the
closure don't have default parameter.
```nushell
❯ let f = {|fib| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} }
❯ generate $f
Error:   × The initial value is missing
   ╭─[entry #5:1:1]
 1 │ generate $f
   · ────┬───
   ·     ╰── Missing intial value
   ╰────
  help: Provide <initial> value in generate, or assigning default value to closure parameter

```

# Tests + Formatting
Added some test cases.

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-07-19 00:22:28 -07:00
Wind
e8764de3c6
don't allow break/continue in each and items command (#13398)
# Description
Fixes: #11451

# User-Facing Changes
### Before
```nushell
❯ [1 2 3] | each {|e| break; print $e }
╭────────────╮
│ empty list │
╰────────────╯
```

### After
```
❯ [1 2 3] | each {|e| break; print $e }
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
   ╭─[entry #9:1:2]
 1 │ [1 2 3] | each {|e| break; print $e }
   ·  ┬
   ·  ╰── source value
   ╰────

Error:   × Break used outside of loop
   ╭─[entry #9:1:21]
 1 │ [1 2 3] | each {|e| break; print $e }
   ·                     ──┬──
   ·                       ╰── used outside of loop
   ╰────
```

# Tests + Formatting
Removes some tests.
2024-07-19 00:21:02 -07:00
Devyn Cairns
f3843a6176
Make plugins able to find and call other commands (#13407)
# Description

Adds functionality to the plugin interface to support calling internal
commands from plugins. For example, using `view ir --json`:

```rust
let closure: Value = call.req(0)?;

let Some(decl_id) = engine.find_decl("view ir")? else {
    return Err(LabeledError::new("`view ir` not found"));
};

let ir_json = engine.call_decl(
    decl_id,
    EvaluatedCall::new(call.head)
        .with_named("json".into_spanned(call.head), Value::bool(true, call.head))
        .with_positional(closure),
    PipelineData::Empty,
    true,
    false,
)?.into_value()?.into_string()?;

let ir = serde_json::from_value(&ir_json);

// ...
```

# User-Facing Changes

Plugin developers can now use `EngineInterface::find_decl()` and
`call_decl()` to call internal commands, which could be handy for
formatters like `to csv` or `to nuon`, or for reflection commands that
help gain insight into the engine.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] release notes
- [ ] update plugin protocol documentation: `FindDecl`, `CallDecl`
engine calls; `Identifier` engine call response
2024-07-19 13:54:21 +08:00
Ian Manske
c19944f291
Refactor window (#13401)
# Description
Following from #13377, this PR refactors the related `window` command.
In the case where `window` should behave exactly like `chunks`, it now
reuses the same code that `chunks` does. Otherwise, the other cases have
been rewritten and have resulted in a performance increase:

| window size | stride | old time (ms) | new time (ms) |
| -----------:| ------:| -------------:| -------------:|
| 20          | 1      | 757           | 722           |
| 2           | 1      | 364           | 333           |
| 1           | 1      | 343           | 293           |
| 20          | 20     | 90            | 63            |
| 2           | 2      | 215           | 175           |
| 20          | 30     | 74            | 60            |
| 2           | 4      | 141           | 110           |


# User-Facing Changes
`window` will now error if the window size or stride is 0, which is
technically a breaking change.
2024-07-19 04:16:09 +00:00
132ikl
22379c9846
Make default config more consistent (#13399)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes some minor config inconsistencies:
- Add default value for `shape_glob_interpolation` to light theme, as it
is missing
- Add right padding space to `shape_garbage` options
- Use a integer value for `footer_mode` instead of a string
- Set `buffer_editor` to null instead of empty string

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
N/A

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
N/A
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
N/A
2024-07-17 18:52:47 -05:00
Devyn Cairns
aa7d7d0cc3
Overhaul $in expressions (#13357)
# Description

This grew quite a bit beyond its original scope, but I've tried to make
`$in` a bit more consistent and easier to work with.

Instead of the parser generating calls to `collect` and creating
closures, this adds `Expr::Collect` which just evaluates in the same
scope and doesn't require any closure.

When `$in` is detected in an expression, it is replaced with a new
variable (also called `$in`) and wrapped in `Expr::Collect`. During
eval, this expression is evaluated directly, with the input and with
that new variable set to the collected value.

Other than being faster and less prone to gotchas, it also makes it
possible to typecheck the output of an expression containing `$in`,
which is nice. This is a breaking change though, because of the lack of
the closure and because now typechecking will actually happen. Also, I
haven't attempted to typecheck the input yet.

The IR generated now just looks like this:

```gas
collect        %in
clone          %tmp, %in
store-variable $in, %tmp
# %out <- ...expression... <- %in
drop-variable  $in
```

(where `$in` is the local variable created for this collection, and not
`IN_VARIABLE_ID`)

which is a lot better than having to create a closure and call `collect
--keep-env`, dealing with all of the capture gathering and allocation
that entails. Ideally we can also detect whether that input is actually
needed, so maybe we don't have to clone, but I haven't tried to do that
yet. Theoretically now that the variable is a unique one every time, it
should be possible to give it a type - I just don't know how to
determine that yet.

On top of that, I've also reworked how `$in` works in pipeline-initial
position. Previously, it was a little bit inconsistent. For example,
this worked:

```nushell
> 3 | do { let x = $in; let y = $in; print $x $y }
3
3
```

However, this causes a runtime variable not found error on the second
`$in`:

```nushell
> def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
Error: nu:🐚:variable_not_found

  × Variable not found
   ╭─[entry #115:1:35]
 1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
   ·                                   ─┬─
   ·                                    ╰── variable not found
   ╰────
```

I've fixed this by making the first element `$in` detection *always*
happen at the block level, so if you use `$in` in pipeline-initial
position anywhere in a block, it will collect with an implicit
subexpression around the whole thing, and you can then use that `$in`
more than once. In doing this I also rewrote `parse_pipeline()` and
hopefully it's a bit more straightforward and possibly more efficient
too now.

Finally, I've tried to make `let` and `mut` a lot more straightforward
with how they handle the rest of the pipeline, and using a redirection
with `let`/`mut` now does what you'd expect if you assume that they
consume the whole pipeline - the redirection is just processed as
normal. These both work now:

```nushell
let x = ^foo err> err.txt
let y = ^foo out+err>| str length
```

It was previously possible to accomplish this with a subexpression, but
it just seemed like a weird gotcha that you couldn't do it. Intuitively,
`let` and `mut` just seem to take the whole line.

- closes #13137

# User-Facing Changes
- `$in` will behave more consistently with blocks and closures, since
the entire block is now just wrapped to handle it if it appears in the
first pipeline element
- `$in` no longer creates a closure, so what can be done within an
expression containing `$in` is less restrictive
- `$in` containing expressions are now type checked, rather than just
resulting in `any`. However, `$in` itself is still `any`, so this isn't
quite perfect yet
- Redirections are now allowed in `let` and `mut` and behave pretty much
how you'd expect

# Tests + Formatting
Added tests to cover the new behaviour.

# After Submitting
- [ ] release notes (definitely breaking change)
2024-07-17 16:02:42 -05:00
dependabot[bot]
63cea44130
Bump uuid from 1.9.1 to 1.10.0 (#13390)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.9.1 to 1.10.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/uuid-rs/uuid/releases">uuid's
releases</a>.</em></p>
<blockquote>
<h2>1.10.0</h2>
<h2>Deprecations</h2>
<p>This release deprecates and renames the following functions:</p>
<ul>
<li><code>Builder::from_rfc4122_timestamp</code> -&gt;
<code>Builder::from_gregorian_timestamp</code></li>
<li><code>Builder::from_sorted_rfc4122_timestamp</code> -&gt;
<code>Builder::from_sorted_gregorian_timestamp</code></li>
<li><code>Timestamp::from_rfc4122</code> -&gt;
<code>Timestamp::from_gregorian</code></li>
<li><code>Timestamp::to_rfc4122</code> -&gt;
<code>Timestamp::to_gregorian</code></li>
</ul>
<h2>What's Changed</h2>
<ul>
<li>Use const identifier in uuid macro by <a
href="https://github.com/Vrajs16"><code>@​Vrajs16</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/764">uuid-rs/uuid#764</a></li>
<li>Rename most methods referring to RFC4122 by <a
href="https://github.com/Mikopet"><code>@​Mikopet</code></a> / <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/765">uuid-rs/uuid#765</a></li>
<li>prepare for 1.10.0 release by <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/766">uuid-rs/uuid#766</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Vrajs16"><code>@​Vrajs16</code></a> made
their first contribution in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/764">uuid-rs/uuid#764</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0">https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4b4c590ae3"><code>4b4c590</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/766">#766</a> from
uuid-rs/cargo/1.10.0</li>
<li><a
href="68eff32640"><code>68eff32</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/765">#765</a> from
uuid-rs/chore/time-fn-deprecations</li>
<li><a
href="3d5384da4b"><code>3d5384d</code></a>
update docs and deprecation messages for timestamp fns</li>
<li><a
href="de50f2091f"><code>de50f20</code></a>
renaming rfc4122 functions</li>
<li><a
href="4a8841792a"><code>4a88417</code></a>
prepare for 1.10.0 release</li>
<li><a
href="66b4fcef14"><code>66b4fce</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/764">#764</a> from
Vrajs16/main</li>
<li><a
href="8896e26c42"><code>8896e26</code></a>
Use expr instead of ident</li>
<li><a
href="09973d6aff"><code>09973d6</code></a>
Added changes</li>
<li><a
href="6edf3e8cd5"><code>6edf3e8</code></a>
Use const identifer in uuid macro</li>
<li>See full diff in <a
href="https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=uuid&package-manager=cargo&previous-version=1.9.1&new-version=1.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-17 09:46:59 +08:00
Maxim Zhiburt
5417c89387
Fix issue with truncation when head on border is used (#13389)
close #13365

@fdncred thanks for reproducible

hopefully there won't be issues with it after this one 😅
2024-07-16 16:21:42 -05:00
Jan Christian Grünhage
b66671d339
Switch from dirs_next 2.0 to dirs 5.0 (#13384)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Replaces the `dirs_next` family of crates with `dirs`. `dirs_next` was
born when the `dirs` crates were abandoned three years ago, but they're
being maintained again and most projects depend on `dirs` nowadays.
`dirs_next` has been abandoned since.

This came up while working on
https://github.com/nushell/nushell/pull/13382.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
None.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests and formatter have been run.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-16 07:16:26 -05:00
Ian Manske
1981c50c8f
Remove unused field in StateWorkingSet (#13387)
# Description
Removes the unused `external_commands` field from `StateWorkingSet`.
2024-07-16 11:28:31 +02:00
Bruce Weirdan
fcb8e36caa
Remove default list-diving behaviour (#13386)
# Description

Before this change `default` would dive into lists and replace `null`
elements with the default value.
Therefore there was no easy way to process `null|list<null|string>`
input and receive `list<null|string>`
(IOW to apply the default on the top level only).

However it's trivially easy to apply default values to list elements
with the `each` command:
```nushell
[null, "a", null] | each { default "b" }
```

So there's no need to have this behavior in `default` command.

# User-Facing Changes

* `default` no longer dives into lists to replace `null` elements with
the default value.

# Tests + Formatting

Added a couple of tests for the new (and old) behavior.

# After Submitting

* Update docs.
2024-07-16 03:54:24 +00:00
Ian Manske
0918050ac8
Deprecate group in favor of chunks (#13377)
# Description
The name of the `group` command is a little unclear/ambiguous.
Everything I look at it, I think of `group-by`. I think `chunks` more
clearly conveys what the `group` command does. Namely, it divides the
input list into chunks of a certain size. For example,
[`slice::chunks`](https://doc.rust-lang.org/std/primitive.slice.html#method.chunks)
has the same name. So, this PR adds a new `chunks` command to replace
the now deprecated `group` command.

The `chunks` command is a refactored version of `group`. As such, there
is a small performance improvement:
```nushell
# $data is a very large list
> bench { $data | chunks 2 } --rounds 30 | get mean
474ms 921µs 190ns

# deprecation warning was disabled here for fairness
> bench { $data | group 2 } --rounds 30 | get mean
592ms 702µs 440ns



> bench { $data | chunks 200 } --rounds 30 | get mean
374ms 188µs 318ns

> bench { $data | group 200 } --rounds 30 | get mean
481ms 264µs 869ns 



> bench { $data | chunks 1 } --rounds 30 | get mean
642ms 574µs 42ns

> bench { $data | group 1 } --rounds 30 | get mean
981ms 602µs 513ns
```

# User-Facing Changes
- `group` command has been deprecated in favor of new `chunks` command.
- `chunks` errors when given a chunk size of `0` whereas `group` returns
chunks with one element.

# Tests + Formatting
Added tests for `chunks`, since `group` did not have any tests.

# After Submitting
Update book if necessary.
2024-07-16 03:49:00 +00:00
Stefan Holderbach
3d1145e759
Fix CI test failure on main (nu-json) (#13374)
Conflict resulting from #13329 and #13326
2024-07-14 10:37:57 +02:00
David
cf4864a9cd
JSON format output keeps braces on same line (issue #13326) (#13352)
# Description
This is a minor breaking change to JSON output syntax/style of the `to
json` command.

This fixes #13326 by setting `braces_same_line` to true when creating a
new `HjsonFormatter`.
This then simply tells `HjsonFormatter` to keep the braces on the same
line when outputting which is what I expected nu's `to json` command to
do.
There are almost no changes to nushell itself, all changes are contained
within `nu-json` crate (minus any documentation updates).

Oh, almost forgot to mention, to get the tests compiling, I added
fancy_regex as a _dev_ dependency to nu-json. I could look into
eliminating that if desirable.

# User-Facing Changes

**Breaking Change**
nushell now outputs the desired result using the reproduction command
from the issue:
```
echo '{"version": "v0.4.4","notes": "blablabla","pub_date": "2024-05-04T16:05:00Z","platforms":{"windows-x86_64":{"signature": "blablabla","url": "https://blablabla"}}}' | from json | to json
```

outputs:
```
{
  "version": "v0.4.4",
  "notes": "blablabla",
  "pub_date": "2024-05-04T16:05:00Z",
  "platforms": {
    "windows-x86_64": {
      "signature": "blablabla",
      "url": "https://blablabla"
    }
  }
}
```

whereas previously it would push the opening braces onto a new line:
```
{
  "version": "v0.4.4",
  "notes": "blablabla",
  "pub_date": "2024-05-04T16:05:00Z",
  "platforms":
  {
    "windows-x86_64":
    {
      "signature": "blablabla",
      "url": "https://blablabla"
    }
  }
}
```

# Tests + Formatting

toolkit check pr mostly passes - there are regrettably some tests not
passing on my windows machine _before making any changes_ (I may look
into this as a separate issue)

I have re-enabled the [hjson
tests](https://github.com/nushell/nushell/blob/main/crates/nu-json/tests/main.rs).
This is done in the second commit 🙂 
They have a crucial difference to what they were previously asserting:
  * nu-json outputs in json syntax, not hjson syntax
I think this is desirable, but I'm not aware of the history of these
tests.

# After Submitting

I suspect there `to json` command examples will need updating to match,
haven't checked yet!
2024-07-14 10:19:09 +02:00
Devyn Cairns
ae40d56fc5
Report parse warns and compile errs when running script files (#13369)
# Description

Report parse warnings and compile errors when running script files.

It's useful to report this information to script authors and users so
they can know if they need to update something in the near future.

I also found that moving some errors from eval to compile time meant
some error tests can fail (in `tests/repl`) so this is a good idea to
keep those passing.

# User-Facing Changes
- Report parse warnings when running script files
- Report compile errors when running script files
2024-07-14 10:12:55 +02:00
Stefan Holderbach
c5aa15c7f6
Add top-level crate documentation/READMEs (#12907)
# Description
Add `README.md` files to each crate in our workspace (-plugins) and also
include it in the `lib.rs` documentation for <docs.rs> (if there is no
existing `lib.rs` crate documentation)

In all new README I added the defensive comment that the crates are not
considered stable for public consumption. If necessary we can adjust
this if we deem a crate useful for plugin authors.
2024-07-14 10:10:41 +02:00
Stefan Holderbach
f5bff8c9c8
Fix select cell path renaming behavior (#13361)
# Description

Fixes #13359

In an attempt to generate names for flat columns resulting from a nested
accesses #3016 generated new column names on nested selection, out of
convenience, that composed the cell path as a string (including `.`) and
then simply replaced all `.` with `_`. As we permit `.` in column names
as long as you quote this surprisingly alters `select`ed columns.


# User-Facing Changes
New columns generated by selection with nested cell paths will for now
be named with a string containing the keys separated by `.` instead of
`_`. We may want to reconsider the semantics for nested access.

# Tests + Formatting
- Alter test to breaking change on nested `select`
2024-07-13 16:54:34 +02:00
Yash Thakur
b0bf54614f
Don't add touch command to default context twice (#13371)
# Description

Touch was added to the shell command context twice, once with the other
filesystem commands and once with the format commands. I removed that
second occurrence of touch, because I'm assuming it was only added there
because "Touch" starts with "To."

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

None
2024-07-13 16:52:39 +02:00
Devyn Cairns
a2758e6c40
Add IR support to the debugger (#13345)
# Description

This adds tracing for each individual instruction to the `Debugger`
trait. Register contents can be inspected both when entering and leaving
an instruction, and if an instruction produced an error, a reference to
the error is also available. It's not the full `EvalContext` but it's
most of the important parts for getting an idea of what's going on.

Added support for all of this to the `Profiler` / `debug profile` as
well, and the output is quite incredible - super verbose, but you can
see every instruction that's executed and also what the result was if
it's an instruction that has a clearly defined output (many do).

# User-Facing Changes

- Added `--instructions` to `debug profile`, which adds the `pc` and
`instruction` columns to the output.
- `--expr` only works in AST mode, and `--instructions` only works in IR
mode. In the wrong mode, the output for those columns is just blank.

# Tests + Formatting

All passing.

# After Submitting

- [ ] release notes
2024-07-13 01:58:21 -07:00
Devyn Cairns
d42cf55431
fix file_count in Debug implementation of IrBlock (#13367)
# Description

Oops.
2024-07-12 21:27:23 -05:00
Darren Schroeder
46b5e510ac
tweak parse usage and examples to be more clear (#13363)
# Description

This PR just tweaks the `parse` command's usage and examples to make it
clearer what's going on "under the hood".

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-12 09:48:27 -05:00
Devyn Cairns
02659b1c8a
Mention the actual output type on an OutputMismatch error (#13355)
# Description

This improves the error when the determined output of a custom command
doesn't match the specified output type by adding the actual determined
output type.

# User-Facing Changes

Previous: `command doesn't output {0}`

New: `expected {0}, but command outputs {1}`

# Tests + Formatting
Passing.

# After Submitting
- [ ] release notes? (minor change, but helpful)
2024-07-12 11:45:53 +02:00
Stefan Holderbach
8f981c1eb4
Use conventional generic bounds (#13360)
https://rust-lang.github.io/rust-clippy/master/index.html#/multiple_bound_locations
2024-07-12 17:13:07 +08:00
Devyn Cairns
0b8d0bcd7a
Fix order of I/O types in take until (#13356)
# Description
Just a quick one, but `List(Any)` has to come before `Table`, because
`List(Any)` is a valid match for `Table`, so it will choose `Table`
output even if the input was actually `List(Any)`. I ended up removing
`Table` because it's just not needed at all anyway.

Though, I'm not really totally sure this is correct - I think the parser
should probably actually just have some idea of what the more specific
type is, and choose the most specific type match, rather than just doing
it in order. I guess this will result in the output just always being
`List(Any)` for now. Still better than a bad typecheck error

# User-Facing Changes

Fixes the following contrived example:

```nushell
def foo []: nothing -> list<int> {
  seq 1 10 | # list<int>
    each { |n| $n * 20 } | # this causes the type to become list<any>
    take until { |x| $x < 10 } } # table is first, so now this is type table
    # ...but table is not compatible with list<int>
}
```

# After Submitting
- [ ] make typechecker type choice more robust
- [ ] release notes
2024-07-12 10:25:44 +02:00
Ian Manske
ee875bb8a3
Edit path form doc comments (#13358)
# Description
Fixes outdated/inaccurate doc comments for `PathForm`s in `nu_path`.
2024-07-12 10:23:51 +02:00
Ian Manske
d56457d63e
Path migration part 2: nu-test-support (#13329)
# Description
Part 2 of replacing `std::path` types with `nu_path` types added in
#13115. This PR targets `nu-test-support`.
2024-07-12 02:43:10 +00:00
Maxim Zhiburt
4bd87d0496
Fix unused space when truncation is used and header on border is configured (#13353)
Somehow this logic was missed on my end. ( I mean I was not even
thinking about it in original patch 😄 )

Please recheck

Added a regression test too.

close #13336

cc: @fdncred
2024-07-11 15:13:16 -05:00
Devyn Cairns
ccd0160c32
Make the store-env IR instruction also update config (#13351)
# Description

Follow up fix to #13332, so that changes to config when running under IR
actually happen as well. Since I merged them around the same time, I
forgot about this.
2024-07-11 10:49:46 -07:00
Devyn Cairns
f65bc97a54
Update config directly at assignment (#13332)
# Description

Allows `Stack` to have a modified local `Config`, which is updated
immediately when `$env.config` is assigned to. This means that even
within a script, commands that come after `$env.config` changes will
always see those changes in `Stack::get_config()`.

Also fixed a lot of cases where `engine_state.get_config()` was used
even when `Stack` was available.

Closes #13324.

# User-Facing Changes
- Config changes apply immediately after the assignment is executed,
rather than whenever config is read by a command that needs it.
- Potentially slower performance when executing a lot of lines that
change `$env.config` one after another. Recommended to get `$env.config`
into a `mut` variable first and do modifications, then assign it back.
- Much faster performance when executing a script that made
modifications to `$env.config`, as the changes are only parsed once.

# Tests + Formatting
All passing.

# After Submitting
- [ ] release notes
2024-07-11 06:09:33 -07:00
Stefan Holderbach
076a29ae19
Document public types in nu-protocol (#12906)
- **Doc-comment public `nu-protocol` modules**
- **Doccomment argument/signature/call stuff**
- **Doccomment cell path types**
- **Doccomment expression stuff**
- **Doccomment import patterns**
- **Doccomment pattern matching AST nodes**
2024-07-11 13:30:12 +02:00
Devyn Cairns
9de7f931c0
Add more argument types to view ir (#13343)
# Description

Add a few more options to `view ir` for finding blocks, which I found
myself wanting while trying to trace through the generated code.

If we end up adding support for plugins to call commands that are in
scope by name, this will also make it possible for
`nu_plugin_explore_ir` to just step through IR automatically (by passing
the block/decl ids) without exposing too many internals. With that I
could potentially add keys that allow you to step in to closures or
decls with the press of a button, just by calling `view ir --json`
appropriately.

# User-Facing Changes

- `view ir` can now take names of custom commands that are in scope.
- integer arguments are treated as block IDs, which sometimes show up in
IR (closure, block, row condition literals).
- `--decl-id` provided to treat the argument as a decl ID instead, which
is also sometimes necessary to access something that isn't in scope.
2024-07-11 06:05:06 -05:00
Devyn Cairns
d97512df8e
Fix the signature of view ir (#13342)
# Description

Fix `view ir` to use `Signature::build()` rather than `new()`, which is
required for `--help` to work. Also add `Category::Debug`, as that's
most appropriate.
2024-07-11 06:00:59 -05:00
Devyn Cairns
801cfae279
Avoid clone in Signature::get_positional() (#13338)
# Description
`Signature::get_positional()` was returning an owned `PositionalArg`,
which contains a bunch of strings. `ClosureEval` uses this in
`try_add_arg`, making all of that unnecessary cloning a little bit hot.

# User-Facing Changes
Slightly better performance
2024-07-11 02:14:05 +00:00
Devyn Cairns
f87cf895c2
Set the capacity of the Vec used in gather_captures() to the number of captures expected (#13339)
# Description

Just more efficient allocation during `Stack::gather_captures()` so that
we don't have to grow the `Vec` needlessly.

# User-Facing Changes
Slightly better performance.
2024-07-11 02:13:35 +00:00
Darren Schroeder
ac561b1b0e
quick fix up for ir pr as_refs (#13340)
# Description

Was having an issue compiling main after the IR pr. Talked to devyn and
he led me to change a couple things real quick and we're compiling once
again.
2024-07-11 09:19:06 +08:00
Devyn Cairns
1a5bf2447a
Use Arc for environment variables on the stack (#13333)
# Description

This is another easy performance lift that just changes `env_vars` and
`env_hidden` on `Stack` to use `Arc`. I noticed that these were being
cloned on essentially every closure invocation during captures
gathering, so we're paying the cost for all of that even when we don't
change anything. On top of that, for `env_vars`, there's actually an
entirely fresh `HashMap` created for each child scope, so it's highly
unlikely that we'll modify the parent ones.

Uses `Arc::make_mut` instead to take care of things when we need to
mutate something, and most of the time nothing has to be cloned at all.

# Benchmarks

The benefits are greater the more calls there are to env-cloning
functions like `captures_to_stack()`. Calling custom commands in a loop
is basically best case for a performance improvement. Plain `each` with
a literal block isn't so badly affected because the stack is set up
once.

## random_bytes.nu

```nushell
use std bench
do {
  const SCRIPT = ../nu_scripts/benchmarks/random-bytes.nu
  let before_change = bench { nu $SCRIPT }
  let after_change = bench { target/release/nu $SCRIPT }
  {
    before: ($before_change | reject times),
    after: ($after_change | reject times)
  }
}
```

```
╭────────┬──────────────────────────────╮
│        │ ╭──────┬───────────────────╮ │
│ before │ │ mean │ 603ms 759µs 727ns │ │
│        │ │ min  │ 593ms 298µs 167ns │ │
│        │ │ max  │ 648ms 612µs 291ns │ │
│        │ │ std  │ 9ms 335µs 251ns   │ │
│        │ ╰──────┴───────────────────╯ │
│        │ ╭──────┬───────────────────╮ │
│ after  │ │ mean │ 518ms 400µs 557ns │ │
│        │ │ min  │ 507ms 762µs 583ns │ │
│        │ │ max  │ 566ms 695µs 166ns │ │
│        │ │ std  │ 9ms 554µs 767ns   │ │
│        │ ╰──────┴───────────────────╯ │
╰────────┴──────────────────────────────╯
```

## gradient_benchmark_no_check.nu

```nushell
use std bench
do {
  const SCRIPT = ../nu_scripts/benchmarks/gradient_benchmark_no_check.nu
  let before_change = bench { nu $SCRIPT }
  let after_change = bench { target/release/nu $SCRIPT }
  {
    before: ($before_change | reject times),
    after: ($after_change | reject times)
  }
}
```

```
╭────────┬──────────────────────────────╮
│        │ ╭──────┬───────────────────╮ │
│ before │ │ mean │ 146ms 543µs 380ns │ │
│        │ │ min  │ 142ms 416µs 166ns │ │
│        │ │ max  │ 189ms 595µs       │ │
│        │ │ std  │ 7ms 140µs 342ns   │ │
│        │ ╰──────┴───────────────────╯ │
│        │ ╭──────┬───────────────────╮ │
│ after  │ │ mean │ 134ms 211µs 678ns │ │
│        │ │ min  │ 132ms 433µs 125ns │ │
│        │ │ max  │ 135ms 722µs 583ns │ │
│        │ │ std  │ 793µs 134ns       │ │
│        │ ╰──────┴───────────────────╯ │
╰────────┴──────────────────────────────╯
```

# User-Facing Changes
Better performance, particularly for custom commands, especially if
there are a lot of environment variables. Nothing else.

# Tests + Formatting
All passing.
2024-07-10 17:34:50 -07:00
Devyn Cairns
d7392f1f3b
Internal representation (IR) compiler and evaluator (#13330)
# Description

This PR adds an internal representation language to Nushell, offering an
alternative evaluator based on simple instructions, stream-containing
registers, and indexed control flow. The number of registers required is
determined statically at compile-time, and the fixed size required is
allocated upon entering the block.

Each instruction is associated with a span, which makes going backwards
from IR instructions to source code very easy.

Motivations for IR:

1. **Performance.** By simplifying the evaluation path and making it
more cache-friendly and branch predictor-friendly, code that does a lot
of computation in Nushell itself can be sped up a decent bit. Because
the IR is fairly easy to reason about, we can also implement
optimization passes in the future to eliminate and simplify code.
2. **Correctness.** The instructions mostly have very simple and
easily-specified behavior, so hopefully engine changes are a little bit
easier to reason about, and they can be specified in a more formal way
at some point. I have made an effort to document each of the
instructions in the docs for the enum itself in a reasonably specific
way. Some of the errors that would have happened during evaluation
before are now moved to the compilation step instead, because they don't
make sense to check during evaluation.
3. **As an intermediate target.** This is a good step for us to bring
the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at
some point, as code generated from new AST can be directly compared to
code generated from old AST. If the IR code is functionally equivalent,
it will behave the exact same way.
4. **Debugging.** With a little bit more work, we can probably give
control over advancing the virtual machine that `IrBlock`s run on to
some sort of external driver, making things like breakpoints and single
stepping possible. Tools like `view ir` and [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than
before to see what exactly is going on with your Nushell code.

The goal is to eventually replace the AST evaluator entirely, once we're
sure it's working just as well. You can help dogfood this by running
Nushell with `$env.NU_USE_IR` set to some value. The environment
variable is checked when Nushell starts, so config runs with IR, or it
can also be set on a line at the REPL to change it dynamically. It is
also checked when running `do` in case within a script you want to just
run a specific piece of code with or without IR.

# Example

```nushell
view ir { |data|
  mut sum = 0
  for n in $data {
    $sum += $n
  }
  $sum
}
```
  
```gas
# 3 registers, 19 instructions, 0 bytes of data
   0: load-literal           %0, int(0)
   1: store-variable         var 904, %0 # let
   2: drain                  %0
   3: drop                   %0
   4: load-variable          %1, var 903
   5: iterate                %0, %1, end 15 # for, label(1), from(14:)
   6: store-variable         var 905, %0
   7: load-variable          %0, var 904
   8: load-variable          %2, var 905
   9: binary-op              %0, Math(Plus), %2
  10: span                   %0
  11: store-variable         var 904, %0
  12: load-literal           %0, nothing
  13: drain                  %0
  14: jump                   5
  15: drop                   %0          # label(0), from(5:)
  16: drain                  %0
  17: load-variable          %0, var 904
  18: return                 %0
```

# Benchmarks

All benchmarks run on a base model Mac Mini M1.

## Iterative Fibonacci sequence

This is about as best case as possible, making use of the much faster
control flow. Most code will not experience a speed improvement nearly
this large.

```nushell
def fib [n: int] {
  mut a = 0
  mut b = 1
  for _ in 2..=$n {
    let c = $a + $b
    $a = $b
    $b = $c
  }
  $b
}
use std bench
bench { 0..50 | each { |n| fib $n } }
```

IR disabled:

```
╭───────┬─────────────────╮
│ mean  │ 1ms 924µs 665ns │
│ min   │ 1ms 700µs 83ns  │
│ max   │ 3ms 450µs 125ns │
│ std   │ 395µs 759ns     │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```

IR enabled:

```
╭───────┬─────────────────╮
│ mean  │ 452µs 820ns     │
│ min   │ 427µs 417ns     │
│ max   │ 540µs 167ns     │
│ std   │ 17µs 158ns      │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```

![explore ir
view](https://github.com/nushell/nushell/assets/10729/d7bccc03-5222-461c-9200-0dce71b83b83)

##
[gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu)

IR disabled:

```
╭───┬──────────────────╮
│ 0 │ 27ms 929µs 958ns │
│ 1 │ 21ms 153µs 459ns │
│ 2 │ 18ms 639µs 666ns │
│ 3 │ 19ms 554µs 583ns │
│ 4 │ 13ms 383µs 375ns │
│ 5 │ 11ms 328µs 208ns │
│ 6 │  5ms 659µs 542ns │
╰───┴──────────────────╯
```

IR enabled:

```
╭───┬──────────────────╮
│ 0 │       22ms 662µs │
│ 1 │ 17ms 221µs 792ns │
│ 2 │ 14ms 786µs 708ns │
│ 3 │ 13ms 876µs 834ns │
│ 4 │  13ms 52µs 875ns │
│ 5 │ 11ms 269µs 666ns │
│ 6 │  6ms 942µs 500ns │
╰───┴──────────────────╯
```

##
[random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu)

I got pretty random results out of this benchmark so I decided not to
include it. Not clear why.

# User-Facing Changes
- IR compilation errors may appear even if the user isn't evaluating
with IR.
- IR evaluation can be enabled by setting the `NU_USE_IR` environment
variable to any value.
- New command `view ir` pretty-prints the IR for a block, and `view ir
--json` can be piped into an external tool like [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir).

# Tests + Formatting
All tests are passing with `NU_USE_IR=1`, and I've added some more eval
tests to compare the results for some very core operations. I will
probably want to add some more so we don't have to always check
`NU_USE_IR=1 toolkit test --workspace` on a regular basis.

# After Submitting
- [ ] release notes
- [ ] further documentation of instructions?
- [ ] post-release: publish `nu_plugin_explore_ir`
2024-07-10 17:33:59 -07:00
Devyn Cairns
ea8c4e3af2
Make pipe redirections consistent, add err>| etc. forms (#13334)
# Description

Fixes the lexer to recognize `out>|`, `err>|`, `out+err>|`, etc.

Previously only the short-style forms were recognized, which was
inconsistent with normal file redirections.

I also integrated it all more into the normal lex path by checking `|`
in a special way, which should be more performant and consistent, and
cleans up the code a bunch.

Closes #13331.

# User-Facing Changes
- Adds `out>|` (error), `err>|`, `out+err>|`, `err+out>|` as recognized
forms of the pipe redirection.

# Tests + Formatting
All passing. Added tests for the new forms.

# After Submitting
- [ ] release notes
2024-07-11 07:16:22 +08:00
Jack Wright
b68c7cf3fa
Make polars unpivot consistent with polars pivot (#13335)
# Description
Makes `polars unpivot` use the same arguments as `polars pivot` and
makes it consistent with the polars' rust api. Additionally, support for
the polar's streaming engine has been exposed on eager dataframes.
Previously, it would only work with lazy dataframes.


# User-Facing Changes
* `polars unpivot` argument `--columns`|`-c` has been renamed to
`--index`|`-i`
* `polars unpivot` argument `--values`|`-v` has been renamed to
`--on`|`-o`
* `polars unpivot` short argument for `--streamable` is now `-t` to make
it consistent with `polars pivot`. It was made `-t` for `polars pivot`
because `-s` is short for `--short`
2024-07-10 16:36:38 -05:00
Darren Schroeder
ad8054ebed
update table comments 2024-07-09 19:52:57 -05:00
Jack Wright
ff27d6a18e
Implemented a command to expose polar's pivot functionality (#13282)
# Description
Implementing pivot support 

The example below is a port of the [python API
example](https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.pivot.html)

<img width="1079" alt="Screenshot 2024-07-01 at 14 29 27"
src="https://github.com/nushell/nushell/assets/56345/277eb7a2-233b-4070-9d24-c2183805c1b8">

# User-Facing Changes
* Introduction of the `polars pivot` command
2024-07-09 10:17:20 -07:00
Maxim Zhiburt
4cdceca1f7
Fix kv table width issue with header_on_border configuration (#13325)
GOOD CATCH.............................................................
SORRY

I've added a test to catch regression just in case.

close #13319

cc: @fdncred
2024-07-09 09:49:04 -05:00
Wind
1964dacaef
Raise error when using o>| pipe (#13323)
# Description
From the feedbacks from @amtoine , it's good to make nushell shows error
for `o>|` syntax.

# User-Facing Changes
## Before
```nushell
'foo' o>| print                                                                                                                                                                                                                     07/09/2024 06:44:23 AM
Error: nu::parser::parse_mismatch

  × Parse mismatch during operation.
   ╭─[entry #6:1:9]
 1 │ 'foo' o>| print
   ·         ┬
   ·         ╰── expected redirection target
```

## After
```nushell
'foo' o>| print                                                                                                                                                                                                                     07/09/2024 06:47:26 AM
Error: nu::parser::parse_mismatch

  × Parse mismatch during operation.
   ╭─[entry #1:1:7]
 1 │ 'foo' o>| print
   ·       ─┬─
   ·        ╰── expected `|`.  Redirection stdout to pipe is the same as piping directly.
   ╰────
```

# Tests + Formatting
Added one test

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-07-09 07:11:25 -05:00
Ian Manske
e98b2ceb8c
Path migration 1 (#13309)
# Description
Part 1 of replacing `std::path` types with `nu_path` types added in
#13115.
2024-07-09 17:25:23 +08:00
Ian Manske
399a7c8836
Add and use new Signals struct (#13314)
# Description
This PR introduces a new `Signals` struct to replace our adhoc passing
around of `ctrlc: Option<Arc<AtomicBool>>`. Doing so has a few benefits:
- We can better enforce when/where resetting or triggering an interrupt
is allowed.
- Consolidates `nu_utils::ctrl_c::was_pressed` and other ad-hoc
re-implementations into a single place: `Signals::check`.
- This allows us to add other types of signals later if we want. E.g.,
exiting or suspension.
- Similarly, we can more easily change the underlying implementation if
we need to in the future.
- Places that used to have a `ctrlc` of `None` now use
`Signals::empty()`, so we can double check these usages for correctness
in the future.
2024-07-07 22:29:01 +00:00
YizhePKU
152fb5be39
Fix PWD-aware command hints (#13024)
This PR fixes PWD-aware command hints by sending PWD to the Reedline
state in every REPL loop. This PR should be merged along with
https://github.com/nushell/reedline/pull/796.

Fixes https://github.com/nushell/nushell/issues/12951.
2024-07-07 11:43:22 -05:00
Reilly Wood
83081f9852
explore: pass config to views at creation time (#13312)
cc: @zhiburt

This is an internal refactoring for `explore`.

Previously, views inside `explore` were created with default/incorrect
configuration and then the correct configuration was passed to them
using a function called `setup()`. I believe this was because
configuration was dynamic and could change while `explore` was running.

After https://github.com/nushell/nushell/pull/10259, configuration can
no longer be changed on the fly. So we can clean this up by removing
`setup()` and passing configuration to views when they are created.
2024-07-07 08:09:59 -05:00
Devyn Cairns
6ce5530fc2
Make into bits produce bitstring stream (#13310)
# Description

Fix `into bits` to have consistent behavior when passed a byte stream.

# User-Facing Changes

Previously, it was returning a binary on stream, even though its
input/output types don't describe this possibility. We don't need this
since we have `into binary` anyway.

# Tests + Formatting
Tests added
2024-07-07 08:00:57 -05:00
goldfish
5af8d62666
Fix from toml to handle toml datetime correctly (#13315)
# Description

fixed #12699

When bare dates or naive times are specified in toml files, `from toml`
returns invalid dates or times.
This PR fixes the problem to correctly handle toml datetime.

The current version command returns the default datetime
(`chrono::DateTime::default()`) if the datetime parse fails. However, I
felt that this behavior was a bit unfriendly, so I changed it to return
`Value::string`.

# User-Facing Changes

The command returns a date with default time and timezone if a bare date
is specified.

```
~/Development/nushell> "dob = 2023-05-27" | from toml
╭─────┬────────────╮
│ dob │ a year ago │
╰─────┴────────────╯
~/Development/nushell> "dob = 2023-05-27" | from toml |
Sat, 27 May 2023 00:00:00 +0000 (a year ago)
~/Development/nushell>                            
```

If a bare time is given, a time string is returned.

```
~/Development/nushell> "tm = 11:00:00" | from toml
╭────┬──────────╮
│ tm │ 11:00:00 │
╰────┴──────────╯
~/Development/nushell> "tm = 11:00:00" | from toml | get tm
11:00:00
~/Development/nushell>  
```

# Tests + Formatting

When I ran tests, `commands::touch::change_file_mtime_to_reference`
failed with the following error.
The error also occurs in the master branch, so it's probably unrelated
to these changes.
(maybe a problem with my dev environment)

```
$ ~/Development/nushell> toolkit check pr

~~~~~~~~

test usage_start_uppercase ... ok
test format_conversions::yaml::convert_dict_to_yaml_with_integer_floats_key ... ok
test format_conversions::yaml::convert_dict_to_yaml_with_boolean_key ... ok
test format_conversions::yaml::table_to_yaml_text_and_from_yaml_text_back_into_table ... ok
test quickcheck_parse ... ok
test format_conversions::yaml::convert_dict_to_yaml_with_integer_key ... ok

failures:

---- commands::touch::change_file_mtime_to_reference stdout ----
=== stderr

thread 'commands::touch::change_file_mtime_to_reference' panicked at crates/nu-command/tests/commands/touch.rs:298:9:
assertion `left == right` failed
  left: SystemTime { tv_sec: 1720344745, tv_nsec: 862392750 }
 right: SystemTime { tv_sec: 1720344745, tv_nsec: 887670417 }


failures:
    commands::touch::change_file_mtime_to_reference

test result: FAILED. 1542 passed; 1 failed; 32 ignored; 0 measured; 0 filtered out; finished in 12.04s

error: test failed, to rerun pass `-p nu-command --test main`
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
-  `toolkit test stdlib`

~/Development/nushell> toolkit test stdlib
   Compiling nu v0.95.1 (/Users/hiroki/Development/nushell)
   Compiling nu-cmd-lang v0.95.1 (/Users/hiroki/Development/nushell/crates/nu-cmd-lang)
    Finished dev [unoptimized + debuginfo] target(s) in 6.64s
     Running `target/debug/nu --no-config-file -c '
        use crates/nu-std/testing.nu
        testing run-tests --path crates/nu-std
    '`
2024-07-07T19:00:20.423|INF|Running from_jsonl_invalid_object in module test_formats
2024-07-07T19:00:20.436|INF|Running env_log-prefix in module test_logger_env

~~~~~~~~~~~

2024-07-07T19:00:22.196|INF|Running debug_short in module test_basic_commands
~/Development/nushell> 
```

# After Submitting

nothing
2024-07-07 07:55:06 -05:00
Maxim Zhiburt
32db5d3aa3
Fix issue with head on separation lines (#13291)
Hi there,

Seems to work

Though I haven't done a lot of testing.


![image](https://github.com/nushell/nushell/assets/20165848/c95aa8d4-a8d2-462c-afc9-35c48f8825f4)

![image](https://github.com/nushell/nushell/assets/20165848/1859dfe5-4a76-4776-a4e0-d3f53fc86862)

![image](https://github.com/nushell/nushell/assets/20165848/b46bb62b-a951-412d-b8fa-65cebcfbfed6)

![image](https://github.com/nushell/nushell/assets/20165848/bff0762e-42d4-41bf-b2c2-641c0436ca2e)

![image](https://github.com/nushell/nushell/assets/20165848/2c3c5664-9b90-44e4-befc-c250174cb630)


close #13287
cc: @fdncred 

PS: Yessssss I do remember about emojie issue..... 😞
2024-07-06 14:47:39 -05:00
Ian Manske
fa183b6669
help operators refactor (#13307)
# Description
Refactors `help operators` so that its output is always up to date with
the parser.

# User-Facing Changes
- The order of output rows for `help operators` was changed.
- `not` is now listed as a boolean operator instead of a comparison
operator.
- Edited some of the descriptions for the operators.
2024-07-06 13:09:12 -05:00
Yash Thakur
de2b752771
Fix variable completion sort order (#13306)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

My last PR (https://github.com/nushell/nushell/pull/13242) made it so
that the last branch in the variable completer doesn't sort suggestions.
Sorry about that. This should fix it.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Variables will now be sorted properly.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added one test case to verify this won't happen again.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-05 17:58:35 -05:00
Devyn Cairns
948b90299d
Preserve attributes on external ByteStreams (#13305)
# Description

Bug fix: `PipelineData::check_external_failed()` was not preserving the
original `type_` and `known_size` attributes of the stream passed in for
streams that come from children, so `external-command | into binary` did
not work properly and always ended up still being unknown type.

# User-Facing Changes
The following test case now works as expected:

```nushell
> head -c 2 /dev/urandom | into binary
# Expected: pretty hex dump of binary
# Previous behavior: just raw binary in the terminal
```

# Tests + Formatting
Added a test to cover this to `into binary`
2024-07-05 21:10:41 +00:00
Himadri Bhattacharjee
34da26d039
fix: exotic types return float on division, self on modulo (#13301)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Related to #13298

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Exotic types like `Duration` and `Filesize` return a float on division
by the same type, i.e., the unit is gone since division results in a
scalar. When using the modulo operator, the output type has the same
unit.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Division results in a float like the following:

```sh
~/Public/nushell> 512sec / 3sec
170.66666666666666
```

Modulo results in an output with the same unit:

```sh
~/Public/nushell> 512sec mod 3sec
2sec
```

Type checking isn't confused with output types:

```sh
~/Public/nushell> (512sec mod 3sec) / 0.5sec
4
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Existing tests are passing.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-05 09:01:27 -05:00
Reilly Wood
8707d14f95
Limit drilling down inside explore (#13293)
This PR fixes an issue with `explore` where you can "drill down" into
the same value forever. For example:

1. Run `ls | explore`
2. Press Enter to enter cursor mode
3. Press Enter again to open the selected string in a new layer
4. Press Enter again to open that string in a new layer
5. Press Enter again to open that string in a new layer
6. Repeat and eventually you have a bajillion layers open with the same
string

IMO it only makes sense to "drill down" into lists and records.

In a separate commit I also did a little refactoring, cleaning up naming
and comments.
2024-07-05 07:18:25 -05:00
Wind
1514b9fbef
don't show result in error make examples (#13296)
# Description
Fixes: #13189 

The issue is caused `error make` returns a `Value::Errror`, and when
nushell pass it to `table -e` in `std help`, it directly stop and render
the error message.
To solve it, I think it's safe to make these examples return None
directly, it doesn't change the reult of `help error make`.

# User-Facing Changes
## Before
```nushell
~> help "error make"
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
     ╭─[NU_STDLIB_VIRTUAL_DIR/std/help.nu:692:21]
 691 │ ] {
 692 │     let commands = (scope commands | sort-by name)
     ·                     ───────┬──────
     ·                            ╰── source value
 693 │
     ╰────

Error:   × my custom error message
```

## After
```nushell
Create an error.

Search terms: panic, crash, throw

Category: core

This command:
- does not create a scope.
- is a built-in command.
- is a subcommand.
- is not part of a plugin.
- is not a custom command.
- is not a keyword.

Usage:
  > error make {flags} <error_struct>


Flags:

  -u, --unspanned - remove the origin label from the error
  -h, --help - Display the help message for this command

Signatures:

  <nothing> | error make[ <record>] -> <any>

Parameters:

  error_struct: <record> The error to create.


Examples:
  Create a simple custom error
  > error make {msg: "my custom error message"}


  Create a more complex custom error
  > error make {
        msg: "my custom error message"
        label: {
            text: "my custom label text"  # not mandatory unless $.label exists
            # optional
            span: {
                # if $.label.span exists, both start and end must be present
                start: 123
                end: 456
            }
        }
        help: "A help string, suggesting a fix to the user"  # optional
    }


  Create a custom error for a custom command that shows the span of the argument
  > def foo [x] {
        error make {
            msg: "this is fishy"
            label: {
                text: "fish right here"
                span: (metadata $x).span
            }
        }
    }
```
# Tests + Formatting
Added 1 test
2024-07-05 07:17:07 -05:00
Andy Gayton
b27cd70fd1
remove the deprecated register command (#13297)
# Description

This PR removes the `register` command which has been
[deprecated](https://www.nushell.sh/blog/2024-04-30-nushell_0_93_0.html#register-toc)
in favor of [`plugin
add`](https://www.nushell.sh/blog/2024-04-30-nushell_0_93_0.html#redesigned-plugin-management-commands-toc)

# User-Facing Changes

`register` is no longer available
2024-07-05 07:16:50 -05:00
Himadri Bhattacharjee
afaa019fae
feat: replace unfold with from_fn for the generate command (#13299)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.


you can also mention related issues, PRs or discussions!
-->
This PR should close #13247 

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

- The deprecated `itertools::unfold` function is replaced with
`std::iter::from_fn` for the generate command.
- The mutable iterator state is no longer passed as an argument to
`from_fn` but it gets captured with the closure's `move`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
No user facing changes

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests for the generate command are passing locally.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-07-05 07:16:21 -05:00
Darren Schroeder
8833d3f89f
change duration mod duration to duration instead of float (#13300)
# Description

closes #13298 so that duration mod duration / duration = duration

### Before
```nushell
(92sec mod 1min) / 1sec
Error: nu::parser::unsupported_operation

  × division is not supported between float and duration.
   ╭─[entry #5:1:1]
 1 │ (92sec mod 1min) / 1sec
   · ────────┬─────── ┬ ──┬─
   ·         │        │   ╰── duration
   ·         │        ╰── doesn't support these values
   ·         ╰── float
   ╰────
```
### After
```nushell
❯ (92sec mod 1min) / 1sec
32
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-05 07:16:03 -05:00
Sang-Heon Jeon
d5e00c0d5d
Support default offset with dateformat option (#13289)
# Description
Fixes #13280. After apply this patch, we can use non-timezone string +
format option `into datetime` cmd

# User-Facing Changes
AS-IS (before fixing)
```
$ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T'
Error: nu:🐚:cant_convert

  × Can't convert to could not parse as datetime using format '%m.%d.%Y %T'.
   ╭─[entry #1:1:25]
 1 │ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T'
   ·                         ──────┬──────
   ·                               ╰── can't convert input is not enough for unique date and time to could not parse as datetime using format '%m.%d.%Y %T'
   ╰────
  help: you can use `into datetime` without a format string to enable flexible parsing

$ "09.02.2024 11:06:11" | into datetime
Mon, 2 Sep 2024 11:06:11 +0900 (in 2 months)
```

TO-BE(After fixing)

```
$ "09.02.2024 11:06:11" | into datetime --format '%m.%d.%Y %T'
Mon, 2 Sep 2024 20:06:11 +0900 (in 2 months)

$ "09.02.2024 11:06:11" | into datetime 
Mon, 2 Sep 2024 11:06:11 +0900 (in 2 months)
```


# Tests + Formatting
If there is agreement on the direction, I will add a test.

# After Submitting

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-07-04 10:44:12 -05:00
Jakub Žádník
3fae77209a
Revert "Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)" (#13292)
This reverts commit 0cfd5fbece.

The original PR messed up syntax higlighting of aliases and causes
panics of completion in the presence of alias.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-04 00:02:13 +03:00
NotTheDr01ds
ca7a2ae1d6
for - remove deprecated --numbered (#13239)
# Description

Complete the `--numbered` removal that was started with the deprecation
in #13112.

# User-Facing Changes

Breaking change - Use `| enumerate` in place of `--numbered` as shown in
the help example

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Searched online doc for `--numbered` to ensure no other usage needed to
be updated.
2024-07-03 07:55:41 -05:00
Darren Schroeder
ba1d900020
create a better error message when saving fails (#13290)
# Description

This PR just creates a better error message when the `save` command
fails.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 07:43:07 -05:00
Yash Thakur
9e738193f3
Force completers to sort in fetch() (#13242)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR fixes the problem pointed out in
https://github.com/nushell/nushell/issues/13204, where the Fish-like
completions aren't sorted properly (this PR doesn't close that issue
because the author there wants more than just fixed sort order).

The cause is all of the file/directory completions being fetched first
and then sorted all together while being treated as strings. Instead,
this PR sorts completions within each individual directory, avoiding
treating `/` as part of the path.

To do this, I removed the `sort` method from the completer trait (as
well as `get_sort_by`) and made all completers sort within the `fetch`
method itself. A generic `sort_completions` helper has been added to
sort lists of completions, and a more specific `sort_suggestions` helper
has been added to sort `Vec<Suggestion>`s.

As for the actual change that fixes the sort order for file/directory
completions, the `complete_rec` helper now sorts the children of each
directory before visiting their children. The file and directory
completers don't bother sorting at the end (except to move hidden files
down).

To reviewers: don't let the 29 changed files scare you, most of those
are just the test fixtures :)

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

This is the current behavior with prefix matching:

![image](https://github.com/nushell/nushell/assets/45539777/6a36e003-8405-45b5-8cbe-d771e0592709)

And with fuzzy matching:

![image](https://github.com/nushell/nushell/assets/45539777/f2cbfdb2-b8fd-491b-a378-779147291d2a)

Notice how `partial/hello.txt` is the last suggestion, even though it
should come before `partial-a`. This is because the ASCII code for `/`
is greater than that of `-`, so `partial-` is put before `partial/`.

This is this PR's behavior with prefix matching (`partial/hello.txt` is
at the start):

![image](https://github.com/nushell/nushell/assets/45539777/3fcea7c9-e017-428f-aa9c-1707e3ab32e0)

And with fuzzy matching:

![image](https://github.com/nushell/nushell/assets/45539777/d55635d4-cdb8-440a-84d6-41111499f9f8)

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- Modified the partial completions test fixture to test whether this PR
even fixed anything
- Modified fixture to test sort order of .nu completions (a previous
version of my changes didn't sort all the completions at the end but
there were no tests catching that)
- Added a test for making sure subcommand completions are sorted by
Levenshtein distance (a previous version of my changes sorted in
alphabetical order but there were no tests catching that)

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 06:48:06 -05:00
Jack Wright
8316a1597e
Polars: Check to see if the cache is empty before enabling GC. More logging (#13286)
There was a bug where anytime the plugin cache remove was called, the
plugin gc was turned back on. This probably happened when I added the
reference counter logic.
2024-07-03 06:44:26 -05:00
Jakub Žádník
0cfd5fbece
Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Part of https://github.com/nushell/nushell/issues/12963, step 2.

This PR refactors Call and related argument structures to remove their
dependency on `Expression::span` which will be removed in the future.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Should be none. If you see some error messages that look broken, please
report.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 09:00:52 +03:00
Jack Wright
122ff1f19c
Add the ability to set content-type metadata with metadata set (#13284)
# Description

With #13254, the content-type pipeline metadata field was added. This
pull request allows it to be manipulated with `metadata set`

# User-Facing Changes
* `metadata set` now has a `--content-type` flag
2024-07-02 13:35:55 -07:00
Jack Wright
0d060aeae8
Use pipeline data for http post|put|patch|delete commands. (#13254)
# Description
Provides the ability to use http commands as part of a pipeline.
Additionally, this pull requests extends the pipeline metadata to add a
content_type field. The content_type metadata field allows commands such
as `to json` to set the metadata in the pipeline allowing the http
commands to use it when making requests.

This pull request also introduces the ability to directly stream http
requests from streaming pipelines.

One other small change is that Content-Type will always be set if it is
passed in to the http commands, either indirectly or throw the content
type flag. Previously it was not preserved with requests that were not
of type json or form data.

# User-Facing Changes
* `http post`, `http put`, `http patch`, `http delete` can be used as
part of a pipeline
* `to text`, `to json`, `from json` all set the content_type metadata
field and the http commands will utilize them when making requests.
2024-07-01 12:34:19 -07:00
Ian Manske
e5cf4863e9
Fix clippy lint (#13277)
# Description
Fixes `items_after_test_module` lint.
2024-06-30 18:28:09 -05:00
alex-tdrn
69e4790b00
Skip decoration lines for detect columns --guess (#13274)
# Description
I introduced a regression in #13272 that resulted in `detect columns
--guess` to panic whenever it had to handle empty, whitespace-only, or
non-whitespace-only lines that go all the way to the last column (and as
such, cannot be considered to be lines that only have entries for the
first colum). I fix this by detecting these cases and skipping them,
since these are usually decoration lines. An example is the second line
output by `winget list`:

![image](https://github.com/nushell/nushell/assets/20356389/06c873fb-0a26-45dd-b020-3bcc737d027f)

What we don't want to skip, however, is lines that contain no
whitespace, and fit into the detected first column, since these lines
represent cases where data is only available for the first column, and
are not just decoration lines. For example (made up example, there are
no such entries in `winget lits`'s output), in this output we would not
want to skip the `Docker Desktop` line :
```
Name                                                        Id                                           Version     Available Source
-------------------------------------------------------------------------------------------------------------------------------------
AMD Software                                                ARPMachineX64AMD Catalyst Install Manager 24.4.1
AMD Ryzen Master                                            ARPMachineX64AMD Ryzen Master             2.13.0.2908
Docker Desktop
Mozilla Firefox (x64 en-US)                                 Mozilla.Firefox                              127.0.2               winget
```

![image](https://github.com/nushell/nushell/assets/20356389/12e31995-a7c1-4759-8c62-fb4fb199fd2e)

NOTE: `winget list | detect columns --guess` does not panic, but sadly
still does not work as expected. I believe this is not a nushell issue
anymore, but a `winget` one. When being piped, `winget` seems to add
extra whitespace and random `\r` symbols at the beginning of the text.
This messes with the column detection, of course.

![image](https://github.com/nushell/nushell/assets/20356389/7d1b7e5f-17d0-41c8-8d2f-7896e0d73d66)

![image](https://github.com/nushell/nushell/assets/20356389/56917954-1231-43e7-bacf-e5760e263054)

![image](https://github.com/nushell/nushell/assets/20356389/630bcfc9-eb78-4a45-9c8f-97efc0c224f4)


# User-Facing Changes
`detect columns --guess` should not panic when receiving output from
`winget list` at all anymore.

A breaking change is the skipping of decoration lines, especially since
scripts probably were doing something like
`winget list | lines | reject 1 | str join "\n" | detect columns
--guess`. This will now cause them to reject a line with valid data.

# Tests + Formatting
Added tests that exercise these edge cases, as well as a single-column
test to make sure that trivial cases keep working.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-30 07:38:41 -05:00
NotTheDr01ds
a2873336bb
Fix do signature (#13216)
Recommend holding until after #13125 is fully digested and *possibly*
until 0.96.

# Description

Fixes one of the issues described in #13125 

The `do` signature included a `SyntaxShape:Any` as one of the possible
first-positional types. This is incorrect. `do` only takes a closure as
a positional. This had the result of:

1. Moving what should have been a parser error to evaluation-time

   ## Before

   ```nu
   > do 1
   Error: nu:🐚:cant_convert

     × Can't convert to Closure.
      ╭─[entry #26:1:4]
    1 │ do 1
      ·    ┬
      ·    ╰── can't convert int to Closure
      ╰────
   ```

   ## After

   ```nu
   > do 1
   Error: nu::parser::parse_mismatch

     × Parse mismatch during operation.
      ╭─[entry #5:1:4]
    1 │ do 1
      ·    ┬
      ·    ╰── expected block, closure or record
      ╰────
   ```  

2. Masking a bad test in `std assert`

This is a bit convoluted, but `std assert` tests included testing
`assert error` to make sure it:

* Asserts on bad code
* Doesn't assert on good code

The good-code test was broken, and was essentially bad-code (really
bad-code) that wasn't getting caught due to the bad signature.

Fixing this resulted in *parse time* failures on every call to
`test_asserts` (not something that particular test was designed to
handle.

This PR also fixes the test case to properly evaluate `std assert error`
against a good code path.

# User-Facing Changes

* Error-type returned (possible breaking change?)

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-06-29 16:17:06 -05:00
Andy Gayton
4fe0f860a8
feat: add query webpage-info to plugin_nu_query (#13252)
# Description

This PR adds a new subcommand `query webpage-info` to `plugin_nu_query`.
The subcommand is a basic wrapper for the
[`webpage`](https://crates.io/crates/webpage) crate.

Usage:

```
http get https://phoronix.com | query webpage-info
```

and it returns a `Record` version of
[`webpage::HTML`](https://docs.rs/webpage/latest/webpage/struct.HTML.html).

The PR also takes a shot at bringing @lily-mara 's
[nu-serde::to_value](https://github.com/nushell/nushell/pull/3878/files)
back to life, updating it for the latest version of nushell. That's not
the main focus of the PR though - I just didn't want to have to
implement a custom converter for `webpage::HTML` 😅. If it looks
reasonable we could move it to `nu_protocol`(?) either in this PR or a
future one (along with adding tests for it).

# User-Facing Changes

no breaking changes
2024-06-29 16:13:31 -05:00
Darren Schroeder
33d0537cae
add str deunicode command (#13270)
# Description

Sometimes it's helpful to deal with only ASCII. This command will take a
unicode string as input and convert it to ASCII using the deunicode
crate.

```nushell
❯ "A…B" | str deunicode
A...B
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-29 16:12:34 -05:00
alex-tdrn
40e629beb1
Fix multibyte codepoint handling in detect columns --guess (#13272)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR fixes #13269. The splitting code in `guess_width.rs` was
creating slices from char indices, instead of byte indices. This works
perfectly fine for 1-byte code points, but panics or returns wrong
results as soon as multibyte codepoints appear in the input. I
originally discovered this by piping `winget list` into `detect columns
--guess`, since winget sometimes uses the unicode ellipsis symbol (`…`)
which is 3 bytes long when encoded in utf-8.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`detect columns --guess` should not crash due to multibyte unicode input
anymore

before:

![image](https://github.com/nushell/nushell/assets/20356389/833cd732-be3b-4158-97f7-0ca2616ce23f)

after:

![image](https://github.com/nushell/nushell/assets/20356389/15358b40-4083-4a33-9f2c-87e63f39d985)


# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- Added tests to `guess_width.rs` for testing handling of multibyte as
well as combining diacritical marks

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-29 16:12:17 -05:00
Tim Martin
153b45bc63
Surprising symlink resolution for std path add (#13258)
# Description
The standard library's `path add` function has some surprising side
effects that I attempt to address in this PR:

1. Paths added, if they are symbolic links, should not be resolved to
their targets. Currently, resolution happens.

   Imagine the following:

   ```nu
# Some time earlier, perhaps even not by the user, a symlink is created
   mkdir real-dir
   ln -s real-dir link-dir

   # Then, step to now, with link-dir that we want in our PATHS variable
   use std
   path add link-dir
   ```

In the current implementation of `path add`, it is _not_ `link-dir` that
will be added, as has been stated in the command. It is instead
`real-dir`. This is surprising. Users have the agency to do this
resolution if they wish with `path expand` (sans a `--no-symlink` flag):
for example, `path add (link-dir | path expand)`

In particular, when I was trying to set up
[fnm](https://github.com/Schniz/fnm), a Node.js version manager, I was
bitten by this fact when `fnm` told me that an expected path had not
been added to the PATHS variable. It was looking for the non-resolved
link. The user in [this
comment](https://github.com/Schniz/fnm/issues/463#issuecomment-1710050737)
was likely affected by this too.

Shells, such as nushell, can handle path symlinks just fine. Binary
lookup is unaffected. Let resolution be opt-in.

Lastly, there is some convention already in place for **not** resolving
path symlinks in the [default $env.ENV_CONVERSIONS
table](57452337ff/crates/nu-utils/src/sample_config/default_env.nu (L65)).
   
2. All existing paths in the path variable should be left untouched.
Currently, they are `path expand`-ed (including symbolic link
resolution).

   Path add should mean just that: prepend/append this path.

Instead, it currently means that, _plus mutate all other paths in the
variable_.

Again, users have the agency to do this with something like `$env.PATH =
$env.PATH | split row (char esep) | path expand`.

3. Minorly, I update documentation on running tests in
`crates/nu-std/CONTRIBUTING.md`. The offered command to run the standard
library test suite was no longer functional. Thanks to @weirdan in [this
Discord
conversation](https://discord.com/channels/601130461678272522/614593951969574961/1256029201119576147)
for the context.

# User-Facing Changes

(Written from the perspective of release notes)

- The standard library's `path add` function no longer resolves symlinks
in either the newly added paths, nor the other paths already in the
variable.

# Tests + Formatting

A test for the changes working correctly has been added to
`crates/nu-std/tests/test_std.nu` under the test named
`path_add_expand`.

You can quickly verify this new test and the existing `path add` test
with the following command:

```nu
cargo run -- -c 'use crates/nu-std/testing.nu; NU_LOG_LEVEL=INFO testing run-tests --path crates/nu-std --test path_add'
```

All commands suggested in the issue template have been run and complete
without error.

# After Submitting
I'll add a release note to [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged.
2024-06-28 18:11:48 -05:00
Bruce Weirdan
4f8d82bb88
Added support for multiple attributes to query web -a (#13256)
# Description

Allows specifying multiple attributes to retrieve from the selected
nodes. E.g. you may want to select both hrefs and targets from the list
of links:

```nushell
.... | query web --query a --attribute [href target]
```
# User-Facing Changes

`query web --attribute` previously accepted a string. Now it accepts
either a string or a list of strings.

The shape definition for this flag was relaxed temporarily, until
nushell/nushell#13253 is fixed.
2024-06-28 12:50:20 -05:00
Jack Wright
720b4cbd01
Polars 0.41 Upgrade (#13238)
# Description
Upgrading to Polars 0.41

# User-Facing Changes
* `polars melt` has been renamed to `polars unpivot` to match the change
in the polars API. Additionally, it now supports lazy dataframes.
Introduced a `--streamable` option to use the polars streaming engine
for lazy frames.
* The parameter `outer` has been replaced with `full` in `polars join`
to match polars change.
* `polars value-count` now supports the column (rename count column),
parallelize (multithread), sort, and normalize options.

The list of polars changes can be found
[here](https://github.com/pola-rs/polars/releases/tag/rs-0.41.2)
2024-06-28 06:37:45 -05:00
Devyn Cairns
a71732ba12
Add context to the I/O error messages in nu_cmd_plugin::util::modify_plugin_file() (#13259)
# Description

This might help @hustcer debug problems with `setup-nu`. The error
messages with the file I/O in `modify_plugin_file()` are not currently
not specific about what file path was involved in the I/O operation.

The spans on those errors have also changed to the span of the custom
path if provided.

# User-Facing Changes

- Slightly better error
2024-06-27 23:49:06 -07:00
Wind
57452337ff
Restrict strings beginning with quote should also ending with quote (#13131)
# Description
Closes: #13010

It adds an additional check inside `parse_string`, and returns
`unbalanced quote` if input string is unbalanced

# User-Facing Changes
After this pr, the following is no longer allowed:
```nushell
❯ "asdfasdf"asdfasdf
Error: nu::parser::extra_token_after_closing_delimiter

  × Invaild characters after closing delimiter
   ╭─[entry #1:1:11]
 1 │ "asdfasdf"asdfasdf
   ·           ────┬───
   ·               ╰── invalid characters
   ╰────
  help: Try removing them.
❯ 'asdfasd'adsfadf
Error: nu::parser::extra_token_after_closing_delimiter

  × Invaild characters after closing delimiter
   ╭─[entry #2:1:10]
 1 │ 'asdfasd'adsfadf
   ·          ───┬───
   ·             ╰── invalid characters
   ╰────
  help: Try removing them.
```

# Tests + Formatting
Added 1 test
2024-06-28 09:47:12 +08:00
Jack Wright
1f1f581357
Converted perf function to be a macro. Utilized the perf macro within the polars plugin. (#13224)
In this pull request, I converted the `perf` function within `nu_utils`
to a macro. This change facilitates easier usage within plugins by
allowing the use of `env_logger` and setting `RUST_LOG=nu_plugin_polars`
(or another plugin). Without this conversion, the `RUST_LOG` variable
would need to be set to `RUST_LOG=nu_utils::utils`, which is less
intuitive and impossible to narrow the perf results to one plugin.
2024-06-27 18:56:56 -05:00
suimong
0d79b63711
Fix find command output bug in the case of taking ByteStream input. (#13246)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

fixes #13245 

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

In addition to addressing #13245, this PR also updated one of the doc
example to the `find` command to document that non-regex mode is case
insensitive, which may surprise some users.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
-  🟢 `toolkit test`
-  🟢 `toolkit test stdlib`


# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Ben Yang <ben@ya.ng>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-06-27 09:46:10 -05:00
Bruce Weirdan
46ed69ab12
Add char nul (#13241)
# Description

Adds `char nul`, `char null_byte` and `char zero_byte` named characters.
All of them generate 0x00 byte.

# User-Facing Changes
Three new named characters are added:
 * `char nul` - generates 0x00 byte
 * `char null_byte` - generates 0x00 byte
 * `char zero_byte` - generates 0x00 byte
2024-06-26 18:49:44 -05:00
goldfish
ee74ec7423
Make the subcommands (from {csv, tsv, ssv}) 0-based for consistency (#13209)
# Description

fixed #11678

The sub-commands of from command (`from {csv, tsv, ssv}`) name columns
starting from index 0.
This behaviour is inconsistent with other commands such as `detect
columns`.
This PR makes the subcommands index 0-based.

# User-Facing Changes

The subcommands (`from {csv, tsv, ssv}`) return a table with the columns
starting at index 0 if no header data is passed.

```
~/Development/nushell> "foo bar baz" | from ssv -n -m 1 
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo     │ bar     │ baz     │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo,bar,baz" | from csv -n 
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo     │ bar     │ baz     │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo\tbar\tbaz" | from tsv -n
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo     │ bar     │ baz     │
╰───┴─────────┴─────────┴─────────╯
```


# Tests + Formatting

When I ran tests, `commands::touch::change_file_mtime_to_reference`
failed with the following error.
The error also occurs in the master branch, so it's probably unrelated
to these changes.
(maybe a problem with my dev environment)

```
$ toolkit check pr

~~~~~~~~

failures:

---- commands::touch::change_file_mtime_to_reference stdout ----
=== stderr

thread 'commands::touch::change_file_mtime_to_reference' panicked at crates/nu-command/tests/commands/touch.rs:298:9:
assertion `left == right` failed
  left: SystemTime { tv_sec: 1719149697, tv_nsec: 57576929 }
 right: SystemTime { tv_sec: 1719149697, tv_nsec: 78219489 }


failures:
    commands::touch::change_file_mtime_to_reference

test result: FAILED. 1533 passed; 1 failed; 32 ignored; 0 measured; 0 filtered out; finished in 10.87s

error: test failed, to rerun pass `-p nu-command --test main`
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
-  `toolkit test stdlib`
```

# After Submitting

nothing
2024-06-26 17:51:47 -05:00
Piepmatz
198aedb6c2
Use IntoValue and FromValue derive macros in nu_plugin_example for example usage (#13220)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The derive macros provided by #13031 are very useful for plugin authors.
In this PR I made use of these macros for two commands.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This Example usage could be highlighted in the changelog for plugin
authors as this is probably very useful for them.
2024-06-26 17:50:14 -05:00
NotTheDr01ds
58e8ea6084
Update and add ls examples (#13222)
# Description

Based on #13219, added several examples to `ls` doc to demonstrate
recursive directory listings. List of changes in this PR:

* Add example for `ls **/*` to demonstrate recursive listing using glob
pattern
* Add example for `ls ...(glob )`... to demonstrate recursive listing
using glob command
* Remove `-s` from an example where it had no use (since it was based on
the current directory and was not recursive)
* Update the description of `ls -a ~ `... to clarify that it lists the
full path of directories
* Update the description of `ls -as ~ `... (the difference being the
`-s`) to clarify that it lists only the filenames, not paths.


# User-Facing Changes

Help only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-06-26 17:49:52 -05:00
dependabot[bot]
020f4436d9
Bump shadow-rs from 0.28.0 to 0.29.0 (#13226) 2024-06-26 22:48:45 +00:00
dependabot[bot]
38ecb6d380
Bump uuid from 1.8.0 to 1.9.1 (#13227) 2024-06-26 06:43:46 +00:00
Ian Manske
5a486029db
Add typed path forms (#13115)
# Description
This PR adds new types to `nu-path` to enforce path invariants. Namely,
this PR adds:
- `Path` and `PathBuf`. These types are different from, but analogous to
`std::path::Path` and `std::path::PathBuf`.
- `RelativePath` and `RelativePathBuf`. These types must be/contain
strictly relative paths.
- `AbsolutePath` and `AbsolutePathBuf`. These types must be/contain
strictly absolute paths.
- `CanonicalPath` and `CanonicalPathBuf`. These types must be/contain
canonical paths.

Operations are prohibited as necessary to ensure that the invariants of
each type are upheld (needs double-checking).

Only paths that are absolute (or canonical) can be easily used as /
converted to `std::path::Path`s. This is to help force us to account for
the emulated current working directory instead of accidentally using the
current directory of the Nushell process (i.e.,
`std::env::current_dir`). Related to #12975 and #12976.

Note that this PR uses several declarative macros, as the file / this PR
would otherwise be 5000 lines long.

# User-Facing Changes
No major changes yet, just adds types to `nu-path` to be used in the
future.

# After Submitting
Actually use the new path types in all our crates where it makes sense,
removing usages of `std::path` types.
2024-06-25 18:33:57 -07:00
Wind
def36865ef
Enable reloading changes to a submodule (#13170)
# Description

Fixes: https://github.com/nushell/nushell/issues/12099

Currently if user run `use voice.nu`, and file is unchanged, then run
`use voice.nu` again. nushell will use the module directly, even if
submodule inside `voice.nu` is changed.

After discussed with @kubouch, I think it's ok to re-parse the module
file when:
1. It exports sub modules which are defined by a file
2. It uses other modules which are defined by a file

## About the change:
To achieve the behavior, we need to add 2 attributes to `Module`:
1. `imported_modules`: it tracks the other modules is imported by the
givem `module`, e.g: `use foo.nu`
2. `file`: the path of a module, if a module is defined by a file, it
will be `Some(path)`, or else it will be `None`.

After the change:

    use voice.nu always read the file and parse it.
    use voice will still use the module which is saved in EngineState.

# User-Facing Changes

use `xxx.nu` will read the file and parse it if it exports submodules or
uses submodules

# Tests + Formatting

Done

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-06-25 18:33:37 -07:00
Ian Manske
55ee476306
Define keywords (#13213)
# Description
Some commands in `nu-cmd-lang` are not classified as keywords even
though they should be.

# User-Facing Changes
In the output of `which`, `scope commands`, and `help commands`, some
commands will now have a `type` of `keyword` instead of `built-in`.
2024-06-25 18:32:54 -07:00
Darren Schroeder
f241110005
implement autoloading (#13217)
# Description

This PR implements script or module autoloading. It does this by finding
the `$nu.vendor-autoload-dir`, lists the contents and sorts them by file
name. These files are evaluated in that order.

To see what's going on, you can use `--log-level warn`
```
❯ cargo r -- --log-level warn
    Finished dev [unoptimized + debuginfo] target(s) in 0.58s
     Running `target\debug\nu.exe --log-level warn`
2024-06-24 09:23:20.494 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: config.nu, key: config-path, config_file_specified: None
2024-06-24 09:23:20.495 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: env.nu, key: env-path, config_file_specified: None
2024-06-24 09:23:20.629 PM [WARN ] nu::config_files: setup_config() config_file_specified: None, env_file_specified: None, login: false
2024-06-24 09:23:20.660 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: true
Hello, from env.nu
2024-06-24 09:23:20.679 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: false
Hello, from config.nu
Hello, from defs.nu
Activating Microsoft Visual Studio environment.
2024-06-24 09:23:21.398 PM [WARN ] nu::config_files: read_vendor_autoload_files() src\config_files.rs:234:9
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: read_vendor_autoload_files: C:\ProgramData\nushell\vendor\autoload
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\01_get-weather.nu"
2024-06-24 09:23:21.675 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\02_temp.nu"
2024-06-24 09:23:21.817 PM [WARN ] nu_cli::repl: Terminal doesn't support use_kitty_protocol config
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-25 18:31:54 -07:00
Jack Wright
0dd35cddcd
Bumping version to 0.95.1 (#13231)
Marks development for hotfix
2024-06-25 18:26:07 -07:00
Jakub Žádník
f93c6680bd
Bump to 0.95.0 (#13221)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-25 21:29:47 +03:00
Devyn Cairns
43e349a274
Mitigate the poor interaction between ndots expansion and non-path strings (#13218)
# Description

@hustcer reported that slashes were disappearing from external args
since #13089:

```
$> ossutil ls oss://abc/b/c
Error: invalid cloud url: "oss:/abc/b/c", please make sure the url starts with: "oss://"

$> ossutil ls 'oss://abc/b/c'
Error: oss: service returned error: StatusCode=403, ErrorCode=UserDisable, ErrorMessage="UserDisable", RequestId=66791EDEFE87B73537120838, Ec=0003-00000801, Bucket=abc, Object=
```

I narrowed this down to the ndots handling, since that does path parsing
and path reconstruction in every case. I decided to change that so that
it only activates if the string contains at least `...`, since that
would be the minimum trigger for ndots, and also to not activate it if
the string contains `://`, since it's probably undesirable for a URL.

Kind of a hack, but I'm not really sure how else we decide whether
someone wants ndots or not.

# User-Facing Changes
- bare strings not containing ndots are not modified
- bare strings containing `://` are not modified

# Tests + Formatting
Added tests to prevent regression.
2024-06-24 16:39:01 -07:00
Bruce Weirdan
4509944988
Fix usage parsing for commands defined in CRLF (windows) files (#13212)
Fixes nushell/nushell#13207

# Description
This fixes the parsing of command usage when that command comes from a
file with CRLF line endings.

See nushell/nushell#13207 for more details.

# User-Facing Changes
Users on Windows will get correct autocompletion for `std` commands.
2024-06-23 18:43:05 -05:00
Piepmatz
9b7f899410
Allow missing fields in derived FromValue::from_value calls (#13206)
# Description
In #13031 I added the derive macros for `FromValue` and `IntoValue`. In
that implementation, in particular for structs with named fields, it was
not possible to omit fields while loading them from a value, when the
field is an `Option`. This PR adds extra handling for this behavior, so
if a field is an `Option` and that field is missing in the `Value`, then
the field becomes `None`. This behavior is also tested in
`nu_protocol::value::test_derive::missing_options`.

# User-Facing Changes
When using structs for options or similar, users can now just emit
fields in the record and the derive `from_value` method will be able to
understand this, if the struct has an `Option` type for that field.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
A showcase for this feature would be great, I tried to use the current
derive macro in a plugin of mine for a config but without this addition,
they are annoying to use. So, when this is done, I would add an example
for such plugin configs that may be loaded via `FromValue`.
2024-06-22 13:31:09 -07:00
NotTheDr01ds
b6bdadbc6f
Suppress column index for default cal output (#13188)
# Description

* As discussed in the comments in #11954, this suppresses the index
column on `cal` output. It does that by running `table -i false` on the
results by default.
* Added new `--as-table/-t` flag to revert to the old behavior and
output the calendar as structured data
* Updated existing tests to use `--as-table`
* Added new tests against the string output
* Updated `length` test which also used `cal`
* Added new example for `--as-table`, with result

# User-Facing Changes

## Breaking change

The *default* `cal` output has changed from a `list` to a `string`. To
obtain structured data from `cal`, use the new `--as-table/-t` flag.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-22 07:41:29 -05:00
NotTheDr01ds
dcb6ab6370
Fixed generate command signature (#13200)
# Description

Removes `list<any>` as an input type for the `generate` command. This
command does not accept pipeline input (and cannot, logically). This can
be seen by the use of `_input` in the command's `run()`.

Also, due to #13199, in order to pass `toolkit check pr`, one of the
examples was changed to remove the `result`. This is probably a better
demonstration of the ability of the command to infinitely generate a
list anyway, and an infinite list can't be represented in a `result`.

# User-Facing Changes

Should only be a change to the help. The input type was never valid and
couldn't have been used.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-06-22 07:37:34 -05:00
Jack Wright
db86dd9f26
Polars default infer (#13193)
Addresses performance issues that @maxim-uvarov found with CSV and JSON
lines.

This ensures that the schema inference follows the polars defaults of
100 lines. Recent changes caused the default values to be override and
caused the entire file to be scanned when inferring the schema.
2024-06-22 07:23:42 -05:00
Maxim Zhiburt
10e84038af
nu-explore: Add vertical lines && fix index/transpose issue (#13147)
Somehow I believe that split lines were implemented originally; (I
haven't got to find it though; from a quick look)
I mean a long time ago before a lot a changes were made.

Probably adding horizontal lines would make also some sense.

ref #13116
close #13140

Take care

________________

If `explore` is used, frequently, or planned to be so.
I guess it would be a good one to create a test suite for it; to not
break things occasionally 😅

I did approached it one time back then using `expectrl` (literally
`expect`), but there was some issues.
Maybe smth. did change.
Or some `clean` mode could be introduced for it, to being able to be
used by outer programs; to control `nu`.

Just thoughts here.
2024-06-21 12:07:57 -07:00
Devyn Cairns
91d44f15c1
Allow plugins to report their own version and store it in the registry (#12883)
# Description

This allows plugins to report their version (and potentially other
metadata in the future). The version is shown in `plugin list` and in
`version`.

The metadata is stored in the registry file, and reflects whatever was
retrieved on `plugin add`, not necessarily the running binary. This can
help you to diagnose if there's some kind of mismatch with what you
expect. We could potentially use this functionality to show a warning or
error if a plugin being run does not have the same version as what was
in the cache file, suggesting `plugin add` be run again, but I haven't
done that at this point.

It is optional, and it requires the plugin author to make some code
changes if they want to provide it, since I can't automatically
determine the version of the calling crate or anything tricky like that
to do it.

Example:

```
> plugin list | select name version is_running pid
╭───┬────────────────┬─────────┬────────────┬─────╮
│ # │      name      │ version │ is_running │ pid │
├───┼────────────────┼─────────┼────────────┼─────┤
│ 0 │ example        │ 0.93.1  │ false      │     │
│ 1 │ gstat          │ 0.93.1  │ false      │     │
│ 2 │ inc            │ 0.93.1  │ false      │     │
│ 3 │ python_example │ 0.1.0   │ false      │     │
╰───┴────────────────┴─────────┴────────────┴─────╯
```

cc @maxim-uvarov (he asked for it)

# User-Facing Changes

- `plugin list` gets a `version` column
- `version` shows plugin versions when available
- plugin authors *should* add `fn metadata()` to their `impl Plugin`,
but don't have to

# Tests + Formatting

Tested the low level stuff and also the `plugin list` column.

# After Submitting
- [ ] update plugin guide docs
- [ ] update plugin protocol docs (`Metadata` call & response)
- [ ] update plugin template (`fn metadata()` should be easy)
- [ ] release notes
2024-06-21 06:27:09 -05:00
Devyn Cairns
dd8f8861ed
Add shape_glob_interpolation to default_config.nu (#13198)
# Description

Just missed this during #13089. Adds `shape_glob_interpolation` to the
config.

This actually isn't really going to be seen at all yet, so I debated
whether it's really needed at all. It's only used to highlight the
quotes themselves, and we don't have any quoted glob interpolations at
the moment.
2024-06-21 06:17:29 -05:00
Devyn Cairns
9845d13347
fix nu-system build on arm64 FreeBSD (#13196)
# Description

Fixes #13194

`ki_stat` is supposed to be a `c_char`, but was defined was `i8`.
Unfortunately, `c_char` is `u8` on Aarch64 (on all platforms), so this
doesn't compile. I fixed it to use `c_char` instead.

Double checked whether NetBSD is affected, but the `libc` code defines
it as `i8` for some reason (erroneously, really) but that doesn't matter
too much. Anyway should be ok there.

Confirmed to be working.
2024-06-21 03:03:10 -07:00
NotTheDr01ds
4c82a748c1
Do example (#13190)
# Description

#12056 added support for default and type-checked arguments in `do`
closures.

This PR adds examples for those features.  It also:

* Fixes the TODO (a closure parameter that wasn't being used) that was
preventing a result from being added
* Removes extraneous commas from the descriptions
* Adds an example demonstrating multiple positional closure arguments

# User-Facing Changes

Help examples only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-20 18:46:56 -05:00
Jack Wright
20834c9d47
Added the ability to turn on performance debugging through and env var for the polars plugin (#13191)
This allows performance debugging to be turned on by setting:

```nushell
$env.POLARS_PLUGIN_PERF = "true"
```

Furthermore, this improves the other plugin debugging by allowing the
env variable for debugging to be set at any time versus having to be
available when nushell is launched:

```nushell
$env.POLARS_PLUGIN_DEBUG = "true"
```

This plugin introduces a `perf` function that will output timing
results. This works very similar to the perf function available in
nu_utils::utils::perf. This version prints everything to std error to
not break the plugin stream and uses the engine interface to see if the
env variable is configured.

This pull requests uses this `perf` function when:
* opening csv files as dataframes
* opening json lines files as dataframes

This will hopefully help provide some more fine grained information on
how long it takes polars to open different dataframes. The `perf` can
also be utilized later for other dataframes use cases.
2024-06-20 16:37:38 -07:00
Jack Wright
7d2d573eb8
Added the ability to open json lines dataframes with polars lazy json lines reader. (#13167)
The `--lazy` flag will now use the polars' LazyJsonLinesReader when
opening a json lines file with `polars open`
2024-06-20 10:55:49 -07:00
Darren Schroeder
c09a8a5ec9
add a system level folder for future autoloading (#13180)
# Description

This PR adds a directory to the `$nu` constant that shows where the
system level autoload directory is located at. This folder is modifiable
at compile time with environment variables.
```rust
    // Create a system level directory for nushell scripts, modules, completions, etc
    // that can be changed by setting the NU_VENDOR_AUTOLOAD_DIR env var on any platform
    // before nushell is compiled OR if NU_VENDOR_AUTOLOAD_DIR is not set for non-windows 
    // systems, the PREFIX env var can be set before compile and used as PREFIX/nushell/vendor/autoload

    // pseudo code
    // if env var NU_VENDOR_AUTOLOAD_DIR is set, in any platform, use it
    // if not, if windows, use ALLUSERPROFILE\nushell\vendor\autoload
    // if not, if non-windows, if env var PREFIX is set, use PREFIX/share/nushell/vendor/autoload
    // if not, use the default /usr/share/nushell/vendor/autoload
```

### Windows default
```nushell
❯ $nu.vendor-autoload-dir
C:\ProgramData\nushell\vendor\autoload
```
### Non-Windows default
```nushell
❯ $nu.vendor-autoload-dir
/usr/local/share/nushell/vendor/autoload
```
### Non-Windows with PREFIX set
```nushell
❯ PREFIX=/usr/bob cargo r
❯ $nu.vendor-autoload-dir
/usr/bob/share/nushell/vendor/autoload
```
### Non-Windows with NU_VENDOR_AUTOLOAD_DIR set
```nushell
❯ NU_VENDOR_AUTOLOAD_DIR=/some/other/path/nushell/stuff cargo r
❯ $nu.vendor-autoload-dir
/some/other/path/nushell/stuff
```

> [!IMPORTANT] 
To be clear, this PR does not do the auto-loading, it just sets up the
folder to support that functionality that can be added in a later PR.
The PR also does not create the folder defined. It's just setting the
$nu constant.
 
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-20 06:30:43 -05:00
Devyn Cairns
bdc32345bd
Move most of the peculiar argument handling for external calls into the parser (#13089)
# Description

We've had a lot of different issues and PRs related to arg handling with
externals since the rewrite of `run-external` in #12921:

- #12950
- #12955
- #13000
- #13001
- #13021
- #13027
- #13028
- #13073

Many of these are caused by the argument handling of external calls and
`run-external` being very special and involving the parser handing
quoted strings over to `run-external` so that it knows whether to expand
tildes and globs and so on. This is really unusual and also makes it
harder to use `run-external`, and also harder to understand it (and
probably is part of the reason why it was rewritten in the first place).

This PR moves a lot more of that work over to the parser, so that by the
time `run-external` gets it, it's dealing with much more normal Nushell
values. In particular:

- Unquoted strings are handled as globs with no expand
- The unescaped-but-quoted handling of strings was removed, and the
parser constructs normal looking strings instead, removing internal
quotes so that `run-external` doesn't have to do it
- Bare word interpolation is now supported and expansion is done in this
case
- Expressions typed as `Glob` containing `Expr::StringInterpolation` now
produce `Value::Glob` instead, with the quoted status from the expr
passed through so we know if it was a bare word
- Bare word interpolation for values typed as `glob` now possible, but
not implemented
- Because expansion is now triggered by `Value::Glob(_, false)` instead
of looking at the expr, externals now support glob types

# User-Facing Changes

- Bare word interpolation works for external command options, and
otherwise embedded in other strings:
  ```nushell
  ^echo --foo=(2 + 2) # prints --foo=4
  ^echo -foo=$"(2 + 2)" # prints -foo=4
  ^echo foo="(2 + 2)" # prints (no interpolation!) foo=(2 + 2)
  ^echo foo,(2 + 2),bar # prints foo,4,bar
  ```

- Bare word interpolation expands for external command head/args:
  ```nushell
  let name = "exa"
  ~/.cargo/bin/($name) # this works, and expands the tilde
  ^$"~/.cargo/bin/($name)" # this doesn't expand the tilde
  ^echo ~/($name)/* # this glob is expanded
  ^echo $"~/($name)/*" # this isn't expanded
  ```

- Ndots are now supported for the head of an external command
(`^.../foo` works)

- Glob values are now supported for head/args of an external command,
and expanded appropriately:
  ```nushell
  ^("~/.cargo/bin/exa" | into glob) # the tilde is expanded
  ^echo ("*.txt" | into glob) # this glob is expanded
  ```

- `run-external` now works more like any other command, without
expecting a special call convention
  for its args:
  ```nushell
  run-external echo "'foo'"
  # before PR: 'foo'
  # after PR:  foo
  run-external echo "*.txt"
  # before PR: (glob is expanded)
  # after PR:  *.txt
  ```

# Tests + Formatting
Lots of tests added and cleaned up. Some tests that weren't active on
Windows changed to use `nu --testbin cococo` so that they can work.
Added a test for Linux only to make sure tilde expansion of commands
works, because changing `HOME` there causes `~` to reliably change.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] release notes: make sure to mention the new syntaxes that are
supported
2024-06-19 21:00:03 -07:00
NotTheDr01ds
44aa0a2de4
Table help rendering (#13182)
# Description

Mostly fixes #13149 with much of the credit to @fdncred.

This PR runs `table --expand` against `help` example results. This is
essentially the same fix that #13146 was for `std help`.

It also changes the shape of the result for the `table --expand`
example, as it was hardcoded wrong.

~Still needed is a fix for the `table --collapse` example.~ Note that
this is also still a bug in `std help` that I didn't noticed before.

# User-Facing Changes

Certain tables are now rendered correctly in the help examples for:

* `table`
* `zip`
* `flatten`
* And almost certainly others

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-06-19 20:12:25 -05:00
Devyn Cairns
12991cd36f
Change the error style during tests to plain (#13061)
# Description

This fixes issues with trying to run the tests with a terminal that is
small enough to cause errors to wrap around, or in cases where the test
environment might produce strings that are reasonably expected to wrap
around anyway. "Fancy" errors are too fancy for tests to work
predictably 😉

cc @abusch

# User-Facing Changes

- Added `--error-style` option for use with `--commands` (like
`--table-mode`)

# Tests + Formatting

Surprisingly, all of the tests pass, including in small windows! I only
had to make one change to a test for `error make` which was looking for
the box drawing characters miette uses to determine whether the span
label was showing up - but the plain error style output is even better
and easier to match on, so this test is actually more specific now.
2024-06-18 21:37:24 -07:00
dependabot[bot]
103f59be52
Bump git2 from 0.18.3 to 0.19.0 (#13179) 2024-06-19 01:09:25 +00:00
Darren Schroeder
a894e9c246
update try command's help (#13173)
# Description

This PR updates the `try` command to show that `catch` is a closure and
can be used as such.

### Before

![image](https://github.com/nushell/nushell/assets/343840/dc330b10-cd68-4d70-9ff8-aa1e7cbda5f3)

### After

![image](https://github.com/nushell/nushell/assets/343840/146a7514-6026-4b53-bdf0-603c77c8a259)


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-18 13:48:06 -05:00
Bruce Weirdan
c171a2b8b7
Update sys users signature (#13172)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Fixes #13171 

# Description
Corrects the `sys users` signature to match the returned type.

Before this change `sys users | where name == root` would result in a
type error.
 
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-18 10:34:18 -05:00
Wind
28ed0fe700
Improves commands that support range input (#13113)
# Description
Fixes: #13105
Fixes: #13077

This pr makes `str substring`, `bytes at` work better with negative
index.

And it also fixes the false range semantic on `detect columns -c` in
some cases.

# User-Facing Changes
For `str substring`, `bytes at`, it will no-longer return an error if
start index is larger than end index. It makes sense to return an empty
string of empty bytes directly.

### Before
```nushell
# str substring
❯ ("aaa" | str substring 2..-3) == ""
Error: nu:🐚:type_mismatch

  × Type mismatch.
   ╭─[entry #23:1:10]
 1 │ ("aaa" | str substring 2..-3) == ""
   ·          ──────┬──────
   ·                ╰── End must be greater than or equal to Start
 2 │ true
   ╰────

# bytes at
❯ ("aaa" | encode utf-8 | bytes at 2..-3) == ("" | encode utf-8)
Error: nu:🐚:type_mismatch

  × Type mismatch.
   ╭─[entry #27:1:25]
 1 │ ("aaa" | encode utf-8 | bytes at 2..-3) == ("" | encode utf-8)
   ·                         ────┬───
   ·                             ╰── End must be greater than or equal to Start
   ╰────
```
### After
```nushell
# str substring
❯ ("aaa" | str substring 2..-3) == ""
true

# bytes at
❯  ("aaa" | encode utf-8 | bytes at 2..-3) == ("" | encode utf-8)
true
```
# Tests + Formatting
Added some tests, adjust existing tests
2024-06-18 07:19:13 -05:00
Devyn Cairns
97876fb0b4
Fix compilation for nu_protocol::value::from_value on 32-bit targets (#13169)
# Description

Needed to cast `usize::MAX` to `i64` for it to compile properly

cc @cptpiepmatz
2024-06-18 07:16:08 -05:00
Piepmatz
b79a2255d2
Add derive macros for FromValue and IntoValue to ease the use of Values in Rust code (#13031)
# Description
After discussing with @sholderbach the cumbersome usage of
`nu_protocol::Value` in Rust, I created a derive macro to simplify it.
I’ve added a new crate called `nu-derive-value`, which includes two
macros, `IntoValue` and `FromValue`. These are re-exported in
`nu-protocol` and should be encouraged to be used via that re-export.

The macros ensure that all types can easily convert from and into
`Value`. For example, as a plugin author, you can define your plugin
configuration using a Rust struct and easily convert it using
`FromValue`. This makes plugin configuration less of a hassle.

I introduced the `IntoValue` trait for a standardized approach to
converting values into `Value` (and a fallible variant `TryIntoValue`).
This trait could potentially replace existing `into_value` methods.
Along with this, I've implemented `FromValue` for several standard types
and refined other implementations to use blanket implementations where
applicable.

I made these design choices with input from @devyn.

There are more improvements possible, but this is a solid start and the
PR is already quite substantial.

# User-Facing Changes

For `nu-protocol` users, these changes simplify the handling of
`Value`s. There are no changes for end-users of nushell itself.

# Tests + Formatting
Documenting the macros itself is not really possible, as they cannot
really reference any other types since they are the root of the
dependency graph. The standard library has the same problem
([std::Debug](https://doc.rust-lang.org/stable/std/fmt/derive.Debug.html)).
However I documented the `FromValue` and `IntoValue` traits completely.

For testing, I made of use `proc-macro2` in the derive macro code. This
would allow testing the generated source code. Instead I just tested
that the derived functionality is correct. This is done in
`nu_protocol::value::test_derive`, as a consumer of `nu-derive-value`
needs to do the testing of the macro usage. I think that these tests
should provide a stable baseline so that users can be sure that the impl
works.

# After Submitting
With these macros available, we can probably use them in some examples
for plugins to showcase the use of them.
2024-06-17 16:05:11 -07:00
NotTheDr01ds
3a6d8aac0b
Return an empty list when no std help --find results are found (#13160)
# Description

Fixes #13143 by returning an empty list when there are no results found
by `std help --find/-f`

# User-Facing Changes

In addition, prints a message to stderr.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-15 12:27:55 -05:00
NotTheDr01ds
b1cf0e258d
Expand tables in help examples in std (#13146)
# Description

Some command help has example results with nested `table` data which is
displayed as the "non-expanded" form. E.g.:

```nu
╭───┬────────────────╮
│ 0 │ [list 2 items] │
│ 1 │ [list 2 items] │
╰───┴────────────────╯
```

For a good example, see `help zip`.

While we could simply remove the offending Example `result`'s from the
command itself, `std help` is capable of expanding the table properly.
It already formats the output of each example result using `table`, so
simply making it a `table -e` fixes the output.

While I wish we had a way of expanding the tables in the builtin `help`,
that seems to be the same type of problem as in formatting the `cal`
output (see #11954).

I personally think it's better to add this feature to `std help` than to
remove the offending example results, but as long as `std help` is
optional, only a small percentage of users are going to see the
"expected" results.

# User-Facing Changes

Excerpt from `std help zip` before change:

```nu
Zip two lists
> [1 2] | zip [3 4]
╭───┬────────────────╮
│ 0 │ [list 2 items] │
│ 1 │ [list 2 items] │
╰───┴────────────────╯
```

After:

```nu
Zip two lists
> [1 2] | zip [3 4]
╭───┬───────────╮
│ 0 │ ╭───┬───╮ │
│   │ │ 0 │ 1 │ │
│   │ │ 1 │ 3 │ │
│   │ ╰───┴───╯ │
│ 1 │ ╭───┬───╮ │
│   │ │ 0 │ 2 │ │
│   │ │ 1 │ 4 │ │
│   │ ╰───┴───╯ │
╰───┴───────────╯
```

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
- 
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-13 19:56:11 -05:00
NotTheDr01ds
af6e1cb5a6
Added search terms to if (#13145)
# Description

In this PR, I continue my tradition of trivial but hopefully helpful
`help` tweaks. As mentioned in #13143, I noticed that `help -f else`
oddly didn't return the `if` statement itself. Perhaps not so oddly,
since who the heck is going to go looking for *"else"* in the help?
Well, I did ...

Added *"else"* and *"conditional"* to the search terms for `if`.

I'll work on the meat of #13143 next - That's more substantiative.

# User-Facing Changes

Help only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
- 
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-13 19:55:17 -05:00
Maxim Zhiburt
323d5457f9
Improve performance of explore - 1 (#13116)
Could be improved further I guess; but not here;

You can test the speed differences using data from #13088

```nu
open data.db | get profiles | explore
```

address: #13062

________

1. Noticed that search does not work anymore (even on `main` branch).
2. Not sure about resolved merged conflicts, seems fine, but maybe
something was lost.

---------

Co-authored-by: Reilly Wood <reilly.wood@icloud.com>
2024-06-12 18:35:04 -07:00
Ian Manske
634361b2d1
Make which-support feature non-optional (#13125)
# Description
Removes the `which-support` cargo feature and makes all of its
feature-gated code enabled by default in all builds. I'm not sure why
this one command is gated behind a feature. It seems to be a relic of
older code where we had features for what seems like every command.
2024-06-12 20:04:12 -05:00
NotTheDr01ds
bdbb096526
Fixed help for banner (#13138)
# Description

`help banner` had several issues:

* It used a Markdown link to an Asciinema recording, but Markdown links
aren't rendered as Markdown links by the help system (and can't be,
since (most?) terminals don't support that)
* Minor grammatical issues
* The Asciinema recording is out of date anyway. It still uses `use
stdn.nu banner` which isn't valid syntax any longer.

Since everyone at this point knows exactly what `banner` does 😉, I chose
to simply remove the link to the recording. Also tweaked the text
(initial caps and removed comma).

# User-Facing Changes

Help only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-06-12 14:26:58 -05:00
Darren Schroeder
0372e8c53c
add $nu.data-dir for completions and $nu.cache-dir for other uses (#13122)
# Description

This PR is an attempt to add a standard location for people to put
completions in. I saw this topic come up again recently and IIRC we
decided to create a standard location. I used the dirs-next crate to
dictate where these locations are. I know some people won't like that
but at least this gets the ball rolling in a direction that has a
standard directory.

This is what the default NU_LIB_DIRS looks like now in the
default_env.nu. It should also be like this when starting nushell with
`nu -n`
```nushell
$env.NU_LIB_DIRS = [
    ($nu.default-config-dir | path join 'scripts') # add <nushell-config-dir>/scripts
    ($nu.data-dir | path join 'completions') # default home for nushell completions
]
```

I also added these default folders to the `$nu` variable so now there is
`$nu.data-path` and `$nu.cache-path`.

## Data Dir Default

![image](https://github.com/nushell/nushell/assets/343840/aeeb7cd6-17b4-43e8-bb6f-986a0c7fce23)

While I was in there, I also decided to add a cache dir

## Cache Dir Default

![image](https://github.com/nushell/nushell/assets/343840/87dead66-4911-4f67-bfb2-acb16f386674)

### This is what the default looks like in Ubuntu.

![image](https://github.com/nushell/nushell/assets/343840/bca8eae8-8c18-47e8-b64f-3efe34f0004f)

### This is what it looks like with XDG_CACHE_HOME and XDG_DATA_HOME
overridden
```nushell
XDG_DATA_HOME=/tmp/data_home XDG_CACHE_HOME=/tmp/cache_home cargo r
```

![image](https://github.com/nushell/nushell/assets/343840/fae86d50-9821-41f1-868e-3814eca3730b)

### This is what the defaults look like in Windows (username scrubbed to
protect the innocent)

![image](https://github.com/nushell/nushell/assets/343840/3ebdb5cd-0150-448c-aff5-c57053e4788a)

How my NU_LIB_DIRS is set in the images above
```nushell
$env.NU_LIB_DIRS = [
    ($nu.default-config-dir | path join 'scripts') # add <nushell-config-dir>/scripts
    '/Users/fdncred/src/nu_scripts'
    ($nu.config-path | path dirname)
    ($nu.data-dir | path join 'completions') # default home for nushell completions
]
```

Let the debate begin.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-11 15:10:31 -04:00
Ian Manske
b0d1b4b182
Remove deprecated --not flag on str contains (#13124)
# Description
Removes the `str contains --not` flag that was deprecated in the last
minor release.

# User-Facing Changes
Breaking change since a flag was removed.
2024-06-11 15:00:00 -04:00
Darren Schroeder
a8376fad40
update uutils crates (#13130)
# Description

This PR updates the uutils/coreutils crates to the latest released
version.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-11 13:44:13 -05:00
NotTheDr01ds
c09488f515
Fix multiple issues with def --wrapped help example (#13123)
# Description

I've noticed this several times but kept forgetting to fix it:

The example given for `help def` for the `--wrapped` flag is:

```nu
Define a custom wrapper for an external command
> def --wrapped my-echo [...rest] { echo $rest }; my-echo spam
  ╭───┬──────╮
  │ 0 │ spam │
  ╰───┴──────╯
```

That's ... odd, since (a) it specifically says *"for an external"*
command, and yet uses (and shows the output from) the builtin `echo`.
Also, (b) I believe `--wrapped` is *only* applicable to external
commands. Finally, (c) the `my-echo spam` doesn't even demonstrate a
wrapped argument.

Unless I'm truly missing something, the example just makes no sense.

This updates the example to really demonstrate `def --wrapped` with the
*external* version of `^echo`. It uses the `-e` command to interpret the
escape-tab character in the string.

```nu
> def --wrapped my-echo [...rest] { ^echo ...$rest }; my-echo -e 'spam\tspam'
spam  spam
```

# User-Facing Changes

Help example only.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-10 19:12:54 -05:00
Ian Manske
944d941dec
path type error and not found changes (#13007)
# Description
Instead of an empty string, this PR changes `path type` to return null
if the path does not exist. If some other IO error is encountered, then
that error is bubbled up instead of treating it as a "not found" case.

# User-Facing Changes
- `path type` will now return null instead of an empty string, which is
technically a breaking change. In most cases though, I think this
shouldn't affect the behavior of scripts too much.
- `path type` can now error instead of returning an empty string if some
other IO error besides a "not found" error occurs.

Since this PR introduces breaking changes, it should be merged after the
0.94.1 patch.
2024-06-11 05:40:09 +08:00
Jakub Žádník
a55a48529d
Fix delta not being merged when evaluating menus (#13120)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

After parsing menu code, the changes weren't merged into the engine
state, which didn't produce any errors (somehow?) until the recent span
ID refactors. With this PR, menus get a new cloned engine state with the
parsed changes correctly merged in.

Hopefully fixes https://github.com/nushell/nushell/issues/13118

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-10 22:33:22 +03:00
Ian Manske
af22bb8d52
Remove old sys command behavior (#13114)
# Description

Removes the old, deprecated behavior of the `sys` command. That is, it
will no longer return the full system information record.

# User-Facing Changes

Breaking change: `sys` no longer outputs anything and will instead
display help text.
2024-06-10 06:31:47 -05:00
NotTheDr01ds
5b7e8bf1d8
Deprecate --numbered from for (#13112)
# Description

#7777 removed the `--numbered` flag from `each`, `par-each`, `reduce`,
and `each while`. It was suggested at the time that it should be removed
from `for` as well, but for several reasons it wasn't.

This PR deprecates `--numbered` in anticipation of removing it in 0.96.

Note: Please review carefully, as this is my first "real" Rust/Nushell
code. I was hoping that some prior commit would be useful as a template,
but since this was an argument on a parser keyword, I didn't find too
much useful. So I had to actually find the relevant helpers in the code
and `nu_protocol` doc and learn how to use them - oh darn ;-) But please
make sure I did it correctly.

# User-Facing Changes

* Use of `--numbered` will result in a deprecation warning.
* Changed help example to demonstrate the new syntax.
* Help shows deprecation notice on the flag
2024-06-10 03:01:22 +00:00
Jack Wright
021b8633cb
Allow the addition of an index column to be optional (#13097)
Per discussion on discord dataframes channel with @maxim-uvarov and pyz.

When converting a dataframe to an nushell value via `polars into-nu`,
the index column should not be added by default and should only be added
when specifying `--index`
2024-06-10 10:45:25 +08:00
Jack Wright
650ae537c3
Fix the use of right hand expressions in operations (#13096)
As reported by @maxim-uvarov and pyz in the dataframes discord channel:

```nushell
[[a b]; [1 1] [1 2] [2 1] [2 2] [3 1] [3 2]] | polars into-df | polars with-column ((polars col a) / (polars col b)) --name c


  × Type mismatch.
   ╭─[entry #45:1:102]
 1 │ [[a b]; [1 1] [1 2] [2 1] [2 2] [3 1] [3 2]] | polars into-df | polars with-column ((polars col a) / (polars col b)) --name c
   ·                                                                                                      ───────┬──────
   ·                                                                                                             ╰── Right hand side not a dataframe expression
   ╰────
```

This pull request corrects the type casting on the right hand side and
allows more than just polars literal expressions.
2024-06-10 10:44:04 +08:00
Sang-Heon Jeon
dc76183cd5
fix wrong casting with into filesize (#13110)
# Description
Fix wrong casting which is related to
https://github.com/nushell/nushell/pull/12974#discussion_r1618598336

# User-Facing Changes
AS-IS (before fixing)
```
$ "-10000PiB" | into filesize
6.2 EiB                                                         <--- Wrong casted value
$ "10000PiB" | into filesize 
-6.2 EiB                                                        <--- Wrong casted value
```

TO-BE (after fixing)
```
$ "-10000PiB" | into filesize
Error: nu:🐚:cant_convert

  × Can't convert to filesize.
   ╭─[entry #6:1:1]
 1 │ "-10000PiB" | into filesize
   · ─────┬─────
   ·      ╰── can't convert string to filesize
   ╰────

$ "10000PiB" | into filesize
Error: nu:🐚:cant_convert

  × Can't convert to filesize.
   ╭─[entry #7:1:1]
 1 │ "10000PiB" | into filesize
   · ─────┬────
   ·      ╰── can't convert string to filesize
   ╰────
```
2024-06-10 10:43:17 +08:00
Jakub Žádník
e52d7bc585
Span ID Refactor (Step 2): Use SpanId of expressions in some places (#13102)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Part of https://github.com/nushell/nushell/issues/12963, step 2.

This PR refactors changes the use of `expression.span` to
`expression.span_id` via a new helper `Expression::span()`. A new
`GetSpan` is added to abstract getting the span from both `EngineState`
and `StateWorkingSet`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

`format pattern` loses the ability to use variables in the pattern,
e.g., `... | format pattern 'value of {$it.name} is {$it.value}'`. This
is because the command did a custom parse-eval cycle, creating spans
that are not merged into the main engine state. We could clone the
engine state, add Clone trait to StateDelta and merge the cloned delta
to the cloned state, but IMO there is not much value from having this
ability, since we have string interpolation nowadays: `... | $"value of
($in.name) is ($in.value)"`.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-09 12:15:53 +03:00
NotTheDr01ds
d2121a155e
Fixes #13093 - Erroneous example in 'touch' help (#13095)
# Description

Fixes #13093 by:

* Removing the mentioned help example
* Updating the `--accessed` and `--modified` flag descriptions to remove
mention of "timestamp/date"

# User-Facing Changes

Help changes

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-06-07 09:33:48 -05:00
ymcx
cfe13397ed
Fix the colors when completing using a relative path (#12898)
# Description
Fixes a bug where the autocompletion menu has the wrong colors due to
'std::fs::symlink_metadata' not being able to handle relative paths.
Attached below are screenshots before and after applying the commit, in
which the colors are wrong when autocompleting on a path prefixed by a
tilde, whereas the same directory is highlighted correctly when prefixed
by a dot.

BEFORE:

![1715982514020](https://github.com/nushell/nushell/assets/89810988/62051f03-0846-430d-b493-e6f3cd6d0e04)

![1715982873231](https://github.com/nushell/nushell/assets/89810988/28c647ab-3b2a-47ef-9967-5d09927e299d)
AFTER:

![1715982490585](https://github.com/nushell/nushell/assets/89810988/7a370138-50af-42fd-9724-a34cc605bede)

![1715982894748](https://github.com/nushell/nushell/assets/89810988/e884f69f-f757-426e-98c4-bc9f7f6fc561)
2024-06-07 08:07:23 -05:00
Ian Manske
d6a9fb0e40
Fix display formatting for command type in help commands (#12996)
# Description
Related to #12832, this PR changes the way `help commands` displays the
command type to be consistent with `scope commands` and `which`.

# User-Facing Changes
Technically a breaking change since the `help commands` output can now
be different.
2024-06-07 08:03:31 -05:00
Access
a246a19387
fix: coredump without any messages (#13034)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.

- this PR should close https://github.com/nushell/nushell/issues/12874
- fixes https://github.com/nushell/nushell/issues/12874
I want to fix the issue which is induced by the fix for
https://github.com/nushell/nushell/issues/12369. after this pr. This pr
induced a new error for unix system, in order to show coredump messages

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

- this PR should close https://github.com/nushell/nushell/issues/12874
- fixes https://github.com/nushell/nushell/issues/12874
I want to fix the issue which is induced by the fix for
https://github.com/nushell/nushell/issues/12369. after this pr. This pr
induced a new error for unix system, in order to show coredump messages

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

after fix for 12874, coredump message is messing, so I want to fix it

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->


![image](https://github.com/nushell/nushell/assets/60290287/6d8ab756-3031-4212-a5f5-5f71be3857f9)

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-06-07 08:02:52 -05:00
Andrej Kolchin
2d0a60ac67
Use native toml datetime type in to toml (#13018)
# Description

Makes `to toml` use the `toml::value::Datetime` type, so that `to toml`
serializes dates properly.


# User-Facing Changes

`to toml` will now encode dates differently, in a native format instead
of a string. This could, in theory, break some workflows:

```Nushell
# Before:
~> {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
Fri, 31 May 2024 00:00:00 +0000 (10 hours ago)
# After:
~> {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
Error: nu:🐚:only_supports_this_input_type

  × Input type not supported.
   ╭─[entry #13:1:36]
 1 │ {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
   ·                                    ────┬────                  ──────┬──────
   ·                                        │                            ╰── only string and int input data is supported
   ·                                        ╰── input type: date
   ╰────
```

Fix #11751
2024-06-07 07:43:30 -05:00
Wind
83cf212e20
run_external.rs: use pathdiff::diff_path to handle relative path (#13056)
# Description
This pr is going to use `pathdiff::diff_path`, so we don't need to
handle for relative path by ourselves.

This is also the behavior before the rewritten of run_external.rs

It's a follow up to https://github.com/nushell/nushell/pull/13028

# User-Facing Changes
NaN

# Tests + Formatting
No need to add tests
2024-06-07 10:14:42 +08:00
Stefan Holderbach
eef4a89ff4
Move format date to Category::Strings (#13083)
The rest of the `format` commands live there.

Closes https://github.com/nushell/nushell.github.io/issues/1435
2024-06-06 18:32:08 -05:00
João Fidalgo
073d8850e9
Allow stor insert and stor update to accept pipeline input (#12882)
- this PR should close #11433 

# Description
This PR implements pipeline input support for the stor insert and stor
update commands,
enabling users to directly pass data to these commands without relying
solely on flag parameters.

Previously, it was only possible to specify the record data using flag
parameters,
which could be less intuitive and become cumbersome:

```bash
stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17}
```

Now it is also possible to pass a record through pipeline input:

```bash
{bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} | stor insert --table-name nudb
{str1: nushell datetime1: 2020-04-17} | stor update --table-name nudb"
```

Changes made on code:

- Modified stor insert and stor update to accept a record from the
pipeline.
- Added logic to handle data from the pipeline record.
- Implemented an error case to prevent simultaneous data input from both
pipeline and flag.

# User-facing changes

Returns an error when both ways of inserting data are used.

The examples for both commands were updated and in each command, when
the -d or -u fags are being used at the same time as input is being
passed through the pipeline, it returns an error:


![image](https://github.com/nushell/nushell/assets/120738170/c5b15c1b-716a-4df4-95e8-3bca8f7ae224)

Also returns an error when both of them are missing:


![image](https://github.com/nushell/nushell/assets/120738170/47f538ab-79f1-4fcc-9c62-d7a7d60f86a1)


# Tests + Formating
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

Co-authored-by: Rodrigo Friães <rodrigo.friaes@tecnico.ulisboa.pt>
2024-06-06 10:30:06 -05:00
Reilly Wood
f2f4b83886
Overhaul explore config (#13075)
Configuration in `explore` has always been confusing to me. This PR
overhauls (and simplifies, I think) how configuration is done.

# Details

1. Configuration is now strongly typed. In `Explore::run()` we create an
`ExploreConfig` struct from the more general Nu configuration and
arguments to `explore`, then pass that struct to other parts of
`explore` that need configuration. IMO this is a lot easier to reason
about and trace than the previous approach of creating a
`HashMap<String, Value>` and then using that to make various structs
elsewhere.
2. We now inherit more configuration from the config used for regular Nu
tables
1. Border/line styling now uses the `separator` style used for regular
Nu tables, the special `explore.split_line` config point has been
retired.
2. Cell padding in tables is now controlled by `table.padding` instead
of the undocumented `column_padding_left`/`column_padding_right` config
3. The (optional, previously not enabled by default) `selected_row` and
`selected_column` configuration has been removed. We now only highlight
the selected cell. I could re-add this if people really like the feature
but I'm guessing nobody uses it.

The interface still looks the same with a default/empty config.nu:


![image](https://github.com/nushell/nushell/assets/26268125/e40161ba-a8ec-407a-932d-5ece6f4dc616)
2024-06-06 08:46:43 -05:00
Wind
5d163c1bcc
run_external: remove inner quotes once nushell gets = sign (#13073)
# Description
Fixes: #13066

nushell should remove argument values' inner quote once it gets `=`.
Whatever it's a flag or not, and it also replace from `\"` to `"` before
passing it to external commands.

# User-Facing Changes
Given the shell script:
```shell
# test.sh
echo $@
```
## Before
```
>  sh test.sh -ldflags="-s -w" github.com
-ldflags="-s -w" github.com
> sh test.sh exp='-s -w' github.com
exp='-s -w' github.com
```
## After
```
>  sh test.sh -ldflags="-s -w" github.com
-ldflags=-s -w github.com
> sh test.sh exp='-s -w' github.com
exp=-s -w github.com
```

# Tests + Formatting
Added some tests
2024-06-06 11:03:34 +08:00
Reilly Wood
75d5807dcd
Fix explore panic on empty lists (#13074)
This fixes up a panic I accidentally introduced when refactoring the
cursor code in `explore`: https://github.com/nushell/nushell/pull/12979

Under certain circumstances (running `:nu []`, opening `:try` with the
hidden `try.reactive` setting enabled), `explore` would panic when
handling an empty list. To fix this for now I've removed the validation
I added to the Cursor constructor in that PR.
2024-06-05 19:49:32 -07:00
Jack Wright
a6b1d1f6d9
Upgrade to polars 0.40 (#13069)
Upgrading to polars 0.40
2024-06-06 07:26:47 +08:00
Embers-of-the-Fire
96493b26d9
Make string related commands parse-time evaluatable (#13032)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Related meta-issue: #10239.

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR will modify some `str`-related commands so that they can be
evaluated at parse time.

See the following list for those implemented by this pr.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Available now:
- `str` subcommands
  - `trim`
  - `contains`
  - `distance`
  - `ends-with`
  - `expand`
  - `index-of`
  - `join`
  - `replace`
  - `reverse`
  - `starts-with`
  - `stats`
  - `substring`
  - `capitalize`
  - `downcase`
  - `upcase`
- `split` subcommands
  - `chars`
  - `column`
  - `list`
  - `row`
  - `words`
- `format` subcommands
  - `date`
  - `duration`
  - `filesize`
- string related commands
  - `parse`
  - `detect columns`
  - `encode` & `decode`

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Unresolved questions:
- [ ] Is there any routine of testing const expressions? I haven't found
any yet.
- [ ] Is const expressions required to behave just like there non-const
version, like what rust promises?

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

Unresolved questions:
- [ ] Do const commands need special marks in the docs?
2024-06-05 22:21:52 +03:00
NotTheDr01ds
36ad7f15c4
cd/def --env examples (#13068)
# Description

Per a Discord question
(https://discord.com/channels/601130461678272522/1244293194603167845/1247794228696711198),
this adds examples to the `help` for both:

* `cd`
* `def`

to demonstrate that `def --env` is required when changing directories in
a custom command.

Since the existing examples for `def` were a bit more complex (and had
output) but the `cd` ones were more simplified, I did use slightly
different examples in each. Either or both could be tweaked if desired.

# User-Facing Changes

Command `help` examples

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-06-05 21:35:31 +03:00
Michael Angerman
78fdb2f4d1
reduce log tracing in nu-cli (#13067)
This one trace message creates thousands of lines of trace messages that
is probably
better suited to using on an *as needed* basis rather than everyone
having to wade
through it...

For now, I just commented it out but eventually this line of code should
be removed
and used simply for the time when someone needs to see it...
2024-06-05 08:54:38 -07:00
Reilly Wood
a9c2349ada
Refactor explore cursor code (#12979)
`explore` has 3 cursor-related structs that are extensively used to
track the currently shown "window" of the data being shown. I was
finding the cursor code quite difficult to follow, so this PR:
- rewrites the base `Cursor` struct from scratch, with some tests
- makes big changes to `WindowCursor`
- renames `XYCursor` to `WindowCursor2D`
- makes some of the cursor functions fallible as a start towards better
error handling
- changes lots of function names to things that I find more intuitive
- adds comments, including ASCII diagrams to explain how the cursors
work

More work could be done (I'd like to review/change more function names
in `WindowCursor` and `WindowCursor2D` and add more tests), but this is
the limit of what I can get done in a weekend. I think this part of the
code is in a better place now.

# Testing performed

I did a lot of manual testing in the record view and binary viewer,
moving around with arrow keys / page up+down / home+end.

This can definitely wait until after the release freeze, this area has
very few automated tests and it'd be good to let the changes bake a bit.
2024-06-04 19:50:11 -07:00
Jakub Žádník
e4104d0792
Span ID Refactor - Step 1 (#12960)
# Description
First part of SpanID refactoring series. This PR adds a `SpanId` type
and a corresponding `span_id` field to `Expression`. Parser creating
expressions will now add them to an array in `StateWorkingSet`,
generates a corresponding ID and saves the ID to the Expression. The IDs
are not used anywhere yet.

For the rough overall plan, see
https://github.com/nushell/nushell/issues/12963.

# User-Facing Changes
Hopefully none. This is only a refactor of Nushell's internals that
shouldn't have visible side effects.

# Tests + Formatting

# After Submitting
2024-06-05 09:57:14 +08:00
Jack Wright
b10325dff1
Allow int values to be converted into floats. (#13025)
Addresses the bug found by @maxim-uvarov when trying to coerce an int
Value to a polars float:

<img width="863" alt="image"
src="https://github.com/nushell/nushell/assets/56345/4d858812-a7b3-4296-98f4-dce0c544b4c6">

Conversion now works correctly:

<img width="891" alt="Screenshot 2024-05-31 at 14 28 51"
src="https://github.com/nushell/nushell/assets/56345/78d9f711-7ad5-4503-abc6-7aba64a2e675">
2024-06-04 18:51:11 -07:00
Antoine Büsch
d4fa014534
Make query xml return nodes in document order (#13047)
# Description

`query xml` used to return results from an XPath query in a random,
non-deterministic order. With this change, results get returned in the
order they appear in the document.

# User-Facing Changes
`query xml` will now return results in a non-random order.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
2024-06-05 09:47:36 +08:00
Devyn Cairns
28e33587d9
msgpackz: increase default compression level (#13035)
# Description
Increase default compression level for brotli on msgpackz to 3. This has
the best compression time generally. Level 0 and 1 give weird results
and sometimes cause extremely inflated outputs rather than being
compressed. So far this hasn't really been a problem for the plugin
registry file, but has been for other data.

The `$example` is the web-app example from https://json.org/example.html

Benchmarked with:

```nushell
seq 0 11 | each { |level|
  let compressed = ($example | to msgpackz --quality $level)
  let time = (timeit { $example | to msgpackz --quality $level })
  {
    level: $level
    time: $time
    length: ($compressed | bytes length)
    ratio: (($uncompressed_length | into float) / ($compressed | bytes length))
  }
}
```

```
╭────┬───────┬─────────────────┬────────┬───────╮
│  # │ level │      time       │ length │ ratio │
├────┼───────┼─────────────────┼────────┼───────┤
│  0 │     0 │ 4ms 611µs 875ns │   3333 │  0.72 │
│  1 │     1 │ 1ms 334µs 500ns │   3333 │  0.72 │
│  2 │     2 │     190µs 333ns │   1185 │  2.02 │
│  3 │     3 │      184µs 42ns │   1128 │  2.12 │
│  4 │     4 │      245µs 83ns │   1098 │  2.18 │
│  5 │     5 │     265µs 584ns │   1040 │  2.30 │
│  6 │     6 │     270µs 792ns │   1040 │  2.30 │
│  7 │     7 │     444µs 708ns │   1040 │  2.30 │
│  8 │     8 │       1ms 801µs │   1040 │  2.30 │
│  9 │     9 │     843µs 875ns │   1037 │  2.31 │
│ 10 │    10 │ 4ms 128µs 375ns │    984 │  2.43 │
│ 11 │    11 │ 6ms 352µs 834ns │    986 │  2.43 │
╰────┴───────┴─────────────────┴────────┴───────╯
```

cc @maxim-uvarov
2024-06-04 17:19:10 -07:00
Antoine Stevan
ff5cb6f1ff
complete the type of --error-label in std assert commands (#12998)
i was looking at the website documentation of `std assert` and i noticed
one thing
- the `--error-label` argument of `assert` and `assert not` was just a
`record` -> now it's that complete type `record<text: string, span:
record<start: int, end: int>>`
2024-06-05 08:02:17 +08:00
Antoine Büsch
65911c125c
Try to preserve the ordering of elements in from toml (#13045)
# Description

Enable the `preserve_order` feature of the `toml` crate to preserve the
ordering of elements when converting from/to toml.

Additionally, use `to_string_pretty()` instead of `to_string()` in `to
toml`. This displays arrays on multiple lines instead of one big single
line. I'm not sure if this one is a good idea or not... Happy to remove
this from this PR if it's not.

# User-Facing Changes
The order of elements will be different when using `from toml`. The
formatting of arrays will also be different when using `to toml`. For
example:

- before
```
❯ "foo=1\nbar=2\ndoo=3" | from toml
╭─────┬───╮
│ bar │ 2 │
│ doo │ 3 │
│ foo │ 1 │
╰─────┴───╯
❯ {a: [a b c d]} | to toml
a = ["a", "b", "c", "d"]
```
- after
```
❯ "foo=1\nbar=2\ndoo=3" | from toml
╭─────┬───╮
│ foo │ 1 │
│ bar │ 2 │
│ doo │ 3 │
╰─────┴───╯
❯ {a: [a b c d]} | to toml
a = [
    "a",
    "b",
    "c",
    "d",
]
```

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
-  `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-05 08:00:39 +08:00
Sang-Heon Jeon
3f0db11ae5
support plus sign for "into filesize" (#12974)
# Description
Fixes https://github.com/nushell/nushell/issues/12968. After apply this
patch, we can use explict plus sign character included string with `into
filesize` cmd.

# User-Facing Changes
AS-IS (before fixing)
```
$ "+8 KiB" | into filesize                                                                                                         
Error: nu:🐚:cant_convert

  × Can't convert to int.
   ╭─[entry #31:1:1]
 1 │ "+8 KiB" | into filesize
   · ────┬───
   ·     ╰── can't convert string to int
   ╰────
```

TO-BE (after fixing)
```
$ "+8KiB" | into filesize                                                                                       
8.0 KiB
```

# Tests + Formatting
Added a test 

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-05 07:43:50 +08:00
Reilly Wood
7746e84199
Make LS_COLORS functionality faster in explore, especially on Windows (#12984)
Closes #12980. More context there, but basically `explore` was getting
file metadata for every row every time the record view was rendered. The
quick fix for now is to do the `LS_COLORS` colouring with a `&str`
instead of a path and file metadata.
2024-06-05 07:43:12 +08:00
Jack Wright
a84fdb1d37
Fixed a couple of incorrect errors messages (#13043)
Fixed a couple of error message that incorrectly reported as parquet
errors instead of CSV errors.
2024-06-05 07:40:02 +08:00
Wind
ad5a6cdc00
bump version to 0.94.3 (#13055) 2024-06-05 06:52:40 +08:00
Devyn Cairns
be8c1dc006
Fix run_external::expand_glob() to return paths that are PWD-relative but reflect the original intent (#13028)
# Description

Fix #13021

This changes the `expand_glob()` function to use
`nu_engine::glob_from()` so that absolute paths are actually preserved,
rather than being made relative to the provided parent. This preserves
the intent of whoever wrote the original path/glob, and also makes it so
that tilde always produces absolute paths.

I also made `expand_glob()` handle Ctrl-C so that it can be interrupted.

cc @YizhePKU

# Tests + Formatting
No additional tests here... but that might be a good idea.
2024-06-03 10:38:55 +03:00
Devyn Cairns
b50903cf58
Fix external command name parsing with backslashes, and add tests (#13027)
# Description

Fixes #13016 and adds tests for many variations of external call
parsing.

I just realized @kubouch took a crack at this too (#13022) so really
whichever is better, but I think the
tests are a good addition.
2024-06-03 10:28:45 +03:00
Devyn Cairns
6635b74d9d
Bump version to 0.94.2 (#13014)
Version bump after 0.94.1 patch release.
2024-06-03 10:28:35 +03:00
Ian Manske
f3cf693ec7
Disallow more characters in arguments for internal cmd commands (#13009)
# Description
Makes `run-external` error if arguments to `cmd.exe` internal commands
contain newlines or a percent sign. This is because the percent sign can
expand environment variables, potentially? allowing command injection.
Newlines I think will truncate the rest of the arguments and should
probably be disallowed to be safe.

# After Submitting
- If the user calls `cmd.exe` directly, then this bypasses our
handling/checking for internal `cmd` commands. Instead, we use the
handling from the Rust std lib which, in this case, does not do special
handling and is potentially unsafe. Then again, it could be the user's
specific intention to run `cmd` with whatever trusted input. The problem
is that since we use the std lib handling, it assumes the exe uses the C
runtime escaping rules and will perform some unwanted escaping. E.g., it
will add backslashes to the quotes in `cmd echo /c '""'`.
- If `cmd` is called indirectly via a `.bat` or `.cmd` file, then we use
the Rust std lib which has separate handling for bat files that should
be safe, but will reject some inputs.
- ~~I'm not sure how we handle `PATHEXT`, that can also cause a file
without an extension to be run as a bat file. If so, I don't know where
the handling, if any, is done for that.~~ It looks like we use the
`which` crate to do the lookup using `PATHEXT`. Then, we pass the exe
path from that to the Rust std lib `Command`, which should be safe
(except for the first `cmd.exe` note).

So, in the future we need to unify and/or fix these different
implementations, including our own special handling for internal `cmd`
commands that this PR tries to fix.
2024-05-30 19:24:48 +00:00
Ian Manske
31f3d2f664
Restore path type behavior (#13006)
# Description
Restores `path type` to return an empty string on error like it did pre
0.94.0.
2024-05-30 13:42:22 +00:00
Wind
40772fea15
fix do closure with both required, options, and rest args (#13002)
# Description
Fixes: #12985

`val_iter` has already handle required positional and optional
positional arguments, it not skip them again while handling rest
arguments.

# User-Facing Changes
Makes `do {|a, ...b| echo $a ...$b} 1 2 3 4` output the following again:
```nushell
╭───┬───╮
│ 0 │ 1 │
│ 1 │ 2 │
│ 2 │ 3 │
│ 3 │ 4 │
╰───┴───╯
```

# Tests + Formatting
Added some test cases
2024-05-30 08:29:46 -05:00
Devyn Cairns
0e1553026e
Restore tilde expansion on external command names (#13001)
# Description

Fix a regression introduced by #12921, where tilde expansion was no
longer done on the external command name, breaking things like

```nushell
> ~/.cargo/bin/exa
```

This properly handles quoted strings, so they don't expand:

```nushell
> ^"~/.cargo/bin/exa"
Error: nu:🐚:external_command

  × External command failed
   ╭─[entry #1:1:2]
 1 │ ^"~/.cargo/bin/exa"
   ·  ─────────┬────────
   ·           ╰── Command `~/.cargo/bin/exa` not found
   ╰────
  help: `~/.cargo/bin/exa` is neither a Nushell built-in or a known external command

```

This required a change to the parser, so the command name is also parsed
in the same way the arguments are - i.e. the quotes on the outside
remain in the expression. Hopefully that doesn't break anything else. 🤞

Fixes #13000. Should include in patch release 0.94.1

cc @YizhePKU

# User-Facing Changes
- Tilde expansion now works again for external commands
- The `command` of `run-external` will now have its quotes removed like
the other arguments if it is a literal string
- The parser is changed to include quotes in the command expression of
`ExternalCall` if they were present

# Tests + Formatting
I would like to add a regression test for this, but it's complicated
because we need a well-known binary within the home directory, which
just isn't a thing. We could drop one there, but that's kind of a bad
behavior for a test to do. I also considered changing the home directory
for the test, but that's so platform-specific - potentially could get it
working on specific platforms though. Changing `HOME` env on Linux
definitely works as far as tilde expansion works.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-29 18:48:29 -07:00
Darren Schroeder
6a2996251c
fixes a bug in OSC9;9 execution (#12994)
# Description

This fixes a bug in the `OSC 9;9` functionality where the path wasn't
being constructed properly and therefore wasn't getting set right for
things like "Duplicate Tab" in Windows Terminal. Thanks to @Araxeus for
finding it.

Related to https://github.com/nushell/nushell/issues/10166

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-29 18:06:47 -05:00
Devyn Cairns
f3991f2080
Bump version to 0.94.1 (#12988)
Merge this PR before merging any other PRs.
2024-05-28 22:41:23 +00:00
Jakub Žádník
61182deb96
Bump version to 0.94.0 (#12987) 2024-05-28 12:04:09 -07:00
Ian Manske
6012af2412
Fix panic when redirecting nothing (#12970)
# Description
Fixes #12969 where the parser can panic if a redirection is applied to
nothing / an empty command.

# Tests + Formatting
Added a test.
2024-05-27 10:03:06 +08:00
YizhePKU
f74dd33ba9
Fix touch --reference using PWD from the environment (#12976)
This PR fixes `touch --reference path` so that it resolves `path` using
PWD from the engine state.
2024-05-26 20:24:00 +03:00
YizhePKU
a1fc41db22
Fix path type using PWD from the environment (#12975)
This PR fixes the `path type` command so that it resolves relative paths
using PWD from the engine state.

As a bonus, it also fixes the issue of `path type` returning an empty
string instead of an error when it fails.
2024-05-26 20:23:52 +03:00
YizhePKU
f38f88d42c
Fixes . expanded incorrectly as external argument (#12950)
This PR fixes a bug where `.` is expanded into an empty string when used
as an argument to external commands. Fixes
https://github.com/nushell/nushell/issues/12948.

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-26 07:06:17 +08:00
Darren Schroeder
0c5a67f4e5
make polars plugin use mimalloc (#12967)
# Description
@maxim-uvarov did a ton of research and work with the dply-rs author and
ritchie from polars and found out that the allocator matters on macos
and it seems to be what was messing up the performance of polars plugin.
ritchie suggested to use jemalloc but i switched it to mimalloc to match
nushell and it seems to run better.

## Before (default allocator)
note - using 1..10 vs 1..100 since it takes so long. also notice how
high the `max` timings are compared to mimalloc below.
```nushell
❯ 1..10 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬─────────────────────────╮
│ mean   │ 4sec 999ms 605µs 995ns  │
│ min    │ 983ms 627µs 42ns        │
│ max    │ 13sec 398ms 135µs 791ns │
│ stddev │ 3sec 476ms 479µs 939ns  │
╰────────┴─────────────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 10
╭───────┬────────────────────────╮
│ mean  │ 6sec 220ms 783µs 983ns │
│ min   │ 1sec 184ms 997µs 708ns │
│ max   │ 18sec 882ms 81µs 708ns │
│ std   │ 5sec 350ms 375µs 697ns │
│ times │ [list 10 items]        │
╰───────┴────────────────────────╯
```

## After (using mimalloc)
```nushell
❯ 1..100 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬───────────────────╮
│ mean   │ 103ms 728µs 902ns │
│ min    │ 97ms 107µs 42ns   │
│ max    │ 149ms 430µs 84ns  │
│ stddev │ 5ms 690µs 664ns   │
╰────────┴───────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬───────────────────╮
│ mean  │ 103ms 620µs 195ns │
│ min   │ 97ms 541µs 166ns  │
│ max   │ 130ms 262µs 166ns │
│ std   │ 4ms 948µs 654ns   │
│ times │ [list 100 items]  │
╰───────┴───────────────────╯
```

## After (using jemalloc - just for comparison)
```nushell
❯ 1..100 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}

╭────────┬───────────────────╮
│ mean   │ 113ms 939µs 777ns │
│ min    │ 108ms 337µs 333ns │
│ max    │ 166ms 467µs 458ns │
│ stddev │ 6ms 175µs 618ns   │
╰────────┴───────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬───────────────────╮
│ mean  │ 114ms 363µs 530ns │
│ min   │ 108ms 804µs 833ns │
│ max   │ 143ms 521µs 459ns │
│ std   │ 5ms 88µs 56ns     │
│ times │ [list 100 items]  │
╰───────┴───────────────────╯
```

## After (using parquet + mimalloc)
```nushell
❯ 1..100 | each {timeit {polars open data.parquet | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬──────────────────╮
│ mean   │ 34ms 255µs 492ns │
│ min    │ 31ms 787µs 250ns │
│ max    │ 76ms 408µs 416ns │
│ stddev │ 4ms 472µs 916ns  │
╰────────┴──────────────────╯
❯ use std bench
❯ bench { polars open data.parquet | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬──────────────────╮
│ mean  │ 34ms 897µs 562ns │
│ min   │ 31ms 518µs 542ns │
│ max   │ 65ms 943µs 625ns │
│ std   │ 3ms 450µs 741ns  │
│ times │ [list 100 items] │
╰───────┴──────────────────╯
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-25 09:10:01 -05:00
Ian Manske
95977faf2d
Do not propagate glob creation error for external args (#12955)
# Description
Instead of returning an error, this PR changes `expand_glob` in
`run_external.rs` to return the original string arg if glob creation
failed. This makes it so that, e.g.,
```nushell
^echo `[`
^echo `***`
```
no longer fail with a shell error. (This follows from #12921.)
2024-05-25 08:59:36 +08:00
Ian Manske
c5d716951f
Allow byte streams with unknown type to be compatiable with binary (#12959)
# Description
Currently, this pipeline doesn't work `open --raw file | take 100`,
since the type of the byte stream is `Unknown`, but `take` expects
`Binary` streams. This PR changes commands that expect
`ByteStreamType::Binary` to also work with `ByteStreamType::Unknown`.
This was done by adding two new methods to `ByteStreamType`:
`is_binary_coercible` and `is_string_coercible`. These return true if
the type is `Unknown` or matches the type in the method name.
2024-05-24 17:54:38 -07:00
Devyn Cairns
b06f31d3c6
Make from json --objects streaming (#12949)
# Description

Makes the `from json --objects` command produce a stream, and read
lazily from an input stream to produce its output.

Also added a helper, `PipelineData::get_type()`, to make it easier to
construct a wrong type error message when matching on `PipelineData`. I
expect checking `PipelineData` for either a string value or an `Unknown`
or `String` typed `ByteStream` will be very, very common. I would have
liked to have a helper that just returns a readable stream from either,
but that would either be a bespoke enum or a `Box<dyn BufRead>`, which
feels like it wouldn't be so great for performance. So instead, taking
the approach I did here is probably better - having a function that
accepts the `impl BufRead` and matching to use it.

# User-Facing Changes

- `from json --objects` no longer collects its input, and can be used
for large datasets or streams that produce values over time.

# Tests + Formatting
All passing.

# After Submitting
- [ ] release notes

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-24 23:37:50 +00:00
Ian Manske
84b7a99adf
Revert "Polars lazy refactor (#12669)" (#12962)
This reverts commit 68adc4657f.

# Description

Reverts the lazyframe refactor (#12669) for the next release, since
there are still a few lingering issues. This temporarily solves #12863
and #12828. After the release, the lazyframes can be added back and
cleaned up.
2024-05-24 18:09:26 -05:00
Darren Schroeder
7d11c28eea
Revert "Remove std::env::set_current_dir() call from EngineState::merge_env()" (#12954)
Reverts nushell/nushell#12922
2024-05-24 11:09:59 -05:00
Ian Manske
bf07806b1b
Use cwd in grid (#12947)
# Description
Fixes #12946. The `grid` command does not use the cwd when trying to get
the icon or color for a file/path.
2024-05-23 20:38:47 +00:00
Reilly Wood
0b5a4c0d95
explore refactoring+clarification (#12940)
Another very boring PR cleaning up and documenting some of `explore`'s
innards. Mostly renaming things that I found confusing or vague when
reading through the code, also adding some comments.
2024-05-23 08:51:39 -05:00
Wind
f53aa6fcbf
fix std help (#12943)
# Description
Fixes: #12941

~~The issue is cause by some columns(is_builtin, is_plugin, is_custom,
is_keyword) are removed in #10023~~
Edit: I'm wrong

# Tests + Formatting
Added one test for `std help`
2024-05-23 08:51:02 -05:00
Ian Manske
2612a167e3
Remove list support in with-env (#12939)
# Description
Following from #12523, this PR removes support for lists of environments
variables in the `with-env` command. Rather, only records will be
supported now.

# After Submitting
Update examples using the list form in the docs and book.
2024-05-23 13:53:55 +08:00
Reilly Wood
c7097ca937
explore cleanup: remove+move binary viewer config (#12920)
Small change, removing 4 more configuration options from `explore`'s
binary viewer:

1. `show_index`
2. `show_data`
3. `show_ascii`
4. `show_split`

These controlled whether the 3 columns in the binary viewer (index, hex
data, ASCII) and the pipe separator (`|`) in between them are shown. I
don't think we need this level of configurability until the `explore`
command is more mature, and maybe even not then; we can just show them
all.

I think it's very unlikely that anyone is using these configuration
points.

Also, the row offset (e.g. how many rows we have scrolled down) was
being stored in config/settings when it's arguably not config; more like
internal state of the binary viewer. I moved it to a more appropriate
location and renamed it.
2024-05-22 20:06:14 -07:00
Wind
58cf0c56f8
add some completion tests (#12908)
# Description
```nushell
❯ ls
╭───┬───────┬──────┬──────┬──────────╮
│ # │ name  │ type │ size │ modified │
├───┼───────┼──────┼──────┼──────────┤
│ 0 │ a.txt │ file │  0 B │ now      │
╰───┴───────┴──────┴──────┴──────────╯

❯ ls a.
NO RECORDS FOUND
```

There is a completion issue on previous version, I think @amtoine have
reproduced it before. But currently I can't reproduce it on latest main.
To avoid such regression, I added some tests for completion.

---------

Co-authored-by: Antoine Stevan <44101798+amtoine@users.noreply.github.com>
2024-05-23 10:47:06 +08:00
YizhePKU
6c649809d3
Rewrite run_external.rs (#12921)
This PR is a complete rewrite of `run_external.rs`. The main goal of the
rewrite is improving readability, but it also fixes some bugs related to
argument handling and the PATH variable (fixes
https://github.com/nushell/nushell/issues/6011).

I'll discuss some technical details to make reviewing easier.

## Argument handling

Quoting arguments for external commands is hard. Like, *really* hard.
We've had more than a dozen issues and PRs dedicated to quoting
arguments (see Appendix) but the current implementation is still buggy.

Here's a demonstration of the buggy behavior:

```nu
let foo = "'bar'"
^touch $foo            # This creates a file named `bar`, but it should be `'bar'`
^touch ...[ "'bar'" ]  # Same
```

I'll describe how this PR deals with argument handling.

First, we'll introduce the concept of **bare strings**. Bare strings are
**string literals** that are either **unquoted** or **quoted by
backticks** [^1]. Strings within a list literal are NOT considered bare
strings, even if they are unquoted or quoted by backticks.

When a bare string is used as an argument to external process, we need
to perform tilde-expansion, glob-expansion, and inner-quotes-removal, in
that order. "Inner-quotes-removal" means transforming from
`--option="value"` into `--option=value`.

## `.bat` files and CMD built-ins

On Windows, `.bat` files and `.cmd` files are considered executable, but
they need `CMD.exe` as the interpreter. The Rust standard library
supports running `.bat` files directly and will spawn `CMD.exe` under
the hood (see
[documentation](https://doc.rust-lang.org/std/process/index.html#windows-argument-splitting)).
However, other extensions are not supported [^2].

Nushell also supports a selected number of CMD built-ins. The problem
with CMD is that it uses a different set of quoting rules. Correctly
quoting for CMD requires using
[Command::raw_arg()](https://doc.rust-lang.org/std/os/windows/process/trait.CommandExt.html#tymethod.raw_arg)
and manually quoting CMD special characters, on top of quoting from the
Nushell side. ~~I decided that this is too complex and chose to reject
special characters in CMD built-ins instead [^3]. Hopefully this will
not affact real-world use cases.~~ I've implemented escaping that works
reasonably well.

## `which-support` feature

The `which` crate is now a hard dependency of `nu-command`, making the
`which-support` feature essentially useless. The `which` crate is
already a hard dependency of `nu-cli`, and we should consider removing
the `which-support` feature entirely.

## Appendix

Here's a list of quoting-related issues and PRs in rough chronological
order.

* https://github.com/nushell/nushell/issues/4609
* https://github.com/nushell/nushell/issues/4631
* https://github.com/nushell/nushell/issues/4601
  * https://github.com/nushell/nushell/pull/5846
* https://github.com/nushell/nushell/issues/5978
  * https://github.com/nushell/nushell/pull/6014
* https://github.com/nushell/nushell/issues/6154
  * https://github.com/nushell/nushell/pull/6161
* https://github.com/nushell/nushell/issues/6399
  * https://github.com/nushell/nushell/pull/6420
  * https://github.com/nushell/nushell/pull/6426
* https://github.com/nushell/nushell/issues/6465
* https://github.com/nushell/nushell/issues/6559
  * https://github.com/nushell/nushell/pull/6560

[^1]: The idea that backtick-quoted strings act like bare strings was
introduced by Kubouch and briefly mentioned in [the language
reference](https://www.nushell.sh/lang-guide/chapters/strings_and_text.html#backtick-quotes).

[^2]: The documentation also said "running .bat scripts in this way may
be removed in the future and so should not be relied upon", which is
another reason to move away from this. But again, quoting for CMD is
hard.

[^3]: If anyone wants to try, the best resource I found on the topic is
[this](https://daviddeley.com/autohotkey/parameters/parameters.htm).
2024-05-23 02:05:27 +00:00
Jakub Žádník
64afb52ffa
Fix leftover wrong column name (#12937)
# Description

Small fixup for https://github.com/nushell/nushell/pull/12930
2024-05-22 21:24:22 +00:00
Wind
ac4125f8ed
fix range semantic in detect_columns, str substring, str index-of (#12894)
# Description
Fixes: https://github.com/nushell/nushell/issues/7761

It's still unsure if we want to change the `range semantic` itself, but
it's good to keep range semantic consistent between nushell commands.

# User-Facing Changes
### Before
```nushell
❯ "abc" | str substring 1..=2
b
```
### After
```nushell
❯ "abc" | str substring 1..=2
bc
```

# Tests + Formatting
Adjust tests to fit new behavior
2024-05-22 20:00:58 +03:00
YizhePKU
7ede90cba5
Remove std::env::set_current_dir() call from EngineState::merge_env() (#12922)
As discussed in https://github.com/nushell/nushell/pull/12749, we no
longer need to call `std::env::set_current_dir()` to sync `$env.PWD`
with the actual working directory. This PR removes the call from
`EngineState::merge_env()`.
2024-05-22 19:58:27 +03:00
Jakub Žádník
75689ec98a
Small improvements to debug profile (#12930)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

1. With the `-l` flag, `debug profile` now collects files and line
numbers of profiled pipeline elements

![profiler_lines](https://github.com/nushell/nushell/assets/25571562/b400a956-d958-4aff-aa4c-7e65da3f78fa)

2. Error from the profiled closure will be reported instead of silently
ignored.

![profiler_lines_error](https://github.com/nushell/nushell/assets/25571562/54f7ad7a-06a3-4d56-92c2-c3466917bee8)


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

New `--lines(-l)` flag to `debug profile`. The command will also fail if
the profiled closure fails, so technically it is a breaking change.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-22 19:56:51 +03:00
Devyn Cairns
7de513a4e0
Implement streaming I/O for CSV and TSV commands (#12918)
# Description

Implements streaming for:

- `from csv`
- `from tsv`
- `to csv`
- `to tsv`

via the new string-typed ByteStream support.

# User-Facing Changes
Commands above. Also:

- `to csv` and `to tsv` now have `--columns <List(String)>`, to provide
the exact columns desired in the output. This is required for them to
have streaming output, because otherwise collecting the entire list is
necessary to determine the output columns. If we introduce
`TableStream`, this may become less necessary.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] release notes

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-22 16:55:24 +00:00
Devyn Cairns
758c5d447a
Add support for the ps command on FreeBSD, NetBSD, and OpenBSD (#12892)
# Description

I feel like it's a little sad that BSDs get to enjoy almost everything
other than the `ps` command, and there are some tests that rely on this
command, so I figured it would be fun to patch that and make it work.

The different BSDs have diverged from each other somewhat, but generally
have a similar enough API for reading process information via
`sysctl()`, with some slightly different args.

This supports FreeBSD with the `freebsd` module, and NetBSD and OpenBSD
with the `netbsd` module. OpenBSD is a fork of NetBSD and the interface
has some minor differences but many things are the same.

I had wanted to try to support DragonFlyBSD too, but their Rust version
in the latest release is only 1.72.0, which is too old for me to want to
try to compile rustc up to 1.77.2... but I will revisit this whenever
they do update it. Dragonfly is a fork of FreeBSD, so it's likely to be
more or less the same - I just don't want to enable it without testing
it.

Fixes #6862 (partially, we probably won't be adding `zfs list`)

# User-Facing Changes
`ps` added for FreeBSD, NetBSD, and OpenBSD.

# Tests + Formatting
The CI doesn't run tests for BSDs, so I'm not entirely sure if
everything was already passing before. (Frankly, it's unlikely.) But
nothing appears to be broken.

# After Submitting
- [ ] release notes?
- [ ] DragonflyBSD, whenever they do update Rust to something close
enough for me to try it
2024-05-22 08:13:45 -07:00
dependabot[bot]
d7e75c0b70
Bump shadow-rs from 0.27.1 to 0.28.0 (#12932)
Bumps [shadow-rs](https://github.com/baoyachi/shadow-rs) from 0.27.1 to
0.28.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/baoyachi/shadow-rs/releases">shadow-rs's
releases</a>.</em></p>
<blockquote>
<h2>fix cargo clippy</h2>
<p><a
href="https://redirect.github.com/baoyachi/shadow-rs/issues/160">#160</a></p>
<p>Thx <a href="https://github.com/qartik"><code>@​qartik</code></a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ba9f8b0c2b"><code>ba9f8b0</code></a>
Update Cargo.toml</li>
<li><a
href="d1b724c1e7"><code>d1b724c</code></a>
Merge pull request <a
href="https://redirect.github.com/baoyachi/shadow-rs/issues/160">#160</a>
from qartik/patch-1</li>
<li><a
href="505108d5d6"><code>505108d</code></a>
Allow missing_docs for deprecated CLAP_VERSION constant</li>
<li>See full diff in <a
href="https://github.com/baoyachi/shadow-rs/compare/v0.27.1...v0.28.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=shadow-rs&package-manager=cargo&previous-version=0.27.1&new-version=0.28.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-22 15:59:33 +08:00
NotTheDr01ds
f83439fdda
Add completer for std help (#12929)
# Description

While each of the `help <subcommands>` in `std` had completers, there
wasn't one for the main `help` command.

This adds all internals and custom commands (as with `help commands`) as
possible completions.

# User-Facing Changes

`help ` + <kbd>Tab</kbd> will now suggest completions for both the `help
<subcommands>` as well as all internal and custom commands.

# Tests + Formatting

Note: Cannot add tests for completion functions since they are
module-internal and not visible to test cases, that I can see.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-21 10:31:14 -05:00
Stefan Holderbach
db37bead64
Remove unused dependencies (#12917)
- **Remove unused `pathdiff` dep in `nu-cli`**
- **Remove unused `serde_json` dep on `nu-protocol`**
- Unnecessary after moving the plugin file to msgpack (still a
dev-dependency)
2024-05-21 01:09:28 +00:00
Reilly Wood
6e050f5634
explore: consolidate padding config, handle ByteStream, tweak naming+comments (#12915)
Some minor changes to `explore`, continuing on my mission to simplify
the command in preparation for a larger UX overhaul:

1. Consolidate padding configuration. I don't think we need separate
config points for the (optional) index column and regular data columns
in the normal pager, they can share padding configuration. Likewise, in
the binary viewer all 3 columns (index, data, ASCII) had their
left+right padding configured independently.
2. Update `explore` so we use the binary viewer for the new `ByteStream`
type. `cat foo.txt | into binary | explore` was not using the binary
viewer after the `ByteStream` changes.
3. Tweak the naming of a few helper functions, add a comment

I've put the changes in separate commits to make them easier to review.

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-05-20 22:03:21 +02:00
Ian Manske
905e3d0715
Remove dataframes crate and feature (#12889)
# Description
Removes the old `nu-cmd-dataframe` crate in favor of the polars plugin.
As such, this PR also removes the `dataframe` feature, related CI, and
full releases of nushell.
2024-05-20 17:22:08 +00:00
Darren Schroeder
4f69ba172e
add math min and math max to bench command (#12913)
# Description

This PR adds min and max to the bench command.
```nushell
❯ use std bench
❯ bench { dply -c 'parquet("./data.parquet") | group_by(year) | summarize(count = n(), sum = sum(geo_count)) | show()' | complete | null } --rounds 100 --verbose
100 / 100
╭───────┬───────────────────╮
│ mean  │ 71ms 358µs 850ns  │
│ min   │ 66ms 457µs 583ns  │
│ max   │ 120ms 338µs 167ns │
│ std   │ 6ms 553µs 949ns   │
│ times │ [list 100 items]  │
╰───────┴───────────────────╯
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-20 10:08:03 -05:00
Ian Manske
c98960d053
Take owned Read and Write (#12909)
# Description
As @YizhePKU pointed out, the [Rust API
guidelines](https://rust-lang.github.io/api-guidelines/interoperability.html#generic-readerwriter-functions-take-r-read-and-w-write-by-value-c-rw-value)
recommend that generic functions take readers and writers by value and
not by reference. This PR changes `copy_with_interupt` and few other
places to take owned `Read` and `Write` instead of mutable references.
2024-05-20 15:10:36 +02:00
Devyn Cairns
c61075e20e
Add string/binary type color to ByteStream (#12897)
# Description

This PR allows byte streams to optionally be colored as being
specifically binary or string data, which guarantees that they'll be
converted to `Binary` or `String` appropriately on `into_value()`,
making them compatible with `Type` guarantees. This makes them
significantly more broadly usable for command input and output.

There is still an `Unknown` type for byte streams coming from external
commands, which uses the same behavior as we previously did where it's a
string if it's UTF-8.

A small number of commands were updated to take advantage of this, just
to prove the point. I will be adding more after this merges.

# User-Facing Changes
- New types in `describe`: `string (stream)`, `binary (stream)`
- These commands now return a stream if their input was a stream:
  - `into binary`
  - `into string`
  - `bytes collect`
  - `str join`
  - `first` (binary)
  - `last` (binary)
  - `take` (binary)
  - `skip` (binary)
- Streams that are explicitly binary colored will print as a streaming
hexdump
  - example:
    ```nushell
    1.. | each { into binary } | bytes collect
    ```

# Tests + Formatting
I've added some tests to cover it at a basic level, and it doesn't break
anything existing, but I do think more would be nice. Some of those will
come when I modify more commands to stream.

# After Submitting
There are a few things I'm not quite satisfied with:

- **String trimming behavior.** We automatically trim newlines from
streams from external commands, but I don't think we should do this with
internal commands. If I call a command that happens to turn my string
into a stream, I don't want the newline to suddenly disappear. I changed
this to specifically do it only on `Child` and `File`, but I don't know
if this is quite right, and maybe we should bring back the old flag for
`trim_end_newline`
- **Known binary always resulting in a hexdump.** It would be nice to
have a `print --raw`, so that we can put binary data on stdout
explicitly if we want to. This PR doesn't change how external commands
work though - they still dump straight to stdout.

Otherwise, here's the normal checklist:

- [ ] release notes
- [ ] docs update for plugin protocol changes (added `type` field)

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-20 00:35:32 +00:00
Ian Manske
baeba19b22
Make get_full_help take &dyn Command (#12903)
# Description
Changes `get_full_help` to take a `&dyn Command` instead of multiple
arguments (`&Signature`, `&Examples` `is_parser_keyword`). All of these
arguments can be gathered from a `Command`, so there is no need to pass
the pieces to `get_full_help`.

This PR also fixes an issue where the search terms are not shown if
`--help` is used on a command.
2024-05-19 19:56:33 +02:00
Ian Manske
474293bf1c
Clear environment for child Commands (#12901)
# Description
There is a bug when `hide-env` is used on environment variables that
were present at shell startup. Namely, child processes still inherit the
hidden environment variable. This PR fixes #12900, fixes #11495, and
fixes #7937.

# Tests + Formatting
Added a test.
2024-05-19 15:35:07 +00:00
Ian Manske
cc9f41e553
Use CommandType in more places (#12832)
# Description
Kind of a vague title, but this PR does two main things:
1. Rather than overriding functions like `Command::is_parser_keyword`,
this PR instead changes commands to override `Command::command_type`.
The `CommandType` returned by `Command::command_type` is then used to
automatically determine whether `Command::is_parser_keyword` and the
other `is_{type}` functions should return true. These changes allow us
to remove the `CommandType::Other` case and should also guarantee than
only one of the `is_{type}` functions on `Command` will return true.
2. Uses the new, reworked `Command::command_type` function in the `scope
commands` and `which` commands.


# User-Facing Changes
- Breaking change for `scope commands`: multiple columns (`is_builtin`,
`is_keyword`, `is_plugin`, etc.) have been merged into the `type`
column.
- Breaking change: the `which` command can now report `plugin` or
`keyword` instead of `built-in` in the `type` column. It may also now
report `external` instead of `custom` in the `type` column for known
`extern`s.
2024-05-18 23:37:31 +00:00
Ian Manske
580c60bb82
Preserve metadata in more places (#12848)
# Description
This PR makes some commands and areas of code preserve pipeline
metadata. This is in an attempt to make the issue described in #12599
and #9456 less likely to occur. That is, reading and writing to the same
file in a pipeline will result in an empty file. Since we preserve
metadata in more places now, there will be a higher chance that we
successfully detect this error case and abort the pipeline.
2024-05-17 17:59:32 +00:00
Devyn Cairns
c10aa2cf09
collect: don't require a closure (#12788)
# Description

This changes the `collect` command so that it doesn't require a closure.
Still allowed, optionally.

Before:

```nushell
open foo.json | insert foo bar | collect { save -f foo.json }
```

After:

```nushell
open foo.json | insert foo bar | collect | save -f foo.json
```

The closure argument isn't really necessary, as collect values are also
supported as `PipelineData`.

# User-Facing Changes
- `collect` command changed

# Tests + Formatting
Example changed to reflect.

# After Submitting
- [ ] release notes
- [ ] we may want to deprecate the closure arg?
2024-05-17 18:46:03 +02:00
Devyn Cairns
e3db6ea04a
Exclude polars from ensure_plugins_built(), for performance reasons (#12896)
# Description

We have been building `nu_plugin_polars` unnecessarily during `cargo
test`, which is very slow. All of its tests are run within its own
crate, which happens during the plugins CI phase.

This should speed up the CI a bit.
2024-05-17 15:04:59 +00:00
Devyn Cairns
59f7c523fa
Fix the way the output of table is printed in print() (#12895)
# Description

Forgot that I fixed this already on my branch, but when printing without
a display output hook, the implicit call to `table` gets its output
mangled with newlines (since #12774). This happens when running `nu -c`
or a script file.

Here's that fix in one PR so it can be merged easily.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-17 07:18:18 -07:00
Wind
8adf3406e5
allow define it as a variable inside closure (#12888)
# Description
Fixes: #12690 

The issue is happened after
https://github.com/nushell/nushell/pull/12056 is merged. It will raise
error if user doesn't supply required parameter when run closure with
do.
And parser adds a `$it` parameter when parsing closure or block
expression.

I believe the previous behavior is because we allow such syntax on
previous version(0.44):
```nushell
let x = { print $it }
```
But it's no longer allowed after 0.60.  So I think they can be removed.

# User-Facing Changes
```nushell
let tmp = {
  let it = 42
  print $it
}

do -c $tmp
```
should be possible again.

# Tests + Formatting
Added 1 test
2024-05-17 00:03:13 +00:00
Ian Manske
6891267b53
Support ByteStreams in bytes starts-with and bytes ends-with (#12887)
# Description
Restores `bytes starts-with` so that it is able to work with byte
streams once again. For parity/consistency, this PR also adds byte
stream support to `bytes ends-with`.

# User-Facing Changes
- `bytes ends-with` now supports byte streams.

# Tests + Formatting
Re-enabled tests for `bytes starts-with` and added tests for `bytes
ends-with`.
2024-05-17 07:59:08 +08:00
Ian Manske
aec41f3df0
Add Span merging functions (#12511)
# Description
This PR adds a few functions to `Span` for merging spans together:
- `Span::append`: merges two spans that are known to be in order.
- `Span::concat`: returns a span that encompasses all the spans in a
slice. The spans must be in order.
- `Span::merge`: merges two spans (no order necessary).
- `Span::merge_many`: merges an iterator of spans into a single span (no
order necessary).

These are meant to replace the free-standing `nu_protocol::span`
function.

The spans in a `LiteCommand` (the `parts`) should always be in order
based on the lite parser and lexer. So, the parser code sees the most
usage of `Span::append` and `Span::concat` where the order is known. In
other code areas, `Span::merge` and `Span::merge_many` are used since
the order between spans is often not known.
2024-05-16 22:34:49 +00:00
Ian Manske
2a09dccc11
Bytestream touchup (#12886)
# Description
Adds some docs and a small fix to `Chunks`.
2024-05-16 21:15:20 +00:00
Ian Manske
6fd854ed9f
Replace ExternalStream with new ByteStream type (#12774)
# Description
This PR introduces a `ByteStream` type which is a `Read`-able stream of
bytes. Internally, it has an enum over three different byte stream
sources:
```rust
pub enum ByteStreamSource {
    Read(Box<dyn Read + Send + 'static>),
    File(File),
    Child(ChildProcess),
}
```

This is in comparison to the current `RawStream` type, which is an
`Iterator<Item = Vec<u8>>` and has to allocate for each read chunk.

Currently, `PipelineData::ExternalStream` serves a weird dual role where
it is either external command output or a wrapper around `RawStream`.
`ByteStream` makes this distinction more clear (via `ByteStreamSource`)
and replaces `PipelineData::ExternalStream` in this PR:
```rust
pub enum PipelineData {
    Empty,
    Value(Value, Option<PipelineMetadata>),
    ListStream(ListStream, Option<PipelineMetadata>),
    ByteStream(ByteStream, Option<PipelineMetadata>),
}
```

The PR is relatively large, but a decent amount of it is just repetitive
changes.

This PR fixes #7017, fixes #10763, and fixes #12369.

This PR also improves performance when piping external commands. Nushell
should, in most cases, have competitive pipeline throughput compared to,
e.g., bash.
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| -------------------------------------------------- | -------------:|
------------:| -----------:|
| `throughput \| rg 'x'` | 3059 | 3744 | 3739 |
| `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 |

# User-Facing Changes
- This is a breaking change for the plugin communication protocol,
because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`.
Plugins now only have to deal with a single input stream, as opposed to
the previous three streams: stdout, stderr, and exit code.
- The output of `describe` has been changed for external/byte streams.
- Temporary breaking change: `bytes starts-with` no longer works with
byte streams. This is to keep the PR smaller, and `bytes ends-with`
already does not work on byte streams.
- If a process core dumped, then instead of having a `Value::Error` in
the `exit_code` column of the output returned from `complete`, it now is
a `Value::Int` with the negation of the signal number.

# After Submitting
- Update docs and book as necessary
- Release notes (e.g., plugin protocol changes)
- Adapt/convert commands to work with byte streams (high priority is
`str length`, `bytes starts-with`, and maybe `bytes ends-with`).
- Refactor the `tee` code, Devyn has already done some work on this.

---------

Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 07:11:18 -07:00
Wind
1b8eb23785
allow passing float value to custom command (#12879)
# Description
Fixes: #12691 

In `parse_short_flag`, it only checks special cases for
`SyntaxShape::Int`, `SyntaxShape::Number` to allow a flag to be a
number. This pr adds `SyntaxShape::Float` to allow a flag to be float
number.

# User-Facing Changes
This is possible after this pr:
```nushell
def spam [val: float] { $val }; 
spam -1.4
```

# Tests + Formatting
Added 1 test
2024-05-16 10:50:29 +02:00
Ian Manske
e20113a0eb
Remove stack debug assert (#12861)
# Description
In order for `Stack::unwrap_unique` to work as intended, we currently
manually track all references to the parent stack and ensure that they
are cleared before calling `Stack::unwrap_unique` in the REPL. We also
only call `Stack::unwrap_unique` after all code from the current REPL
entry has finished executing. Since `Value`s cannot store `Stack`
references, then this should have worked in theory. However, we forgot
to account for threads. `run-external` (and maybe the plugin writers)
can spawn threads that clone the `Stack`, holding on to references of
the parent stack. These threads are not waited/joined upon, and so may
finish after the eval has already returned. This PR removes the
`Stack::unwrap_unique` function and associated debug assert that was
[causing
panics](https://gist.github.com/cablehead/f3d2608a1629e607c2d75290829354f7)
like @cablehead found.

# After Submitting
Make values cheaper to clone as a more robust solution to the
performance issues with cloning the stack.

---------

Co-authored-by: Wind <WindSoilder@outlook.com>
2024-05-15 22:59:10 +00:00
Jack Wright
6f3dbc97bb
fixed syntax shape requirements for --quantiles option for polars summary (#12878)
Fix for #12730

All of the code expected a list of floats, but the syntax shape expected
a table. Resolved by changing the syntax shape to list of floats.

cc: @maxim-uvarov
2024-05-15 16:55:07 -05:00
Ian Manske
06fe7d1e16
Remove usages of Call::positional_nth (#12871)
# Description
Following from #12867, this PR replaces usages of `Call::positional_nth`
with existing spans. This removes several `expect`s from the code.

Also remove unused `positional_nth_mut` and `positional_iter_mut`
2024-05-15 19:59:42 +02:00
NotTheDr01ds
b08135d877
Fixed small error in the help-examples for the get command (#12877)
# Description

Another small error in Help, this time for the `get` command example.

# User-Facing Changes

Help only
2024-05-15 19:49:08 +02:00
NotTheDr01ds
72b880662b
Fixed a nitpick usage-help error - closure v. block (#12876)
# Description

So minor, but had to be fixed sometime. `help each while` used the term
"block" in the "usage", but the argument type is a closure.

# User-Facing Changes

help-only
2024-05-15 18:16:59 +02:00
Ian Manske
0cfbdc909e
Fix sys panic (#12846)
# Description
This should fix #10155 where the `sys` command can panic due to date
math in certain cases / on certain systems.

# User-Facing Changes
The `boot_time` column now has a date value instead of a formatted date
string. This is technically a breaking change.
2024-05-15 15:40:04 +08:00
Wind
155934f783
make better messages for incomplete string (#12868)
# Description
Fixes: #12795

The issue is caused by an empty position of `ParseError::UnexpectedEof`.
So no detailed message is displayed.
To fix the issue, I adjust the start of span to `span.end - 1`. In this
way, we can make sure that it never points to an empty position.

After lexing item, I also reorder the unclosed character checking . Now
it will be checking unclosed opening delimiters first.

# User-Facing Changes
After this pr, it outputs detailed error message for incomplete string
when running scripts.

## Before
```
❯ nu -c "'ab"
Error: nu::parser::unexpected_eof

  × Unexpected end of code.
   ╭─[source:1:4]
 1 │ 'ab
   ╰────
> ./target/debug/nu -c "r#'ab"
Error: nu::parser::unexpected_eof

  × Unexpected end of code.
   ╭─[source:1:6]
 1 │ r#'ab
   ╰────
```
## After
```
> nu -c "'ab"
Error: nu::parser::unexpected_eof

  × Unexpected end of code.
   ╭─[source:1:3]
 1 │ 'ab
   ·   ┬
   ·   ╰── expected closing '
   ╰────
> ./target/debug/nu -c "r#'ab"
Error: nu::parser::unexpected_eof

  × Unexpected end of code.
   ╭─[source:1:5]
 1 │ r#'ab
   ·     ┬
   ·     ╰── expected closing '#
   ╰────
```


# Tests + Formatting
Added some tests for incomplete string.

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-15 01:14:11 +00:00
Ian Manske
c3da44cbb7
Fix char panic (#12867)
# Description
The `char` command can panic due to a failed `expect`: `char --integer
...[77 78 79]`

This PR fixes the panic for the `--integer` flag and also the
`--unicode` flag.

# After Submitting
Check other commands and places where similar bugs can occur due to
usages of `Call::positional_nth` and related methods.
2024-05-14 21:10:06 +00:00
NotTheDr01ds
aa46bc97b3
Search terms for compact command (#12864)
# Description

There was a question in Discord today about how to remove empty rows
from a table. The user found the `compact` command on their own, but I
realized that there were no search terms on the command. I've added
'empty' and 'remove', although I subsequently figured out that 'empty'
is found in the "usage" anyway. That said, I don't think it hurts to
have good search terms behind it regardless.

# User-Facing Changes

Just the help

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
2024-05-14 09:21:50 -05:00
Maxime Jacob
2ed77aef1d
Fix panic when exploring empty dictionary (#12860)
- fixes #12841 

# Description
Add boundary checks to ensure that the row and column chosen in
RecordView are not over the length of the possible row and columns. If
we are out of bounds, we default to Value::nothing.

# Tests + Formatting
Tests ran and formatting done
2024-05-14 14:13:49 +00:00
Maxime Jacob
cd381b74e0
Fix improperly escaped strings in stor insert (#12820)
- fixes #12764 

Replaced the custom logic with values_to_sql method that is already used
in crate::database.
This will ensure that handling of parameters is the same between sqlite
and stor.
2024-05-13 20:22:39 -05:00
Jack Wright
98369985b1
Allow custom value operations to work on eager and lazy dataframes interchangeably. (#12819)
Fixes Bug #12809 

The example that @maxim-uvarov posted now works as expected:

<img width="1223" alt="Screenshot 2024-05-09 at 16 21 01"
src="https://github.com/nushell/nushell/assets/56345/a4df62e3-e432-4c09-8e25-9a6c198741a3">
2024-05-13 18:17:31 -05:00
Piepmatz
aaf973bbba
Add Stack::stdout_file and Stack::stderr_file to capture stdout/-err of external commands (#12857)
# Description
In this PR I added two new methods to `Stack`, `stdout_file` and
`stderr_file`. These two modify the inner `StackOutDest` and set a
`File` into the `stdout` and `stderr` respectively. Different to the
`push_redirection` methods, these do not require to hold a guard up all
the time but require ownership of the stack.

This is primarly useful for applications that use `nu` as a language but
not the `nushell`.

This PR replaces my first attempt #12851 to add a way to capture
stdout/-err of external commands. Capturing the stdout without having to
write into a file is possible with crates like
[`os_pipe`](https://docs.rs/os_pipe), an example for this is given in
the doc comment of the `stdout_file` command and can be executed as a
doctest (although it doesn't validate that you actually got any data).

This implementation takes `File` as input to make it easier to implement
on different operating systems without having to worry about
`OwnedHandle` or `OwnedFd`. Also this doesn't expose any use `os_pipe`
to not leak its types into this API, making it depend on it.

As in my previous attempt, @IanManske guided me here.

# User-Facing Changes
This change has no effect on `nushell` and therefore no user-facing
changes.

# Tests + Formatting
This only exposes a new way of using already existing code and has
therefore no further testing. The doctest succeeds on my machine at
least (x86 Windows, 64 Bit).

# After Submitting
All the required documentation is already part of this PR.
2024-05-13 18:48:38 +00:00
Ian Manske
905ec88091
Update PR template (#12838)
# Description
Updates the command listed in the PR template to test the standard
library, following from #11151.
2024-05-13 08:45:44 -05:00
NotTheDr01ds
c70c43aae9
Add example and search term for 'repeat' to the fill command (#12844)
# Description

It's commonly forgotten or overlooked that a lot of `std repeat`
functionality can be handled with the built-in `fill`. Added 'repeat` as
a search term for `fill` to improve discoverability.

Also replaced one of the existing examples with one `fill`ing an empty
string, a la `repeat`. There were 6 examples already, and 3 of them
pretty much were variations on the same theme, so I repurposed one of
those rather than adding a 7th.

# User-Facing Changes

Changes to `help` only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

I assume the "Commands" doc is auto-generated from the `help`, but I'll
double-check that assumption.
2024-05-12 20:55:07 -05:00
Ian Manske
30fc832035
Fix custom converters with save (#12833)
# Description
Fixes #10429 where `save` fails if a custom command is used as the file
format converter.

# Tests + Formatting
Added a test.
2024-05-12 13:19:28 +02:00
Brage Ingebrigtsen
075535f869
remove --not flag for 'str contains' (#12837)
# Description
This PR resolves an inconsistency between different `str` subcommands,
notably `str contains`, `str starts-with` and `str ends-with`. Only the
`str contains` command has the `--not` flag and a desicion was made in
this #12781 PR to remove the `--not` flag and use the `not` operator
instead.

Before:
`"blob" | str contains --not o`
After:
`not ("blob" | str contains o)` OR `"blob" | str contains o | not $in`

> Note, you can currently do all three, but the first will be broken
after this PR is merged.

# User-Facing Changes
- remove `--not(-n)` flag from `str contains` command
  - This is a breaking change!

# Tests + Formatting
- [x] Added tests
- [x] Ran `cargo fmt --all`
- [x] Ran `cargo clippy --workspace -- -D warnings -D
clippy::unwrap_used`
- [x] Ran `cargo test --workspace`
- [ ] Ran `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"`
    - I was unable to get this working.
```
Error: nu::parser::export_not_found

  × Export not found.
   ╭─[source:1:9]
 1 │ use std testing; testing run-tests --path crates/nu-std
   ·         ───┬───
   ·            ╰── could not find imports
   ╰────
```
^ I still can't figure out how to make this work 😂 

# After Submitting
Requires update of documentation
2024-05-11 23:13:36 +00:00
Ian Manske
cab86f49c0
Fix pipe redirection into complete (#12818)
# Description
Fixes #12796 where a combined out and err pipe redirection (`o+e>|`)
into `complete` still provides separate `stdout` and `stderr` columns in
the record. Now, the combined output will be in the `stdout` column.
This PR also fixes a similar error with the `e>|` pipe redirection.

# Tests + Formatting
Added two tests.
2024-05-11 15:32:00 +00:00
YizhePKU
b9a7faad5a
Implement PWD recovery (#12779)
This PR has two parts. The first part is the addition of the
`Stack::set_pwd()` API. It strips trailing slashes from paths for
convenience, but will reject otherwise bad paths, leaving PWD in a good
state. This should reduce the impact of faulty code incorrectly trying
to set PWD.
(https://github.com/nushell/nushell/pull/12760#issuecomment-2095393012)

The second part is implementing a PWD recovery mechanism. PWD can become
bad even when we did nothing wrong. For example, Unix allows you to
remove any directory when another process might still be using it, which
means PWD can just "disappear" under our nose. This PR makes it possible
to use `cd` to reset PWD into a good state. Here's a demonstration:

```sh
mkdir /tmp/foo
cd /tmp/foo

# delete "/tmp/foo" in a subshell, because Nushell is smart and refuse to delete PWD
nu -c 'cd /; rm -r /tmp/foo'

ls          # Error:   × $env.PWD points to a non-existent directory
            # help: Use `cd` to reset $env.PWD into a good state

cd /
pwd         # prints /
```

Also, auto-cd should be working again.
2024-05-10 11:06:33 -05:00
Ian Manske
70c01bbb26
Fix raw strings as external argument (#12817)
# Description
As discovered by @YizhePKU in a
[comment](https://github.com/nushell/nushell/pull/9956#issuecomment-2103123797)
in #9956, raw strings are not parsed properly when they are used as an
argument to an external command. This PR fixes that.

# Tests + Formatting
Added a test.
2024-05-10 07:50:31 +08:00
Ian Manske
72d3860d05
Refactor the CLI code a bit (#12782)
# Description
Refactors the code in `nu-cli`, `main.rs`, `run.rs`, and few others.
Namely, I added `EngineState::generate_nu_constant` function to
eliminate some duplicate code. Otherwise, I changed a bunch of areas to
return errors instead of calling `std::process::exit`.

# User-Facing Changes
Should be none.
2024-05-10 07:29:27 +08:00
Ian Manske
1b2e680059
Fix syntax highlighting for not (#12815)
# Description
Fixes #12813 where a panic occurs when syntax highlighting `not`. Also
fixes #12814 where syntax highlighting for `not` no longer works.

# User-Facing Changes
Bug fix.
2024-05-10 07:09:44 +08:00
Ian Manske
7271ad7909
Pass Stack ref to Completer::fetch (#12783)
# Description
Adds an additional `&Stack` parameter to `Completer::fetch` so that the
completers don't have to store a `Stack` themselves. I also removed
unnecessary `EngineState`s from the completers, since the same
`EngineState` is available in the `working_set.permanent_state` also
passed to `Completer::fetch`.
2024-05-09 13:38:24 +08:00
Ian Manske
3b3f48202c
Refactor message printing in rm (#12799)
# Description
Changes the iterator in `rm` to be an iterator over
`Result<Option<String>, ShellError>` (an optional message or error)
instead of an iterator over `Value`. Then, the iterator is consumed and
each message is printed. This allows the
`PipelineData::print_not_formatted` method to be removed.
2024-05-09 13:36:47 +08:00
Stefan Holderbach
ba6f38510c
Shrink Value by boxing Range/Closure (#12784)
# Description
On 64-bit platforms the current size of `Value` is 56 bytes. The
limiting variants were `Closure` and `Range`. Boxing the two reduces the
size of Value to 48 bytes. This is the minimal size possible with our
current 16-byte `Span` and any 24-byte `Vec` container which we use in
several variants. (Note the extra full 8-bytes necessary for the
discriminant or other smaller values due to the 8-byte alignment of
`usize`)

This is leads to a size reduction of ~15% for `Value` and should overall
be beneficial as both `Range` and `Closure` are rarely used compared to
the primitive types or even our general container types.

# User-Facing Changes
Less memory used, potential runtime benefits.

(Too late in the evening to run the benchmarks myself right now)
2024-05-09 08:10:58 +08:00
Andy Gayton
92831d7efc
feat: add an echo command to nu_plugin_example (#12754)
# Description

This PR adds a new `echo` command to the `nu_plugin_example` plugin that
simply [streams all of its input to its
output](https://github.com/nushell/nushell/pull/12754/files#diff-de9fcf086b8c373039dadcc2bcb664c6014c0b2af8568eab68c0b6666ac5ccceR47).

```
: "hi" | example echo
hi
```

The motivation for adding it is to have a convenient command to exercise
interactivity on slow pipelines.

I'll follow up on that front with [another
PR](https://github.com/cablehead/nushell/pull/1/files)

# Tests + Formatting

https://github.com/nushell/nushell/pull/12754/files#diff-de9fcf086b8c373039dadcc2bcb664c6014c0b2af8568eab68c0b6666ac5ccceR51-R55
2024-05-08 12:45:44 -07:00
Darren Schroeder
5466da3b52
cleanup osc calls for shell_integration (#12810)
# Description

This PR is a continuation of #12629 and meant to address [Reilly's
stated
issue](https://github.com/nushell/nushell/pull/12629#issuecomment-2099660609).

With this PR, nushell should work more consistently with WezTerm on
Windows. However, that means continued scrolling with typing if osc133
is enabled. If it's possible to run WezTerm inside of vscode, then
having osc633 enabled will also cause the display to scroll with every
character typed. I think the cause of this is that reedline paints the
entire prompt on each character typed. We need to figure out how to fix
that, but that's in reedline.

For my purposes, I keep osc133 and osc633 set to true and don't use
WezTerm on Windows.

Thanks @rgwood for reporting the issue. I found several logic errors.
It's often good to come back to PRs and look at them with fresh eyes. I
think this is pretty close to logically correct now. However, I'm
approaching burn out on ansi escape codes so i could've missed
something.

Kudos to [escape-artist](https://github.com/rgwood/escape-artist) for
helping me debug an ansi escape codes that are actually being sent to
the terminal. It was an invaluable tool.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-08 13:34:04 -05:00
Ian Manske
3b26c08dab
Refactor parse command (#12791)
# Description
- Switches the `excess` in the `ParserStream` and
`ParseStreamerExternal` types from a `Vec` to a `VecDeque`
- Removes unnecessary clones to `stream_helper`
- Other simplifications and loop restructuring
- Merges the `ParseStreamer` and `ParseStreamerExternal` types into a
common `ParseIter`
- `parse` now streams for list values
2024-05-08 06:50:58 -05:00
Devyn Cairns
e462b6cd99
Make the message when running a plugin exe directly clearer (#12806)
# Description

This changes the message that shows up when running a plugin executable
directly rather than as a plugin to direct the user to run `plugin add
--help`, which should have enough information to figure out what's going
on. The message previously just vaguely suggested that the user needs to
run the plugin "from within Nushell", which is not really enough - it
has to be added with `plugin add` to be used as a plugin.

Also fix docs for `plugin add` to mention `plugin use` rather than
`register` (oops)
2024-05-07 20:12:32 -07:00
YizhePKU
7a86b98f61
Migrate to a new PWD API (part 2) (#12749)
Refer to #12603 for part 1.

We need to be careful when migrating to the new API, because the new API
has slightly different semantics (PWD can contain symlinks). This PR
handles the "obviously safe" part of the migrations. Namely, it handles
two specific use cases:

* Passing PWD into `canonicalize_with()`
* Passing PWD into `EngineState::merge_env()`

The first case is safe because symlinks are canonicalized away. The
second case is safe because `EngineState::merge_env()` only uses PWD to
call `std::env::set_current_dir()`, which shouldn't affact Nushell. The
commit message contains detailed stats on the updated files.

Because these migrations touch a lot of files, I want to keep these PRs
small to avoid merge conflicts.
2024-05-07 18:17:49 +03:00
Ian Manske
b9331d1b08
Add sys users command (#12787)
# Description
Add a new `sys users` command which returns a table of the users of the
system. This is the same table that is currently present as
`(sys).host.sessions`. The same table has been removed from the recently
added `sys host` command.

# User-Facing Changes
Adds a new command. (The old `sys` command is left as is.)
2024-05-07 07:52:02 -05:00
Ian Manske
c54d223ea0
Fix list spread syntax highlighting (#12793)
# Description
I broke syntax highlighting for list spreads in #12529. This should fix
#12792 😅. I just copied the code for highlighting record
spreads.
2024-05-07 13:41:47 +08:00
Ian Manske
eccc558a4e
describe refactor (#12770)
# Description

Refactors `describe` a bit. Namely, I added a `Description` enum to get
rid of `compact_primitive_description` and its awkward `Value` pattern
matching.
2024-05-06 23:20:46 +00:00
Ian Manske
1038c64f80
Add sys subcommands (#12747)
# Description
Adds subcommands to `sys` corresponding to each column of the record
returned by `sys`. This is to alleviate the fact that `sys` now returns
a regular record, meaning that it must compute every column which might
take a noticeable amount of time. The subcommands, on the other hand,
only need to compute and return a subset of the data which should be
much faster. In fact, it should be as fast as before, since this is how
the lazy record worked (it would compute only each column as necessary).

I choose to add subcommands instead of having an optional cell-path
parameter on `sys`, since the cell-path parameter would:
- increase the code complexity (can access any value at any row or
nested column)
- prevents discovery with tab-completion
- hinders type checking and allows users to pass potentially invalid
columns

# User-Facing Changes
Deprecates `sys` in favor of the new `sys` subcommands.
2024-05-06 23:20:27 +00:00
Jack Wright
68adc4657f
Polars lazy refactor (#12669)
This moves to predominantly supporting only lazy dataframes for most
operations. It removes a lot of the type conversion between lazy and
eager dataframes based on what was inputted into the command.

For the most part the changes will mean:
* You will need to run `polars collect` after performing operations
* The into-lazy command has been removed as it is redundant.
* When opening files a lazy frame will be outputted by default if the
reader supports lazy frames

A list of individual command changes can be found
[here](https://hackmd.io/@nucore/Bk-3V-hW0)

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-06 23:19:11 +00:00
Wind
97fc190cc5
allow raw string to be used inside subexpression, list, and closure (#12776)
# Description
Fixes: #12744

This pr is moving raw string lex logic into `lex_item` function, so we
can use raw string inside subexpression, list, closure.
```nushell
> [r#'abc'#]
╭───┬─────╮
│ 0 │ abc │
╰───┴─────╯
> (r#'abc'#)
abc
> do {r#'aa'#}
aa
```

# Tests + Formatting
Done

# After Submitting
NaN
2024-05-06 15:53:58 -05:00
Wind
460a1c8f87
Allow ls works inside dir with [] brackets (#12625)
# Description
Fixes: #12429

To fix the issue, we need to pass the `input pattern` itself to
`glob_from` function, but currently on latest main, nushell pass
`expanded path of input pattern` to `glob_from` function.
It causes globbing failed if expanded path includes `[]` brackets.

It's a pity that I have to duplicate `nu_engine::glob_from` function
into `ls`, because `ls` might convert from `NuGlob::NotExpand` to
`NuGlob::Expand`, in that case, `nu_engine::glob_from` won't work if
user want to ls for a directory which includes tilde:
```
mkdir "~abc"
ls "~abc"
```
So I need to duplicate `glob_from` function and pass original
`expand_tilde` information.

# User-Facing Changes
Nan

# Tests + Formatting
Done

# After Submitting
Nan
2024-05-06 14:01:32 +08:00
Ian Manske
e879d4ecaf
ListStream touchup (#12524)
# Description

Does some misc changes to `ListStream`:
- Moves it into its own module/file separate from `RawStream`.
- `ListStream`s now have an associated `Span`.
- This required changes to `ListStreamInfo` in `nu-plugin`. Note sure if
this is a breaking change for the plugin protocol.
- Hides the internals of `ListStream` but also adds a few more methods.
- This includes two functions to more easily alter a stream (these take
a `ListStream` and return a `ListStream` instead of having to go through
the whole `into_pipeline_data(..)` route).
  -  `map`: takes a `FnMut(Value) -> Value`
  - `modify`: takes a function to modify the inner stream.
2024-05-05 16:00:59 +00:00
Antoine Stevan
ce3bc470ba
improve NUON documentation (#12717)
# Description
this PR
- moves the documentation from `lib.rs` to `README.md` while still
including it in the lib file, so that both the [crates.io
page](https://crates.io/crates/nuon) and the
[documentation](https://docs.rs/nuon/latest/nuon/) show the top-level
doc
- mention that comments are allowed in NUON
- add a JSON-NUON example
- put back the formatting of NOTE blocks in the doc

# User-Facing Changes

# Tests + Formatting

# After Submitting
2024-05-05 15:34:22 +02:00
Ian Manske
2f8e397365
Refactor flattening to reduce intermediate allocations (#12756)
# Description
Our current flattening code creates a bunch of intermediate `Vec`s for
each function call. These intermediate `Vec`s are then usually appended
to the current `output` `Vec`. By instead passing a mutable reference of
the `output` `Vec` to each flattening function, this `Vec` can be
reused/appended to directly thereby eliminating the need for
intermediate `Vec`s in most cases.
2024-05-05 10:43:20 +02:00
Viktor Szépe
8eefb7313e
Minimize future false positive typos (#12751)
# Description

Make typos config more strict: ignore false positives where they occur.

1. Ignore only files with typos
2. Add regexp-s with context
3. Ignore variable names only in Rust code
4. Ignore only 1 "identifier"
5. Check dot files

🎁 Extra bonus: fix typos!!
2024-05-04 15:00:44 +00:00
Maxime Jacob
3ae6fe2114
Enable columns with spaces for into_sqlite by adding quotes to column names (#12759)
# Description
Spaces were causing an issue with into_sqlite when they appeared in
column names.

This is because the column names were not properly wrapped with
backticks that allow sqlite to properly interpret the column.

The issue has been addressed by adding backticks to the column names of
into sqlite. The output of the column names when using open is
unchanged, and the column names appear without backticks as expected.

fixes #12700 

# User-Facing Changes
N/A

# Tests + Formatting
Formatting has been respected.

Repro steps from the issue have been done, and ran multiple times. New
values get added to the correct columns as expected.
2024-05-04 08:12:44 -05:00
Ian Manske
1e71cd4777
Bump base64 to 0.22.1 (#12757)
# Description
Bumps `base64` to 0.22.1 which fixes the alphabet used for binhex
encoding and decoding. This required updating some test expected output.

Related to PR #12469 where `base64` was also bumped and ran into the
failing tests.

# User-Facing Changes
Bug fix, but still changes binhex encoding and decoding output.

# Tests + Formatting
Updated test expected output.
2024-05-04 15:56:16 +03:00
YizhePKU
0d6fbdde4a
Fix PWD cannot point to root paths (#12761)
PR https://github.com/nushell/nushell/pull/12603 made it so that PWD can
never contain a trailing slash. However, a root path (such as `/` or
`C:\`) technically counts as "having a trailing slash", so now `cd /`
doesn't work.

I feel dumb for missing such an obvious edge case. Let's just merge this
quickly before anyone else finds out...

EDIT: It appears I'm too late.
2024-05-04 13:05:54 +03:00
Devyn Cairns
709b2479d9
Fix trailing slash in PWD set by cd (#12760)
# Description

Fixes #12758.

#12662 introduced a bug where calling `cd` with a path with a trailing
slash would cause `PWD` to be set to a path including a trailing slash,
which is not allowed. This adds a helper to `nu_path` to remove this,
and uses it in the `cd` command to clean it up before setting `PWD`.

# Tests + Formatting
I added some tests to make sure we don't regress on this in the future.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-04 12:38:37 +03:00
Himadri Bhattacharjee
35a0f7a369
fix: prevent relative directory traversal from crashing (#12438)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

- fixes #11922
- fixes #12203

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This is a rewrite for some parts of the recursive completion system. The
Rust `std::path` structures often ignores things like a trailing `.`
because for a complete path, it implies the current directory. We are
replacing the use of some of these structs for Strings.

A side effect is the slashes being normalized in Windows. For example if
we were to type `foo/bar/b`, it would complete it to `foo\bar\baz`
because a backward slash is the main separator in windows.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Relative paths are preserved. `..`s in the paths won't eagerly show
completions from the parent path. For example, `asd/foo/../b` will now
complete to `asd/foo/../bar` instead of `asd/bar`.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-03 20:17:50 -05:00
Maxim Uvarov
a1287f7b3f
add more tests to the polars plugin (#12719)
# Description

I added some more tests to our mighty `polars` ~~, yet I don't know how
to add expected results in some of them. I would like to ask for help.~~

~~My experiments are in the last commit: [polars:
experiments](f7e5e72019).
Without those experiments `cargo test` goes well.~~
 
UPD. I moved out my unsuccessful test experiments into a separate
[branch](https://github.com/maxim-uvarov/nushell/blob/polars-tests-broken2/).
So, this branch seems ready for a merge.

@ayax79, maybe you'll find time for me please? It's not urgent for sure.

P.S. I'm very new to git. Please feel free to give me any suggestions on
how I should use it better
2024-05-03 20:14:55 -05:00
Stefan Holderbach
406df7f208
Avoid taking unnecessary ownership of intermediates (#12740)
# Description

Judiciously try to avoid allocations/clone by changing the signature of
functions

- **Don't pass str by value unnecessarily if only read**
- **Don't require a vec in `Sandbox::with_files`**
- **Remove unnecessary string clone**
- **Fixup unnecessary borrow**
- **Use `&str` in shape color instead**
- **Vec -> Slice**
- **Elide string clone**
- **Elide `Path` clone**
- **Take &str to elide clone in tests**

# User-Facing Changes
None

# Tests + Formatting
This touches many tests purely in changing from owned to borrowed/static
data
2024-05-04 00:53:15 +00:00
Ian Manske
e6f473695c
Fix typo (#12752) 2024-05-03 16:14:13 -05:00
Raphael Gaschignard
eff7f33086
Report errors that occur on file operations in ls (#12033)
Currently errors just create empty entries inside of resulting
dataframes.

This changeset is meant to help debug #12004, though generally speaking
I do think it's worth having ways to make errors be visible in this kind
of pipeline be visible

An example of what this looks like

<img width="954" alt="image"
src="https://github.com/nushell/nushell/assets/1408472/2c3c9167-2aaf-4f87-bab5-e8302d7a1170">
2024-05-03 10:12:43 -05:00
YizhePKU
bdb6daa4b5
Migrate to a new PWD API (#12603)
This is the first PR towards migrating to a new `$env.PWD` API that
returns potentially un-canonicalized paths. Refer to PR #12515 for
motivations.

## New API: `EngineState::cwd()`

The goal of the new API is to cover both parse-time and runtime use
case, and avoid unintentional misuse. It takes an `Option<Stack>` as
argument, which if supplied, will search for `$env.PWD` on the stack in
additional to the engine state. I think with this design, there's less
confusion over parse-time and runtime environments. If you have access
to a stack, just supply it; otherwise supply `None`.

## Deprecation of other PWD-related APIs

Other APIs are re-implemented using `EngineState::cwd()` and properly
documented. They're marked deprecated, but their behavior is unchanged.
Unused APIs are deleted, and code that accesses `$env.PWD` directly
without using an API is rewritten.

Deprecated APIs:

* `EngineState::current_work_dir()`
* `StateWorkingSet::get_cwd()`
* `env::current_dir()`
* `env::current_dir_str()`
* `env::current_dir_const()`
* `env::current_dir_str_const()`

Other changes:

* `EngineState::get_cwd()` (deleted)
* `StateWorkingSet::list_env()` (deleted)
* `repl::do_run_cmd()` (rewritten with `env::current_dir_str()`)

## `cd` and `pwd` now use logical paths by default

This pulls the changes from PR #12515. It's currently somewhat broken
because using non-canonicalized paths exposed a bug in our path
normalization logic (Issue #12602). Once that is fixed, this should
work.

## Future plans

This PR needs some tests. Which test helpers should I use, and where
should I put those tests?

I noticed that unquoted paths are expanded within `eval_filepath()` and
`eval_directory()` before they even reach the `cd` command. This means
every paths is expanded twice. Is this intended?

Once this PR lands, the plan is to review all usages of the deprecated
APIs and migrate them to `EngineState::cwd()`. In the meantime, these
usages are annotated with `#[allow(deprecated)]` to avoid breaking CI.

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-05-03 14:33:09 +03:00
Ian Manske
f32ecc641f
Remove some macros (#12742)
# Description
Replaces some macros with regular functions or other code.
2024-05-03 10:35:37 +02:00
Devyn Cairns
72f3942c37
Upgrade to interprocess 2.0.0 (#12729)
# Description

This fixes #12724. NetBSD confirmed to work with this change.

The update also behaves a bit better in some ways - it automatically
unlinks and reclaims sockets on Unix, and doesn't try to flush/sync the
socket on Windows, so I was able to remove that platform-specific logic.

They also have a way to split the socket so I could just use one socket
now, but I haven't tried to do that yet. That would be more of a
breaking change but I think it's more straightforward.

# User-Facing Changes

- Hopefully more platforms work

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-02 22:31:33 -07:00
Antoine Stevan
bc6d934fa1
add support for cell-paths to NUON (#12718)
# Description
_cell paths_ can be easily serialized back and forth to NUON with the
leading `$.` syntax.

# User-Facing Changes
```nushell
$.foo.bar.0 | to nuon
```
and
```nushell
"$.foo.bar.0" | from nuon
```
are now possible

# Tests + Formatting
a new `cell_path` test has been added to `nuon`

# After Submitting
2024-05-03 09:25:19 +08:00
Stefan Holderbach
944ebac1c2
Eliminate dead code in nu-explore (#12735)
# Description
Nightly clippy found some unused fields leading me down a rabbit hole of
dead code hidden behind `pub`

Generally removing any already dead code or premature configurability
that is not exposed to the user.

# User-Facing Changes

None in effect.

Removed some options from the `$env.config.explore.hex-dump` record that
were only read into a struct but never used and also not validated.
2024-05-03 08:36:58 +08:00
Ian Manske
847646e44e
Remove lazy records (#12682)
# Description
Removes lazy records from the language, following from the reasons
outlined in #12622. Namely, this should make semantics more clear and
will eliminate concerns regarding maintainability.

# User-Facing Changes
- Breaking change: `lazy make` is removed.
- Breaking change: `describe --collect-lazyrecords` flag is removed.
- `sys` and `debug info` now return regular records.

# After Submitting
- Update nushell book if necessary.
- Explore new `sys` and `debug info` APIs to prevent them from taking
too long (e.g., subcommands or taking an optional column/cell-path
argument).
2024-05-03 08:36:10 +08:00
Devyn Cairns
ad6deadf24
Flush on every plugin Data message (#12728)
# Description

This helps to ensure data produced on a stream is immediately available
to the consumer of the stream. The BufWriter introduced for performance
reasons in 0.93 exposed the behavior that data messages wouldn't make it
to the other side until they filled the buffer in @cablehead's
[`nu_plugin_from_sse`](https://github.com/cablehead/nu_plugin_from_sse).

I had originally not flushed on every `Data` message because I figured
that it isn't really critical that the other side sees those messages
immediately, since they're not used for control and they are flushed
when waiting for acknowledgement or when the buffer is too full anyway.

Increasing the amount of data that can be sent with a single underlying
write increases performance, but this interferes with some plugins that
want to use streams in a more real-time way. In the future I would like
to make this configurable, maybe even per-command, so that a command can
decide what the priority is. But for now I think this is reasonable.

In the worst case, this decreases performance by about 40%, when sending
very small values (just numbers). But for larger values, this PR
actually increases performance by about 20%, because I've increased the
buffer size about 2x to 16,384 bytes. The previous value of 8,192 bytes
was too small to fit a full buffer coming from an external command, so
doubling it makes sense, and now a write of a buffer from an external
command can be done in exactly one write call, which I think makes
sense. I'm doing this at the same time because flushing each data
message would make it very likely that each individual data message from
an external stream would require exactly two writes rather than
approximately one (amortized).

Again, hopefully the tradeoff isn't too bad, and if it is I'll just make
it configurable.

# User-Facing Changes

- Performance of plugin streams will be a bit different
- Plugins that expect to send streams in real-time will work again

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-02 23:51:16 +00:00
Stefan Holderbach
be6137d136
Fix clippy::wrong_self_convention in polars plugin (#12737)
Expected `into_` for `fn(self) -> T`
2024-05-02 19:31:51 +02:00
Stefan Holderbach
b88d8726d0
Rework for new clippy lints (#12736)
- **Clippy lint `assigning_clones`**
- **Clippy lint `legacy_numeric_constants`**
- **`clippy::float_equality_without_abs`**
- **`nu-table`: clippy::zero_repeat_side_effects**

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-02 19:29:03 +02:00
Darren Schroeder
0805f1fd90
overhaul shell_integration to enable individual control over ansi escape sequences (#12629)
# Description

This PR overhauls the shell_integration system by allowing individual
control over which ansi escape sequences are used. As we continue to
broaden our support for more ansi escape sequences, we can't really have
an all-or-nothing strategy. Some ansi escapes cause problems in certain
operating systems or terminals. We should allow the user to choose which
escapes they want.

TODO:
* Gather feedback
* Should osc7, osc9_9 and osc633p be mutually exclusive?
* Is the naming convention for these settings too nerdy osc2, osc7, etc?

closes #11301

# User-Facing Changes
shell_integration is no longer a boolean value. This is what is
supported in the default_config.nu
```nushell
  shell_integration: {
    # osc2 abbreviates the path if in the home_dir, sets the tab/window title, shows the running command in the tab/window title
    osc2: true
    # osc7 is a way to communicate the path to the terminal, this is helpful for spawning new tabs in the same directory
    osc7: true
    # osc8 is also implemented as the deprecated setting ls.show_clickable_links, it shows clickable links in ls output if your terminal supports it
    osc8: true
    # osc9_9 is from ConEmu and is starting to get wider support. It's similar to osc7 in that it communicates the path to the terminal
    osc9_9: false
    # osc133 is several escapes invented by Final Term which include the supported ones below.
    # 133;A - Mark prompt start
    # 133;B - Mark prompt end
    # 133;C - Mark pre-execution
    # 133;D;exit - Mark execution finished with exit code
    # This is used to enable terminals to know where the prompt is, the command is, where the command finishes, and where the output of the command is
    osc133: true
    # osc633 is closely related to osc133 but only exists in visual studio code (vscode) and supports their shell integration features
    # 633;A - Mark prompt start
    # 633;B - Mark prompt end
    # 633;C - Mark pre-execution
    # 633;D;exit - Mark execution finished with exit code
    # 633;E - NOT IMPLEMENTED - Explicitly set the command line with an optional nonce
    # 633;P;Cwd=<path> - Mark the current working directory and communicate it to the terminal
    # and also helps with the run recent menu in vscode
    osc633: true
    # reset_application_mode is escape \x1b[?1l and was added to help ssh work better
    reset_application_mode: true
  }
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-02 09:56:50 -04:00
Darren Schroeder
8ed0d84d6a
add raw-string literal support (#9956)
# Description

This PR adds raw string support by using `r#` at the beginning of single
quoted strings and `#` at the end.

Notice that escapes do not process, even within single quotes,
parentheses don't mean anything, $variables don't mean anything. It's
just a string.
```nushell
❯ echo r#'one\ntwo (blah) ($var)'#
one\ntwo (blah) ($var)
```
Notice how they work without `echo` or `print` and how they work without
carriage returns.
```nushell
❯ r#'adsfa'#
adsfa
❯ r##"asdfa'@qpejq'##
asdfa'@qpejq
❯ r#'asdfasdfasf
∙ foqwejfqo@'23rfjqf'#
```
They also have a special configurable color in the repl. (use single
quotes though)

![image](https://github.com/nushell/nushell/assets/343840/8780e21d-de4c-45b3-9880-2425f5fe10ef)

They should work like rust raw literals and allow `r##`, `r###`,
`r####`, etc, to help with having one or many `#`'s in the middle of
your raw-string.

They should work with `let` as well.

```nushell
r#'some\nraw\nstring'# | str upcase
```

closes https://github.com/nushell/nushell/issues/5091
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: WindSoilder <WindSoilder@outlook.com>
Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-02 09:36:37 -04:00
Reilly Wood
3d340657b5
explore: adopt anyhow, support CustomValue, remove help system (#12692)
This PR:
1. Adds basic support for `CustomValue` to `explore`. Previously `open
foo.db | explore` didn't really work, now we "materialize" the whole
database to a `Value` before loading it
2. Adopts `anyhow` for error handling in `explore`. Previously we were
kind of rolling our own version of `anyhow` by shoving all errors into a
`std::io::Error`; I think this is much nicer. This was necessary because
as part of 1), collecting input is now fallible...
3. Removes a lot of `explore`'s fancy command help system.
- Previously each command (`:help`, `:try`, etc.) had a sophisticated
help system with examples etc... but this was not very visible to users.
You had to know to run `:help :try` or view a list of commands with
`:help :`
- As discussed previously, we eventually want to move to a less modal
approach for `explore`, without the Vim-like commands. And so I don't
think it's worth keeping this command help system around (it's
intertwined with other stuff, and making these changes would have been
harder if keeping it).
4. Rename the `--reverse` flag to `--tail`. The flag scrolls to the end
of the data, which IMO is described better by "tail"
5. Does some renaming+commenting to clear up things I found difficult to
understand when navigating the `explore` code


I initially thought 1) would be just a few lines, and then this PR blew
up into much more extensive changes 😅


## Before
The whole database was being displayed as a single Nuon/JSON line 🤔 

![image](https://github.com/nushell/nushell/assets/26268125/6383f43b-fdff-48b4-9604-398438ad1499)


## After
The database gets displayed like a record

![image](https://github.com/nushell/nushell/assets/26268125/2f00ed7b-a3c4-47f4-a08c-98d07efc7bb4)


## Future work

It is sort of annoying that we have to load a whole SQLite database into
memory to make this work; it will be impractical for large databases.
I'd like to explore improvements to `CustomValue` that can make this
work more efficiently.
2024-05-01 17:34:37 -05:00
Devyn Cairns
2970d48d41
Make bytes build accept integer values as individual bytes (#12685)
# Description
This creates an option for building binary data from byte integers.
Previously I think you could only do this by formatting the integers to
hex and using `decode hex`.

One potentially confusing thing is that this is different from the `into
binary` behavior. But since this doesn't support any of the other `into
binary` behaviors, it might be okay.

# User-Facing Changes
- `bytes build` accepts single byte arguments as integers

# Tests + Formatting
Example added.

# After Submitting
- [ ] release notes
2024-05-01 17:29:33 -05:00
YizhePKU
f184a77fe1
Path expansion no longer removes trailing slashes (#12662)
This PR changes `nu_path::expand_path_with()` to no longer remove
trailing slashes. It also fixes bugs in the current implementation due
to ineffective tests (Fixes #12602).
2024-05-01 17:28:54 -05:00
pwygab
b22d131279
Prevent each from swallowing errors when eval_block returns a ListStream (#12412)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Prior, it seemed that nested errors would not get detected and shown.
This PR fixes that.

Resolves #10176:
```
~/CodingProjects/nushell> [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } }                        05/04/2024 21:34:08
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
   ╭─[entry #1:1:3]
 1 │ [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } }
   ·   ┬
   ·   ╰── source value
   ╰────

Error:   × oh noes
   ╭─[entry #1:1:36]
 1 │ [[1,2]] | each {|x| $x | each {|y| error make {msg: "oh noes"} } }
   ·                                    ─────┬────
   ·                                         ╰── originates from here
   ╰────
```

Resolves #11224:
```
~/CodingProjects/nushell> [0] | each { |_|                                                                          05/04/2024 21:35:40
:::     [0] | each { |_|
:::         non-existent-command
:::     }
::: }
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
   ╭─[entry #1:2:6]
 1 │ [0] | each { |_|
 2 │     [0] | each { |_|
   ·      ┬
   ·      ╰── source value
 3 │         non-existent-command
   ╰────

Error: nu:🐚:external_command

  × External command failed
   ╭─[entry #1:3:9]
 2 │     [0] | each { |_|
 3 │         non-existent-command
   ·         ──────────┬─────────
   ·                   ╰── executable was not found
 4 │     }
   ╰────
  help: No such file or directory (os error 2)
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-01 17:24:54 -05:00
YizhePKU
52d99cc60c
Change environment variables to be case-preserving (#12701)
This PR changes `$env` to be **case-preserving** instead of
case-sensitive. That is, it preserves the case of the environment
variable when it is first assigned, but subsequent retrieval and update
ignores the case.

Notably, both `$env.PATH` and `$env.Path` can now be used to read or set
the environment variable, but child processes will always see the
correct case based on the platform.

Fixes #11268.

---

This feature was surprising simple to implement, because most of the
infrastructure to support case-insensitive cell path access already
exists. The `get` command extracts data using a cell path in a
case-insensitive way (!), but accepts a `--sensitive` flag. (I think
this should be flipped around?)
2024-05-01 17:22:34 -05:00
Devyn Cairns
21ebdfe8d7
Bump version to 0.93.1 (#12710)
# Description

Next patch/dev release, `0.93.1`
2024-05-01 17:19:20 -05:00
Devyn Cairns
3b220e07e3
Bump version to 0.93.0 (#12709)
# Description

Bump version to `0.93.0`
2024-04-30 15:51:13 -07:00
Ian Manske
e83123dcca
Add fs feature to nix dependency (#12702)
# Description
Caught a compilation error using `cargo hack` -- `nu-utils` will not
compile without the `fs` feature enabled for `nix`.
2024-04-29 22:50:39 +00:00
Devyn Cairns
648486400c
Fix missing local socket feature (#12698)
# Description

So sorry to do this during the pre-release freeze, but my plugin crate
split PR broke local socket mode, because `nu-plugin-protocol` didn't
have the compile feature to advertise the `LocalSocket` protocol
feature.

This is a very simple, configuration-only bugfix that I think really
needs to be merged before the release, or else local socket mode won't
work at all.

# Tests + Formatting

There's an oversight in my testing that caused this to not be caught:
the engine really did have the feature, but it just wasn't advertising
it, so for `stress_internals` it was still able to use it successfully.
Post-release I'll try to make sure this is properly handled somehow.
2024-04-29 15:02:56 +08:00
Darren Schroeder
59ee96c70d
fixes a rust-analyzer warning (#12694)
# Description

Minor change but fixes a few rust-analyzer warnings.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-28 04:52:05 -05:00
Darren Schroeder
365f5954ee
restore query web --as-table to working order (#12693)
# Description

This PR fixes a problem introduced with PR
https://github.com/nushell/nushell/pull/12236. That PR accidentally
stopped `--as-table` from working.

Closes https://github.com/nushell/nushell/issues/12689

It works again.

![image](https://github.com/nushell/nushell/assets/343840/b517507f-6b92-4e39-a389-5c69907d77c0)

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-28 04:21:26 -05:00
Devyn Cairns
0c4d5330ee
Split the plugin crate (#12563)
# Description

This breaks `nu-plugin` up into four crates:

- `nu-plugin-protocol`: just the type definitions for the protocol, no
I/O. If someone wanted to wire up something more bare metal, maybe for
async I/O, they could use this.
- `nu-plugin-core`: the shared stuff between engine/plugin. Less stable
interface.
- `nu-plugin-engine`: everything required for the engine to talk to
plugins. Less stable interface.
- `nu-plugin`: everything required for the plugin to talk to the engine,
what plugin developers use. Should be the most stable interface.

No changes are made to the interface exposed by `nu-plugin` - it should
all still be there. Re-exports from `nu-plugin-protocol` or
`nu-plugin-core` are used as required. Plugins shouldn't ever have to
use those crates directly.

This should be somewhat faster to compile as `nu-plugin-engine` and
`nu-plugin` can compile in parallel, and the engine doesn't need
`nu-plugin` and plugins don't need `nu-plugin-engine` (except for test
support), so that should reduce what needs to be compiled too.

The only significant change here other than splitting stuff up was to
break the `source` out of `PluginCustomValue` and create a new
`PluginCustomValueWithSource` type that contains that instead. One bonus
of that is we get rid of the option and it's now more type-safe, but it
also means that the logic for that stuff (actually running the plugin
for custom value ops) can live entirely within the `nu-plugin-engine`
crate.

# User-Facing Changes
- New crates.
- Added `local-socket` feature for `nu` to try to make it possible to
compile without that support if needed.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-27 12:08:12 -05:00
Maxim Uvarov
884d5312bb
add tests to polars unique (#12683)
# Description

I would like to help with `polars` plugin development and add tests to
all the `polars` command's existing params.

Since I have never written any lines of Rust, even though the task of
creating tests is relatively simple, I would like to ask for feedback to
ensure I did everything correctly here.
2024-04-27 12:04:54 -05:00
Devyn Cairns
02de69de92
Fix inconsistent print behavior (#12675)
# Description

I found a bunch of issues relating to the specialized reimplementation
of `print()` that's done in `nu-cli` and it just didn't seem necessary.
So I tried to unify the behavior reasonably. `PipelineData::print()`
already handles the call to `table` and it even has a `no_newline`
option.

One of the most major issues before was that we were using the value
iterator, and then converting to string, and then printing each with
newlines. This doesn't work well for an external stream, because its
iterator ends up creating `Value::binary()` with each buffer... so we
were doing lossy UTF-8 conversion on those and then printing them with
newlines, which was very weird:


![Screenshot_2024-04-26_02-02-29](https://github.com/nushell/nushell/assets/10729/131c2224-08ee-4582-8617-6ecbb3ce8da5)

You can see the random newline inserted in a break between buffers, but
this would be even worse if it were on a multibyte UTF-8 character. You
can produce this by writing a large amount of text to a text file, and
then doing `nu -c 'open file.txt'` - in my case I just wrote `^find .`;
it just has to be large enough to trigger a buffer break.

Using `print()` instead led to a new issue though, because it doesn't
abort on errors. This is so that certain commands can produce a stream
of errors and have those all printed. There are tests for e.g. `rm` that
depend on this behavior. I assume we want to keep that, so instead I
made my target `BufferedReader`, and had that fuse closed if an error
was encountered. I can't imagine we want to keep reading from a wrapped
I/O stream if an error occurs; more often than not the error isn't going
to magically resolve itself, it's not going to be a different error each
time, and it's just going to lead to an infinite stream of the same
error.

The test that broke without that was `open . | lines`, because `lines`
doesn't fuse closed on error. But I don't know if it's expected or not
for it to do that, so I didn't target that.

I think this PR makes things better but I'll keep looking for ways to
improve on how errors and streams interact, especially trying to
eliminate cases where infinite error loops can happen.

# User-Facing Changes
- **Breaking**: `BufferedReader` changes + no more public fields
- A raw I/O stream from e.g. `open` won't produce infinite errors
anymore, but I consider that to be a plus
- the implicit `print` on script output is the same as the normal one
now

# Tests + Formatting
Everything passes but I didn't add anything specific.
2024-04-27 00:25:11 +00:00
Ian Manske
533603b72c
Add deprecation warning to describe --collect-lazyrecords (#12667)
# Description
Missed a spot for lazy record deprecation. This adds a warning for the
`--collect-lazyrecords` flag on `describe`.
2024-04-26 16:36:30 +00:00
Ian Manske
1ecbb3e09f
Make exit code available in catch block (#12648)
# Description
Bandaid fix for #12643, where it is not possible to get the exit code of
a failed external command while also having the external command inherit
nushell's stdout and stderr. This changes `try` so that the exit code of
external command is available in the `catch` block via the usual
`$env.LAST_EXIT_CODE`.

# Tests + Formatting
Added one test.

# After Submitting
Rework I/O redirection and possibly exit codes.
2024-04-26 16:35:08 +00:00
pwygab
d23a3737c0
make grid throw an error when not enough columns (#12672)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Resolves #12654. 

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

`grid` can now throw an error.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added relevant test.
2024-04-26 06:33:00 -05:00
Devyn Cairns
d126793290
Add plugin error propagation on write/flush (#12670)
# Description
Yet another attempt to fix the `stress_internals::test_wrong_version()`
test...

This time I think it's probably because we are getting a broken pipe
write error when we try to send `Hello` or perhaps something after it,
because the pipe has already been closed by the reader when it saw the
invalid version. In that case, an error should be available in state. It
probably makes more sense to send that back to the user rather than an
unhelpful I/O error.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-26 06:23:58 -05:00
Devyn Cairns
adf38c7c76
Msgpack commands (#12664)
# Description

I thought about bringing `nu_plugin_msgpack` in, but that is MPL with a
clause that prevents other licenses, so rather than adapt that code I
decided to take a crack at just doing it straight from `rmp` to `Value`
without any `rmpv` in the middle. It seems like it's probably faster,
though I can't say for sure how much with the plugin overhead.

@IanManske I started on a `Read` implementation for `RawStream` but just
specialized to `from msgpack` here, but I'm thinking after release maybe
we can polish it up and make it a real one. It works!

# User-Facing Changes
New commands:

- `from msgpack`
- `from msgpackz`
- `to msgpack`
- `to msgpackz`

# Tests + Formatting
Pretty thorough tests added for the format deserialization, with a
roundtrip for serialization. Some example tests too for both `from
msgpack` and `to msgpack`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
- [ ] update release notes
2024-04-26 06:23:16 -05:00
Ian Manske
79ebf0c0a9
Fix an into bits example (#12668)
# Description
One example for `into bits` says it uses binary value when it actually
uses a filesize. This lead to issue #11412, but I never got around to
fixing the example until this PR.
2024-04-25 19:38:28 -05:00
Ian Manske
f234a0ea33
each signature fix (#12666)
# Description
Removes the second `Int` parameter from the closure in the signature of
`each`. This parameter doesn't exist / isn't supported.
2024-04-25 22:47:30 +02:00
Andrej Kolchin
2466a39574
Fix example wording in seq date (#12665)
# Description

The previous messages said that the command printed dates separated by
newlines. But the current iteration of `seq date` returns a list.

# User-Facing Changes

Minor wording edit.

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-04-25 22:12:42 +02:00
Ian Manske
1239e86320
Remove deprecated flags on run-external (#12659)
# Description
Removes the `run-external` flags that were deprecated in 0.92.0 with
#11934.
2024-04-25 14:17:21 +00:00
Ian Manske
5b0546cfce
Remove deprecated flags on commandline (#12658)
# Description
Removes the `commandline` flags and API that was deprecated in 0.91.0
with #11877.

# User-Facing Changes
Users need to migrate to the new `commandline` subcommands introduced in
0.91.0.
2024-04-25 14:16:42 +00:00
Ian Manske
b6d765a2d8
nu-cmd-lang cleanup (#12609)
# Description
This PR does miscellaneous cleanup in some of the commands from
`nu-cmd-lang`.

# User-Facing Changes
None.

# After Submitting
Cleanup the other commands in `nu-cmd-lang`.
2024-04-25 14:16:12 +00:00
Ian Manske
530162b4c4
Lazy record deprecation (#12656)
# Description
In this week's nushell meeting, we decided to go ahead with #12622 and
remove lazy records in 0.94.0. For 0.93.0, we will only deprecate `lazy
make`, and so this PR makes `lazy make` print a deprecation warning.

# User-Facing Changes
None, besides the deprecation warning.

# After Submitting
Remove lazy records.
2024-04-25 07:05:24 +08:00
Devyn Cairns
25cbcb511d
Rename plugin cache file ⇒ plugin registry file (#12634)
# Description
So far this seems like the winner of my poll on what the name should be.
I'll take this off draft once the poll expires, if this is indeed the
winner.
2024-04-24 17:40:39 -05:00
Ian Manske
9996e4a1f8
Shrink the size of Expr (#12610)
# Description
Continuing from #12568, this PR further reduces the size of `Expr` from
64 to 40 bytes. It also reduces `Expression` from 128 to 96 bytes and
`Type` from 32 to 24 bytes.

This was accomplished by:
- for `Expr` with multiple fields (e.g., `Expr::Thing(A, B, C)`),
merging the fields into new AST struct types and then boxing this struct
(e.g. `Expr::Thing(Box<ABC>)`).
- replacing `Vec<T>` with `Box<[T]>` in multiple places. `Expr`s and
`Expression`s should rarely be mutated, if at all, so this optimization
makes sense.

By reducing the size of these types, I didn't notice a large performance
improvement (at least compared to #12568). But this PR does reduce the
memory usage of nushell. My config is somewhat light so I only noticed a
difference of 1.4MiB (38.9MiB vs 37.5MiB).

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-24 15:46:35 +00:00
Devyn Cairns
c52884b3c8
Fix (and test) for a deadlock that can happen while waiting for protocol info (#12633)
# Description

The local socket PR introduced a `Waitable` type, which could either
hold a value or be waited on until a value is available. Unlike a
channel, it would always return that value once set.

However, one issue with this design was that there was no way to detect
whether a value would ever be written. This splits the writer into a
different type `WaitableMut`, so that when it is dropped, waiting
threads can fail (because they'll never get a value).

# Tests + Formatting

A test has been added to `stress_internals` to make sure this fails in
the right way.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-24 08:44:04 -05:00
Devyn Cairns
0f645b3bb6
Futher improve messages related to error propagation on plugin calls (#12646)
# Description
Trying to give as much context as possible. Now there should be a
spanned error with the call span if possible, and the propagated error
as an inner error if there was one in every case.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-24 08:39:04 -05:00
dependabot[bot]
9d65c47313
Bump rust-ini from 0.20.0 to 0.21.0 (#12637) 2024-04-24 11:55:20 +00:00
Devyn Cairns
b576123b0a
Accept filenames in other plugin management commands (#12639)
# Description

This allows the following commands to all accept a filename instead of a
plugin name:

- `plugin use`
- `plugin rm`
- `plugin stop`

Slightly complicated because of the need to also check against
`NU_PLUGIN_DIRS`, but I also fixed some issues with that at the same
time

Requested by @fdncred

# User-Facing Changes

The new commands are updated as described.

# Tests + Formatting

Tests for `NU_PLUGIN_DIRS` handling also made more robust.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Double check new docs to make sure they describe this capability
2024-04-24 06:28:45 -05:00
Devyn Cairns
c9bc0c7d3e
Fix error message propagation on plugin call failure (#12632)
# Description

This should fix the sometimes failing wrong version test for
stress_internals.

The plugin interface state stores an error if some kind of critical
error happened, and this error should be propagated to any future
operations on the interface, but this wasn't being propagated to plugin
calls that were already waiting.

During plugin registration, the wrong version error needs to be received
as a response to the `get_signature()` to show up properly, but this
would only happen if `get_signature()` started after the `Hello` was
already received and processed. That would be a race condition, which
this commit solves.

cc @sholderbach - this should fix the CI issue

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-23 17:30:51 -05:00
Devyn Cairns
1f4131532d
Deprecate register and add plugin use (#12607)
# Description

Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:

```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```

This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.

The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.

`register` now prints a deprecation parse warning.

This should fix #11923, as we now have a complete alternative to
`register`.

# User-Facing Changes

- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`

# Tests + Formatting

I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Update plugins sections of book
- [ ] Release notes
2024-04-23 06:37:50 -05:00
Devyn Cairns
5c7f7883c8
Add ErrSpan extension trait for Result (#12626)
# Description
This adds an extension trait to `Result` that wraps errors in `Spanned`,
saving the effort of calling `.map_err(|err| err.into_spanned(span))`
every time. This will hopefully make it even more likely that someone
will want to use a spanned `io::Error` and make it easier to remove the
impl for `From<io::Error> for ShellError` because that doesn't have span
information.
2024-04-23 10:39:55 +02:00
Wind
b0acc1d890
Avoid panic when pipe a variable to a custom command which have recursive call (#12491)
# Description
Fixes: #11351

And comment here is also fixed:
https://github.com/nushell/nushell/issues/11351#issuecomment-1996191537

The panic can happened if we pipe a variable to a custom command which
recursively called itself inside another block.

TBH, I think I figure out how it works to panic, but I'm not sure if
there is a potention issue if nushell don't mutate a block in such case.

# User-Facing Changes
Nan

# Tests + Formatting
Done

# After Submitting
Done

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-23 06:10:35 +08:00
Ian Manske
797a90520c
Improve the "input and output are the same file" error text (#12619)
# Description
Continuing from #12601, this PR improves the error message help text and
adds an example to `collect`.
2024-04-22 09:00:38 -05:00
Ian Manske
bae6d694ca
Refactor using ClosureEval types (#12541)
# Description
Adds two new types in `nu-engine` for evaluating closures: `ClosureEval`
and `ClosureEvalOnce`. This removed some duplicate code and centralizes
our logic for setting up, running, and cleaning up closures. For
example, in the future if we are able to reduce the cloning necessary to
run a closure, then we only have to change the code related to these
types.

`ClosureEval` and `ClosureEvalOnce` are designed with a builder API.
`ClosureEval` is used to run a closure multiple times whereas
`ClosureEvalOnce` is used for a one-shot closure.

# User-Facing Changes
Should be none, unless I messed up one of the command migrations.
Actually, this will fix any unreported environment bugs for commands
that didn't reset the env after running a closure.
2024-04-22 14:15:09 +08:00
Ian Manske
83720a9f30
Make the same file error more likely to appear (#12601)
# Description
When saving to a file we currently try to check if the data source in
the pipeline metadata is the same as the file we are saving to. If so,
we create an error, since reading and writing to a file at the same time
is currently not supported/handled gracefully. However, there are still
a few instances where this error is not properly triggered, and so this
PR attempts to reduce these cases. Inspired by #12599.

# Tests + Formatting
Added a few tests.

# After Submitting
Some commands still do not properly preserve metadata (e.g., `str trim`)
and so prevent us from detecting this error.
2024-04-22 01:12:13 +00:00
Jack Wright
a60381a932
Added commands for working with the plugin cache. (#12576)
# Description
This pull request provides three new commands:
`polars store-ls` - moved from `polars ls`. It provides the list of all
object stored in the plugin cache
`polars store-rm` - deletes a cached object
`polars store-get` - gets an object from the cache. 

The addition of `polars store-get` required adding a reference_count to
cached entries. `polars get` is the only command that will increment
this value. `polars rm` will remove the value despite it's count. Calls
to PolarsPlugin::custom_value_dropped will decrement the value.

The prefix store- was chosen due to there already being a `polars cache`
command. These commands were not made sub-commands as there isn't a way
to display help for sub commands in plugins (e.g. `polars store`
displaying help) and I felt the store- seemed fine anyways.

The output of `polars store-ls` now shows the reference count for each
object.

# User-Facing Changes
polars ls has now moved to polars store-ls

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-21 19:43:43 -05:00
Devyn Cairns
8b7696f4c1
stress_internals: exit(1) on io error (#12612)
# Description

The `stress_internals` tests can fail sometimes, but usually not on the
CI, because Nushell exits while the plugin is still trying to read or
maybe write something, leading to a broken pipe.

`nu-plugin` already exits with 1 without printing a message on a
protocol-level I/O error, so this just doing the same thing.

I think there's probably a way to correct the plugin handling so that we
wait for plugins to shut down before exiting and this doesn't happen,
but this is the quick fix in the meantime.
2024-04-21 23:48:09 +02:00
Antoine Stevan
a900166e27
fix typo in the documentation of nuon::ToStyle (#12608)
follow-up to
- https://github.com/nushell/nushell/pull/12591

cc/ @fdncred 

# Description
there was a typo in the doc of `nuon::ToStyle`.

# User-Facing Changes

# Tests + Formatting

# After Submitting
2024-04-21 10:53:53 -05:00
Devyn Cairns
2595f31541
Overhaul the plugin cache file with a new msgpack+brotli format (#12579)
# Description

- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed

# User-Facing Changes

- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
  - `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`

# Tests + Formatting

Tests added for the format and for the invalid handling.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Check for documentation changes
- [ ] Definitely needs release notes
2024-04-21 07:36:26 -05:00
Stefan Holderbach
6cba7c6b40
Small refactor in cal (#12604)
Removed some unnecessary code found when looking at #12597

- **Simplify day of the week selection**
- **Use `Record` directly instead of `IndexMap`**
2024-04-21 10:41:26 +02:00
Ian Manske
3b1d405b96
Remove the Value::Block case (#12582)
# Description
`Value` describes the types of first-class values that users and scripts
can create, manipulate, pass around, and store. However, `Block`s are
not first-class values in the language, so this PR removes it from
`Value`. This removes some unnecessary code, and this change should be
invisible to the user except for the change to `scope modules` described
below.

# User-Facing Changes
Breaking change: the output of `scope modules` was changed so that
`env_block` is now `has_env_block` which is a boolean value instead of a
`Block`.

# After Submitting
Update the language guide possibly.
2024-04-21 07:03:33 +02:00
Maxim Uvarov
5fd34320e9
add search_term "str extract" to parse command (#12600)
# Description

For a long time, I was searching for the `str extract` command to
extract regexes from strings. I often painfully used `str replace -r
'(.*)(pattern_to_find)(.*)' '$2'` for such purposes.
Only this morning did I realize that `parse` is what I needed for so
many times, which I had only used for parsing data in tables.
2024-04-21 07:01:42 +02:00
woosaaahh
5e52bd77e0
Change cal --week-start examples + error message (#12597)
This PR fixes the example for `cal --week-start` and adds the list of
expected values to the error message.
2024-04-20 22:39:46 +02:00
Poliorcetics
47867a58df
Ab/version details (#12593)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Closes #12561

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Add version details in `version`'s output. The intended use is for
third-party tools to be able to quickly check version numbers without
having to the parsing from `(version).version` or `$env.NU_VERSION`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

This adds 4 new values to the record from `version`:

```
...
│ major              │ 0                                               │
│ minor              │ 92                                              │
│ patch              │ 3                                               │
│ pre                │ a-value                                         │
...
```

`pre` is optional and won't be present most of the time I think.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

I ran the new command using `cargo run -- -c version`:

```
╭────────────────────┬─────────────────────────────────────────────────╮
│ version            │ 0.92.3                                          │
│ major              │ 0                                               │
│ minor              │ 92                                              │
│ patch              │ 3                                               │
│ branch             │                                                 │
│ commit_hash        │                                                 │
│ build_os           │ macos-aarch64                                   │
│ build_target       │ aarch64-apple-darwin                            │
│ rust_version       │ rustc 1.77.2 (25ef9e3d8 2024-04-09)             │
│ rust_channel       │ 1.77.2-aarch64-apple-darwin                     │
│ cargo_version      │ cargo 1.77.2 (e52e36006 2024-03-26)             │
│ build_time         │ 2024-04-20 15:09:36 +02:00                      │
│ build_rust_channel │ release                                         │
│ allocator          │ mimalloc                                        │
│ features           │ default, sqlite, system-clipboard, trash, which │
│ installed_plugins  │                                                 │
╰────────────────────┴─────────────────────────────────────────────────╯
```

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

After this is merged I would like to write docs somewhere for scripts
writer to advise using these members instead of the string values. Where
should I put said docs ?
2024-04-20 10:01:51 -05:00
Darren Schroeder
b274ec19fd
improve nu --lsp command tooltips (#12589)
# Description

This PR improves the `nu --lsp` tooltips by using nu code blocks around
the examples and a few other places.

This is what it looks like in Zed.
![Screenshot 2024-04-19 at 8 20
53 PM](https://github.com/nushell/nushell/assets/343840/20d51dcc-f3b2-4f2b-9d43-5817dd3913df)

Here it is in Helix.

![image](https://github.com/nushell/nushell/assets/343840/a9e7d6b9-cd21-4a5a-9c88-9af17a2b2363)

This coloring is far from perfect, but it's what the tree-sitter-nu
queries generate.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-20 07:16:38 -05:00
Antoine Stevan
be5ed3290c
add "to nuon" enumeration of possible styles (#12591)
# Description
in order to change the style of the _serialized_ NUON data,
`nuon::to_nuon` takes three mutually exclusive arguments, `raw: bool`,
`tabs: Option<usize>` and `indent: Option<usize>` 🤔
this begs to use an enumeration with all possible alternatives, right?

this PR changes the signature of `nuon::to_nuon` to use `nuon::ToStyle`
which has three variants
- `Raw`: no newlines
- `Tabs(n: usize)`: newlines and `n` tabulations as indent
- `Spaces(n: usize)`: newlines and `n` spaces as indent

# User-Facing Changes
the signature of `nuon::to_nuon` changes from
```rust
to_nuon(
    input: &Value,
    raw: bool,
    tabs: Option<usize>,
    indent: Option<usize>,
    span: Option<Span>,
) -> Result<String, ShellError>
```
to
```rust
to_nuon(
    input: &Value,
    style: ToStyle,
    span: Option<Span>
) -> Result<String, ShellError>
```

# Tests + Formatting

# After Submitting
2024-04-20 11:40:52 +02:00
Wind
187b87c61c
Don't allow skip on external stream (#12559)
# Description
Close: #12514

# User-Facing Changes
`^ls | skip 1` will raise an error
```nushell
❯ ^ls | skip 1
Error: nu:🐚:only_supports_this_input_type

  × Input type not supported.
   ╭─[entry #1:1:2]
 1 │ ^ls | skip 1
   ·  ─┬   ──┬─
   ·   │     ╰── only list, binary or range input data is supported
   ·   ╰── input type: raw data
   ╰────
```

# Tests + Formatting
Sorry I can't add it because of the issue:
https://github.com/nushell/nushell/issues/12558

# After Submitting
Nan
2024-04-19 14:54:59 +00:00
Devyn Cairns
f2169c8022
Switch plugin msgpack protocol to named format (#12580)
# Description
In conflict with the documentation, the msgpack serializer for plugins
is actually using the compact format, which doesn't name struct fields,
and is instead dependent on their ordering, rendering them as tuples.
This is not a good idea for a robust protocol even if it makes the
serialization and deserialization faster.

I expect this to have some impact on performance, but I think the
robustness is probably worth it.

Deserialization always accepts either format, so this shouldn't cause
too many incompatibilities.

# User-Facing Changes
This does technically change the protocol, but it makes it reflect the
documentation. It shouldn't break deserialization, so plugins shouldn't
necessarily need a recompile.

Performance is likely worse and I should benchmark the difference.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-19 07:50:51 -05:00
Jack Wright
9fb59a6f43
Removed the polars dtypes command (#12577)
# Description
The polars dtype command is largerly redundant since the introduction of
the schema command. The schema command also has the added benefit that
it's output can be used as a parameter to other schema commands:

```nushell
[[a b]; [5 6] [5 7]] | polars into-df -s ($df | polars schema
```

# User-Facing Changes
`polars dtypes` has been removed. Users should use `polars schema`
instead.

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-19 07:01:47 -05:00
Antoine Stevan
55edef5dda
create nuon crate from from nuon and to nuon (#12553)
# Description
playing with the NUON format in Rust code in some plugins, we agreed
with the team it was a great time to create a standalone NUON format to
allow Rust devs to use this Nushell file format.

> **Note**
> this PR almost copy-pastes the code from
`nu_commands/src/formats/from/nuon.rs` and
`nu_commands/src/formats/to/nuon.rs` to `nuon/src/from.rs` and
`nuon/src/to.rs`, with minor tweaks to make then standalone functions,
e.g. remove the rest of the command implementations

### TODO
- [x] add tests
- [x] add documentation

# User-Facing Changes
devs will have access to a new crate, `nuon`, and two functions,
`from_nuon` and `to_nuon`
```rust
from_nuon(
    input: &str,
    span: Option<Span>,
) -> Result<Value, ShellError>
```
```rust
to_nuon(
    input: &Value,
    raw: bool,
    tabs: Option<usize>,
    indent: Option<usize>,
    span: Option<Span>,
) -> Result<String, ShellError>
```

# Tests + Formatting
i've basically taken all the tests from
`crates/nu-command/tests/format_conversions/nuon.rs` and converted them
to use `from_nuon` and `to_nuon` instead of Nushell commands
- i've created a `nuon_end_to_end` to run both conversions with an
optional middle value to check that all is fine

> **Note** 
> the `nuon::tests::read_code_should_fail_rather_than_panic` test does
give different results locally and in the CI...
> i've left it ignored with comments to help future us :)

# After Submitting
mention that in the release notes for sure!!
2024-04-19 13:54:16 +02:00
Devyn Cairns
fac2f43aa4
Add an example Nushell plugin written in Nushell itself (#12574)
# Description

As suggested by @fdncred.

It's neat that this is possible, but the particularly useful part of
this is that we can actually
test it because it doesn't have any external dependencies, unlike the
python plugin.

Right now this just implements exactly the same behavior as the python
plugin, but we could have it
exercise a few more things.

Also fixes a couple of bugs:

- `.nu` plugins were not run with `nu --stdin`, so they couldn't take
input.
- `register` couldn't be called if `--no-config-file` was set, because
it would error on trying to
  update the plugin file.

# User-Facing Changes

- `nu_plugin_nu_example` plugin added.
- `register` now works in `--no-config-file` mode.

# Tests + Formatting
Tests added for `nu_plugin_nu_example`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Add the version bump to the release script just like for python
2024-04-19 09:53:30 +03:00
YizhePKU
6d2cb4382a
Fix circular source causing Nushell to crash (#12262)
# Description

EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).

Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).

Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->


<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

It shouldn't have any user facing changes.
2024-04-19 09:38:08 +03:00
Brooks Rady
351bff8233
fix(shell_integration): set window title on startup (#12569)
Should close #10833 — though I'd imagine that should have already been
closed.

# Description

Very minor tweak, but it was quite noticeable when using Zellij which
relies on OSC 2 to set pane titles. Before the change:

![image](https://github.com/nushell/nushell/assets/6251883/b944bbce-2040-4886-9955-3c5b57d368e9)

Note that the default `Pane #1` is still showing for the untouched
shell, but running a command like `htop` or `ls` correctly sets the
title during / afterwards.

After this PR:

![image](https://github.com/nushell/nushell/assets/6251883/dd513cfe-923c-450f-b0f2-c66938b0d6f0)

There are now no-longer any unset titles — even if the shell hasn't been
touched.

**As an aside:** I feel quite strongly that (at least OSC 2) shell
integration should be enabled by default, as it is for every other Linux
shell I've used, but I'm not sure which issues that caused that the
default config refers to? Which terminals are broken by shell
integration, and could some of the shell integrations be turned on by
default after splitting things into sub-options as suggested in #11301 ?

# User-Facing Changes

You'll just have shell integrations working from right after the shell
has been launched, instead of needing to run something first.

# Tests + Formatting

Not quite sure how to test this one? Are there any other tests that
currently exist for shell integration? I couldn't quite track them
down...

# After Submitting

Let me know if you think this needs any user-facing docs changes!
2024-04-18 20:30:37 -05:00
Darren Schroeder
7fe2e60af7
add ability to set metadata (#12564)
# Description

This PR adds the ability to set metadata. This is especially useful for
activating LS_COLORS when using table literals.


![image](https://github.com/nushell/nushell/assets/343840/feef6433-f592-43ea-890a-38cb2df35686)

You can also set the filepath metadata, although I'm not really user how
useful this is. We may end up removing this option entirely.
```nushell
❯ "crates" | metadata set --datasource-filepath $'(pwd)/crates' | metadata
╭────────┬───────────────────────────────────╮
│ source │ /Users/fdncred/src/nushell/crates │
╰────────┴───────────────────────────────────╯
```

No file paths are checked. You could also do this.
```nushell
❯ "crates" | metadata set --datasource-filepath $'a/b/c/d/crates' | metadata
╭────────┬────────────────╮
│ source │ a/b/c/d/crates │
╰────────┴────────────────╯
```

The command name and parameter names are still WIP. We could change
them.

There are currently 3 kinds of metadata in nushell.
```rust
pub enum DataSource {
    Ls,
    HtmlThemes,
    FilePath(PathBuf),
}
```

I've skipped adding `HtmlThemes` because it seems to be specific to our
`to html` command only.
2024-04-19 09:03:59 +08:00
Devyn Cairns
999dfdf936
Fix the error output in the python plugin to match LabeledError (#12575)
# Description

Forgot to update this after updating the format of `LabeledError`.

# Tests + Formatting

Not applicable to the python plugin.
2024-04-18 20:01:35 -05:00
Jack Wright
cc7b5c5a26
Only mark collected dataframes as from_lazy=false when collect is called from the collect command. (#12571)
I had previously changed NuLazyFrame::collect to set the NuDataFrame's
from_lazy field to false to prevent conversion back to a lazy frame. It
appears there are cases where this should happen. Instead, I am only
setting from_lazy=false inside the `polars collect` command.

[Related discord
message](https://discord.com/channels/601130461678272522/1227612017171501136/1230600465159421993)

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-18 17:10:38 -05:00
Ian Manske
9a265847e2
Box ImportPattern in Expr (#12568)
# Description
Adds a `Box` around the `ImportPattern` in `Expr` which decreases the
size of `Expr` from 152 to 64 bytes (and `Expression` from 216 to 128
bytes). This seems to speed up parsing a little bit according to the
benchmarks (main is top, PR is bottom):
```
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ parser_benchmarks                           │               │               │               │         │
├─ parser_benchmarks                           │               │               │               │         │
│  ├─ parse_default_config_file  2.287 ms      │ 4.532 ms      │ 2.311 ms      │ 2.437 ms      │ 100     │ 100
│  ├─ parse_default_config_file  2.255 ms      │ 2.781 ms      │ 2.281 ms      │ 2.312 ms      │ 100     │ 100
│  ╰─ parse_default_env_file     421.8 µs      │ 824.6 µs      │ 494.3 µs      │ 527.5 µs      │ 100     │ 100
│  ╰─ parse_default_env_file     402 µs        │ 486.6 µs      │ 414.8 µs      │ 416.2 µs      │ 100     │ 100

```
2024-04-18 17:57:01 +02:00
Stefan Holderbach
b088f395dc
Update crate feature flags (#12566)
# Description
Remove unused/effect-less features, make sure we show all relevant
features in `version`

# User-Facing Changes
- **Remove unused feature `wasi`**
- will cause failure to build should you enable it. Otherwise no effect
- **Include feat `system-clipboard` in `version`**
2024-04-18 16:33:41 +02:00
Ian Manske
6ccd547d81
Add ListItem type for Expr::List (#12529)
# Description
This PR adds a `ListItem` enum to our set of AST types. It encodes the
two possible expressions inside of list expression: a singular item or a
spread. This is similar to the existing `RecordItem` enum. Adding
`ListItem` allows us to remove the existing `Expr::Spread` case which
was previously used for list spreads. As a consequence, this guarantees
(via the type system) that spreads can only ever occur inside lists,
records, or as command args.

This PR also does a little bit of cleanup in relevant parser code.
2024-04-18 13:21:05 +02:00
Jack Wright
57b0c722c6
Upgrading nu-cmd-dataframe to polars 0.39 (#12554)
#Description
Upgrading nu-cmd-dataframe to polars 0.39

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-17 12:50:17 -05:00
Devyn Cairns
13160b3ec3
Replace subtraction of Instants and Durations with saturating subtractions (#12549)
# Description
Duration can not be negative, and an underflow causes a panic.

This should fix #12539 as from what I can tell that bug was caused in
`nu-explore:📟:events` from subtracting durations, but I figured
this might be more widespread, and saturating to zero generally makes
sense.

I also added the relevant clippy lint to try to prevent this from
happening in the future. I can't think of a reason we would ever want to
subtract durations without checking first.

cc @fdncred

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-17 07:25:16 -05:00
Jack Wright
410f3c5c8a
Upgrading nu_plugin_polars to polars 0.39.1 (#12551)
# Description
Upgrading nu_plugin_polars to polars 0.39.1

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-17 06:35:09 -05:00
Devyn Cairns
b296d6ee3c
Improve safety of get_unchecked_str in nu_system::macos (#12550)
# Description
The implementation of this function had a few issues before:

- It didn't check that the `cp` pointer is actually ahead of the `start`
pointer, so `len` could potentially underflow and wrap around, which
would be a violation of memory safety
- It used `Vec::from_raw_parts` even though the buffer is borrowed, not
owned. Although `std::mem::forget` is used later to ensure the
destructor doesn't run, there is a risk that the destructor would run if
a panic happened during `String::from_utf8_unchecked`, which would lead
to a `free()` of a pointer we don't own
2024-04-17 17:10:05 +08:00
dependabot[bot]
9a739d9f0d
Bump rmp-serde from 1.1.2 to 1.2.0 (#12547)
Bumps [rmp-serde](https://github.com/3Hren/msgpack-rust) from 1.1.2 to
1.2.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/3Hren/msgpack-rust/commits/rmp-serde/v1.2.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rmp-serde&package-manager=cargo&previous-version=1.1.2&new-version=1.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-17 16:44:54 +08:00
Stefan Holderbach
055aae9f5c
Impl FusedIterator for record iterators (#12542)
This is good practice as all our iterators will never return a value
after reaching `None`

The benefit should be minimal as only `Iterator::fuse` is directly
specialized and itself rarely used (sometimes in `itertools` adaptors)

Thus it is mostly a documentation thing
2024-04-17 00:34:16 +02:00
Stefan Holderbach
a67dad3d15
Minor housekeeping in the parser (#12540)
- **Move lone `check_name` to the alias place**
- **Restrict visibility of `check_call` helper**
2024-04-17 00:33:50 +02:00