Commit Graph

431 Commits

Author SHA1 Message Date
Ian Manske
a8890d5cca
Fix potential panic in ls (#14655)
# Description

Fixes a potential panic in `ls`.

# User-Facing Changes

Entries in the same directory are sorted first based on whether or not
they errored. Errors will be listed first, potentially stopping the
pipeline short.
2024-12-21 15:09:46 -08:00
Wind
e26364f885
Remove -a/-all flag in du. (#14618)
Just noticed that I forget to remove `-a/-all` flag in `du`'s signature
in #14407

This pr is going to remove it
2024-12-18 10:45:54 -06:00
Darren Schroeder
e2c4ff8180
Revert "Feature: PWD-per-drive to facilitate working on multiple drives at Windows" (#14598)
Reverts nushell/nushell#14411
2024-12-16 13:52:07 -06:00
Wind
dff6268d66
du: add -l/--long flag, remove -a/--all flag (#14407)
# Description
Closes:  #14387 
~To make it happen, just need to added `-l` flag to `du`, and pass it to
`DirBuilder`, `DirInfo`, `FileInfo`
Then tweak `impl From<DirInfo> for Value` and `impl From<FileInfo> for
Value` impl.~

---

Edit: this PR is going to:
1. Exclude directories and files columns by default
2. Added `-l/--long` flag to output directories and files columns
3. When running `du`, it will output the files as well. Previously it
doesn't output the size of file.

To make it happen, just need to added `-r` flag to `du`, and pass it to
`DirBuilder`, `DirInfo`, `FileInfo`
Then tweak `impl From<DirInfo> for Value` and `impl From<FileInfo> for
Value` impl.

And rename some variables.

# User-Facing Changes
`du` is no longer output `directories` and `file` columns by default,
added `-r` flag will show `directories` column, `-f` flag will show
`files` column.

```nushell
> du nushell
╭───┬────────────────────────────────────┬──────────┬──────────╮
│ # │                path                │ apparent │ physical │
├───┼────────────────────────────────────┼──────────┼──────────┤
│ 0 │ /home/windsoilder/projects/nushell │ 34.6 GiB │ 34.7 GiB │
├───┼────────────────────────────────────┼──────────┼──────────┤
│ # │                path                │ apparent │ physical │
╰───┴────────────────────────────────────┴──────────┴──────────╯
> du nushell --recursive --files # It outputs two more columns, `directories` and `files`, but the output is too long to paste here.
```
# Tests + Formatting
Added 1 test

# After Submitting
NaN
2024-12-10 11:22:56 -06:00
Darren Schroeder
c16f49cf19
add coreutils to search terms 2024-12-07 07:20:46 -06:00
132ikl
424efdaafe
Make glob stream (#14495)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Makes the `glob` command stream

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

The glob command now streams

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
N/A
2024-12-03 15:21:09 -06:00
PegasusPlusUS
c8b5909ee8
Feature: PWD-per-drive to facilitate working on multiple drives at Windows (#14411)
This PR implements PWD-per-drive as described in discussion #14355

# Description
On Windows, CMD or PowerShell assigns each drive its own current
directory. For example, if you are in 'C:\Windows', switch to 'D:', and
navigate to 'D:\Game', you can return to 'C:\Windows' by simply typing
'C:'.

This PR enables Nushell on Windows to have the same capability, allowing
each drive to maintain its own PWD (Present Working Directory).

# User-Facing Changes
Currently, 'cd' or 'ls' only accept absolute paths if the path starts
with 'C:' or another drive letter. With PWD-per-drive, users can use
'cd' (or auto cd) and 'ls' in the same way as 'cd' and 'dir' in
PowerShell, or similarly to 'cd' and 'dir' in CMD (noting that cd in CMD
has slightly different behavior, 'cd' for another drive only changes
current directory of that drive, but does not switch there).

Interaction example on switching between drives:
```Nushell
~>D:
D:\>cd Test
D:\Test\>C:
~>D:
D:\Test\>C:
~>cd D:..
D:\>C:x/../y/../z/..
~>cd D:Test\Test
D:\Test\Test>C:
~>D:...
D:\>
```
Interaction example on auto-completion at cmd line:
```Nushell
~>cd D:\test[Enter]
D:\test>~[Enter]
~>D:[TAB]
~>D:\test[Enter]
D:\test>c:.c[TAB]
c:\users\nushell\.cargo\ c:\users\nushell\.config\
```
Interaction example on pass PWD-per-drive to child process: (Note CMD
will use it, but PowerShell will ignore it though it still prepares such
info for child process)
```Nushell
~>cd D:\Test
D:\Test>cd E:\Test
E:\Test\>~
~>CMD
Microsoft Windows [Version 10.0.22631.4460]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Nushell>d:
D:\Test>e:
E:\Test>
```

# Brief Change Description
 
1.Added 'crates/nu-path/src/pwd_per_drive.rs' to implement a 26-slot
array mapping drive letters to PWDs. Test cases are included in the same
file, along with a doctest for the usage of PWD-per-drive.
2. Modified 'crates/nu-path/src/lib.rs' to declare module of
pwd_per_drive and export struct for PWD-per-drive.
3. Modified 'crates/nu-protocol/src/engine/stack.rs' to sync PWD when
set_cwd() is called. Add PWD-per-drive map as member. Clone between
parent and child. Stub/proxy for nu_path::expand_path_with() to
facilitate filesystem commands using PWD-per-drive.
4. Modified 'crates/nu-cli/src/repl.rs' auto_cd uses PWD-per-drive to
expand path.
5. Modified 'crates/nu-cli/src/completions/completion_common.rs' to
expand relative path when press [TAB] at command line.
6. Modified 'crates/nu-engine/src/env.rs' to collect PWD-per-drive info
as env vars for child process as CMD or PowerShell do, this can let
child process inherit PWD-per-drive info.
7. Modified 'crates/nu-engine/src/eval.rs', caller clone callee's
PWD-per-drive info, supporting 'def --env'
8. Modified 'crates/nu-engine/src/eval_ir.rs', 'def --env' support.
Remove duplicated fn redirect_env()
9. Modified 'src/run.rs', to init PWD-per-drive when startup.

filesystem commands that modified:
1. Modified 'crates/nu-command/src/filesystem/cd.rs', 1 line change to
use stackscoped PWD-per-drive.
Other commands, commit pending....

Local test def --env OK:
```nushell
E:\study\nushell> def --env env_cd_demo [] {                 
:::     cd ~
:::     cd D:\Project
:::     cd E:Crates
::: }
E:\study\nushell>                                                   
E:\study\nushell> def cd_no_demo [] {                   
:::     cd ~
:::     cd D:\Project
:::     cd E:Crates
::: }
E:\study\nushell> cd_no_demo                                 
E:\study\nushell> C:
C:\>D:
D:\>E:                                     
E:\study\nushell>env_cd_demo
E:\study\nushell\crates> C:
~>D:
D:\Project>E:                                     
E:\study\nushell\crates>     
```

# Tests + Formatting

- `cargo fmt --all -- --check` passed.
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
passed.
- `cargo test --workspace` passed on Windows developer mode and Ubuntu.
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` passed.
- nushell:
```
> use toolkit.nu  # or use an `env_change` hook to activate it automatically
> toolkit check pr
> ```
passed

---------

Co-authored-by: pegasus.cadence@gmail.com <pegasus.cadence@gmail.com>
2024-12-02 12:17:46 -06:00
Ian Manske
bcd85b6f3e
Remove duplicate implementations of CallExt::rest (#14484)
# Description

Removes unnecessary usages of `Call::rest_iter_flattened` and
`get_rest_for_glob_pattern` and replaces them with `CallExt::rest`.

# User-Facing Changes

None
2024-12-01 15:03:45 +01:00
Piepmatz
3d5f853b03
Start to Add WASM Support Again (#14418)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The [nushell/demo](https://github.com/nushell/demo) project successfully
demonstrated running Nushell in the browser using WASM. However, the
current version of Nushell cannot be easily built for the
`wasm32-unknown-unknown` target, the default for `wasm-bindgen`.

This PR introduces initial support for the `wasm32-unknown-unknown`
target by disabling OS-dependent features such as filesystem access, IO,
and platform/system-specific functionality. This separation is achieved
using a new `os` feature in the following crates:

 - `nu-cmd-lang`
 - `nu-command`
 - `nu-engine`
 - `nu-protocol`

The `os` feature includes all functionality that interacts with an
operating system. It is enabled by default, but can be disabled using
`--no-default-features`. All crates that depend on these core crates now
use `--no-default-features` to allow compilation for WASM.

To demonstrate compatibility, the following script builds all crates
expected to work with WASM. Direct user interaction, running external
commands, working with plugins, and features requiring `openssl` are out
of scope for now due to their complexity or reliance on C libraries,
which are difficult to compile and link in a WASM environment.

```nushell
[ # compatible crates
	"nu-cmd-base",
	"nu-cmd-extra",
	"nu-cmd-lang",
	"nu-color-config",
	"nu-command",
	"nu-derive-value",
	"nu-engine",
	"nu-glob",
	"nu-json",
	"nu-parser",
	"nu-path",
	"nu-pretty-hex",
	"nu-protocol",
	"nu-std",
	"nu-system",
	"nu-table",
	"nu-term-grid",
	"nu-utils",
	"nuon"
] | each {cargo build -p $in --target wasm32-unknown-unknown --no-default-features}
```

## Caveats
This PR has a few caveats:
1. **`miette` and `terminal-size` Dependency Issue**
`miette` depends on `terminal-size`, which uses `rustix` when the target
is not Windows. However, `rustix` requires `std::os::unix`, which is
unavailable in WASM. To address this, I opened a
[PR](https://github.com/eminence/terminal-size/pull/68) for
`terminal-size` to conditionally compile `rustix` only when the target
is Unix. For now, the `Cargo.toml` includes patches to:
    - Use my forked version of `terminal-size`.
- ~~Use an unreleased version of `miette` that depends on
`terminal-size@0.4`.~~

These patches are temporary and can be removed once the upstream changes
are merged and released.

2. **Test Output Adjustments**
Due to the slight bump in the `miette` version, one test required
adjustments to accommodate minor formatting changes in the error output,
such as shifted newlines.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This shouldn't break anything but allows using some crates for targeting
`wasm32-unknown-unknown` to revive the demo page eventually.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

I did not add any extra tests, I just checked that compiling works, also
when using the host target but unselecting the `os` feature.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
~~Breaking the wasm support can be easily done by adding some `use`s or
by adding a new dependency, we should definitely add some CI that also
at least builds against wasm to make sure that building for it keep
working.~~
I added a job to build wasm.

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-11-30 07:57:11 -06:00
132ikl
d69e131450
Rely on display_output hook for formatting values from evaluations (#14361)
# Description

I was reading through the documentation yesterday, when I stumbled upon
[this
section](https://www.nushell.sh/book/pipelines.html#behind-the-scenes)
explaining how command output is formatted using the `table` command. I
was surprised that this section didn't mention the `display_output`
hook, so I took a look in the code and was shocked to discovered that
the documentation was correct, and the `table` command _is_
automatically applied to printed pipelines.

This auto-tabling has two ramifications for the `display_output` hook:

1. The `table` command is called on the output of a pipeline after the
`display_output` has run, even if `display_output` contains the table
command. This means each pipeline output is roughly equivalent to the
following (using `ls` as an example):
    ```nushell
    ls | do $config.hooks.display_output | table
    ```
2. If `display_output` returns structured data, it will _still_ be
formatted through the table command.

This PR removes the auto-table when the `display_output` hook is set.
The auto-table made sense before `display_output` was introduced, but to
me, it now seems like unnecessary "automagic" which can be accomplished
using existing Nushell features.

This means that you can now pull back the curtain a bit, and replace
your `display_output` hook with an empty closure
(`$env.config.hooks.display_output = {||}`, setting it to null retains
the previous behavior) to see the values printed normally without the
table formatting. I think this is a good thing, and makes it easier to
understand Nushell fundamentals.

It is important to note that this PR does not change how `print` and
other commands (well, specifically only `watch`) print out values. They
continue to use `table` with no arguments, so changing your
config/`display_output` hook won't affect what `print`ing a value does.

Rel: [Discord
discussion](https://discord.com/channels/601130461678272522/615329862395101194/1307102690848931904)
(cc @dcarosone)

# User-Facing Changes

Pipelines are no longer automatically formatted using the `table`
command. Instead, the `display_output` hook is used to format pipeline
output. Most users should see no impact, as the default `display_output`
hook already uses the `table` command.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting


Will update mentioned docs page to call out `display_output` hook.
2024-11-19 21:04:29 +08:00
Darren Schroeder
13ce9e4f64
update uutils crates (#14371)
# Description

This PR updates the uutils/coreutils crates to the latest version. I
hard-coded debug to false, a new uu_mv parameter. It may be interesting
to add that but I just wanted to get all the uu crates on the same
version.

I had to update the tests because --no-clobber works but doesn't say
anything when it's not clobbering and previously we were checking for an
error message.


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-17 19:31:36 -06:00
Yash Thakur
f63f8cb154
Add utouch command from uutils/coreutils (#11817)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Part of https://github.com/nushell/nushell/issues/11549

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR adds a `utouch` command that uses the `touch` command from
https://github.com/uutils/coreutils. Eventually, `utouch` may be able to
replace `touch`.

The conflicts in Cargo.lock and Cargo.toml are because I'm using the
uutils/coreutils main rather than the latest release, since the changes
that expose `uu_touch`'s internal functionality aren't available in the
latest release.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Users will have access to a new `utouch` command with the following
flags:
todo

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-17 18:03:21 -06:00
Solomon
a04c90e22d
make ls return "Permission denied" for CWD instead of empty results (#14310)
Fixes #14265

# User-Facing Changes

`ls` without a path argument now errors when the current working
directory is unreadable due to missing permissions:

```diff
mkdir foo
chmod 100 foo
cd foo
ls | to nuon
-[]
+Error:   × Permission denied
```
2024-11-15 12:09:02 +08:00
Bark
a84d410f11
Fix inconsistency in ls sort-order (#13875)
Fixes #13267 

As we can see from the bisect done in the comments.
Bisected to https://github.com/nushell/nushell/pull/12625 /
460a1c8f87

We can see that this update brought the use of `read_dir` and for it, it
is mentioned in the [rust
docs](https://doc.rust-lang.org/std/fs/fn.read_dir.html#platform-specific-behavior)
that it does **not** provide any specific order of files.
As was the advice there, I went and applied a manual `sort` to the
entries and tested it manually on my local machine.

If required I could probably try and add tests for the order
consistency, would need some time to find my way around them, so I'm
sending the PR first.
2024-11-15 07:39:41 +08:00
Doru
a935e0720f
no deref in touch (#14214)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Adds --no-deref flag to `touch`. Nice and backwards compatible, and I
get to touch symlinks. I still don't get to set their dates directly,
but maybe that'll come with utouch.

Some sadness in the implementation, since `set_symlink_file_times`
doesn't take Option values and we call it twice with the old "read"
values from reference (or now, if missing). This shouldn't be a big
concern since `touch` already did two calls if you set both mtime and
atime. Also, `--no-deref` applies both to the reference file, and to the
target file. No splitting them up, because that's silly.

Can always bikeshed. I nicked `--no-deref` from the uutils flag, and
made the short flag `-d` because it obviously can't be `-h`. I thought
of `-S` like in `glob`, for the "negative/filter out" uppercase short
letters. Ultimately I don't think it matters much.

Should fix #14212 since it's not really tied to uutils, besides the
comment about setting a `datetime` value directly.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
New flag.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Maybe.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-03 00:56:05 -04:00
Darren Schroeder
af9c31152a
Add metadata on open --raw with bytestreams (#14141)
# Description

This PR closes #14137 and allows the display hook to be set on byte
streams. So, with a hook like this below.
```nushell
display_output: {
    metadata access {|meta| match $meta.content_type? {
        "application/x-nuscript" | "application/x-nuon" | "text/x-nushell" => { nu-highlight },
        "application/json" => { ^bat --language=json --color=always --style=plain --paging=never },
        _ => {},
        }
    } | table
}
```
You could type `open toolkit.nu` and the text of toolkit.nu would be
highlighted by nu-highlight. This PR also changes the way content-type
is assigned with `open`. Previously it would only assign it if `--raw`
was specified.

Lastly, it changes the `is_external()` function to only say
`ByteStreamSource::Child`'s are external instead of both Child and
`ByteStreamSource::File`. Again, this was to allow the hook to function
properly. I'm not sure what negative ramifications changing
`is_external()` could have, but there may be some?

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-10-23 16:50:15 -05:00
132ikl
a11c9e9d70
Ratelimit save command progress bar updates (#14075)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Currently, the `save -p` command updates the progress animation each
time any data is written. This PR rate limits the animation so it
doesn't play as fast.

Here's an asciinema of [current
behavior](https://asciinema.org/a/8RWrWTozQSceqx6tYY7kzblqj) and
[proposed behavior](https://asciinema.org/a/E1pi0gMwMwFcxVHOy9Fv1Kk6R).

# User-Facing Changes

* `save -p` progress bar has a smoother animation

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
N/A
2024-10-13 07:01:03 -05:00
Darren Schroeder
e3fd4d3f81
since windows allows slash or backslash, allow both instead of MAIN_SEPARATOR (#13996)
# Description

I mean't to do this small change the other day but forgot. We probably
shouldn't be using MAIN_SEPARATOR because **\\*.rs is an illegal glob.
So, update this to just use slash.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-10-03 10:10:11 -05:00
Ian Manske
157494e803
Make get_env_var return a reference to a Value (#13987)
# Description
Title says it all, changes `EngineState::get_env_var` to return a
`Option<&'a Value>` instead of an owned `Option<Value>`. This avoids
some unnecessary clones.

I also made a similar change to the `PluginExecutionContext` trait.
2024-10-02 13:05:48 +02:00
Ian Manske
f03ba6793e
Fix non-zero exit code errors in middle of pipeline (#13899)
# Description
Fixes #13868. Should come after #13885.

# User-Facing Changes
Bug fix.

# Tests + Formatting
Added a test.
2024-10-02 06:04:18 -05:00
YizhePKU
13df0af514
Set current working directory at startup (#12953)
This PR sets the current working directory to the location of the
Nushell executable at startup, using `std::env::set_current_dir()`. This
is desirable because after PR
https://github.com/nushell/nushell/pull/12922, we no longer change our
current working directory even after `cd` is executed, and some OS might
lock the directory where Nushell started.

The location of the Nushell executable is chosen because it cannot be
removed while Nushell is running anyways, so we don't have to worry
about OS locking it.

This PR has the side effect that it breaks buggy command even harder.
I'll keep this PR as a draft until these commands are fixed, but it
might be helpful to pull this PR if you're working on fixing one of
those bugs.

---------

Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-09-25 13:04:26 -05:00
Darren Schroeder
1afff777a6
update folder_depth algorithm for glob command (#13915)
# Description

This PR updates the `folder_depth` algorithm in order to make `glob` a
bit faster. The algorithm works like this. Since `folder_depth` is
always used we need to set it to a value. If the glob pattern contains
`**` then make `folder_depth` `usize::MAX`. If `--depth` is not
provided, make it 1, otherwise use the provided value.

closes #13914

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-09-25 08:15:18 -05:00
Darren Schroeder
65bb0ff167
Add threads to the ls command in order to increase performance in some circumstances (#13836)
# Description

This PR tries to allow the `ls` command to use multiple threads if so
specified. The reason why you'd want to use threads is if you notice
`ls` taking a long time. The one place I see that happening is from WSL.

I'm not sure how real-world this test is but you can see that this
simple `ls` of a folder with length takes a while 9366 ms. I've run this
test many times and it ranges from about 15 seconds to about 10 seconds.
But with the `--threads` parameter, it takes less time, 2744ms in this
screenshot.

![image](https://github.com/user-attachments/assets/e5c4afa2-7837-4437-8e6e-5d4bc3894ae1)

The only way forward I could find was to _always_ use threading and
adjust the number of threads based on if the user provides a flag. That
seemed the easiest way to do it after applying @devyn's interleave
advice.

No feelings hurt if this doesn't land. It's more of an experiment but I
think it has potential.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-09-24 08:40:48 -05:00
Ian Manske
03ee54a4df
Fix try not working with let, etc. (#13885)
# Description
Partialy addresses #13868. `try` does not catch non-zero exit code
errors from the last command in a pipeline if the result is assigned to
a variable using `let` (or `mut`).

This was fixed by adding a new `OutDest::Value` case. This is used when
the pipeline is in a "value" position. I.e., it will be collected into a
value. This ended up replacing most of the usages of `OutDest::Capture`.
So, this PR also renames `OutDest::Capture` to `OutDest::PipeSeparate`
to better fit the few remaining use cases for it.

# User-Facing Changes
Bug fix.

# Tests + Formatting
Added two tests.
2024-09-23 06:44:25 -05:00
Ian Manske
3d008e2c4e
Error on non-zero exit statuses (#13515)
# Description
This PR makes it so that non-zero exit codes and termination by signal
are treated as a normal `ShellError`. Currently, these are silent
errors. That is, if an external command fails, then it's code block is
aborted, but the parent block can sometimes continue execution. E.g.,
see #8569 and this example:
```nushell
[1 2] | each { ^false }
```

Before this would give:
```
╭───┬──╮
│ 0 │  │
│ 1 │  │
╰───┴──╯
```

Now, this shows an error:
```
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
   ╭─[entry #1:1:2]
 1 │ [1 2] | each { ^false }
   ·  ┬
   ·  ╰── source value
   ╰────

Error: nu:🐚:non_zero_exit_code

  × External command had a non-zero exit code
   ╭─[entry #1:1:17]
 1 │ [1 2] | each { ^false }
   ·                 ──┬──
   ·                   ╰── exited with code 1
   ╰────
```

This PR fixes #12874, fixes #5960, fixes #10856, and fixes #5347. This
PR also partially addresses #10633 and #10624 (only the last command of
a pipeline is currently checked). It looks like #8569 is already fixed,
but this PR will make sure it is definitely fixed (fixes #8569).

# User-Facing Changes
- Non-zero exit codes and termination by signal now cause an error to be
thrown.
- The error record value passed to a `catch` block may now have an
`exit_code` column containing the integer exit code if the error was due
to an external command.
- Adds new config values, `display_errors.exit_code` and
`display_errors.termination_signal`, which determine whether an error
message should be printed in the respective error cases. For
non-interactive sessions, these are set to `true`, and for interactive
sessions `display_errors.exit_code` is false (via the default config).

# Tests
Added a few tests.

# After Submitting
- Update docs and book.
- Future work:
- Error if other external commands besides the last in a pipeline exit
with a non-zero exit code. Then, deprecate `do -c` since this will be
the default behavior everywhere.
- Add a better mechanism for exit codes and deprecate
`$env.LAST_EXIT_CODE` (it's buggy).
2024-09-07 06:44:26 +00:00
Ian Manske
abd230e12e
Use IntoValue in config code (#13751)
# Description

Cleans up and refactors the config code using the `IntoValue` macro.
Shoutout to @cptpiepmatz for making the macro!

# User-Facing Changes

Should be none.

# After Submitting

Somehow refactor the reverse transformation.
2024-09-05 09:44:23 +02:00
Stefan Holderbach
95b78eee25
Change the usage misnomer to "description" (#13598)
# Description
    
The meaning of the word usage is specific to describing how a command
function is *used* and not a synonym for general description. Usage can
be used to describe the SYNOPSIS or EXAMPLES sections of a man page
where the permitted argument combinations are shown or example *uses*
are given.
Let's not confuse people and call it what it is a description.

Our `help` command already creates its own *Usage* section based on the
available arguments and doesn't refer to the description with usage.

# User-Facing Changes

`help commands` and `scope commands` will now use `description` or
`extra_description`
`usage`-> `description`
`extra_usage` -> `extra_description`

Breaking change in the plugin protocol:

In the signature record communicated with the engine.
`usage`-> `description`
`extra_usage` -> `extra_description`

The same rename also takes place for the methods on
`SimplePluginCommand` and `PluginCommand`

# Tests + Formatting
- Updated plugin protocol specific changes
# After Submitting
- [ ] update plugin protocol doc
2024-08-22 12:02:08 +02:00
Bruce Weirdan
7e9d32d64e
Drop outdated touch -d example (#13631)
Fixes nushell/nushell#8723

# Description

The example was showing the flag that no longer exists.

# User-Facing Changes

Help no longer shows the example with `-d` flag.

# Tests + Formatting

I trust in CI.

# After Submitting

Nothing.
2024-08-15 23:57:25 +02:00
Stefan Holderbach
31b3104af7
Remove code duplication in glob (#13626) 2024-08-14 23:11:00 +02:00
Darren Schroeder
f122065772
create a better error message for bad glob patterns (#13613)
# Description

@sholderbach pointed out that I could've made this error message better.
So, here's my attempt to make it better.

This should work. I had a hard time figuring out how to trigger the
error anyway because the type checker doesn't allow "bad" parameters to
begin with.

### Before

![image](https://github.com/user-attachments/assets/ac60ce27-4b9a-49ca-910c-74422ae31bc4)


### After

![image](https://github.com/user-attachments/assets/fe939339-67df-4d30-a8dd-5ce3fe623a95)


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-13 18:40:17 -05:00
Darren Schroeder
4732507f46
allow glob to take a glob or a string as the input (#13612)
# Description

This PR changes glob to take either a string or a glob as a parameter.

Closes #13611

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-13 14:27:42 -05:00
Wind
a55d172e52
make ls -lf outputs full path in symbolic target (#13605)
# Description
Fixes: #13479 

# User-Facing Changes
Given the following setup:
```
cd /tmp
touch src_file.txt
ln -s src_file.txt link1
```
### Before 
```
ls -lf link1 | get target.0  # It outputs src_file.txt
```
### After
```
ls -lf link1 | get target.0  # It outputs /tmp/src_file.txt
```
# Tests + Formatting
Added a test for the change
2024-08-13 06:34:10 -05:00
Jack Wright
73e8de9753
Attempt to guess the content type of a file when opening with --raw (#13521)
# Description
Attempt to guess the content type of a file when opening with --raw and
set it in the pipeline metadata.

<img width="644" alt="Screenshot 2024-08-02 at 11 30 10"
src="https://github.com/user-attachments/assets/071f0967-c4dd-405a-b8c8-f7aa073efa98">


# User-Facing Changes
- Content of files can be directly piped into commands like `http post`
with the content type set appropriately when using `--raw`.
2024-08-06 11:36:24 +02:00
NotTheDr01ds
ca8eb856e8
Doc and examples for multi-dot directory traversal (#13513)
# Description

With this PR, we should be able to close
https://github.com/nushell/nushell.github.io/issues/1225

Help/doc/examples updated for:

* `cd` to show multi-dot traversal
* `cd` to show implicit `cd` with bare directory path
* Fixed/clarified another example that mentioned `$OLDPATH` while I was
in there
* `mv` and `cp` examples for multi-dot traversal
* Updated `cp` examples to use more consistent (and clear) filenames

# User-Facing Changes

Help/doc only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-08-01 15:22:25 -05:00
Stefan Holderbach
2c6b1471e1
Contentious clippy fixes (#13498)
Lints from stable or nightly toolchain that may have questionable added
value.

- **Contentious lint to contract into single `if let`**
- **Potential false positive around `AsRef`/`Deref` fun**
2024-08-01 11:02:55 +02:00
Ian Manske
ae5fed41ed
Path migration part 3: $nu paths (#13368)
# Description
Part 3 of replacing `std::path` types with `nu_path` types added in
#13115. This PR targets the paths listed in `$nu`. That is, the home,
config, data, and cache directories.
2024-08-01 10:16:31 +02:00
Embers-of-the-Fire
ca73d85c09
Add --upgrade switch for mv command (#13505)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Add `--upgrade, -u` switch for `mv` command, corresponding to `cp`.

Closes #13458.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
```plain
❯ help mv | find update
╭──────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│        0 │   -u, --update - move and overwite only when the SOURCE file is newer than the destination file or when the destination file is missing       │
╰──────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:
-->
- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used`
to check that you're using the standard code style
- [x] `cargo test --workspace` to check that all tests pass (on Windows
make sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- [x] `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
<!--
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

P.S.  
The standard test kit (`nu-test-support`) doesn't provide utility to
create file with modification timestamp, and I didn't find any test for
this in `cp` command. I had tested on my local machine but I'm not sure
how to integrate it into ci. If unit testing is required, I may need
your guidance.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
- [x] Command docs are auto generated.
2024-07-31 20:43:06 -05:00
Wind
928c57db41
save: print to stderr for bytestream (#13422)
# Description
Fixes: #13260 

When user run a command like this:
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```

`save` command sinks the output of previous commands' stderr output. I
think it should be `stderr`.

# User-Facing Changes
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```
The command will output ` New Err` to stderr.

# Tests + Formatting
Added 2 cases.
2024-07-31 18:19:35 +02:00
Zoey Hewll
5db57abc7d
add --quiet flag to watch command (#13415)
# Description
Adds a `--quiet` flag to the `watch` command, silencing the message
usually shown when invoking `watch`.

Resolves #13411

As implemented, `--quiet` is orthogonal to `--verbose`. I'm open to
improving the flag's name, behaviour and/or documentation to make it
more user-friendly, as I realise this may be surprising as-is.

```
> watch path {|a b c| echo $a $b $c}
Now watching files at "/home/user/path". Press ctrl+c to abort.
───┬───────────────────────
 0 │ Remove                
 1 │ /home/user/path 
 2 │                       
───┴───────────────────────
^C
> watch --quiet path {|a b c| echo $a $b $c}
───┬───────────────────────
 0 │ Remove                
 1 │ /home/user/path 
 2 │                       
───┴───────────────────────
^C
```

# User-Facing Changes

Adds `--quiet`/`-q` flag to `watch`, which removes the initial status
message.
2024-07-20 12:34:27 +02:00
Devyn Cairns
f65bc97a54
Update config directly at assignment (#13332)
# Description

Allows `Stack` to have a modified local `Config`, which is updated
immediately when `$env.config` is assigned to. This means that even
within a script, commands that come after `$env.config` changes will
always see those changes in `Stack::get_config()`.

Also fixed a lot of cases where `engine_state.get_config()` was used
even when `Stack` was available.

Closes #13324.

# User-Facing Changes
- Config changes apply immediately after the assignment is executed,
rather than whenever config is read by a command that needs it.
- Potentially slower performance when executing a lot of lines that
change `$env.config` one after another. Recommended to get `$env.config`
into a `mut` variable first and do modifications, then assign it back.
- Much faster performance when executing a script that made
modifications to `$env.config`, as the changes are only parsed once.

# Tests + Formatting
All passing.

# After Submitting
- [ ] release notes
2024-07-11 06:09:33 -07:00
Devyn Cairns
d7392f1f3b
Internal representation (IR) compiler and evaluator (#13330)
# Description

This PR adds an internal representation language to Nushell, offering an
alternative evaluator based on simple instructions, stream-containing
registers, and indexed control flow. The number of registers required is
determined statically at compile-time, and the fixed size required is
allocated upon entering the block.

Each instruction is associated with a span, which makes going backwards
from IR instructions to source code very easy.

Motivations for IR:

1. **Performance.** By simplifying the evaluation path and making it
more cache-friendly and branch predictor-friendly, code that does a lot
of computation in Nushell itself can be sped up a decent bit. Because
the IR is fairly easy to reason about, we can also implement
optimization passes in the future to eliminate and simplify code.
2. **Correctness.** The instructions mostly have very simple and
easily-specified behavior, so hopefully engine changes are a little bit
easier to reason about, and they can be specified in a more formal way
at some point. I have made an effort to document each of the
instructions in the docs for the enum itself in a reasonably specific
way. Some of the errors that would have happened during evaluation
before are now moved to the compilation step instead, because they don't
make sense to check during evaluation.
3. **As an intermediate target.** This is a good step for us to bring
the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at
some point, as code generated from new AST can be directly compared to
code generated from old AST. If the IR code is functionally equivalent,
it will behave the exact same way.
4. **Debugging.** With a little bit more work, we can probably give
control over advancing the virtual machine that `IrBlock`s run on to
some sort of external driver, making things like breakpoints and single
stepping possible. Tools like `view ir` and [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than
before to see what exactly is going on with your Nushell code.

The goal is to eventually replace the AST evaluator entirely, once we're
sure it's working just as well. You can help dogfood this by running
Nushell with `$env.NU_USE_IR` set to some value. The environment
variable is checked when Nushell starts, so config runs with IR, or it
can also be set on a line at the REPL to change it dynamically. It is
also checked when running `do` in case within a script you want to just
run a specific piece of code with or without IR.

# Example

```nushell
view ir { |data|
  mut sum = 0
  for n in $data {
    $sum += $n
  }
  $sum
}
```
  
```gas
# 3 registers, 19 instructions, 0 bytes of data
   0: load-literal           %0, int(0)
   1: store-variable         var 904, %0 # let
   2: drain                  %0
   3: drop                   %0
   4: load-variable          %1, var 903
   5: iterate                %0, %1, end 15 # for, label(1), from(14:)
   6: store-variable         var 905, %0
   7: load-variable          %0, var 904
   8: load-variable          %2, var 905
   9: binary-op              %0, Math(Plus), %2
  10: span                   %0
  11: store-variable         var 904, %0
  12: load-literal           %0, nothing
  13: drain                  %0
  14: jump                   5
  15: drop                   %0          # label(0), from(5:)
  16: drain                  %0
  17: load-variable          %0, var 904
  18: return                 %0
```

# Benchmarks

All benchmarks run on a base model Mac Mini M1.

## Iterative Fibonacci sequence

This is about as best case as possible, making use of the much faster
control flow. Most code will not experience a speed improvement nearly
this large.

```nushell
def fib [n: int] {
  mut a = 0
  mut b = 1
  for _ in 2..=$n {
    let c = $a + $b
    $a = $b
    $b = $c
  }
  $b
}
use std bench
bench { 0..50 | each { |n| fib $n } }
```

IR disabled:

```
╭───────┬─────────────────╮
│ mean  │ 1ms 924µs 665ns │
│ min   │ 1ms 700µs 83ns  │
│ max   │ 3ms 450µs 125ns │
│ std   │ 395µs 759ns     │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```

IR enabled:

```
╭───────┬─────────────────╮
│ mean  │ 452µs 820ns     │
│ min   │ 427µs 417ns     │
│ max   │ 540µs 167ns     │
│ std   │ 17µs 158ns      │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```

![explore ir
view](https://github.com/nushell/nushell/assets/10729/d7bccc03-5222-461c-9200-0dce71b83b83)

##
[gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu)

IR disabled:

```
╭───┬──────────────────╮
│ 0 │ 27ms 929µs 958ns │
│ 1 │ 21ms 153µs 459ns │
│ 2 │ 18ms 639µs 666ns │
│ 3 │ 19ms 554µs 583ns │
│ 4 │ 13ms 383µs 375ns │
│ 5 │ 11ms 328µs 208ns │
│ 6 │  5ms 659µs 542ns │
╰───┴──────────────────╯
```

IR enabled:

```
╭───┬──────────────────╮
│ 0 │       22ms 662µs │
│ 1 │ 17ms 221µs 792ns │
│ 2 │ 14ms 786µs 708ns │
│ 3 │ 13ms 876µs 834ns │
│ 4 │  13ms 52µs 875ns │
│ 5 │ 11ms 269µs 666ns │
│ 6 │  6ms 942µs 500ns │
╰───┴──────────────────╯
```

##
[random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu)

I got pretty random results out of this benchmark so I decided not to
include it. Not clear why.

# User-Facing Changes
- IR compilation errors may appear even if the user isn't evaluating
with IR.
- IR evaluation can be enabled by setting the `NU_USE_IR` environment
variable to any value.
- New command `view ir` pretty-prints the IR for a block, and `view ir
--json` can be piped into an external tool like [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir).

# Tests + Formatting
All tests are passing with `NU_USE_IR=1`, and I've added some more eval
tests to compare the results for some very core operations. I will
probably want to add some more so we don't have to always check
`NU_USE_IR=1 toolkit test --workspace` on a regular basis.

# After Submitting
- [ ] release notes
- [ ] further documentation of instructions?
- [ ] post-release: publish `nu_plugin_explore_ir`
2024-07-10 17:33:59 -07:00
Ian Manske
e98b2ceb8c
Path migration 1 (#13309)
# Description
Part 1 of replacing `std::path` types with `nu_path` types added in
#13115.
2024-07-09 17:25:23 +08:00
Ian Manske
399a7c8836
Add and use new Signals struct (#13314)
# Description
This PR introduces a new `Signals` struct to replace our adhoc passing
around of `ctrlc: Option<Arc<AtomicBool>>`. Doing so has a few benefits:
- We can better enforce when/where resetting or triggering an interrupt
is allowed.
- Consolidates `nu_utils::ctrl_c::was_pressed` and other ad-hoc
re-implementations into a single place: `Signals::check`.
- This allows us to add other types of signals later if we want. E.g.,
exiting or suspension.
- Similarly, we can more easily change the underlying implementation if
we need to in the future.
- Places that used to have a `ctrlc` of `None` now use
`Signals::empty()`, so we can double check these usages for correctness
in the future.
2024-07-07 22:29:01 +00:00
Jakub Žádník
3fae77209a
Revert "Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)" (#13292)
This reverts commit 0cfd5fbece.

The original PR messed up syntax higlighting of aliases and causes
panics of completion in the presence of alias.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-04 00:02:13 +03:00
Darren Schroeder
ba1d900020
create a better error message when saving fails (#13290)
# Description

This PR just creates a better error message when the `save` command
fails.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 07:43:07 -05:00
Jakub Žádník
0cfd5fbece
Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Part of https://github.com/nushell/nushell/issues/12963, step 2.

This PR refactors Call and related argument structures to remove their
dependency on `Expression::span` which will be removed in the future.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Should be none. If you see some error messages that look broken, please
report.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 09:00:52 +03:00
Jack Wright
0d060aeae8
Use pipeline data for http post|put|patch|delete commands. (#13254)
# Description
Provides the ability to use http commands as part of a pipeline.
Additionally, this pull requests extends the pipeline metadata to add a
content_type field. The content_type metadata field allows commands such
as `to json` to set the metadata in the pipeline allowing the http
commands to use it when making requests.

This pull request also introduces the ability to directly stream http
requests from streaming pipelines.

One other small change is that Content-Type will always be set if it is
passed in to the http commands, either indirectly or throw the content
type flag. Previously it was not preserved with requests that were not
of type json or form data.

# User-Facing Changes
* `http post`, `http put`, `http patch`, `http delete` can be used as
part of a pipeline
* `to text`, `to json`, `from json` all set the content_type metadata
field and the http commands will utilize them when making requests.
2024-07-01 12:34:19 -07:00
NotTheDr01ds
58e8ea6084
Update and add ls examples (#13222)
# Description

Based on #13219, added several examples to `ls` doc to demonstrate
recursive directory listings. List of changes in this PR:

* Add example for `ls **/*` to demonstrate recursive listing using glob
pattern
* Add example for `ls ...(glob )`... to demonstrate recursive listing
using glob command
* Remove `-s` from an example where it had no use (since it was based on
the current directory and was not recursive)
* Update the description of `ls -a ~ `... to clarify that it lists the
full path of directories
* Update the description of `ls -as ~ `... (the difference being the
`-s`) to clarify that it lists only the filenames, not paths.


# User-Facing Changes

Help only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-06-26 17:49:52 -05:00
NotTheDr01ds
d2121a155e
Fixes #13093 - Erroneous example in 'touch' help (#13095)
# Description

Fixes #13093 by:

* Removing the mentioned help example
* Updating the `--accessed` and `--modified` flag descriptions to remove
mention of "timestamp/date"

# User-Facing Changes

Help changes

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-06-07 09:33:48 -05:00
NotTheDr01ds
36ad7f15c4
cd/def --env examples (#13068)
# Description

Per a Discord question
(https://discord.com/channels/601130461678272522/1244293194603167845/1247794228696711198),
this adds examples to the `help` for both:

* `cd`
* `def`

to demonstrate that `def --env` is required when changing directories in
a custom command.

Since the existing examples for `def` were a bit more complex (and had
output) but the `cd` ones were more simplified, I did use slightly
different examples in each. Either or both could be tweaked if desired.

# User-Facing Changes

Command `help` examples

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-06-05 21:35:31 +03:00