Commit Graph

9643 Commits

Author SHA1 Message Date
Darren Schroeder
1940b36e07
Add environment variables for sourced files (#14486)
# Description

I always wondered why the module env vars `CURRENT_FILE`, `FILE_PWD`,
`PROCESS_PATH` weren't available in the source command. I tried to add
them here. I think it could be helpful but I'm not sure. I'm also not
sure this hack is what we should do but I thought I'd put it out there
for fun.

Thoughts?

### Run Module (works as it did before)

```nushell
❯ open test_module.nu
def main [] {
  print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
  print $"$env.FILE_PWD = ($env.FILE_PWD?)"
  print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"
}
❯ nu test_module.nu
$env.CURRENT_FILE = /Users/fdncred/src/nushell/test_module.nu
$env.FILE_PWD = /Users/fdncred/src/nushell
$env.PROCESS_PATH = test_module.nu
```
### Use Module (works as it did before)
```nushell
❯ open test_module2.nu
export-env {
  print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
  print $"$env.FILE_PWD = ($env.FILE_PWD?)"
  print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"
}
❯ use test_module2.nu
$env.CURRENT_FILE = /Users/fdncred/src/nushell/test_module.nu
$env.FILE_PWD = /Users/fdncred/src/nushell
$env.PROCESS_PATH =
```
### Sourced non-module script (this is the new part)

> [!NOTE] 
> Note: We intentionally left out PROCESS_PATH since it's supposed to
> to work like argv[0] in C, which is the name of the program being
executed.
> Since we're not executing a program, we don't need to set it.


```nushell
❯ open test_source.nu
print $"$env.CURRENT_FILE = ($env.CURRENT_FILE?)"
print $"$env.FILE_PWD = ($env.FILE_PWD?)"
print $"$env.PROCESS_PATH = ($env.PROCESS_PATH?)"
❯ source test_source.nu
$env.CURRENT_FILE = /Users/fdncred/src/nushell/test_source.nu
$env.FILE_PWD = /Users/fdncred/src/nushell
$env.PROCESS_PATH = 
```

Also, what is PROCESS_PATH even supposed to be?

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-12-02 06:19:20 -06:00
Bahex
dfec687a46
term query: refactor, add --beginning flag (#14446)
# Description

- Refactor code to be simpler.
- Make the mentioned changes.
- `scopeguard` is added as a direct dependency. Helps simplify the code.
Rather than roll an ad-hoc version of it myself, I thought it would be
better to use `scopeguard` as it was already an indirect dependency.

# User-Facing Changes

- Add `--beginning` flag, which is used to validate the response and
provide early errors in case of unexpected inputs.
- Both `terminator` and `beginning` sequences (when provided) are not
included in the command's output. Turns out they are almost always
removed from the output, and because they are known beforehand they can
be added back by the user.
2024-12-01 20:02:48 -06:00
Ian Manske
bcd85b6f3e
Remove duplicate implementations of CallExt::rest (#14484)
# Description

Removes unnecessary usages of `Call::rest_iter_flattened` and
`get_rest_for_glob_pattern` and replaces them with `CallExt::rest`.

# User-Facing Changes

None
2024-12-01 15:03:45 +01:00
Jasha Sommer-Simpson
c4b919b24c
enable test_cp_recurse on macos (#14358)
# Description

This PR enables some tests that were disabled on macos.

We shall see if the CI passes. (Update: CI has passed.)

# User-Facing Changes

Should be no user-facing changes as only a test-file is modified.

# Tests + Formatting

Test coverage should increase

Co-authored-by: Jasha <jsimpson@hiddenroad.com>
2024-12-01 05:59:40 -06:00
Ian Manske
c560bac13f
Add --long flag for sys cpu (#14485)
# Description

Fixes #14470 where the `sys cpu` command is slow. This was done by
removing the `cpu_usage` column from the default output, since it takes
400ms to calculate. Instead a `--long` flag was added that, when
provided, adds back the `cpu_usage` column.

```nu
# Before
> bench { sys cpu | length } | get mean
401ms 591µs 896ns

# After
> bench { sys cpu | length } | get mean
500µs 13ns # around 1-2ms in practice
```

# User-Facing Changes

- `sys cpu` no longer has a `cpu_usage` column by default.
- Added  a `--long` flag for `sys cpu` to add back the removed column.
2024-12-01 05:56:42 -06:00
paulie4
88d27fd607
explore: add more less key bindings and add Transition::None (#14468)
# Description
The `explore` command is `less`-like, but it's missing the `Emacs`
keybindings for up/down and PageUp/PageDown as well as the "q" to quit
out. When I looked into adding those additional keybindings, I noticed
there was a lot of duplicated code in the various views, so I refactored
the code into a new `trait CursorMoveHandler`. I also noticed that there
was an existing `TODO: should we add a noop transition instead of doing
Option<Transition> everywhere?` comment in the code. I went ahead and
implemented a new `Transition::None`, and that made the new `trait
CursorMoveHandler` code MUCH cleaner, in addition to making some of the
old code a little cleaner as well.

# User-Facing Changes
Users that are used to the keybindings for `less` should feel much more
comfortable using `explore`.

# Tests + Formatting
Unfortunately, there aren't any existing tests for the `explore`
command, so I didn't know where I should add new tests to cover my code
changes.

---------

Co-authored-by: paulie4 <203125+paulie4@users.noreply.github.com>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-11-30 08:22:52 -06:00
Piepmatz
3d5f853b03
Start to Add WASM Support Again (#14418)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The [nushell/demo](https://github.com/nushell/demo) project successfully
demonstrated running Nushell in the browser using WASM. However, the
current version of Nushell cannot be easily built for the
`wasm32-unknown-unknown` target, the default for `wasm-bindgen`.

This PR introduces initial support for the `wasm32-unknown-unknown`
target by disabling OS-dependent features such as filesystem access, IO,
and platform/system-specific functionality. This separation is achieved
using a new `os` feature in the following crates:

 - `nu-cmd-lang`
 - `nu-command`
 - `nu-engine`
 - `nu-protocol`

The `os` feature includes all functionality that interacts with an
operating system. It is enabled by default, but can be disabled using
`--no-default-features`. All crates that depend on these core crates now
use `--no-default-features` to allow compilation for WASM.

To demonstrate compatibility, the following script builds all crates
expected to work with WASM. Direct user interaction, running external
commands, working with plugins, and features requiring `openssl` are out
of scope for now due to their complexity or reliance on C libraries,
which are difficult to compile and link in a WASM environment.

```nushell
[ # compatible crates
	"nu-cmd-base",
	"nu-cmd-extra",
	"nu-cmd-lang",
	"nu-color-config",
	"nu-command",
	"nu-derive-value",
	"nu-engine",
	"nu-glob",
	"nu-json",
	"nu-parser",
	"nu-path",
	"nu-pretty-hex",
	"nu-protocol",
	"nu-std",
	"nu-system",
	"nu-table",
	"nu-term-grid",
	"nu-utils",
	"nuon"
] | each {cargo build -p $in --target wasm32-unknown-unknown --no-default-features}
```

## Caveats
This PR has a few caveats:
1. **`miette` and `terminal-size` Dependency Issue**
`miette` depends on `terminal-size`, which uses `rustix` when the target
is not Windows. However, `rustix` requires `std::os::unix`, which is
unavailable in WASM. To address this, I opened a
[PR](https://github.com/eminence/terminal-size/pull/68) for
`terminal-size` to conditionally compile `rustix` only when the target
is Unix. For now, the `Cargo.toml` includes patches to:
    - Use my forked version of `terminal-size`.
- ~~Use an unreleased version of `miette` that depends on
`terminal-size@0.4`.~~

These patches are temporary and can be removed once the upstream changes
are merged and released.

2. **Test Output Adjustments**
Due to the slight bump in the `miette` version, one test required
adjustments to accommodate minor formatting changes in the error output,
such as shifted newlines.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This shouldn't break anything but allows using some crates for targeting
`wasm32-unknown-unknown` to revive the demo page eventually.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

I did not add any extra tests, I just checked that compiling works, also
when using the host target but unselecting the `os` feature.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
~~Breaking the wasm support can be easily done by adding some `use`s or
by adding a new dependency, we should definitely add some CI that also
at least builds against wasm to make sure that building for it keep
working.~~
I added a job to build wasm.

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-11-30 07:57:11 -06:00
Yash Thakur
07a37f9b47
fix: Respect sort in custom completions (#14424)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description

This PR makes it so that when a custom completer sets `options.sort` to
true, completions aren't sorted. Previously, in #13311, I'd made it so
that setting `sort` to true would sort in alphabetical order, while
omitting it or setting it to false would sort it in the default order
for the chosen match algorithm (alphabetical for prefix matching, fuzzy
match score for fuzzy matching). I'd assumed that you'd always want to
sort completions and the important thing was choosing alphabetical
sorting vs the default sort order for your match algorithm. However,
this assumption was incorrect (see #13696 and [this
thread](https://discord.com/channels/601130461678272522/1302332259227144294)
in Discord).

An alternative would be to make `sort` accept `"alphabetical"`,
`"smart"`, and `"none"`/`null` rather than keeping it a boolean. But
that would be a breaking change and require more discussion, and I
wanted to keep this PR simple/small so that we can go back to the
sensible behavior as soon as possible.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Here are the different scenarios:
- If your custom completer returns a record with an `options` field
that's a record:
- If `options` contains `sort: true`, completions **will be sorted
according to the order set in the user's config**. Previously, they
would have been sorted in alphabetical order. This does mean that
**custom completers cannot explicitly choose to sort in alphabetical
order** anymore. I think that's an acceptable trade-off, though.
- If `options` contains `sort: false`, completions will not be sorted.
#13311 broke things so they would be sorted in the default order for the
match algorithm used. Before that PR, completions would not have been
sorted.
- If there's no `sort` option, that **will be treated as `sort: true`**.
Previously, this would have been treated as `sort: false`.
- Otherwise, nothing changes. Completions will still be sorted.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added 1 test to make sure that completions aren't sorted with `sort:
false` explicitly set.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-29 20:47:57 -05:00
Jack Wright
0172ad8461
Upgrading to polars 0.44 (#14478)
Upgrading to polars 0.44
2024-11-29 19:39:07 -06:00
132ikl
e1f74a6d57
Add label rendering to try/catch rendered errors (#14477)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->



# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Before this PR, you can access rendered error values that are raised in
a `try/catch` block by accessing the `rendered` element of the catch
error value:
```
$ try { ls nonexist.txt } catch {|e| print "my cool error:" $e.rendered }
my cool error:
nu:🐚:directory_not_found

  × Directory not found
  help: /home/rose/nonexist.txt does not exist
```

However, the rendered errors don't include the labels present in the
real rendered error, which would look like this:
```
$ ls nonexist.txt
Error: nu:🐚:directory_not_found

  × Directory not found
   ╭─[entry #46:1:4]
 1 │ ls nonexist.txt
   ·    ──────┬─────
   ·          ╰── directory not found
   ╰────
  help: /home/rose/nonexist.txt does not exist
```

After this PR, the rendered error includes the labels:

```
$ try { ls nonexist.txt } catch {|e| print "my cool error:" $e.rendered }
my cool error:
Error: nu:🐚:directory_not_found

  × Directory not found
   ╭─[entry #4:1:10]
 1 │ try { ls nonexist.txt } catch {|e| print "my cool error:" $e.rendered }
   ·          ──────┬─────
   ·                ╰── directory not found
   ╰────
  help: /home/rose/nonexist.txt does not exist
```

This change is accomplished by using the standard error formatting code
to render an error. This respects the error theme as before without any
extra scaffolding, but it means that e.g., the terminal size is also
respected. I think this is fine because the way the error is rendered
already changed based on config, and I think that a "rendered" error
should give back _exactly_ what would be shown to the user anyway.

@fdncred, let me know if you have any concerns with the way this is
handled since you were the one who implemented this feature in the first
place.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

The `rendered` element of the `try`/`catch` error record now includes
labels in the error output.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
N/A
2024-11-29 19:02:26 -06:00
Douglas
e17f6d654c
Deprecate date to-record and date to-table (#14319)
# Description

Implements #11234 based on the comments there:

* (Previously implemented): `into record` handles nanoseconds (as well
as milliseconds and microseconds, which the deprecated commands didn't
support).
* Added deprecation warning to `date to-record` and `date to-table`
* Added new example for `into record` showing the conversion to a table
* Changed `std/dt` to use `into record`
* Added "Deprecated" category back to nu-protocol::Signature
* Assigned the deprecated commands to the Deprecated category so be
categorized properly in the online Doc.

# User-Facing Changes

Deprecated command warning

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Searched doc for existing uses of `date to-record` and `date to-table`:

* For primary English-language docs, there are no uses other than in the
auto-generated command help, which will be updated based on this PR
* Other language translations appear to have an old use in several
places and will need to be updated to match the English-language doc.
2024-11-29 23:06:26 +01:00
Wind
817830940b
raise ParseError if assign to a non-variable or non-mutable-variable (#14405)
# Description
While reviewing #14388, I think we can make some improvement on parser.

For the following code:
```nushell
let a = 3
a = 10   # should be error
$a = 10 # another error
```
I think they can raise `ParseError`, so nushell doesn't need to move
forward compiling IR block.

# User-Facing Changes
```nushell
let a = 3
a = 10
```
Will raise parse error instead of compile error.

# Tests + Formatting
Added 1 test.
2024-11-29 23:02:21 +01:00
Renan Ribeiro
dc9e8161d9
Implement chunk_by operation (#14410)
# Description

This pull requests implements a new ~~partition-by~~ `chunk-by` command.
The operation takes a closure and partitions the input list into
sublists based on the return value of the closure.
- fixes #14149

Examples, tests and and documentation were added accordingly.


![image](https://github.com/user-attachments/assets/c272e2ec-9af3-4a88-832b-ddca4eb14c8f)


![image](https://github.com/user-attachments/assets/178968e7-c165-4d8c-858c-98584d653b0a)
2024-11-29 13:37:27 -08:00
Ian Manske
7f61cbbfd6
Add Filesize type (#14369)
# Description
Adds a new `Filesize` type so that `FromValue` can be used to convert a
`Value::Filesize` to a `Filesize`. Currently, to extract a filesize from
a `Value` using `FromValue`, you have to extract an `i64` which coerces
`Value::Int`, `Value::Duration`, and `Value::Filesize` to an `i64`.

Having a separate type also allows us to enforce checked math to catch
overflows. Similarly, it allows us to specify other trait
implementations like `Display` in a common place.

# User-Facing Changes
Multiplication with filesizes now error on overflow. Should not be a
breaking change for plugins (i.e., serialization) since `Filesize` is
marked with `serde(transparent)`.

# Tests + Formatting
Updated some tests.
2024-11-29 21:24:17 +00:00
Stefan Holderbach
acca56f77c
Remove unused FlatShapes And/Or (#14476)
# Description
This removes the need for the `shape_and` and `shape_or` entries in the
themes. We did not color those underlying FlatShapes or operators
differently.

Closes #14372
# User-Facing Changes
Our theme handling currently doesn't reject invalid entries so should
not cause an error. The non-functional nature was already documented.
2024-11-29 22:23:40 +01:00
Ian Manske
6bc695f251
Make Hooks fields non-optional to match the new config defaults (#14345)
# Description
Follow up to #14341. Changes the fields of `Hooks` to `Vec` or `Hashmap`
to match the new config defaults.

# User-Facing Changes
Mostly the same as #14341. `pre_prompt` and `pre_execution` must now be
a list, and `env_change` must be a record.
2024-11-29 21:11:09 +00:00
Darren Schroeder
91bb566ee6
udpate rust toolchain to rust 1.81.0 (#14473)
# Description

With the release of rust 1.83.0 it's time to update to rust 1.81.0.
2024-11-29 21:46:58 +01:00
132ikl
5f04bbbb8b
Make length only operate on supported input types (#14475)
# Description


Before this PR, `length` did not check its input type at run-time, so it
would attempt to calculate a length for any input with indeterminate
type (e.g., `echo` which has an `any` output type). This PR makes
`length` only work on the types specifically supported in its
input/output types (list/table, binary, and nothing), making the
behavior the same at parse-time and at run-time.

Fixes #14462

# User-Facing Changes


Length will error if passed an unsupported type:

Before (only caught at parse-time):
```nushell
"hello" | length
Error: nu::parser::input_type_mismatch

  × Command does not support string input.
   ╭─[entry #2:1:11]
 1 │ "hello" | length
   ·           ───┬──
   ·              ╰── command doesn't support string input
   ╰────

echo "hello" | length
# => 1
```

After (caught at parse-time and run-time):
```nushell
"hello" | length
Error: nu::parser::input_type_mismatch

  × Command does not support string input.
   ╭─[entry #22:1:11]
 1 │ "hello" | length
   ·           ───┬──
   ·              ╰── command doesn't support string input
   ╰────

echo "hello" | length
Error: nu:🐚:only_supports_this_input_type

  × Input type not supported.
   ╭─[entry #23:1:6]
 1 │ echo "hello" | length
   ·      ───┬───   ───┬──
   ·         │         ╰── only list, table, binary, and nothing input data is supported
   ·         ╰── input type: string
   ╰────
```
2024-11-29 21:45:27 +01:00
Dziuba Maksym
49fb5cb1a8
fix: sample_config (#14465)
path to sample_config

crates/nu-utils/src/sample_config ->
crates/nu-utils/src/default_files/sample_config.nu

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-11-29 08:06:14 -06:00
Wind
6e036ca09a
update unicode-width to 0.2 (#14456)
# Description
When looking into #14395, I found that `unicode-width` from 0.1 to 0.2
contains a breaking change, the mainly change is it treats newlines as
width 1. So relative tests(str stats) are broken.
But I think it's ok to adjust the test.

# User-Facing Changes
The output of `str stats` might change if there are `\n` in the input.
### Before
```nushell
> "a\nb" | str stats | get unicode-width
2
```
### After
```nushell
> "a\nb" | str stats | get unicode-width
3
```
# Tests + Formatting
Adjusted 2 tests.

# After Submitting
NaN
2024-11-29 09:09:45 +08:00
Douglas
8d1e36fa3c
Allow inherited environment variables (#14467)
# Description

Due to #14249 loading `default_env.nu` before the user's `env.nu`,
variables that were defined there were overriding:

* Inherited values
* Some values that were set in the Rust code, such as the `NU_LIB_PATH`
when set using `--include-path`.

This change checks to see if a variable already exists, uses its value
if so, and sets the default value otherwise.

Note: `ENV_CONVERSIONS` is still "forced" to a default value regardless,
as it needs to run reliably. There's probably not much reason to inherit
it, but I'm open to the idea if there's a use-case.

# User-Facing Changes

* Before: Variables that were set in `default_env.nu` always overrode
those that were inherited from the parent process or set internally
* After: Inherited and internal environment variables will take
priority.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Will try to find a good place to mention this behavior in the Config
chapter updates
2024-11-28 12:37:32 -06:00
Douglas
bccff3b237
Update default-files README (#14461)
# Description

Someone noticed today that I had left a TODO in the Readme. It has since
been completed and needed to be removed. Also made some other minor
fixes and wordsmithing while I was in it.

# User-Facing Changes

None

# Tests + Formatting

Clippy and fmt passed, and that should be all that matters on the
Readme.

# After Submitting

N/A
2024-11-28 15:04:42 +08:00
Wind
a13a024ac8
update miette to 7.3 (#14454)
# Description
The test is failed when updating miette from 7.2 to 7.3. After looking
into the test, I think it's ok to adjust test.

# User-Facing Changes
For the given custom command:
```nushell
def force_error [ x: any ] {
    error make {
        msg: "oh no!"
        label: {
            text: "here's the error"
            span: (metadata $x).span
        }
    }
}
```
### Before
```
> force_error "My error"
Error:   × oh no!
   ╭─[entry #8:1:13]
 1 │ force_error "My error"
   ·             ─────┬────
   ·                  ╰── here's the error
   ╰────

```

### After
```
> force_error "My error"
Error:
  × oh no!
   ╭─[entry #9:1:13]
 1 │ force_error "My error"
   ·             ─────┬────
   ·                  ╰── here's the error
   ╰────
```
As we can see, the message `oh no!` is output in a new line, and there
is one less trailing line. I have makes some testing, and it seems that
it only happened on `error make` command.

# Tests + Formatting
Changed 1 test

# After Submitting
NaN
2024-11-27 22:43:36 +01:00
Douglas
5e7263cd1a
Bump reedline to current main (#14455)
# Description

@fdncred mentioned that we should be dogfooding the latest Reedline
changes in Nushell. Hoping I got the steps correct.

# User-Facing Changes

New keybindings for:

* Insert Newline: <kbd>Alt</kbd>+<kbd>Enter</kbd> and
<kbd>Shift</kbd>+<kbd>Enter</kbd>
* Enter:  <kbd>Ctrl</kbd>+<kbd>J</kbd>

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
2024-11-27 23:42:30 +08:00
Bahex
0aafc29fb5
Propagate existing errors in insert and merge (#14453)
# Description
Propagate existing errors in the pipeline, rather than a type error.

# User-Facing Changes
Nothing that previously worked should be affected, this should just
change the errors.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-11-27 06:37:21 -06:00
PegasusPlusUS
bd37473515
Fix unstable test case: One time my windows report drive letter as lowercase (#14451)
As I'm working on PWD-per-drive feature. Once the plugin test of env
failed. I checked the log, found sometime Windows can give drive letter
as lowercase, so the test case should be rewrite to check first letter
caseinsensitive equal, and following part normal equal.

```
assert_eq! failed at tests/plugins/env.rs:43:5
left: r"e:\Study\Nushell"
right: r"E:\Study\Nushell"
```

---------

Co-authored-by: Zhenping Zhao <pegasus.cadence@gmail.com>
2024-11-27 06:27:06 -06:00
Douglas
1c18e37a7c
Always populate config record during startup (#14435)
# Description

As a bit of a follow-on to #13802 and #14249, this (pretty much a
"one-line" change) really does *always* populate the `$env.config`
record with the `nu-protocol::config` defaults during startup. This
means that an `$env.config` record is value (with defaults) even during:

* `nu -n` to suppress loading of config files
* `nu -c <commandstring>`
* `nu <script>`

# User-Facing Changes

There should be no case in which there isn't a valid `$env.config`.

* Before:

  ```nushell
  nu -c "$env.config"
  # -> Error
  ```

* After:

  ```nushell
  nu -c "$env.config"
  # -> Default $env.config record
  ```

Startup time impact is negligible (17.072µs from `perf!` on my system) -
Seems well worth it.

# Tests + Formatting

Added tests for several `-n -c` cases.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Config chapter update still in progress.
2024-11-27 13:52:47 +08:00
Antoine Stevan
547c436281
add from ndnuon and to ndnuon to stdlib (#14334)
# Description
i was playing with the NDNUON format and using local definitions of
`from ndnuon` and `to ndnuon` but then i thought they could live in the
standard library next to `from ndjson` and `to ndjson` 😋

# User-Facing Changes
users can now add the following to their configs and get NDNUON ready to
go
```nushell
use std formats ["from ndnuon" "to ndnuon"]
```

# Tests + Formatting
i did simply mimic the tests for `from ndjson` and `to ndjson`, i hope
it's fine since the recent big change to the standard library

# After Submitting

---------

Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com>
2024-11-27 09:43:49 +08:00
Wind
e0c0d39ede
deprecate --ignore-shell-errors and --ignore-program-errors in do (#14385)
# Description
As title, this pr is going to deprecate `--ignore-shell-errors` and
`--ignore-program-errors`.

Because I think these two flags makes `do` command complicate, and it
should be easy to use `-i` instead.

# User-Facing Changes
After the pr, using these two flags will raise deprecated warning.
```nushell
> do --ignore-program-errors { ^pwd }
Error:   × Deprecated option
   ╭─[entry #2:1:1]
 1 │ do --ignore-program-errors { ^pwd }
   · ─┬
   ·  ╰── `--ignore-program-errors` is deprecated and will be removed in 0.102.0.
   ╰────
  help: Please use the `--ignore-errors(-i)`
/home/windsoilder/projects/nushell
> do --ignore-shell-errors { ^pwd }
Error:   × Deprecated option
   ╭─[entry #3:1:1]
 1 │ do --ignore-shell-errors { ^pwd }
   · ─┬
   ·  ╰── `--ignore-shell-errors` is deprecated and will be removed in 0.102.0.
   ╰────
  help: Please use the `--ignore-errors(-i)`
/home/windsoilder/projects/nushell
```

# Tests + Formatting
NaN
2024-11-27 09:36:30 +08:00
Ian Manske
4edce44689
Remove ListStream type (#14425)
# Description
List values and list streams have the same type (`list<>`). Rather,
streaming is a separate property of the pipeline/command output. This PR
removes the unnecessary `ListStream` type.

# User-Facing Changes
Should be none, except `random dice` now has a more specific output
type.
2024-11-27 09:35:55 +08:00
Wind
186c08467f
make std help more user friendly (#14347)
# Description
Fixes:  #13159

After the change, `std help` will no-longer print out "double error"
messages.

Actually I think it's tricky to make it right. To make `help <cmd>`
keeps paging feature from fallback `man` command. I have to split
`commands` into `scope-commands` and `external-commands`.

If we don't split it, simply call `let commands = (try { commands
$target_item --find $find })` in `help main` will cause us to lost
paging feature, which is not we want.

A comment from original issue:

> If there are no objections, I'd like to remove the man page fallback
code from std help for the moment. While it's probably fixable, it's
also platform specific and requires testing on all platforms. It also
seems like a low-value add here.

Actually I think it's a beautiful feature of `std help`, so I want to
keep it here.

# User-Facing Changes
### Before
```nushell
> help commands asdfadsf
Help pages from external command asdfadsf:
No manual entry for asdfadsf
Error:   × std::help::command_not_found
   ╭─[entry #11:1:15]
 1 │ help commands asdfadsf
   ·               ────┬───
   ·                   ╰── command not found
   ╰────
```

### After
```nushell
> help commands asdfasdf
Help pages from external command asdfasdf:
No manual entry for asdfasdf
```

# Tests + Formatting
Actually it's a little hard to add test because it required user input
(especially for fallback `man` command)
2024-11-27 09:29:25 +08:00
dependabot[bot]
367fb9b504
Bump crate-ci/typos from 1.27.3 to 1.28.1 (#14447)
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.27.3 to
1.28.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/crate-ci/typos/releases">crate-ci/typos's
releases</a>.</em></p>
<blockquote>
<h2>v1.28.1</h2>
<h2>[1.28.1] - 2024-11-26</h2>
<h3>Fixes</h3>
<ul>
<li>Add back in <code>lock</code> file types accidentally removed in
1.28 (<code>go.sum</code>, <code>requirements.txt</code>)</li>
</ul>
<h2>v1.28.0</h2>
<h2>[1.28.0] - 2024-11-25</h2>
<h3>Features</h3>
<ul>
<li>Updated the dictionary with the <a
href="https://redirect.github.com/crate-ci/typos/issues/1139">November
2024</a> changes</li>
<li>Add many new types and file extensions to the
<code>--type-list</code>, including ada, alire, bat, candid, carp, cml,
devicetree, dita, dockercompose, grpbuild, graphql, hare, lean, meson,
prolog, raku, reasonml, rescript, solidity, svelte, usd, v, wgsl</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/crate-ci/typos/blob/master/CHANGELOG.md">crate-ci/typos's
changelog</a>.</em></p>
<blockquote>
<h2>[1.28.1] - 2024-11-26</h2>
<h3>Fixes</h3>
<ul>
<li>Add back in <code>lock</code> file types accidentally removed in
1.28 (<code>go.sum</code>, <code>requirements.txt</code>)</li>
</ul>
<h2>[1.28.0] - 2024-11-25</h2>
<h3>Features</h3>
<ul>
<li>Updated the dictionary with the <a
href="https://redirect.github.com/crate-ci/typos/issues/1139">November
2024</a> changes</li>
<li>Add many new types and file extensions to the
<code>--type-list</code>, including ada, alire, bat, candid, carp, cml,
devicetree, dita, dockercompose, grpbuild, graphql, hare, lean, meson,
prolog, raku, reasonml, rescript, solidity, svelte, usd, v, wgsl</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bd36f89fcd"><code>bd36f89</code></a>
chore: Release</li>
<li><a
href="9b3917ceee"><code>9b3917c</code></a>
docs: Update changelog</li>
<li><a
href="8a0dae6793"><code>8a0dae6</code></a>
Merge pull request <a
href="https://redirect.github.com/crate-ci/typos/issues/1159">#1159</a>
from epage/go</li>
<li><a
href="70f236086f"><code>70f2360</code></a>
fix: Re-add go.sum, requirements.txt to 'lock' file type</li>
<li><a
href="78d6d22744"><code>78d6d22</code></a>
chore: Release</li>
<li><a
href="e75389e6fb"><code>e75389e</code></a>
chore: Release</li>
<li><a
href="cd4f2950fc"><code>cd4f295</code></a>
docs: Update changelog</li>
<li><a
href="bdcd9c54e3"><code>bdcd9c5</code></a>
Merge pull request <a
href="https://redirect.github.com/crate-ci/typos/issues/1157">#1157</a>
from epage/dict</li>
<li><a
href="31e5b4f5b5"><code>31e5b4f</code></a>
feat(dict): November updates</li>
<li><a
href="ea6fdd1371"><code>ea6fdd1</code></a>
Merge pull request <a
href="https://redirect.github.com/crate-ci/typos/issues/1154">#1154</a>
from dseight/master</li>
<li>Additional commits viewable in <a
href="https://github.com/crate-ci/typos/compare/v1.27.3...v1.28.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=crate-ci/typos&package-manager=github_actions&previous-version=1.27.3&new-version=1.28.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-27 09:27:47 +08:00
Douglas
ac75562296
Remove long-unused autoenv tests (#14436)
# Description

The `.nu-env` file feature was removed some time ago (probably in the
engine-q upgrade?). The tests, however, still remained as dead-code, so
this is just some basic clean-up.

If this feature was ever implemented again, the tests would need to be
rewritten anyway due to the changes in the way config is handled.

# User-Facing Changes

None

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
- 
# After Submitting

N/A
2024-11-26 10:22:18 +08:00
Douglas
7a9b14b49d
Add example for PROMPT_COMMAND_RIGHT (#14439)
# Description

I just completely left out `$env.PROMPT_COMMAND_RIGHT` in the
`sample_env.nu`. This adds it in.

# User-Facing Changes

`config env --sample` will now include doc for `PROMPT_COMMAND_RIGHT`.

# Tests + Formatting

Doc-only

# After Submitting

n/a
2024-11-25 19:25:43 -06:00
Bahex
32196cfe78
Add term query, for querying information from terminals. (#14427)
## Related
- #10150
- https://github.com/nushell/nushell/pull/10150#issuecomment-1721238336
- #10387
- https://github.com/nushell/nushell/pull/10387#issuecomment-1722228185

# Description
`term query`: a command for querying information from the terminal.

Prints the `$query`, and immediately starts reading raw bytes from
stdin.

The standard input will be read until the `terminator` sequence is
encountered.
The `terminator` is not removed from the output.

It also stops on <kbd>Ctrl-C</kbd> with an error.

```
Usage:
  > term query {flags} <query> 

Flags:
  -h, --help: Display the help message for this command
  -t, --terminator (required parameter) <one_of(binary, string)>: stdin will be read until this sequence is encountered

Parameters:
  query <one_of(binary, string)>: The query that will be printed to stdout
```

This was previously possible with `input` until #10150.
`input` command's features such as cursor control, deleting input etc.
are useful, but interfere with this use case.

`term query` makes the following uses possible:

```nushell
# get the terminal size with ansi escape codes
def terminal-size [] {
    let response = term query (ansi size) --terminator 'R'
    # $response should look like this
    # Length: 9 (0x9) bytes | printable whitespace ascii_other non_ascii
    # 00000000:   1b 5b 33 38  3b 31 35 30  52             •[38;150R

    let sz = $response | bytes at 2..<-1 | decode
    # 38;150

    # $sz should look like 38;150
    let size = ($sz | split row ';' | each {into int})

    # output in record syntax
    {
        rows: $size.0
        columns: $size.1
    }
}
```

```nushell
# read clipboard content using OSC 52
term query $"(ansi --osc '52;c;?')(ansi st)" --terminator (ansi st)
| bytes at 7..<-2
| decode
| decode base64
| decode
```

# User-Facing Changes
- added `ansi query`

# Tests + Formatting
- Integration tests should be added if possible.
2024-11-25 15:13:11 -06:00
Ian Manske
4d3283e235
Change append operator to concatenation operator (#14344)
# Description

The "append" operator currently serves as both the append operator and
the concatenation operator. This dual role creates ambiguity when
operating on nested lists.

```nu
[1 2] ++ 3     # appends a value to a list [1 2 3]
[1 2] ++ [3 4] # concatenates two lists    [1 2 3 4]

[[1 2] [3 4]] ++ [5 6]
# does this give [[1 2] [3 4] [5 6]]
# or             [[1 2] [3 4] 5 6]  
```

Another problem is that `++=` can change the type of a variable:
```nu
mut str = 'hello '
$str ++= ['world']
($str | describe) == list<string>
```

Note that appending is only relevant for lists, but concatenation is
relevant for lists, strings, and binary values. Additionally, appending
can be expressed in terms of concatenation (see example below). So, this
PR changes the `++` operator to only perform concatenation.

# User-Facing Changes

Using the `++` operator with a list and a non-list value will now be a
compile time or runtime error.
```nu
mut list = []
$list ++= 1 # error
```
Instead, concatenate a list with one element:
```nu
$list ++= [1]
```
Or use `append`:
```nu
$list = $list | append 1
```

# After Submitting

Update book and docs.

---------

Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com>
2024-11-24 10:59:54 -08:00
Darren Schroeder
dd3a3a2717
remove terminal_size crate everywhere it makes sense (#14423)
# Description

This PR removes the `terminal_size` crate everywhere that it made sense.
I replaced it with crossterm's version called `size`. The places I
didn't remove it were the places that did not have a dependency on
crossterm. So, I thought it was "cheaper" to have a dep on term_size vs
crossterm in those locations.
2024-11-23 19:37:12 -08:00
Douglas
83d8e936ad
Fix small typos in std/dirs (#14422)
# Description

Typos in the command doc-help.
2024-11-23 16:04:27 -06:00
Beinsezii
58576630db
command/http/client use CRLF for headers join instead of LF (#14417)
# Description
Apparently it should be joint CRLF for the EOL marker

https://www.rfc-editor.org/rfc/rfc2616#section-2.2

Plain LF isn't particularly standardized and many backends don't
recognize it. Tested on `starlette`

# User-Facing Changes
None

# Tests + Formatting
It's two characters; everything passes

# After Submitting
Not needed
2024-11-23 13:49:25 -08:00
Solomon
7c84634e3f
return accurate type errors from blocks/expressions in type unions (#14420)
# User-Facing Changes

- `expected <type>` errors are now propagated from
  `Closure | Block | Expression` instead of falling back to
  "expected one of..." for the block:

Before:

```nushell
def foo [bar: bool] {}
if true {} else { foo 1 }
                ────┬────
                    ╰── expected one of a list of accepted shapes: [Block, Expression]
```

After:

```nushell
if true {} else { foo 1 }
                      ┬
                      ╰── expected bool
```
2024-11-23 13:42:00 -08:00
Yash Thakur
671640b0a9
Avoid recomputing fuzzy match scores (#13700)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR makes it so that when using fuzzy matching, the score isn't
recomputed when sorting. Instead, filtering and sorting suggestions is
handled by a new `NuMatcher` struct. This struct accepts suggestions
and, if they match the user's typed text, stores those suggestions
(along with their scores and values). At the end, it returns a sorted
list of suggestions.

This probably won't have a noticeable impact on performance, but it
might be helpful if we start using Nucleo in the future.

Minor change: Makes `find_commands_by_predicate` in `StateWorkingSet`
and `EngineState` take `FnMut` rather than `Fn` for the predicate.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

When using case-insensitive matching, if you have two matches `FOO` and
`abc`, `abc` will be shown before `FOO` rather than the other way
around. I think this way makes more sense than the current behavior.
When I brought this up on Discord, WindSoilder did say it would make
sense to show uppercase matches first if the user typed, say, `F`.
However, that would be a lot more complicated to implement.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added a test for the changes in
https://github.com/nushell/nushell/pull/13302.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-22 06:29:00 -06:00
Bahex
5f7082f053
truly flexible csv/tsv parsing (#14399)
- fixes #14398

I will properly fill out this PR and fix any tests that might break when
I have the time, this was a quick fix.

# Description

This PR makes `from csv` and `from tsv`, with the `--flexible` flag,
stop dropping extra/unexpected columns.

# User-Facing Changes

`$text`'s contents
```csv
value
1,aaa
2,bbb
3
4,ddd
5,eee,extra
```

Old behavior
```nushell
> $text | from csv --flexible --noheaders 
╭─#─┬─column0─╮
│ 0 │ value   │
│ 1 │       1 │
│ 2 │       2 │
│ 3 │       3 │
│ 4 │       4 │
│ 5 │       5 │
╰─#─┴─column0─╯
```

New behavior
```nushell
> $text | from csv --flexible --noheaders 
╭─#─┬─column0─┬─column1─┬─column2─╮
│ 0 │ value   │       │
│ 1 │       1 │ aaa     │       │
│ 2 │       2 │ bbb     │       │
│ 3 │       3 │       │
│ 4 │       4 │ ddd     │       │
│ 5 │       5 │ eee     │ extra   │
╰─#─┴─column0─┴─column1─┴─column2─╯
```

- The first line in a csv (or tsv) document no longer limits the number
of columns
- Missing values in columns are longer automatically filled with `null`
with this change, as a later row can introduce new columns. **BREAKING
CHANGE**

Because missing columns are different from empty columns, operations on
possibly missing columns will have to use optional access syntax e.g.
`get foo` => `get foo?`
  
# Tests + Formatting
Added examples that run as tests and adjusted existing tests to confirm
the new behavior.

# After Submitting

Update the workaround with fish completer mentioned
[here](https://www.nushell.sh/cookbook/external_completers.html#fish-completer)
2024-11-21 15:58:31 -06:00
Rikuki IX
2a90cb7355
Update SHLVL (only when interactive) on startup (#14404)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Make NuShell correctly inherit and update `SHLVL` from other shells
(obviously including itself) in Unix environment.

See issue #14384

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

None

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

New code formatted.

New feature works well in interactive usage.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-21 15:57:33 -06:00
Marc Schreiber
e63976df7e
Bump Calamine (#14403)
This commit upgrades calamine in order to benefit from recent
developments, e.g. ignore annotations in column headers (see
https://github.com/tafia/calamine/pull/467 for reference).
2024-11-21 20:31:14 +08:00
Ian Manske
d8c2493658
Deprecate split-by command (#14019)
# Description
I'm not quite sure what the point of the `split-by` command is. The only
example for the command seems to suggest it's an additional grouping
command. I.e., a record that seems to be the output of the `group-by`
command is passed to `split-by` which then adds an additional layer of
grouping based on a different column.

# User-Facing Changes
Breaking change, deprecated the command.
2024-11-21 10:47:03 +01:00
Douglas
4ed25b63a6
Always load default env/config values (#14249)
# Release-Notes Short Description

* Nushell now always loads its internal `default_env.nu` before the user
`env.nu` is loaded, then loads the internal `default_config.nu` before
the user's `config.nu` is loaded. This allows for a simpler
user-configuration experience. The Configuration Chapter of the Book
will be updated soon with the new behavior.

# Description

Implements the main ideas in #13671 and a few more:

* Users can now specify only the environment and config options they
want to override in *their* `env.nu` and `config.nu`and yet still have
access to all of the defaults:
* `default_env.nu` (internally defined) will be loaded whenever (and
before) the user's `env.nu` is loaded.
* `default_config.nu` (internally defined) will be loaded whenever (and
before) the user's `config.nu` is loaded.
* No more 900+ line config out-of-the-box.
* Faster startup (again): ~40-45% improvement in launch time with a
default configuration.
* New keys that are added to the defaults in the future will
automatically be available to all users after updating Nushell. No need
to regenerate config to get the new defaults.
* It is now possible to have different internal defaults (which will be
used with `-c` and scripts) vs. REPL defaults. This would have solved
many of the user complaints about the [`display_errors`
implementation](https://www.nushell.sh/blog/2024-09-17-nushell_0_98_0.html#non-zero-exit-codes-are-now-errors-toc).
* A basic "scaffold" `config.nu` and `env.nu` are created on first
launch (if the config directory isn't present).
* Improved "out-of-the-box" experience (OOBE) - No longer asks to create
the files; the minimal scaffolding will be automatically created. If
deleted, they will not be regenerated. This provides a better
"out-of-the-box" experience for the user as they no longer have to make
this decision (without much info on the pros or cons) when first
launching.
* <s>(New: 2024-11-07) Runs the env_conversions process after the
`default_env.nu` is loaded so that users can treat `Path`/`PATH` as
lists in their own config.</s>
* (New: 2024-11-08) Given the changes in #13802, `default_config.nu`
will be a minimal file to minimize load-times. This shaves another (on
my system) ~3ms off the base launch time.
* Related: Keybindings, menus, and hooks that are already internal
defaults are no longer duplicated in `$env.config`. The documentation
will be updated to cover these scenarios.
* (New: 2024-11-08) Move existing "full" `default_config.nu` to
`sample_config.nu` for short-term "documentation" purposes.
* (New: 2024-11-18) Move the `dark-theme` and `light-theme` to Standard
Library and demonstrate their use - Also improves startup times, but
we're reaching the limit of optimization.
* (New: 2024-11-18) Extensively documented/commented `sample_env.nu` and
`sample_config.nu`. These can be displayed in-shell using (for example)
`config nu --sample | nu-highlight | less -R`. Note: Much of this will
eventually be moved to or (some) duplicated in the Doc. But for now,
this some nice in-shell doc that replaces the older
"commented/documented default".
* (New: 2024-11-20) Runs the `ENV_CONVERSIONS` process (1) after the
`default_env.nu` (allows `PATH` to be used as a list in user's `env.nu`)
and (2) before `default_config.nu` is loaded (allows user's
`ENV_CONVERSIONS` from their `env.nu` to be used in their `config.nu`).
* <s>(New: 2024-11-20) The default `ENV_CONVERSIONS` is now an empty
record. The internal Rust code handles `PATH` (and variants) conversions
regardless of the `ENV_CONVERSIONS` variable. This shaves a *very* small
amount of time off the startup.</s> Reset - Looks like there might be a
bug in `nu-enginer::env::ensure_path()` on Windows that would need to be
fixed in order for this to work.

# User-Facing Changes

By default, you shouldn't see much, if any, change when running this
with your existing configuration.

To see the greatest benefit from these changes, you'll probably want to
start with a "fresh" config. This can be easily tested using something
like:

```nushell
let temp_home = (mktemp -d)
$env.XDG_CONFIG_HOME = $temp_home
$env.XDG_DATA_HOME = $temp_home
./target/release/nu
```

You should see a message where the (mostly empty) `env.nu` and
`config.nu` are created on first start. Defaults should be the same (or
similar to) those before the PR. Please let me know if you notice any
differences.

---

Users should now specify configuration in terms of overrides of each
setting. For instance, rather than modifying `history` settings in the
monolithic `config.nu`, the following is recommended in an updated
`config.nu`:

```nu
$env.config.history = {
  file_format: sqlite,
  sync_on_enter: true
  isolation: true
  max_size: 1_000_000
}
```

or even just:

```nu
$env.config.history.file_format = sqlite
$env.config.history.isolation: true
$env.config.history.max_size = 1_000_000
```

Note: It seems many users are already appending a `source my_config.nu`
(or similar pattern) to the end of the existing `config.nu` to make
updates easier. In this case, they will likely want to remove all of the
previous defaults and just move their `my_config.nu` to `config.nu`.

Note: It should be unlikely that there are any breaking changes here,
but there's a slim chance that some code, somewhere, *expects* an
absence of certain config values. Otherwise, all config values are
available before and after this change.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Configuration Chapter (and related) of the doc is currently WIP and will
be finished in time for 0.101 release.
2024-11-20 16:15:15 -06:00
Darren Schroeder
b318d588fe
add new --flatten parameter to the ast command (#14400)
# Description

By request, this PR introduces a new `--flatten` parameter to the ast
command for generating a more readable version of the AST output. This
enhancement improves usability by allowing users to easily visualize the
structure of the AST.


![image](https://github.com/user-attachments/assets/a66644ef-5fff-4d3d-a334-4e9f80edb39d)

```nushell
❯ ast 'ls | sort-by type name -i' --flatten --json
[
  {
    "content": "ls",
    "shape": "shape_internalcall",
    "span": {
      "start": 0,
      "end": 2
    }
  },
  {
    "content": "|",
    "shape": "shape_pipe",
    "span": {
      "start": 3,
      "end": 4
    }
  },
  {
    "content": "sort-by",
    "shape": "shape_internalcall",
    "span": {
      "start": 5,
      "end": 12
    }
  },
  {
    "content": "type",
    "shape": "shape_string",
    "span": {
      "start": 13,
      "end": 17
    }
  },
  {
    "content": "name",
    "shape": "shape_string",
    "span": {
      "start": 18,
      "end": 22
    }
  },
  {
    "content": "-i",
    "shape": "shape_flag",
    "span": {
      "start": 23,
      "end": 25
    }
  }
]
❯ ast 'ls | sort-by type name -i' --flatten --json --minify
[{"content":"ls","shape":"shape_internalcall","span":{"start":0,"end":2}},{"content":"|","shape":"shape_pipe","span":{"start":3,"end":4}},{"content":"sort-by","shape":"shape_internalcall","span":{"start":5,"end":12}},{"content":"type","shape":"shape_string","span":{"start":13,"end":17}},{"content":"name","shape":"shape_string","span":{"start":18,"end":22}},{"content":"-i","shape":"shape_flag","span":{"start":23,"end":25}}]
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-11-20 11:39:15 -06:00
Darren Schroeder
42d2adc3e0
allow ps1 files to be executed without pwsh/powershell -c file.ps1 (#14379)
# Description

This PR allows nushell to run powershell scripts easier. You can already
do `powershell -c script.ps1` but this PR takes it a step further by
doing the `powershell -c` part for you. So, if you have script.ps1 you
can execute it by running it in the command position of the repl.

![image](https://github.com/user-attachments/assets/0661a746-27d9-4d21-b576-c244ff7fab2b)

or once it's in json, just consume it with nushell.

![image](https://github.com/user-attachments/assets/38f5c5d8-3659-41f0-872b-91a14909760b)

# User-Facing Changes
Easier to run powershell scripts. It should work on Windows with
powershell.exe.

# Tests + Formatting
Added 1 test

# After Submitting


---------

Co-authored-by: Wind <WindSoilder@outlook.com>
2024-11-20 21:55:26 +08:00
Devyn Cairns
5d1eb031eb
Turn compile errors into fatal errors (#14388)
# Description

Because the IR compiler was previously optional, compile errors were not
treated as fatal errors, and were just logged like parse warnings are.
This unfortunately meant that if a user encountered a compile error,
they would see "Can't evaluate block in IR mode" as the actual error in
addition to (hopefully) logging the compile error.

This changes compile errors to be treated like parse errors so that they
show up as the last error, helping users understand what's wrong a
little bit more easily.

Fixes #14333.

# User-Facing Changes
- Shouldn't see "Can't evaluate block in IR mode"
- Should only see compile error
- No evaluation should happen

# Tests + Formatting
Didn't add any tests specifically for this, but it might be good to have
at least one that checks to ensure the compile error shows up and the
"can't evaluate" error does not.
2024-11-20 19:24:03 +08:00
dependabot[bot]
1e7840c376
Bump terminal_size from 0.3.0 to 0.4.0 (#14393)
Bumps [terminal_size](https://github.com/eminence/terminal-size) from
0.3.0 to 0.4.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/eminence/terminal-size/releases">terminal_size's
releases</a>.</em></p>
<blockquote>
<h2>v0.4.0</h2>
<h2>Breaking changes</h2>
<p>The big change in this release is the API change in <a
href="https://redirect.github.com/eminence/terminal-size/issues/66">#66</a>:</p>
<ul>
<li>If you were using the <code>terminal_size_using_fd</code> or
<code>terminal_size_using_handle</code> functions, these are now
deprecated and unsafe. Instead you should use the
<code>terminal_size_of</code> function, which does the same thing but is
safer.</li>
</ul>
<h2>What's Changed</h2>
<ul>
<li>Add <code>rust-version</code> in Cargo.toml by <a
href="https://github.com/cgwalters"><code>@​cgwalters</code></a> in <a
href="https://redirect.github.com/eminence/terminal-size/pull/60">eminence/terminal-size#60</a></li>
<li>Update <code>windows-sys</code> to 0.52 by <a
href="https://github.com/barrbrain"><code>@​barrbrain</code></a> in <a
href="https://redirect.github.com/eminence/terminal-size/pull/62">eminence/terminal-size#62</a></li>
<li>Update windows-sys to 0.59 by <a
href="https://github.com/eminence"><code>@​eminence</code></a> in <a
href="https://redirect.github.com/eminence/terminal-size/pull/67">eminence/terminal-size#67</a></li>
<li>Update the API for I/O safety by <a
href="https://github.com/sunfishcode"><code>@​sunfishcode</code></a> in
<a
href="https://redirect.github.com/eminence/terminal-size/pull/66">eminence/terminal-size#66</a></li>
<li>Fix typo, link to docs, update docs by <a
href="https://github.com/waywardmonkeys"><code>@​waywardmonkeys</code></a>
in <a
href="https://redirect.github.com/eminence/terminal-size/pull/63">eminence/terminal-size#63</a></li>
<li>Update CI: Use current actions, remove unused build step by <a
href="https://github.com/waywardmonkeys"><code>@​waywardmonkeys</code></a>
in <a
href="https://redirect.github.com/eminence/terminal-size/pull/64">eminence/terminal-size#64</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/cgwalters"><code>@​cgwalters</code></a>
made their first contribution in <a
href="https://redirect.github.com/eminence/terminal-size/pull/60">eminence/terminal-size#60</a></li>
<li><a href="https://github.com/barrbrain"><code>@​barrbrain</code></a>
made their first contribution in <a
href="https://redirect.github.com/eminence/terminal-size/pull/62">eminence/terminal-size#62</a></li>
<li><a
href="https://github.com/waywardmonkeys"><code>@​waywardmonkeys</code></a>
made their first contribution in <a
href="https://redirect.github.com/eminence/terminal-size/pull/63">eminence/terminal-size#63</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/eminence/terminal-size/compare/v0.3.0...v0.4.0">https://github.com/eminence/terminal-size/compare/v0.3.0...v0.4.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f6b81b5714"><code>f6b81b5</code></a>
Bump to version 0.4.0</li>
<li><a
href="5cbc616cf3"><code>5cbc616</code></a>
Merge pull request <a
href="https://redirect.github.com/eminence/terminal-size/issues/64">#64</a>
from waywardmonkeys/update-ci</li>
<li><a
href="68ceb8dc9d"><code>68ceb8d</code></a>
Merge pull request <a
href="https://redirect.github.com/eminence/terminal-size/issues/63">#63</a>
from waywardmonkeys/fix-typo</li>
<li><a
href="53077475d5"><code>5307747</code></a>
Merge pull request <a
href="https://redirect.github.com/eminence/terminal-size/issues/66">#66</a>
from sunfishcode/main</li>
<li><a
href="a29b904580"><code>a29b904</code></a>
Mark <code>terminal_size_using_handle</code> as unsafe too.</li>
<li><a
href="ea92388054"><code>ea92388</code></a>
Mark <code>terminal_size_using_fd</code> as unsafe.</li>
<li><a
href="78e81fa487"><code>78e81fa</code></a>
Merge pull request <a
href="https://redirect.github.com/eminence/terminal-size/issues/67">#67</a>
from eminence/windows-sys</li>
<li><a
href="c69ff4e55f"><code>c69ff4e</code></a>
Update windows-sys to 0.59</li>
<li><a
href="76b0caeb6f"><code>76b0cae</code></a>
Merge pull request <a
href="https://redirect.github.com/eminence/terminal-size/issues/62">#62</a>
from barrbrain/windows-sys</li>
<li><a
href="56334c3cea"><code>56334c3</code></a>
Update the API for I/O safety</li>
<li>Additional commits viewable in <a
href="https://github.com/eminence/terminal-size/compare/v0.3.0...v0.4.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=terminal_size&package-manager=cargo&previous-version=0.3.0&new-version=0.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-20 09:20:47 +08:00