Commit Graph

8766 Commits

Author SHA1 Message Date
Antoine Büsch
b2092df27e
Upgrade to ratatui 0.26 (#11742)
# Description
Upgrade `ratatui` to 0.26

# User-Facing Changes
n/a
2024-02-08 08:15:45 +08:00
KITAGAWA Yasutaka
09f513bb53
Allow comments in match blocks (#11717)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Fix #9878 

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Writing comments in match blocks will be allowed.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-08 07:22:42 +08:00
Ian Manske
857c522808
Fix #11750: LazyRecord error message (#11772)
# Description
Makes `LazyRecord`s have the same error message as regular `Records` for
`Value::follow_cell_path`. Fixes #11750.
2024-02-08 07:22:15 +08:00
Antoine Büsch
9fa2b77611
Allow specifying a cellpath in input list to get the value to display (#11748)
# Description
When using a table (or a list of records) as input to `input list`,
allow specifying a cellpath for the field/column to use as the display
value.

For instance, at the moment, using a table as input results in the
following:

```
❯ [[name price]; [Banana 12] [Kiwi 4] [Pear 7]] | input list
> {name: Banana, price: 12}
  {name: Kiwi, price: 4}
  {name: Pear, price: 7}
```

With the new `--display` flag introduced by this PR, you can do the
following:

```
❯ [[name price]; [Banana 12] [Kiwi 4] [Pear 7]] | input list -d name
> Banana
  Kiwi
  Pear
```

Note that it doesn't change what gets returned after selecting an item:
the full row/record is still returned.

# User-Facing Changes
A new optional flag is allowed.

# Tests + Formatting

# After Submitting
2024-02-08 07:21:47 +08:00
Ian Manske
f8a8eca836
Record cleanup (#11726)
# Description
Does a little cleanup in `record.rs`:
- Makes the `record!` macro more hygienic.
- Converts regular comments to doc comments from #11718.
- Converts the `Record` iterators to new types.
2024-02-08 06:43:12 +08:00
Wind
20aa59085b
Fix file completions which contains glob pattern (#11766)
# Description
Fixes: https://github.com/nushell/nushell/issues/11762

The auto-completion is somehow annoying if a path contains a glob
pattern, let's say if user type `ls` and it auto-completes to <code>ls
`[a] bc.txt`</code>, and user can't list the file because it's backtick
quoted.

This pr is going to fix it.

# User-Facing Changes
### Before
```
❯ | ls
`[a] bc.txt`        `a bc`
```
### After
```
❯ | ls
"[a] bc.txt"        `a bc`
```
# Tests + Formatting
Done

# After Submitting
NaN
2024-02-08 06:42:50 +08:00
TrMen
4b91ed57dd
Enforce call stack depth limit for all calls (#11729)
# Description
Previously, only direcly-recursive calls were checked for recursion
depth. But most recursive calls in nushell are mutually recursive since
expressions like `for`, `where`, `try` and `do` all execute a separte
block.

```nushell
def f [] {
    do { f }
}
```
Calling `f` would crash nushell with a stack overflow.

I think the only general way to prevent such a stack overflow is to
enforce a maximum call stack depth instead of only disallowing directly
recursive calls.

This commit also moves that logic into `eval_call()` instead of
`eval_block()` because the recursion limit is tracked in the `Stack`,
but not all blocks are evaluated in a new stack. Incrementing the
recursion depth of the caller's stack would permanently increment that
for all future calls.

Fixes #11667

# User-Facing Changes
Any function call can now fail with `recursion_limit_reached` instead of
just directly recursive calls. Mutually-recursive calls no longer crash
nushell.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-08 06:42:24 +08:00
Darren Schroeder
366348dea0
make char command const (#11771)
# Description

This PR makes the `char` command a `const` command. The only real
changes are to get the arguments different and I extracted code into
functions so they could be called via run and run_cost. No algorithms
were harmed in the making of this PR.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-07 16:29:00 -06:00
Darren Schroeder
b6b36e00c6
allow ansi strip to work better on other nushell values (#11781)
# Description

This PR help `ansi strip` work on more nushell values. It does this by
converting values like filesize and dates to strings. This may not be
precisely correct but I think it does more what the user expects.

### Before

![image](https://github.com/nushell/nushell/assets/343840/768ffbb2-e3d7-424e-8e3b-1d20c9aa7d91)


### After

![image](https://github.com/nushell/nushell/assets/343840/6141aebb-481f-45a9-9cb7-084ca9ca1ea5)


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-07 16:28:40 -06:00
Darren Schroeder
c79432f33c
for ints, provide an option to convert all of them to filesizes with the into value command (#11797)
# Description

This PR allows `into value` to recognize ints and change them into file
sizes if you prefer.
### Before
```nushell
❯ free | ^column -t | lines | update 0 {$"type  ($in)"} | to text | ^column -t | detect columns | into value 
╭─#─┬─type──┬──total───┬──used───┬───free───┬shared┬buff/cache┬available─╮
│ 0 │ Mem:  │ 24614036 │ 3367680 │ 16196240 │ 3688 │  5449736 │ 21246356 │
│ 1 │ Swap: │  6291456 │       0 │  6291456 │      │          │          │
╰───┴───────┴──────────┴─────────┴──────────┴──────┴──────────┴──────────╯
```
### After
```nushell
❯ free | ^column -t | lines | update 0 {$"type  ($in)"} | to text | ^column -t | detect columns | into value --prefer-filesizes
╭─#─┬─type──┬──total──┬──used──┬──free───┬─shared─┬buff/cache┬available╮
│ 0 │ Mem:  │ 24.6 MB │ 3.4 MB │ 16.2 MB │ 3.7 KB │   5.4 MB │ 21.2 MB │
│ 1 │ Swap: │  6.3 MB │    0 B │  6.3 MB │        │          │         │
╰───┴───────┴─────────┴────────┴─────────┴────────┴──────────┴─────────╯
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-07 16:28:17 -06:00
Darren Schroeder
08931e976e
bump to dev release of nushell 0.90.2 (#11793)
# Description

Bump nushell version to the dev version of 0.90.2

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-07 16:26:03 -06:00
Jakub Žádník
c2992d5d8b
Bump to 0.90.1 (#11787)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Merge after https://github.com/nushell/nushell/pull/11786

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-06 16:28:49 -06:00
Ian Manske
342907c04d
Add serde feature for byte-unit (#11786)
Fixes compilation error for `nu-protocol`
2024-02-06 16:20:09 -06:00
Jakub Žádník
f5f21aca2d
Bump to 0.90 (#11730)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-06 22:42:43 +02:00
Jakub Žádník
9342ad6ae1
Pin reedline to version 0.29.0 (#11782)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

See full release notes:
https://github.com/nushell/reedline/releases/tag/v0.29.0

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-06 22:23:41 +02:00
dependabot[bot]
cb4ca157fd
Bump crate-ci/typos from 1.17.2 to 1.18.0 (#11743) 2024-02-05 01:00:27 +00:00
Darren Schroeder
d1c807230b
default to having border in ide_menu 2024-02-04 08:19:09 -06:00
Darren Schroeder
4e5d3db952
allow strings with thousands separators to be converted to filesize or ints (#11724)
# Description

This PR changes `into int` and `into filesize` so that they allow
thousands separators.

### Before
```nushell
❯ '1,000' | into filesize
Error: nu:🐚:cant_convert

  × Can't convert to int.
   ╭─[entry #1:1:1]
 1 │ '1,000' | into filesize
   · ───┬───
   ·    ╰── can't convert string to int
   ╰────

❯ '1,000' | into int
Error: nu:🐚:cant_convert

  × Can't convert to int.
   ╭─[entry #2:1:1]
 1 │ '1,000' | into int
   ·           ────┬───
   ·               ╰── can't convert string to int
   ╰────
  help: string "1,000" does not represent a valid integer
```
### After
```nushell
❯ '1,000' | into filesize
1.0 KB
❯ '1,000' | into int
1000
```

This works by getting the system locale and from that, determining what
the thousands separator is. So, hopefully, this will work across
locales.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-03 10:42:44 -06:00
Jakub Žádník
b8d37a7541
Fix panic in rotate; Add safe record creation function (#11718)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes https://github.com/nushell/nushell/issues/11716

The problem is in our [record creation
API](0d518bf813/crates/nu-protocol/src/value/record.rs (L33))
which panics if the numbers of columns and values are different. I added
a safe variant that returns a `Result` and used it in the `rotate`
command.

## TODO in another PR:

Go through all `from_raw_cols_vals_unchecked()` (this includes the
`record!` macro which uses the unchecked version) and make sure that
either
a) it is guaranteed the number of cols and vals is the same, or
b) convert the call to `from_raw_cols_vals()`

Reason: Nushell should never panic.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-03 13:23:16 +02:00
Jakub Žádník
c7a8aac883
Tighten def body parsing (#11719)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes https://github.com/nushell/nushell/issues/11711

Previously, syntax `def a [] (echo 4)` was allowed to parse and then
failed with panic duting eval.

Current error:
```
Error: nu::parser::parse_mismatch

  × Parse mismatch during operation.
   ╭─[entry #1:1:1]
 1 │ def a [] (echo 4)
   ·          ────┬───
   ·              ╰── expected definition body closure { ... }
   ╰────
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-03 13:20:40 +02:00
kik4444
0d518bf813
query web --query should return list<list<string>> like the scraper crate's ElementRef::text() (#11705)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
## Problem
I tried converting one of my Rust web scrapers to Nushell just to see
how it would be done, but quickly ran into an issue that proved annoying
to fix without diving into the source.

For instance, let's say we have the following HTML
```html
<p>Hello there, <span style="color: red;">World</span></p>
```
and we want to extract only the text within the `p` element, but not the
`span`. With the current version of nu_plugin_query, if we run this code
```nushell
echo `<p>Hello there, <span style="color: red;">World</span></p>` | query web -q "p" | get 0
# returns "Hello there, World"

# but we want only "Hello there, "
```
we will get back a `list<string>` that contains 1 string `Hello there,
World`.
To avoid scraping the span, we would have to do something like this
```nushell
const html = `<p>Hello there, <span style="color: red;">World</span></p>`
$html
| query web -q "p"
| get 0
| str replace ($html | query web -q "p > span" | get 0) ""
# returns "Hello there, "
```
In other words, we would have to make a sub scrape of the text we
*don't* want in order to subtract it from the text we *do* want.

## Solution
I didn't like this behavior, so I decided to change it. I modified the
`execute_selector_query` function to collect all text nodes in the HTML
element matching the query. Now `query web --query` will return a
`list<list<string>>`
```nushell
echo `<p>Hello there, <span style="color: red;">World</span></p>` | query web -q "p" | get 0 | to json --raw
# returns ["Hello there, ","World"]
```
This also brings `query web --query`'s behavior more in line with
[scraper's
ElementRef::text()](https://docs.rs/scraper/latest/scraper/element_ref/struct.ElementRef.html#method.text)
which "Returns an iterator over descendent text nodes", allowing you to
choose how much of an element's text you want to scrape without
resorting to string substitutions.

## Consequences
As this is a user-facing change, the usage examples will produce
different results than before. For example
```nushell
http get https://phoronix.com | query web --query 'header'
```
will return a list of lists of 1 string each, whereas before it was just
a list of strings.

I only modified the 3rd example
```nushell
# old
http get https://www.nushell.sh | query web --query 'h2, h2 + p' | group 2 | each {rotate --ccw tagline description} | flatten
# new
http get https://www.nushell.sh | query web --query 'h2, h2 + p' | each {str join} | group 2 | each {rotate --ccw tagline description} | flatten
```
to make it behave like before because I thought this one ought to show
the same results as before.
However, the second reason I changed the 3rd example is because it
otherwise panics! If we run the original 3rd example with my
modifications, we get a panic
```
thread 'main' panicked at crates/nu-protocol/src/value/record.rs:34:9:
assertion `left == right` failed
  left: 2
 right: 17
```
This happens because `rotate` receives a list of lists where the inner
lists have a different number of elements.

However this panic is unrelated to the changes I've made, because it can
be triggered easily without using the plugin. For instance
```nushell
# this is fine
[[[one] [two]] [[three] [four]]] | each {rotate --ccw tagline description}

# this panics!
[[[one] [two]] [[three] [four five]]] | each {rotate --ccw tagline description}
```
Though beyond the scope of this PR, I thought I'd mention this bug since
I found it while testing the usage examples. However, I intend to make a
proper issue about it tomorrow.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`query web --query "css selector"` now returns a `list<list<string>>`
instead of a `list<string>` to make it more in line with [scraper's
ElementRef::text()](https://docs.rs/scraper/latest/scraper/element_ref/struct.ElementRef.html#method.text).

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I ran `cargo fmt --all -- --check`, `cargo clippy --workspace -- -D
warnings -D clippy::unwrap_used` and the tests in the plugin.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
PR that updates the documentation to match the new 3rd example:
https://github.com/nushell/nushell.github.io/pull/1235
2024-02-02 19:40:47 -06:00
Darren Schroeder
e083b75aef
update to latest reedline after column menu fix (#11715)
# Description

This PR updates to the latest reedline after the column menu fix.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-03 08:30:28 +08:00
Darren Schroeder
2e5a857983
update query web wiki example (#11709)
# Description

This PR tries to make `query web` more resilient and easier to debug
with the `--inspect` parameter when trying to scrape tables. Previously
it would just fail, now at least it tries to give you a hint.

This is some example output now of when something went wrong.
```
❯ http get https://en.wikipedia.org/wiki/List_of_cities_in_India_by_population | query web --as-table [Rank City 'Population(2011)[3]' 'Population(2001)[3][a]' 'State or union territory'] --inspect
Passed in Column Headers = ["Rank", "City", "Population(2011)[3]", "Population(2001)[3][a]", "State or union territory"]

First 2048 HTML chars = <!DOCTYPE html>
<html class="client-nojs vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-sticky-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-0 vector-feature-client-preferences-disabled vector-feature-client-prefs-pinned-disabled vector-toc-available" lang="en" dir="ltr">
<head>
<meta charset="UTF-8">
<title>List of cities in India by population - Wikipedia</title>
<script>(function(){var className="client-js vector-feature-language-in-header-enabled vector-feature-language-in-main-page-header-disabled vector-feature-sticky-header-disabled vector-feature-page-tools-pinned-disabled vector-feature-toc-pinned-clientpref-1 vector-feature-main-menu-pinned-disabled vector-feature-limited-width-clientpref-1 vector-feature-limited-width-content-enabled vector-feature-custom-font-size-clientpref-0 vector-feature-client-preferences-disabled vector-feature-client-prefs-pinned-disabled vector-toc-available";var cookie=document.cookie.match(/(?:^|; )enwikimwclientpreferences=([^;]+)/);if(cookie){cookie[1].split('%2C').forEach(function(pref){className=className.replace(new RegExp('(^| )'+pref.replace(/-clientpref-\w+$|[^\w-]+/g,'')+'-clientpref-\\w+( |$)'),'$1'+pref+'$2');});}document.documentElement.className=className;}());RLCONF={"wgBreakFrames":false,"wgSeparatorTransformTable":["",""],"wgDigitTransformTable":["",""],"wgDefaultDateFormat":"dmy","wgMonthNames":["",
"January","February","March","April","May","June","July","August","September","October","November","December"],"wgRequestId":"9ecdad8f-2dbd-4245-b54d-9c57aea5ca45","wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"List_of_cities_in_India_by_population","wgTitle":"List of cities in India by population","wgCurRevisionId":1192093210,"wgRev

Potential HTML Headers = ["City", "Population(2011)[3]", "Population(2001)[3][a]", "State or unionterritory", "Ref"]

Potential HTML Headers = ["City", "Population(2011)[5]", "Population(2001)", "State or unionterritory"]

Potential HTML Headers = [".mw-parser-output .navbar{display:inline;font-size:88%;font-weight:normal}.mw-parser-output .navbar-collapse{float:left;text-align:left}.mw-parser-output .navbar-boxtext{word-spacing:0}.mw-parser-output .navbar ul{display:inline-block;white-space:nowrap;line-height:inherit}.mw-parser-output .navbar-brackets::before{margin-right:-0.125em;content:\"[ \"}.mw-parser-output .navbar-brackets::after{margin-left:-0.125em;content:\" ]\"}.mw-parser-output .navbar li{word-spacing:-0.125em}.mw-parser-output .navbar a>span,.mw-parser-output .navbar a>abbr{text-decoration:inherit}.mw-parser-output .navbar-mini abbr{font-variant:small-caps;border-bottom:none;text-decoration:none;cursor:inherit}.mw-parser-output .navbar-ct-full{font-size:114%;margin:0 7em}.mw-parser-output .navbar-ct-mini{font-size:114%;margin:0 4em}vtePopulation of cities in India"]

Potential HTML Headers = ["vteGeography of India"]

╭──────────────────────────┬─────────────────────────────────────────────────────╮
│ Rank                     │ error: no data found (column name may be incorrect) │
│ City                     │ error: no data found (column name may be incorrect) │
│ Population(2011)[3]      │ error: no data found (column name may be incorrect) │
│ Population(2001)[3][a]   │ error: no data found (column name may be incorrect) │
│ State or union territory │ error: no data found (column name may be incorrect) │
╰──────────────────────────┴─────────────────────────────────────────────────────╯
```
The key here is to look at the `Passed in Column Headers` and compare
them to the `Potential HTML Headers` and couple that with the error
table at the bottom should give you a hint that, in this situation,
wikipedia has changed the column names, yet again. So we need to update
our query web statement's tables to get closer to what we want.

```
❯ http get https://en.wikipedia.org/wiki/List_of_cities_in_India_by_population | query web --as-table [City 'Population(2011)[3]' 'Population(2001)[3][a]' 'State or unionterritory' 'Ref']
╭─#──┬───────City───────┬─Population(2011)[3]─┬─Population(2001)[3][a]─┬─State or unionterritory─┬──Ref───╮
│ 0  │ Mumbai           │ 12,442,373          │ 11,978,450             │ Maharashtra             │ [3]    │
│ 1  │ Delhi            │ 11,034,555          │ 9,879,172              │ Delhi                   │ [3]    │
│ 2  │ Bangalore        │ 8,443,675           │ 5,682,293              │ Karnataka               │ [3]    │
│ 3  │ Hyderabad        │ 6,993,262           │ 5,496,960              │ Telangana               │ [3]    │
│ 4  │ Ahmedabad        │ 5,577,940           │ 4,470,006              │ Gujarat                 │ [3]    │
│ 5  │ Chennai          │ 4,646,732           │ 4,343,645              │ Tamil Nadu              │ [3]    │
│ 6  │ Kolkata          │ 4,496,694           │ 4,580,546              │ West Bengal             │ [3]    │
│ 7  │ Surat            │ 4,467,797           │ 2,788,126              │ Gujarat                 │ [3]    │
│ 8  │ Pune             │ 3,124,458           │ 2,538,473              │ Maharashtra             │ [3]    │
│ 9  │ Jaipur           │ 3,046,163           │ 2,322,575              │ Rajasthan               │ [3]    │
│ 10 │ Lucknow          │ 2,817,105           │ 2,185,927              │ Uttar Pradesh           │ [3]    │
│ 11 │ Kanpur           │ 2,765,348           │ 2,551,337              │ Uttar Pradesh           │ [3]    │
│ 12 │ Nagpur           │ 2,405,665           │ 2,052,066              │ Maharashtra             │ [3]    │
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-02 09:03:28 -06:00
Justin Ma
0b6f15b69e
Disable riscv64 build target temporarily to make release and nightly-build work (#11700)
Let's disable riscv64 build target temporarily to make release and
nightly-build workflow work, close #11604
We will add this target later if the related issue got fixed
2024-02-01 06:44:07 -06:00
WindSoilder
16f3d9b4e1
cp: expand target path before checking (#11692)
# Description
Fixes: #11683

# User-Facing Changes
NaN

# Tests + Formatting
~~I don't think we need to add a test, or else it'll copy some file to
user's directory, it seems bad.~~
Done.

# After Submitting
NaN
2024-02-01 09:06:03 +08:00
ellis
3e0fa8ff85
Allow 'url join' to print username without password (#11697)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
Fixes: https://github.com/nushell/nushell/issues/11677
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
```
'https://example.com' | url parse | update scheme ssh | upda
te username user | url join
# => ssh://user@example.com/
'https://example.com' | url parse | update scheme ssh | upda
te password hackme | url join
# => ssh://example.com/
'https://example.com' | url parse | update scheme ssh | update username user | update password hackme | url join
# => ssh://user:hackme@example.com/
```
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Richard Westhaver <ellis@rwest.io>
2024-01-31 16:52:23 -06:00
Raphael Gaschignard
d40019a141
Tidy up the REPL main loop (#11655)
While working on #11288 I was having some trouble tracking the main REPL
loop, so I've sent in a bunch of tiny refactorings on this branch.

These are almost all of the "move code from one place to another"
variety, and each commit is meant to be independent, _except for the
last one_, which is trying to be a bit more clever to handle the
decision of autocd'ing vs running a command. Feel free to just go
through each commit and cherry pick the ones that look good.

This leads to `evaluate_repl` going from ending on line 715 to ending on
line 395. Again, this is mostly just moving code around, but I think
this set of changes will make other changes around juggling the stack to
avoid cloning easier to review.
2024-01-31 09:32:19 -08:00
Darren Schroeder
d9b324c5d3
rollback polars 0.37.0 (#11695)
# Description

This PR rolls back the polars updates to 0.37.0 back to 0.36.2 since it
won't compile yet for some reason.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-31 08:39:38 -06:00
nibon7
00176c5666
Bump wayland (#11694)
# Description
Bumps wayland, and remove `quick-xml v0.30.0` and duplicate `nix
v0.26.4`.
2024-01-31 08:06:59 -06:00
Darren Schroeder
f16ac886a8
change update cells column param from Table to List (#11691)
# Description

This PR fixes `update cells` parameter `--columns`/`-c` so that it takes
a `SyntaxShape::List` instead of `SyntaxShape::Table`.

closes #11689

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-30 19:36:03 -06:00
dependabot[bot]
c77ec0b667
Bump polars-arrow from 0.36.2 to 0.37.0 (#11660) 2024-01-30 22:26:42 +00:00
dependabot[bot]
6779b248e4
Bump sqlparser from 0.41.0 to 0.43.1 (#11662) 2024-01-30 22:26:31 +00:00
Darren Schroeder
0a355db5c0
make the ansi command const (#11682)
# Description

This PR changes the `ansi` command to be a `const` command. 

- ~~It's breaking because I found that I had to change the way `ansi` is
used in scripts a little bit.
https://github.com/nushell/nu_scripts/pull/751~~

- I had to change one of the examples because apparently `const` can't
be tested yet.

- ~~I'm not sure this is right at all
https://github.com/nushell/nushell/pull/11682/files#diff-ba932369a40eb40d6e1985eac1c784af403dab4500a7f0568e593900bf6cd740R654-R655.
I just didn't want to duplicate a ton of code. Maybe if I duplicated the
code it wouldn't be a breaking change because it would have a run and
run_const?~~

- I had to add `opt_const` to CallExt.

/cc @kubouch Can you take a look at this? I'm a little iffy if I'm doing
this right, or even if we should do this at all.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-30 16:09:43 -06:00
Ian Manske
cf9813cbf8
Refactor lines command (#11685)
# Description
This PR uses `str::lines` to simplify the `lines` command (and one other
section of code). This has two main benefits:
1. We no longer need to use regex to split on lines, as `str::lines`
splits on `\r\n` or `\n`.
2. We no longer need to handle blank empty lines at the end. E.g.,
`str::lines` results in `["text"]` for both `"test\n"` and `"text"`.

These changes give a slight boost to performance for the following
benchmarks:
1. lines of `Value::String`:
    ```nushell
    let data = open Cargo.lock
    1..10000 | each { $data | timeit { lines } } | math avg 
    ```
    current main: 392µs
    this PR: 270µs
2. lines of external stream:
    ```nushell
    1..10000 | each { open Cargo.lock | timeit { lines } } | math avg 
    ```
    current main: 794µs
    this PR: 489µs
2024-01-30 15:56:19 -06:00
WindSoilder
c371d1a535
fix exit_code handling when running a scripts with ctrlc (#11466)
# Description
Fixes: #11394

When run `^sleep 3` we have an `exit_code ListStream`, and when we press
ctrl-c, this `ListStream` will return None. But it's not expected,
because `exit_code` sender in `run_external` always send an exit code
out.

This pr is trying to fix the issue by introducing a `first_guard` into
ListStream, it will always generate a value from underlying stream if
`first_guard` is true, so it's guarantee to have at least one value to
return.

And the pr also do a little refactor, which makes use of
`ListStream::from_stream` rather than construct it manually.

# User-Facing Changes
## Before
```
> nu -c "^sleep 3"  # press ctrl-c
> echo $env.LAST_EXIT_CODE
0
```

## After
```
> nu -c "^sleep 3"  # press ctrl-c
> echo $env.LAST_EXIT_CODE
255
```

# Tests + Formatting
None, sorry that I don't think it's easy to test the ctrlc behavior.

# After Submitting
None
2024-01-30 22:41:14 +08:00
Ian Manske
4e0a65c822
Strict JSON parsing (#11592)
# Description
Adds the `--strict` flag for `from json` which will try to parse text
while following the exact JSON specification (e.g., no comments or
trailing commas allowed). Fixes issue #11548.
2024-01-30 08:10:19 -06:00
Georgiana Grigoreanu
6530403ff8
Highlights find upgrade (#11509)
this PR should close #9105

# Description
I have implemented highlights for find which work for all strings. The
implementation also works for lists, but with exceptions (for example,
it does not work for list of lists). The implementation is also not
implemented for --regex.

---------

Co-authored-by: Georgiana <geo@LAPTOP-EQP6H37N>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-01-30 08:06:20 -06:00
Yash Thakur
c08f46f836
Respect SyntaxShape when parsing spread operator (#11674)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

This fixes an issue brought up by nihilander in
[Discord](https://discord.com/channels/601130461678272522/614593951969574961/1201594105986285649).

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Nushell panics when the spread operator is used like this (the
`...$rest` shouldn't actually be parsed as a spread operator at all):

```nu
$ def foo [...rest: string] {...$rest}                      
$ foo bar baz                                               
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/nu-protocol-0.89.0/src/signature.rs:650:9:
Internal error: can't run a predeclaration without a body
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: <nu_protocol::signature::Predeclaration as nu_protocol::engine::command::Command>::run
   3: nu_engine::eval::eval_call
   4: nu_engine::eval::eval_expression_with_input
   5: nu_engine::eval::eval_element_with_input
   6: nu_engine::eval::eval_block
   7: nu_cli::util::eval_source
   8: nu_cli::repl::evaluate_repl
   9: nu::run::run_repl
  10: nu::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
```

The problem was that whenever the parser saw something like `{...$`,
`{...(`, or `{...[`, it would treat that as a record with a spread
expression, ignoring the syntax shape of the block it was parsing. This
should now be fixed, and the snippet above instead gives the following
error:

```nu
Error: nu:🐚:external_command

  × External command failed
   ╭─[entry #1:1:1]
 1 │  def foo [...rest] {...$rest}
   ·                     ────┬───
   ·                         ╰── executable was not found
   ╰────
  help: No such file or directory (os error 2)
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Stuff like `do { ...$rest }` will now try to run a command `...$rest`
rather than complaining that variable `$rest` doesn't exist.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

Sorry about the issue, I am not touching the parser again for a long
time :)
2024-01-30 13:49:42 +08:00
Jack Wright
175dab4898
"[11611] fixing dataframe column comparisons" (#11676)
fixes #11611

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-01-29 17:28:12 -06:00
Sophia June Turner
798ae7b251
Fix precedence of 'not' operator (#11672)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description

A bit hackish but this fixes the precedence of the `not` operator.

Before: `not false and false` => true

Now: `not false and false` => false

Fixes #11633

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-01-29 21:42:27 +02:00
Jack Wright
f879c00f9d
The ability to specify a schema when using dfr open and dfr into-df (#11634)
# Description

There are times where explicitly specifying a schema for a dataframe is
needed such as:
- Opening CSV and JSON lines files and needing provide more information
to polars to keep it from failing or in a desire to override default
type conversion
- When converting a nushell value to a dataframe and wanting to override
the default conversion behaviors.

This pull requests provides:
- A flag to allow specifying a schema when using dfr into-df
- A flag to allow specifying a schema when using dfr open that works for
CSV and JSON types
- A new command `dfr schema` which displays schema information and will
allow display support schema dtypes

Schema is specified creating a record that has the key value and the
dtype. Examples usages:

```
{a:1, b:{a:2}} | dfr into-df -s {a: u8, b: {a: i32}} | dfr schema
{a: 1, b: {a: [1 2 3]}, c: [a b c]} | dfr into-df -s {a: u8, b: {a: list<u64>}, c: list<str>} | dfr schema
 dfr open -s {pid: i32, ppid: i32, name: str, status: str, cpu: f64, mem: i64, virtual: i64} /tmp/ps.jsonl  | dfr schema
```

Supported dtypes:
null                                                   
bool                                                   
u8                                                     
u16                                                    
u32                                                    
u64                                                    
i8                                                     
i16                                                    
i32                                                    
i64                                                    
f32                                                    
f64                                                    
str                                                    
binary                                                 
date                                                   
datetime[time_unit: (ms, us, ns) timezone (optional)]  
duration[time_unit: (ms, us, ns)]                      
time                                                   
object                                                 
unknown                                                
list[dtype]


structs are also supported but are specified via another record:
{a: u8, b: {d: str}}

Another feature with the dfr schema command is that it returns the data
back in a format that can be passed to provide a valid schema that can
be passed in as schema argument:

<img width="638" alt="Screenshot 2024-01-29 at 10 23 58"
src="https://github.com/nushell/nushell/assets/56345/b49c3bff-5cda-4c86-975a-dfd91d991373">

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-01-29 13:26:04 -06:00
Darren Schroeder
d03ad6a257
update to latest reedline for the quick completions fix (#11673)
# Description

This PR updates nushell to the latest reedline main to fix the quick
completions.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-29 13:22:07 -06:00
David Horner
14e4d05a9f
use constant instead of <0 for ls fix (#11642)
https://github.com/nushell/nushell/pull/10558#issuecomment-1911076784

more correct than <0.  safer check.
2024-01-29 13:17:04 -06:00
maxomatic458
86dd045554
add match-text style + config setting for ide menu (#11670)
the match-text style (https://github.com/nushell/reedline/pull/730) is
now configurable via the config.nu file.
the option ``correct_cursor_pos`` can now also be set in the config.nu
file.
2024-01-29 09:59:01 -06:00
WindSoilder
0e023eaa84
add str escape-glob command (#11664)
# Description
This pr is a follow up to #11621, it introduces a `str escape-glob`
command as a workaround for the case:

```nushell
let f = "a[123]b"
ls $f
```

It will glob `a[123]b`, we can get rid of the behavior through `str
escape-glob` command:

```nushll
let f = "a[123]b"
ls ($f | str escape-glob)
```

It's more useful in the `each` context:
`ls | get name | str escape-glob | each {|it| ls $it}`

# User-Facing Changes
NaN

# Tests + Formatting
Done

# After Submitting

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-01-29 23:00:15 +08:00
Andrej Kolchin
427857a78e
Fix wrong error for raw streams in into record (#11668)
Fix #11632
2024-01-29 08:32:43 -06:00
dependabot[bot]
eea3f79c3c
Bump ical from 0.9.0 to 0.10.0 (#11661)
Bumps [ical](https://github.com/Peltoche/ical-rs) from 0.9.0 to 0.10.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/Peltoche/ical-rs/releases">ical's
releases</a>.</em></p>
<blockquote>
<h2>v0.10.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Fix newlines by <a
href="https://github.com/westy92"><code>@​westy92</code></a> in <a
href="https://redirect.github.com/Peltoche/ical-rs/pull/56">Peltoche/ical-rs#56</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Peltoche/ical-rs/compare/v0.9.0...v0.10.0">https://github.com/Peltoche/ical-rs/compare/v0.9.0...v0.10.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="2e83429867"><code>2e83429</code></a>
chore: Release ical version 0.10.0</li>
<li><a
href="6c265a53f0"><code>6c265a5</code></a>
Add test to verify parser handles \r\n.</li>
<li><a
href="79b0e6501b"><code>79b0e65</code></a>
Fixes &amp; clippy.</li>
<li><a
href="821bd46bfb"><code>821bd46</code></a>
Fix split lines.</li>
<li><a
href="8bb242a4ba"><code>8bb242a</code></a>
Fix newlines.</li>
<li>See full diff in <a
href="https://github.com/Peltoche/ical-rs/compare/v0.9.0...v0.10.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=ical&package-manager=cargo&previous-version=0.9.0&new-version=0.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 09:34:06 +08:00
Michael Angerman
b004e80f77
Bump Reedline for the Menu Refactor (#11658)
* [Refactor Menu System with Composition of Menu
Functions](https://github.com/nushell/reedline/issues/706)
* Move Description Menu over to Reedline to consolidate location of the
Menus which will simplify further changes and maintenance to the Menu
system going forward
* Removes lots of code duplication in the Menu system on the Reedline
side which should ease a developers ability to develop new cool menus
for Reedline moving forward
2024-01-28 08:26:03 -08:00
WindSoilder
25b62c2ac3
fix force rm: should suppress error if directory is not found (#11656)
# Description
Fix a breaking change which is introduced by #11621

`rm -f /tmp/aaa` shouldn't return error if `/tmp/aaa/` doesn't exist.

# User-Facing Changes
NaN

# Tests + Formatting
Done
2024-01-28 09:01:19 -06:00
Antoine Stevan
859f7b3dc7
add $.extra_usage to modules (#11649)
- should fix https://github.com/nushell/nushell/issues/11648

# Description
this PR
- adds a test that should pass but fails
- adds `$.extra_usage` to the output of `scope modules`, fixing both the
new test and the linked issue

# User-Facing Changes
`$.extra_usage` is now a column in the output of `scope modules`

# Tests + Formatting
a new test case has been added to `correct_scope_modules_fields`

# After Submitting
2024-01-27 17:49:21 +02:00