Commit Graph

8816 Commits

Author SHA1 Message Date
Devyn Cairns
1f4131532d
Deprecate register and add plugin use (#12607)
# Description

Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:

```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```

This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.

The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.

`register` now prints a deprecation parse warning.

This should fix #11923, as we now have a complete alternative to
`register`.

# User-Facing Changes

- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`

# Tests + Formatting

I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Update plugins sections of book
- [ ] Release notes
2024-04-23 06:37:50 -05:00
Devyn Cairns
5c7f7883c8
Add ErrSpan extension trait for Result (#12626)
# Description
This adds an extension trait to `Result` that wraps errors in `Spanned`,
saving the effort of calling `.map_err(|err| err.into_spanned(span))`
every time. This will hopefully make it even more likely that someone
will want to use a spanned `io::Error` and make it easier to remove the
impl for `From<io::Error> for ShellError` because that doesn't have span
information.
2024-04-23 10:39:55 +02:00
Wind
b0acc1d890
Avoid panic when pipe a variable to a custom command which have recursive call (#12491)
# Description
Fixes: #11351

And comment here is also fixed:
https://github.com/nushell/nushell/issues/11351#issuecomment-1996191537

The panic can happened if we pipe a variable to a custom command which
recursively called itself inside another block.

TBH, I think I figure out how it works to panic, but I'm not sure if
there is a potention issue if nushell don't mutate a block in such case.

# User-Facing Changes
Nan

# Tests + Formatting
Done

# After Submitting
Done

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-23 06:10:35 +08:00
Michael Angerman
bed236362a
bump reedline to latest commit point on main (#12621)
https://github.com/nushell/reedline/pull/781

This will allow others to test and make sure that this reedline fix
works on other terminals and platforms...
2024-04-22 11:48:30 -07:00
Ian Manske
797a90520c
Improve the "input and output are the same file" error text (#12619)
# Description
Continuing from #12601, this PR improves the error message help text and
adds an example to `collect`.
2024-04-22 09:00:38 -05:00
Stefan Holderbach
20bf3c587f
Update ratatui to deduplicate syn in build (#12606)
`ratatui` introduced a dependency on `stability` which until recently
used `syn 1.x`, with this update all our crates in the default `cargo
build` path are using `syn 2.x`
2024-04-22 14:48:33 +02:00
Ian Manske
bae6d694ca
Refactor using ClosureEval types (#12541)
# Description
Adds two new types in `nu-engine` for evaluating closures: `ClosureEval`
and `ClosureEvalOnce`. This removed some duplicate code and centralizes
our logic for setting up, running, and cleaning up closures. For
example, in the future if we are able to reduce the cloning necessary to
run a closure, then we only have to change the code related to these
types.

`ClosureEval` and `ClosureEvalOnce` are designed with a builder API.
`ClosureEval` is used to run a closure multiple times whereas
`ClosureEvalOnce` is used for a one-shot closure.

# User-Facing Changes
Should be none, unless I messed up one of the command migrations.
Actually, this will fix any unreported environment bugs for commands
that didn't reset the env after running a closure.
2024-04-22 14:15:09 +08:00
Ian Manske
83720a9f30
Make the same file error more likely to appear (#12601)
# Description
When saving to a file we currently try to check if the data source in
the pipeline metadata is the same as the file we are saving to. If so,
we create an error, since reading and writing to a file at the same time
is currently not supported/handled gracefully. However, there are still
a few instances where this error is not properly triggered, and so this
PR attempts to reduce these cases. Inspired by #12599.

# Tests + Formatting
Added a few tests.

# After Submitting
Some commands still do not properly preserve metadata (e.g., `str trim`)
and so prevent us from detecting this error.
2024-04-22 01:12:13 +00:00
Jack Wright
a60381a932
Added commands for working with the plugin cache. (#12576)
# Description
This pull request provides three new commands:
`polars store-ls` - moved from `polars ls`. It provides the list of all
object stored in the plugin cache
`polars store-rm` - deletes a cached object
`polars store-get` - gets an object from the cache. 

The addition of `polars store-get` required adding a reference_count to
cached entries. `polars get` is the only command that will increment
this value. `polars rm` will remove the value despite it's count. Calls
to PolarsPlugin::custom_value_dropped will decrement the value.

The prefix store- was chosen due to there already being a `polars cache`
command. These commands were not made sub-commands as there isn't a way
to display help for sub commands in plugins (e.g. `polars store`
displaying help) and I felt the store- seemed fine anyways.

The output of `polars store-ls` now shows the reference count for each
object.

# User-Facing Changes
polars ls has now moved to polars store-ls

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-21 19:43:43 -05:00
Devyn Cairns
aad3ac11da
update toolkit register pluginstoolkit add plugins (#12613)
# Description

Updates `toolkit.nu` to use `plugin add`.

There's no need to spawn a separate `nu` to do `register` anymore, since
`plugin add` is not a keyword
2024-04-21 18:23:12 -05:00
Devyn Cairns
8b7696f4c1
stress_internals: exit(1) on io error (#12612)
# Description

The `stress_internals` tests can fail sometimes, but usually not on the
CI, because Nushell exits while the plugin is still trying to read or
maybe write something, leading to a broken pipe.

`nu-plugin` already exits with 1 without printing a message on a
protocol-level I/O error, so this just doing the same thing.

I think there's probably a way to correct the plugin handling so that we
wait for plugins to shut down before exiting and this doesn't happen,
but this is the quick fix in the meantime.
2024-04-21 23:48:09 +02:00
Antoine Stevan
a900166e27
fix typo in the documentation of nuon::ToStyle (#12608)
follow-up to
- https://github.com/nushell/nushell/pull/12591

cc/ @fdncred 

# Description
there was a typo in the doc of `nuon::ToStyle`.

# User-Facing Changes

# Tests + Formatting

# After Submitting
2024-04-21 10:53:53 -05:00
Devyn Cairns
2595f31541
Overhaul the plugin cache file with a new msgpack+brotli format (#12579)
# Description

- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed

# User-Facing Changes

- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
  - `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`

# Tests + Formatting

Tests added for the format and for the invalid handling.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Check for documentation changes
- [ ] Definitely needs release notes
2024-04-21 07:36:26 -05:00
Stefan Holderbach
6cba7c6b40
Small refactor in cal (#12604)
Removed some unnecessary code found when looking at #12597

- **Simplify day of the week selection**
- **Use `Record` directly instead of `IndexMap`**
2024-04-21 10:41:26 +02:00
Ian Manske
3b1d405b96
Remove the Value::Block case (#12582)
# Description
`Value` describes the types of first-class values that users and scripts
can create, manipulate, pass around, and store. However, `Block`s are
not first-class values in the language, so this PR removes it from
`Value`. This removes some unnecessary code, and this change should be
invisible to the user except for the change to `scope modules` described
below.

# User-Facing Changes
Breaking change: the output of `scope modules` was changed so that
`env_block` is now `has_env_block` which is a boolean value instead of a
`Block`.

# After Submitting
Update the language guide possibly.
2024-04-21 07:03:33 +02:00
Maxim Uvarov
5fd34320e9
add search_term "str extract" to parse command (#12600)
# Description

For a long time, I was searching for the `str extract` command to
extract regexes from strings. I often painfully used `str replace -r
'(.*)(pattern_to_find)(.*)' '$2'` for such purposes.
Only this morning did I realize that `parse` is what I needed for so
many times, which I had only used for parsing data in tables.
2024-04-21 07:01:42 +02:00
woosaaahh
5e52bd77e0
Change cal --week-start examples + error message (#12597)
This PR fixes the example for `cal --week-start` and adds the list of
expected values to the error message.
2024-04-20 22:39:46 +02:00
Wind
cf8fcef9bf
set the type of default NU_LIB_DIRS and NU_PLUGIN_DIRS to list<string> (#12573)
# Description
Fix: #12489

I believe the issue it's because the default value of `NU_LIB_DIRS` and
`NU_PLUGIN_DIRS` is a string, but it should be a list.

So if users don't set up these values in `env.nu`, we will get a
problem.
2024-04-20 10:04:41 -05:00
Poliorcetics
47867a58df
Ab/version details (#12593)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Closes #12561

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Add version details in `version`'s output. The intended use is for
third-party tools to be able to quickly check version numbers without
having to the parsing from `(version).version` or `$env.NU_VERSION`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

This adds 4 new values to the record from `version`:

```
...
│ major              │ 0                                               │
│ minor              │ 92                                              │
│ patch              │ 3                                               │
│ pre                │ a-value                                         │
...
```

`pre` is optional and won't be present most of the time I think.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

I ran the new command using `cargo run -- -c version`:

```
╭────────────────────┬─────────────────────────────────────────────────╮
│ version            │ 0.92.3                                          │
│ major              │ 0                                               │
│ minor              │ 92                                              │
│ patch              │ 3                                               │
│ branch             │                                                 │
│ commit_hash        │                                                 │
│ build_os           │ macos-aarch64                                   │
│ build_target       │ aarch64-apple-darwin                            │
│ rust_version       │ rustc 1.77.2 (25ef9e3d8 2024-04-09)             │
│ rust_channel       │ 1.77.2-aarch64-apple-darwin                     │
│ cargo_version      │ cargo 1.77.2 (e52e36006 2024-03-26)             │
│ build_time         │ 2024-04-20 15:09:36 +02:00                      │
│ build_rust_channel │ release                                         │
│ allocator          │ mimalloc                                        │
│ features           │ default, sqlite, system-clipboard, trash, which │
│ installed_plugins  │                                                 │
╰────────────────────┴─────────────────────────────────────────────────╯
```

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

After this is merged I would like to write docs somewhere for scripts
writer to advise using these members instead of the string values. Where
should I put said docs ?
2024-04-20 10:01:51 -05:00
Darren Schroeder
b274ec19fd
improve nu --lsp command tooltips (#12589)
# Description

This PR improves the `nu --lsp` tooltips by using nu code blocks around
the examples and a few other places.

This is what it looks like in Zed.
![Screenshot 2024-04-19 at 8 20
53 PM](https://github.com/nushell/nushell/assets/343840/20d51dcc-f3b2-4f2b-9d43-5817dd3913df)

Here it is in Helix.

![image](https://github.com/nushell/nushell/assets/343840/a9e7d6b9-cd21-4a5a-9c88-9af17a2b2363)

This coloring is far from perfect, but it's what the tree-sitter-nu
queries generate.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-20 07:16:38 -05:00
Antoine Stevan
be5ed3290c
add "to nuon" enumeration of possible styles (#12591)
# Description
in order to change the style of the _serialized_ NUON data,
`nuon::to_nuon` takes three mutually exclusive arguments, `raw: bool`,
`tabs: Option<usize>` and `indent: Option<usize>` 🤔
this begs to use an enumeration with all possible alternatives, right?

this PR changes the signature of `nuon::to_nuon` to use `nuon::ToStyle`
which has three variants
- `Raw`: no newlines
- `Tabs(n: usize)`: newlines and `n` tabulations as indent
- `Spaces(n: usize)`: newlines and `n` spaces as indent

# User-Facing Changes
the signature of `nuon::to_nuon` changes from
```rust
to_nuon(
    input: &Value,
    raw: bool,
    tabs: Option<usize>,
    indent: Option<usize>,
    span: Option<Span>,
) -> Result<String, ShellError>
```
to
```rust
to_nuon(
    input: &Value,
    style: ToStyle,
    span: Option<Span>
) -> Result<String, ShellError>
```

# Tests + Formatting

# After Submitting
2024-04-20 11:40:52 +02:00
Wind
187b87c61c
Don't allow skip on external stream (#12559)
# Description
Close: #12514

# User-Facing Changes
`^ls | skip 1` will raise an error
```nushell
❯ ^ls | skip 1
Error: nu:🐚:only_supports_this_input_type

  × Input type not supported.
   ╭─[entry #1:1:2]
 1 │ ^ls | skip 1
   ·  ─┬   ──┬─
   ·   │     ╰── only list, binary or range input data is supported
   ·   ╰── input type: raw data
   ╰────
```

# Tests + Formatting
Sorry I can't add it because of the issue:
https://github.com/nushell/nushell/issues/12558

# After Submitting
Nan
2024-04-19 14:54:59 +00:00
Devyn Cairns
f2169c8022
Switch plugin msgpack protocol to named format (#12580)
# Description
In conflict with the documentation, the msgpack serializer for plugins
is actually using the compact format, which doesn't name struct fields,
and is instead dependent on their ordering, rendering them as tuples.
This is not a good idea for a robust protocol even if it makes the
serialization and deserialization faster.

I expect this to have some impact on performance, but I think the
robustness is probably worth it.

Deserialization always accepts either format, so this shouldn't cause
too many incompatibilities.

# User-Facing Changes
This does technically change the protocol, but it makes it reflect the
documentation. It shouldn't break deserialization, so plugins shouldn't
necessarily need a recompile.

Performance is likely worse and I should benchmark the difference.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-19 07:50:51 -05:00
Jack Wright
9fb59a6f43
Removed the polars dtypes command (#12577)
# Description
The polars dtype command is largerly redundant since the introduction of
the schema command. The schema command also has the added benefit that
it's output can be used as a parameter to other schema commands:

```nushell
[[a b]; [5 6] [5 7]] | polars into-df -s ($df | polars schema
```

# User-Facing Changes
`polars dtypes` has been removed. Users should use `polars schema`
instead.

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-19 07:01:47 -05:00
Antoine Stevan
55edef5dda
create nuon crate from from nuon and to nuon (#12553)
# Description
playing with the NUON format in Rust code in some plugins, we agreed
with the team it was a great time to create a standalone NUON format to
allow Rust devs to use this Nushell file format.

> **Note**
> this PR almost copy-pastes the code from
`nu_commands/src/formats/from/nuon.rs` and
`nu_commands/src/formats/to/nuon.rs` to `nuon/src/from.rs` and
`nuon/src/to.rs`, with minor tweaks to make then standalone functions,
e.g. remove the rest of the command implementations

### TODO
- [x] add tests
- [x] add documentation

# User-Facing Changes
devs will have access to a new crate, `nuon`, and two functions,
`from_nuon` and `to_nuon`
```rust
from_nuon(
    input: &str,
    span: Option<Span>,
) -> Result<Value, ShellError>
```
```rust
to_nuon(
    input: &Value,
    raw: bool,
    tabs: Option<usize>,
    indent: Option<usize>,
    span: Option<Span>,
) -> Result<String, ShellError>
```

# Tests + Formatting
i've basically taken all the tests from
`crates/nu-command/tests/format_conversions/nuon.rs` and converted them
to use `from_nuon` and `to_nuon` instead of Nushell commands
- i've created a `nuon_end_to_end` to run both conversions with an
optional middle value to check that all is fine

> **Note** 
> the `nuon::tests::read_code_should_fail_rather_than_panic` test does
give different results locally and in the CI...
> i've left it ignored with comments to help future us :)

# After Submitting
mention that in the release notes for sure!!
2024-04-19 13:54:16 +02:00
Devyn Cairns
fac2f43aa4
Add an example Nushell plugin written in Nushell itself (#12574)
# Description

As suggested by @fdncred.

It's neat that this is possible, but the particularly useful part of
this is that we can actually
test it because it doesn't have any external dependencies, unlike the
python plugin.

Right now this just implements exactly the same behavior as the python
plugin, but we could have it
exercise a few more things.

Also fixes a couple of bugs:

- `.nu` plugins were not run with `nu --stdin`, so they couldn't take
input.
- `register` couldn't be called if `--no-config-file` was set, because
it would error on trying to
  update the plugin file.

# User-Facing Changes

- `nu_plugin_nu_example` plugin added.
- `register` now works in `--no-config-file` mode.

# Tests + Formatting
Tests added for `nu_plugin_nu_example`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Add the version bump to the release script just like for python
2024-04-19 09:53:30 +03:00
YizhePKU
6d2cb4382a
Fix circular source causing Nushell to crash (#12262)
# Description

EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).

Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).

Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->


<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

It shouldn't have any user facing changes.
2024-04-19 09:38:08 +03:00
Brooks Rady
351bff8233
fix(shell_integration): set window title on startup (#12569)
Should close #10833 — though I'd imagine that should have already been
closed.

# Description

Very minor tweak, but it was quite noticeable when using Zellij which
relies on OSC 2 to set pane titles. Before the change:

![image](https://github.com/nushell/nushell/assets/6251883/b944bbce-2040-4886-9955-3c5b57d368e9)

Note that the default `Pane #1` is still showing for the untouched
shell, but running a command like `htop` or `ls` correctly sets the
title during / afterwards.

After this PR:

![image](https://github.com/nushell/nushell/assets/6251883/dd513cfe-923c-450f-b0f2-c66938b0d6f0)

There are now no-longer any unset titles — even if the shell hasn't been
touched.

**As an aside:** I feel quite strongly that (at least OSC 2) shell
integration should be enabled by default, as it is for every other Linux
shell I've used, but I'm not sure which issues that caused that the
default config refers to? Which terminals are broken by shell
integration, and could some of the shell integrations be turned on by
default after splitting things into sub-options as suggested in #11301 ?

# User-Facing Changes

You'll just have shell integrations working from right after the shell
has been launched, instead of needing to run something first.

# Tests + Formatting

Not quite sure how to test this one? Are there any other tests that
currently exist for shell integration? I couldn't quite track them
down...

# After Submitting

Let me know if you think this needs any user-facing docs changes!
2024-04-18 20:30:37 -05:00
Darren Schroeder
7fe2e60af7
add ability to set metadata (#12564)
# Description

This PR adds the ability to set metadata. This is especially useful for
activating LS_COLORS when using table literals.


![image](https://github.com/nushell/nushell/assets/343840/feef6433-f592-43ea-890a-38cb2df35686)

You can also set the filepath metadata, although I'm not really user how
useful this is. We may end up removing this option entirely.
```nushell
❯ "crates" | metadata set --datasource-filepath $'(pwd)/crates' | metadata
╭────────┬───────────────────────────────────╮
│ source │ /Users/fdncred/src/nushell/crates │
╰────────┴───────────────────────────────────╯
```

No file paths are checked. You could also do this.
```nushell
❯ "crates" | metadata set --datasource-filepath $'a/b/c/d/crates' | metadata
╭────────┬────────────────╮
│ source │ a/b/c/d/crates │
╰────────┴────────────────╯
```

The command name and parameter names are still WIP. We could change
them.

There are currently 3 kinds of metadata in nushell.
```rust
pub enum DataSource {
    Ls,
    HtmlThemes,
    FilePath(PathBuf),
}
```

I've skipped adding `HtmlThemes` because it seems to be specific to our
`to html` command only.
2024-04-19 09:03:59 +08:00
Devyn Cairns
999dfdf936
Fix the error output in the python plugin to match LabeledError (#12575)
# Description

Forgot to update this after updating the format of `LabeledError`.

# Tests + Formatting

Not applicable to the python plugin.
2024-04-18 20:01:35 -05:00
Jack Wright
cc7b5c5a26
Only mark collected dataframes as from_lazy=false when collect is called from the collect command. (#12571)
I had previously changed NuLazyFrame::collect to set the NuDataFrame's
from_lazy field to false to prevent conversion back to a lazy frame. It
appears there are cases where this should happen. Instead, I am only
setting from_lazy=false inside the `polars collect` command.

[Related discord
message](https://discord.com/channels/601130461678272522/1227612017171501136/1230600465159421993)

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-18 17:10:38 -05:00
Ian Manske
9a265847e2
Box ImportPattern in Expr (#12568)
# Description
Adds a `Box` around the `ImportPattern` in `Expr` which decreases the
size of `Expr` from 152 to 64 bytes (and `Expression` from 216 to 128
bytes). This seems to speed up parsing a little bit according to the
benchmarks (main is top, PR is bottom):
```
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ parser_benchmarks                           │               │               │               │         │
├─ parser_benchmarks                           │               │               │               │         │
│  ├─ parse_default_config_file  2.287 ms      │ 4.532 ms      │ 2.311 ms      │ 2.437 ms      │ 100     │ 100
│  ├─ parse_default_config_file  2.255 ms      │ 2.781 ms      │ 2.281 ms      │ 2.312 ms      │ 100     │ 100
│  ╰─ parse_default_env_file     421.8 µs      │ 824.6 µs      │ 494.3 µs      │ 527.5 µs      │ 100     │ 100
│  ╰─ parse_default_env_file     402 µs        │ 486.6 µs      │ 414.8 µs      │ 416.2 µs      │ 100     │ 100

```
2024-04-18 17:57:01 +02:00
Stefan Holderbach
b088f395dc
Update crate feature flags (#12566)
# Description
Remove unused/effect-less features, make sure we show all relevant
features in `version`

# User-Facing Changes
- **Remove unused feature `wasi`**
- will cause failure to build should you enable it. Otherwise no effect
- **Include feat `system-clipboard` in `version`**
2024-04-18 16:33:41 +02:00
Ian Manske
6ccd547d81
Add ListItem type for Expr::List (#12529)
# Description
This PR adds a `ListItem` enum to our set of AST types. It encodes the
two possible expressions inside of list expression: a singular item or a
spread. This is similar to the existing `RecordItem` enum. Adding
`ListItem` allows us to remove the existing `Expr::Spread` case which
was previously used for list spreads. As a consequence, this guarantees
(via the type system) that spreads can only ever occur inside lists,
records, or as command args.

This PR also does a little bit of cleanup in relevant parser code.
2024-04-18 13:21:05 +02:00
Jack Wright
57b0c722c6
Upgrading nu-cmd-dataframe to polars 0.39 (#12554)
#Description
Upgrading nu-cmd-dataframe to polars 0.39

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-17 12:50:17 -05:00
Devyn Cairns
13160b3ec3
Replace subtraction of Instants and Durations with saturating subtractions (#12549)
# Description
Duration can not be negative, and an underflow causes a panic.

This should fix #12539 as from what I can tell that bug was caused in
`nu-explore:📟:events` from subtracting durations, but I figured
this might be more widespread, and saturating to zero generally makes
sense.

I also added the relevant clippy lint to try to prevent this from
happening in the future. I can't think of a reason we would ever want to
subtract durations without checking first.

cc @fdncred

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-17 07:25:16 -05:00
Jack Wright
410f3c5c8a
Upgrading nu_plugin_polars to polars 0.39.1 (#12551)
# Description
Upgrading nu_plugin_polars to polars 0.39.1

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-17 06:35:09 -05:00
Devyn Cairns
b296d6ee3c
Improve safety of get_unchecked_str in nu_system::macos (#12550)
# Description
The implementation of this function had a few issues before:

- It didn't check that the `cp` pointer is actually ahead of the `start`
pointer, so `len` could potentially underflow and wrap around, which
would be a violation of memory safety
- It used `Vec::from_raw_parts` even though the buffer is borrowed, not
owned. Although `std::mem::forget` is used later to ensure the
destructor doesn't run, there is a risk that the destructor would run if
a panic happened during `String::from_utf8_unchecked`, which would lead
to a `free()` of a pointer we don't own
2024-04-17 17:10:05 +08:00
dependabot[bot]
9a739d9f0d
Bump rmp-serde from 1.1.2 to 1.2.0 (#12547)
Bumps [rmp-serde](https://github.com/3Hren/msgpack-rust) from 1.1.2 to
1.2.0.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/3Hren/msgpack-rust/commits/rmp-serde/v1.2.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rmp-serde&package-manager=cargo&previous-version=1.1.2&new-version=1.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-17 16:44:54 +08:00
dependabot[bot]
8cba6678b3
Bump crate-ci/typos from 1.20.7 to 1.20.9 (#12543) 2024-04-17 01:26:46 +00:00
Stefan Holderbach
055aae9f5c
Impl FusedIterator for record iterators (#12542)
This is good practice as all our iterators will never return a value
after reaching `None`

The benefit should be minimal as only `Iterator::fuse` is directly
specialized and itself rarely used (sometimes in `itertools` adaptors)

Thus it is mostly a documentation thing
2024-04-17 00:34:16 +02:00
Stefan Holderbach
a67dad3d15
Minor housekeeping in the parser (#12540)
- **Move lone `check_name` to the alias place**
- **Restrict visibility of `check_call` helper**
2024-04-17 00:33:50 +02:00
pwygab
43814dcb0f
use abbreviated string instead of debug string for DatetimeParseErrors (#12517)
Resolves #12444. Prevents debug string from being printed out.

---------

Co-authored-by: sholderbach <sholderbach@users.noreply.github.com>
2024-04-16 23:19:03 +02:00
Ian Manske
cc781a1ecd
Make group-by return errors in closure (#12508)
# Description
When a closure if provided to `group-by`, errors that occur in the
closure are currently ignored. That is, `group-by` will fall back and
use the `"error"` key if an error occurs. For example, the code snippet
below will group all `ls` entries under the `"error"` column.
```nushell
ls | group-by { get nope } 
```

This PR changes `group-by` to instead bubble up any errors triggered
inside the closure. In addition, this PR also does some refactoring and
cleanup inside `group-by`.

# User-Facing Changes
Errors are now returned from the closure provided to `group-by` instead
of falling back to the `"error"` group/key.
2024-04-16 21:52:21 +02:00
Jack Wright
a7a5ec31be
Fixing NuLazyFrame/NuDataFrame conversion issues (#12538)
# Description

@maxim-uvarov brought up another case where converting back and forth
between eager and lazy dataframes was not working correctly:

```
> [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars append -c ([[a b]; [6 2] [1 4] [4 1]] | polars into-df)
Error: nu:🐚:cant_convert

  × Can't convert to NuDataFrame.
   ╭─[entry #1:1:49]
 1 │ [[a b]; [6 2] [1 4] [4 1]] | polars into-lazy | polars append -c ([[a b]; [6 2] [1 4] [4 1]] | polars into-df)
   ·                                                 ──────┬──────
   ·                                                       ╰── can't convert NuLazyFrameCustomValue to NuDataFrame
   ╰────
```

This pull request fixes this case and glaringly obvious similar cases I
could find.

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-16 11:16:37 -05:00
Devyn Cairns
48e4448e55
Add a panic unwind handler during plugin calls (#12526)
# Description
If a panic happens during a plugin call, because it always happens
outside of the main thread, it currently just hangs Nushell because the
plugin stays running without ever producing a response to the call.

This adds a panic handler that calls `exit(1)` after the unwind finishes
to the plugin runner. The panic error is still printed to stderr as
always, and waiting for the unwind to finish helps to ensure that
anything on the stack with `Drop` behavior that needed to run still
runs, at least on that thread.

# User-Facing Changes
Panics now look like this, which is what they looked like before the
plugin behavior was moved to a separate thread:

```
thread 'plugin runner (primary)' panicked at crates/nu_plugin_example/src/commands/main.rs:45:9:
Test panic
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: nu:🐚:plugin_failed_to_decode

  × Plugin failed to decode: Failed to receive response to plugin call

```

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-16 15:00:32 +00:00
Wind
e6fbf7d01d
Unify working_set.error usage. (#12531)
# Description
A little refactor that use `working_set.error` rather than
`working_set.parse_errors.push`, which is reported here:
https://github.com/nushell/nushell/pull/12238

> Inconsistent error reporting. Usage of both working_set.error() and
working_set.parse_errors.push(). Using ParseError::Expected for an
invalid variable name when there's ParseError::VariableNotValid (from
parser.rs:5237). Checking variable names manually when there's
is_variable() (from parser.rs:2905).

# User-Facing Changes
NaN

# Tests + Formatting
Done
2024-04-16 15:47:10 +02:00
Wind
62555c997e
remove useless path.rs (#12534)
# Description
Sorry for introducing a useless file in previous pr, I have renamed it
from `glob.rs` to `path.rs`, but forget removing it.
2024-04-16 06:41:43 -05:00
Devyn Cairns
03317ff92e
Python plugin: remove unnecessary fields from signature (#12533)
# Description
Remove a couple of legacy fields (`input_type`, `output_type`), and
`var_id` which is optional and not required for deserialization.

I think until I document this in the plugin protocol ref, most people
will probably be using this example to get started, so it should be as
correct as possible

# After Submitting
- [ ] TODO: document `Signature` in plugin protocol reference
2024-04-16 06:40:04 -05:00
Jack Wright
1661bb68f9
Cleaning up to_pipe_line_data and cache_and_to_value, making them part of CustomValueSupport (#12528)
# Description

This is just some cleanup. I moved to_pipeline_data and to_cache_value
to the CustomValueSupport trait, where I should've put them to begin
with.

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-04-16 06:35:52 -05:00