Commit Graph

1408 Commits

Author SHA1 Message Date
YizhePKU
52d99cc60c
Change environment variables to be case-preserving (#12701)
This PR changes `$env` to be **case-preserving** instead of
case-sensitive. That is, it preserves the case of the environment
variable when it is first assigned, but subsequent retrieval and update
ignores the case.

Notably, both `$env.PATH` and `$env.Path` can now be used to read or set
the environment variable, but child processes will always see the
correct case based on the platform.

Fixes #11268.

---

This feature was surprising simple to implement, because most of the
infrastructure to support case-insensitive cell path access already
exists. The `get` command extracts data using a cell path in a
case-insensitive way (!), but accepts a `--sensitive` flag. (I think
this should be flipped around?)
2024-05-01 17:22:34 -05:00
Devyn Cairns
21ebdfe8d7
Bump version to 0.93.1 (#12710)
# Description

Next patch/dev release, `0.93.1`
2024-05-01 17:19:20 -05:00
Devyn Cairns
3b220e07e3
Bump version to 0.93.0 (#12709)
# Description

Bump version to `0.93.0`
2024-04-30 15:51:13 -07:00
Devyn Cairns
02de69de92
Fix inconsistent print behavior (#12675)
# Description

I found a bunch of issues relating to the specialized reimplementation
of `print()` that's done in `nu-cli` and it just didn't seem necessary.
So I tried to unify the behavior reasonably. `PipelineData::print()`
already handles the call to `table` and it even has a `no_newline`
option.

One of the most major issues before was that we were using the value
iterator, and then converting to string, and then printing each with
newlines. This doesn't work well for an external stream, because its
iterator ends up creating `Value::binary()` with each buffer... so we
were doing lossy UTF-8 conversion on those and then printing them with
newlines, which was very weird:


![Screenshot_2024-04-26_02-02-29](https://github.com/nushell/nushell/assets/10729/131c2224-08ee-4582-8617-6ecbb3ce8da5)

You can see the random newline inserted in a break between buffers, but
this would be even worse if it were on a multibyte UTF-8 character. You
can produce this by writing a large amount of text to a text file, and
then doing `nu -c 'open file.txt'` - in my case I just wrote `^find .`;
it just has to be large enough to trigger a buffer break.

Using `print()` instead led to a new issue though, because it doesn't
abort on errors. This is so that certain commands can produce a stream
of errors and have those all printed. There are tests for e.g. `rm` that
depend on this behavior. I assume we want to keep that, so instead I
made my target `BufferedReader`, and had that fuse closed if an error
was encountered. I can't imagine we want to keep reading from a wrapped
I/O stream if an error occurs; more often than not the error isn't going
to magically resolve itself, it's not going to be a different error each
time, and it's just going to lead to an infinite stream of the same
error.

The test that broke without that was `open . | lines`, because `lines`
doesn't fuse closed on error. But I don't know if it's expected or not
for it to do that, so I didn't target that.

I think this PR makes things better but I'll keep looking for ways to
improve on how errors and streams interact, especially trying to
eliminate cases where infinite error loops can happen.

# User-Facing Changes
- **Breaking**: `BufferedReader` changes + no more public fields
- A raw I/O stream from e.g. `open` won't produce infinite errors
anymore, but I consider that to be a plus
- the implicit `print` on script output is the same as the normal one
now

# Tests + Formatting
Everything passes but I didn't add anything specific.
2024-04-27 00:25:11 +00:00
Devyn Cairns
adf38c7c76
Msgpack commands (#12664)
# Description

I thought about bringing `nu_plugin_msgpack` in, but that is MPL with a
clause that prevents other licenses, so rather than adapt that code I
decided to take a crack at just doing it straight from `rmp` to `Value`
without any `rmpv` in the middle. It seems like it's probably faster,
though I can't say for sure how much with the plugin overhead.

@IanManske I started on a `Read` implementation for `RawStream` but just
specialized to `from msgpack` here, but I'm thinking after release maybe
we can polish it up and make it a real one. It works!

# User-Facing Changes
New commands:

- `from msgpack`
- `from msgpackz`
- `to msgpack`
- `to msgpackz`

# Tests + Formatting
Pretty thorough tests added for the format deserialization, with a
roundtrip for serialization. Some example tests too for both `from
msgpack` and `to msgpack`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
- [ ] update release notes
2024-04-26 06:23:16 -05:00
Ian Manske
b6d765a2d8
nu-cmd-lang cleanup (#12609)
# Description
This PR does miscellaneous cleanup in some of the commands from
`nu-cmd-lang`.

# User-Facing Changes
None.

# After Submitting
Cleanup the other commands in `nu-cmd-lang`.
2024-04-25 14:16:12 +00:00
Devyn Cairns
25cbcb511d
Rename plugin cache file ⇒ plugin registry file (#12634)
# Description
So far this seems like the winner of my poll on what the name should be.
I'll take this off draft once the poll expires, if this is indeed the
winner.
2024-04-24 17:40:39 -05:00
Ian Manske
9996e4a1f8
Shrink the size of Expr (#12610)
# Description
Continuing from #12568, this PR further reduces the size of `Expr` from
64 to 40 bytes. It also reduces `Expression` from 128 to 96 bytes and
`Type` from 32 to 24 bytes.

This was accomplished by:
- for `Expr` with multiple fields (e.g., `Expr::Thing(A, B, C)`),
merging the fields into new AST struct types and then boxing this struct
(e.g. `Expr::Thing(Box<ABC>)`).
- replacing `Vec<T>` with `Box<[T]>` in multiple places. `Expr`s and
`Expression`s should rarely be mutated, if at all, so this optimization
makes sense.

By reducing the size of these types, I didn't notice a large performance
improvement (at least compared to #12568). But this PR does reduce the
memory usage of nushell. My config is somewhat light so I only noticed a
difference of 1.4MiB (38.9MiB vs 37.5MiB).

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-24 15:46:35 +00:00
Devyn Cairns
b576123b0a
Accept filenames in other plugin management commands (#12639)
# Description

This allows the following commands to all accept a filename instead of a
plugin name:

- `plugin use`
- `plugin rm`
- `plugin stop`

Slightly complicated because of the need to also check against
`NU_PLUGIN_DIRS`, but I also fixed some issues with that at the same
time

Requested by @fdncred

# User-Facing Changes

The new commands are updated as described.

# Tests + Formatting

Tests for `NU_PLUGIN_DIRS` handling also made more robust.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Double check new docs to make sure they describe this capability
2024-04-24 06:28:45 -05:00
Devyn Cairns
1f4131532d
Deprecate register and add plugin use (#12607)
# Description

Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:

```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```

This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.

The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.

`register` now prints a deprecation parse warning.

This should fix #11923, as we now have a complete alternative to
`register`.

# User-Facing Changes

- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`

# Tests + Formatting

I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Update plugins sections of book
- [ ] Release notes
2024-04-23 06:37:50 -05:00
Devyn Cairns
5c7f7883c8
Add ErrSpan extension trait for Result (#12626)
# Description
This adds an extension trait to `Result` that wraps errors in `Spanned`,
saving the effort of calling `.map_err(|err| err.into_spanned(span))`
every time. This will hopefully make it even more likely that someone
will want to use a spanned `io::Error` and make it easier to remove the
impl for `From<io::Error> for ShellError` because that doesn't have span
information.
2024-04-23 10:39:55 +02:00
Devyn Cairns
2595f31541
Overhaul the plugin cache file with a new msgpack+brotli format (#12579)
# Description

- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed

# User-Facing Changes

- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
  - `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`

# Tests + Formatting

Tests added for the format and for the invalid handling.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Check for documentation changes
- [ ] Definitely needs release notes
2024-04-21 07:36:26 -05:00
Ian Manske
3b1d405b96
Remove the Value::Block case (#12582)
# Description
`Value` describes the types of first-class values that users and scripts
can create, manipulate, pass around, and store. However, `Block`s are
not first-class values in the language, so this PR removes it from
`Value`. This removes some unnecessary code, and this change should be
invisible to the user except for the change to `scope modules` described
below.

# User-Facing Changes
Breaking change: the output of `scope modules` was changed so that
`env_block` is now `has_env_block` which is a boolean value instead of a
`Block`.

# After Submitting
Update the language guide possibly.
2024-04-21 07:03:33 +02:00
Devyn Cairns
fac2f43aa4
Add an example Nushell plugin written in Nushell itself (#12574)
# Description

As suggested by @fdncred.

It's neat that this is possible, but the particularly useful part of
this is that we can actually
test it because it doesn't have any external dependencies, unlike the
python plugin.

Right now this just implements exactly the same behavior as the python
plugin, but we could have it
exercise a few more things.

Also fixes a couple of bugs:

- `.nu` plugins were not run with `nu --stdin`, so they couldn't take
input.
- `register` couldn't be called if `--no-config-file` was set, because
it would error on trying to
  update the plugin file.

# User-Facing Changes

- `nu_plugin_nu_example` plugin added.
- `register` now works in `--no-config-file` mode.

# Tests + Formatting
Tests added for `nu_plugin_nu_example`.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Add the version bump to the release script just like for python
2024-04-19 09:53:30 +03:00
YizhePKU
6d2cb4382a
Fix circular source causing Nushell to crash (#12262)
# Description

EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).

Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).

Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->


<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

It shouldn't have any user facing changes.
2024-04-19 09:38:08 +03:00
Ian Manske
9a265847e2
Box ImportPattern in Expr (#12568)
# Description
Adds a `Box` around the `ImportPattern` in `Expr` which decreases the
size of `Expr` from 152 to 64 bytes (and `Expression` from 216 to 128
bytes). This seems to speed up parsing a little bit according to the
benchmarks (main is top, PR is bottom):
```
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
benchmarks                       fastest       │ slowest       │ median        │ mean          │ samples │ iters
├─ parser_benchmarks                           │               │               │               │         │
├─ parser_benchmarks                           │               │               │               │         │
│  ├─ parse_default_config_file  2.287 ms      │ 4.532 ms      │ 2.311 ms      │ 2.437 ms      │ 100     │ 100
│  ├─ parse_default_config_file  2.255 ms      │ 2.781 ms      │ 2.281 ms      │ 2.312 ms      │ 100     │ 100
│  ╰─ parse_default_env_file     421.8 µs      │ 824.6 µs      │ 494.3 µs      │ 527.5 µs      │ 100     │ 100
│  ╰─ parse_default_env_file     402 µs        │ 486.6 µs      │ 414.8 µs      │ 416.2 µs      │ 100     │ 100

```
2024-04-18 17:57:01 +02:00
Ian Manske
6ccd547d81
Add ListItem type for Expr::List (#12529)
# Description
This PR adds a `ListItem` enum to our set of AST types. It encodes the
two possible expressions inside of list expression: a singular item or a
spread. This is similar to the existing `RecordItem` enum. Adding
`ListItem` allows us to remove the existing `Expr::Spread` case which
was previously used for list spreads. As a consequence, this guarantees
(via the type system) that spreads can only ever occur inside lists,
records, or as command args.

This PR also does a little bit of cleanup in relevant parser code.
2024-04-18 13:21:05 +02:00
Stefan Holderbach
055aae9f5c
Impl FusedIterator for record iterators (#12542)
This is good practice as all our iterators will never return a value
after reaching `None`

The benefit should be minimal as only `Iterator::fuse` is directly
specialized and itself rarely used (sometimes in `itertools` adaptors)

Thus it is mostly a documentation thing
2024-04-17 00:34:16 +02:00
Wind
62555c997e
remove useless path.rs (#12534)
# Description
Sorry for introducing a useless file in previous pr, I have renamed it
from `glob.rs` to `path.rs`, but forget removing it.
2024-04-16 06:41:43 -05:00
Devyn Cairns
c06ef201b7
Local socket mode and foreground terminal control for plugins (#12448)
# Description

Adds support for running plugins using local socket communication
instead of stdio. This will be an optional thing that not all plugins
have to support.

This frees up stdio for use to make plugins that use stdio to create
terminal UIs, cc @amtoine, @fdncred.

This uses the [`interprocess`](https://crates.io/crates/interprocess)
crate (298 stars, MIT license, actively maintained), which seems to be
the best option for cross-platform local socket support in Rust. On
Windows, a local socket name is provided. On Unixes, it's a path. The
socket name is kept to a relatively small size because some operating
systems have pretty strict limits on the whole path (~100 chars), so on
macOS for example we prefer `/tmp/nu.{pid}.{hash64}.sock` where the hash
includes the plugin filename and timestamp to be unique enough.

This also adds an API for moving plugins in and out of the foreground
group, which is relevant for Unixes where direct terminal control
depends on that.

TODO:

- [x] Generate local socket path according to OS conventions
- [x] Add support for passing `--local-socket` to the plugin executable
instead of `--stdio`, and communicating over that instead
- [x] Test plugins that were broken, including
[amtoine/nu_plugin_explore](https://github.com/amtoine/nu_plugin_explore)
- [x] Automatically upgrade to using local sockets when supported,
falling back if it doesn't work, transparently to the user without any
visible error messages
  - Added protocol feature: `LocalSocket`
- [x] Reset preferred mode to `None` on `register`
- [x] Allow plugins to detect whether they're running on a local socket
and can use stdio freely, so that TUI plugins can just produce an error
message otherwise
  - Implemented via `EngineInterface::is_using_stdio()`
- [x] Clean up foreground state when plugin command exits on the engine
side too, not just whole plugin
- [x] Make sure tests for failure cases work as intended
  - `nu_plugin_stress_internals` added

# User-Facing Changes
- TUI plugins work
- Non-Rust plugins could optionally choose to use this
- This might behave differently, so will need to test it carefully
across different operating systems

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] Document local socket option in plugin contrib docs
- [ ] Document how to do a terminal UI plugin in plugin contrib docs
- [ ] Document: `EnterForeground` engine call
- [ ] Document: `LeaveForeground` engine call
- [ ] Document: `LocalSocket` protocol feature
2024-04-15 18:28:18 +00:00
Ian Manske
67e7eec7da
Add Record::into_columns (#12324)
# Description
Add `Record::into_columns` to complement `Record::columns` and
`Record::into_values`.
2024-04-14 22:43:47 +02:00
Devyn Cairns
2ae9ad8676
Copy-on-write for record values (#12305)
# Description
This adds a `SharedCow` type as a transparent copy-on-write pointer that
clones to unique on mutate.

As an initial test, the `Record` within `Value::Record` is shared.

There are some pretty big wins for performance. I'll post benchmark
results in a comment. The biggest winner is nested access, as that would
have cloned the records for each cell path follow before and it doesn't
have to anymore.

The reusability of the `SharedCow` type is nice and I think it could be
used to clean up the previous work I did with `Arc` in `EngineState`.
It's meant to be a mostly transparent clone-on-write that just clones on
`.to_mut()` or `.into_owned()` if there are actually multiple
references, but avoids cloning if the reference is unique.

# User-Facing Changes
- `Value::Record` field is a different type (plugin authors)

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] use for `EngineState`
- [ ] use for `Value::List`
2024-04-14 01:42:03 +00:00
Stefan Holderbach
872945ae8e
Bump version to 0.92.3 (#12476) 2024-04-12 08:00:43 -05:00
Ian Manske
d7ba8872bf
Rename IoStream to OutDest (#12433)
# Description
I spent a while trying to come up with a good name for what is currently
`IoStream`. Looking back, this name is not the best, because it:
1. Implies that it is a stream, when it all it really does is specify
the output destination for a stream/pipeline.
2. Implies that it handles input and output, when it really only handles
output.

So, this PR renames `IoStream` to `OutDest` instead, which should be
more clear.
2024-04-09 16:48:32 +00:00
Texas Toland
6536fa5ff7
Ensure currently_parsed_cwd is set for config files (#12338)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes #7849, #11465 based on @kubouch's suggestion in
https://github.com/nushell/nushell/issues/11465#issuecomment-1883847806.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Can source files relative to `env.nu` or `config.nu` like in #6150.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Adds test that previously failed.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-09 10:06:41 -04:00
Stefan Holderbach
01c79bbefc
Implement De-/Serialize for Record manually (#12365)
# Description
This decouples the serialized representation of `Record` from its
internal implementation. It now gets treated as a map type in `serde`.

This has several benefits:
- more efficient representation (not showing inner fields)
- human readable e.g. as a JSON record
- no breaking changes when refactoring the `Record` internals in the
future (see #12326, or potential introduction of `indexmap::IndexMap`
for large N)
- we now deny the creation of invalid records a non-cooperating plugin
could produce
  - guaranteed key-value correspondence
  - checking for unique keys

# Breaking change to the plugin protocol:
Now expects a record/map directly as the `Record.val` field instead of a
serialization of it.
2024-04-07 07:21:03 +08:00
Ian Manske
03667bdf8c
Fix merging child stack into parent (#12426)
# Description
Fixes #12423 where changes to mutable variables are not properly
persisted after a REPL entry.
2024-04-06 15:03:22 +00:00
Ian Manske
7a7d43344e
Range refactor (#12405)
# Description
Currently, `Range` is a struct with a `from`, `to`, and `incr` field,
which are all type `Value`. This PR changes `Range` to be an enum over
`IntRange` and `FloatRange` for better type safety / stronger compile
time guarantees.

Fixes: #11778 Fixes: #11777 Fixes: #11776 Fixes: #11775 Fixes: #11774
Fixes: #11773 Fixes: #11769.

# User-Facing Changes
Hopefully none, besides bug fixes.

Although, the `serde` representation might have changed.
2024-04-06 09:04:56 -05:00
Devyn Cairns
2562e306b6
Improve handling of custom values in plugin examples (#12409)
# Description
Requested by @ayax79. This makes the custom value behavior more correct,
by calling the methods on the plugin to handle the custom values in
examples rather than the methods on the custom values themselves. This
helps for handle-type custom values (like what he's doing with
dataframes).

- Equality checking in `PluginTest::test_examples()` changed to use
`PluginInterface::custom_value_partial_cmp()`
- Base value rendering for `PluginSignature` changed to use
`Plugin::custom_value_to_base_value()`
- Had to be moved closer to `serve_plugin` for this reason, so the test
for writing signatures containing custom values was removed
- That behavior should still be tested to some degree, since if custom
values are not handled, signatures will fail to parse, so all of the
other tests won't work.

# User-Facing Changes

- `Record::sort_cols()` method added to share functionality required by
`PartialCmp`, and it might also be slightly faster
- Otherwise, everything should mostly be the same but better. Plugins
that don't implement special handling for custom values will still work
the same way, because the default implementation is just a pass-through
to the `CustomValue` methods.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-05 21:57:20 -05:00
Stefan Holderbach
394487b3a7
Bump version to 0.92.2 (#12402) 2024-04-05 10:24:00 -04:00
Stefan Holderbach
c00a05a762
Bump version to 0.92.1 (#12380) 2024-04-04 16:18:54 +02:00
Stefan Holderbach
e810995cf8
Bump crate-ci/typos and fix typos (#12381)
Supersede #12376
2024-04-04 09:59:21 +02:00
Ian Manske
cd00a489af
Fix hooks on 0.92.0 (#12383)
# Description
Fixes #12382, where overlay changes from hooks were not preserved into
the global state. This was due to creating child stacks for hooks, when
the global stack should have been used instead.
2024-04-04 09:25:54 +02:00
Devyn Cairns
cbf7feef22
Make drop notification timing for plugin custom values more consistent (#12341)
# Description
This keeps plugin custom values that have requested drop notification
around during the lifetime of a plugin call / stream by sending them to
a channel that gets persisted during the lifetime of the call.

Before this change, it was very likely that the drop notification would
be sent before the plugin ever had a chance to handle the value it
received.

Tests have been added to make sure this works - see the `custom_values`
plugin.

cc @ayax79 

# User-Facing Changes
This is basically just a bugfix, just a slightly big one.

However, I did add an `as_mut_any()` function for custom values, to
avoid having to clone them. This is a breaking change.
2024-04-04 09:13:25 +02:00
Stefan Holderbach
c3428b891a
Bump version for 0.92.0 release (#12349)
- [x] `cargo hack` feature flag compatibility run
- [x] reedline released and pinned
- [x] `nu-plugin-test-support` added to release script
- [x] dependency tree checked
- [x] release notes
2024-04-02 20:50:26 +03:00
Stefan Holderbach
0cf7de598b
Refactor Record to use a single backing Vec (#12326)
# Description
This shrinks `Record`'s size in half and and allows you to include it in
`Value` without growing the size.

Changing the `Record` internals may have slightly different performance
characteristics as the cache locality changes on lookups (if you
directly need the value, it should be closer, but in other cases may
blow up the cache line budget)

Also different perf characteristics on creation expected. 
`Record::from_raw_cols_vals` now probably worse.

## Benchmarking

Comparison with the main branch (boxed Record) revealed no significant
change to the creation but an improvement when accessing larger N.
The fact that this was more pronounced for nested access (still cloning
before nushell/nushell#12325) leads to the conclusion that this may
still be dominated by the smaller clone necessary for a 24-byte `Record`
over the previous 48 bytes.

# User-Facing Changes

Reduced memory usage
2024-03-31 00:47:17 +01:00
Devyn Cairns
714a0ccd24
Remove serde derive for ShellError, replace via LabeledError (#12319)
# Description

This changes the interface for plugins to always represent errors as
`LabeledError`s. This is good for altlang plugins, as it would suck for
them to have to implement and track `ShellError`. We save a lot of
generated code from the `ShellError` serde impl too, so `nu` and plugins
get to have a smaller binary size.

Reduces the release binary size by 1.2 MiB on my build configuration.

# User-Facing Changes

- Changes plugin protocol. `ShellError` no longer serialized.
- `ShellError` serialize output is different
- `ShellError` no longer deserializes to exactly the same value as
serialized

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Document in plugin protocol reference
2024-03-30 14:21:40 +01:00
Stefan Holderbach
cc39069e13
Reuse existing small allocations if possible (#12335)
Those allocations are all small and insignificant in the grand scheme of
things and the optimizer may be able to resolve some of those but better
to be nice anyways.

Primarily inspired by the new
[`clippy::assigning_clones`](https://rust-lang.github.io/rust-clippy/master/index.html#/assigning_clones)

- **Avoid reallocs with `clone_from` in `nu-parser`**
- **Avoid realloc on assignment in `Stack`**
- **Fix `clippy::assigning_clones` in `nu-cli`**
- **Reuse allocations in `nu-explore` if possible**
2024-03-30 14:04:11 +01:00
Stefan Holderbach
ce581a80a6
Elide clone in V::follow_cell_path for record (#12325)
# Description
This clone is not necessary and tanks the performance of deep nested
access.
As soon as we found the value, we know we discard the old value, so can
`std::mem::take` the inner (`impl Default for Value` to the rescue)

We may be able to further optimize this but not having to clone the
value is vital.
2024-03-30 14:03:31 +01:00
Stefan Holderbach
e889679d42
Use nightly clippy to kill dead code/fix style (#12334)
- **Remove duplicated imports**
- **Remove unused field in `CompletionOptions`**
- **Remove unused struct in `nu-table`**
- **Clarify generic bounds**
- **Simplify a subtrait bound for `ExactSizeIterator`**
- **Use `unwrap_or_default`**
- **Use `Option` directly instead of empty string**
- **Elide unneeded clone in `to html`**
2024-03-30 09:17:28 +08:00
Devyn Cairns
3857e368ff
Fix build of nu-protocol without plugin feature enabled (#12323)
# Description

I broke this, I think in #12279, because I forgot a `#[cfg(plugin)]`
2024-03-28 22:39:57 +01:00
pwygab
04531357b4
Exposed the recursion limit value as a config option (#12308)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Closes #12253.

Exposes the option as "recursion_limit" under config.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

The config file now has a new option!

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

Nothing else...? Do let me know if there's something I've missed!
2024-03-28 15:40:45 -05:00
Ian Manske
442faa5576
Make Record.cols private (#12317)
# Description
Makes the `cols` field in `Record` private and fixes the implementation
of `rename` to account for this change.
2024-03-28 20:18:43 +00:00
Stefan Holderbach
b19da158d5
Rename Value::CustomValue to Value::Custom (#12309)
# Description
The second `Value` is redundant and will consume five extra bytes on
each transmission of a custom value to/from a plugin.

# User-Facing Changes
This is a breaking change to the plugin protocol.

The [example in the protocol
reference](https://www.nushell.sh/contributor-book/plugin_protocol_reference.html#value)
becomes

```json
{
  "Custom": {
    "val": {
      "type": "PluginCustomValue",
      "name": "database",
      "data": [36, 190, 127, 40, 12, 3, 46, 83],
      "notify_on_drop": true
    },
    "span": {
      "start": 320,
      "end": 340
    }
  }
}
```

instead of 

```json
{
  "CustomValue": {
    ...
  }
}
```


# After Submitting
Update plugin protocol reference
2024-03-27 22:10:56 +01:00
Devyn Cairns
01d30a416b
Change PluginCommand API to be more like Command (#12279)
# Description

This is something that was discussed in the core team meeting last
Wednesday. @ayax79 is building `nu-plugin-polars` with all of the
dataframe commands into a plugin, and there are a lot of them, so it
would help to make the API more similar. At the same time, I think the
`Command` API is just better anyway. I don't think the difference is
justified, and the types for core commands have the benefit of requiring
less `.into()` because they often don't own their data

- Broke `signature()` up into `name()`, `usage()`, `extra_usage()`,
`search_terms()`, `examples()`
- `signature()` returns `nu_protocol::Signature`
- `examples()` returns `Vec<nu_protocol::Example>`
- `PluginSignature` and `PluginExample` no longer need to be used by
plugin developers

# User-Facing Changes
Breaking API for plugins yet again 😄
2024-03-27 11:59:57 +01:00
Ian Manske
c747ec75c9
Add command_prelude module (#12291)
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
    record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
    ShellError, Signature, Span, Type, Value,
};
```

This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
    ast::{Call, CellPath},
    engine::{Command, EngineState, Stack},
    record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
    PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```

This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
2024-03-26 21:17:30 +00:00
Filip Andersson
b70766e6f5
Boxes record for smaller Value enum. (#12252)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Boxes `Record` inside `Value` to reduce memory usage, `Value` goes from
`72` -> `56` bytes after this change.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-03-26 17:17:44 +02:00
Stefan Holderbach
24d2c8dd8e
Follow API guidelines for public types (#12283)
# Description
Follow the [API guideline naming
conventions](https://rust-lang.github.io/api-guidelines/naming.html)
also for our externally exposed types

(See
[`clippy::wrong_self_convention`](https://rust-lang.github.io/rust-clippy/master/index.html#/wrong_self_convention)
with [`avoid-breaking-exported-api =
false`](https://doc.rust-lang.org/clippy/lint_configuration.html#avoid-breaking-exported-api)
)

Also be a good citizen around doccomments

- **Fix `Unit::to_value` to `Unit::build_value`**
- **Fix `PipelineData::is_external_failed` to `check_external_failed`**
- **Fix doccomment on `check_external_failed`**
- **Fix `Value::into_config` naming to `parse_as_config`**
- **Document `Value::parse_as_config`**

# Plugin-Author-Facing Changes
See renames above
2024-03-26 12:12:25 +01:00
Stefan Holderbach
b2c5dc204a
Style: move some Option if/else to method chains (#12285)
- **Use `bool::then` where it simplifies readability**
- **More debatable uses of `bool::then`**
- **Use `Option::filter` in `find_active_overlay`**
2024-03-26 08:35:51 +08:00
Wind
87c5f6e455
ls, rm, cp, open, touch, mkdir: Don't expand tilde if input path is quoted string or a variable. (#12232)
# Description
Fixes:  #11887
Fixes: #11626

This pr unify the tilde expand behavior over several filesystem relative
commands. It follows the same rule with glob expansion:
|  command  |  result |
| ----------- |  ------ |
| ls ~/aaa  | expand tilde
| ls "~/aaa"  | don't expand tilde
| let f = "~/aaa"; ls $f | don't expand tilde, if you want to: use `ls
($f \| path expand)`
| let f: glob = "~/aaa"; ls $f | expand tilde, they don't expand on
`mkdir`, `touch` comamnd.

Actually I'm not sure for 4th item, currently it's expanding is just
because it followes the same rule with glob expansion.

### About the change
It changes `expand_path_with` to accept a new argument called
`expand_tilde`, if it's true, expand it, if not, just keep it as `~`
itself.

# User-Facing Changes
After this change, `ls "~/aaa"` won't expand tilde.

# Tests + Formatting
Done
2024-03-25 10:08:38 +08:00