Commit Graph

5861 Commits

Author SHA1 Message Date
Jakub Žádník
3fae77209a
Revert "Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)" (#13292)
This reverts commit 0cfd5fbece.

The original PR messed up syntax higlighting of aliases and causes
panics of completion in the presence of alias.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-04 00:02:13 +03:00
NotTheDr01ds
ca7a2ae1d6
for - remove deprecated --numbered (#13239)
# Description

Complete the `--numbered` removal that was started with the deprecation
in #13112.

# User-Facing Changes

Breaking change - Use `| enumerate` in place of `--numbered` as shown in
the help example

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Searched online doc for `--numbered` to ensure no other usage needed to
be updated.
2024-07-03 07:55:41 -05:00
Darren Schroeder
ba1d900020
create a better error message when saving fails (#13290)
# Description

This PR just creates a better error message when the `save` command
fails.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 07:43:07 -05:00
Yash Thakur
9e738193f3
Force completers to sort in fetch() (#13242)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR fixes the problem pointed out in
https://github.com/nushell/nushell/issues/13204, where the Fish-like
completions aren't sorted properly (this PR doesn't close that issue
because the author there wants more than just fixed sort order).

The cause is all of the file/directory completions being fetched first
and then sorted all together while being treated as strings. Instead,
this PR sorts completions within each individual directory, avoiding
treating `/` as part of the path.

To do this, I removed the `sort` method from the completer trait (as
well as `get_sort_by`) and made all completers sort within the `fetch`
method itself. A generic `sort_completions` helper has been added to
sort lists of completions, and a more specific `sort_suggestions` helper
has been added to sort `Vec<Suggestion>`s.

As for the actual change that fixes the sort order for file/directory
completions, the `complete_rec` helper now sorts the children of each
directory before visiting their children. The file and directory
completers don't bother sorting at the end (except to move hidden files
down).

To reviewers: don't let the 29 changed files scare you, most of those
are just the test fixtures :)

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

This is the current behavior with prefix matching:

![image](https://github.com/nushell/nushell/assets/45539777/6a36e003-8405-45b5-8cbe-d771e0592709)

And with fuzzy matching:

![image](https://github.com/nushell/nushell/assets/45539777/f2cbfdb2-b8fd-491b-a378-779147291d2a)

Notice how `partial/hello.txt` is the last suggestion, even though it
should come before `partial-a`. This is because the ASCII code for `/`
is greater than that of `-`, so `partial-` is put before `partial/`.

This is this PR's behavior with prefix matching (`partial/hello.txt` is
at the start):

![image](https://github.com/nushell/nushell/assets/45539777/3fcea7c9-e017-428f-aa9c-1707e3ab32e0)

And with fuzzy matching:

![image](https://github.com/nushell/nushell/assets/45539777/d55635d4-cdb8-440a-84d6-41111499f9f8)

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- Modified the partial completions test fixture to test whether this PR
even fixed anything
- Modified fixture to test sort order of .nu completions (a previous
version of my changes didn't sort all the completions at the end but
there were no tests catching that)
- Added a test for making sure subcommand completions are sorted by
Levenshtein distance (a previous version of my changes sorted in
alphabetical order but there were no tests catching that)

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 06:48:06 -05:00
Jack Wright
8316a1597e
Polars: Check to see if the cache is empty before enabling GC. More logging (#13286)
There was a bug where anytime the plugin cache remove was called, the
plugin gc was turned back on. This probably happened when I added the
reference counter logic.
2024-07-03 06:44:26 -05:00
Jakub Žádník
0cfd5fbece
Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Part of https://github.com/nushell/nushell/issues/12963, step 2.

This PR refactors Call and related argument structures to remove their
dependency on `Expression::span` which will be removed in the future.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Should be none. If you see some error messages that look broken, please
report.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-03 09:00:52 +03:00
Jack Wright
122ff1f19c
Add the ability to set content-type metadata with metadata set (#13284)
# Description

With #13254, the content-type pipeline metadata field was added. This
pull request allows it to be manipulated with `metadata set`

# User-Facing Changes
* `metadata set` now has a `--content-type` flag
2024-07-02 13:35:55 -07:00
Jack Wright
0d060aeae8
Use pipeline data for http post|put|patch|delete commands. (#13254)
# Description
Provides the ability to use http commands as part of a pipeline.
Additionally, this pull requests extends the pipeline metadata to add a
content_type field. The content_type metadata field allows commands such
as `to json` to set the metadata in the pipeline allowing the http
commands to use it when making requests.

This pull request also introduces the ability to directly stream http
requests from streaming pipelines.

One other small change is that Content-Type will always be set if it is
passed in to the http commands, either indirectly or throw the content
type flag. Previously it was not preserved with requests that were not
of type json or form data.

# User-Facing Changes
* `http post`, `http put`, `http patch`, `http delete` can be used as
part of a pipeline
* `to text`, `to json`, `from json` all set the content_type metadata
field and the http commands will utilize them when making requests.
2024-07-01 12:34:19 -07:00
Ian Manske
e5cf4863e9
Fix clippy lint (#13277)
# Description
Fixes `items_after_test_module` lint.
2024-06-30 18:28:09 -05:00
alex-tdrn
69e4790b00
Skip decoration lines for detect columns --guess (#13274)
# Description
I introduced a regression in #13272 that resulted in `detect columns
--guess` to panic whenever it had to handle empty, whitespace-only, or
non-whitespace-only lines that go all the way to the last column (and as
such, cannot be considered to be lines that only have entries for the
first colum). I fix this by detecting these cases and skipping them,
since these are usually decoration lines. An example is the second line
output by `winget list`:

![image](https://github.com/nushell/nushell/assets/20356389/06c873fb-0a26-45dd-b020-3bcc737d027f)

What we don't want to skip, however, is lines that contain no
whitespace, and fit into the detected first column, since these lines
represent cases where data is only available for the first column, and
are not just decoration lines. For example (made up example, there are
no such entries in `winget lits`'s output), in this output we would not
want to skip the `Docker Desktop` line :
```
Name                                                        Id                                           Version     Available Source
-------------------------------------------------------------------------------------------------------------------------------------
AMD Software                                                ARPMachineX64AMD Catalyst Install Manager 24.4.1
AMD Ryzen Master                                            ARPMachineX64AMD Ryzen Master             2.13.0.2908
Docker Desktop
Mozilla Firefox (x64 en-US)                                 Mozilla.Firefox                              127.0.2               winget
```

![image](https://github.com/nushell/nushell/assets/20356389/12e31995-a7c1-4759-8c62-fb4fb199fd2e)

NOTE: `winget list | detect columns --guess` does not panic, but sadly
still does not work as expected. I believe this is not a nushell issue
anymore, but a `winget` one. When being piped, `winget` seems to add
extra whitespace and random `\r` symbols at the beginning of the text.
This messes with the column detection, of course.

![image](https://github.com/nushell/nushell/assets/20356389/7d1b7e5f-17d0-41c8-8d2f-7896e0d73d66)

![image](https://github.com/nushell/nushell/assets/20356389/56917954-1231-43e7-bacf-e5760e263054)

![image](https://github.com/nushell/nushell/assets/20356389/630bcfc9-eb78-4a45-9c8f-97efc0c224f4)


# User-Facing Changes
`detect columns --guess` should not panic when receiving output from
`winget list` at all anymore.

A breaking change is the skipping of decoration lines, especially since
scripts probably were doing something like
`winget list | lines | reject 1 | str join "\n" | detect columns
--guess`. This will now cause them to reject a line with valid data.

# Tests + Formatting
Added tests that exercise these edge cases, as well as a single-column
test to make sure that trivial cases keep working.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-30 07:38:41 -05:00
NotTheDr01ds
a2873336bb
Fix do signature (#13216)
Recommend holding until after #13125 is fully digested and *possibly*
until 0.96.

# Description

Fixes one of the issues described in #13125 

The `do` signature included a `SyntaxShape:Any` as one of the possible
first-positional types. This is incorrect. `do` only takes a closure as
a positional. This had the result of:

1. Moving what should have been a parser error to evaluation-time

   ## Before

   ```nu
   > do 1
   Error: nu:🐚:cant_convert

     × Can't convert to Closure.
      ╭─[entry #26:1:4]
    1 │ do 1
      ·    ┬
      ·    ╰── can't convert int to Closure
      ╰────
   ```

   ## After

   ```nu
   > do 1
   Error: nu::parser::parse_mismatch

     × Parse mismatch during operation.
      ╭─[entry #5:1:4]
    1 │ do 1
      ·    ┬
      ·    ╰── expected block, closure or record
      ╰────
   ```  

2. Masking a bad test in `std assert`

This is a bit convoluted, but `std assert` tests included testing
`assert error` to make sure it:

* Asserts on bad code
* Doesn't assert on good code

The good-code test was broken, and was essentially bad-code (really
bad-code) that wasn't getting caught due to the bad signature.

Fixing this resulted in *parse time* failures on every call to
`test_asserts` (not something that particular test was designed to
handle.

This PR also fixes the test case to properly evaluate `std assert error`
against a good code path.

# User-Facing Changes

* Error-type returned (possible breaking change?)

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-06-29 16:17:06 -05:00
Andy Gayton
4fe0f860a8
feat: add query webpage-info to plugin_nu_query (#13252)
# Description

This PR adds a new subcommand `query webpage-info` to `plugin_nu_query`.
The subcommand is a basic wrapper for the
[`webpage`](https://crates.io/crates/webpage) crate.

Usage:

```
http get https://phoronix.com | query webpage-info
```

and it returns a `Record` version of
[`webpage::HTML`](https://docs.rs/webpage/latest/webpage/struct.HTML.html).

The PR also takes a shot at bringing @lily-mara 's
[nu-serde::to_value](https://github.com/nushell/nushell/pull/3878/files)
back to life, updating it for the latest version of nushell. That's not
the main focus of the PR though - I just didn't want to have to
implement a custom converter for `webpage::HTML` 😅. If it looks
reasonable we could move it to `nu_protocol`(?) either in this PR or a
future one (along with adding tests for it).

# User-Facing Changes

no breaking changes
2024-06-29 16:13:31 -05:00
Darren Schroeder
33d0537cae
add str deunicode command (#13270)
# Description

Sometimes it's helpful to deal with only ASCII. This command will take a
unicode string as input and convert it to ASCII using the deunicode
crate.

```nushell
❯ "A…B" | str deunicode
A...B
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-29 16:12:34 -05:00
alex-tdrn
40e629beb1
Fix multibyte codepoint handling in detect columns --guess (#13272)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR fixes #13269. The splitting code in `guess_width.rs` was
creating slices from char indices, instead of byte indices. This works
perfectly fine for 1-byte code points, but panics or returns wrong
results as soon as multibyte codepoints appear in the input. I
originally discovered this by piping `winget list` into `detect columns
--guess`, since winget sometimes uses the unicode ellipsis symbol (`…`)
which is 3 bytes long when encoded in utf-8.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`detect columns --guess` should not crash due to multibyte unicode input
anymore

before:

![image](https://github.com/nushell/nushell/assets/20356389/833cd732-be3b-4158-97f7-0ca2616ce23f)

after:

![image](https://github.com/nushell/nushell/assets/20356389/15358b40-4083-4a33-9f2c-87e63f39d985)


# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- Added tests to `guess_width.rs` for testing handling of multibyte as
well as combining diacritical marks

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-29 16:12:17 -05:00
Tim Martin
153b45bc63
Surprising symlink resolution for std path add (#13258)
# Description
The standard library's `path add` function has some surprising side
effects that I attempt to address in this PR:

1. Paths added, if they are symbolic links, should not be resolved to
their targets. Currently, resolution happens.

   Imagine the following:

   ```nu
# Some time earlier, perhaps even not by the user, a symlink is created
   mkdir real-dir
   ln -s real-dir link-dir

   # Then, step to now, with link-dir that we want in our PATHS variable
   use std
   path add link-dir
   ```

In the current implementation of `path add`, it is _not_ `link-dir` that
will be added, as has been stated in the command. It is instead
`real-dir`. This is surprising. Users have the agency to do this
resolution if they wish with `path expand` (sans a `--no-symlink` flag):
for example, `path add (link-dir | path expand)`

In particular, when I was trying to set up
[fnm](https://github.com/Schniz/fnm), a Node.js version manager, I was
bitten by this fact when `fnm` told me that an expected path had not
been added to the PATHS variable. It was looking for the non-resolved
link. The user in [this
comment](https://github.com/Schniz/fnm/issues/463#issuecomment-1710050737)
was likely affected by this too.

Shells, such as nushell, can handle path symlinks just fine. Binary
lookup is unaffected. Let resolution be opt-in.

Lastly, there is some convention already in place for **not** resolving
path symlinks in the [default $env.ENV_CONVERSIONS
table](57452337ff/crates/nu-utils/src/sample_config/default_env.nu (L65)).
   
2. All existing paths in the path variable should be left untouched.
Currently, they are `path expand`-ed (including symbolic link
resolution).

   Path add should mean just that: prepend/append this path.

Instead, it currently means that, _plus mutate all other paths in the
variable_.

Again, users have the agency to do this with something like `$env.PATH =
$env.PATH | split row (char esep) | path expand`.

3. Minorly, I update documentation on running tests in
`crates/nu-std/CONTRIBUTING.md`. The offered command to run the standard
library test suite was no longer functional. Thanks to @weirdan in [this
Discord
conversation](https://discord.com/channels/601130461678272522/614593951969574961/1256029201119576147)
for the context.

# User-Facing Changes

(Written from the perspective of release notes)

- The standard library's `path add` function no longer resolves symlinks
in either the newly added paths, nor the other paths already in the
variable.

# Tests + Formatting

A test for the changes working correctly has been added to
`crates/nu-std/tests/test_std.nu` under the test named
`path_add_expand`.

You can quickly verify this new test and the existing `path add` test
with the following command:

```nu
cargo run -- -c 'use crates/nu-std/testing.nu; NU_LOG_LEVEL=INFO testing run-tests --path crates/nu-std --test path_add'
```

All commands suggested in the issue template have been run and complete
without error.

# After Submitting
I'll add a release note to [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged.
2024-06-28 18:11:48 -05:00
Bruce Weirdan
4f8d82bb88
Added support for multiple attributes to query web -a (#13256)
# Description

Allows specifying multiple attributes to retrieve from the selected
nodes. E.g. you may want to select both hrefs and targets from the list
of links:

```nushell
.... | query web --query a --attribute [href target]
```
# User-Facing Changes

`query web --attribute` previously accepted a string. Now it accepts
either a string or a list of strings.

The shape definition for this flag was relaxed temporarily, until
nushell/nushell#13253 is fixed.
2024-06-28 12:50:20 -05:00
Jack Wright
720b4cbd01
Polars 0.41 Upgrade (#13238)
# Description
Upgrading to Polars 0.41

# User-Facing Changes
* `polars melt` has been renamed to `polars unpivot` to match the change
in the polars API. Additionally, it now supports lazy dataframes.
Introduced a `--streamable` option to use the polars streaming engine
for lazy frames.
* The parameter `outer` has been replaced with `full` in `polars join`
to match polars change.
* `polars value-count` now supports the column (rename count column),
parallelize (multithread), sort, and normalize options.

The list of polars changes can be found
[here](https://github.com/pola-rs/polars/releases/tag/rs-0.41.2)
2024-06-28 06:37:45 -05:00
Devyn Cairns
a71732ba12
Add context to the I/O error messages in nu_cmd_plugin::util::modify_plugin_file() (#13259)
# Description

This might help @hustcer debug problems with `setup-nu`. The error
messages with the file I/O in `modify_plugin_file()` are not currently
not specific about what file path was involved in the I/O operation.

The spans on those errors have also changed to the span of the custom
path if provided.

# User-Facing Changes

- Slightly better error
2024-06-27 23:49:06 -07:00
Wind
57452337ff
Restrict strings beginning with quote should also ending with quote (#13131)
# Description
Closes: #13010

It adds an additional check inside `parse_string`, and returns
`unbalanced quote` if input string is unbalanced

# User-Facing Changes
After this pr, the following is no longer allowed:
```nushell
❯ "asdfasdf"asdfasdf
Error: nu::parser::extra_token_after_closing_delimiter

  × Invaild characters after closing delimiter
   ╭─[entry #1:1:11]
 1 │ "asdfasdf"asdfasdf
   ·           ────┬───
   ·               ╰── invalid characters
   ╰────
  help: Try removing them.
❯ 'asdfasd'adsfadf
Error: nu::parser::extra_token_after_closing_delimiter

  × Invaild characters after closing delimiter
   ╭─[entry #2:1:10]
 1 │ 'asdfasd'adsfadf
   ·          ───┬───
   ·             ╰── invalid characters
   ╰────
  help: Try removing them.
```

# Tests + Formatting
Added 1 test
2024-06-28 09:47:12 +08:00
Jack Wright
1f1f581357
Converted perf function to be a macro. Utilized the perf macro within the polars plugin. (#13224)
In this pull request, I converted the `perf` function within `nu_utils`
to a macro. This change facilitates easier usage within plugins by
allowing the use of `env_logger` and setting `RUST_LOG=nu_plugin_polars`
(or another plugin). Without this conversion, the `RUST_LOG` variable
would need to be set to `RUST_LOG=nu_utils::utils`, which is less
intuitive and impossible to narrow the perf results to one plugin.
2024-06-27 18:56:56 -05:00
suimong
0d79b63711
Fix find command output bug in the case of taking ByteStream input. (#13246)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

fixes #13245 

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

In addition to addressing #13245, this PR also updated one of the doc
example to the `find` command to document that non-regex mode is case
insensitive, which may surprise some users.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
-  🟢 `toolkit test`
-  🟢 `toolkit test stdlib`


# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Ben Yang <ben@ya.ng>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-06-27 09:46:10 -05:00
Bruce Weirdan
46ed69ab12
Add char nul (#13241)
# Description

Adds `char nul`, `char null_byte` and `char zero_byte` named characters.
All of them generate 0x00 byte.

# User-Facing Changes
Three new named characters are added:
 * `char nul` - generates 0x00 byte
 * `char null_byte` - generates 0x00 byte
 * `char zero_byte` - generates 0x00 byte
2024-06-26 18:49:44 -05:00
goldfish
ee74ec7423
Make the subcommands (from {csv, tsv, ssv}) 0-based for consistency (#13209)
# Description

fixed #11678

The sub-commands of from command (`from {csv, tsv, ssv}`) name columns
starting from index 0.
This behaviour is inconsistent with other commands such as `detect
columns`.
This PR makes the subcommands index 0-based.

# User-Facing Changes

The subcommands (`from {csv, tsv, ssv}`) return a table with the columns
starting at index 0 if no header data is passed.

```
~/Development/nushell> "foo bar baz" | from ssv -n -m 1 
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo     │ bar     │ baz     │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo,bar,baz" | from csv -n 
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo     │ bar     │ baz     │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo\tbar\tbaz" | from tsv -n
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo     │ bar     │ baz     │
╰───┴─────────┴─────────┴─────────╯
```


# Tests + Formatting

When I ran tests, `commands::touch::change_file_mtime_to_reference`
failed with the following error.
The error also occurs in the master branch, so it's probably unrelated
to these changes.
(maybe a problem with my dev environment)

```
$ toolkit check pr

~~~~~~~~

failures:

---- commands::touch::change_file_mtime_to_reference stdout ----
=== stderr

thread 'commands::touch::change_file_mtime_to_reference' panicked at crates/nu-command/tests/commands/touch.rs:298:9:
assertion `left == right` failed
  left: SystemTime { tv_sec: 1719149697, tv_nsec: 57576929 }
 right: SystemTime { tv_sec: 1719149697, tv_nsec: 78219489 }


failures:
    commands::touch::change_file_mtime_to_reference

test result: FAILED. 1533 passed; 1 failed; 32 ignored; 0 measured; 0 filtered out; finished in 10.87s

error: test failed, to rerun pass `-p nu-command --test main`
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
-  `toolkit test stdlib`
```

# After Submitting

nothing
2024-06-26 17:51:47 -05:00
Piepmatz
198aedb6c2
Use IntoValue and FromValue derive macros in nu_plugin_example for example usage (#13220)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The derive macros provided by #13031 are very useful for plugin authors.
In this PR I made use of these macros for two commands.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This Example usage could be highlighted in the changelog for plugin
authors as this is probably very useful for them.
2024-06-26 17:50:14 -05:00
NotTheDr01ds
58e8ea6084
Update and add ls examples (#13222)
# Description

Based on #13219, added several examples to `ls` doc to demonstrate
recursive directory listings. List of changes in this PR:

* Add example for `ls **/*` to demonstrate recursive listing using glob
pattern
* Add example for `ls ...(glob )`... to demonstrate recursive listing
using glob command
* Remove `-s` from an example where it had no use (since it was based on
the current directory and was not recursive)
* Update the description of `ls -a ~ `... to clarify that it lists the
full path of directories
* Update the description of `ls -as ~ `... (the difference being the
`-s`) to clarify that it lists only the filenames, not paths.


# User-Facing Changes

Help only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2024-06-26 17:49:52 -05:00
dependabot[bot]
020f4436d9
Bump shadow-rs from 0.28.0 to 0.29.0 (#13226) 2024-06-26 22:48:45 +00:00
dependabot[bot]
38ecb6d380
Bump uuid from 1.8.0 to 1.9.1 (#13227) 2024-06-26 06:43:46 +00:00
Ian Manske
5a486029db
Add typed path forms (#13115)
# Description
This PR adds new types to `nu-path` to enforce path invariants. Namely,
this PR adds:
- `Path` and `PathBuf`. These types are different from, but analogous to
`std::path::Path` and `std::path::PathBuf`.
- `RelativePath` and `RelativePathBuf`. These types must be/contain
strictly relative paths.
- `AbsolutePath` and `AbsolutePathBuf`. These types must be/contain
strictly absolute paths.
- `CanonicalPath` and `CanonicalPathBuf`. These types must be/contain
canonical paths.

Operations are prohibited as necessary to ensure that the invariants of
each type are upheld (needs double-checking).

Only paths that are absolute (or canonical) can be easily used as /
converted to `std::path::Path`s. This is to help force us to account for
the emulated current working directory instead of accidentally using the
current directory of the Nushell process (i.e.,
`std::env::current_dir`). Related to #12975 and #12976.

Note that this PR uses several declarative macros, as the file / this PR
would otherwise be 5000 lines long.

# User-Facing Changes
No major changes yet, just adds types to `nu-path` to be used in the
future.

# After Submitting
Actually use the new path types in all our crates where it makes sense,
removing usages of `std::path` types.
2024-06-25 18:33:57 -07:00
Wind
def36865ef
Enable reloading changes to a submodule (#13170)
# Description

Fixes: https://github.com/nushell/nushell/issues/12099

Currently if user run `use voice.nu`, and file is unchanged, then run
`use voice.nu` again. nushell will use the module directly, even if
submodule inside `voice.nu` is changed.

After discussed with @kubouch, I think it's ok to re-parse the module
file when:
1. It exports sub modules which are defined by a file
2. It uses other modules which are defined by a file

## About the change:
To achieve the behavior, we need to add 2 attributes to `Module`:
1. `imported_modules`: it tracks the other modules is imported by the
givem `module`, e.g: `use foo.nu`
2. `file`: the path of a module, if a module is defined by a file, it
will be `Some(path)`, or else it will be `None`.

After the change:

    use voice.nu always read the file and parse it.
    use voice will still use the module which is saved in EngineState.

# User-Facing Changes

use `xxx.nu` will read the file and parse it if it exports submodules or
uses submodules

# Tests + Formatting

Done

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-06-25 18:33:37 -07:00
Ian Manske
55ee476306
Define keywords (#13213)
# Description
Some commands in `nu-cmd-lang` are not classified as keywords even
though they should be.

# User-Facing Changes
In the output of `which`, `scope commands`, and `help commands`, some
commands will now have a `type` of `keyword` instead of `built-in`.
2024-06-25 18:32:54 -07:00
Darren Schroeder
f241110005
implement autoloading (#13217)
# Description

This PR implements script or module autoloading. It does this by finding
the `$nu.vendor-autoload-dir`, lists the contents and sorts them by file
name. These files are evaluated in that order.

To see what's going on, you can use `--log-level warn`
```
❯ cargo r -- --log-level warn
    Finished dev [unoptimized + debuginfo] target(s) in 0.58s
     Running `target\debug\nu.exe --log-level warn`
2024-06-24 09:23:20.494 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: config.nu, key: config-path, config_file_specified: None
2024-06-24 09:23:20.495 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: env.nu, key: env-path, config_file_specified: None
2024-06-24 09:23:20.629 PM [WARN ] nu::config_files: setup_config() config_file_specified: None, env_file_specified: None, login: false
2024-06-24 09:23:20.660 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: true
Hello, from env.nu
2024-06-24 09:23:20.679 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: false
Hello, from config.nu
Hello, from defs.nu
Activating Microsoft Visual Studio environment.
2024-06-24 09:23:21.398 PM [WARN ] nu::config_files: read_vendor_autoload_files() src\config_files.rs:234:9
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: read_vendor_autoload_files: C:\ProgramData\nushell\vendor\autoload
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\01_get-weather.nu"
2024-06-24 09:23:21.675 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\02_temp.nu"
2024-06-24 09:23:21.817 PM [WARN ] nu_cli::repl: Terminal doesn't support use_kitty_protocol config
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-25 18:31:54 -07:00
Jack Wright
0dd35cddcd
Bumping version to 0.95.1 (#13231)
Marks development for hotfix
2024-06-25 18:26:07 -07:00
Jakub Žádník
f93c6680bd
Bump to 0.95.0 (#13221)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-25 21:29:47 +03:00
Devyn Cairns
43e349a274
Mitigate the poor interaction between ndots expansion and non-path strings (#13218)
# Description

@hustcer reported that slashes were disappearing from external args
since #13089:

```
$> ossutil ls oss://abc/b/c
Error: invalid cloud url: "oss:/abc/b/c", please make sure the url starts with: "oss://"

$> ossutil ls 'oss://abc/b/c'
Error: oss: service returned error: StatusCode=403, ErrorCode=UserDisable, ErrorMessage="UserDisable", RequestId=66791EDEFE87B73537120838, Ec=0003-00000801, Bucket=abc, Object=
```

I narrowed this down to the ndots handling, since that does path parsing
and path reconstruction in every case. I decided to change that so that
it only activates if the string contains at least `...`, since that
would be the minimum trigger for ndots, and also to not activate it if
the string contains `://`, since it's probably undesirable for a URL.

Kind of a hack, but I'm not really sure how else we decide whether
someone wants ndots or not.

# User-Facing Changes
- bare strings not containing ndots are not modified
- bare strings containing `://` are not modified

# Tests + Formatting
Added tests to prevent regression.
2024-06-24 16:39:01 -07:00
Bruce Weirdan
4509944988
Fix usage parsing for commands defined in CRLF (windows) files (#13212)
Fixes nushell/nushell#13207

# Description
This fixes the parsing of command usage when that command comes from a
file with CRLF line endings.

See nushell/nushell#13207 for more details.

# User-Facing Changes
Users on Windows will get correct autocompletion for `std` commands.
2024-06-23 18:43:05 -05:00
Piepmatz
9b7f899410
Allow missing fields in derived FromValue::from_value calls (#13206)
# Description
In #13031 I added the derive macros for `FromValue` and `IntoValue`. In
that implementation, in particular for structs with named fields, it was
not possible to omit fields while loading them from a value, when the
field is an `Option`. This PR adds extra handling for this behavior, so
if a field is an `Option` and that field is missing in the `Value`, then
the field becomes `None`. This behavior is also tested in
`nu_protocol::value::test_derive::missing_options`.

# User-Facing Changes
When using structs for options or similar, users can now just emit
fields in the record and the derive `from_value` method will be able to
understand this, if the struct has an `Option` type for that field.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
A showcase for this feature would be great, I tried to use the current
derive macro in a plugin of mine for a config but without this addition,
they are annoying to use. So, when this is done, I would add an example
for such plugin configs that may be loaded via `FromValue`.
2024-06-22 13:31:09 -07:00
NotTheDr01ds
b6bdadbc6f
Suppress column index for default cal output (#13188)
# Description

* As discussed in the comments in #11954, this suppresses the index
column on `cal` output. It does that by running `table -i false` on the
results by default.
* Added new `--as-table/-t` flag to revert to the old behavior and
output the calendar as structured data
* Updated existing tests to use `--as-table`
* Added new tests against the string output
* Updated `length` test which also used `cal`
* Added new example for `--as-table`, with result

# User-Facing Changes

## Breaking change

The *default* `cal` output has changed from a `list` to a `string`. To
obtain structured data from `cal`, use the new `--as-table/-t` flag.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`


# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-22 07:41:29 -05:00
NotTheDr01ds
dcb6ab6370
Fixed generate command signature (#13200)
# Description

Removes `list<any>` as an input type for the `generate` command. This
command does not accept pipeline input (and cannot, logically). This can
be seen by the use of `_input` in the command's `run()`.

Also, due to #13199, in order to pass `toolkit check pr`, one of the
examples was changed to remove the `result`. This is probably a better
demonstration of the ability of the command to infinitely generate a
list anyway, and an infinite list can't be represented in a `result`.

# User-Facing Changes

Should only be a change to the help. The input type was never valid and
couldn't have been used.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-06-22 07:37:34 -05:00
Jack Wright
db86dd9f26
Polars default infer (#13193)
Addresses performance issues that @maxim-uvarov found with CSV and JSON
lines.

This ensures that the schema inference follows the polars defaults of
100 lines. Recent changes caused the default values to be override and
caused the entire file to be scanned when inferring the schema.
2024-06-22 07:23:42 -05:00
Maxim Zhiburt
10e84038af
nu-explore: Add vertical lines && fix index/transpose issue (#13147)
Somehow I believe that split lines were implemented originally; (I
haven't got to find it though; from a quick look)
I mean a long time ago before a lot a changes were made.

Probably adding horizontal lines would make also some sense.

ref #13116
close #13140

Take care

________________

If `explore` is used, frequently, or planned to be so.
I guess it would be a good one to create a test suite for it; to not
break things occasionally 😅

I did approached it one time back then using `expectrl` (literally
`expect`), but there was some issues.
Maybe smth. did change.
Or some `clean` mode could be introduced for it, to being able to be
used by outer programs; to control `nu`.

Just thoughts here.
2024-06-21 12:07:57 -07:00
Devyn Cairns
91d44f15c1
Allow plugins to report their own version and store it in the registry (#12883)
# Description

This allows plugins to report their version (and potentially other
metadata in the future). The version is shown in `plugin list` and in
`version`.

The metadata is stored in the registry file, and reflects whatever was
retrieved on `plugin add`, not necessarily the running binary. This can
help you to diagnose if there's some kind of mismatch with what you
expect. We could potentially use this functionality to show a warning or
error if a plugin being run does not have the same version as what was
in the cache file, suggesting `plugin add` be run again, but I haven't
done that at this point.

It is optional, and it requires the plugin author to make some code
changes if they want to provide it, since I can't automatically
determine the version of the calling crate or anything tricky like that
to do it.

Example:

```
> plugin list | select name version is_running pid
╭───┬────────────────┬─────────┬────────────┬─────╮
│ # │      name      │ version │ is_running │ pid │
├───┼────────────────┼─────────┼────────────┼─────┤
│ 0 │ example        │ 0.93.1  │ false      │     │
│ 1 │ gstat          │ 0.93.1  │ false      │     │
│ 2 │ inc            │ 0.93.1  │ false      │     │
│ 3 │ python_example │ 0.1.0   │ false      │     │
╰───┴────────────────┴─────────┴────────────┴─────╯
```

cc @maxim-uvarov (he asked for it)

# User-Facing Changes

- `plugin list` gets a `version` column
- `version` shows plugin versions when available
- plugin authors *should* add `fn metadata()` to their `impl Plugin`,
but don't have to

# Tests + Formatting

Tested the low level stuff and also the `plugin list` column.

# After Submitting
- [ ] update plugin guide docs
- [ ] update plugin protocol docs (`Metadata` call & response)
- [ ] update plugin template (`fn metadata()` should be easy)
- [ ] release notes
2024-06-21 06:27:09 -05:00
Devyn Cairns
dd8f8861ed
Add shape_glob_interpolation to default_config.nu (#13198)
# Description

Just missed this during #13089. Adds `shape_glob_interpolation` to the
config.

This actually isn't really going to be seen at all yet, so I debated
whether it's really needed at all. It's only used to highlight the
quotes themselves, and we don't have any quoted glob interpolations at
the moment.
2024-06-21 06:17:29 -05:00
Devyn Cairns
9845d13347
fix nu-system build on arm64 FreeBSD (#13196)
# Description

Fixes #13194

`ki_stat` is supposed to be a `c_char`, but was defined was `i8`.
Unfortunately, `c_char` is `u8` on Aarch64 (on all platforms), so this
doesn't compile. I fixed it to use `c_char` instead.

Double checked whether NetBSD is affected, but the `libc` code defines
it as `i8` for some reason (erroneously, really) but that doesn't matter
too much. Anyway should be ok there.

Confirmed to be working.
2024-06-21 03:03:10 -07:00
NotTheDr01ds
4c82a748c1
Do example (#13190)
# Description

#12056 added support for default and type-checked arguments in `do`
closures.

This PR adds examples for those features.  It also:

* Fixes the TODO (a closure parameter that wasn't being used) that was
preventing a result from being added
* Removes extraneous commas from the descriptions
* Adds an example demonstrating multiple positional closure arguments

# User-Facing Changes

Help examples only

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-20 18:46:56 -05:00
Jack Wright
20834c9d47
Added the ability to turn on performance debugging through and env var for the polars plugin (#13191)
This allows performance debugging to be turned on by setting:

```nushell
$env.POLARS_PLUGIN_PERF = "true"
```

Furthermore, this improves the other plugin debugging by allowing the
env variable for debugging to be set at any time versus having to be
available when nushell is launched:

```nushell
$env.POLARS_PLUGIN_DEBUG = "true"
```

This plugin introduces a `perf` function that will output timing
results. This works very similar to the perf function available in
nu_utils::utils::perf. This version prints everything to std error to
not break the plugin stream and uses the engine interface to see if the
env variable is configured.

This pull requests uses this `perf` function when:
* opening csv files as dataframes
* opening json lines files as dataframes

This will hopefully help provide some more fine grained information on
how long it takes polars to open different dataframes. The `perf` can
also be utilized later for other dataframes use cases.
2024-06-20 16:37:38 -07:00
Jack Wright
7d2d573eb8
Added the ability to open json lines dataframes with polars lazy json lines reader. (#13167)
The `--lazy` flag will now use the polars' LazyJsonLinesReader when
opening a json lines file with `polars open`
2024-06-20 10:55:49 -07:00
Darren Schroeder
c09a8a5ec9
add a system level folder for future autoloading (#13180)
# Description

This PR adds a directory to the `$nu` constant that shows where the
system level autoload directory is located at. This folder is modifiable
at compile time with environment variables.
```rust
    // Create a system level directory for nushell scripts, modules, completions, etc
    // that can be changed by setting the NU_VENDOR_AUTOLOAD_DIR env var on any platform
    // before nushell is compiled OR if NU_VENDOR_AUTOLOAD_DIR is not set for non-windows 
    // systems, the PREFIX env var can be set before compile and used as PREFIX/nushell/vendor/autoload

    // pseudo code
    // if env var NU_VENDOR_AUTOLOAD_DIR is set, in any platform, use it
    // if not, if windows, use ALLUSERPROFILE\nushell\vendor\autoload
    // if not, if non-windows, if env var PREFIX is set, use PREFIX/share/nushell/vendor/autoload
    // if not, use the default /usr/share/nushell/vendor/autoload
```

### Windows default
```nushell
❯ $nu.vendor-autoload-dir
C:\ProgramData\nushell\vendor\autoload
```
### Non-Windows default
```nushell
❯ $nu.vendor-autoload-dir
/usr/local/share/nushell/vendor/autoload
```
### Non-Windows with PREFIX set
```nushell
❯ PREFIX=/usr/bob cargo r
❯ $nu.vendor-autoload-dir
/usr/bob/share/nushell/vendor/autoload
```
### Non-Windows with NU_VENDOR_AUTOLOAD_DIR set
```nushell
❯ NU_VENDOR_AUTOLOAD_DIR=/some/other/path/nushell/stuff cargo r
❯ $nu.vendor-autoload-dir
/some/other/path/nushell/stuff
```

> [!IMPORTANT] 
To be clear, this PR does not do the auto-loading, it just sets up the
folder to support that functionality that can be added in a later PR.
The PR also does not create the folder defined. It's just setting the
$nu constant.
 
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-20 06:30:43 -05:00
Devyn Cairns
bdc32345bd
Move most of the peculiar argument handling for external calls into the parser (#13089)
# Description

We've had a lot of different issues and PRs related to arg handling with
externals since the rewrite of `run-external` in #12921:

- #12950
- #12955
- #13000
- #13001
- #13021
- #13027
- #13028
- #13073

Many of these are caused by the argument handling of external calls and
`run-external` being very special and involving the parser handing
quoted strings over to `run-external` so that it knows whether to expand
tildes and globs and so on. This is really unusual and also makes it
harder to use `run-external`, and also harder to understand it (and
probably is part of the reason why it was rewritten in the first place).

This PR moves a lot more of that work over to the parser, so that by the
time `run-external` gets it, it's dealing with much more normal Nushell
values. In particular:

- Unquoted strings are handled as globs with no expand
- The unescaped-but-quoted handling of strings was removed, and the
parser constructs normal looking strings instead, removing internal
quotes so that `run-external` doesn't have to do it
- Bare word interpolation is now supported and expansion is done in this
case
- Expressions typed as `Glob` containing `Expr::StringInterpolation` now
produce `Value::Glob` instead, with the quoted status from the expr
passed through so we know if it was a bare word
- Bare word interpolation for values typed as `glob` now possible, but
not implemented
- Because expansion is now triggered by `Value::Glob(_, false)` instead
of looking at the expr, externals now support glob types

# User-Facing Changes

- Bare word interpolation works for external command options, and
otherwise embedded in other strings:
  ```nushell
  ^echo --foo=(2 + 2) # prints --foo=4
  ^echo -foo=$"(2 + 2)" # prints -foo=4
  ^echo foo="(2 + 2)" # prints (no interpolation!) foo=(2 + 2)
  ^echo foo,(2 + 2),bar # prints foo,4,bar
  ```

- Bare word interpolation expands for external command head/args:
  ```nushell
  let name = "exa"
  ~/.cargo/bin/($name) # this works, and expands the tilde
  ^$"~/.cargo/bin/($name)" # this doesn't expand the tilde
  ^echo ~/($name)/* # this glob is expanded
  ^echo $"~/($name)/*" # this isn't expanded
  ```

- Ndots are now supported for the head of an external command
(`^.../foo` works)

- Glob values are now supported for head/args of an external command,
and expanded appropriately:
  ```nushell
  ^("~/.cargo/bin/exa" | into glob) # the tilde is expanded
  ^echo ("*.txt" | into glob) # this glob is expanded
  ```

- `run-external` now works more like any other command, without
expecting a special call convention
  for its args:
  ```nushell
  run-external echo "'foo'"
  # before PR: 'foo'
  # after PR:  foo
  run-external echo "*.txt"
  # before PR: (glob is expanded)
  # after PR:  *.txt
  ```

# Tests + Formatting
Lots of tests added and cleaned up. Some tests that weren't active on
Windows changed to use `nu --testbin cococo` so that they can work.
Added a test for Linux only to make sure tilde expansion of commands
works, because changing `HOME` there causes `~` to reliably change.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
- [ ] release notes: make sure to mention the new syntaxes that are
supported
2024-06-19 21:00:03 -07:00
NotTheDr01ds
44aa0a2de4
Table help rendering (#13182)
# Description

Mostly fixes #13149 with much of the credit to @fdncred.

This PR runs `table --expand` against `help` example results. This is
essentially the same fix that #13146 was for `std help`.

It also changes the shape of the result for the `table --expand`
example, as it was hardcoded wrong.

~Still needed is a fix for the `table --collapse` example.~ Note that
this is also still a bug in `std help` that I didn't noticed before.

# User-Facing Changes

Certain tables are now rendered correctly in the help examples for:

* `table`
* `zip`
* `flatten`
* And almost certainly others

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-06-19 20:12:25 -05:00
Devyn Cairns
12991cd36f
Change the error style during tests to plain (#13061)
# Description

This fixes issues with trying to run the tests with a terminal that is
small enough to cause errors to wrap around, or in cases where the test
environment might produce strings that are reasonably expected to wrap
around anyway. "Fancy" errors are too fancy for tests to work
predictably 😉

cc @abusch

# User-Facing Changes

- Added `--error-style` option for use with `--commands` (like
`--table-mode`)

# Tests + Formatting

Surprisingly, all of the tests pass, including in small windows! I only
had to make one change to a test for `error make` which was looking for
the box drawing characters miette uses to determine whether the span
label was showing up - but the plain error style output is even better
and easier to match on, so this test is actually more specific now.
2024-06-18 21:37:24 -07:00