Commit Graph

9393 Commits

Author SHA1 Message Date
Wind
c150af4279
Don't panic on detect columns with --guess flag (#13752)
# Description
This pr addresses the comment:
https://github.com/nushell/nushell/issues/11791#issuecomment-2308384155

It's caused by if the last row have a very different format to the first
row, the value of `end_char` will exceed the `line_char_boundaries`.
Adding a guard for it should avoid such panic.

# User-Facing Changes
The following code should no longer panic:
```shell
"nu_plugin_highlight = '1.2.2+0.97.1'    # A nushell plugin for syntax highlighting
trace_nu_plugin = '0.3.1'               # A wrapper to trace Nu plugins
nu_plugin_bash_env = '0.13.0'           # Nu plugin bash-env
nu_plugin_from_sse = '0.4.0'            # Nushell plugin to convert a HTTP server sent event stream to structured data
... and 90 crates more (use --limit N to see more)" | detect columns -n --guess
```

# Tests + Formatting
Added 1 test.
2024-09-02 16:29:53 +02:00
Devyn Cairns
39bda8986e
Make tee work more nicely with non-collections (#13652)
# Description

This changes the behavior of `tee` to be more transparent when given a
value that isn't a list or range. Previously, anything that wasn't a
byte stream would converted to a list stream using the iterator
implementation, which led to some surprising results. Instead, now, if
the value is a string or binary, it will be treated the same way a byte
stream is, and the output of `tee` is a byte stream instead of the
original value. This is done so that we can synchronize with the other
thread on collect, and potentially capture any error produced by the
closure.

For values that can't be converted to streams, the closure is just run
with a clone of the value instead on another thread. Because we can't
wait for the other thread, there is no way to send an error back to the
original thread, so instead it's just written to stderr using
`report_error_new()`.

There are a couple of follow up edge cases I see where byte streams
aren't necessarily treated exactly the same way strings are, but this
should mostly be a good experience.

Fixes #13489.

# User-Facing Changes

Breaking change.

- `tee` now outputs and sends string/binary stream for string/binary
input.
- `tee` now outputs and sends the original value for any other input
other than lists/ranges.

# Tests + Formatting

Added for new behavior.

# After Submitting

- [ ] release notes: breaking change, command change
2024-09-01 19:03:46 +02:00
Stefan Holderbach
ee997ef3dd
Remove unneeded serde feature on byte-unit dep (#13749)
removing the `std` feature as well would drop some dependencies tied to
`rust_decimal` from the `Cargo.lock` but unclear to me what the actual
impact on compile times is.

We may want to consider dropping the `byte-unit` dependency altogether
as we have a significant fraction of our own logic to support the byte
units with 1024 and 1000 prefixes. Not sure which fraction is covered by
us or the dependency.
2024-09-01 19:02:28 +02:00
Ian Manske
e3f59910b8
Implement IntoValue for more types (#13744)
# Description

Implements `IntoValue` for `&str` and `DateTime` as well as other
nushell types like `Record` and `Closure`. Also allows `HashMap`s with
keys besides `String` to implement `IntoValue`.
2024-09-01 19:02:12 +02:00
Devyn Cairns
f4940e115f
Remove bincode and use MessagePack instead for plugin custom values (#13745)
# Description

This changes the serialization of custom values within the plugin
protocol to use MessagePack instead of bincode, removing the dependency
on bincode entirely.

Bincode does not seem to be very maintained anymore, and the externally
tagged enum representation doesn't seem to always work now even though
it should. Since we use MessagePack already anyway for the plugin
protocol, this seems like an obvious choice. This uses the unnamed
variant of the serialization rather than the named variant, which is
what the plugin protocol in general uses. The unnamed variant does not
include field names, which aren't really required here, so this should
give us something that's more or less as efficient as bincode is.

Should fix #13743.

# User-Facing Changes

- Will need to recompile plugins (but always do anyway)
- Doesn't technically break the plugin protocol (custom value data is a
black box / plugin implementation specific), but breaks compatibility
between `nu-plugin-engine` and `nu-plugin` so they do need to both be
updated to match.

# Tests + Formatting

All tests pass.

# After Submitting

- [ ] release notes
2024-09-01 17:33:10 +02:00
Darren Schroeder
3f31ca7b8e
remove cfg_atter tarpaulin (#13739)
# Description

Remove the `#[cfg_attr(tarpaulin, ignore)]` code coverage attributes to
get rid warnings when compiling plugins with a more recent rust version
than nushell.
2024-08-31 16:45:15 +02:00
Jack Wright
0119534f61
Expression support polars replace and polars replace-all (#13726)
# Description
Adds the ability for `polars replace` and `polars replace-all` to work
as expressions.

# User-Facing Changes
- `polars replace` can be used with polars expressions
- `polars replace-all` can be used with polars expressions
2024-08-29 13:59:44 -07:00
Stefan Holderbach
84e1ac27e5
Setup global cargo lint configuration (#13691)
# Description
`cargo` somewhat recently gained the capability to store `lints`
settings for the crate and workspace, that can override the defaults
from `rustc` and `clippy` lints. This means we can enforce some lints
without having to actively pass them to clippy via `cargo clippy -- -W
...`. So users just forking the repo have an easier time to follow
similar requirements like our CI.

## Limitation

An exception that remains is that those lints apply to both the primary
code base and the tests. Thus we can't include e.g. `unwrap_used`
without generating noise in the tests. Here the setup in the CI remains
the most helpful.

## Included lints

- Add `clippy::unchecked_duration_subtraction` (added by #12549)
# User-Facing Changes
Running `cargo clippy --workspace` should be closer to the CI. This has
benefits for editor configured runs of clippy and saves you from having
to use `toolkit` to be close to CI in more cases.
2024-08-28 23:37:17 +02:00
Jack Wright
644bebf4c6
Expression support for polars uppercase and polars lowercase (#13724) 2024-08-28 14:08:16 -07:00
Bahex
f58a4b5017
Add split cell-path (#13705)
this PR should close #12168

# Description
Add `split cell-path`, inverse of `into cell-path`.

# User-Facing Changes
Currently there is no way to make use of cell-path values as a user,
other than passing them to builtin commands. This PR makes more use
cases possible.
2024-08-28 23:01:26 +02:00
Kira
ae0e13733d
Fix parsing record values containing colons (#13413)
This PR is an attempt to fix #8257 and fix #10985 (which is
duplicate-ish)

# Description
The parser currently doesn't know how to deal with colons appearing
while lexing whitespace-terminated tokens specifying a record value.
Most notably, this means you can't use datetime literals in record value
position (and as a consequence, `| to nuon | from nuon` roundtrips can
fail), but it also means that bare words containing colons cause a
non-useful error message.

![image](https://github.com/user-attachments/assets/f04a8417-ee18-44e7-90eb-a0ecef943a0f)

`parser::parse_record` calls `lex::lex` with the `:` colon character in
the `special_tokens` argument. This allows colons to terminate record
keys, but as a side effect, it also causes colons to terminate record
*values*. I added a new function `lex::lex_n_tokens`, which allows the
caller to drive the lexing process more explicitly, and used it in
`parser::parse_record` to let colons terminate record keys while not
giving them special treatment when appearing in record values.

This PR description previously said: *Another approach suggested in one
of the issues was to support an additional datetime literal format that
doesn't require colons. I like that that wouldn't require new
`lex::lex_internal` behaviour, but an advantage of my approach is that
it also newly allows for string record values given as bare words
containing colons. I think this eliminates another possible source of
confusion.* It was determined that this is undesirable, and in the
current state of this PR, bare word record values with colons are
rejected explicitly. The better error message is still a win.

# User-Facing Changes
In addition to the above, this PR also disables the use of "special"
(non-item) tokens in record key and value position, and the use of a
single bare `:` as a record key.

Examples of behaviour *before* this PR:
```nu
{ a: b } # Valid, same as { 'a': 'b' }
{ a: b:c } # Error: expected ':'
{ a: 2024-08-13T22:11:09 } # Error: expected ':'
{ :: 1 } # Valid, same as { ':': 1 }
{ ;: 1 } # Valid, same as { ';': 1 }
{ a: || } # Valid, same as { 'a': '||' }
```

Examples of behaviour *after* this PR:
```nu
{ a: b } # (Unchanged) Valid, same as { 'a': 'b' }
{ a: b:c } # Error: colon in bare word specifying record value
{ a: 2024-08-13T22:11:09 } # Valid, same as { a: (2024-08-13T22:11:09) }
{ :: 1 } # Error: colon in bare word specifying record key
{ ;: 1 } # Error: expected item in record key position
{ a: || } # Error: expected item in record value position
```

# Tests + Formatting
I added tests, but I'm not sure if they're sufficient and in the right
place.

# After Submitting
I don't think documentation changes are needed for this, but please let
me know if you disagree.
2024-08-28 22:53:56 +02:00
Stefan Holderbach
2c379cba71
Remove system-clipboard from the default build (#13694)
# Description
This feature tried to connect reedline with the system clipboard for
three special bindings.
To do so it uses the `arboard` crate with heavy dependencies for the
system X or Wayland server or the Windows APIs. We had issues in the
headless CI with it and builds with musl seem to stall.

Removing it from the default build should negatively impact only a small
subset of users aware of the extra bindings. You can still use the
internal clipboard for binding based selection and the terminals extra
bindings to copy arbitrary content into the system clipboard.

For all other users it removes potential sources of failure and a whole
1 MB of release mode binary size (> 2% reduction). Furthermore a
potentially substantial attack surface for Nushell is gone for default
builds.

- Should resolve #13019
- Work in the spirit of #13603


# User-Facing Changes

The `edit` entries
`copyselectionsystem`/`copyselectionsystem`/`pastesystem` for
keybindings are gone in the default build

If you strictly depend on this behavior, you can still build with the
addition of `--features system-clipboard`
2024-08-28 22:19:13 +02:00
dependabot[bot]
7171c9b84a
Bump shadow-rs from 0.31.1 to 0.33.0 (#13713) 2024-08-28 13:05:11 +00:00
Piepmatz
055d7e27e9
Use heck instead of convert_case for nu-derive-value (#13708)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
@sholderbach mentioned that I introduced `convert_case` as a dependency
while we already had `heck` for case conversion. So in this PR replaced
the use `convert_case` with `heck`. Mostly I rebuilt the `convert_case`
API with `heck` to work with it as I like the API of `convert_case` more
than `heck`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Nothing changed, the use of `convert_case` wasn't exposed anywhere and
all case conversions are still available.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
No new tests required but my tests in `test_derive` captured some errors
I made while developing this change, (hurray, tests work 🎉)
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-28 08:02:25 -05:00
Stefan Holderbach
af76e11dd6
Remove str deunicode (#13693)
# Description
Closes #13677

Remove the command `str deunicode`, as it has a narrow application, is
loosely defined by the data provided by the `deunicode` crate and thus a
stabilization liability post-1.0.

Furthermore the data to perform the look-up is quite substantial.

Removing the command and the `deunicode` dependency saves 0.9 MB of
binary data in release mode (~ 2% of total)

(checked via `cargo bloat --release` for a linux x86 build)


# User-Facing Changes
The `str deunicode` command recently added in #13270 is gone
2024-08-28 07:58:38 -05:00
Stefan Holderbach
7dda39a89e
Simplify our bug reporting form (#13695)
The two additional boxes for "additional context" and screenshots may be
somewhat redundant to the primary `Steps to reproduce`. Sadly folks are
already a bit lazy with the core task of providing a succinct
reproducing example. Having additional fields may not actually improve
the quality and lead to waffling or if left empty some deadspace to
scroll past.
2024-08-28 07:58:10 -05:00
Bruce Weirdan
4f822e263f
Respect user-defined $env.NU_LOG_FORMAT and $env.NU_LOG_DATE_FORMAT (#13692)
Fixes nushell/nushell#13689

# Description

Respect user-defined `$env.NU_LOG_FORMAT` and `$env.NU_LOG_DATE_FORMAT`

Additionally I fixed `nu_with_std!()` macro (it was not working
correctly)

# User-Facing Changes

Users now may set `$env.NU_LOG_FORMAT` and `$env.NU_LOG_DATE_FORMAT` in
`env.nu` and it will work even if `use std` is used after that.

# Tests + Formatting

Added a couple of tests for the new functionality.

# After Submitting
2024-08-28 07:57:43 -05:00
Jack Wright
a39e94de8a
Added polars commands for converting string columns to integer and decimal columns (#13711)
# Description
Introduces two new polars commands for converting string columns to
decimal and integer columns:

<img width="740" alt="Screenshot 2024-08-27 at 15 32 28"
src="https://github.com/user-attachments/assets/f9573b6e-48f6-4bbf-8782-39ffb95eb934">

<img width="720" alt="Screenshot 2024-08-27 at 15 33 46"
src="https://github.com/user-attachments/assets/90a66bb5-fa78-4ed3-8b2b-ae05cddd2f3a">

# User-Facing Changes
- Addition of the `polars integer` command
- Addition of the `polars decimal` command
2024-08-28 07:54:31 -05:00
Darren Schroeder
a88f46c6c9
update virtual terminal processing (#13710)
# Description

With Windows Terminal Canary 1.23.240826001-llm, this enables nushell to
query the terminal and receive a response.


![image](https://github.com/user-attachments/assets/c4c43328-c431-47e4-b377-8b3a2bc12b74)

The red component here is
```nushell
❯ ("0c0c" | into int -r 16) / 256 | math round | fmt | get lowerhex
0xc
```

This example queries the background and the response is a r/g/b color.
The response really should be
```
␛]11;1;rgb:0c0c/0c0c/0c0c
```
I'm not sure why nushell's input is eating the first part.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-28 07:54:01 -05:00
dependabot[bot]
4ca1f95b6c
Bump crate-ci/typos from 1.23.6 to 1.24.1 (#13716) 2024-08-28 12:36:03 +00:00
Jack Wright
71ced35987
Changed category for panic and added search terms and examples (#13707)
# Description
Cosmetic changes around `panic` command. Changed category, added search
terms, and examples.

# User-Facing Changes
See above
2024-08-27 16:58:05 -07:00
Piepmatz
1128df2d29
Added record key renaming for derive macros IntoValue and FromValue (#13699)
# Description

Using derived `IntoValue` and `FromValue` implementations on structs
with named fields currently produce `Value::Record`s where each key is
the key of the Rust struct. For records like the `$nu` constant, that
won't work as this record uses `kebab-case` for it's keys. To accomodate
this, I upgraded the `#[nu_value(rename_all = "...")]` helper attribute
to also work on structs with named fields which will rename the keys via
the same case conversion as the enums already have.

# User-Facing Changes
Users of these macros may choose different key styles for their in
`Value` representation.

# Tests + Formatting
I added the same test suite as enums already have and updated the traits
documentation with more examples that also pass the doc test.

# After Submitting
I played around with the `$nu` constant but got stuck at the point that
these keys are kebab-cased, with this, I can play around more with it.
2024-08-27 20:00:44 +02:00
Yash Thakur
da98c23ab3
Use right options in custom completions (#13698)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This issue was reported by kira in
[Discord](https://discord.com/channels/601130461678272522/1276981416307069019).
In https://github.com/nushell/nushell/pull/13311, I accidentally made it
so that custom completions are filtered according to the user's
configured completion options (`$env.config.completions`) rather than
the completion options provided as part of the custom completions. This
PR is a quick fix for that.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

It should once again be possible to override match algorithm, case
sensitivity, and substring matching (`positional`) in custom
completions.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added a couple tests.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

fdncred and I discussed this in Discord a bit and we thought it might be
better to not allow custom completions to override the user's config.
However, `positional` can't currently be set inside one's config, so you
can only do strict prefix matching, no substring matching. Another PR
could do one of the following:
- Document the fact that you can provide completion options inside
custom completions
- Remove the ability to provide completion options with custom
completions and add a `$env.config.completions.positional: true` option
- Remove the ability to provide completion options with custom
completions and add a new match algorithm `substring` (this is the one I
like most, since `positional` only applies to prefix matching anyway)

Separately from these options, we could also allow completers to specify
that they don't Nushell to do any filtering and sorting on the provided
custom completions.
2024-08-26 12:14:57 -05:00
Stefan Holderbach
e3efc8da9f
Remove unnecessary sort in explore search fn (#13690)
Noticed when playing with the `stable_sort_primitive` lint that the
elements from `enumerate` are already sorted.
2024-08-25 20:13:05 +02:00
Andrej Kolčin
3f332bef35
Fix encode/decode todo's (#13683)
Mistakes have been made. I forgot about a bunch of `todo`s in the helper
functions. So, this PR replaces them with proper errors. It also adds
tests for parse-time evaluation, because one `todo` I missed was in a
`run_const` function.
2024-08-24 09:02:02 -05:00
Maxim Zhiburt
525eac1afd
[DRAFT] Check fix for emojie, wrap issues (#13430)
Hi there

Here I am using latest tabled.

My tests shows it does fixes panics, but I am wanna be sure.

@fdncred could you verify that it does fixes those panics/errors?

Closes #13405 
Closes #12786
2024-08-23 17:35:42 -05:00
Poliorcetics
7003b007d5
doc: fix broken doc links (#13644)
Some broken doc links I saw when compiling with `cargo +stable doc
--no-deps --document-private-items --workspace --open`
2024-08-23 21:17:44 +02:00
Gwendolyn
dfdb2b5d31
Improve help output for scripts (#13445)
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Currently the parser and the documentation generation use the signature
of the command, which means that it doesn't pick up on the changed name
of the `main` block, and therefore shows the name of the command as
"main" and doesn't find the subcommands. This PR changes the
aforementioned places to use the block signature to fix these issues.
This closes #13397. Incidentally it also causes input/output types to be
shown in the help, which is kinda pointless for scripts since they don't
operate on structured data but maybe not worth the effort to remove.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
```
# example.nu
export def main [] { help main }
export def 'main sub' [] { print 'sub' }
```
Before:

![image](https://github.com/user-attachments/assets/49fdcf8d-e56a-4c27-b7c8-7d2902c2a807)

![image](https://github.com/user-attachments/assets/4d1f4faa-5928-4269-b0b5-fd654563bb8b)


After:

![image](https://github.com/user-attachments/assets/a7232a1f-f997-4988-808c-8fa957e39bae)

![image](https://github.com/user-attachments/assets/c5628dc6-69b5-443a-b103-9e5faa9bb4ba)

# Tests
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Tests are still missing for the subcommands and the input/output types

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-08-23 21:08:27 +02:00
Andrej Kolčin
822007dbbb
Remove unused same-file workspace dependency (#13678)
A small no-op change. It was used in a two years old `mv` fix
(848550771a, pre `uu_mv`). But now it's redundant.
2024-08-23 20:51:34 +02:00
Andrej Kolčin
0560826414
encode/decode for multiple alphabets (#13428)
Based on the discussion in #13419.


## Description

Reworks the `decode`/`encode` commands by adding/changing the following
bases:

- `base32`
- `base32hex`
- `hex`
- `new-base64`

The `hex` base is compatible with the previous version of `hex` out of
the box (it only adds more flags). `base64` isn't, so the PR adds a new
version and deprecates the old one.

All commands have `string -> binary` signature for decoding and `string
| binary -> string` signature for encoding. A few `base64` encodings,
which are not a part of the
[RFC4648](https://datatracker.ietf.org/doc/html/rfc4648#section-6), have
been dropped.


## Example usage

```Nushell
~/fork/nushell> "string" | encode base32 | decode base32 | decode
string
```

```Nushell
~/fork/nushell> "ORSXG5A=" | decode base32
# `decode` always returns a binary value
Length: 4 (0x4) bytes | printable whitespace ascii_other non_ascii
00000000:   74 65 73 74                                          test
```


## User-Facing Changes

- New commands: `encode/decode base32/base32hex`.
- `encode hex` gets a `--lower` flag.
- `encode/decode base64` deprecated in favor of `encode/decode
new-base64`.
2024-08-23 11:18:51 -05:00
Piepmatz
39b0f3bdda
Change expected type for derived FromValue implementations via attribute (#13647)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
In this PR I expanded the helper attribute `#[nu_value]` on
`#[derive(FromValue)]`. It now allows the usage of `#[nu_value(type_name
= "...")]` to set a type name for the `FromValue::expected_type`
implementation. Currently it only uses the default implementation but
I'd like to change that without having to manually implement the entire
trait on my own.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Users that derive `FromValue` may now change the name of the expected
type.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I added some tests that check if this feature work and updated the
documentation about the derive macro.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-23 06:47:15 -05:00
Piepmatz
712fec166d
Improve working with IntoValue and FromValue for byte collections (#13641)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
I was working with byte collections like `Vec<u8>` and
[`bytes::Bytes`](https://docs.rs/bytes/1.7.1/bytes/struct.Bytes.html),
both are currently not possible to be used directly in a struct that
derives `IntoValue` and `FromValue` at the same time. The `Vec<u8>` will
convert itself into a `Value::List` but expects a `Value::String` or
`Value::Binary` to load from. I now also implemented that it can load
from `Value::List` just like the other `Vec<uX>` versions. For further
working with byte collections the type `bytes::Bytes` is wildly used,
therefore I added a implementation for it. `bytes` is already part of
the dependency graph as many crates (more than 5000 to crates.io) use
it.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
User of `nu-protocol` as library, e.g. plugin developers, can now use
byte collections more easily in their data structures and derive
`IntoValue` and `FromValue` for it.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
I added a few tests that check that these byte collections are correctly
translated in and from `Value`. They live in `test_derive.rs` as part of
the `ByteContainer` and I also explicitely tested that `FromValue` for
`Vec<u8>` works as expected.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Maybe it should be explored if `Value::Binary` should use `bytes::Bytes`
instead of `Vec<u8>`.
2024-08-22 17:59:00 -05:00
Stefan Holderbach
43dcf19ac3
Fix bits ror/bits rol implementation (#13673)
# Description
`bits rol` and `bits ror` were both undefined for the full byte rotates
and panicked when exceeding the byte rotation range.
`bits ror` further more produced nonsensical results by pulling bits
from the following byte instead of the preceding byte.

Those bugs are now fixed

# User-Facing Changes
Sound Nushell `IncorrectValue` error when exceeding the available bits

# Tests + Formatting
Added the necessary tests
2024-08-22 21:22:10 +02:00
Darren Schroeder
ffddee5678
add more granularity for into record with dates (#13650)
# Description

This PR adds a little more granularity when using `into record` with
dates.

### Before
```nushell
❯ date now | into record
╭──────────┬────────╮
│ year     │ 2024   │
│ month    │ 8      │
│ day      │ 19     │
│ hour     │ 18     │
│ minute   │ 31     │
│ second   │ 27     │
│ timezone │ -05:00 │
╰──────────┴────────╯
```

### After
```nushell
❯ date now | into record
╭─────────────┬────────╮
│ year        │ 2024   │
│ month       │ 8      │
│ day         │ 19     │
│ hour        │ 18     │
│ minute      │ 30     │
│ second      │ 51     │
│ millisecond │ 928    │
│ microsecond │ 980    │
│ nanosecond  │ 0      │
│ timezone    │ -05:00 │
╰─────────────┴────────╯
```
2024-08-22 12:09:27 +02:00
Stefan Holderbach
95b78eee25
Change the usage misnomer to "description" (#13598)
# Description
    
The meaning of the word usage is specific to describing how a command
function is *used* and not a synonym for general description. Usage can
be used to describe the SYNOPSIS or EXAMPLES sections of a man page
where the permitted argument combinations are shown or example *uses*
are given.
Let's not confuse people and call it what it is a description.

Our `help` command already creates its own *Usage* section based on the
available arguments and doesn't refer to the description with usage.

# User-Facing Changes

`help commands` and `scope commands` will now use `description` or
`extra_description`
`usage`-> `description`
`extra_usage` -> `extra_description`

Breaking change in the plugin protocol:

In the signature record communicated with the engine.
`usage`-> `description`
`extra_usage` -> `extra_description`

The same rename also takes place for the methods on
`SimplePluginCommand` and `PluginCommand`

# Tests + Formatting
- Updated plugin protocol specific changes
# After Submitting
- [ ] update plugin protocol doc
2024-08-22 12:02:08 +02:00
Stefan Holderbach
3ab9f0b90a
Fix bugs and UB in bit shifting ops (#13663)
# Description
Fixes #11267

Shifting by a `shift >= num_bits` is undefined in the underlying
operation. Previously we also had an overflow on negative shifts for the
operators `bit-shl` and `bit-shr`
Furthermore I found a severe bug in the implementation of shifting of
`binary` data with the commands `bits shl` and `bits shr`, this
categorically produced incorrect results with shifts that were not
`shift % 4 == 0`. `bits shr` also was able to produce outputs with
different size to the input if the shift was exceeding the length of the
input data by more than a byte.

# User-Facing Changes
It is now an error trying to shift by more than the available bits with:
- `bit-shl` operator
- `bit-shr` operator
- command `bits shl`
- command `bits shr`

# Tests + Formatting
Added testing for all relevant cases
2024-08-22 11:54:27 +02:00
Ian Manske
9261c0c55a
Be explicit about reduce args and input (#13646)
# Description

`run_with_value` is meant for running simple closures with one arg.
Using `run_with_value` after `add_arg` is slightly confusing.
2024-08-22 11:39:21 +02:00
Devyn Cairns
7a888c9e9b
Change behavior of into record on lists to be more useful (#13637)
# Description

The previous behaviour of `into record` on lists was to create a new
record with each list index as the key. This was not very useful for
creating meaningful records, though, and most people would end up using
commands like `headers` or `transpose` to turn a list of keys and values
into a record.

This PR changes that instead to do what I think the most ergonomic thing
is, and instead:

- A list of records is merged into one record.
- A list of pairs (two element lists) is folded into a record with the
first element of each pair being the key, and the second being the
value.

The former is just generally more useful than having to use `reduce`
with `merge` for such a common operation, and the latter is useful
because it means that `$a | zip $b | into record` *just works* in the
way that seems most obvious.

Example:

```nushell
[[foo bar] [baz quux]] | into record # => {foo: bar, baz: quux}
[{foo: bar} {baz: quux}] | into record # => {foo: bar, baz: quux}
[foo baz] | zip [bar quux] | into record # => {foo: bar, baz: quux}
```

The support for range input has been removed, as it would no longer
reflect the treatment of an equivalent list.

The following is equivalent to the old behavior, in case that's desired:

```
0.. | zip [a b c] | into record # => {0: a, 1: b, 2: c}
```

# User-Facing Changes
- `into record` changed as described above (breaking)
- `into record` no longer supports range input (breaking)

# Tests + Formatting
Examples changed to match, everything works. Some usage in stdlib and
`nu_plugin_nu_example` had to be changed.

# After Submitting
- [ ] release notes (commands, breaking change)
2024-08-22 11:38:43 +02:00
Stefan Holderbach
e211b7ba53
Bump version to 0.97.2 (#13666) 2024-08-22 11:36:32 +02:00
Devyn Cairns
60769ac1ba
Bump version to 0.97.1 (#13659)
# Description

Bump version to `0.97.1`, which will be the actual next major release.
(`0.97.0` had a bug.)
2024-08-20 20:21:12 -07:00
Yash Thakur
34e7bd861c
Fix bug introduced by #13595 (#13658)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

@devyn found that https://github.com/nushell/nushell/pull/13595, which
made ranges be type-checked at parse time, introduced a bug that caused
`../foo` to be parsed as a string rather than a command call. This was
caused by `parse_range` returning a `Some` despite there being parse
errors (`/foo` doesn't match `SyntaxShape::Number`). To go back to the
old behavior, `parse_range` now returns `None` anytime there's any parse
errors met while parsing the range.

Unfortunately, this means that something like `..$foo` will be parsed as
a string if `$foo` isn't defined and as a range if it is defined. That
was the behavior before #13595, and it should probably be fixed at some
point, but I'm just trying to quickly fix the bug.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Things should go back to the way they were before #13595, except the
type-checking stuff from that PR is still here.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Added a test. Reverted another test that tests that `0..<$day` is parsed
successfully as a string if the variable isn't defined.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-08-20 19:35:13 -07:00
Jack Wright
d667b3c0bc
bumped version number to 0.97 (#13655) 2024-08-20 16:28:19 -07:00
Ian Manske
b63c3514c4
Fix clippy lints (#13645) 2024-08-19 11:12:45 +02:00
Bruce Weirdan
7e9d32d64e
Drop outdated touch -d example (#13631)
Fixes nushell/nushell#8723

# Description

The example was showing the flag that no longer exists.

# User-Facing Changes

Help no longer shows the example with `-d` flag.

# Tests + Formatting

I trust in CI.

# After Submitting

Nothing.
2024-08-15 23:57:25 +02:00
Darren Schroeder
621fb25670
update to latest reedline f2b4 (#13629)
# Description

Update to the latest reedline comment. BTW `f2b4` sounds like a new star
wars droid. 😆
2024-08-15 21:03:13 +02:00
Bruce Weirdan
a80273bd7b
Drop unused fs_extra and hamcrest2 dependencies (#13628)
Fixes nushell/nushell#7995

# Description

This dependency is no longer used by nushell itself.

# User-Facing Changes

None.

# Tests + Formatting

Pased.

# After Submitting

None.
2024-08-15 19:25:50 +02:00
Piotr Kufel
5473def7ef
Prefer process name over executable path (#13618)
# Description
Prefer process name over executable path. This in practice causes the
`name` column to use just the base executable name.

Also set start_time to nothing on error, because why not.

# User-Facing Changes

Before:

> /opt/google/chrome/chrome

After:

> chrome

Also picks up changes due to `echo test-proc > /proc/$$/comm`.

# Tests + Formatting

No new coverage.
2024-08-15 13:44:01 +08:00
Darren Schroeder
04746b8e2d
add osc633e (#13625)
# Description

This PR adds OSC 633E for vscode users to enable `Terminal: Run Recent
Command` with ctrl-shift-p.


![image](https://github.com/user-attachments/assets/21b92206-3c44-4060-aa0b-78718c099336)
2024-08-15 13:40:34 +08:00
Stefan Holderbach
31b3104af7
Remove code duplication in glob (#13626) 2024-08-14 23:11:00 +02:00
Jack Wright
803bc9c63f
Incrementing the eager dataframe cache value before returning it (#13624)
# Description

Fixes issue [12828](https://github.com/nushell/nushell/issues/12828).

When attempting a `polars collect` on an eager dataframe, we return
dataframe as is. However, before this fix I failed to increment the
internal cache reference count. This caused the value to be dropped from
the internal cache when the references were decremented again.

This fix adds a call to cache.get to increment the value before
returning.
2024-08-14 15:38:46 -05:00