Commit Graph

9334 Commits

Author SHA1 Message Date
Jakub Žádník
48a34ffe6d
Fix test failure when running tests with nextest (#13101)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Test was failing with “did you mean” due to the `NEXTEST` env var being
present when running tests via `cargo nextest run`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-08 15:11:49 +03:00
NotTheDr01ds
d2121a155e
Fixes #13093 - Erroneous example in 'touch' help (#13095)
# Description

Fixes #13093 by:

* Removing the mentioned help example
* Updating the `--accessed` and `--modified` flag descriptions to remove
mention of "timestamp/date"

# User-Facing Changes

Help changes

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-06-07 09:33:48 -05:00
ymcx
cfe13397ed
Fix the colors when completing using a relative path (#12898)
# Description
Fixes a bug where the autocompletion menu has the wrong colors due to
'std::fs::symlink_metadata' not being able to handle relative paths.
Attached below are screenshots before and after applying the commit, in
which the colors are wrong when autocompleting on a path prefixed by a
tilde, whereas the same directory is highlighted correctly when prefixed
by a dot.

BEFORE:

![1715982514020](https://github.com/nushell/nushell/assets/89810988/62051f03-0846-430d-b493-e6f3cd6d0e04)

![1715982873231](https://github.com/nushell/nushell/assets/89810988/28c647ab-3b2a-47ef-9967-5d09927e299d)
AFTER:

![1715982490585](https://github.com/nushell/nushell/assets/89810988/7a370138-50af-42fd-9724-a34cc605bede)

![1715982894748](https://github.com/nushell/nushell/assets/89810988/e884f69f-f757-426e-98c4-bc9f7f6fc561)
2024-06-07 08:07:23 -05:00
Ian Manske
d6a9fb0e40
Fix display formatting for command type in help commands (#12996)
# Description
Related to #12832, this PR changes the way `help commands` displays the
command type to be consistent with `scope commands` and `which`.

# User-Facing Changes
Technically a breaking change since the `help commands` output can now
be different.
2024-06-07 08:03:31 -05:00
Access
a246a19387
fix: coredump without any messages (#13034)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.

- this PR should close https://github.com/nushell/nushell/issues/12874
- fixes https://github.com/nushell/nushell/issues/12874
I want to fix the issue which is induced by the fix for
https://github.com/nushell/nushell/issues/12369. after this pr. This pr
induced a new error for unix system, in order to show coredump messages

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

- this PR should close https://github.com/nushell/nushell/issues/12874
- fixes https://github.com/nushell/nushell/issues/12874
I want to fix the issue which is induced by the fix for
https://github.com/nushell/nushell/issues/12369. after this pr. This pr
induced a new error for unix system, in order to show coredump messages

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

after fix for 12874, coredump message is messing, so I want to fix it

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->


![image](https://github.com/nushell/nushell/assets/60290287/6d8ab756-3031-4212-a5f5-5f71be3857f9)

---------

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2024-06-07 08:02:52 -05:00
Andrej Kolchin
2d0a60ac67
Use native toml datetime type in to toml (#13018)
# Description

Makes `to toml` use the `toml::value::Datetime` type, so that `to toml`
serializes dates properly.


# User-Facing Changes

`to toml` will now encode dates differently, in a native format instead
of a string. This could, in theory, break some workflows:

```Nushell
# Before:
~> {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
Fri, 31 May 2024 00:00:00 +0000 (10 hours ago)
# After:
~> {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
Error: nu:🐚:only_supports_this_input_type

  × Input type not supported.
   ╭─[entry #13:1:36]
 1 │ {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
   ·                                    ────┬────                  ──────┬──────
   ·                                        │                            ╰── only string and int input data is supported
   ·                                        ╰── input type: date
   ╰────
```

Fix #11751
2024-06-07 07:43:30 -05:00
Wind
83cf212e20
run_external.rs: use pathdiff::diff_path to handle relative path (#13056)
# Description
This pr is going to use `pathdiff::diff_path`, so we don't need to
handle for relative path by ourselves.

This is also the behavior before the rewritten of run_external.rs

It's a follow up to https://github.com/nushell/nushell/pull/13028

# User-Facing Changes
NaN

# Tests + Formatting
No need to add tests
2024-06-07 10:14:42 +08:00
dependabot[bot]
e3a20e90b0
Bump os_pipe from 1.1.5 to 1.2.0 (#13087)
Bumps [os_pipe](https://github.com/oconnor663/os_pipe.rs) from 1.1.5 to
1.2.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="6268f86399"><code>6268f86</code></a>
bump version to 1.2.0</li>
<li><a
href="b392c7e3c9"><code>b392c7e</code></a>
remove unsafe code from dup()</li>
<li><a
href="c7b2277bdb"><code>c7b2277</code></a>
move <code>mod sys</code> to the top of lib.rs</li>
<li><a
href="282b1884f6"><code>282b188</code></a>
update ci.yml</li>
<li><a
href="432da0803a"><code>432da08</code></a>
add rust-version to Cargo.toml</li>
<li><a
href="12868e41e6"><code>12868e4</code></a>
edition 2018 -&gt; 2021</li>
<li><a
href="d9e8d61593"><code>d9e8d61</code></a>
activate IO safety integration by default</li>
<li><a
href="d02b96eddb"><code>d02b96e</code></a>
added visionos</li>
<li>See full diff in <a
href="https://github.com/oconnor663/os_pipe.rs/compare/1.1.5...1.2.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=os_pipe&package-manager=cargo&previous-version=1.1.5&new-version=1.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-07 08:08:02 +08:00
dependabot[bot]
83628f04ff Bump crate-ci/typos from 1.22.0 to 1.22.1
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.22.0 to 1.22.1.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.22.0...v1.22.1)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-06 23:48:44 +00:00
Stefan Holderbach
460df0e99a
Group polars crate updates in dependabot (#13084)
We depend on `polars` and some of its subcrates explicitly. This leads
to the unfortunate situtation that we have dependabot opening PRs that
are not compiling, independent of whether there are breaking changes we
need to account for. By introducing a [dependabot
group](https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/customizing-dependency-updates#grouping-dependabot-updates-into-one-pull-request)
for `polars` we can at least limit that failure mode.
2024-06-06 18:33:21 -05:00
Stefan Holderbach
eef4a89ff4
Move format date to Category::Strings (#13083)
The rest of the `format` commands live there.

Closes https://github.com/nushell/nushell.github.io/issues/1435
2024-06-06 18:32:08 -05:00
João Fidalgo
073d8850e9
Allow stor insert and stor update to accept pipeline input (#12882)
- this PR should close #11433 

# Description
This PR implements pipeline input support for the stor insert and stor
update commands,
enabling users to directly pass data to these commands without relying
solely on flag parameters.

Previously, it was only possible to specify the record data using flag
parameters,
which could be less intuitive and become cumbersome:

```bash
stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17}
```

Now it is also possible to pass a record through pipeline input:

```bash
{bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} | stor insert --table-name nudb
{str1: nushell datetime1: 2020-04-17} | stor update --table-name nudb"
```

Changes made on code:

- Modified stor insert and stor update to accept a record from the
pipeline.
- Added logic to handle data from the pipeline record.
- Implemented an error case to prevent simultaneous data input from both
pipeline and flag.

# User-facing changes

Returns an error when both ways of inserting data are used.

The examples for both commands were updated and in each command, when
the -d or -u fags are being used at the same time as input is being
passed through the pipeline, it returns an error:


![image](https://github.com/nushell/nushell/assets/120738170/c5b15c1b-716a-4df4-95e8-3bca8f7ae224)

Also returns an error when both of them are missing:


![image](https://github.com/nushell/nushell/assets/120738170/47f538ab-79f1-4fcc-9c62-d7a7d60f86a1)


# Tests + Formating
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

Co-authored-by: Rodrigo Friães <rodrigo.friaes@tecnico.ulisboa.pt>
2024-06-06 10:30:06 -05:00
Reilly Wood
f2f4b83886
Overhaul explore config (#13075)
Configuration in `explore` has always been confusing to me. This PR
overhauls (and simplifies, I think) how configuration is done.

# Details

1. Configuration is now strongly typed. In `Explore::run()` we create an
`ExploreConfig` struct from the more general Nu configuration and
arguments to `explore`, then pass that struct to other parts of
`explore` that need configuration. IMO this is a lot easier to reason
about and trace than the previous approach of creating a
`HashMap<String, Value>` and then using that to make various structs
elsewhere.
2. We now inherit more configuration from the config used for regular Nu
tables
1. Border/line styling now uses the `separator` style used for regular
Nu tables, the special `explore.split_line` config point has been
retired.
2. Cell padding in tables is now controlled by `table.padding` instead
of the undocumented `column_padding_left`/`column_padding_right` config
3. The (optional, previously not enabled by default) `selected_row` and
`selected_column` configuration has been removed. We now only highlight
the selected cell. I could re-add this if people really like the feature
but I'm guessing nobody uses it.

The interface still looks the same with a default/empty config.nu:


![image](https://github.com/nushell/nushell/assets/26268125/e40161ba-a8ec-407a-932d-5ece6f4dc616)
2024-06-06 08:46:43 -05:00
Wind
5d163c1bcc
run_external: remove inner quotes once nushell gets = sign (#13073)
# Description
Fixes: #13066

nushell should remove argument values' inner quote once it gets `=`.
Whatever it's a flag or not, and it also replace from `\"` to `"` before
passing it to external commands.

# User-Facing Changes
Given the shell script:
```shell
# test.sh
echo $@
```
## Before
```
>  sh test.sh -ldflags="-s -w" github.com
-ldflags="-s -w" github.com
> sh test.sh exp='-s -w' github.com
exp='-s -w' github.com
```
## After
```
>  sh test.sh -ldflags="-s -w" github.com
-ldflags=-s -w github.com
> sh test.sh exp='-s -w' github.com
exp=-s -w github.com
```

# Tests + Formatting
Added some tests
2024-06-06 11:03:34 +08:00
Reilly Wood
75d5807dcd
Fix explore panic on empty lists (#13074)
This fixes up a panic I accidentally introduced when refactoring the
cursor code in `explore`: https://github.com/nushell/nushell/pull/12979

Under certain circumstances (running `:nu []`, opening `:try` with the
hidden `try.reactive` setting enabled), `explore` would panic when
handling an empty list. To fix this for now I've removed the validation
I added to the Cursor constructor in that PR.
2024-06-05 19:49:32 -07:00
Mikkel Roald-Arbøl
f378c72f6f
Create CITATION.cff (#12983)
# Description
This PR creates a `CITATION.cff` file. 

# User-Facing Changes
Users will now be able to properly cite `nushell` when used e.g. in
scientific work. The `CITATION.cff` file also enables users to get the
citation withe the Github integration, see
[documentation](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-citation-files).
More information about the `.cff` standard
[here](https://citation-file-format.github.io/).

# After Submitting
Do you want it documented in the docs somewhere? If so, where exactly?
Happy to make a PR there.

One feature that could well be implemented later on is automatic updates
to the CITATION file with a Github Action (this is outside of my comfort
zone, so I'll leave that for a separate PR should anyone want to
implement it). Ideally, the `CITATION.cff` file points to the latest
release , and could do so with the following fields:
```
commit: 
version: 
date-released: 
```

---------

Co-authored-by: Mikkel Roald-Arbøl <github.ggb9a@simplelogin.co>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
Co-authored-by: NotTheDr01ds <32344964+NotTheDr01ds@users.noreply.github.com>
2024-06-06 08:53:34 +08:00
Jack Wright
a6b1d1f6d9
Upgrade to polars 0.40 (#13069)
Upgrading to polars 0.40
2024-06-06 07:26:47 +08:00
Embers-of-the-Fire
96493b26d9
Make string related commands parse-time evaluatable (#13032)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Related meta-issue: #10239.

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

This PR will modify some `str`-related commands so that they can be
evaluated at parse time.

See the following list for those implemented by this pr.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Available now:
- `str` subcommands
  - `trim`
  - `contains`
  - `distance`
  - `ends-with`
  - `expand`
  - `index-of`
  - `join`
  - `replace`
  - `reverse`
  - `starts-with`
  - `stats`
  - `substring`
  - `capitalize`
  - `downcase`
  - `upcase`
- `split` subcommands
  - `chars`
  - `column`
  - `list`
  - `row`
  - `words`
- `format` subcommands
  - `date`
  - `duration`
  - `filesize`
- string related commands
  - `parse`
  - `detect columns`
  - `encode` & `decode`

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

Unresolved questions:
- [ ] Is there any routine of testing const expressions? I haven't found
any yet.
- [ ] Is const expressions required to behave just like there non-const
version, like what rust promises?

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

Unresolved questions:
- [ ] Do const commands need special marks in the docs?
2024-06-05 22:21:52 +03:00
NotTheDr01ds
36ad7f15c4
cd/def --env examples (#13068)
# Description

Per a Discord question
(https://discord.com/channels/601130461678272522/1244293194603167845/1247794228696711198),
this adds examples to the `help` for both:

* `cd`
* `def`

to demonstrate that `def --env` is required when changing directories in
a custom command.

Since the existing examples for `def` were a bit more complex (and had
output) but the `cd` ones were more simplified, I did use slightly
different examples in each. Either or both could be tweaked if desired.

# User-Facing Changes

Command `help` examples

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-06-05 21:35:31 +03:00
Michael Angerman
78fdb2f4d1
reduce log tracing in nu-cli (#13067)
This one trace message creates thousands of lines of trace messages that
is probably
better suited to using on an *as needed* basis rather than everyone
having to wade
through it...

For now, I just commented it out but eventually this line of code should
be removed
and used simply for the time when someone needs to see it...
2024-06-05 08:54:38 -07:00
Devyn Cairns
a3bc85bb5f
Add options for filtering the log output from nu (#13044)
# Description

Add `--log-include` and `--log-exclude` options to filter the log output
from `nu` to specific module prefixes. For example,

```nushell
nu --log-level trace --log-exclude '[nu_parser]'
```

This avoids having to scan through parser spam when trying to debug
something else at `TRACE` level, and should make it feel much more
reasonable to add logging, particularly at `TRACE` level, to various
places in the codebase. It can also be used to debug non-Nushell crates
that support the Rust logging infrastructure, as many do.

You can also include a specific module instead of excluding the parser
log output:

```nushell
nu --log-level trace --log-include '[nu_plugin]'
```

Pinging #13041 for reference, but hesitant to outright say that this
closes that. I think it address that concern though. I've also struggled
with debugging plugin stuff with all of the other output, so this will
really help me there.

# User-Facing Changes

- New `nu` option: `--log-include`
- New `nu` option: `--log-exclude`
2024-06-05 16:42:55 +08:00
Reilly Wood
a9c2349ada
Refactor explore cursor code (#12979)
`explore` has 3 cursor-related structs that are extensively used to
track the currently shown "window" of the data being shown. I was
finding the cursor code quite difficult to follow, so this PR:
- rewrites the base `Cursor` struct from scratch, with some tests
- makes big changes to `WindowCursor`
- renames `XYCursor` to `WindowCursor2D`
- makes some of the cursor functions fallible as a start towards better
error handling
- changes lots of function names to things that I find more intuitive
- adds comments, including ASCII diagrams to explain how the cursors
work

More work could be done (I'd like to review/change more function names
in `WindowCursor` and `WindowCursor2D` and add more tests), but this is
the limit of what I can get done in a weekend. I think this part of the
code is in a better place now.

# Testing performed

I did a lot of manual testing in the record view and binary viewer,
moving around with arrow keys / page up+down / home+end.

This can definitely wait until after the release freeze, this area has
very few automated tests and it'd be good to let the changes bake a bit.
2024-06-04 19:50:11 -07:00
Jakub Žádník
e4104d0792
Span ID Refactor - Step 1 (#12960)
# Description
First part of SpanID refactoring series. This PR adds a `SpanId` type
and a corresponding `span_id` field to `Expression`. Parser creating
expressions will now add them to an array in `StateWorkingSet`,
generates a corresponding ID and saves the ID to the Expression. The IDs
are not used anywhere yet.

For the rough overall plan, see
https://github.com/nushell/nushell/issues/12963.

# User-Facing Changes
Hopefully none. This is only a refactor of Nushell's internals that
shouldn't have visible side effects.

# Tests + Formatting

# After Submitting
2024-06-05 09:57:14 +08:00
Jack Wright
b10325dff1
Allow int values to be converted into floats. (#13025)
Addresses the bug found by @maxim-uvarov when trying to coerce an int
Value to a polars float:

<img width="863" alt="image"
src="https://github.com/nushell/nushell/assets/56345/4d858812-a7b3-4296-98f4-dce0c544b4c6">

Conversion now works correctly:

<img width="891" alt="Screenshot 2024-05-31 at 14 28 51"
src="https://github.com/nushell/nushell/assets/56345/78d9f711-7ad5-4503-abc6-7aba64a2e675">
2024-06-04 18:51:11 -07:00
Antoine Büsch
d4fa014534
Make query xml return nodes in document order (#13047)
# Description

`query xml` used to return results from an XPath query in a random,
non-deterministic order. With this change, results get returned in the
order they appear in the document.

# User-Facing Changes
`query xml` will now return results in a non-random order.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting
2024-06-05 09:47:36 +08:00
dependabot[bot]
c310175a1e
Bump hustcer/setup-nu from 3.10 to 3.11 (#13058) 2024-06-05 09:17:27 +08:00
dependabot[bot]
6b2012bdfa Bump crate-ci/typos from 1.21.0 to 1.22.0
Bumps [crate-ci/typos](https://github.com/crate-ci/typos) from 1.21.0 to 1.22.0.
- [Release notes](https://github.com/crate-ci/typos/releases)
- [Changelog](https://github.com/crate-ci/typos/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crate-ci/typos/compare/v1.21.0...v1.22.0)

---
updated-dependencies:
- dependency-name: crate-ci/typos
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-05 01:01:54 +00:00
Devyn Cairns
28e33587d9
msgpackz: increase default compression level (#13035)
# Description
Increase default compression level for brotli on msgpackz to 3. This has
the best compression time generally. Level 0 and 1 give weird results
and sometimes cause extremely inflated outputs rather than being
compressed. So far this hasn't really been a problem for the plugin
registry file, but has been for other data.

The `$example` is the web-app example from https://json.org/example.html

Benchmarked with:

```nushell
seq 0 11 | each { |level|
  let compressed = ($example | to msgpackz --quality $level)
  let time = (timeit { $example | to msgpackz --quality $level })
  {
    level: $level
    time: $time
    length: ($compressed | bytes length)
    ratio: (($uncompressed_length | into float) / ($compressed | bytes length))
  }
}
```

```
╭────┬───────┬─────────────────┬────────┬───────╮
│  # │ level │      time       │ length │ ratio │
├────┼───────┼─────────────────┼────────┼───────┤
│  0 │     0 │ 4ms 611µs 875ns │   3333 │  0.72 │
│  1 │     1 │ 1ms 334µs 500ns │   3333 │  0.72 │
│  2 │     2 │     190µs 333ns │   1185 │  2.02 │
│  3 │     3 │      184µs 42ns │   1128 │  2.12 │
│  4 │     4 │      245µs 83ns │   1098 │  2.18 │
│  5 │     5 │     265µs 584ns │   1040 │  2.30 │
│  6 │     6 │     270µs 792ns │   1040 │  2.30 │
│  7 │     7 │     444µs 708ns │   1040 │  2.30 │
│  8 │     8 │       1ms 801µs │   1040 │  2.30 │
│  9 │     9 │     843µs 875ns │   1037 │  2.31 │
│ 10 │    10 │ 4ms 128µs 375ns │    984 │  2.43 │
│ 11 │    11 │ 6ms 352µs 834ns │    986 │  2.43 │
╰────┴───────┴─────────────────┴────────┴───────╯
```

cc @maxim-uvarov
2024-06-04 17:19:10 -07:00
Antoine Stevan
ff5cb6f1ff
complete the type of --error-label in std assert commands (#12998)
i was looking at the website documentation of `std assert` and i noticed
one thing
- the `--error-label` argument of `assert` and `assert not` was just a
`record` -> now it's that complete type `record<text: string, span:
record<start: int, end: int>>`
2024-06-05 08:02:17 +08:00
Antoine Büsch
65911c125c
Try to preserve the ordering of elements in from toml (#13045)
# Description

Enable the `preserve_order` feature of the `toml` crate to preserve the
ordering of elements when converting from/to toml.

Additionally, use `to_string_pretty()` instead of `to_string()` in `to
toml`. This displays arrays on multiple lines instead of one big single
line. I'm not sure if this one is a good idea or not... Happy to remove
this from this PR if it's not.

# User-Facing Changes
The order of elements will be different when using `from toml`. The
formatting of arrays will also be different when using `to toml`. For
example:

- before
```
❯ "foo=1\nbar=2\ndoo=3" | from toml
╭─────┬───╮
│ bar │ 2 │
│ doo │ 3 │
│ foo │ 1 │
╰─────┴───╯
❯ {a: [a b c d]} | to toml
a = ["a", "b", "c", "d"]
```
- after
```
❯ "foo=1\nbar=2\ndoo=3" | from toml
╭─────┬───╮
│ foo │ 1 │
│ bar │ 2 │
│ doo │ 3 │
╰─────┴───╯
❯ {a: [a b c d]} | to toml
a = [
    "a",
    "b",
    "c",
    "d",
]
```

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
-  `toolkit test stdlib`

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-05 08:00:39 +08:00
Sang-Heon Jeon
3f0db11ae5
support plus sign for "into filesize" (#12974)
# Description
Fixes https://github.com/nushell/nushell/issues/12968. After apply this
patch, we can use explict plus sign character included string with `into
filesize` cmd.

# User-Facing Changes
AS-IS (before fixing)
```
$ "+8 KiB" | into filesize                                                                                                         
Error: nu:🐚:cant_convert

  × Can't convert to int.
   ╭─[entry #31:1:1]
 1 │ "+8 KiB" | into filesize
   · ────┬───
   ·     ╰── can't convert string to int
   ╰────
```

TO-BE (after fixing)
```
$ "+8KiB" | into filesize                                                                                       
8.0 KiB
```

# Tests + Formatting
Added a test 

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-05 07:43:50 +08:00
Reilly Wood
7746e84199
Make LS_COLORS functionality faster in explore, especially on Windows (#12984)
Closes #12980. More context there, but basically `explore` was getting
file metadata for every row every time the record view was rendered. The
quick fix for now is to do the `LS_COLORS` colouring with a `&str`
instead of a path and file metadata.
2024-06-05 07:43:12 +08:00
Jack Wright
a84fdb1d37
Fixed a couple of incorrect errors messages (#13043)
Fixed a couple of error message that incorrectly reported as parquet
errors instead of CSV errors.
2024-06-05 07:40:02 +08:00
Wind
ad5a6cdc00
bump version to 0.94.3 (#13055) 2024-06-05 06:52:40 +08:00
Devyn Cairns
be8c1dc006
Fix run_external::expand_glob() to return paths that are PWD-relative but reflect the original intent (#13028)
# Description

Fix #13021

This changes the `expand_glob()` function to use
`nu_engine::glob_from()` so that absolute paths are actually preserved,
rather than being made relative to the provided parent. This preserves
the intent of whoever wrote the original path/glob, and also makes it so
that tilde always produces absolute paths.

I also made `expand_glob()` handle Ctrl-C so that it can be interrupted.

cc @YizhePKU

# Tests + Formatting
No additional tests here... but that might be a good idea.
2024-06-03 10:38:55 +03:00
Devyn Cairns
b50903cf58
Fix external command name parsing with backslashes, and add tests (#13027)
# Description

Fixes #13016 and adds tests for many variations of external call
parsing.

I just realized @kubouch took a crack at this too (#13022) so really
whichever is better, but I think the
tests are a good addition.
2024-06-03 10:28:45 +03:00
Devyn Cairns
6635b74d9d
Bump version to 0.94.2 (#13014)
Version bump after 0.94.1 patch release.
2024-06-03 10:28:35 +03:00
Ian Manske
f3cf693ec7
Disallow more characters in arguments for internal cmd commands (#13009)
# Description
Makes `run-external` error if arguments to `cmd.exe` internal commands
contain newlines or a percent sign. This is because the percent sign can
expand environment variables, potentially? allowing command injection.
Newlines I think will truncate the rest of the arguments and should
probably be disallowed to be safe.

# After Submitting
- If the user calls `cmd.exe` directly, then this bypasses our
handling/checking for internal `cmd` commands. Instead, we use the
handling from the Rust std lib which, in this case, does not do special
handling and is potentially unsafe. Then again, it could be the user's
specific intention to run `cmd` with whatever trusted input. The problem
is that since we use the std lib handling, it assumes the exe uses the C
runtime escaping rules and will perform some unwanted escaping. E.g., it
will add backslashes to the quotes in `cmd echo /c '""'`.
- If `cmd` is called indirectly via a `.bat` or `.cmd` file, then we use
the Rust std lib which has separate handling for bat files that should
be safe, but will reject some inputs.
- ~~I'm not sure how we handle `PATHEXT`, that can also cause a file
without an extension to be run as a bat file. If so, I don't know where
the handling, if any, is done for that.~~ It looks like we use the
`which` crate to do the lookup using `PATHEXT`. Then, we pass the exe
path from that to the Rust std lib `Command`, which should be safe
(except for the first `cmd.exe` note).

So, in the future we need to unify and/or fix these different
implementations, including our own special handling for internal `cmd`
commands that this PR tries to fix.
2024-05-30 19:24:48 +00:00
Ian Manske
31f3d2f664
Restore path type behavior (#13006)
# Description
Restores `path type` to return an empty string on error like it did pre
0.94.0.
2024-05-30 13:42:22 +00:00
Wind
40772fea15
fix do closure with both required, options, and rest args (#13002)
# Description
Fixes: #12985

`val_iter` has already handle required positional and optional
positional arguments, it not skip them again while handling rest
arguments.

# User-Facing Changes
Makes `do {|a, ...b| echo $a ...$b} 1 2 3 4` output the following again:
```nushell
╭───┬───╮
│ 0 │ 1 │
│ 1 │ 2 │
│ 2 │ 3 │
│ 3 │ 4 │
╰───┴───╯
```

# Tests + Formatting
Added some test cases
2024-05-30 08:29:46 -05:00
Devyn Cairns
0e1553026e
Restore tilde expansion on external command names (#13001)
# Description

Fix a regression introduced by #12921, where tilde expansion was no
longer done on the external command name, breaking things like

```nushell
> ~/.cargo/bin/exa
```

This properly handles quoted strings, so they don't expand:

```nushell
> ^"~/.cargo/bin/exa"
Error: nu:🐚:external_command

  × External command failed
   ╭─[entry #1:1:2]
 1 │ ^"~/.cargo/bin/exa"
   ·  ─────────┬────────
   ·           ╰── Command `~/.cargo/bin/exa` not found
   ╰────
  help: `~/.cargo/bin/exa` is neither a Nushell built-in or a known external command

```

This required a change to the parser, so the command name is also parsed
in the same way the arguments are - i.e. the quotes on the outside
remain in the expression. Hopefully that doesn't break anything else. 🤞

Fixes #13000. Should include in patch release 0.94.1

cc @YizhePKU

# User-Facing Changes
- Tilde expansion now works again for external commands
- The `command` of `run-external` will now have its quotes removed like
the other arguments if it is a literal string
- The parser is changed to include quotes in the command expression of
`ExternalCall` if they were present

# Tests + Formatting
I would like to add a regression test for this, but it's complicated
because we need a well-known binary within the home directory, which
just isn't a thing. We could drop one there, but that's kind of a bad
behavior for a test to do. I also considered changing the home directory
for the test, but that's so platform-specific - potentially could get it
working on specific platforms though. Changing `HOME` env on Linux
definitely works as far as tilde expansion works.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-05-29 18:48:29 -07:00
Darren Schroeder
6a2996251c
fixes a bug in OSC9;9 execution (#12994)
# Description

This fixes a bug in the `OSC 9;9` functionality where the path wasn't
being constructed properly and therefore wasn't getting set right for
things like "Duplicate Tab" in Windows Terminal. Thanks to @Araxeus for
finding it.

Related to https://github.com/nushell/nushell/issues/10166

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-29 18:06:47 -05:00
Devyn Cairns
f3991f2080
Bump version to 0.94.1 (#12988)
Merge this PR before merging any other PRs.
2024-05-28 22:41:23 +00:00
Jakub Žádník
61182deb96
Bump version to 0.94.0 (#12987) 2024-05-28 12:04:09 -07:00
Ian Manske
4ab2c3238a
Disable reedline patch for 0.94.0 (#12986)
Disable crates.io git patch for reedline for 0.94.0 release.
2024-05-28 18:53:51 +00:00
Ian Manske
6012af2412
Fix panic when redirecting nothing (#12970)
# Description
Fixes #12969 where the parser can panic if a redirection is applied to
nothing / an empty command.

# Tests + Formatting
Added a test.
2024-05-27 10:03:06 +08:00
YizhePKU
f74dd33ba9
Fix touch --reference using PWD from the environment (#12976)
This PR fixes `touch --reference path` so that it resolves `path` using
PWD from the engine state.
2024-05-26 20:24:00 +03:00
YizhePKU
a1fc41db22
Fix path type using PWD from the environment (#12975)
This PR fixes the `path type` command so that it resolves relative paths
using PWD from the engine state.

As a bonus, it also fixes the issue of `path type` returning an empty
string instead of an error when it fails.
2024-05-26 20:23:52 +03:00
YizhePKU
f38f88d42c
Fixes . expanded incorrectly as external argument (#12950)
This PR fixes a bug where `.` is expanded into an empty string when used
as an argument to external commands. Fixes
https://github.com/nushell/nushell/issues/12948.

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-26 07:06:17 +08:00
Darren Schroeder
0c5a67f4e5
make polars plugin use mimalloc (#12967)
# Description
@maxim-uvarov did a ton of research and work with the dply-rs author and
ritchie from polars and found out that the allocator matters on macos
and it seems to be what was messing up the performance of polars plugin.
ritchie suggested to use jemalloc but i switched it to mimalloc to match
nushell and it seems to run better.

## Before (default allocator)
note - using 1..10 vs 1..100 since it takes so long. also notice how
high the `max` timings are compared to mimalloc below.
```nushell
❯ 1..10 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬─────────────────────────╮
│ mean   │ 4sec 999ms 605µs 995ns  │
│ min    │ 983ms 627µs 42ns        │
│ max    │ 13sec 398ms 135µs 791ns │
│ stddev │ 3sec 476ms 479µs 939ns  │
╰────────┴─────────────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 10
╭───────┬────────────────────────╮
│ mean  │ 6sec 220ms 783µs 983ns │
│ min   │ 1sec 184ms 997µs 708ns │
│ max   │ 18sec 882ms 81µs 708ns │
│ std   │ 5sec 350ms 375µs 697ns │
│ times │ [list 10 items]        │
╰───────┴────────────────────────╯
```

## After (using mimalloc)
```nushell
❯ 1..100 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬───────────────────╮
│ mean   │ 103ms 728µs 902ns │
│ min    │ 97ms 107µs 42ns   │
│ max    │ 149ms 430µs 84ns  │
│ stddev │ 5ms 690µs 664ns   │
╰────────┴───────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬───────────────────╮
│ mean  │ 103ms 620µs 195ns │
│ min   │ 97ms 541µs 166ns  │
│ max   │ 130ms 262µs 166ns │
│ std   │ 4ms 948µs 654ns   │
│ times │ [list 100 items]  │
╰───────┴───────────────────╯
```

## After (using jemalloc - just for comparison)
```nushell
❯ 1..100 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}

╭────────┬───────────────────╮
│ mean   │ 113ms 939µs 777ns │
│ min    │ 108ms 337µs 333ns │
│ max    │ 166ms 467µs 458ns │
│ stddev │ 6ms 175µs 618ns   │
╰────────┴───────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬───────────────────╮
│ mean  │ 114ms 363µs 530ns │
│ min   │ 108ms 804µs 833ns │
│ max   │ 143ms 521µs 459ns │
│ std   │ 5ms 88µs 56ns     │
│ times │ [list 100 items]  │
╰───────┴───────────────────╯
```

## After (using parquet + mimalloc)
```nushell
❯ 1..100 | each {timeit {polars open data.parquet | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} |   | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬──────────────────╮
│ mean   │ 34ms 255µs 492ns │
│ min    │ 31ms 787µs 250ns │
│ max    │ 76ms 408µs 416ns │
│ stddev │ 4ms 472µs 916ns  │
╰────────┴──────────────────╯
❯ use std bench
❯ bench { polars open data.parquet | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬──────────────────╮
│ mean  │ 34ms 897µs 562ns │
│ min   │ 31ms 518µs 542ns │
│ max   │ 65ms 943µs 625ns │
│ std   │ 3ms 450µs 741ns  │
│ times │ [list 100 items] │
╰───────┴──────────────────╯
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-25 09:10:01 -05:00