In this pull request, I converted the `perf` function within `nu_utils`
to a macro. This change facilitates easier usage within plugins by
allowing the use of `env_logger` and setting `RUST_LOG=nu_plugin_polars`
(or another plugin). Without this conversion, the `RUST_LOG` variable
would need to be set to `RUST_LOG=nu_utils::utils`, which is less
intuitive and impossible to narrow the perf results to one plugin.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
fixes#13245
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
In addition to addressing #13245, this PR also updated one of the doc
example to the `find` command to document that non-regex mode is case
insensitive, which may surprise some users.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Ben Yang <ben@ya.ng>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
fixed#11678
The sub-commands of from command (`from {csv, tsv, ssv}`) name columns
starting from index 0.
This behaviour is inconsistent with other commands such as `detect
columns`.
This PR makes the subcommands index 0-based.
# User-Facing Changes
The subcommands (`from {csv, tsv, ssv}`) return a table with the columns
starting at index 0 if no header data is passed.
```
~/Development/nushell> "foo bar baz" | from ssv -n -m 1
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo │ bar │ baz │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo,bar,baz" | from csv -n
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo │ bar │ baz │
╰───┴─────────┴─────────┴─────────╯
~/Development/nushell> "foo\tbar\tbaz" | from tsv -n
╭───┬─────────┬─────────┬─────────╮
│ # │ column0 │ column1 │ column2 │
├───┼─────────┼─────────┼─────────┤
│ 0 │ foo │ bar │ baz │
╰───┴─────────┴─────────┴─────────╯
```
# Tests + Formatting
When I ran tests, `commands::touch::change_file_mtime_to_reference`
failed with the following error.
The error also occurs in the master branch, so it's probably unrelated
to these changes.
(maybe a problem with my dev environment)
```
$ toolkit check pr
~~~~~~~~
failures:
---- commands::touch::change_file_mtime_to_reference stdout ----
=== stderr
thread 'commands::touch::change_file_mtime_to_reference' panicked at crates/nu-command/tests/commands/touch.rs:298:9:
assertion `left == right` failed
left: SystemTime { tv_sec: 1719149697, tv_nsec: 57576929 }
right: SystemTime { tv_sec: 1719149697, tv_nsec: 78219489 }
failures:
commands::touch::change_file_mtime_to_reference
test result: FAILED. 1533 passed; 1 failed; 32 ignored; 0 measured; 0 filtered out; finished in 10.87s
error: test failed, to rerun pass `-p nu-command --test main`
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
- ⚫ `toolkit test stdlib`
```
# After Submitting
nothing
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The derive macros provided by #13031 are very useful for plugin authors.
In this PR I made use of these macros for two commands.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This Example usage could be highlighted in the changelog for plugin
authors as this is probably very useful for them.
# Description
Based on #13219, added several examples to `ls` doc to demonstrate
recursive directory listings. List of changes in this PR:
* Add example for `ls **/*` to demonstrate recursive listing using glob
pattern
* Add example for `ls ...(glob )`... to demonstrate recursive listing
using glob command
* Remove `-s` from an example where it had no use (since it was based on
the current directory and was not recursive)
* Update the description of `ls -a ~ `... to clarify that it lists the
full path of directories
* Update the description of `ls -as ~ `... (the difference being the
`-s`) to clarify that it lists only the filenames, not paths.
# User-Facing Changes
Help only
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
N/A
# Description
This PR adds new types to `nu-path` to enforce path invariants. Namely,
this PR adds:
- `Path` and `PathBuf`. These types are different from, but analogous to
`std::path::Path` and `std::path::PathBuf`.
- `RelativePath` and `RelativePathBuf`. These types must be/contain
strictly relative paths.
- `AbsolutePath` and `AbsolutePathBuf`. These types must be/contain
strictly absolute paths.
- `CanonicalPath` and `CanonicalPathBuf`. These types must be/contain
canonical paths.
Operations are prohibited as necessary to ensure that the invariants of
each type are upheld (needs double-checking).
Only paths that are absolute (or canonical) can be easily used as /
converted to `std::path::Path`s. This is to help force us to account for
the emulated current working directory instead of accidentally using the
current directory of the Nushell process (i.e.,
`std::env::current_dir`). Related to #12975 and #12976.
Note that this PR uses several declarative macros, as the file / this PR
would otherwise be 5000 lines long.
# User-Facing Changes
No major changes yet, just adds types to `nu-path` to be used in the
future.
# After Submitting
Actually use the new path types in all our crates where it makes sense,
removing usages of `std::path` types.
# Description
Fixes: https://github.com/nushell/nushell/issues/12099
Currently if user run `use voice.nu`, and file is unchanged, then run
`use voice.nu` again. nushell will use the module directly, even if
submodule inside `voice.nu` is changed.
After discussed with @kubouch, I think it's ok to re-parse the module
file when:
1. It exports sub modules which are defined by a file
2. It uses other modules which are defined by a file
## About the change:
To achieve the behavior, we need to add 2 attributes to `Module`:
1. `imported_modules`: it tracks the other modules is imported by the
givem `module`, e.g: `use foo.nu`
2. `file`: the path of a module, if a module is defined by a file, it
will be `Some(path)`, or else it will be `None`.
After the change:
use voice.nu always read the file and parse it.
use voice will still use the module which is saved in EngineState.
# User-Facing Changes
use `xxx.nu` will read the file and parse it if it exports submodules or
uses submodules
# Tests + Formatting
Done
---------
Co-authored-by: Jakub Žádník <kubouch@gmail.com>
# Description
Some commands in `nu-cmd-lang` are not classified as keywords even
though they should be.
# User-Facing Changes
In the output of `which`, `scope commands`, and `help commands`, some
commands will now have a `type` of `keyword` instead of `built-in`.
# Description
This PR implements script or module autoloading. It does this by finding
the `$nu.vendor-autoload-dir`, lists the contents and sorts them by file
name. These files are evaluated in that order.
To see what's going on, you can use `--log-level warn`
```
❯ cargo r -- --log-level warn
Finished dev [unoptimized + debuginfo] target(s) in 0.58s
Running `target\debug\nu.exe --log-level warn`
2024-06-24 09:23:20.494 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: config.nu, key: config-path, config_file_specified: None
2024-06-24 09:23:20.495 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: env.nu, key: env-path, config_file_specified: None
2024-06-24 09:23:20.629 PM [WARN ] nu::config_files: setup_config() config_file_specified: None, env_file_specified: None, login: false
2024-06-24 09:23:20.660 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: true
Hello, from env.nu
2024-06-24 09:23:20.679 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: false
Hello, from config.nu
Hello, from defs.nu
Activating Microsoft Visual Studio environment.
2024-06-24 09:23:21.398 PM [WARN ] nu::config_files: read_vendor_autoload_files() src\config_files.rs:234:9
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: read_vendor_autoload_files: C:\ProgramData\nushell\vendor\autoload
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\01_get-weather.nu"
2024-06-24 09:23:21.675 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\02_temp.nu"
2024-06-24 09:23:21.817 PM [WARN ] nu_cli::repl: Terminal doesn't support use_kitty_protocol config
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
@hustcer reported that slashes were disappearing from external args
since #13089:
```
$> ossutil ls oss://abc/b/c
Error: invalid cloud url: "oss:/abc/b/c", please make sure the url starts with: "oss://"
$> ossutil ls 'oss://abc/b/c'
Error: oss: service returned error: StatusCode=403, ErrorCode=UserDisable, ErrorMessage="UserDisable", RequestId=66791EDEFE87B73537120838, Ec=0003-00000801, Bucket=abc, Object=
```
I narrowed this down to the ndots handling, since that does path parsing
and path reconstruction in every case. I decided to change that so that
it only activates if the string contains at least `...`, since that
would be the minimum trigger for ndots, and also to not activate it if
the string contains `://`, since it's probably undesirable for a URL.
Kind of a hack, but I'm not really sure how else we decide whether
someone wants ndots or not.
# User-Facing Changes
- bare strings not containing ndots are not modified
- bare strings containing `://` are not modified
# Tests + Formatting
Added tests to prevent regression.
Fixesnushell/nushell#13207
# Description
This fixes the parsing of command usage when that command comes from a
file with CRLF line endings.
See nushell/nushell#13207 for more details.
# User-Facing Changes
Users on Windows will get correct autocompletion for `std` commands.
# Description
In #13031 I added the derive macros for `FromValue` and `IntoValue`. In
that implementation, in particular for structs with named fields, it was
not possible to omit fields while loading them from a value, when the
field is an `Option`. This PR adds extra handling for this behavior, so
if a field is an `Option` and that field is missing in the `Value`, then
the field becomes `None`. This behavior is also tested in
`nu_protocol::value::test_derive::missing_options`.
# User-Facing Changes
When using structs for options or similar, users can now just emit
fields in the record and the derive `from_value` method will be able to
understand this, if the struct has an `Option` type for that field.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
A showcase for this feature would be great, I tried to use the current
derive macro in a plugin of mine for a config but without this addition,
they are annoying to use. So, when this is done, I would add an example
for such plugin configs that may be loaded via `FromValue`.
# Description
* As discussed in the comments in #11954, this suppresses the index
column on `cal` output. It does that by running `table -i false` on the
results by default.
* Added new `--as-table/-t` flag to revert to the old behavior and
output the calendar as structured data
* Updated existing tests to use `--as-table`
* Added new tests against the string output
* Updated `length` test which also used `cal`
* Added new example for `--as-table`, with result
# User-Facing Changes
## Breaking change
The *default* `cal` output has changed from a `list` to a `string`. To
obtain structured data from `cal`, use the new `--as-table/-t` flag.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Removes `list<any>` as an input type for the `generate` command. This
command does not accept pipeline input (and cannot, logically). This can
be seen by the use of `_input` in the command's `run()`.
Also, due to #13199, in order to pass `toolkit check pr`, one of the
examples was changed to remove the `result`. This is probably a better
demonstration of the ability of the command to infinitely generate a
list anyway, and an infinite list can't be represented in a `result`.
# User-Facing Changes
Should only be a change to the help. The input type was never valid and
couldn't have been used.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
Addresses performance issues that @maxim-uvarov found with CSV and JSON
lines.
This ensures that the schema inference follows the polars defaults of
100 lines. Recent changes caused the default values to be override and
caused the entire file to be scanned when inferring the schema.
Somehow I believe that split lines were implemented originally; (I
haven't got to find it though; from a quick look)
I mean a long time ago before a lot a changes were made.
Probably adding horizontal lines would make also some sense.
ref #13116close#13140
Take care
________________
If `explore` is used, frequently, or planned to be so.
I guess it would be a good one to create a test suite for it; to not
break things occasionally 😅
I did approached it one time back then using `expectrl` (literally
`expect`), but there was some issues.
Maybe smth. did change.
Or some `clean` mode could be introduced for it, to being able to be
used by outer programs; to control `nu`.
Just thoughts here.
# Description
This allows plugins to report their version (and potentially other
metadata in the future). The version is shown in `plugin list` and in
`version`.
The metadata is stored in the registry file, and reflects whatever was
retrieved on `plugin add`, not necessarily the running binary. This can
help you to diagnose if there's some kind of mismatch with what you
expect. We could potentially use this functionality to show a warning or
error if a plugin being run does not have the same version as what was
in the cache file, suggesting `plugin add` be run again, but I haven't
done that at this point.
It is optional, and it requires the plugin author to make some code
changes if they want to provide it, since I can't automatically
determine the version of the calling crate or anything tricky like that
to do it.
Example:
```
> plugin list | select name version is_running pid
╭───┬────────────────┬─────────┬────────────┬─────╮
│ # │ name │ version │ is_running │ pid │
├───┼────────────────┼─────────┼────────────┼─────┤
│ 0 │ example │ 0.93.1 │ false │ │
│ 1 │ gstat │ 0.93.1 │ false │ │
│ 2 │ inc │ 0.93.1 │ false │ │
│ 3 │ python_example │ 0.1.0 │ false │ │
╰───┴────────────────┴─────────┴────────────┴─────╯
```
cc @maxim-uvarov (he asked for it)
# User-Facing Changes
- `plugin list` gets a `version` column
- `version` shows plugin versions when available
- plugin authors *should* add `fn metadata()` to their `impl Plugin`,
but don't have to
# Tests + Formatting
Tested the low level stuff and also the `plugin list` column.
# After Submitting
- [ ] update plugin guide docs
- [ ] update plugin protocol docs (`Metadata` call & response)
- [ ] update plugin template (`fn metadata()` should be easy)
- [ ] release notes
# Description
Just missed this during #13089. Adds `shape_glob_interpolation` to the
config.
This actually isn't really going to be seen at all yet, so I debated
whether it's really needed at all. It's only used to highlight the
quotes themselves, and we don't have any quoted glob interpolations at
the moment.
# Description
Fixes#13194
`ki_stat` is supposed to be a `c_char`, but was defined was `i8`.
Unfortunately, `c_char` is `u8` on Aarch64 (on all platforms), so this
doesn't compile. I fixed it to use `c_char` instead.
Double checked whether NetBSD is affected, but the `libc` code defines
it as `i8` for some reason (erroneously, really) but that doesn't matter
too much. Anyway should be ok there.
Confirmed to be working.
# Description
#12056 added support for default and type-checked arguments in `do`
closures.
This PR adds examples for those features. It also:
* Fixes the TODO (a closure parameter that wasn't being used) that was
preventing a result from being added
* Removes extraneous commas from the descriptions
* Adds an example demonstrating multiple positional closure arguments
# User-Facing Changes
Help examples only
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This allows performance debugging to be turned on by setting:
```nushell
$env.POLARS_PLUGIN_PERF = "true"
```
Furthermore, this improves the other plugin debugging by allowing the
env variable for debugging to be set at any time versus having to be
available when nushell is launched:
```nushell
$env.POLARS_PLUGIN_DEBUG = "true"
```
This plugin introduces a `perf` function that will output timing
results. This works very similar to the perf function available in
nu_utils::utils::perf. This version prints everything to std error to
not break the plugin stream and uses the engine interface to see if the
env variable is configured.
This pull requests uses this `perf` function when:
* opening csv files as dataframes
* opening json lines files as dataframes
This will hopefully help provide some more fine grained information on
how long it takes polars to open different dataframes. The `perf` can
also be utilized later for other dataframes use cases.
# Description
This PR adds a directory to the `$nu` constant that shows where the
system level autoload directory is located at. This folder is modifiable
at compile time with environment variables.
```rust
// Create a system level directory for nushell scripts, modules, completions, etc
// that can be changed by setting the NU_VENDOR_AUTOLOAD_DIR env var on any platform
// before nushell is compiled OR if NU_VENDOR_AUTOLOAD_DIR is not set for non-windows
// systems, the PREFIX env var can be set before compile and used as PREFIX/nushell/vendor/autoload
// pseudo code
// if env var NU_VENDOR_AUTOLOAD_DIR is set, in any platform, use it
// if not, if windows, use ALLUSERPROFILE\nushell\vendor\autoload
// if not, if non-windows, if env var PREFIX is set, use PREFIX/share/nushell/vendor/autoload
// if not, use the default /usr/share/nushell/vendor/autoload
```
### Windows default
```nushell
❯ $nu.vendor-autoload-dir
C:\ProgramData\nushell\vendor\autoload
```
### Non-Windows default
```nushell
❯ $nu.vendor-autoload-dir
/usr/local/share/nushell/vendor/autoload
```
### Non-Windows with PREFIX set
```nushell
❯ PREFIX=/usr/bob cargo r
❯ $nu.vendor-autoload-dir
/usr/bob/share/nushell/vendor/autoload
```
### Non-Windows with NU_VENDOR_AUTOLOAD_DIR set
```nushell
❯ NU_VENDOR_AUTOLOAD_DIR=/some/other/path/nushell/stuff cargo r
❯ $nu.vendor-autoload-dir
/some/other/path/nushell/stuff
```
> [!IMPORTANT]
To be clear, this PR does not do the auto-loading, it just sets up the
folder to support that functionality that can be added in a later PR.
The PR also does not create the folder defined. It's just setting the
$nu constant.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
We've had a lot of different issues and PRs related to arg handling with
externals since the rewrite of `run-external` in #12921:
- #12950
- #12955
- #13000
- #13001
- #13021
- #13027
- #13028
- #13073
Many of these are caused by the argument handling of external calls and
`run-external` being very special and involving the parser handing
quoted strings over to `run-external` so that it knows whether to expand
tildes and globs and so on. This is really unusual and also makes it
harder to use `run-external`, and also harder to understand it (and
probably is part of the reason why it was rewritten in the first place).
This PR moves a lot more of that work over to the parser, so that by the
time `run-external` gets it, it's dealing with much more normal Nushell
values. In particular:
- Unquoted strings are handled as globs with no expand
- The unescaped-but-quoted handling of strings was removed, and the
parser constructs normal looking strings instead, removing internal
quotes so that `run-external` doesn't have to do it
- Bare word interpolation is now supported and expansion is done in this
case
- Expressions typed as `Glob` containing `Expr::StringInterpolation` now
produce `Value::Glob` instead, with the quoted status from the expr
passed through so we know if it was a bare word
- Bare word interpolation for values typed as `glob` now possible, but
not implemented
- Because expansion is now triggered by `Value::Glob(_, false)` instead
of looking at the expr, externals now support glob types
# User-Facing Changes
- Bare word interpolation works for external command options, and
otherwise embedded in other strings:
```nushell
^echo --foo=(2 + 2) # prints --foo=4
^echo -foo=$"(2 + 2)" # prints -foo=4
^echo foo="(2 + 2)" # prints (no interpolation!) foo=(2 + 2)
^echo foo,(2 + 2),bar # prints foo,4,bar
```
- Bare word interpolation expands for external command head/args:
```nushell
let name = "exa"
~/.cargo/bin/($name) # this works, and expands the tilde
^$"~/.cargo/bin/($name)" # this doesn't expand the tilde
^echo ~/($name)/* # this glob is expanded
^echo $"~/($name)/*" # this isn't expanded
```
- Ndots are now supported for the head of an external command
(`^.../foo` works)
- Glob values are now supported for head/args of an external command,
and expanded appropriately:
```nushell
^("~/.cargo/bin/exa" | into glob) # the tilde is expanded
^echo ("*.txt" | into glob) # this glob is expanded
```
- `run-external` now works more like any other command, without
expecting a special call convention
for its args:
```nushell
run-external echo "'foo'"
# before PR: 'foo'
# after PR: foo
run-external echo "*.txt"
# before PR: (glob is expanded)
# after PR: *.txt
```
# Tests + Formatting
Lots of tests added and cleaned up. Some tests that weren't active on
Windows changed to use `nu --testbin cococo` so that they can work.
Added a test for Linux only to make sure tilde expansion of commands
works, because changing `HOME` there causes `~` to reliably change.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] release notes: make sure to mention the new syntaxes that are
supported
# Description
Mostly fixes#13149 with much of the credit to @fdncred.
This PR runs `table --expand` against `help` example results. This is
essentially the same fix that #13146 was for `std help`.
It also changes the shape of the result for the `table --expand`
example, as it was hardcoded wrong.
~Still needed is a fix for the `table --collapse` example.~ Note that
this is also still a bug in `std help` that I didn't noticed before.
# User-Facing Changes
Certain tables are now rendered correctly in the help examples for:
* `table`
* `zip`
* `flatten`
* And almost certainly others
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
This fixes issues with trying to run the tests with a terminal that is
small enough to cause errors to wrap around, or in cases where the test
environment might produce strings that are reasonably expected to wrap
around anyway. "Fancy" errors are too fancy for tests to work
predictably 😉
cc @abusch
# User-Facing Changes
- Added `--error-style` option for use with `--commands` (like
`--table-mode`)
# Tests + Formatting
Surprisingly, all of the tests pass, including in small windows! I only
had to make one change to a test for `error make` which was looking for
the box drawing characters miette uses to determine whether the span
label was showing up - but the plain error style output is even better
and easier to match on, so this test is actually more specific now.
# Description
This PR updates the `try` command to show that `catch` is a closure and
can be used as such.
### Before
### After
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
Fixes#13171
# Description
Corrects the `sys users` signature to match the returned type.
Before this change `sys users | where name == root` would result in a
type error.
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes: #13105Fixes: #13077
This pr makes `str substring`, `bytes at` work better with negative
index.
And it also fixes the false range semantic on `detect columns -c` in
some cases.
# User-Facing Changes
For `str substring`, `bytes at`, it will no-longer return an error if
start index is larger than end index. It makes sense to return an empty
string of empty bytes directly.
### Before
```nushell
# str substring
❯ ("aaa" | str substring 2..-3) == ""
Error: nu:🐚:type_mismatch
× Type mismatch.
╭─[entry #23:1:10]
1 │ ("aaa" | str substring 2..-3) == ""
· ──────┬──────
· ╰── End must be greater than or equal to Start
2 │ true
╰────
# bytes at
❯ ("aaa" | encode utf-8 | bytes at 2..-3) == ("" | encode utf-8)
Error: nu:🐚:type_mismatch
× Type mismatch.
╭─[entry #27:1:25]
1 │ ("aaa" | encode utf-8 | bytes at 2..-3) == ("" | encode utf-8)
· ────┬───
· ╰── End must be greater than or equal to Start
╰────
```
### After
```nushell
# str substring
❯ ("aaa" | str substring 2..-3) == ""
true
# bytes at
❯ ("aaa" | encode utf-8 | bytes at 2..-3) == ("" | encode utf-8)
true
```
# Tests + Formatting
Added some tests, adjust existing tests
# Description
After discussing with @sholderbach the cumbersome usage of
`nu_protocol::Value` in Rust, I created a derive macro to simplify it.
I’ve added a new crate called `nu-derive-value`, which includes two
macros, `IntoValue` and `FromValue`. These are re-exported in
`nu-protocol` and should be encouraged to be used via that re-export.
The macros ensure that all types can easily convert from and into
`Value`. For example, as a plugin author, you can define your plugin
configuration using a Rust struct and easily convert it using
`FromValue`. This makes plugin configuration less of a hassle.
I introduced the `IntoValue` trait for a standardized approach to
converting values into `Value` (and a fallible variant `TryIntoValue`).
This trait could potentially replace existing `into_value` methods.
Along with this, I've implemented `FromValue` for several standard types
and refined other implementations to use blanket implementations where
applicable.
I made these design choices with input from @devyn.
There are more improvements possible, but this is a solid start and the
PR is already quite substantial.
# User-Facing Changes
For `nu-protocol` users, these changes simplify the handling of
`Value`s. There are no changes for end-users of nushell itself.
# Tests + Formatting
Documenting the macros itself is not really possible, as they cannot
really reference any other types since they are the root of the
dependency graph. The standard library has the same problem
([std::Debug](https://doc.rust-lang.org/stable/std/fmt/derive.Debug.html)).
However I documented the `FromValue` and `IntoValue` traits completely.
For testing, I made of use `proc-macro2` in the derive macro code. This
would allow testing the generated source code. Instead I just tested
that the derived functionality is correct. This is done in
`nu_protocol::value::test_derive`, as a consumer of `nu-derive-value`
needs to do the testing of the macro usage. I think that these tests
should provide a stable baseline so that users can be sure that the impl
works.
# After Submitting
With these macros available, we can probably use them in some examples
for plugins to showcase the use of them.
# Description
Fixes#13143 by returning an empty list when there are no results found
by `std help --find/-f`
# User-Facing Changes
In addition, prints a message to stderr.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Some command help has example results with nested `table` data which is
displayed as the "non-expanded" form. E.g.:
```nu
╭───┬────────────────╮
│ 0 │ [list 2 items] │
│ 1 │ [list 2 items] │
╰───┴────────────────╯
```
For a good example, see `help zip`.
While we could simply remove the offending Example `result`'s from the
command itself, `std help` is capable of expanding the table properly.
It already formats the output of each example result using `table`, so
simply making it a `table -e` fixes the output.
While I wish we had a way of expanding the tables in the builtin `help`,
that seems to be the same type of problem as in formatting the `cal`
output (see #11954).
I personally think it's better to add this feature to `std help` than to
remove the offending example results, but as long as `std help` is
optional, only a small percentage of users are going to see the
"expected" results.
# User-Facing Changes
Excerpt from `std help zip` before change:
```nu
Zip two lists
> [1 2] | zip [3 4]
╭───┬────────────────╮
│ 0 │ [list 2 items] │
│ 1 │ [list 2 items] │
╰───┴────────────────╯
```
After:
```nu
Zip two lists
> [1 2] | zip [3 4]
╭───┬───────────╮
│ 0 │ ╭───┬───╮ │
│ │ │ 0 │ 1 │ │
│ │ │ 1 │ 3 │ │
│ │ ╰───┴───╯ │
│ 1 │ ╭───┬───╮ │
│ │ │ 0 │ 2 │ │
│ │ │ 1 │ 4 │ │
│ │ ╰───┴───╯ │
╰───┴───────────╯
```
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
-
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
In this PR, I continue my tradition of trivial but hopefully helpful
`help` tweaks. As mentioned in #13143, I noticed that `help -f else`
oddly didn't return the `if` statement itself. Perhaps not so oddly,
since who the heck is going to go looking for *"else"* in the help?
Well, I did ...
Added *"else"* and *"conditional"* to the search terms for `if`.
I'll work on the meat of #13143 next - That's more substantiative.
# User-Facing Changes
Help only
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
-
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Could be improved further I guess; but not here;
You can test the speed differences using data from #13088
```nu
open data.db | get profiles | explore
```
address: #13062
________
1. Noticed that search does not work anymore (even on `main` branch).
2. Not sure about resolved merged conflicts, seems fine, but maybe
something was lost.
---------
Co-authored-by: Reilly Wood <reilly.wood@icloud.com>
# Description
Removes the `which-support` cargo feature and makes all of its
feature-gated code enabled by default in all builds. I'm not sure why
this one command is gated behind a feature. It seems to be a relic of
older code where we had features for what seems like every command.
# Description
`help banner` had several issues:
* It used a Markdown link to an Asciinema recording, but Markdown links
aren't rendered as Markdown links by the help system (and can't be,
since (most?) terminals don't support that)
* Minor grammatical issues
* The Asciinema recording is out of date anyway. It still uses `use
stdn.nu banner` which isn't valid syntax any longer.
Since everyone at this point knows exactly what `banner` does 😉, I chose
to simply remove the link to the recording. Also tweaked the text
(initial caps and removed comma).
# User-Facing Changes
Help only
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This PR is an attempt to add a standard location for people to put
completions in. I saw this topic come up again recently and IIRC we
decided to create a standard location. I used the dirs-next crate to
dictate where these locations are. I know some people won't like that
but at least this gets the ball rolling in a direction that has a
standard directory.
This is what the default NU_LIB_DIRS looks like now in the
default_env.nu. It should also be like this when starting nushell with
`nu -n`
```nushell
$env.NU_LIB_DIRS = [
($nu.default-config-dir | path join 'scripts') # add <nushell-config-dir>/scripts
($nu.data-dir | path join 'completions') # default home for nushell completions
]
```
I also added these default folders to the `$nu` variable so now there is
`$nu.data-path` and `$nu.cache-path`.
## Data Dir Default
While I was in there, I also decided to add a cache dir
## Cache Dir Default
### This is what the default looks like in Ubuntu.
### This is what it looks like with XDG_CACHE_HOME and XDG_DATA_HOME
overridden
```nushell
XDG_DATA_HOME=/tmp/data_home XDG_CACHE_HOME=/tmp/cache_home cargo r
```
### This is what the defaults look like in Windows (username scrubbed to
protect the innocent)
How my NU_LIB_DIRS is set in the images above
```nushell
$env.NU_LIB_DIRS = [
($nu.default-config-dir | path join 'scripts') # add <nushell-config-dir>/scripts
'/Users/fdncred/src/nu_scripts'
($nu.config-path | path dirname)
($nu.data-dir | path join 'completions') # default home for nushell completions
]
```
Let the debate begin.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Removes the `str contains --not` flag that was deprecated in the last
minor release.
# User-Facing Changes
Breaking change since a flag was removed.
# Description
This PR updates the uutils/coreutils crates to the latest released
version.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
I've noticed this several times but kept forgetting to fix it:
The example given for `help def` for the `--wrapped` flag is:
```nu
Define a custom wrapper for an external command
> def --wrapped my-echo [...rest] { echo $rest }; my-echo spam
╭───┬──────╮
│ 0 │ spam │
╰───┴──────╯
```
That's ... odd, since (a) it specifically says *"for an external"*
command, and yet uses (and shows the output from) the builtin `echo`.
Also, (b) I believe `--wrapped` is *only* applicable to external
commands. Finally, (c) the `my-echo spam` doesn't even demonstrate a
wrapped argument.
Unless I'm truly missing something, the example just makes no sense.
This updates the example to really demonstrate `def --wrapped` with the
*external* version of `^echo`. It uses the `-e` command to interpret the
escape-tab character in the string.
```nu
> def --wrapped my-echo [...rest] { ^echo ...$rest }; my-echo -e 'spam\tspam'
spam spam
```
# User-Facing Changes
Help example only.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Instead of an empty string, this PR changes `path type` to return null
if the path does not exist. If some other IO error is encountered, then
that error is bubbled up instead of treating it as a "not found" case.
# User-Facing Changes
- `path type` will now return null instead of an empty string, which is
technically a breaking change. In most cases though, I think this
shouldn't affect the behavior of scripts too much.
- `path type` can now error instead of returning an empty string if some
other IO error besides a "not found" error occurs.
Since this PR introduces breaking changes, it should be merged after the
0.94.1 patch.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
After parsing menu code, the changes weren't merged into the engine
state, which didn't produce any errors (somehow?) until the recent span
ID refactors. With this PR, menus get a new cloned engine state with the
parsed changes correctly merged in.
Hopefully fixes https://github.com/nushell/nushell/issues/13118
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Removes the old, deprecated behavior of the `sys` command. That is, it
will no longer return the full system information record.
# User-Facing Changes
Breaking change: `sys` no longer outputs anything and will instead
display help text.
# Description
#7777 removed the `--numbered` flag from `each`, `par-each`, `reduce`,
and `each while`. It was suggested at the time that it should be removed
from `for` as well, but for several reasons it wasn't.
This PR deprecates `--numbered` in anticipation of removing it in 0.96.
Note: Please review carefully, as this is my first "real" Rust/Nushell
code. I was hoping that some prior commit would be useful as a template,
but since this was an argument on a parser keyword, I didn't find too
much useful. So I had to actually find the relevant helpers in the code
and `nu_protocol` doc and learn how to use them - oh darn ;-) But please
make sure I did it correctly.
# User-Facing Changes
* Use of `--numbered` will result in a deprecation warning.
* Changed help example to demonstrate the new syntax.
* Help shows deprecation notice on the flag
Per discussion on discord dataframes channel with @maxim-uvarov and pyz.
When converting a dataframe to an nushell value via `polars into-nu`,
the index column should not be added by default and should only be added
when specifying `--index`
As reported by @maxim-uvarov and pyz in the dataframes discord channel:
```nushell
[[a b]; [1 1] [1 2] [2 1] [2 2] [3 1] [3 2]] | polars into-df | polars with-column ((polars col a) / (polars col b)) --name c
× Type mismatch.
╭─[entry #45:1:102]
1 │ [[a b]; [1 1] [1 2] [2 1] [2 2] [3 1] [3 2]] | polars into-df | polars with-column ((polars col a) / (polars col b)) --name c
· ───────┬──────
· ╰── Right hand side not a dataframe expression
╰────
```
This pull request corrects the type casting on the right hand side and
allows more than just polars literal expressions.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Part of https://github.com/nushell/nushell/issues/12963, step 2.
This PR refactors changes the use of `expression.span` to
`expression.span_id` via a new helper `Expression::span()`. A new
`GetSpan` is added to abstract getting the span from both `EngineState`
and `StateWorkingSet`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`format pattern` loses the ability to use variables in the pattern,
e.g., `... | format pattern 'value of {$it.name} is {$it.value}'`. This
is because the command did a custom parse-eval cycle, creating spans
that are not merged into the main engine state. We could clone the
engine state, add Clone trait to StateDelta and merge the cloned delta
to the cloned state, but IMO there is not much value from having this
ability, since we have string interpolation nowadays: `... | $"value of
($in.name) is ($in.value)"`.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes#13093 by:
* Removing the mentioned help example
* Updating the `--accessed` and `--modified` flag descriptions to remove
mention of "timestamp/date"
# User-Facing Changes
Help changes
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
Related to #12832, this PR changes the way `help commands` displays the
command type to be consistent with `scope commands` and `which`.
# User-Facing Changes
Technically a breaking change since the `help commands` output can now
be different.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close https://github.com/nushell/nushell/issues/12874
- fixes https://github.com/nushell/nushell/issues/12874
I want to fix the issue which is induced by the fix for
https://github.com/nushell/nushell/issues/12369. after this pr. This pr
induced a new error for unix system, in order to show coredump messages
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
- this PR should close https://github.com/nushell/nushell/issues/12874
- fixes https://github.com/nushell/nushell/issues/12874
I want to fix the issue which is induced by the fix for
https://github.com/nushell/nushell/issues/12369. after this pr. This pr
induced a new error for unix system, in order to show coredump messages
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
after fix for 12874, coredump message is messing, so I want to fix it
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
Makes `to toml` use the `toml::value::Datetime` type, so that `to toml`
serializes dates properly.
# User-Facing Changes
`to toml` will now encode dates differently, in a native format instead
of a string. This could, in theory, break some workflows:
```Nushell
# Before:
~> {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
Fri, 31 May 2024 00:00:00 +0000 (10 hours ago)
# After:
~> {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
Error: nu:🐚:only_supports_this_input_type
× Input type not supported.
╭─[entry #13:1:36]
1 │ {datetime: 2024-05-31} | to toml | from toml | get datetime | into datetime
· ────┬──── ──────┬──────
· │ ╰── only string and int input data is supported
· ╰── input type: date
╰────
```
Fix#11751
# Description
This pr is going to use `pathdiff::diff_path`, so we don't need to
handle for relative path by ourselves.
This is also the behavior before the rewritten of run_external.rs
It's a follow up to https://github.com/nushell/nushell/pull/13028
# User-Facing Changes
NaN
# Tests + Formatting
No need to add tests
- this PR should close#11433
# Description
This PR implements pipeline input support for the stor insert and stor
update commands,
enabling users to directly pass data to these commands without relying
solely on flag parameters.
Previously, it was only possible to specify the record data using flag
parameters,
which could be less intuitive and become cumbersome:
```bash
stor insert --table-name nudb --data-record {bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17}
stor update --table-name nudb --update-record {str1: nushell datetime1: 2020-04-17}
```
Now it is also possible to pass a record through pipeline input:
```bash
{bool1: true, int1: 5, float1: 1.1, str1: fdncred, datetime1: 2023-04-17} | stor insert --table-name nudb
{str1: nushell datetime1: 2020-04-17} | stor update --table-name nudb"
```
Changes made on code:
- Modified stor insert and stor update to accept a record from the
pipeline.
- Added logic to handle data from the pipeline record.
- Implemented an error case to prevent simultaneous data input from both
pipeline and flag.
# User-facing changes
Returns an error when both ways of inserting data are used.
The examples for both commands were updated and in each command, when
the -d or -u fags are being used at the same time as input is being
passed through the pipeline, it returns an error:
Also returns an error when both of them are missing:
# Tests + Formating
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
Co-authored-by: Rodrigo Friães <rodrigo.friaes@tecnico.ulisboa.pt>
Configuration in `explore` has always been confusing to me. This PR
overhauls (and simplifies, I think) how configuration is done.
# Details
1. Configuration is now strongly typed. In `Explore::run()` we create an
`ExploreConfig` struct from the more general Nu configuration and
arguments to `explore`, then pass that struct to other parts of
`explore` that need configuration. IMO this is a lot easier to reason
about and trace than the previous approach of creating a
`HashMap<String, Value>` and then using that to make various structs
elsewhere.
2. We now inherit more configuration from the config used for regular Nu
tables
1. Border/line styling now uses the `separator` style used for regular
Nu tables, the special `explore.split_line` config point has been
retired.
2. Cell padding in tables is now controlled by `table.padding` instead
of the undocumented `column_padding_left`/`column_padding_right` config
3. The (optional, previously not enabled by default) `selected_row` and
`selected_column` configuration has been removed. We now only highlight
the selected cell. I could re-add this if people really like the feature
but I'm guessing nobody uses it.
The interface still looks the same with a default/empty config.nu:
# Description
Fixes: #13066
nushell should remove argument values' inner quote once it gets `=`.
Whatever it's a flag or not, and it also replace from `\"` to `"` before
passing it to external commands.
# User-Facing Changes
Given the shell script:
```shell
# test.sh
echo $@
```
## Before
```
> sh test.sh -ldflags="-s -w" github.com
-ldflags="-s -w" github.com
> sh test.sh exp='-s -w' github.com
exp='-s -w' github.com
```
## After
```
> sh test.sh -ldflags="-s -w" github.com
-ldflags=-s -w github.com
> sh test.sh exp='-s -w' github.com
exp=-s -w github.com
```
# Tests + Formatting
Added some tests
This fixes up a panic I accidentally introduced when refactoring the
cursor code in `explore`: https://github.com/nushell/nushell/pull/12979
Under certain circumstances (running `:nu []`, opening `:try` with the
hidden `try.reactive` setting enabled), `explore` would panic when
handling an empty list. To fix this for now I've removed the validation
I added to the Cursor constructor in that PR.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
Related meta-issue: #10239.
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR will modify some `str`-related commands so that they can be
evaluated at parse time.
See the following list for those implemented by this pr.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Available now:
- `str` subcommands
- `trim`
- `contains`
- `distance`
- `ends-with`
- `expand`
- `index-of`
- `join`
- `replace`
- `reverse`
- `starts-with`
- `stats`
- `substring`
- `capitalize`
- `downcase`
- `upcase`
- `split` subcommands
- `chars`
- `column`
- `list`
- `row`
- `words`
- `format` subcommands
- `date`
- `duration`
- `filesize`
- string related commands
- `parse`
- `detect columns`
- `encode` & `decode`
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Unresolved questions:
- [ ] Is there any routine of testing const expressions? I haven't found
any yet.
- [ ] Is const expressions required to behave just like there non-const
version, like what rust promises?
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Unresolved questions:
- [ ] Do const commands need special marks in the docs?
# Description
Per a Discord question
(https://discord.com/channels/601130461678272522/1244293194603167845/1247794228696711198),
this adds examples to the `help` for both:
* `cd`
* `def`
to demonstrate that `def --env` is required when changing directories in
a custom command.
Since the existing examples for `def` were a bit more complex (and had
output) but the `cd` ones were more simplified, I did use slightly
different examples in each. Either or both could be tweaked if desired.
# User-Facing Changes
Command `help` examples
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
N/A
---------
Co-authored-by: Jakub Žádník <kubouch@gmail.com>
This one trace message creates thousands of lines of trace messages that
is probably
better suited to using on an *as needed* basis rather than everyone
having to wade
through it...
For now, I just commented it out but eventually this line of code should
be removed
and used simply for the time when someone needs to see it...
`explore` has 3 cursor-related structs that are extensively used to
track the currently shown "window" of the data being shown. I was
finding the cursor code quite difficult to follow, so this PR:
- rewrites the base `Cursor` struct from scratch, with some tests
- makes big changes to `WindowCursor`
- renames `XYCursor` to `WindowCursor2D`
- makes some of the cursor functions fallible as a start towards better
error handling
- changes lots of function names to things that I find more intuitive
- adds comments, including ASCII diagrams to explain how the cursors
work
More work could be done (I'd like to review/change more function names
in `WindowCursor` and `WindowCursor2D` and add more tests), but this is
the limit of what I can get done in a weekend. I think this part of the
code is in a better place now.
# Testing performed
I did a lot of manual testing in the record view and binary viewer,
moving around with arrow keys / page up+down / home+end.
This can definitely wait until after the release freeze, this area has
very few automated tests and it'd be good to let the changes bake a bit.
# Description
First part of SpanID refactoring series. This PR adds a `SpanId` type
and a corresponding `span_id` field to `Expression`. Parser creating
expressions will now add them to an array in `StateWorkingSet`,
generates a corresponding ID and saves the ID to the Expression. The IDs
are not used anywhere yet.
For the rough overall plan, see
https://github.com/nushell/nushell/issues/12963.
# User-Facing Changes
Hopefully none. This is only a refactor of Nushell's internals that
shouldn't have visible side effects.
# Tests + Formatting
# After Submitting
# Description
`query xml` used to return results from an XPath query in a random,
non-deterministic order. With this change, results get returned in the
order they appear in the document.
# User-Facing Changes
`query xml` will now return results in a non-random order.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
i was looking at the website documentation of `std assert` and i noticed
one thing
- the `--error-label` argument of `assert` and `assert not` was just a
`record` -> now it's that complete type `record<text: string, span:
record<start: int, end: int>>`
# Description
Enable the `preserve_order` feature of the `toml` crate to preserve the
ordering of elements when converting from/to toml.
Additionally, use `to_string_pretty()` instead of `to_string()` in `to
toml`. This displays arrays on multiple lines instead of one big single
line. I'm not sure if this one is a good idea or not... Happy to remove
this from this PR if it's not.
# User-Facing Changes
The order of elements will be different when using `from toml`. The
formatting of arrays will also be different when using `to toml`. For
example:
- before
```
❯ "foo=1\nbar=2\ndoo=3" | from toml
╭─────┬───╮
│ bar │ 2 │
│ doo │ 3 │
│ foo │ 1 │
╰─────┴───╯
❯ {a: [a b c d]} | to toml
a = ["a", "b", "c", "d"]
```
- after
```
❯ "foo=1\nbar=2\ndoo=3" | from toml
╭─────┬───╮
│ foo │ 1 │
│ bar │ 2 │
│ doo │ 3 │
╰─────┴───╯
❯ {a: [a b c d]} | to toml
a = [
"a",
"b",
"c",
"d",
]
```
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🔴 `toolkit test`
- ⚫ `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes https://github.com/nushell/nushell/issues/12968. After apply this
patch, we can use explict plus sign character included string with `into
filesize` cmd.
# User-Facing Changes
AS-IS (before fixing)
```
$ "+8 KiB" | into filesize
Error: nu:🐚:cant_convert
× Can't convert to int.
╭─[entry #31:1:1]
1 │ "+8 KiB" | into filesize
· ────┬───
· ╰── can't convert string to int
╰────
```
TO-BE (after fixing)
```
$ "+8KiB" | into filesize
8.0 KiB
```
# Tests + Formatting
Added a test
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Closes#12980. More context there, but basically `explore` was getting
file metadata for every row every time the record view was rendered. The
quick fix for now is to do the `LS_COLORS` colouring with a `&str`
instead of a path and file metadata.
# Description
Fix#13021
This changes the `expand_glob()` function to use
`nu_engine::glob_from()` so that absolute paths are actually preserved,
rather than being made relative to the provided parent. This preserves
the intent of whoever wrote the original path/glob, and also makes it so
that tilde always produces absolute paths.
I also made `expand_glob()` handle Ctrl-C so that it can be interrupted.
cc @YizhePKU
# Tests + Formatting
No additional tests here... but that might be a good idea.
# Description
Fixes#13016 and adds tests for many variations of external call
parsing.
I just realized @kubouch took a crack at this too (#13022) so really
whichever is better, but I think the
tests are a good addition.
# Description
Makes `run-external` error if arguments to `cmd.exe` internal commands
contain newlines or a percent sign. This is because the percent sign can
expand environment variables, potentially? allowing command injection.
Newlines I think will truncate the rest of the arguments and should
probably be disallowed to be safe.
# After Submitting
- If the user calls `cmd.exe` directly, then this bypasses our
handling/checking for internal `cmd` commands. Instead, we use the
handling from the Rust std lib which, in this case, does not do special
handling and is potentially unsafe. Then again, it could be the user's
specific intention to run `cmd` with whatever trusted input. The problem
is that since we use the std lib handling, it assumes the exe uses the C
runtime escaping rules and will perform some unwanted escaping. E.g., it
will add backslashes to the quotes in `cmd echo /c '""'`.
- If `cmd` is called indirectly via a `.bat` or `.cmd` file, then we use
the Rust std lib which has separate handling for bat files that should
be safe, but will reject some inputs.
- ~~I'm not sure how we handle `PATHEXT`, that can also cause a file
without an extension to be run as a bat file. If so, I don't know where
the handling, if any, is done for that.~~ It looks like we use the
`which` crate to do the lookup using `PATHEXT`. Then, we pass the exe
path from that to the Rust std lib `Command`, which should be safe
(except for the first `cmd.exe` note).
So, in the future we need to unify and/or fix these different
implementations, including our own special handling for internal `cmd`
commands that this PR tries to fix.
# Description
Fix a regression introduced by #12921, where tilde expansion was no
longer done on the external command name, breaking things like
```nushell
> ~/.cargo/bin/exa
```
This properly handles quoted strings, so they don't expand:
```nushell
> ^"~/.cargo/bin/exa"
Error: nu:🐚:external_command
× External command failed
╭─[entry #1:1:2]
1 │ ^"~/.cargo/bin/exa"
· ─────────┬────────
· ╰── Command `~/.cargo/bin/exa` not found
╰────
help: `~/.cargo/bin/exa` is neither a Nushell built-in or a known external command
```
This required a change to the parser, so the command name is also parsed
in the same way the arguments are - i.e. the quotes on the outside
remain in the expression. Hopefully that doesn't break anything else. 🤞Fixes#13000. Should include in patch release 0.94.1
cc @YizhePKU
# User-Facing Changes
- Tilde expansion now works again for external commands
- The `command` of `run-external` will now have its quotes removed like
the other arguments if it is a literal string
- The parser is changed to include quotes in the command expression of
`ExternalCall` if they were present
# Tests + Formatting
I would like to add a regression test for this, but it's complicated
because we need a well-known binary within the home directory, which
just isn't a thing. We could drop one there, but that's kind of a bad
behavior for a test to do. I also considered changing the home directory
for the test, but that's so platform-specific - potentially could get it
working on specific platforms though. Changing `HOME` env on Linux
definitely works as far as tilde expansion works.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This fixes a bug in the `OSC 9;9` functionality where the path wasn't
being constructed properly and therefore wasn't getting set right for
things like "Duplicate Tab" in Windows Terminal. Thanks to @Araxeus for
finding it.
Related to https://github.com/nushell/nushell/issues/10166
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This PR fixes the `path type` command so that it resolves relative paths
using PWD from the engine state.
As a bonus, it also fixes the issue of `path type` returning an empty
string instead of an error when it fails.
This PR fixes a bug where `.` is expanded into an empty string when used
as an argument to external commands. Fixes
https://github.com/nushell/nushell/issues/12948.
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
# Description
@maxim-uvarov did a ton of research and work with the dply-rs author and
ritchie from polars and found out that the allocator matters on macos
and it seems to be what was messing up the performance of polars plugin.
ritchie suggested to use jemalloc but i switched it to mimalloc to match
nushell and it seems to run better.
## Before (default allocator)
note - using 1..10 vs 1..100 since it takes so long. also notice how
high the `max` timings are compared to mimalloc below.
```nushell
❯ 1..10 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} | | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬─────────────────────────╮
│ mean │ 4sec 999ms 605µs 995ns │
│ min │ 983ms 627µs 42ns │
│ max │ 13sec 398ms 135µs 791ns │
│ stddev │ 3sec 476ms 479µs 939ns │
╰────────┴─────────────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 10
╭───────┬────────────────────────╮
│ mean │ 6sec 220ms 783µs 983ns │
│ min │ 1sec 184ms 997µs 708ns │
│ max │ 18sec 882ms 81µs 708ns │
│ std │ 5sec 350ms 375µs 697ns │
│ times │ [list 10 items] │
╰───────┴────────────────────────╯
```
## After (using mimalloc)
```nushell
❯ 1..100 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} | | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬───────────────────╮
│ mean │ 103ms 728µs 902ns │
│ min │ 97ms 107µs 42ns │
│ max │ 149ms 430µs 84ns │
│ stddev │ 5ms 690µs 664ns │
╰────────┴───────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬───────────────────╮
│ mean │ 103ms 620µs 195ns │
│ min │ 97ms 541µs 166ns │
│ max │ 130ms 262µs 166ns │
│ std │ 4ms 948µs 654ns │
│ times │ [list 100 items] │
╰───────┴───────────────────╯
```
## After (using jemalloc - just for comparison)
```nushell
❯ 1..100 | each {timeit {polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} | | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬───────────────────╮
│ mean │ 113ms 939µs 777ns │
│ min │ 108ms 337µs 333ns │
│ max │ 166ms 467µs 458ns │
│ stddev │ 6ms 175µs 618ns │
╰────────┴───────────────────╯
❯ use std bench
❯ bench { polars open Data7602DescendingYearOrder.csv | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬───────────────────╮
│ mean │ 114ms 363µs 530ns │
│ min │ 108ms 804µs 833ns │
│ max │ 143ms 521µs 459ns │
│ std │ 5ms 88µs 56ns │
│ times │ [list 100 items] │
╰───────┴───────────────────╯
```
## After (using parquet + mimalloc)
```nushell
❯ 1..100 | each {timeit {polars open data.parquet | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null}} | | {mean: ($in | math avg), min: ($in | math min), max: ($in | math max), stddev: ($in | into int | into float | math stddev | into int | $'($in)ns' | into duration)}
╭────────┬──────────────────╮
│ mean │ 34ms 255µs 492ns │
│ min │ 31ms 787µs 250ns │
│ max │ 76ms 408µs 416ns │
│ stddev │ 4ms 472µs 916ns │
╰────────┴──────────────────╯
❯ use std bench
❯ bench { polars open data.parquet | polars group-by year | polars agg (polars col geo_count | polars sum) | polars collect | null } -n 100
╭───────┬──────────────────╮
│ mean │ 34ms 897µs 562ns │
│ min │ 31ms 518µs 542ns │
│ max │ 65ms 943µs 625ns │
│ std │ 3ms 450µs 741ns │
│ times │ [list 100 items] │
╰───────┴──────────────────╯
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Instead of returning an error, this PR changes `expand_glob` in
`run_external.rs` to return the original string arg if glob creation
failed. This makes it so that, e.g.,
```nushell
^echo `[`
^echo `***`
```
no longer fail with a shell error. (This follows from #12921.)
# Description
Currently, this pipeline doesn't work `open --raw file | take 100`,
since the type of the byte stream is `Unknown`, but `take` expects
`Binary` streams. This PR changes commands that expect
`ByteStreamType::Binary` to also work with `ByteStreamType::Unknown`.
This was done by adding two new methods to `ByteStreamType`:
`is_binary_coercible` and `is_string_coercible`. These return true if
the type is `Unknown` or matches the type in the method name.
# Description
Makes the `from json --objects` command produce a stream, and read
lazily from an input stream to produce its output.
Also added a helper, `PipelineData::get_type()`, to make it easier to
construct a wrong type error message when matching on `PipelineData`. I
expect checking `PipelineData` for either a string value or an `Unknown`
or `String` typed `ByteStream` will be very, very common. I would have
liked to have a helper that just returns a readable stream from either,
but that would either be a bespoke enum or a `Box<dyn BufRead>`, which
feels like it wouldn't be so great for performance. So instead, taking
the approach I did here is probably better - having a function that
accepts the `impl BufRead` and matching to use it.
# User-Facing Changes
- `from json --objects` no longer collects its input, and can be used
for large datasets or streams that produce values over time.
# Tests + Formatting
All passing.
# After Submitting
- [ ] release notes
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
This reverts commit 68adc4657f.
# Description
Reverts the lazyframe refactor (#12669) for the next release, since
there are still a few lingering issues. This temporarily solves #12863
and #12828. After the release, the lazyframes can be added back and
cleaned up.
