1415 Commits

Author	SHA1	Message	Date
Yash Thakur	f6853fd636	Use XDG_CONFIG_HOME before default config directory (#12118) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Closes #12103 # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> As described in #12103, this PR makes Nushell use `XDG_CONFIG_HOME` as the config directory if it exists. Otherwise, it uses the old behavior, which was to use `dirs_next::config_dir()`. Edit: We discussed choosing between `XDG_CONFIG_HOME` and the default config directory in Discord and decided against it, at least for now. <s>@kubouch also suggested letting users choose between `XDG_CONFIG_HOME` and the default config directory if config files aren't found on startup and `XDG_CONFIG_HOME` is set to a value different from the default config directory</s> On Windows and MacOS, if the `XDG_CONFIG_HOME` variable is set but `XDG_CONFIG_HOME` is either empty or doesn't exist *and* the old config directory is non-empty, Nushell will issue a warning on startup saying that it won't move files from the old config directory to the new one. To do this, I had to add a `nu_path::config_dir_old()` function. I assume that at some point, we will remove the warning message and the function can be removed too. Alternatively, instead of having that function there, `main.rs` could directly call `dirs_next::config_dir()`. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> When `$env.XDG_CONFIG_HOME` is set to an absolute path, Nushell will use `$"($env.XDG_CONFIG_HOME)/nushell"` as its config directory (previously, this only worked on Linux). To use `App Data\Roaming` (Windows) or `Library/Application Support` (MacOS) instead (the old behavior), one can either leave `XDG_CONFIG_HOME` unset or set it to an empty string. If `XDG_CONFIG_HOME` is set, but to a non-absolute/invalid path, Nushell will report an error on startup and use the default config directory instead:  On Windows and MacOS, if the `XDG_CONFIG_HOME` variable is set but `XDG_CONFIG_HOME` is either empty or doesn't exist *and* the old config directory is non-empty, Nushell will issue a warning on startup saying that it won't move files from the old config directory to the new one.  # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> The existing config path tests have been modified to use `XDG_CONFIG_HOME` to change the config directory on all OSes, not just Linux. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> The documentation will have to be updated to note that Nushell uses `XDG_CONFIG_HOME` now. As @fdncred pointed out, it's possible for people to set `XDG_CONFIG_HOME` to, say, `~/.config/nushell` rather than `~/.config`, so the documentation could warn about that mistake.	2024-03-11 06:15:46 -05:00
Stefan Holderbach	f695ba408a	Restructure nu-protocol in more meaningful units (#11917) ... This is partially "feng-shui programming" of moving things to new separate places. The later commits include "`git blame` tollbooths" by moving out chunks of code into new files, which requires an extra step to track things with `git blame`. We can negiotiate if you want to keep particular things in their original place. If egregious I tried to add a bit of documentation. If I see something that is unused/unnecessarily `pub` I will try to remove that. - Move `nu_protocol::Exportable` to `nu-parser` - Guess doccomment for `Exportable` - Move `Unit` enum from `value` to `AST` - Move engine state `Variable` def into its folder - Move error-related files in `nu-protocol` subdir - Move `pipeline_data` module into its own folder - Move `stream.rs` over into the `pipeline_data` mod - Move `PipelineMetadata` into its own file - Doccomment `PipelineMetadata` - Remove unused `is_leap_year` in `value/mod` - Note about criminal `type_compatible` helper - Move duration fmting into new `value/duration.rs` - Move filesize fmting logic to new `value/filesize` - Split reexports from standard imports in `value/mod` - Doccomment trait `CustomValue` - Polish doccomments and intradoc links	2024-03-10 18:45:45 +01:00
Devyn Cairns	bc19be25b1	Keep plugins persistently running in the background (#12064) ... # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere	2024-03-09 17:10:22 -06:00
Devyn Cairns	430fb1fcb6	Add support for engine calls from plugins (#12029) ... # Description This allows plugins to make calls back to the engine to get config, evaluate closures, and do other things that must be done within the engine process. Engine calls can both produce and consume streams as necessary. Closures passed to plugins can both accept stream input and produce stream output sent back to the plugin. Engine calls referring to a plugin call's context can be processed as long either the response hasn't been received, or the response created streams that haven't ended yet. This is a breaking API change for plugins. There are some pretty major changes to the interface that plugins must implement, including: 1. Plugins now run with `&self` and must be `Sync`. Executing multiple plugin calls in parallel is supported, and there's a chance that a closure passed to a plugin could invoke the same plugin. Supporting state across plugin invocations is left up to the plugin author to do in whichever way they feel best, but the plugin object itself is still shared. Even though the engine doesn't run multiple plugin calls through the same process yet, I still considered it important to break the API in this way at this stage. We might want to consider an optional threadpool feature for performance. 2. Plugins take a reference to `EngineInterface`, which can be cloned. This interface allows plugins to make calls back to the engine, including for getting config and running closures. 3. Plugins no longer take the `config` parameter. This can be accessed from the interface via the `.get_plugin_config()` engine call. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Not only does this have plugin protocol changes, it will require plugins to make some code changes before they will work again. But on the plus side, the engine call feature is extensible, and we can add more things to it as needed. Plugin maintainers will have to change the trait signature at the very least. If they were using `config`, they will have to call `engine.get_plugin_config()` instead. If they were using the mutable reference to the plugin, they will have to come up with some strategy to work around it (for example, for `Inc` I just cloned it). This shouldn't be such a big deal at the moment as it's not like plugins have ever run as daemons with persistent state in the past, and they don't in this PR either. But I thought it was important to make the change before we support plugins as daemons, as an exclusive mutable reference is not compatible with parallel plugin calls. I suggest this gets merged sometime *after* the current pending release, so that we have some time to adjust to the previous plugin protocol changes that don't require code changes before making ones that do. # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting I will document the additional protocol features (`EngineCall`, `EngineCallResponse`), and constraints on plugin call processing if engine calls are used - basically, to be aware that an engine call could result in a nested plugin call, so the plugin should be able to handle that.	2024-03-09 11:26:30 -06:00
Raphael Gaschignard	d8f13b36b1	Allow for stacks to have parents (#11654) ... This is another attempt on #11288 This allows for a `Stack` to have a parent stack (behind an `Arc`). This is being added to avoid constant stack copying in REPL code. Concretely the following changes are included here: - `Stack` can now have a `parent_stack`, pointing to another stack - variable lookups can fallback to this parent stack (env vars and everything else is still copied) - REPL code has been reworked so that we use parenting rather than cloning. A REPL-code-specific trait helps to ensure that we do not accidentally trigger a full clone of the main stack - A property test has been added to make sure that parenting "looks the same" as cloning for consumers of `Stack` objects --------- Co-authored-by: Raphael Gaschignard <rtpg@rokkenjima.local> Co-authored-by: Ian Manske <ian.manske@pm.me>	2024-03-09 17:55:39 +01:00
Jakub Žádník	14d1c67863	Debugger experiments (#11441) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a new evaluator path with callbacks to a mutable trait object implementing a Debugger trait. The trait object can do anything, e.g., profiling, code coverage, step debugging. Currently, entering/leaving a block and a pipeline element is marked with callbacks, but more callbacks can be added as necessary. Not all callbacks need to be used by all debuggers; unused ones are simply empty calls. A simple profiler is implemented as a proof of concept. The debugging support is implementing by making `eval_xxx()` functions generic depending on whether we're debugging or not. This has zero computational overhead, but makes the binary slightly larger (see benchmarks below). `eval_xxx()` variants called from commands (like `eval_block_with_early_return()` in `each`) are chosen with a dynamic dispatch for two reasons: to not grow the binary size due to duplicating the code of many commands, and for the fact that it isn't possible because it would make Command trait objects object-unsafe. In the future, I hope it will be possible to allow plugin callbacks such that users would be able to implement their profiler plugins instead of having to recompile Nushell. [DAP](https://microsoft.github.io/debug-adapter-protocol/) would also be interesting to explore. Try `help debug profile`. ## Screenshots Basic output:  To profile with more granularity, increase the profiler depth (you'll see that repeated `is-windows` calls take a large chunk of total time, making it a good candidate for optimizing):  ## Benchmarks ### Binary size Binary size increase vs. main: **+40360 bytes**. _(Both built with `--release --features=extra,dataframe`.)_ ### Time ```nushell # bench_debug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'debug:' let res2 = bench { debug profile $test } --pretty print $res2 ``` ```nushell # bench_nodebug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'no debug:' let res1 = bench { do $test } --pretty print $res1 ``` `cargo run --release -- bench_debug.nu` is consistently 1--2 ms slower than `cargo run --release -- bench_nodebug.nu` due to the collection overhead + gathering the report. This is expected. When gathering more stuff, the overhead is obviously higher. `cargo run --release -- bench_nodebug.nu` vs. `nu bench_nodebug.nu` I didn't measure any difference. Both benchmarks report times between 97 and 103 ms randomly, without one being consistently higher than the other. This suggests that at least in this particular case, when not running any debugger, there is no runtime overhead. ## API changes This PR adds a generic parameter to all `eval_xxx` functions that forces you to specify whether you use the debugger. You can resolve it in two ways: * Use a provided helper that will figure it out for you. If you wanted to use `eval_block(&engine_state, ...)`, call `let eval_block = get_eval_block(&engine_state); eval_block(&engine_state, ...)` * If you know you're in an evaluation path that doesn't need debugger support, call `eval_block::<WithoutDebug>(&engine_state, ...)` (this is the case of hooks, for example). I tried to add more explanation in the docstring of `debugger_trait.rs`. ## TODO - [x] Better profiler output to reduce spam of iterative commands like `each` - [x] Resolve `TODO: DEBUG` comments - [x] Resolve unwraps - [x] Add doc comments - [x] Add usage and extra usage for `debug profile`, explaining all columns # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Hopefully none. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-03-08 20:21:35 +02:00
Ian Manske	dfe072fd30	Fix chrono deprecation warnings (#12091) ... # Description Bumps `chrono` to 0.4.35 and fixes any deprecation warnings.	2024-03-07 06:01:30 -06:00
Ian Manske	a18de999c2	Fix broken doc link (#12092) ... Fixes a doc comment link in `Value::to_parsable_string`.	2024-03-06 19:50:31 -08:00
Yash Thakur	4cda183103	Canonicalize default-config-dir and plugin-path (#11999) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR makes sure `$nu.default-config-dir` and `$nu.plugin-path` are canonicalized. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> `$nu.default-config-dir` (and `$nu.plugin-path`) will now give canonical paths, with symlinks and whatnot resolved. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> I've added a couple of tests to check that even if the config folder and/or any of the config files within are symlinks, the `$nu.*` variables are properly canonicalized. These tests unfortunately only run on Linux and MacOS, because I couldn't figure out how to change the config directory on Windows. Also, given that they involve creating files, I'm not sure if they're excessive, so I could remove one or two of them. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-03-02 11:15:31 -06:00
Devyn Cairns	626d597527	Replace panics with errors in thread spawning (#12040) ... # Description Replace panics with errors in thread spawning. Also adds `IntoSpanned` trait for easily constructing `Spanned`, and an implementation of `From<Spanned<std::io::Error>>` for `ShellError`, which is used to provide context for the error wherever there was a span conveniently available. In general this should make it more convenient to do the right thing with `std::io::Error` and always add a span to it when it's possible to do so. # User-Facing Changes Fewer panics! # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib`	2024-03-02 11:14:02 -06:00
Wind	387328fe73	Glob: don't allow implicit casting between glob and string (#11992) ... # Description As title, currently on latest main, nushell confused user if it allows implicit casting between glob and string: ```nushell let x = "*.txt" def glob-test [g: glob] { open $g } glob-test $x ``` It always expand the glob although `$x` is defined as a string. This pr implements a solution from @kubouch : > We could make it really strict and disallow all autocasting between globs and strings because that's what's causing the "magic" confusion. Then, modify all builtins that accept globs to accept oneof(glob, string) and the rules would be that globs always expand and strings never expand # User-Facing Changes After this pr, user needs to use `into glob` to invoke `glob-test`, if user pass a string variable: ```nushell let x = "*.txt" def glob-test [g: glob] { open $g } glob-test ($x | into glob) ``` Or else nushell will return an error. ``` 3 │ glob-test $x · ─┬ · ╰── can't convert string to glob ``` # Tests + Formatting Done # After Submitting Nan	2024-02-28 23:05:35 +08:00
Yash Thakur	c0ff0f12f0	Add ConfigDirNotFound error (#11849) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Currently, there's multiple places that look for a config directory, and each of them has different error messages when it can't be found. This PR makes a `ConfigDirNotFound` error to standardize the error message for all of these cases. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Previously, the errors in `create_nu_constant()` would say which config file Nushell was trying to get when it couldn't find the config directory. Now it doesn't. However, I think that's fine, given that it doesn't matter whether it couldn't find the config directory while looking for `login.nu` or `env.nu`, it only matters that it couldn't find it. This is what the error looks like:  # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: Antoine Stevan <44101798+amtoine@users.noreply.github.com>	2024-02-26 15:42:20 +08:00
Devyn Cairns	88f1f386bb	Bidirectional communication and streams for plugins (#11911)	2024-02-25 16:32:50 -06:00
Devyn Cairns	461f69ac5d	Rename spans in the serialized form of Value (#11972) ... [Discord context](https://discord.com/channels/601130461678272522/615962413203718156/1211158641793695744) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Span fields were previously renamed to `internal_span` to discourage their use in Rust code, but this change also affected the serde I/O for Value. I don't believe the Python plugin was ever updated to reflect this change. This effectively changes it back, but just for the serialized form. There are good reasons for doing this: 1. `internal_span` is a much longer name, and would be one of the most common strings found in serialized Value data, probably bulking up the plugin I/O 2. This change was never really meant to have implications for plugins, and was just meant to be a hint that `.span()` should be used instead in Rust code. When Span refactoring is complete, the serialized form of Value will probably change again in some significant way, so I think for now it's best that it's left like this. This has implications for #11911, particularly for documentation and for the Python plugin as that was already updated in that PR to reflect `internal_span`. If this is merged first, I will update that PR. This would probably be considered a breaking change as it would break plugin I/O compatibility (but not Rust code). I think it can probably go in any major release though - all things considered, it's pretty minor, and users are already expected to recompile plugins for new major versions. However, it may also be worth holding off to do it together with #11911 as that PR makes breaking changes in general a little bit friendlier. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Requires plugin recompile. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` Nothing outside of `Value` itself had to be changed to make tests pass. I did not check the Python plugin and whether it works now, but it was broken before. It may work again as I think the main incompatibility it had was expecting to use `span` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-25 15:49:10 -06:00
Stefan Holderbach	7884de1941	Remove some unnecessary static Vecs (#11947) ... Avoid unnecessary allocations or larger iterator structs - Turn static `Vec`s into arrays when possible - Use `std::iter::once`/`empty` where applicable - Use `bool::then_some` in `detect column` `.chain` - Drop in the bucket: de-vec-ing tests	2024-02-24 20:58:01 +01:00
Devyn Cairns	098527b263	Print stderr streams to stderr in pipeline_data::print_if_stream() (#11929) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Related to #11928 - `tee --stderr` doesn't really work as expected without it # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Print stderr streams to stderr in `pipeline_data::print_if_stream()` This corrects unexpected behavior if a stream from an external program is transformed while still preserving its stderr output. Before this change, that output is just drained and discarded. Worse, it's drained to a buffer, which could be really slow and memory hungry if there's a lot of output on stderr. This is needed to make `tee --stderr` function in a non-surprising way. See #11928 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> A script that was erroneously not producing stderr output before might now, but I can't think of a lot of examples of an external stream being transformed without being converted. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-24 15:32:39 +00:00
Wind	f7d647ac3c	open, rm, umv, cp, rm and du: Don't globs if inputs are variables or string interpolation (#11886) ... # Description This is a follow up to https://github.com/nushell/nushell/pull/11621#issuecomment-1937484322 Also Fixes: #11838 ## About the code change It applys the same logic when we pass variables to external commands: 0487e9ffcb/crates/nu-command/src/system/run_external.rs (L162-L170) That is: if user input dynamic things(like variables, sub-expression, or string interpolation), it returns a quoted `NuPath`, then user input won't be globbed # User-Facing Changes Given two input files: `a*c.txt`, `abc.txt` * `let f = "a*c.txt"; rm $f` will remove one file: `a*c.txt`. ~* `let f = "a*c.txt"; rm --glob $f` will remove `a*c.txt` and `abc.txt`~ * `let f: glob = "a*c.txt"; rm $f` will remove `a*c.txt` and `abc.txt` ## Rules about globbing with *variable* Given two files: `a*c.txt`, `abc.txt` | Cmd Type | example | Result | | ----- | ------------------ | ------ | | builtin | let f = "a*c.txt"; rm $f | remove `a*c.txt` | | builtin | let f: glob = "a*c.txt"; rm $f | remove `a*c.txt` and `abc.txt` | builtin | let f = "a*c.txt"; rm ($f \| into glob) | remove `a*c.txt` and `abc.txt` | custom | def crm [f: glob] { rm $f }; let f = "a*c.txt"; crm $f | remove `a*c.txt` and `abc.txt` | custom | def crm [f: glob] { rm ($f \| into string) }; let f = "a*c.txt"; crm $f | remove `a*c.txt` | custom | def crm [f: string] { rm $f }; let f = "a*c.txt"; crm $f | remove `a*c.txt` | custom | def crm [f: string] { rm $f }; let f = "a*c.txt"; crm ($f \| into glob) | remove `a*c.txt` and `abc.txt` In general, if a variable is annotated with `glob` type, nushell will expand glob pattern. Or else, we need to use `into | glob` to expand glob pattern # Tests + Formatting Done # After Submitting I think `str glob-escape` command will be no-longer required. We can remove it.	2024-02-23 09:17:09 +08:00
Devyn Cairns	28f58057b6	Replace debug_assert! with assert! in Signature::check_names (#11937) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Debug assertions don't run at release, which means that `cargo test --release` fails because the tests for name checks don't run properly. These checks are not really expensive, and there shouldn't be any noticeable difference to startup time, so there isn't much reason not to just leave them in. It's valuable to be able to run `cargo test --release`, as that can expose race conditions and dependencies on undefined behavior that aren't exposed in debug builds. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> This shouldn't affect anything. Any violations of this rule were being caught with debug tests, which are run by the CI. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-22 16:17:06 -06:00
Jack Wright	f17f857b1f	wrapping run_repl with catch_unwind and restarting the repl on panic (#11860) ... Provides the ability to cleanly recover from panics, falling back to the last known good state of EngineState and Stack. This pull request also utilizes miette's panic handler for better formatting of panics. <img width="642" alt="Screenshot 2024-02-21 at 08 34 35" src="https://github.com/nushell/nushell/assets/56345/f81efaba-aa45-4e47-991c-1a2cf99e06ff"> --------- Co-authored-by: Jack Wright <jack.wright@disqo.com>	2024-02-22 12:14:10 -06:00
Yash Thakur	6ff3a4180b	Specify which file not found in error (#11868) ... # Description Currently, `ShellError::FileNotFound` shows the span where the error occurred but doesn't say which file wasn't found. This PR makes it so the help includes that (like the `DirectoryNotFound` error). # User-Facing Changes No breaking changes, it's just that when a file can't be found, the help will say which file couldn't be found: 	2024-02-21 21:27:13 +08:00
Stefan Holderbach	6e590fe0a2	Remove unused Index(Mut) impls on AST types (#11903) ... # Description Both `Block` and `Pipeline` had `Index`/`IndexMut` implementations to access their elements, that are currently unused. Explicit helpers or iteration would generally be preferred anyways but in the current state the inner containers are `pub` and are liberally used. (Sometimes with potentially panicking indexing or also iteration) As it is potentially unclear what the meaning of the element from a block or pipeline queried by a usize is, let's remove it entirely until we come up with a better API. # User-Facing Changes None Plugin authors shouldn't dig into AST internals	2024-02-21 18:02:30 +08:00
David Matos	123bf2d736	fix format date based on users locale (#11908) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description Hi, Fixes #10838, where before the `date` would be formatted incorrectly, and was not picking `LC_TIME` for time formatting, but it picked the first locale returned by the `sys-locale` crate instead. Now it will format time based on `LC_TIME`. For example, ``` // my locale `nl_NL.UTF-8` ❯ date now | format date '%x %X' 20-02-24 17:17:12 $env.LC_TIME = "en_US.UTF-8" ❯ date now | format date '%x %X' 02/20/2024 05:16:28 PM ``` Note that I also changed the `default_env.nu` as otherwise the Time will show AM/PM twice. Also reason for the `chrono` update is because this relies on a fix to upstream repo, which i initially submitted an [issue](https://github.com/chronotope/chrono/issues/1349#event-11765363286) <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - [X] `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - [X] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - [X] `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - [X] `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-20 11:08:49 -06:00
Ian Manske	68fcd71898	Add Value::coerce_str (#11885) ... # Description Following #11851, this PR adds one final conversion function for `Value`. `Value::coerce_str` takes a `&Value` and converts it to a `Cow<str>`, creating an owned `String` for types that needed converting. Otherwise, it returns a borrowed `str` for `String` and `Binary` `Value`s which avoids a clone/allocation. Where possible, `coerce_str` and `coerce_into_string` should be used instead of `coerce_string`, since `coerce_string` always allocates a new `String`.	2024-02-18 17:47:10 +01:00
Ian Manske	fb4251aba7	Remove Record::from_raw_cols_vals_unchecked (#11810) ... # Description Follows from #11718 and replaces all usages of `Record::from_raw_cols_vals_unchecked` with iterator or `record!` equivalents.	2024-02-18 14:20:22 +02:00
Stefan Holderbach	28f0f32ae7	Prune unused ShellError variants (#11883) ... # Description Same procedure as #11881 Remove unused variants to avoid confusion and foster better practices around error variants. - Remove `SE::PermissionDeniedError` - Remove `SE::OutOfMemoryError` - Remove `SE::DirectoryNotFoundCustom` - Remove `SE::MoveNotPossibleSingle` - Remove `SE::NonUnicodeInput` # User-Facing Changes Plugin authors may have matched against or emitted those variants	2024-02-18 15:31:36 +08:00
Stefan Holderbach	06c590d894	Prune unused ParseError variants (#11881) ... # Description Error variants never raised should be removed to avoid confusion and make sure we choose the proper variants in the future - Remove unused `ParseError::InvalidModuleFileName` - Remove unused `ParseError::NotFound` - Remove unused `ParseError::MissingImportPattern` - Remove unused `ParseError::ReadingFile` # User-Facing Changes None for users. Insignificant for plugin authors as they interact only with `ShellError`	2024-02-18 15:31:02 +08:00
Ian Manske	1c49ca503a	Name the Value conversion functions more clearly (#11851) ... # Description This PR renames the conversion functions on `Value` to be more consistent. It follows the Rust [API guidelines](https://rust-lang.github.io/api-guidelines/naming.html#ad-hoc-conversions-follow-as_-to_-into_-conventions-c-conv) for ad-hoc conversions. The conversion functions on `Value` now come in a few forms: - `coerce_{type}` takes a `&Value` and attempts to convert the value to `type` (e.g., `i64` are converted to `f64`). This is the old behavior of some of the `as_{type}` functions -- these functions have simply been renamed to better reflect what they do. - The new `as_{type}` functions take a `&Value` and returns an `Ok` result only if the value is of `type` (no conversion is attempted). The returned value will be borrowed if `type` is non-`Copy`, otherwise an owned value is returned. - `into_{type}` exists for non-`Copy` types, but otherwise does not attempt conversion just like `as_type`. It takes an owned `Value` and always returns an owned result. - `coerce_into_{type}` has the same relationship with `coerce_{type}` as `into_{type}` does with `as_{type}`. - `to_{kind}_string`: conversion to different string formats (debug, abbreviated, etc.). Only two of the old string conversion functions were removed, the rest have been renamed only. - `to_{type}`: other conversion functions. Currently, only `to_path` exists. (And `to_string` through `Display`.) This table summaries the above: | Form | Cost | Input Ownership | Output Ownership | Converts `Value` case/`type` | | ---------------------------- | ----- | --------------- | ---------------- | -------- | | `as_{type}` | Cheap | Borrowed | Borrowed/Owned | No | | `into_{type}` | Cheap | Owned | Owned | No | | `coerce_{type}` | Cheap | Borrowed | Borrowed/Owned | Yes | | `coerce_into_{type}` | Cheap | Owned | Owned | Yes | | `to_{kind}_string` | Expensive | Borrowed | Owned | Yes | | `to_{type}` | Expensive | Borrowed | Owned | Yes | # User-Facing Changes Breaking API change for `Value` in `nu-protocol` which is exposed as part of the plugin API.	2024-02-17 18:14:16 +00:00
Yash Thakur	cb67de675e	Disallow spreading lists automatically when calling externals (#11857) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Spreading lists automatically when calling externals was deprecated in 0.89 (#11289), and this PR is to remove it in 0.91. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> The new error message looks like this: ``` > ^echo [1 2] Error: nu:🐚:cannot_pass_list_to_external × Lists are not automatically spread when calling external commands ╭─[entry #13:1:8] 1 │ ^echo [1 2] · ──┬── · ╰── Spread operator (...) is necessary to spread lists ╰──── help: Either convert the list to a string or use the spread operator, like so: ...[1 2] ``` The old error message didn't say exactly where to put the `...` and seemed to confuse a lot of people, so hopefully this helps. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> There was one test to check that implicit spread was deprecated before, updated that to check that it's disallowed now. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-14 18:16:19 -05:00
Darren Schroeder	a603b067e5	update default_config with new defaults (#11856) ... # Description Update a few defaults. 1. use_ls_colors_completeions defaults to true. 2. make ide_menu only offer 10 completions at a time with `max_completion_height = 10` instead of taking the entire screen. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-14 13:01:27 -06:00
Steven	5042f19d1b	colored file-like completions (#11702) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> `ls` and other file completions uses `LS_COLORS`.  # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>	2024-02-08 14:29:28 -06:00
Wind	58c6fea60b	Support redirect stderr and stdout+stderr with a pipe (#11708) ... # Description Close: #9673 Close: #8277 Close: #10944 This pr introduces the following syntax: 1. `e>|`, pipe stderr to next command. Example: `$env.FOO=bar nu --testbin echo_env_stderr FOO e>| str length` 2. `o+e>|` and `e+o>|`, pipe both stdout and stderr to next command, example: `$env.FOO=bar nu --testbin echo_env_mixed out-err FOO FOO e+o>| str length` Note: it only works for external commands. ~There is no different for internal commands, that is, the following three commands do the same things:~ Edit: it raises errors if we want to pipes for internal commands ``` ❯ ls e>| str length Error: × `e>|` only works with external streams ╭─[entry #1:1:1] 1 │ ls e>| str length · ─┬─ · ╰── `e>|` only works on external streams ╰──── ❯ ls e+o>| str length Error: × `o+e>|` only works with external streams ╭─[entry #2:1:1] 1 │ ls e+o>| str length · ──┬── · ╰── `o+e>|` only works on external streams ╰──── ``` This can help us to avoid some strange issues like the following: `$env.FOO=bar (nu --testbin echo_env_stderr FOO) e>| str length` Which is hard to understand and hard to explain to users. # User-Facing Changes Nan # Tests + Formatting To be done # After Submitting Maybe update documentation about these syntax.	2024-02-09 01:30:46 +08:00
nibon7	84517138bc	Bump miette from 5.10.0 to 7.0.0 (#11788) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description Bump miette from 5.10.0 to 7.0.0 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>	2024-02-07 19:26:18 -06:00
Andrej Kolchin	fb7f6fc08b	Fix a panic when parsing empty file (#11314) ... The previous implementation presumed that if files were given, they had contents. The change makes the fallback to permanent files uniform. Fix #11256	2024-02-07 18:47:44 -06:00
Ian Manske	857c522808	Fix #11750: LazyRecord error message (#11772) ... # Description Makes `LazyRecord`s have the same error message as regular `Records` for `Value::follow_cell_path`. Fixes #11750.	2024-02-08 07:22:15 +08:00
Ian Manske	f8a8eca836	Record cleanup (#11726) ... # Description Does a little cleanup in `record.rs`: - Makes the `record!` macro more hygienic. - Converts regular comments to doc comments from #11718. - Converts the `Record` iterators to new types.	2024-02-08 06:43:12 +08:00
TrMen	4b91ed57dd	Enforce call stack depth limit for all calls (#11729) ... # Description Previously, only direcly-recursive calls were checked for recursion depth. But most recursive calls in nushell are mutually recursive since expressions like `for`, `where`, `try` and `do` all execute a separte block. ```nushell def f [] { do { f } } ``` Calling `f` would crash nushell with a stack overflow. I think the only general way to prevent such a stack overflow is to enforce a maximum call stack depth instead of only disallowing directly recursive calls. This commit also moves that logic into `eval_call()` instead of `eval_block()` because the recursion limit is tracked in the `Stack`, but not all blocks are evaluated in a new stack. Incrementing the recursion depth of the caller's stack would permanently increment that for all future calls. Fixes #11667 # User-Facing Changes Any function call can now fail with `recursion_limit_reached` instead of just directly recursive calls. Mutually-recursive calls no longer crash nushell. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-08 06:42:24 +08:00
Jakub Žádník	b8d37a7541	Fix panic in rotate; Add safe record creation function (#11718) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Fixes https://github.com/nushell/nushell/issues/11716 The problem is in our [record creation API](0d518bf813/crates/nu-protocol/src/value/record.rs (L33)) which panics if the numbers of columns and values are different. I added a safe variant that returns a `Result` and used it in the `rotate` command. ## TODO in another PR: Go through all `from_raw_cols_vals_unchecked()` (this includes the `record!` macro which uses the unchecked version) and make sure that either a) it is guaranteed the number of cols and vals is the same, or b) convert the call to `from_raw_cols_vals()` Reason: Nushell should never panic. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-02-03 13:23:16 +02:00
WindSoilder	c371d1a535	fix exit_code handling when running a scripts with ctrlc (#11466) ... # Description Fixes: #11394 When run `^sleep 3` we have an `exit_code ListStream`, and when we press ctrl-c, this `ListStream` will return None. But it's not expected, because `exit_code` sender in `run_external` always send an exit code out. This pr is trying to fix the issue by introducing a `first_guard` into ListStream, it will always generate a value from underlying stream if `first_guard` is true, so it's guarantee to have at least one value to return. And the pr also do a little refactor, which makes use of `ListStream::from_stream` rather than construct it manually. # User-Facing Changes ## Before ``` > nu -c "^sleep 3" # press ctrl-c > echo $env.LAST_EXIT_CODE 0 ``` ## After ``` > nu -c "^sleep 3" # press ctrl-c > echo $env.LAST_EXIT_CODE 255 ``` # Tests + Formatting None, sorry that I don't think it's easy to test the ctrlc behavior. # After Submitting None	2024-01-30 22:41:14 +08:00
WindSoilder	d646903161	Unify glob behavior on open, rm, cp-old, mv, umv, cp and du commands (#11621) ... # Description This pr is a follow up to [#11569](https://github.com/nushell/nushell/pull/11569#issuecomment-1902279587) > Revert the logic in https://github.com/nushell/nushell/pull/10694 and apply the logic in this pr to mv, cp, rv will require a larger change, I need to think how to achieve the bahavior And sorry @bobhy for reverting some of your changes. This pr is going to unify glob behavior on the given commands: * open * rm * cp-old * mv * umv * cp * du So they have the same behavior to `ls`, which is: If given parameter is quoted by single quote(`'`) or double quote(`"`), don't auto-expand the glob pattern. If not quoted, auto-expand the glob pattern. Fixes: #9558 Fixes: #10211 Fixes: #9310 Fixes: #10364 # TODO But there is one thing remains: if we give a variable to the command, it will always auto-expand the glob pattern, e.g: ```nushell let path = "a[123]b" rm $path ``` I don't think it's expected. But I also think user might want to auto-expand the glob pattern in variables. So I'll introduce a new command called `glob escape`, then if user doesn't want to auto-expand the glob pattern, he can just do this: `rm ($path | glob escape)` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting Done # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> ## NOTE This pr changes the semantic of `GlobPattern`, before this pr, it will `expand path` after evaluated, this makes `nu_engine::glob_from` have no chance to glob things right if a path contains glob pattern. e.g: [#9310 ](https://github.com/nushell/nushell/issues/9310#issuecomment-1886824030) #10211 I think changing the semantic is fine, because it makes glob works if path contains something like '*'. It maybe a breaking change if a custom command's argument are annotated by `: glob`.	2024-01-26 21:57:35 +08:00
Eric Hodel	2a65d43c13	Add into cell-path for dynamic cell-path creation (#11322) ... # Description The `cell-path` is a type that can be created statically with `$.nested.structure.5`, but can't be created from user input. This makes it difficult to take advantage of commands that accept a cell-path to operate on data structures. This PR adds `into cell-path` for dynamic cell-path creation. `into cell-path` accepts the following input shapes: * Bare integer (equivalent to `$.1`) * List of strings and integers * List of records with entries `value` and `optional` * String (parsed into a cell-path) ## Example usage An example of where `into cell-path` can be used is in working with `git config --list`. The git configuration has a tree structure that maps well to nushell records. With dynamic cell paths it is easy to convert `git config list` to a record: ```nushell git config --list | lines | parse -r '^(?<key>[^=]+)=(?<value>.*)' | reduce --fold {} {|entry, result| let path = $entry.key | into cell-path $result | upsert $path {|| $entry.value } } | select remote ``` Output: ``` ╭────────┬──────────────────────────────────────────────────────────────────╮ │ │ ╭──────────┬───────────────────────────────────────────────────╮ │ │ remote │ │ │ ╭───────┬───────────────────────────────────────╮ │ │ │ │ │ upstream │ │ url │ git@github.com:nushell/nushell.git │ │ │ │ │ │ │ │ fetch │ +refs/heads/*:refs/remotes/upstream/* │ │ │ │ │ │ │ ╰───────┴───────────────────────────────────────╯ │ │ │ │ │ │ ╭───────┬─────────────────────────────────────╮ │ │ │ │ │ origin │ │ url │ git@github.com:drbrain/nushell │ │ │ │ │ │ │ │ fetch │ +refs/heads/*:refs/remotes/origin/* │ │ │ │ │ │ │ ╰───────┴─────────────────────────────────────╯ │ │ │ │ ╰──────────┴───────────────────────────────────────────────────╯ │ ╰────────┴──────────────────────────────────────────────────────────────────╯ ``` ## Errors `lex()` + `parse_cell_path()` are forgiving about what is allowed in a cell-path so it will allow what appears to be nonsense to become a cell-path: ```nushell let table = [["!@$%^&*" value]; [key value]] $table | get ("!@$%^&*.0" | into cell-path) # => key ``` But it will reject bad cell-paths: ``` ❯ "a b" | into cell-path Error: nu:🐚:cant_convert × Can't convert to cell-path. ╭─[entry #14:1:1] 1 │ "a b" | into cell-path · ───────┬────── · ╰── can't convert string to cell-path ╰──── help: "a b" is not a valid cell-path (Parse mismatch during operation.) ``` # User-Facing Changes New conversion command `into cell-path` # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting Automatic documentation updates	2024-01-24 16:20:46 -06:00
Yash Thakur	90d65bb987	Evaluate string interpolation at parse time (#11562) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Closes #11561 # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR will allow string interpolation at parse time. Since the actual config hasn't been loaded at parse time, this uses the `get_config()` method on `StateWorkingSet`. So file sizes and datetimes (I think those are the only things whose string representations depend on the config) may be formatted differently from how users have configured things, which may come as a surprise to some. It does seem unlikely that anyone would be formatting file sizes or date times at parse time. Still, something to think about if/before this PR merged. Also, I changed the `ModuleNotFound` error to include the name of the module. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Users will be able to do stuff like: ```nu const x = [1 2 3] const y = $"foo($x)" // foo[1, 2, 3] ``` The main use case is `use`-ing and `source`-ing files at parse time: ```nu const file = "foo.nu" use $"($file)" ``` If the module isn't found, you'll see an error like this: ``` Error: nu::parser::module_not_found × Module not found. ╭─[entry #3:1:1] 1 │ use $"($file)" · ─────┬──── · ╰── module foo.nu not found ╰──── help: module files and their paths must be available before your script is run as parsing occurs before anything is evaluated ``` # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> Although there's user-facing changes, there's probably no need to change the docs since people probably already expect string interpolation to work at parse time. Edit: @kubouch pointed out that we'd need to document the fact that stuff like file sizes and datetimes won't get formatted according to user's runtime configs, so I'll make a PR to nushell.github.io after this one	2024-01-22 09:13:48 +02:00
Yash Thakur	188aca8fe6	Upgrade byte-unit from 4.0 to 5.1 (#11584) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR is for using version 5.1 of [byte_unit](https://docs.rs/byte-unit/latest/byte_unit/index.html) instead of 4.0. dependabot opened https://github.com/nushell/nushell/pull/11499 to do this but it's a major version increment so some minor changes were necessary. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> If something is on the boundary of a unit (e.g. 1024 bytes = 1 kibibytes), that will now be formatted as `1.0 KiB` where it used to be formatted as `1,024 B`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-01-21 14:17:28 -06:00
WindSoilder	c59d6d31bc	do not attempt to glob expand if the file path is wrapped in quotes (#11569) ... # Description Fixes: #11455 ### For arguments which is annotated with `:path/:directory/:glob` To fix the issue, we need to have a way to know if a path is originally quoted during runtime. So the information needed to be added at several levels: * parse time (from user input to expression) We need to add quoted information into `Expr::Filepath`, `Expr::Directory`, `Expr::GlobPattern` * eval time When convert from `Expr::Filepath`, `Expr::Directory`, `Expr::GlobPattern` to `Value::String` during runtime, we won't auto expanded the path if it's quoted ### For `ls` It's really special, because it accepts a `String` as a pattern, and it generates `glob` expression inside the command itself. So the idea behind the change is introducing a special SyntaxShape to ls: `SyntaxShape::LsGlobPattern`. So we can track if the pattern is originally quoted easier, and we don't auto expand the path either. Then when constructing a glob pattern inside ls, we check if input pattern is quoted, if so: we escape the input pattern, so we can run `ls a[123]b`, because it's already escaped. Finally, to accomplish the checking process, we also need to introduce a new value type called `Value::QuotedString` to differ from `Value::String`, it's used to generate an enum called `NuPath`, which is finally used in `ls` function. `ls` learned from `NuPath` to know if user input is quoted. # User-Facing Changes Actually it contains several changes ### For arguments which is annotated with `:path/:directory/:glob` #### Before ```nushell > def foo [p: path] { echo $p }; print (foo "~/a"); print (foo '~/a') /home/windsoilder/a /home/windsoilder/a > def foo [p: directory] { echo $p }; print (foo "~/a"); print (foo '~/a') /home/windsoilder/a /home/windsoilder/a > def foo [p: glob] { echo $p }; print (foo "~/a"); print (foo '~/a') /home/windsoilder/a /home/windsoilder/a ``` #### After ```nushell > def foo [p: path] { echo $p }; print (foo "~/a"); print (foo '~/a') ~/a ~/a > def foo [p: directory] { echo $p }; print (foo "~/a"); print (foo '~/a') ~/a ~/a > def foo [p: glob] { echo $p }; print (foo "~/a"); print (foo '~/a') ~/a ~/a ``` ### For ls command `touch '[uwu]'` #### Before ``` ❯ ls -D "[uwu]" Error: × No matches found for [uwu] ╭─[entry #6:1:1] 1 │ ls -D "[uwu]" · ───┬─── · ╰── Pattern, file or folder not found ╰──── help: no matches found ``` #### After ``` ❯ ls -D "[uwu]" ╭───┬───────┬──────┬──────┬──────────╮ │ # │ name │ type │ size │ modified │ ├───┼───────┼──────┼──────┼──────────┤ │ 0 │ [uwu] │ file │ 0 B │ now │ ╰───┴───────┴──────┴──────┴──────────╯ ``` # Tests + Formatting Done # After Submitting NaN	2024-01-21 23:22:25 +08:00
Ian Manske	55bf4d847f	Add CLI flag to disable history (#11550) ... # Description Adds a CLI flag for nushell that disables reading and writing to the history file. This will be useful for future testing and possibly our users as well. To borrow `fish` shell's terminology, this allows users to start nushell in "private" mode. # User-Facing Changes Breaking API change for `nu-protocol` (changed `Config`).	2024-01-17 09:40:59 -06:00
Eric Hodel	7071617f18	Allow plugins to receive configuration from the nushell configuration (#10955) ... # Description When nushell calls a plugin it now sends a configuration `Value` from the nushell config under `$env.config.plugins.PLUGIN_SHORT_NAME`. This allows plugin authors to read configuration provided by plugin users. The `PLUGIN_SHORT_NAME` must match the registered filename after `nu_plugin_`. If you register `target/debug/nu_plugin_config` the `PLUGIN_NAME` will be `config` and the nushell config will loook like: $env.config = { # ... plugins: { config: [ some values ] } } Configuration may also use a closure which allows passing values from `$env` to a plugin: $env.config = { # ... plugins: { config: {|| $env.some_value } } } This is a breaking change for the plugin API as the `Plugin::run()` function now accepts a new configuration argument which is an `&Option<Value>`. If no configuration was supplied the value is `None`. Plugins compiled after this change should work with older nushell, and will behave as if the configuration was not set. Initially discussed in #10867 # User-Facing Changes * Plugins can read configuration data stored in `$env.config.plugins` * The plugin `CallInfo` now includes a `config` entry, existing plugins will require updates # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting - [ ] Update [Creating a plugin (in Rust)](https://www.nushell.sh/contributor-book/plugins.html#creating-a-plugin-in-rust) [source](https://github.com/nushell/nushell.github.io/blob/main/contributor-book/plugins.md) - [ ] Add "Configuration" section to [Plugins documentation](https://www.nushell.sh/contributor-book/plugins.html)	2024-01-15 16:59:47 +08:00
WindSoilder	e72a4116ec	adjust some commansd input_output type (#11436) ... # Description 1. Make table to be a subtype of `list<any>`, so some input_output_types of filter commands are unnecessary 2. Change some commands which accept an input type, but generates different output types. In this case, delete duplicate entry, and change relative output type to `<any>` Yeah it makes some commands more permissive, but I think it's better to run into strange issue that why my script runs to failed during parse time. Fixes #11193 # User-Facing Changes NaN # Tests + Formatting NaN # After Submitting NaN	2024-01-15 16:58:26 +08:00
WindSoilder	724818030d	add type check during eval time (#11475) ... # Description Fixes: #11438 Take the following as example: ```nushell def spam [foo: string] { $'foo: ($foo | describe)' } def outer [--foo: string] { spam $foo } outer ``` When we call `outer`, type checker only check the all for `outer`, but doesn't check inside the body of `outer`. This pr is trying to introduce a type checking process through `Type::is_subtype()` during eval time. ## NOTE I'm not really sure if it's easy to make a check inside the body of `outer`. Adding an eval time type checker seems like an easier solution. As a result: `outer` will be caught by runtime, not parse time type checker cc @kubouch # User-Facing Changes After this pr the following call will failed: ```nushell > outer Error: nu:🐚:cant_convert × Can't convert to string. ╭─[entry #27:1:1] 1 │ def outer [--foo: any] { 2 │ spam $foo · ──┬─ · ╰── can't convert nothing to string 3 │ } ╰──── ``` # Tests + Formatting Done # After Submitting NaN	2024-01-12 23:48:53 +08:00
Artemiy	1867bb1a88	Fix incorrect handling of boolean flags for builtin commands (#11492) ... # Description Possible fix of #11456 This PR fixes a bug where builtin commands did not respect the logic of dynamically passed boolean flags. The reason is [has_flag](6f59abaf43/crates/nu-protocol/src/ast/call.rs (L204C5-L212C6)) method did not evaluate and take into consideration expression used with flag. To address this issue a solution is proposed: 1. `has_flag` method is moved to `CallExt` and new logic to evaluate expression and check if it is a boolean value is added 2. `has_flag_const` method is added to `CallExt` which is a constant version of `has_flag` 3. `has_named` method is added to `Call` which is basically the old logic of `has_flag` 4. All usages of `has_flag` in code are updated, mostly to pass `engine_state` and `stack` to new `has_flag`. In `run_const` commands it is replaced with `has_flag_const`. And in a few select places: parser, `to nuon` and `into string` old logic via `has_named` is used. # User-Facing Changes Explicit values of boolean flags are now respected in builtin commands. Before:  After:  Another example: Before:  After:  # Tests + Formatting Added test reproducing some variants of original issue.	2024-01-11 17:19:48 +02:00
Yash Thakur	21b3eeed99	Allow spreading arguments to commands (#11289) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Finishes implementing https://github.com/nushell/nushell/issues/10598, which asks for a spread operator in lists, in records, and when calling commands. # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR will allow spreading arguments to commands (both internal and external). It will also deprecate spreading arguments automatically when passing to external commands. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> - Users will be able to use `...` to spread arguments to custom/builtin commands that have rest parameters or allow unknown arguments, or to any external command - If a custom command doesn't have a rest parameter and it doesn't allow unknown arguments either, the spread operator will not be allowed - Passing lists to external commands without `...` will work for now but will cause a deprecation warning saying that it'll stop working in 0.91 (is 2 versions enough time?) Here's a function to help with demonstrating some behavior: ```nushell > def foo [ a, b, c?, d?, ...rest ] { [$a $b $c $d $rest] | to nuon } ``` You can pass a list of arguments to fill in the `rest` parameter using `...`: ```nushell > foo 1 2 3 4 ...[5 6] [1, 2, 3, 4, [5, 6]] ``` If you don't use `...`, the list `[5 6]` will be treated as a single argument: ```nushell > foo 1 2 3 4 [5 6] # Note the double [[]] [1, 2, 3, 4, [[5, 6]]] ``` You can omit optional parameters before the spread arguments: ```nushell > foo 1 2 3 ...[4 5] # d is omitted here [1, 2, 3, null, [4, 5]] ``` If you have multiple lists, you can spread them all: ```nushell > foo 1 2 3 ...[4 5] 6 7 ...[8] ...[] [1, 2, 3, null, [4, 5, 6, 7, 8]] ``` Here's the kind of error you get when you try to spread arguments to a command with no rest parameter:  And this is the warning you get when you pass a list to an external now (without `...`):  # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Added tests to cover the following cases: - Spreading arguments to a command that doesn't have a rest parameter (unexpected spread argument error) - Spreading arguments to a command that doesn't have a rest parameter *but* there's also a missing positional argument (missing positional error) - Spreading arguments to a command that doesn't have a rest parameter but does allow unknown arguments, such as `exec` (allowed) - Spreading a list literal containing arguments of the wrong type (parse error) - Spreading a non-list value, both to internal and external commands - Having named arguments in the middle of rest arguments - `explain`ing a command call that spreads its arguments # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> # Examples Suppose you have multiple tables: ```nushell let people = [[id name age]; [0 alice 100] [1 bob 200] [2 eve 300]] let evil_twins = [[id name age]; [0 ecila 100] [-1 bob 200] [-2 eve 300]] ``` Maybe you often find yourself needing to merge multiple tables and want a utility to do that. You could write a function like this: ```nushell def merge_all [ ...tables ] { $tables | reduce { |it, acc| $acc | merge $it } } ``` Then you can use it like this: ```nushell > merge_all ...([$people $evil_twins] | each { |$it| $it | select name age }) ╭───┬───────┬─────╮ │ # │ name │ age │ ├───┼───────┼─────┤ │ 0 │ ecila │ 100 │ │ 1 │ bob │ 200 │ │ 2 │ eve │ 300 │ ╰───┴───────┴─────╯ ``` Except they had duplicate columns, so now you first want to suffix every column with a number to tell you which table the column came from. You can make a command for that: ```nushell def select_and_merge [ --cols: list<string>, ...tables ] { let renamed_tables = $tables | enumerate | each { |it| $it.item | select $cols | rename ...($cols | each { |col| $col + ($it.index | into string) }) }; merge_all ...$renamed_tables } ``` And call it like this: ```nushell > select_and_merge --cols [name age] $people $evil_twins ╭───┬───────┬──────┬───────┬──────╮ │ # │ name0 │ age0 │ name1 │ age1 │ ├───┼───────┼──────┼───────┼──────┤ │ 0 │ alice │ 100 │ ecila │ 100 │ │ 1 │ bob │ 200 │ bob │ 200 │ │ 2 │ eve │ 300 │ eve │ 300 │ ╰───┴───────┴──────┴───────┴──────╯ ``` --- Suppose someone's made a command to search for APT packages: ```nushell # The main command def search-pkgs [ --install # Whether to install any packages it finds log_level: int # Pretend it's a good idea to make this a required positional parameter exclude?: list<string> # Packages to exclude repositories?: list<string> # Which repositories to look in (searches in all if not given) ...pkgs # Package names to search for ] { { install: $install, log_level: $log_level, exclude: ($exclude | to nuon), repositories: ($repositories | to nuon), pkgs: ($pkgs | to nuon) } } ``` It has a lot of parameters to configure it, so you might make your own helper commands to wrap around it for specific cases. Here's one example: ```nushell # Only look for packages locally def search-pkgs-local [ --install # Whether to install any packages it finds log_level: int exclude?: list<string> # Packages to exclude ...pkgs # Package names to search for ] { # All required and optional positional parameters are given search-pkgs --install=$install $log_level [] ["<local URI or something>"] ...$pkgs } ``` And you can run it like this: ```nushell > search-pkgs-local --install=false 5 ...["python2.7" "vim"] ╭──────────────┬──────────────────────────────╮ │ install │ false │ │ log_level │ 5 │ │ exclude │ [] │ │ repositories │ ["<local URI or something>"] │ │ pkgs │ ["python2.7", vim] │ ╰──────────────┴──────────────────────────────╯ ``` One thing I realized when writing this was that if we decide to not allow passing optional arguments using the spread operator, then you can (mis?)use the spread operator to skip optional parameters. Here, I didn't want to give `exclude` explicitly, so I used a spread operator to pass the packages to install. Without it, I would've needed to do `search-pkgs-local --install=false 5 [] "python2.7" "vim"` (explicitly pass `[]` (or `null`, in the general case) to `exclude`). There are probably more idiomatic ways to do this, but I just thought it was something interesting. If you're a virologist of the [xkcd](https://xkcd.com/350/) kind, another helper command you might make is this: ```nushell # Install any packages it finds def live-dangerously [ ...pkgs ] { # One optional argument was given (exclude), while another was not (repositories) search-pkgs 0 [] ...$pkgs --install # Flags can go after spread arguments } ``` Running it: ```nushell > live-dangerously "git" "*vi*" # *vi* because I don't feel like typing out vim and neovim ╭──────────────┬─────────────╮ │ install │ true │ │ log_level │ 0 │ │ exclude │ [] │ │ repositories │ null │ │ pkgs │ [git, *vi*] │ ╰──────────────┴─────────────╯ ``` Here's an example that uses the spread operator more than once within the same command call: ```nushell let extras = [ chrome firefox python java git ] def search-pkgs-curated [ ...pkgs ] { (search-pkgs 1 [emacs] ["example.com", "foo.com"] vim # A must for everyone! ...($pkgs | filter { |p| not ($p | str contains "*") }) # Remove packages with globs python # Good tool to have ...$extras --install=false python3) # I forget, did I already put Python in extras? } ``` Running it: ```nushell > search-pkgs-curated "git" "*vi*" ╭──────────────┬───────────────────────────────────────────────────────────────────╮ │ install │ false │ │ log_level │ 1 │ │ exclude │ [emacs] │ │ repositories │ [example.com, foo.com] │ │ pkgs │ [vim, git, python, chrome, firefox, python, java, git, "python3"] │ ╰──────────────┴───────────────────────────────────────────────────────────────────╯ ```	2023-12-28 15:43:20 +08:00
Ian Manske	ba880277bf	Remove unnecessary replace_in_variable (#11424) ... # Description `Expression::replace_in_variable` is only called in one place, and it is called with `new_var_id` = `IN_VARIABLE_ID`. So, it ends up doing nothing. E.g., adding `debug_assert_eq!(new_var_id, IN_VARIABLE_ID)` in `replace_in_variable` does not trigger any panic. # User-Facing Changes Breaking change for `nu_protocol`.	2023-12-26 18:46:49 +01:00