21 Commits

Author	SHA1	Message	Date
Stefan Holderbach	95b78eee25	Change the usage misnomer to "description" (#13598) ... # Description The meaning of the word usage is specific to describing how a command function is *used* and not a synonym for general description. Usage can be used to describe the SYNOPSIS or EXAMPLES sections of a man page where the permitted argument combinations are shown or example *uses* are given. Let's not confuse people and call it what it is a description. Our `help` command already creates its own *Usage* section based on the available arguments and doesn't refer to the description with usage. # User-Facing Changes `help commands` and `scope commands` will now use `description` or `extra_description` `usage`-> `description` `extra_usage` -> `extra_description` Breaking change in the plugin protocol: In the signature record communicated with the engine. `usage`-> `description` `extra_usage` -> `extra_description` The same rename also takes place for the methods on `SimplePluginCommand` and `PluginCommand` # Tests + Formatting - Updated plugin protocol specific changes # After Submitting - [ ] update plugin protocol doc	2024-08-22 12:02:08 +02:00
Devyn Cairns	d7392f1f3b	Internal representation (IR) compiler and evaluator (#13330) ... # Description This PR adds an internal representation language to Nushell, offering an alternative evaluator based on simple instructions, stream-containing registers, and indexed control flow. The number of registers required is determined statically at compile-time, and the fixed size required is allocated upon entering the block. Each instruction is associated with a span, which makes going backwards from IR instructions to source code very easy. Motivations for IR: 1. **Performance.** By simplifying the evaluation path and making it more cache-friendly and branch predictor-friendly, code that does a lot of computation in Nushell itself can be sped up a decent bit. Because the IR is fairly easy to reason about, we can also implement optimization passes in the future to eliminate and simplify code. 2. **Correctness.** The instructions mostly have very simple and easily-specified behavior, so hopefully engine changes are a little bit easier to reason about, and they can be specified in a more formal way at some point. I have made an effort to document each of the instructions in the docs for the enum itself in a reasonably specific way. Some of the errors that would have happened during evaluation before are now moved to the compilation step instead, because they don't make sense to check during evaluation. 3. **As an intermediate target.** This is a good step for us to bring the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at some point, as code generated from new AST can be directly compared to code generated from old AST. If the IR code is functionally equivalent, it will behave the exact same way. 4. **Debugging.** With a little bit more work, we can probably give control over advancing the virtual machine that `IrBlock`s run on to some sort of external driver, making things like breakpoints and single stepping possible. Tools like `view ir` and [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than before to see what exactly is going on with your Nushell code. The goal is to eventually replace the AST evaluator entirely, once we're sure it's working just as well. You can help dogfood this by running Nushell with `$env.NU_USE_IR` set to some value. The environment variable is checked when Nushell starts, so config runs with IR, or it can also be set on a line at the REPL to change it dynamically. It is also checked when running `do` in case within a script you want to just run a specific piece of code with or without IR. # Example ```nushell view ir { |data| mut sum = 0 for n in $data { $sum += $n } $sum } ``` ```gas # 3 registers, 19 instructions, 0 bytes of data 0: load-literal %0, int(0) 1: store-variable var 904, %0 # let 2: drain %0 3: drop %0 4: load-variable %1, var 903 5: iterate %0, %1, end 15 # for, label(1), from(14:) 6: store-variable var 905, %0 7: load-variable %0, var 904 8: load-variable %2, var 905 9: binary-op %0, Math(Plus), %2 10: span %0 11: store-variable var 904, %0 12: load-literal %0, nothing 13: drain %0 14: jump 5 15: drop %0 # label(0), from(5:) 16: drain %0 17: load-variable %0, var 904 18: return %0 ``` # Benchmarks All benchmarks run on a base model Mac Mini M1. ## Iterative Fibonacci sequence This is about as best case as possible, making use of the much faster control flow. Most code will not experience a speed improvement nearly this large. ```nushell def fib [n: int] { mut a = 0 mut b = 1 for _ in 2..=$n { let c = $a + $b $a = $b $b = $c } $b } use std bench bench { 0..50 | each { |n| fib $n } } ``` IR disabled: ``` ╭───────┬─────────────────╮ │ mean │ 1ms 924µs 665ns │ │ min │ 1ms 700µs 83ns │ │ max │ 3ms 450µs 125ns │ │ std │ 395µs 759ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ``` IR enabled: ``` ╭───────┬─────────────────╮ │ mean │ 452µs 820ns │ │ min │ 427µs 417ns │ │ max │ 540µs 167ns │ │ std │ 17µs 158ns │ │ times │ [list 50 items] │ ╰───────┴─────────────────╯ ```  ## [gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu) IR disabled: ``` ╭───┬──────────────────╮ │ 0 │ 27ms 929µs 958ns │ │ 1 │ 21ms 153µs 459ns │ │ 2 │ 18ms 639µs 666ns │ │ 3 │ 19ms 554µs 583ns │ │ 4 │ 13ms 383µs 375ns │ │ 5 │ 11ms 328µs 208ns │ │ 6 │ 5ms 659µs 542ns │ ╰───┴──────────────────╯ ``` IR enabled: ``` ╭───┬──────────────────╮ │ 0 │ 22ms 662µs │ │ 1 │ 17ms 221µs 792ns │ │ 2 │ 14ms 786µs 708ns │ │ 3 │ 13ms 876µs 834ns │ │ 4 │ 13ms 52µs 875ns │ │ 5 │ 11ms 269µs 666ns │ │ 6 │ 6ms 942µs 500ns │ ╰───┴──────────────────╯ ``` ## [random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu) I got pretty random results out of this benchmark so I decided not to include it. Not clear why. # User-Facing Changes - IR compilation errors may appear even if the user isn't evaluating with IR. - IR evaluation can be enabled by setting the `NU_USE_IR` environment variable to any value. - New command `view ir` pretty-prints the IR for a block, and `view ir --json` can be piped into an external tool like [`explore ir`](https://github.com/devyn/nu_plugin_explore_ir). # Tests + Formatting All tests are passing with `NU_USE_IR=1`, and I've added some more eval tests to compare the results for some very core operations. I will probably want to add some more so we don't have to always check `NU_USE_IR=1 toolkit test --workspace` on a regular basis. # After Submitting - [ ] release notes - [ ] further documentation of instructions? - [ ] post-release: publish `nu_plugin_explore_ir`	2024-07-10 17:33:59 -07:00
Jakub Žádník	3fae77209a	Revert "Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268)" (#13292) ... This reverts commit 0cfd5fbece6f25b54ab9dc417a9e06af9d83f282. The original PR messed up syntax higlighting of aliases and causes panics of completion in the presence of alias. <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-07-04 00:02:13 +03:00
Jakub Žádník	0cfd5fbece	Span ID Refactor (Step 2): Make Call SpanId-friendly (#13268) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Part of https://github.com/nushell/nushell/issues/12963, step 2. This PR refactors Call and related argument structures to remove their dependency on `Expression::span` which will be removed in the future. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Should be none. If you see some error messages that look broken, please report. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-07-03 09:00:52 +03:00
Devyn Cairns	2970d48d41	Make bytes build accept integer values as individual bytes (#12685) ... # Description This creates an option for building binary data from byte integers. Previously I think you could only do this by formatting the integers to hex and using `decode hex`. One potentially confusing thing is that this is different from the `into binary` behavior. But since this doesn't support any of the other `into binary` behaviors, it might be okay. # User-Facing Changes - `bytes build` accepts single byte arguments as integers # Tests + Formatting Example added. # After Submitting - [ ] release notes	2024-05-01 17:29:33 -05:00
Ian Manske	c747ec75c9	Add command_prelude module (#12291) ... # Description When implementing a `Command`, one must also import all the types present in the function signatures for `Command`. This makes it so that we often import the same set of types in each command implementation file. E.g., something like this: ```rust use nu_protocol::ast::Call; use nu_protocol::engine::{Command, EngineState, Stack}; use nu_protocol::{ record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData, ShellError, Signature, Span, Type, Value, }; ``` This PR adds the `nu_engine::command_prelude` module which contains the necessary and commonly used types to implement a `Command`: ```rust // command_prelude.rs pub use crate::CallExt; pub use nu_protocol::{ ast::{Call, CellPath}, engine::{Command, EngineState, Stack}, record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned, PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value, }; ``` This should reduce the boilerplate needed to implement a command and also gives us a place to track the breadth of the `Command` API. I tried to be conservative with what went into the prelude modules, since it might be hard/annoying to remove items from the prelude in the future. Let me know if something should be included or excluded.	2024-03-26 21:17:30 +00:00
Jakub Žádník	14d1c67863	Debugger experiments (#11441) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a new evaluator path with callbacks to a mutable trait object implementing a Debugger trait. The trait object can do anything, e.g., profiling, code coverage, step debugging. Currently, entering/leaving a block and a pipeline element is marked with callbacks, but more callbacks can be added as necessary. Not all callbacks need to be used by all debuggers; unused ones are simply empty calls. A simple profiler is implemented as a proof of concept. The debugging support is implementing by making `eval_xxx()` functions generic depending on whether we're debugging or not. This has zero computational overhead, but makes the binary slightly larger (see benchmarks below). `eval_xxx()` variants called from commands (like `eval_block_with_early_return()` in `each`) are chosen with a dynamic dispatch for two reasons: to not grow the binary size due to duplicating the code of many commands, and for the fact that it isn't possible because it would make Command trait objects object-unsafe. In the future, I hope it will be possible to allow plugin callbacks such that users would be able to implement their profiler plugins instead of having to recompile Nushell. [DAP](https://microsoft.github.io/debug-adapter-protocol/) would also be interesting to explore. Try `help debug profile`. ## Screenshots Basic output:  To profile with more granularity, increase the profiler depth (you'll see that repeated `is-windows` calls take a large chunk of total time, making it a good candidate for optimizing):  ## Benchmarks ### Binary size Binary size increase vs. main: **+40360 bytes**. _(Both built with `--release --features=extra,dataframe`.)_ ### Time ```nushell # bench_debug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'debug:' let res2 = bench { debug profile $test } --pretty print $res2 ``` ```nushell # bench_nodebug.nu use std bench let test = { 1..100 | each { ls | each {|row| $row.name | str length } } | flatten | math avg } print 'no debug:' let res1 = bench { do $test } --pretty print $res1 ``` `cargo run --release -- bench_debug.nu` is consistently 1--2 ms slower than `cargo run --release -- bench_nodebug.nu` due to the collection overhead + gathering the report. This is expected. When gathering more stuff, the overhead is obviously higher. `cargo run --release -- bench_nodebug.nu` vs. `nu bench_nodebug.nu` I didn't measure any difference. Both benchmarks report times between 97 and 103 ms randomly, without one being consistently higher than the other. This suggests that at least in this particular case, when not running any debugger, there is no runtime overhead. ## API changes This PR adds a generic parameter to all `eval_xxx` functions that forces you to specify whether you use the debugger. You can resolve it in two ways: * Use a provided helper that will figure it out for you. If you wanted to use `eval_block(&engine_state, ...)`, call `let eval_block = get_eval_block(&engine_state); eval_block(&engine_state, ...)` * If you know you're in an evaluation path that doesn't need debugger support, call `eval_block::<WithoutDebug>(&engine_state, ...)` (this is the case of hooks, for example). I tried to add more explanation in the docstring of `debugger_trait.rs`. ## TODO - [x] Better profiler output to reduce spam of iterative commands like `each` - [x] Resolve `TODO: DEBUG` comments - [x] Resolve unwraps - [x] Add doc comments - [x] Add usage and extra usage for `debug profile`, explaining all columns # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Hopefully none. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2024-03-08 20:21:35 +02:00
Artemiy	1867bb1a88	Fix incorrect handling of boolean flags for builtin commands (#11492) ... # Description Possible fix of #11456 This PR fixes a bug where builtin commands did not respect the logic of dynamically passed boolean flags. The reason is [has_flag](6f59abaf43/crates/nu-protocol/src/ast/call.rs (L204C5-L212C6)) method did not evaluate and take into consideration expression used with flag. To address this issue a solution is proposed: 1. `has_flag` method is moved to `CallExt` and new logic to evaluate expression and check if it is a boolean value is added 2. `has_flag_const` method is added to `CallExt` which is a constant version of `has_flag` 3. `has_named` method is added to `Call` which is basically the old logic of `has_flag` 4. All usages of `has_flag` in code are updated, mostly to pass `engine_state` and `stack` to new `has_flag`. In `run_const` commands it is replaced with `has_flag_const`. And in a few select places: parser, `to nuon` and `into string` old logic via `has_named` is used. # User-Facing Changes Explicit values of boolean flags are now respected in builtin commands. Before:  After:  Another example: Before:  After:  # Tests + Formatting Added test reproducing some variants of original issue.	2024-01-11 17:19:48 +02:00
Yash Thakur	21b3eeed99	Allow spreading arguments to commands (#11289) ... <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> Finishes implementing https://github.com/nushell/nushell/issues/10598, which asks for a spread operator in lists, in records, and when calling commands. # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR will allow spreading arguments to commands (both internal and external). It will also deprecate spreading arguments automatically when passing to external commands. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> - Users will be able to use `...` to spread arguments to custom/builtin commands that have rest parameters or allow unknown arguments, or to any external command - If a custom command doesn't have a rest parameter and it doesn't allow unknown arguments either, the spread operator will not be allowed - Passing lists to external commands without `...` will work for now but will cause a deprecation warning saying that it'll stop working in 0.91 (is 2 versions enough time?) Here's a function to help with demonstrating some behavior: ```nushell > def foo [ a, b, c?, d?, ...rest ] { [$a $b $c $d $rest] | to nuon } ``` You can pass a list of arguments to fill in the `rest` parameter using `...`: ```nushell > foo 1 2 3 4 ...[5 6] [1, 2, 3, 4, [5, 6]] ``` If you don't use `...`, the list `[5 6]` will be treated as a single argument: ```nushell > foo 1 2 3 4 [5 6] # Note the double [[]] [1, 2, 3, 4, [[5, 6]]] ``` You can omit optional parameters before the spread arguments: ```nushell > foo 1 2 3 ...[4 5] # d is omitted here [1, 2, 3, null, [4, 5]] ``` If you have multiple lists, you can spread them all: ```nushell > foo 1 2 3 ...[4 5] 6 7 ...[8] ...[] [1, 2, 3, null, [4, 5, 6, 7, 8]] ``` Here's the kind of error you get when you try to spread arguments to a command with no rest parameter:  And this is the warning you get when you pass a list to an external now (without `...`):  # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Added tests to cover the following cases: - Spreading arguments to a command that doesn't have a rest parameter (unexpected spread argument error) - Spreading arguments to a command that doesn't have a rest parameter *but* there's also a missing positional argument (missing positional error) - Spreading arguments to a command that doesn't have a rest parameter but does allow unknown arguments, such as `exec` (allowed) - Spreading a list literal containing arguments of the wrong type (parse error) - Spreading a non-list value, both to internal and external commands - Having named arguments in the middle of rest arguments - `explain`ing a command call that spreads its arguments # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> # Examples Suppose you have multiple tables: ```nushell let people = [[id name age]; [0 alice 100] [1 bob 200] [2 eve 300]] let evil_twins = [[id name age]; [0 ecila 100] [-1 bob 200] [-2 eve 300]] ``` Maybe you often find yourself needing to merge multiple tables and want a utility to do that. You could write a function like this: ```nushell def merge_all [ ...tables ] { $tables | reduce { |it, acc| $acc | merge $it } } ``` Then you can use it like this: ```nushell > merge_all ...([$people $evil_twins] | each { |$it| $it | select name age }) ╭───┬───────┬─────╮ │ # │ name │ age │ ├───┼───────┼─────┤ │ 0 │ ecila │ 100 │ │ 1 │ bob │ 200 │ │ 2 │ eve │ 300 │ ╰───┴───────┴─────╯ ``` Except they had duplicate columns, so now you first want to suffix every column with a number to tell you which table the column came from. You can make a command for that: ```nushell def select_and_merge [ --cols: list<string>, ...tables ] { let renamed_tables = $tables | enumerate | each { |it| $it.item | select $cols | rename ...($cols | each { |col| $col + ($it.index | into string) }) }; merge_all ...$renamed_tables } ``` And call it like this: ```nushell > select_and_merge --cols [name age] $people $evil_twins ╭───┬───────┬──────┬───────┬──────╮ │ # │ name0 │ age0 │ name1 │ age1 │ ├───┼───────┼──────┼───────┼──────┤ │ 0 │ alice │ 100 │ ecila │ 100 │ │ 1 │ bob │ 200 │ bob │ 200 │ │ 2 │ eve │ 300 │ eve │ 300 │ ╰───┴───────┴──────┴───────┴──────╯ ``` --- Suppose someone's made a command to search for APT packages: ```nushell # The main command def search-pkgs [ --install # Whether to install any packages it finds log_level: int # Pretend it's a good idea to make this a required positional parameter exclude?: list<string> # Packages to exclude repositories?: list<string> # Which repositories to look in (searches in all if not given) ...pkgs # Package names to search for ] { { install: $install, log_level: $log_level, exclude: ($exclude | to nuon), repositories: ($repositories | to nuon), pkgs: ($pkgs | to nuon) } } ``` It has a lot of parameters to configure it, so you might make your own helper commands to wrap around it for specific cases. Here's one example: ```nushell # Only look for packages locally def search-pkgs-local [ --install # Whether to install any packages it finds log_level: int exclude?: list<string> # Packages to exclude ...pkgs # Package names to search for ] { # All required and optional positional parameters are given search-pkgs --install=$install $log_level [] ["<local URI or something>"] ...$pkgs } ``` And you can run it like this: ```nushell > search-pkgs-local --install=false 5 ...["python2.7" "vim"] ╭──────────────┬──────────────────────────────╮ │ install │ false │ │ log_level │ 5 │ │ exclude │ [] │ │ repositories │ ["<local URI or something>"] │ │ pkgs │ ["python2.7", vim] │ ╰──────────────┴──────────────────────────────╯ ``` One thing I realized when writing this was that if we decide to not allow passing optional arguments using the spread operator, then you can (mis?)use the spread operator to skip optional parameters. Here, I didn't want to give `exclude` explicitly, so I used a spread operator to pass the packages to install. Without it, I would've needed to do `search-pkgs-local --install=false 5 [] "python2.7" "vim"` (explicitly pass `[]` (or `null`, in the general case) to `exclude`). There are probably more idiomatic ways to do this, but I just thought it was something interesting. If you're a virologist of the [xkcd](https://xkcd.com/350/) kind, another helper command you might make is this: ```nushell # Install any packages it finds def live-dangerously [ ...pkgs ] { # One optional argument was given (exclude), while another was not (repositories) search-pkgs 0 [] ...$pkgs --install # Flags can go after spread arguments } ``` Running it: ```nushell > live-dangerously "git" "*vi*" # *vi* because I don't feel like typing out vim and neovim ╭──────────────┬─────────────╮ │ install │ true │ │ log_level │ 0 │ │ exclude │ [] │ │ repositories │ null │ │ pkgs │ [git, *vi*] │ ╰──────────────┴─────────────╯ ``` Here's an example that uses the spread operator more than once within the same command call: ```nushell let extras = [ chrome firefox python java git ] def search-pkgs-curated [ ...pkgs ] { (search-pkgs 1 [emacs] ["example.com", "foo.com"] vim # A must for everyone! ...($pkgs | filter { |p| not ($p | str contains "*") }) # Remove packages with globs python # Good tool to have ...$extras --install=false python3) # I forget, did I already put Python in extras? } ``` Running it: ```nushell > search-pkgs-curated "git" "*vi*" ╭──────────────┬───────────────────────────────────────────────────────────────────╮ │ install │ false │ │ log_level │ 1 │ │ exclude │ [emacs] │ │ repositories │ [example.com, foo.com] │ │ pkgs │ [vim, git, python, chrome, firefox, python, java, git, "python3"] │ ╰──────────────┴───────────────────────────────────────────────────────────────────╯ ```	2023-12-28 15:43:20 +08:00
Eric Hodel	5b01685fc3	Enforce required, optional, and rest positional arguments start with an uppercase and end with a period. (#11285) ... # Description This updates all the positional arguments (except with `--features=dataframe` or `--features=extra`) to start with an uppercase letter and end with a period. Part of #5066, specifically [this comment](/nushell/nushell/issues/5066#issuecomment-1421528910) Some arguments had example data removed from them because it also appears in the examples. There are other inconsistencies in positional arguments I noticed while making the tests pass which I will bring up in #5066. # User-Facing Changes Positional arguments are now consistent # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting Automatic documentation updates	2023-12-15 14:32:37 +08:00
JT	6cdfee3573	Move Value to helpers, separate span call (#10121) ... # Description As part of the refactor to split spans off of Value, this moves to using helper functions to create values, and using `.span()` instead of matching span out of Value directly. Hoping to get a few more helping hands to finish this, as there are a lot of commands to update :) # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com> Co-authored-by: WindSoilder <windsoilder@outlook.com>	2023-09-03 07:27:29 -07:00
JT	1e3e034021	Spanned Value step 1: span all value cases (#10042) ... # Description This doesn't really do much that the user could see, but it helps get us ready to do the steps of the refactor to split the span off of Value, so that values can be spanless. This allows us to have top-level values that can hold both a Value and a Span, without requiring that all values have them. We expect to see significant memory reduction by removing so many unnecessary spans from values. For example, a table of 100,000 rows and 5 columns would have a savings of ~8megs in just spans that are almost always duplicated. # User-Facing Changes Nothing yet # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->	2023-08-25 08:48:05 +12:00
JT	2ae1de2470	move 'bytes' back to commands (#10051) ... # Description Moves the `bytes XXXX` commands back to the default set.	2023-08-19 22:43:53 +02:00
Michael Angerman	d1449c4ee7	cratification: move the bytes command to nu-cmd-extra (#9509) ... now that #9455 has landed we can move the bytes command to nu-cmd-extra in concert with moving nu_command::util to nu-cmd-base	2023-06-23 12:23:08 -07:00
Stefan Holderbach	a52386e837	Box ShellError in Value::Error (#8375) ... # Description Our `ShellError` at the moment has a `std::mem::size_of<ShellError>` of 136 bytes (on AMD64). As a result `Value` directly storing the struct also required 136 bytes (thanks to alignment requirements). This change stores the `Value::Error` `ShellError` on the heap. Pro: - Value now needs just 80 bytes - Should be 1 cacheline less (still at least 2 cachelines) Con: - More small heap allocations when dealing with `Value::Error` - More heap fragmentation - Potential for additional required memcopies # Further code changes Includes a small refactor of `try` due to a type mismatch in its large match. # User-Facing Changes None for regular users. Plugin authors may have to update their matches on `Value` if they use `nu-protocol` Needs benchmarking to see if there is a benefit in real world workloads. **Update** small improvements in runtime for workloads with high volume of values. Significant reduction in maximum resident set size, when many values are held in memory. # Tests + Formatting	2023-03-12 09:57:27 +01:00
Stefan Holderbach	f7b8f97873	Document and critically review ShellError variants - Ep. 2 (#8326) ... Continuation of #8229 # Description The `ShellError` enum at the moment is kind of messy. Many variants are basic tuple structs where you always have to reference the implementation with its macro invocation to know which field serves which purpose. Furthermore we have both variants that are kind of redundant or either overly broad to be useful for the user to match on or overly specific with few uses. So I set out to start fixing the lacking documentation and naming to make it feasible to critically review the individual usages and fix those. Furthermore we can decide to join or split up variants that don't seem to be fit for purpose. **Everyone:** Feel free to add review comments if you spot inconsistent use of `ShellError` variants. - Name fields of `SE::IncorrectValue` - Merge and name fields on `SE::TypeMismatch` - Name fields on `SE::UnsupportedOperator` - Name fields on `AssignmentRequires*` and fix doc - Name fields on `SE::UnknownOperator` - Name fields on `SE::MissingParameter` - Name fields on `SE::DelimiterError` - Name fields on `SE::IncompatibleParametersSingle` # User-Facing Changes (None now, end goal more explicit and consistent error messages) # Tests + Formatting (No additional tests needed so far)	2023-03-06 11:31:07 +01:00
Jérémy Audiger	99076af18b	Use imported names in Command::run signatures (#7967) ... # Description _(Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes.)_ I opened this PR to unify the run command method. It's mainly to improve consistency across the tree. # User-Facing Changes None. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.	2023-02-05 22:17:46 +01:00
Kian-Meng Ang	79000aa5e0	Fix typos by codespell (#7600) ... # Description Found via `codespell -S target -L crate,ser,numer,falsy,ro,te,nd,bu,ndoes,statics,ons,fo,rouge,pard` # User-Facing Changes None. # Tests + Formatting None and done. # After Submitting None.	2022-12-26 02:31:26 -05:00
Leon	dd7b7311b3	Standardise the use of ShellError::UnsupportedInput and ShellError::TypeMismatch and add spans to every instance of the former (#7217) ... # Description * I was dismayed to discover recently that UnsupportedInput and TypeMismatch are used *extremely* inconsistently across the codebase. UnsupportedInput is sometimes used for input type-checks (as per the name!!), but *also* used for argument type-checks. TypeMismatch is also used for both. I thus devised the following standard: input type-checking *only* uses UnsupportedInput, and argument type-checking *only* uses TypeMismatch. Moreover, to differentiate them, UnsupportedInput now has *two* error arrows (spans), one pointing at the command and the other at the input origin, while TypeMismatch only has the one (because the command should always be nearby) * In order to apply that standard, a very large number of UnsupportedInput uses were changed so that the input's span could be retrieved and delivered to it. * Additionally, I noticed many places where **errors are not propagated correctly**: there are lots of `match` sites which take a Value::Error, then throw it away and replace it with a new Value::Error with less/misleading information (such as reporting the error as an "incorrect type"). I believe that the earliest errors are the most important, and should always be propagated where possible. * Also, to standardise one broad subset of UnsupportedInput error messages, who all used slightly different wordings of "expected `<type>`, got `<type>`", I created OnlySupportsThisInputType as a variant of it. * Finally, a bunch of error sites that had "repeated spans" - i.e. where an error expected two spans, but `call.head` was given for both - were fixed to use different spans. # Example BEFORE ``` 〉20b | str starts-with 'a' Error: nu:🐚:unsupported_input (link) × Unsupported input ╭─[entry #31:1:1] 1 │ 20b | str starts-with 'a' · ┬ · ╰── Input's type is filesize. This command only works with strings. ╰──── 〉'a' | math cos Error: nu:🐚:unsupported_input (link) × Unsupported input ╭─[entry #33:1:1] 1 │ 'a' | math cos · ─┬─ · ╰── Only numerical values are supported, input type: String ╰──── 〉0x[12] | encode utf8 Error: nu:🐚:unsupported_input (link) × Unsupported input ╭─[entry #38:1:1] 1 │ 0x[12] | encode utf8 · ───┬── · ╰── non-string input ╰──── ``` AFTER ``` 〉20b | str starts-with 'a' Error: nu:🐚:pipeline_mismatch (link) × Pipeline mismatch. ╭─[entry #1:1:1] 1 │ 20b | str starts-with 'a' · ┬ ───────┬─────── · │ ╰── only string input data is supported · ╰── input type: filesize ╰──── 〉'a' | math cos Error: nu:🐚:pipeline_mismatch (link) × Pipeline mismatch. ╭─[entry #2:1:1] 1 │ 'a' | math cos · ─┬─ ────┬─── · │ ╰── only numeric input data is supported · ╰── input type: string ╰──── 〉0x[12] | encode utf8 Error: nu:🐚:pipeline_mismatch (link) × Pipeline mismatch. ╭─[entry #3:1:1] 1 │ 0x[12] | encode utf8 · ───┬── ───┬── · │ ╰── only string input data is supported · ╰── input type: binary ╰──── ``` # User-Facing Changes Various error messages suddenly make more sense (i.e. have two arrows instead of one). # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.	2022-12-23 01:48:53 -05:00
Dan Davison	df94052180	Declare input and output types of commands (#6796) ... * Add failing test that list of ints and floats is List<Number> * Start defining subtype relation * Make it possible to declare input and output types for commands - Enforce them in tests * Declare input and output types of commands * Add formatted signatures to `help commands` table * Revert SyntaxShape::Table -> Type::Table change * Revert unnecessary derive(Hash) on SyntaxShape Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>	2022-11-10 10:55:05 +13:00
WindSoilder	9e3c64aa84	Add bytes collect, bytes remove, bytes build cmd (#6008) ... * add bytes collect * index_of support searching from end * add bytes remove * make bytes replace work better for empty pattern * add bytes build * remove comment * tweak words Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>	2022-07-11 06:26:00 -05:00