# Description
As a bit of a follow-on to #13802 and #14249, this (pretty much a
"one-line" change) really does *always* populate the `$env.config`
record with the `nu-protocol::config` defaults during startup. This
means that an `$env.config` record is value (with defaults) even during:
* `nu -n` to suppress loading of config files
* `nu -c <commandstring>`
* `nu <script>`
# User-Facing Changes
There should be no case in which there isn't a valid `$env.config`.
* Before:
```nushell
nu -c "$env.config"
# -> Error
```
* After:
```nushell
nu -c "$env.config"
# -> Default $env.config record
```
Startup time impact is negligible (17.072µs from `perf!` on my system) -
Seems well worth it.
# Tests + Formatting
Added tests for several `-n -c` cases.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
Config chapter update still in progress.
# Description
The `.nu-env` file feature was removed some time ago (probably in the
engine-q upgrade?). The tests, however, still remained as dead-code, so
this is just some basic clean-up.
If this feature was ever implemented again, the tests would need to be
rewritten anyway due to the changes in the way config is handled.
# User-Facing Changes
None
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
-
# After Submitting
N/A
# Description
The "append" operator currently serves as both the append operator and
the concatenation operator. This dual role creates ambiguity when
operating on nested lists.
```nu
[1 2] ++ 3 # appends a value to a list [1 2 3]
[1 2] ++ [3 4] # concatenates two lists [1 2 3 4]
[[1 2] [3 4]] ++ [5 6]
# does this give [[1 2] [3 4] [5 6]]
# or [[1 2] [3 4] 5 6]
```
Another problem is that `++=` can change the type of a variable:
```nu
mut str = 'hello '
$str ++= ['world']
($str | describe) == list<string>
```
Note that appending is only relevant for lists, but concatenation is
relevant for lists, strings, and binary values. Additionally, appending
can be expressed in terms of concatenation (see example below). So, this
PR changes the `++` operator to only perform concatenation.
# User-Facing Changes
Using the `++` operator with a list and a non-list value will now be a
compile time or runtime error.
```nu
mut list = []
$list ++= 1 # error
```
Instead, concatenate a list with one element:
```nu
$list ++= [1]
```
Or use `append`:
```nu
$list = $list | append 1
```
# After Submitting
Update book and docs.
---------
Co-authored-by: Douglas <32344964+NotTheDr01ds@users.noreply.github.com>
# User-Facing Changes
- `expected <type>` errors are now propagated from
`Closure | Block | Expression` instead of falling back to
"expected one of..." for the block:
Before:
```nushell
def foo [bar: bool] {}
if true {} else { foo 1 }
────┬────
╰── expected one of a list of accepted shapes: [Block, Expression]
```
After:
```nushell
if true {} else { foo 1 }
┬
╰── expected bool
```
This commit upgrades calamine in order to benefit from recent
developments, e.g. ignore annotations in column headers (see
https://github.com/tafia/calamine/pull/467 for reference).
# Release-Notes Short Description
* Nushell now always loads its internal `default_env.nu` before the user
`env.nu` is loaded, then loads the internal `default_config.nu` before
the user's `config.nu` is loaded. This allows for a simpler
user-configuration experience. The Configuration Chapter of the Book
will be updated soon with the new behavior.
# Description
Implements the main ideas in #13671 and a few more:
* Users can now specify only the environment and config options they
want to override in *their* `env.nu` and `config.nu`and yet still have
access to all of the defaults:
* `default_env.nu` (internally defined) will be loaded whenever (and
before) the user's `env.nu` is loaded.
* `default_config.nu` (internally defined) will be loaded whenever (and
before) the user's `config.nu` is loaded.
* No more 900+ line config out-of-the-box.
* Faster startup (again): ~40-45% improvement in launch time with a
default configuration.
* New keys that are added to the defaults in the future will
automatically be available to all users after updating Nushell. No need
to regenerate config to get the new defaults.
* It is now possible to have different internal defaults (which will be
used with `-c` and scripts) vs. REPL defaults. This would have solved
many of the user complaints about the [`display_errors`
implementation](https://www.nushell.sh/blog/2024-09-17-nushell_0_98_0.html#non-zero-exit-codes-are-now-errors-toc).
* A basic "scaffold" `config.nu` and `env.nu` are created on first
launch (if the config directory isn't present).
* Improved "out-of-the-box" experience (OOBE) - No longer asks to create
the files; the minimal scaffolding will be automatically created. If
deleted, they will not be regenerated. This provides a better
"out-of-the-box" experience for the user as they no longer have to make
this decision (without much info on the pros or cons) when first
launching.
* <s>(New: 2024-11-07) Runs the env_conversions process after the
`default_env.nu` is loaded so that users can treat `Path`/`PATH` as
lists in their own config.</s>
* (New: 2024-11-08) Given the changes in #13802, `default_config.nu`
will be a minimal file to minimize load-times. This shaves another (on
my system) ~3ms off the base launch time.
* Related: Keybindings, menus, and hooks that are already internal
defaults are no longer duplicated in `$env.config`. The documentation
will be updated to cover these scenarios.
* (New: 2024-11-08) Move existing "full" `default_config.nu` to
`sample_config.nu` for short-term "documentation" purposes.
* (New: 2024-11-18) Move the `dark-theme` and `light-theme` to Standard
Library and demonstrate their use - Also improves startup times, but
we're reaching the limit of optimization.
* (New: 2024-11-18) Extensively documented/commented `sample_env.nu` and
`sample_config.nu`. These can be displayed in-shell using (for example)
`config nu --sample | nu-highlight | less -R`. Note: Much of this will
eventually be moved to or (some) duplicated in the Doc. But for now,
this some nice in-shell doc that replaces the older
"commented/documented default".
* (New: 2024-11-20) Runs the `ENV_CONVERSIONS` process (1) after the
`default_env.nu` (allows `PATH` to be used as a list in user's `env.nu`)
and (2) before `default_config.nu` is loaded (allows user's
`ENV_CONVERSIONS` from their `env.nu` to be used in their `config.nu`).
* <s>(New: 2024-11-20) The default `ENV_CONVERSIONS` is now an empty
record. The internal Rust code handles `PATH` (and variants) conversions
regardless of the `ENV_CONVERSIONS` variable. This shaves a *very* small
amount of time off the startup.</s> Reset - Looks like there might be a
bug in `nu-enginer::env::ensure_path()` on Windows that would need to be
fixed in order for this to work.
# User-Facing Changes
By default, you shouldn't see much, if any, change when running this
with your existing configuration.
To see the greatest benefit from these changes, you'll probably want to
start with a "fresh" config. This can be easily tested using something
like:
```nushell
let temp_home = (mktemp -d)
$env.XDG_CONFIG_HOME = $temp_home
$env.XDG_DATA_HOME = $temp_home
./target/release/nu
```
You should see a message where the (mostly empty) `env.nu` and
`config.nu` are created on first start. Defaults should be the same (or
similar to) those before the PR. Please let me know if you notice any
differences.
---
Users should now specify configuration in terms of overrides of each
setting. For instance, rather than modifying `history` settings in the
monolithic `config.nu`, the following is recommended in an updated
`config.nu`:
```nu
$env.config.history = {
file_format: sqlite,
sync_on_enter: true
isolation: true
max_size: 1_000_000
}
```
or even just:
```nu
$env.config.history.file_format = sqlite
$env.config.history.isolation: true
$env.config.history.max_size = 1_000_000
```
Note: It seems many users are already appending a `source my_config.nu`
(or similar pattern) to the end of the existing `config.nu` to make
updates easier. In this case, they will likely want to remove all of the
previous defaults and just move their `my_config.nu` to `config.nu`.
Note: It should be unlikely that there are any breaking changes here,
but there's a slim chance that some code, somewhere, *expects* an
absence of certain config values. Otherwise, all config values are
available before and after this change.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
Configuration Chapter (and related) of the doc is currently WIP and will
be finished in time for 0.101 release.
# Release Notes Excerpt
* Hooks now default to an empty value of the proper type (e.g., `[]` or
`{}`) when not otherwise specified
# Description
```nushell
# Start with no config
nu -n
# Populate with defaults
$env.config = {}
$env.config.hooks
```
* Before: All hooks other than `display_output` were set to `null`.
Attempting to append a hook using `++=` would fail unless it had already
been assigned.
* After:
* `pre_prompt`, `pre_execution`, and `command_not_found` are set to
empty lists. This allows the user to simply append new hooks using
`++=`.
* `env_change` is set to an empty record. This allows the user to add
new hooks using `merge`, although a "helper" command would still be
useful (TODO: stdlib).
Also fixed a typo in an error message.
# User-Facing Changes
There shouldn't be any breaking changes since (before) there were no
guarantees of the hook's value/type. Previously, users would have to
check for `null` and `default` to an empty list before appending. Any
user-strategies for dealing with the problem should continue to work
after this change.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
Note that, for reasons I cannot ascertain, this PR appears to have
*fixed* the `command_not_found_error_recognizes_non_executable_file`
test that was previously broken by #12953. That PR essentially rewrote
the test to match the new behavior, but it no longer tested what it was
intended to test.
Now, the test is working again as designed (and as it works in the
REPL).
# After Submitting
This will be covered in the Configuration update for #14249. This PR
will simplify several examples in the doc.
# Description
Removes the `NU_DISABLE_IR` option and some code related to evaluating
blocks with the AST
evaluator.
Does not entirely remove the AST evaluator yet. We still have some
dependencies on expression
evaluation in a few minor places which will take a little bit of effort
to fix.
Also changes `debug profile` to always include instructions, because the
output is a little
confusing otherwise, and removes the different options for
instructions/exprs.
# User-Facing Changes
- `NU_DISABLE_IR` no longer has any effect, and is removed. There is no
way to use the AST
evaluator.
- `debug profile` no longer has `--exprs`, `--instructions` options.
- `debug profile` lists `pc` and `instruction` columns by default now.
# Tests + Formatting
Eval tests fixed to only use IR.
# After Submitting
- [ ] release notes
- [ ] finish removing AST evaluator, come up with solutions for the
expression evaluation.
# Description
Fixes#14294 - Turned out to be a whole lot easier than I expected, but
please double-check me on this, since it's an area I haven't been in
before.
# User-Facing Changes
Allow date to be added to a duration type.
# Tests + Formatting
Tests added:
* Duration + Date is allowed
* Duration - Date is not allowed
Fixes#13776
# User-Facing Changes
Arguments to aliased externals no longer include nested import paths:
```diff
module foo { export alias bar = ^echo }
use foo
foo bar baz
-bar baz
+baz
```
# User-Facing Changes
Table literal arguments to list parameters are now correctly parsed:
```diff
def a [l: list<any>] { $l | to nuon }; a [[a]; [2]]
-[[a]]
+[[a]; [2]]
```
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Fixes: #13362
This PR fixes the `Display` impl for `CellPath`, as laid out in #13362
and #14090:
```nushell
> $.0."0"
$.0."0"
> $."foo.bar".baz
$."foo.bar".baz
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Cell-paths are now printed using the same `$.` notation that is used to
create them, and ambiguous column names are properly quoted.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes: #14202
After looking into the issue, I think #13910 it's not good to cut the
span if it's in external argument.
This pr is somehow revert the change, and fix
https://github.com/nushell/nushell/issues/13431 in another way.
It introduce a new state named `State::BackTickQuote`, so if an external
arg include backtick quote, it enters the state, so backtick quote won't
be the body of a string.
# User-Facing Changes
### Before
```nushell
> ^echo `(echo aa)`
aa
> ^echo `"aa"` # maybe it's not right to remove the inner quote.
aa
```
### After
```nushell
> ^echo `(echo aa)`
(echo aa)
> ^echo `"aa"` # inner quote is keeped if there are backtick quote outside.
"aa"
```
# Tests + Formatting
Added 3 tests.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
This PR is supposed to fix#13582, #11522, as well as related goto
definition/reference issues (wrong position if non ascii characters
ahead).
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
<img width="411" alt="image"
src="https://github.com/user-attachments/assets/9a81953c-81b2-490d-a842-14ccaefd6972">
Changes:
1. span/completion should use byte offset instead of character index
2. lsp Postions related ops in Ropey remain to use character index
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Should be none, tested in neovim with config:
```lua
require("lspconfig").nushell.setup({
cmd = {
"nu",
"-I",
vim.fn.getcwd(),
"--no-config-file",
"--lsp",
},
filetypes = { "nu" },
})
```
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
tests::complete_command_with_utf_line parameters fixed to align with
true lsp requests (in character index, not byte).
As for the issue_11522.nu, manually tested:
<img width="520" alt="image"
src="https://github.com/user-attachments/assets/45496ba8-5a2d-4998-9190-d7bde31ee72c">
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Fixes#13757, fixes#9562
# User-Facing Changes
- `unclosed |` is returned for malformed closure parameters:
```
{ |a }
```
- Parameter list closing pipes are highlighted as part of the closure
# Description
This PR changes the range contains logic to take the step into account.
```nushell
# before
2 in 1..3.. # true
# now
2 in 1..3.. # false
```
---
I encountered another issue while adding tests. Due to floating point
precision, `2.1 in 1..1.1..3` will return `false`. The floating point
error is even bigger than `f64::EPSILON` (`0.09999999999999876` vs
`2.220446049250313e-16`). This issue disappears with bigger numbers.
I tried a different algorithm (checking if the estimated number of steps
is close enough to any integer) but the results are still pretty bad:
```rust
let n_steps = (value - self.start) / self.step; // 14.999999999999988
(n_steps - n_steps.round()).abs() < f64::EPSILON // returns false
```
Maybe it can be shipped like this, the REPL already has floating point
errors (`1.1 - 1` returns `0.10000000000000009`). Or maybe there's a way
to fix this that I didn't think of. I'm open to ideas! But in any case
performing this kind of checks on a range of floats seems more niche
than doing it on a range of ints.
# User-Facing Changes
Code that depended on this behavior to check if a number is between
`start` and `end` will potentially return a different value.
# Tests + Formatting
# After Submitting
# Description
Fixes: #13967
The key changes lays in `nu-protocol/src/module.rs`, when resolving
import pattern, nushell only needs to bring `$module` with a record
value if it defines any constants.
# User-Facing Changes
```nushell
module spam {}
use spam
```
Will no longer create a `$spam` variable with an empty record.
# Tests + Formatting
Adjusted some tests and added some tests.
Closes#13654
# User-Facing Changes
- Short flags are now fully type-checked,
including null and record signatures for literal arguments:
```nushell
def test [-v: record<l: int>] {};
test -v null # error
test -v {l: ""} # error
def test2 [-v: int] {};
let v = ""
test2 -v $v # error
```
- `polars unpivot` `--index`/`--on` and `into value --columns`
now accept `list` values
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
# Description
Fixes: #13967
The key changes lays in `nu-protocol/src/module.rs`, when resolving
import pattern, nushell only needs to bring `$module` with a record
value if it defines any constants.
# User-Facing Changes
```nushell
module spam {}
use spam
```
Will no longer create a `$spam` variable with an empty record.
# Tests + Formatting
Adjusted some tests and added some tests.
Closes#13654
# User-Facing Changes
- Short flags are now fully type-checked,
including null and record signatures for literal arguments:
```nushell
def test [-v: record<l: int>] {};
test -v null # error
test -v {l: ""} # error
def test2 [-v: int] {};
let v = ""
test2 -v $v # error
```
- `polars unpivot` `--index`/`--on` and `into value --columns`
now accept `list` values
# Description
[Context on
Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583)
**This is a breaking change, due to the removal of `is_running`.**
Some users find the `plugin list` command confusing, because it doesn't
show anything different after running `plugin add` or `plugin rm`. This
modifies the `plugin list` command to also look at the plugin registry
file to give some idea of how the plugins in engine state differ from
those in the plugin registry file.
The following values of `status` are now produced instead of
`is_running`:
- `added`: The plugin is present in the plugin registry file, but not in
the engine.
- `loaded`: The plugin is present both in the plugin registry file and
in the engine, but is not running.
- `running`: The plugin is currently running, and the `pid` column
should contain its process ID.
- `modified`: The plugin state present in the plugin registry file is
different from the state in the engine.
- `removed`: The plugin is still loaded in the engine, but is not
present in the plugin registry file.
- `invalid`: The data in the plugin registry file couldn't be
deserialized, and the plugin most likely needs to be added again.
Example (`commands` omitted):
```
╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮
│ # │ name │ version │ status │ pid │ filename │ shell │
├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤
│ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │
│ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │
│ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │
│ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │
│ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │
│ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │
│ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │
│ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │
│ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │
│ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │
╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯
```
# User-Facing Changes
To `plugin list`:
* **Breaking:** The `is_running` column is removed and replaced with
`status`. Use `status == running` to filter equivalently.
* The `--plugin-config` from other plugin management commands is now
supported.
* Added an `--engine` flag which behaves more or less like before, and
doesn't load the plugin registry file at all.
* Added a `--registry` flag which only checks the plugin registry file.
All plugins appear as `added` since there is no state to compare with.
Because the default is to check both, the `plugin list` command might be
a little bit slower. If you don't need to check the plugin registry
file, the `--engine` flag does not load the plugin registry file at all,
so it should be just as fast as before.
# Tests + Formatting
Added tests for `added` and `removed` statuses. `modified` and `invalid`
are a bit more tricky so I didn't try.
# After Submitting
- [ ] update documentation that references the `plugin list` command
- [ ] release notes
The idea comes from @amtoine, I think it would be good to keey
`display_error.exit_code` same value, if user is using default config or
using no config file at all.
# Description
This PR standardizes updates to the config through a new
`UpdateFromValue` trait. For now, this trait is private in case we need
to make changes to it.
Note that this PR adds some additional `ShellError` cases to create
standard error messages for config errors. A follow-up PR will move
usages of the old error cases to these new ones. This PR also uses
`Type::custom` in lots of places (e.g., for string enums). Not sure if
this is something we want to encourage.
# User-Facing Changes
Should be none.
# Description
Currently there is a bit of chaos regarding construction of history file
paths. Various pieces of code across a number of crates reimplement the
same/similar logic:
- There is `get_history_path`, but it requires a directory parameter (it
really just joins it with a file name).
- Some places use a const for the directory parameter, others use a
string literal - in all cases the value seems to be `"nushell"`.
- Some places assume the `"nushell"` value, other plumb it down from
close to the top of the call stack.
- Some places use a constant for history file names while others assume
it.
This PR tries to make it so that the history/config path format is
defined in a single places and so dependencies on it are easier to
follow:
- It removes `get_history_path` and adds a `file_path` method to
`HistoryConfig` instead (an extra motivation being, this is a convenient
place that can be used from all creates that need a history file path)
- Adds a `nu_config_dir` function that returns the nushell configuration
directory.
- Updates existing code to rely on the above, effectively removing
duplicate uses of `"nushell"` and `NUSHELL_FOLDER` and assumptions about
file names associated with different history formats
# User-Facing Changes
None
# Description
Fixes a small side-issue in #10977 - If a command flag didn't have a
comment/description, it would still show an unnecessary separator at the
end of the line.
This fixes that, plus uses the `: ` (colon) to separate the flag from
the description. This aligns with the way that named parameters are
handled.
# User-Facing Changes
Help/doc only
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
N/A
Updated summary for commit
[612e0e2](612e0e2160)
- While folks are welcome to read through the entire comments, the core
information is summarized here.
# Description
This PR drastically improves startup times of Nushell by only parsing a
single submodule of the Standard Library that provides the `banner` and
`pwd` commands. All other Standard Library commands and submodules are
parsed when imported by the user. This cuts startup times by more than
60%.
At the moment, we have stopped adding to `std-lib` because every
addition adds a small amount to the Nushell startup time.
With this change, we should once again be able to allow new
functionality to be added to the Standard Library without it impacting
`nu` startup times.
# User-Facing Changes
* Nushell now starts about 60% faster
* Breaking change: The `dirs` (Shells) aliases will return a warning
message that it will not be auto-loaded in the following release, along
with instructions on how to restore it (and disable the message)
* The `use std <submodule> *` syntax is available for convenience, but
should be avoided in scripts as it parses the entire `std` module and
all other submodules and places it in scope. The correct syntax to
*just* load a submodule is `use std/<submodule> *` (asterisk optional).
The slash is important. This will be documented.
* `use std *` can be used for convenience to load all of the library but
still incurs the full loading-time.
* `std/dirs`: Semi-breaking change. The `dirs` command replaces the
`show` command. This is more in line with the directory-stack
functionality found in other shells. Existing users will not be impacted
by this as the alias (`shells`) remains the same.
* Breaking-change: Technically a breaking change, but probably only
impacts maintainers of `std`. The virtual path for the standard library
has changed. It could previously be imported using its virtual path (and
technically, this would have been the correct way to do it):
```nu
use NU_STDLIB_VIRTUAL_DIR/std
```
The path is now simply `std/`:
```nu
use std
```
All submodules have moved accordingly.
# Timings
Comparisons below were made:
* In a temporary, clean config directory using `$env.XDG_CONFIG_HOME =
(mktemp -d)`.
* `nu` was run with a release build
* `nu` was run one time to generate the default `config.nu` (etc.) files
- Otherwise timings would include the user-prompt
* The shell was exited and then restarted several times to get timing
samples
(Note: Old timings based on 0.97 rather than 0.98, but in the range of
being accurate)
| Scenario | `$nu.startup-time` |
| --- | --- |
| 0.97.2
([aaaab8e](aaaab8e070))
Without this PR | 23ms - 24ms |
| This PR with deprecated commands | 9ms - <11ms |
| This PR after deprecated commands are removed in following release |
8ms - <10ms |
| Final PR (remove deprecated), using `--no-std-lib` | 6.1ms to 6.4ms |
| Final PR (remove deprecated), using `--no-config-file` | 3.1ms - 3.6ms
|
| Final PR (remove deprecated), using `--no-config-file --no-std-lib` |
1ms - 1.5ms |
*These last two timings point to the opportunity for further
optimization (see comment in thread below (will link once I write it).*
# Implementation details for future maintenance
* `use std banner` is a ridiculously deceptive call. That call parses
and imports *all* of `std` into scope. Simply replacing it with `use
std/core *` is essentially what saves ~14-15ms. This *only* imports the
submodule with the `banner` and `pwd` commands.
* From the code-comments, the reason that `NU_STDLIB_VIRTUAL_DIR` was
used as a prefix was so that there wouldn't be an issue if a user had a
`./std/mod.nu` in the current directory. This does **not** appear to be
an issue. After removing the prefix, I tested with both a relative
module as well as one in the `$env.NU_LIB_DIRS` path, and in all cases
the *internal* `std` still took precedence.
* By removing the prefix, users can now `use std` (and variants) without
requiring that it already be parsed and in scope.
* In the next release, we'll stop autoloading the `dirs` (shells)
functionality. While this only costs an additional 1-1.5ms, I think it's
better moved to the `config.nu` where the user can optionally remove it.
The main reason is its use of aliases (which have also caused issues) -
The `n`, `p`, and `g` short-commands are valuable real-estate, and users
may want to map these to something else.
For this release, there's an `deprecated_dirs` module that is still
autoloaded. As with the top-level commands, use of these will give a
deprecation warning with instructions on how to handle going forward.
To help with this, moved the aliases to their own submodule inside the
`dirs` module.
* Also sneaks in a small change where the top-level `dirs` command is
now the replacement for `dirs show`
* Fixed a double-import of `assert` in `dirs.nu`
* The `show_banner` step is replaced with simply `banner` rather than
re-importing it.
* A `virtual_path` may now be referenced with either a forward-slash or
a backward-slash on Windows. This allows `use std/<submodule>` to work
on all platforms.
# Performance side-notes:
* Future parsing and/or IR improvements should improve performance even
further.
* While the existing load time penalty of `std-lib` was not noticeable
on many systems, Nushell runs on a wide-variety of hardware and OS
platforms. Slower platforms will naturally see a bigger jump in
performance here. For users starting multiple Nushell sessions
frequently (e.g., `tmux`, Zellij, `screen`, et. al.) it is recommended
to keep total startup time (including user configuration) under ~250ms.
# Tests + Formatting
* All tests are green
* Updated tests:
- Removed the test that confirmed that `std` was loaded (since we
don't).
- Removed the `shells` test since it is not autoloaded. Main `dirs.nu`
functionality is tested through `stdlib-test`.
- Many tests assumed that the library was fully loaded, because it was
(even though we didn't intend for it to be). Fixed those tests.
- Tests now import only the necessary submodules (e.g., `use
std/assert`, rather than `use std assert`)
- Some tests *thought* they were loading `std/log`, but were doing so
improperly. This was masked by the now-fixed "load-everything-into-scope
bug". Local CI would pass due the `$env.NU_LOG_<...>` variables being
inherited from the calling process, but would fail in the "clean" GitHub
CI environment. These tests have also been fixed.
* Added additional tests for the changes
# After Submitting
Will update the Standard Library doc page
# Description
Old code was comparing remaining positional arguments with total number
of arguments, where it should've compared remaining positional with
with remaining arguments of any kind. This means that if a function was
given too few arguments, `calculate_end_span` would believe that it
actually had too many arguments, since after parsing the first few
arguments, the number of remaining arguments needed were fewer than the
*total* number of arguments, of which we had used several.
Fixes#9072
Fixes: https://github.com/nushell/nushell/issues/13930
Fixes: https://github.com/nushell/nushell/issues/12069
Fixes: https://github.com/nushell/nushell/issues/8385
Extracted from #10381
## Bonus
It also improves the error handling on missing positional arguments
before keywords (no longer crashing since #9851). Instead of just giving
the keyword to the parser for the missing positional, we give an
explicit error about a missing positional argument. I would like better
descriptions than "missing var_name" though, but I'm not sure if that's
available without
Old error
```
Error: nu::parser::parse_mismatch
× Parse mismatch during operation.
╭─[entry #1:1:1]
1 │ let = if foo
· ┬
· ╰── expected valid variable name
╰────
```
New error
```
Error: nu::parser::missing_positional
× Missing required positional argument.
╭─[entry #18:1:1]
1 │ let = foo
· ┬
· ╰── missing var_name
╰────
help: Usage: let <var_name> = <initial_value>
```
# User-Facing Changes
The program `alias = = =` is no longer accepted by the parser
This PR sets the current working directory to the location of the
Nushell executable at startup, using `std::env::set_current_dir()`. This
is desirable because after PR
https://github.com/nushell/nushell/pull/12922, we no longer change our
current working directory even after `cd` is executed, and some OS might
lock the directory where Nushell started.
The location of the Nushell executable is chosen because it cannot be
removed while Nushell is running anyways, so we don't have to worry
about OS locking it.
This PR has the side effect that it breaks buggy command even harder.
I'll keep this PR as a draft until these commands are fixed, but it
might be helpful to pull this PR if you're working on fixing one of
those bugs.
---------
Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
# Description
Fixes a bug with `set_last_error()` introduced by @IanManske not being
called during the jump to an error handler in IR eval. Without this,
`$env.LAST_EXIT_CODE` wasn't getting set in the `catch` block for an
external.
# Tests + Formatting
Added a `tests/eval` test to cover this in both IR and non-IR eval
# Description
This is my first PR, and I'm looking for feedback to help me improve!
This PR fixes#13380 by expanding the path prior to parsing it.
Also I've removed some unused code in
[completion_common.rs](84e92bb02c/crates/nu-cli/src/completions/completion_common.rs
)
# User-Facing Changes
Auto-completion for "cd .../" now works by expanding to "cd ../../".
# Tests + Formatting
Formatted and added 2 tests for triple dots in the middle of a path and
at the end.
Also added a test for the expand_ndots() function.
# Description
This PR makes it so that non-zero exit codes and termination by signal
are treated as a normal `ShellError`. Currently, these are silent
errors. That is, if an external command fails, then it's code block is
aborted, but the parent block can sometimes continue execution. E.g.,
see #8569 and this example:
```nushell
[1 2] | each { ^false }
```
Before this would give:
```
╭───┬──╮
│ 0 │ │
│ 1 │ │
╰───┴──╯
```
Now, this shows an error:
```
Error: nu:🐚:eval_block_with_input
× Eval block failed with pipeline input
╭─[entry #1:1:2]
1 │ [1 2] | each { ^false }
· ┬
· ╰── source value
╰────
Error: nu:🐚:non_zero_exit_code
× External command had a non-zero exit code
╭─[entry #1:1:17]
1 │ [1 2] | each { ^false }
· ──┬──
· ╰── exited with code 1
╰────
```
This PR fixes#12874, fixes#5960, fixes#10856, and fixes#5347. This
PR also partially addresses #10633 and #10624 (only the last command of
a pipeline is currently checked). It looks like #8569 is already fixed,
but this PR will make sure it is definitely fixed (fixes#8569).
# User-Facing Changes
- Non-zero exit codes and termination by signal now cause an error to be
thrown.
- The error record value passed to a `catch` block may now have an
`exit_code` column containing the integer exit code if the error was due
to an external command.
- Adds new config values, `display_errors.exit_code` and
`display_errors.termination_signal`, which determine whether an error
message should be printed in the respective error cases. For
non-interactive sessions, these are set to `true`, and for interactive
sessions `display_errors.exit_code` is false (via the default config).
# Tests
Added a few tests.
# After Submitting
- Update docs and book.
- Future work:
- Error if other external commands besides the last in a pipeline exit
with a non-zero exit code. Then, deprecate `do -c` since this will be
the default behavior everywhere.
- Add a better mechanism for exit codes and deprecate
`$env.LAST_EXIT_CODE` (it's buggy).
Fixesnushell/nushell#13689
# Description
Respect user-defined `$env.NU_LOG_FORMAT` and `$env.NU_LOG_DATE_FORMAT`
Additionally I fixed `nu_with_std!()` macro (it was not working
correctly)
# User-Facing Changes
Users now may set `$env.NU_LOG_FORMAT` and `$env.NU_LOG_DATE_FORMAT` in
`env.nu` and it will work even if `use std` is used after that.
# Tests + Formatting
Added a couple of tests for the new functionality.
# After Submitting
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Currently the parser and the documentation generation use the signature
of the command, which means that it doesn't pick up on the changed name
of the `main` block, and therefore shows the name of the command as
"main" and doesn't find the subcommands. This PR changes the
aforementioned places to use the block signature to fix these issues.
This closes#13397. Incidentally it also causes input/output types to be
shown in the help, which is kinda pointless for scripts since they don't
operate on structured data but maybe not worth the effort to remove.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
```
# example.nu
export def main [] { help main }
export def 'main sub' [] { print 'sub' }
```
Before:
After:
# Tests
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests are still missing for the subcommands and the input/output types
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
`bits rol` and `bits ror` were both undefined for the full byte rotates
and panicked when exceeding the byte rotation range.
`bits ror` further more produced nonsensical results by pulling bits
from the following byte instead of the preceding byte.
Those bugs are now fixed
# User-Facing Changes
Sound Nushell `IncorrectValue` error when exceeding the available bits
# Tests + Formatting
Added the necessary tests
# Description
The meaning of the word usage is specific to describing how a command
function is *used* and not a synonym for general description. Usage can
be used to describe the SYNOPSIS or EXAMPLES sections of a man page
where the permitted argument combinations are shown or example *uses*
are given.
Let's not confuse people and call it what it is a description.
Our `help` command already creates its own *Usage* section based on the
available arguments and doesn't refer to the description with usage.
# User-Facing Changes
`help commands` and `scope commands` will now use `description` or
`extra_description`
`usage`-> `description`
`extra_usage` -> `extra_description`
Breaking change in the plugin protocol:
In the signature record communicated with the engine.
`usage`-> `description`
`extra_usage` -> `extra_description`
The same rename also takes place for the methods on
`SimplePluginCommand` and `PluginCommand`
# Tests + Formatting
- Updated plugin protocol specific changes
# After Submitting
- [ ] update plugin protocol doc
# Description
Fixes#11267
Shifting by a `shift >= num_bits` is undefined in the underlying
operation. Previously we also had an overflow on negative shifts for the
operators `bit-shl` and `bit-shr`
Furthermore I found a severe bug in the implementation of shifting of
`binary` data with the commands `bits shl` and `bits shr`, this
categorically produced incorrect results with shifts that were not
`shift % 4 == 0`. `bits shr` also was able to produce outputs with
different size to the input if the shift was exceeding the length of the
input data by more than a byte.
# User-Facing Changes
It is now an error trying to shift by more than the available bits with:
- `bit-shl` operator
- `bit-shr` operator
- command `bits shl`
- command `bits shr`
# Tests + Formatting
Added testing for all relevant cases
# Description
Fixes Issue #13477
This adds a check to see if a user is trying to invoke a
(non-executable) file as a command and returns a helpful error if so.
EDIT: this will not work on Windows, and is arguably not relevant there,
because of the different semantics of executables. I think the
equivalent on Windows would be if a user tries to invoke `./foo`, we
should look for `foo.exe` or `foo.bat` in the directory and recommend
that if it exists.
# User-Facing Changes
When a user invokes an unrecognized command that is the path to an
existing file, the error used to say:
`{name} is neither a Nushell built-in or a known external command`
This PR proposes to change the message to:
`{name} refers to a file that is not executable. Did you forget to to
set execute permissions?`
# Tests + Formatting
Ran cargo fmt, clippy and test on the workspace.
EDIT: added test asserting the new behavior
# Description
As per our Wednesday meeting, this adds a parse error when something
that would be parsed as an external call is present at the top level,
unless the head of the external call begins with a caret (to make it
explicit).
I tried to make the error quite descriptive about what should be done.
# User-Facing Changes
These now cause a parse error:
```nushell
$foo = bar
$foo = `bar`
```
These would have been interpreted as strings before this version, but
now they'd be interpreted as external calls. This behavior is consistent
with `let`/`mut` (which is unaffected by this change).
Here is an example of the error:
```
Error: × External command calls must be explicit in assignments
╭─[entry #3:1:8]
1 │ $foo = bar
· ─┬─
· ╰── add a caret (^) before the command name if you intended to run and capture its output
╰────
help: the parsing of assignments was changed in 0.97.0, and this would have previously been treated as a string.
Alternatively, quote the string with single or double quotes to avoid it being interpreted as a command name. This
restriction may be removed in a future release.
```
# Tests + Formatting
Tests added to cover the change. Note made about it being temporary.
# Description
Part 4 of replacing std::path types with nu_path types added in
https://github.com/nushell/nushell/pull/13115. This PR migrates various
tests throughout the code base.
# Description
Part 3 of replacing `std::path` types with `nu_path` types added in
#13115. This PR targets the paths listed in `$nu`. That is, the home,
config, data, and cache directories.
# Description
This makes assignment operations and `const` behave the same way `let`
and `mut` do, absorbing the rest of the pipeline.
Changes the lexer to be able to recognize assignment operators as a
separate token, and then makes the lite parser continue to push spans
into the same command regardless of any redirections or pipes if an
assignment operator is encountered. Because the pipeline is no longer
split up by the lite parser at this point, it's trivial to just parse
the right hand side as if it were a subexpression not contained within
parentheses.
# User-Facing Changes
Big breaking change. These are all now possible:
```nushell
const path = 'a' | path join 'b'
mut x = 2
$x = random int
$x = [1 2 3] | math sum
$env.FOO = random chars
```
In the past, these would have led to (an attempt at) bare word string
parsing. So while `$env.FOO = bar` would have previously set the
environment variable `FOO` to the string `"bar"`, it now tries to run
the command named `bar`, hence the major breaking change.
However, this is desirable because it is very consistent - if you see
the `=`, you can just assume it absorbs everything else to the right of
it.
# Tests + Formatting
Added tests for the new behaviour. Adjusted some existing tests that
depended on the right hand side of assignments being parsed as
barewords.
# After Submitting
- [ ] release notes (breaking change!)
# Description
[Discovered](https://discord.com/channels/601130461678272522/614593951969574961/1266503282554179604)
by `@warp` on Discord.
The IR compiler was not properly setting redirect modes for
subexpressions because `FullCellPath` was always being compiled with
capture-out redirection. This is the correct behavior if there is a tail
to the `FullCellPath`, as we need the value in order to try to extract
anything from it (although this is unlikely to work) - however, the
parser also generates `FullCellPath`s with an empty tail quite often,
including for bare subexpressions.
Because of this, the following did not behave as expected:
```nushell
(docker run -it --rm alpine)
```
Capturing the output meant that `docker` didn't have direct access to
the terminal as a TTY.
As this is a minor bug fix, it should be okay to include in the 0.96.1
patch release.
# User-Facing Changes
- Fixes the bug as described when running with IR evaluation enabled.
# Tests + Formatting
I added a test for this, though we're not currently running all tests
with IR on the CI, but it should ensure this behaviour is consistent.
The equivalent minimum repro I could find was:
```nushell
(nu --testbin cococo); null
```
as this should cause the `cococo` message to appear on stdout, and if
Nushell is capturing the output, it would be discarded instead.
# Description
Fixes#13441.
I must have forgotten that `Expr::Range` can contain other expressions,
so I wasn't searching for `$in` to replace within it. Easy fix.
# User-Facing Changes
Bug fix, ranges like `6 | 3..$in` work as expected now.
# Tests + Formatting
Added regression test.
# Description
This corrects the parsing of unknown arguments provided to known
externals to behave exactly like external arguments passed to normal
external calls.
I've done this by adding a `SyntaxShape::ExternalArgument` which
triggers the same parsing rules.
Because I didn't like how the highlighting looked, I modified the
flattener to emit `ExternalArg` flat shapes for arguments that have that
syntax shape and are plain strings/globs. This is the same behavior that
external calls have.
Aside from passing the tests, I've also checked manually that the
completer seems to work adequately. I can confirm that specified
positional arguments get completion according to their specified type
(including custom completions), and then anything remaining gets
filepath style completion, as you'd expect from an external command.
Thanks to @OJarrisonn for originally finding this issue.
# User-Facing Changes
- Unknown args are now parsed according to their specified syntax shape,
rather than `Any`. This may be a breaking change, though I think it's
extremely unlikely in practice.
- The unspecified arguments of known externals are now highlighted /
flattened identically to normal external arguments, which makes it more
clear how they're being interpreted, and should help the completer
function properly.
- Known externals now have an implicit rest arg if not specified named
`args`, with a syntax shape of `ExternalArgument`.
# Tests + Formatting
Tests added for the new behaviour. Some old tests had to be corrected to
match.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] release notes (bugfix, and debatable whether it's a breaking
change)
# Description
Adds functionality to the plugin interface to support calling internal
commands from plugins. For example, using `view ir --json`:
```rust
let closure: Value = call.req(0)?;
let Some(decl_id) = engine.find_decl("view ir")? else {
return Err(LabeledError::new("`view ir` not found"));
};
let ir_json = engine.call_decl(
decl_id,
EvaluatedCall::new(call.head)
.with_named("json".into_spanned(call.head), Value::bool(true, call.head))
.with_positional(closure),
PipelineData::Empty,
true,
false,
)?.into_value()?.into_string()?;
let ir = serde_json::from_value(&ir_json);
// ...
```
# User-Facing Changes
Plugin developers can now use `EngineInterface::find_decl()` and
`call_decl()` to call internal commands, which could be handy for
formatters like `to csv` or `to nuon`, or for reflection commands that
help gain insight into the engine.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] release notes
- [ ] update plugin protocol documentation: `FindDecl`, `CallDecl`
engine calls; `Identifier` engine call response
# Description
This grew quite a bit beyond its original scope, but I've tried to make
`$in` a bit more consistent and easier to work with.
Instead of the parser generating calls to `collect` and creating
closures, this adds `Expr::Collect` which just evaluates in the same
scope and doesn't require any closure.
When `$in` is detected in an expression, it is replaced with a new
variable (also called `$in`) and wrapped in `Expr::Collect`. During
eval, this expression is evaluated directly, with the input and with
that new variable set to the collected value.
Other than being faster and less prone to gotchas, it also makes it
possible to typecheck the output of an expression containing `$in`,
which is nice. This is a breaking change though, because of the lack of
the closure and because now typechecking will actually happen. Also, I
haven't attempted to typecheck the input yet.
The IR generated now just looks like this:
```gas
collect %in
clone %tmp, %in
store-variable $in, %tmp
# %out <- ...expression... <- %in
drop-variable $in
```
(where `$in` is the local variable created for this collection, and not
`IN_VARIABLE_ID`)
which is a lot better than having to create a closure and call `collect
--keep-env`, dealing with all of the capture gathering and allocation
that entails. Ideally we can also detect whether that input is actually
needed, so maybe we don't have to clone, but I haven't tried to do that
yet. Theoretically now that the variable is a unique one every time, it
should be possible to give it a type - I just don't know how to
determine that yet.
On top of that, I've also reworked how `$in` works in pipeline-initial
position. Previously, it was a little bit inconsistent. For example,
this worked:
```nushell
> 3 | do { let x = $in; let y = $in; print $x $y }
3
3
```
However, this causes a runtime variable not found error on the second
`$in`:
```nushell
> def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
Error: nu:🐚:variable_not_found
× Variable not found
╭─[entry #115:1:35]
1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
· ─┬─
· ╰── variable not found
╰────
```
I've fixed this by making the first element `$in` detection *always*
happen at the block level, so if you use `$in` in pipeline-initial
position anywhere in a block, it will collect with an implicit
subexpression around the whole thing, and you can then use that `$in`
more than once. In doing this I also rewrote `parse_pipeline()` and
hopefully it's a bit more straightforward and possibly more efficient
too now.
Finally, I've tried to make `let` and `mut` a lot more straightforward
with how they handle the rest of the pipeline, and using a redirection
with `let`/`mut` now does what you'd expect if you assume that they
consume the whole pipeline - the redirection is just processed as
normal. These both work now:
```nushell
let x = ^foo err> err.txt
let y = ^foo out+err>| str length
```
It was previously possible to accomplish this with a subexpression, but
it just seemed like a weird gotcha that you couldn't do it. Intuitively,
`let` and `mut` just seem to take the whole line.
- closes#13137
# User-Facing Changes
- `$in` will behave more consistently with blocks and closures, since
the entire block is now just wrapped to handle it if it appears in the
first pipeline element
- `$in` no longer creates a closure, so what can be done within an
expression containing `$in` is less restrictive
- `$in` containing expressions are now type checked, rather than just
resulting in `any`. However, `$in` itself is still `any`, so this isn't
quite perfect yet
- Redirections are now allowed in `let` and `mut` and behave pretty much
how you'd expect
# Tests + Formatting
Added tests to cover the new behaviour.
# After Submitting
- [ ] release notes (definitely breaking change)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Replaces the `dirs_next` family of crates with `dirs`. `dirs_next` was
born when the `dirs` crates were abandoned three years ago, but they're
being maintained again and most projects depend on `dirs` nowadays.
`dirs_next` has been abandoned since.
This came up while working on
https://github.com/nushell/nushell/pull/13382.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
None.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests and formatter have been run.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
This PR adds an internal representation language to Nushell, offering an
alternative evaluator based on simple instructions, stream-containing
registers, and indexed control flow. The number of registers required is
determined statically at compile-time, and the fixed size required is
allocated upon entering the block.
Each instruction is associated with a span, which makes going backwards
from IR instructions to source code very easy.
Motivations for IR:
1. **Performance.** By simplifying the evaluation path and making it
more cache-friendly and branch predictor-friendly, code that does a lot
of computation in Nushell itself can be sped up a decent bit. Because
the IR is fairly easy to reason about, we can also implement
optimization passes in the future to eliminate and simplify code.
2. **Correctness.** The instructions mostly have very simple and
easily-specified behavior, so hopefully engine changes are a little bit
easier to reason about, and they can be specified in a more formal way
at some point. I have made an effort to document each of the
instructions in the docs for the enum itself in a reasonably specific
way. Some of the errors that would have happened during evaluation
before are now moved to the compilation step instead, because they don't
make sense to check during evaluation.
3. **As an intermediate target.** This is a good step for us to bring
the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at
some point, as code generated from new AST can be directly compared to
code generated from old AST. If the IR code is functionally equivalent,
it will behave the exact same way.
4. **Debugging.** With a little bit more work, we can probably give
control over advancing the virtual machine that `IrBlock`s run on to
some sort of external driver, making things like breakpoints and single
stepping possible. Tools like `view ir` and [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than
before to see what exactly is going on with your Nushell code.
The goal is to eventually replace the AST evaluator entirely, once we're
sure it's working just as well. You can help dogfood this by running
Nushell with `$env.NU_USE_IR` set to some value. The environment
variable is checked when Nushell starts, so config runs with IR, or it
can also be set on a line at the REPL to change it dynamically. It is
also checked when running `do` in case within a script you want to just
run a specific piece of code with or without IR.
# Example
```nushell
view ir { |data|
mut sum = 0
for n in $data {
$sum += $n
}
$sum
}
```
```gas
# 3 registers, 19 instructions, 0 bytes of data
0: load-literal %0, int(0)
1: store-variable var 904, %0 # let
2: drain %0
3: drop %0
4: load-variable %1, var 903
5: iterate %0, %1, end 15 # for, label(1), from(14:)
6: store-variable var 905, %0
7: load-variable %0, var 904
8: load-variable %2, var 905
9: binary-op %0, Math(Plus), %2
10: span %0
11: store-variable var 904, %0
12: load-literal %0, nothing
13: drain %0
14: jump 5
15: drop %0 # label(0), from(5:)
16: drain %0
17: load-variable %0, var 904
18: return %0
```
# Benchmarks
All benchmarks run on a base model Mac Mini M1.
## Iterative Fibonacci sequence
This is about as best case as possible, making use of the much faster
control flow. Most code will not experience a speed improvement nearly
this large.
```nushell
def fib [n: int] {
mut a = 0
mut b = 1
for _ in 2..=$n {
let c = $a + $b
$a = $b
$b = $c
}
$b
}
use std bench
bench { 0..50 | each { |n| fib $n } }
```
IR disabled:
```
╭───────┬─────────────────╮
│ mean │ 1ms 924µs 665ns │
│ min │ 1ms 700µs 83ns │
│ max │ 3ms 450µs 125ns │
│ std │ 395µs 759ns │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```
IR enabled:
```
╭───────┬─────────────────╮
│ mean │ 452µs 820ns │
│ min │ 427µs 417ns │
│ max │ 540µs 167ns │
│ std │ 17µs 158ns │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```
##
[gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu)
IR disabled:
```
╭───┬──────────────────╮
│ 0 │ 27ms 929µs 958ns │
│ 1 │ 21ms 153µs 459ns │
│ 2 │ 18ms 639µs 666ns │
│ 3 │ 19ms 554µs 583ns │
│ 4 │ 13ms 383µs 375ns │
│ 5 │ 11ms 328µs 208ns │
│ 6 │ 5ms 659µs 542ns │
╰───┴──────────────────╯
```
IR enabled:
```
╭───┬──────────────────╮
│ 0 │ 22ms 662µs │
│ 1 │ 17ms 221µs 792ns │
│ 2 │ 14ms 786µs 708ns │
│ 3 │ 13ms 876µs 834ns │
│ 4 │ 13ms 52µs 875ns │
│ 5 │ 11ms 269µs 666ns │
│ 6 │ 6ms 942µs 500ns │
╰───┴──────────────────╯
```
##
[random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu)
I got pretty random results out of this benchmark so I decided not to
include it. Not clear why.
# User-Facing Changes
- IR compilation errors may appear even if the user isn't evaluating
with IR.
- IR evaluation can be enabled by setting the `NU_USE_IR` environment
variable to any value.
- New command `view ir` pretty-prints the IR for a block, and `view ir
--json` can be piped into an external tool like [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir).
# Tests + Formatting
All tests are passing with `NU_USE_IR=1`, and I've added some more eval
tests to compare the results for some very core operations. I will
probably want to add some more so we don't have to always check
`NU_USE_IR=1 toolkit test --workspace` on a regular basis.
# After Submitting
- [ ] release notes
- [ ] further documentation of instructions?
- [ ] post-release: publish `nu_plugin_explore_ir`
# Description
Fixes the lexer to recognize `out>|`, `err>|`, `out+err>|`, etc.
Previously only the short-style forms were recognized, which was
inconsistent with normal file redirections.
I also integrated it all more into the normal lex path by checking `|`
in a special way, which should be more performant and consistent, and
cleans up the code a bunch.
Closes#13331.
# User-Facing Changes
- Adds `out>|` (error), `err>|`, `out+err>|`, `err+out>|` as recognized
forms of the pipe redirection.
# Tests + Formatting
All passing. Added tests for the new forms.
# After Submitting
- [ ] release notes
# Description
From the feedbacks from @amtoine , it's good to make nushell shows error
for `o>|` syntax.
# User-Facing Changes
## Before
```nushell
'foo' o>| print 07/09/2024 06:44:23 AM
Error: nu::parser::parse_mismatch
× Parse mismatch during operation.
╭─[entry #6:1:9]
1 │ 'foo' o>| print
· ┬
· ╰── expected redirection target
```
## After
```nushell
'foo' o>| print 07/09/2024 06:47:26 AM
Error: nu::parser::parse_mismatch
× Parse mismatch during operation.
╭─[entry #1:1:7]
1 │ 'foo' o>| print
· ─┬─
· ╰── expected `|`. Redirection stdout to pipe is the same as piping directly.
╰────
```
# Tests + Formatting
Added one test
---------
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR fixes the problem pointed out in
https://github.com/nushell/nushell/issues/13204, where the Fish-like
completions aren't sorted properly (this PR doesn't close that issue
because the author there wants more than just fixed sort order).
The cause is all of the file/directory completions being fetched first
and then sorted all together while being treated as strings. Instead,
this PR sorts completions within each individual directory, avoiding
treating `/` as part of the path.
To do this, I removed the `sort` method from the completer trait (as
well as `get_sort_by`) and made all completers sort within the `fetch`
method itself. A generic `sort_completions` helper has been added to
sort lists of completions, and a more specific `sort_suggestions` helper
has been added to sort `Vec<Suggestion>`s.
As for the actual change that fixes the sort order for file/directory
completions, the `complete_rec` helper now sorts the children of each
directory before visiting their children. The file and directory
completers don't bother sorting at the end (except to move hidden files
down).
To reviewers: don't let the 29 changed files scare you, most of those
are just the test fixtures :)
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This is the current behavior with prefix matching:
And with fuzzy matching:
Notice how `partial/hello.txt` is the last suggestion, even though it
should come before `partial-a`. This is because the ASCII code for `/`
is greater than that of `-`, so `partial-` is put before `partial/`.
This is this PR's behavior with prefix matching (`partial/hello.txt` is
at the start):
And with fuzzy matching:
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- Modified the partial completions test fixture to test whether this PR
even fixed anything
- Modified fixture to test sort order of .nu completions (a previous
version of my changes didn't sort all the completions at the end but
there were no tests catching that)
- Added a test for making sure subcommand completions are sorted by
Levenshtein distance (a previous version of my changes sorted in
alphabetical order but there were no tests catching that)
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The derive macros provided by #13031 are very useful for plugin authors.
In this PR I made use of these macros for two commands.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
This Example usage could be highlighted in the changelog for plugin
authors as this is probably very useful for them.
# Description
Fixes: https://github.com/nushell/nushell/issues/12099
Currently if user run `use voice.nu`, and file is unchanged, then run
`use voice.nu` again. nushell will use the module directly, even if
submodule inside `voice.nu` is changed.
After discussed with @kubouch, I think it's ok to re-parse the module
file when:
1. It exports sub modules which are defined by a file
2. It uses other modules which are defined by a file
## About the change:
To achieve the behavior, we need to add 2 attributes to `Module`:
1. `imported_modules`: it tracks the other modules is imported by the
givem `module`, e.g: `use foo.nu`
2. `file`: the path of a module, if a module is defined by a file, it
will be `Some(path)`, or else it will be `None`.
After the change:
use voice.nu always read the file and parse it.
use voice will still use the module which is saved in EngineState.
# User-Facing Changes
use `xxx.nu` will read the file and parse it if it exports submodules or
uses submodules
# Tests + Formatting
Done
---------
Co-authored-by: Jakub Žádník <kubouch@gmail.com>
Fixesnushell/nushell#13207
# Description
This fixes the parsing of command usage when that command comes from a
file with CRLF line endings.
See nushell/nushell#13207 for more details.
# User-Facing Changes
Users on Windows will get correct autocompletion for `std` commands.
# Description
This allows plugins to report their version (and potentially other
metadata in the future). The version is shown in `plugin list` and in
`version`.
The metadata is stored in the registry file, and reflects whatever was
retrieved on `plugin add`, not necessarily the running binary. This can
help you to diagnose if there's some kind of mismatch with what you
expect. We could potentially use this functionality to show a warning or
error if a plugin being run does not have the same version as what was
in the cache file, suggesting `plugin add` be run again, but I haven't
done that at this point.
It is optional, and it requires the plugin author to make some code
changes if they want to provide it, since I can't automatically
determine the version of the calling crate or anything tricky like that
to do it.
Example:
```
> plugin list | select name version is_running pid
╭───┬────────────────┬─────────┬────────────┬─────╮
│ # │ name │ version │ is_running │ pid │
├───┼────────────────┼─────────┼────────────┼─────┤
│ 0 │ example │ 0.93.1 │ false │ │
│ 1 │ gstat │ 0.93.1 │ false │ │
│ 2 │ inc │ 0.93.1 │ false │ │
│ 3 │ python_example │ 0.1.0 │ false │ │
╰───┴────────────────┴─────────┴────────────┴─────╯
```
cc @maxim-uvarov (he asked for it)
# User-Facing Changes
- `plugin list` gets a `version` column
- `version` shows plugin versions when available
- plugin authors *should* add `fn metadata()` to their `impl Plugin`,
but don't have to
# Tests + Formatting
Tested the low level stuff and also the `plugin list` column.
# After Submitting
- [ ] update plugin guide docs
- [ ] update plugin protocol docs (`Metadata` call & response)
- [ ] update plugin template (`fn metadata()` should be easy)
- [ ] release notes
# Description
Removes the `which-support` cargo feature and makes all of its
feature-gated code enabled by default in all builds. I'm not sure why
this one command is gated behind a feature. It seems to be a relic of
older code where we had features for what seems like every command.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Test was failing with “did you mean” due to the `NEXTEST` env var being
present when running tests via `cargo nextest run`.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Description
Fixes: #13066
nushell should remove argument values' inner quote once it gets `=`.
Whatever it's a flag or not, and it also replace from `\"` to `"` before
passing it to external commands.
# User-Facing Changes
Given the shell script:
```shell
# test.sh
echo $@
```
## Before
```
> sh test.sh -ldflags="-s -w" github.com
-ldflags="-s -w" github.com
> sh test.sh exp='-s -w' github.com
exp='-s -w' github.com
```
## After
```
> sh test.sh -ldflags="-s -w" github.com
-ldflags=-s -w github.com
> sh test.sh exp='-s -w' github.com
exp=-s -w github.com
```
# Tests + Formatting
Added some tests
This PR fixes a bug where `.` is expanded into an empty string when used
as an argument to external commands. Fixes
https://github.com/nushell/nushell/issues/12948.
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
# Description
```nushell
❯ ls
╭───┬───────┬──────┬──────┬──────────╮
│ # │ name │ type │ size │ modified │
├───┼───────┼──────┼──────┼──────────┤
│ 0 │ a.txt │ file │ 0 B │ now │
╰───┴───────┴──────┴──────┴──────────╯
❯ ls a.
NO RECORDS FOUND
```
There is a completion issue on previous version, I think @amtoine have
reproduced it before. But currently I can't reproduce it on latest main.
To avoid such regression, I added some tests for completion.
---------
Co-authored-by: Antoine Stevan <44101798+amtoine@users.noreply.github.com>
# Description
Fixes: https://github.com/nushell/nushell/issues/7761
It's still unsure if we want to change the `range semantic` itself, but
it's good to keep range semantic consistent between nushell commands.
# User-Facing Changes
### Before
```nushell
❯ "abc" | str substring 1..=2
b
```
### After
```nushell
❯ "abc" | str substring 1..=2
bc
```
# Tests + Formatting
Adjust tests to fit new behavior
# Description
Changes `get_full_help` to take a `&dyn Command` instead of multiple
arguments (`&Signature`, `&Examples` `is_parser_keyword`). All of these
arguments can be gathered from a `Command`, so there is no need to pass
the pieces to `get_full_help`.
This PR also fixes an issue where the search terms are not shown if
`--help` is used on a command.
# Description
There is a bug when `hide-env` is used on environment variables that
were present at shell startup. Namely, child processes still inherit the
hidden environment variable. This PR fixes#12900, fixes#11495, and
fixes#7937.
# Tests + Formatting
Added a test.
# Description
This PR introduces a `ByteStream` type which is a `Read`-able stream of
bytes. Internally, it has an enum over three different byte stream
sources:
```rust
pub enum ByteStreamSource {
Read(Box<dyn Read + Send + 'static>),
File(File),
Child(ChildProcess),
}
```
This is in comparison to the current `RawStream` type, which is an
`Iterator<Item = Vec<u8>>` and has to allocate for each read chunk.
Currently, `PipelineData::ExternalStream` serves a weird dual role where
it is either external command output or a wrapper around `RawStream`.
`ByteStream` makes this distinction more clear (via `ByteStreamSource`)
and replaces `PipelineData::ExternalStream` in this PR:
```rust
pub enum PipelineData {
Empty,
Value(Value, Option<PipelineMetadata>),
ListStream(ListStream, Option<PipelineMetadata>),
ByteStream(ByteStream, Option<PipelineMetadata>),
}
```
The PR is relatively large, but a decent amount of it is just repetitive
changes.
This PR fixes#7017, fixes#10763, and fixes#12369.
This PR also improves performance when piping external commands. Nushell
should, in most cases, have competitive pipeline throughput compared to,
e.g., bash.
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| -------------------------------------------------- | -------------:|
------------:| -----------:|
| `throughput \| rg 'x'` | 3059 | 3744 | 3739 |
| `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 |
# User-Facing Changes
- This is a breaking change for the plugin communication protocol,
because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`.
Plugins now only have to deal with a single input stream, as opposed to
the previous three streams: stdout, stderr, and exit code.
- The output of `describe` has been changed for external/byte streams.
- Temporary breaking change: `bytes starts-with` no longer works with
byte streams. This is to keep the PR smaller, and `bytes ends-with`
already does not work on byte streams.
- If a process core dumped, then instead of having a `Value::Error` in
the `exit_code` column of the output returned from `complete`, it now is
a `Value::Int` with the negation of the signal number.
# After Submitting
- Update docs and book as necessary
- Release notes (e.g., plugin protocol changes)
- Adapt/convert commands to work with byte streams (high priority is
`str length`, `bytes starts-with`, and maybe `bytes ends-with`).
- Refactor the `tee` code, Devyn has already done some work on this.
---------
Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
# Description
Fixes: #12691
In `parse_short_flag`, it only checks special cases for
`SyntaxShape::Int`, `SyntaxShape::Number` to allow a flag to be a
number. This pr adds `SyntaxShape::Float` to allow a flag to be float
number.
# User-Facing Changes
This is possible after this pr:
```nushell
def spam [val: float] { $val };
spam -1.4
```
# Tests + Formatting
Added 1 test
# Description
This PR adds a single test to assert interactivity on slow pipelines
Currently the timeout is set to 6 seconds, as the test can sometimes
take ~3secs to run on my local m1 mac air, which I don't think is an
indication of a slow pipeline, but rather slow test start up time...
# Description
Fixes: #12795
The issue is caused by an empty position of `ParseError::UnexpectedEof`.
So no detailed message is displayed.
To fix the issue, I adjust the start of span to `span.end - 1`. In this
way, we can make sure that it never points to an empty position.
After lexing item, I also reorder the unclosed character checking . Now
it will be checking unclosed opening delimiters first.
# User-Facing Changes
After this pr, it outputs detailed error message for incomplete string
when running scripts.
## Before
```
❯ nu -c "'ab"
Error: nu::parser::unexpected_eof
× Unexpected end of code.
╭─[source:1:4]
1 │ 'ab
╰────
> ./target/debug/nu -c "r#'ab"
Error: nu::parser::unexpected_eof
× Unexpected end of code.
╭─[source:1:6]
1 │ r#'ab
╰────
```
## After
```
> nu -c "'ab"
Error: nu::parser::unexpected_eof
× Unexpected end of code.
╭─[source:1:3]
1 │ 'ab
· ┬
· ╰── expected closing '
╰────
> ./target/debug/nu -c "r#'ab"
Error: nu::parser::unexpected_eof
× Unexpected end of code.
╭─[source:1:5]
1 │ r#'ab
· ┬
· ╰── expected closing '#
╰────
```
# Tests + Formatting
Added some tests for incomplete string.
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
This PR should close#7147
# Description
Merged src/tests into /tests to produce a single binary.
# User-Facing Changes
No user facing changes
# Tests + Formatting
Moved tests. Tollkit check pr pass.
# After Submitting
---------
Co-authored-by: Ian Manske <ian.manske@pm.me>
# Description
Make typos config more strict: ignore false positives where they occur.
1. Ignore only files with typos
2. Add regexp-s with context
3. Ignore variable names only in Rust code
4. Ignore only 1 "identifier"
5. Check dot files
🎁 Extra bonus: fix typos!!
# Description
Judiciously try to avoid allocations/clone by changing the signature of
functions
- **Don't pass str by value unnecessarily if only read**
- **Don't require a vec in `Sandbox::with_files`**
- **Remove unnecessary string clone**
- **Fixup unnecessary borrow**
- **Use `&str` in shape color instead**
- **Vec -> Slice**
- **Elide string clone**
- **Elide `Path` clone**
- **Take &str to elide clone in tests**
# User-Facing Changes
None
# Tests + Formatting
This touches many tests purely in changing from owned to borrowed/static
data
# Description
This PR adds raw string support by using `r#` at the beginning of single
quoted strings and `#` at the end.
Notice that escapes do not process, even within single quotes,
parentheses don't mean anything, $variables don't mean anything. It's
just a string.
```nushell
❯ echo r#'one\ntwo (blah) ($var)'#
one\ntwo (blah) ($var)
```
Notice how they work without `echo` or `print` and how they work without
carriage returns.
```nushell
❯ r#'adsfa'#
adsfa
❯ r##"asdfa'@qpejq'##
asdfa'@qpejq
❯ r#'asdfasdfasf
∙ foqwejfqo@'23rfjqf'#
```
They also have a special configurable color in the repl. (use single
quotes though)
They should work like rust raw literals and allow `r##`, `r###`,
`r####`, etc, to help with having one or many `#`'s in the middle of
your raw-string.
They should work with `let` as well.
```nushell
r#'some\nraw\nstring'# | str upcase
```
closes https://github.com/nushell/nushell/issues/5091
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
---------
Co-authored-by: WindSoilder <WindSoilder@outlook.com>
Co-authored-by: Ian Manske <ian.manske@pm.me>
This PR changes `$env` to be **case-preserving** instead of
case-sensitive. That is, it preserves the case of the environment
variable when it is first assigned, but subsequent retrieval and update
ignores the case.
Notably, both `$env.PATH` and `$env.Path` can now be used to read or set
the environment variable, but child processes will always see the
correct case based on the platform.
Fixes#11268.
---
This feature was surprising simple to implement, because most of the
infrastructure to support case-insensitive cell path access already
exists. The `get` command extracts data using a cell path in a
case-insensitive way (!), but accepts a `--sensitive` flag. (I think
this should be flipped around?)
# Description
I've found that sometimes on Linux, this test fails to find the created
process even after it should definitely be running.
Trying to add a little delay.
# Description
I thought about bringing `nu_plugin_msgpack` in, but that is MPL with a
clause that prevents other licenses, so rather than adapt that code I
decided to take a crack at just doing it straight from `rmp` to `Value`
without any `rmpv` in the middle. It seems like it's probably faster,
though I can't say for sure how much with the plugin overhead.
@IanManske I started on a `Read` implementation for `RawStream` but just
specialized to `from msgpack` here, but I'm thinking after release maybe
we can polish it up and make it a real one. It works!
# User-Facing Changes
New commands:
- `from msgpack`
- `from msgpackz`
- `to msgpack`
- `to msgpackz`
# Tests + Formatting
Pretty thorough tests added for the format deserialization, with a
roundtrip for serialization. Some example tests too for both `from
msgpack` and `to msgpack`.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] update release notes
# Description
So far this seems like the winner of my poll on what the name should be.
I'll take this off draft once the poll expires, if this is indeed the
winner.
# Description
The local socket PR introduced a `Waitable` type, which could either
hold a value or be waited on until a value is available. Unlike a
channel, it would always return that value once set.
However, one issue with this design was that there was no way to detect
whether a value would ever be written. This splits the writer into a
different type `WaitableMut`, so that when it is dropped, waiting
threads can fail (because they'll never get a value).
# Tests + Formatting
A test has been added to `stress_internals` to make sure this fails in
the right way.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# Description
This allows the following commands to all accept a filename instead of a
plugin name:
- `plugin use`
- `plugin rm`
- `plugin stop`
Slightly complicated because of the need to also check against
`NU_PLUGIN_DIRS`, but I also fixed some issues with that at the same
time
Requested by @fdncred
# User-Facing Changes
The new commands are updated as described.
# Tests + Formatting
Tests for `NU_PLUGIN_DIRS` handling also made more robust.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Double check new docs to make sure they describe this capability
# Description
Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:
```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```
This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.
The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.
`register` now prints a deprecation parse warning.
This should fix#11923, as we now have a complete alternative to
`register`.
# User-Facing Changes
- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`
# Tests + Formatting
I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Update plugins sections of book
- [ ] Release notes
# Description
Fixes: #11351
And comment here is also fixed:
https://github.com/nushell/nushell/issues/11351#issuecomment-1996191537
The panic can happened if we pipe a variable to a custom command which
recursively called itself inside another block.
TBH, I think I figure out how it works to panic, but I'm not sure if
there is a potention issue if nushell don't mutate a block in such case.
# User-Facing Changes
Nan
# Tests + Formatting
Done
# After Submitting
Done
---------
Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
# Description
- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed
# User-Facing Changes
- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
- `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`
# Tests + Formatting
Tests added for the format and for the invalid handling.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Check for documentation changes
- [ ] Definitely needs release notes
# Description
`Value` describes the types of first-class values that users and scripts
can create, manipulate, pass around, and store. However, `Block`s are
not first-class values in the language, so this PR removes it from
`Value`. This removes some unnecessary code, and this change should be
invisible to the user except for the change to `scope modules` described
below.
# User-Facing Changes
Breaking change: the output of `scope modules` was changed so that
`env_block` is now `has_env_block` which is a boolean value instead of a
`Block`.
# After Submitting
Update the language guide possibly.
# Description
As suggested by @fdncred.
It's neat that this is possible, but the particularly useful part of
this is that we can actually
test it because it doesn't have any external dependencies, unlike the
python plugin.
Right now this just implements exactly the same behavior as the python
plugin, but we could have it
exercise a few more things.
Also fixes a couple of bugs:
- `.nu` plugins were not run with `nu --stdin`, so they couldn't take
input.
- `register` couldn't be called if `--no-config-file` was set, because
it would error on trying to
update the plugin file.
# User-Facing Changes
- `nu_plugin_nu_example` plugin added.
- `register` now works in `--no-config-file` mode.
# Tests + Formatting
Tests added for `nu_plugin_nu_example`.
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Add the version bump to the release script just like for python
# Description
EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).
Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).
Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
It shouldn't have any user facing changes.
# Description
Work for #7149
- **Error `with-env` given uneven count in list form**
- **Fix `with-env` `CantConvert` to record**
- **Error `with-env` when given protected env vars**
- **Deprecate list/table input of vars to `with-env`**
- **Remove examples for deprecated input**
# User-Facing Changes
## Deprecation of the following forms
```
> with-env [MYENV "my env value"] { $env.MYENV }
my env value
> with-env [X Y W Z] { $env.X }
Y
> with-env [[X W]; [Y Z]] { $env.W }
Z
```
## recommended standardized form
```
# Set by key-value record
> with-env {X: "Y", W: "Z"} { [$env.X $env.W] }
╭───┬───╮
│ 0 │ Y │
│ 1 │ Z │
╰───┴───╯
```
## (Side effect) Repeated definitions in an env shorthand are now
disallowed
```
> FOO=bar FOO=baz $env
Error: nu:🐚:column_defined_twice
× Record field or table column used twice: FOO
╭─[entry #1:1:1]
1 │ FOO=bar FOO=baz $env
· ─┬─ ─┬─
· │ ╰── field redefined here
· ╰── field first defined here
╰────
```
# Description
Adds support for running plugins using local socket communication
instead of stdio. This will be an optional thing that not all plugins
have to support.
This frees up stdio for use to make plugins that use stdio to create
terminal UIs, cc @amtoine, @fdncred.
This uses the [`interprocess`](https://crates.io/crates/interprocess)
crate (298 stars, MIT license, actively maintained), which seems to be
the best option for cross-platform local socket support in Rust. On
Windows, a local socket name is provided. On Unixes, it's a path. The
socket name is kept to a relatively small size because some operating
systems have pretty strict limits on the whole path (~100 chars), so on
macOS for example we prefer `/tmp/nu.{pid}.{hash64}.sock` where the hash
includes the plugin filename and timestamp to be unique enough.
This also adds an API for moving plugins in and out of the foreground
group, which is relevant for Unixes where direct terminal control
depends on that.
TODO:
- [x] Generate local socket path according to OS conventions
- [x] Add support for passing `--local-socket` to the plugin executable
instead of `--stdio`, and communicating over that instead
- [x] Test plugins that were broken, including
[amtoine/nu_plugin_explore](https://github.com/amtoine/nu_plugin_explore)
- [x] Automatically upgrade to using local sockets when supported,
falling back if it doesn't work, transparently to the user without any
visible error messages
- Added protocol feature: `LocalSocket`
- [x] Reset preferred mode to `None` on `register`
- [x] Allow plugins to detect whether they're running on a local socket
and can use stdio freely, so that TUI plugins can just produce an error
message otherwise
- Implemented via `EngineInterface::is_using_stdio()`
- [x] Clean up foreground state when plugin command exits on the engine
side too, not just whole plugin
- [x] Make sure tests for failure cases work as intended
- `nu_plugin_stress_internals` added
# User-Facing Changes
- TUI plugins work
- Non-Rust plugins could optionally choose to use this
- This might behave differently, so will need to test it carefully
across different operating systems
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
# After Submitting
- [ ] Document local socket option in plugin contrib docs
- [ ] Document how to do a terminal UI plugin in plugin contrib docs
- [ ] Document: `EnterForeground` engine call
- [ ] Document: `LeaveForeground` engine call
- [ ] Document: `LocalSocket` protocol feature
# Description
This is an attempt to isolate the unit tests from whatever might be in
the user's config. If the
user's config is broken in some way or incompatible with this version
(for example, especially if
there are plugins that aren't built for this version), tests can
spuriously fail.
This makes tests more reliably pass the same way they would on CI even
if the user has config, and
should also make them run faster.
I think this is _good enough_, but I still think we should have a
specific config dir env variable for nushell specifically (rather than
having to use `XDG_CONFIG_HOME`, which would mess with other things) and
then we can just have `nu-test-support` set that to a temporary dir
containing the shipped default config files.
# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Fixes#7849, #11465 based on @kubouch's suggestion in
https://github.com/nushell/nushell/issues/11465#issuecomment-1883847806.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
Can source files relative to `env.nu` or `config.nu` like in #6150.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Adds test that previously failed.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
