Commit Graph

1896 Commits

Author SHA1 Message Date
Andy Gayton
7b82c6b482
feat: make ctrlc available to plugins (#13181)
# Description

This PR adds a new method to `EngineInterface`: `register_ctrlc_handler`
which takes a closure to run when the plugin's driving engine receives a
ctrlc-signal. It also adds a mirror of the `signals` attribute from the
main shell `EngineState`.

This is an example of how a plugin which makes a long poll http request
can end the request on ctrlc:
https://github.com/cablehead/nu_plugin_http/blob/main/src/commands/request.rs#L68-L77

To facilitate the feature, a new attribute has been added to
`EngineState`: `ctrlc_handlers`. This is a Vec of closures that will be
run when the engine's process receives a ctrlc signal.

When plugins are added to an `engine_state` during a `merge_delta`, the
engine passes the ctrlc_handlers to the plugin's
`.configure_ctrlc_handler` method, which gives the plugin a chance to
register a handler that sends a ctrlc packet through the
`PluginInterface`, if an instance of the plugin is currently running.

On the plugin side: `EngineInterface` also has a ctrlc_handlers Vec of
closures. Plugin calls can use `register_ctrlc_handler` to register a
closure that will be called in the plugin process when the
PluginInput::Ctrlc command is received.

For future reference these are some alternate places that were
investigated for tying the ctrlc trigger to transmitting a Ctrlc packet
through the `PluginInterface`:

- Directly from `src/signals.rs`: the handler there would need a
reference to the Vec<Arc<RegisteredPlugins>>, which would require us to
wrap the plugins in a Mutex, which we don't want to do.

- have `PersistentPlugin.get_plugin` pass down the engine's
CtrlcHandlers to .get and then to .spawn (if the plugin isn't already
running). Once we have CtrlcHandlers in spawn, we can register a handler
to write directly to PluginInterface. We don't want to double down on
passing engine_state to spawn this way though, as it's unpredictable
because it would depend on whether the plugin has already been spawned
or not.

- pass `ctrlc_handlers` to PersistentPlugin::new so it can store it on
itself so it's available to spawn.

- in `PersistentPlugin.spawn`, create a handler that sends to a clone of
the GC event loop's tx. this has the same issues with regards to how to
get CtrlcHandlers to the spawn method, and is more complicated than a
handler that writes directly to PluginInterface

# User-Facing Changes

No breaking changes

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-07-30 08:29:18 -05:00
Jan Christian Grünhage
4665323bb4
Use directories for autoloading (#13382)
fixes https://github.com/nushell/nushell/issues/13378

# Description

This PR tries to improve usage of system APIs to determine the location
of vendored autoload files.

# User-Facing Changes
The paths listed in #13180 and #13217 are changing. This has not been
part of a release yet, so arguably the user facing changes are only to
unreleased features anyway.

# Tests + Formatting
Haven't done, but if someone wants to help me here, I'm open to doing
it. I just don't know how to properly test this.

# After Submitting
2024-07-19 03:47:07 -07:00
Jan Christian Grünhage
b66671d339
Switch from dirs_next 2.0 to dirs 5.0 (#13384)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Replaces the `dirs_next` family of crates with `dirs`. `dirs_next` was
born when the `dirs` crates were abandoned three years ago, but they're
being maintained again and most projects depend on `dirs` nowadays.
`dirs_next` has been abandoned since.

This came up while working on
https://github.com/nushell/nushell/pull/13382.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
None.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests and formatter have been run.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-16 07:16:26 -05:00
Devyn Cairns
d7392f1f3b
Internal representation (IR) compiler and evaluator (#13330)
# Description

This PR adds an internal representation language to Nushell, offering an
alternative evaluator based on simple instructions, stream-containing
registers, and indexed control flow. The number of registers required is
determined statically at compile-time, and the fixed size required is
allocated upon entering the block.

Each instruction is associated with a span, which makes going backwards
from IR instructions to source code very easy.

Motivations for IR:

1. **Performance.** By simplifying the evaluation path and making it
more cache-friendly and branch predictor-friendly, code that does a lot
of computation in Nushell itself can be sped up a decent bit. Because
the IR is fairly easy to reason about, we can also implement
optimization passes in the future to eliminate and simplify code.
2. **Correctness.** The instructions mostly have very simple and
easily-specified behavior, so hopefully engine changes are a little bit
easier to reason about, and they can be specified in a more formal way
at some point. I have made an effort to document each of the
instructions in the docs for the enum itself in a reasonably specific
way. Some of the errors that would have happened during evaluation
before are now moved to the compilation step instead, because they don't
make sense to check during evaluation.
3. **As an intermediate target.** This is a good step for us to bring
the [`new-nu-parser`](https://github.com/nushell/new-nu-parser) in at
some point, as code generated from new AST can be directly compared to
code generated from old AST. If the IR code is functionally equivalent,
it will behave the exact same way.
4. **Debugging.** With a little bit more work, we can probably give
control over advancing the virtual machine that `IrBlock`s run on to
some sort of external driver, making things like breakpoints and single
stepping possible. Tools like `view ir` and [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir) make it easier than
before to see what exactly is going on with your Nushell code.

The goal is to eventually replace the AST evaluator entirely, once we're
sure it's working just as well. You can help dogfood this by running
Nushell with `$env.NU_USE_IR` set to some value. The environment
variable is checked when Nushell starts, so config runs with IR, or it
can also be set on a line at the REPL to change it dynamically. It is
also checked when running `do` in case within a script you want to just
run a specific piece of code with or without IR.

# Example

```nushell
view ir { |data|
  mut sum = 0
  for n in $data {
    $sum += $n
  }
  $sum
}
```
  
```gas
# 3 registers, 19 instructions, 0 bytes of data
   0: load-literal           %0, int(0)
   1: store-variable         var 904, %0 # let
   2: drain                  %0
   3: drop                   %0
   4: load-variable          %1, var 903
   5: iterate                %0, %1, end 15 # for, label(1), from(14:)
   6: store-variable         var 905, %0
   7: load-variable          %0, var 904
   8: load-variable          %2, var 905
   9: binary-op              %0, Math(Plus), %2
  10: span                   %0
  11: store-variable         var 904, %0
  12: load-literal           %0, nothing
  13: drain                  %0
  14: jump                   5
  15: drop                   %0          # label(0), from(5:)
  16: drain                  %0
  17: load-variable          %0, var 904
  18: return                 %0
```

# Benchmarks

All benchmarks run on a base model Mac Mini M1.

## Iterative Fibonacci sequence

This is about as best case as possible, making use of the much faster
control flow. Most code will not experience a speed improvement nearly
this large.

```nushell
def fib [n: int] {
  mut a = 0
  mut b = 1
  for _ in 2..=$n {
    let c = $a + $b
    $a = $b
    $b = $c
  }
  $b
}
use std bench
bench { 0..50 | each { |n| fib $n } }
```

IR disabled:

```
╭───────┬─────────────────╮
│ mean  │ 1ms 924µs 665ns │
│ min   │ 1ms 700µs 83ns  │
│ max   │ 3ms 450µs 125ns │
│ std   │ 395µs 759ns     │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```

IR enabled:

```
╭───────┬─────────────────╮
│ mean  │ 452µs 820ns     │
│ min   │ 427µs 417ns     │
│ max   │ 540µs 167ns     │
│ std   │ 17µs 158ns      │
│ times │ [list 50 items] │
╰───────┴─────────────────╯
```

![explore ir
view](https://github.com/nushell/nushell/assets/10729/d7bccc03-5222-461c-9200-0dce71b83b83)

##
[gradient_benchmark_no_check.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/gradient_benchmark_no_check.nu)

IR disabled:

```
╭───┬──────────────────╮
│ 0 │ 27ms 929µs 958ns │
│ 1 │ 21ms 153µs 459ns │
│ 2 │ 18ms 639µs 666ns │
│ 3 │ 19ms 554µs 583ns │
│ 4 │ 13ms 383µs 375ns │
│ 5 │ 11ms 328µs 208ns │
│ 6 │  5ms 659µs 542ns │
╰───┴──────────────────╯
```

IR enabled:

```
╭───┬──────────────────╮
│ 0 │       22ms 662µs │
│ 1 │ 17ms 221µs 792ns │
│ 2 │ 14ms 786µs 708ns │
│ 3 │ 13ms 876µs 834ns │
│ 4 │  13ms 52µs 875ns │
│ 5 │ 11ms 269µs 666ns │
│ 6 │  6ms 942µs 500ns │
╰───┴──────────────────╯
```

##
[random-bytes.nu](https://github.com/nushell/nu_scripts/blob/main/benchmarks/random-bytes.nu)

I got pretty random results out of this benchmark so I decided not to
include it. Not clear why.

# User-Facing Changes
- IR compilation errors may appear even if the user isn't evaluating
with IR.
- IR evaluation can be enabled by setting the `NU_USE_IR` environment
variable to any value.
- New command `view ir` pretty-prints the IR for a block, and `view ir
--json` can be piped into an external tool like [`explore
ir`](https://github.com/devyn/nu_plugin_explore_ir).

# Tests + Formatting
All tests are passing with `NU_USE_IR=1`, and I've added some more eval
tests to compare the results for some very core operations. I will
probably want to add some more so we don't have to always check
`NU_USE_IR=1 toolkit test --workspace` on a regular basis.

# After Submitting
- [ ] release notes
- [ ] further documentation of instructions?
- [ ] post-release: publish `nu_plugin_explore_ir`
2024-07-10 17:33:59 -07:00
Ian Manske
399a7c8836
Add and use new Signals struct (#13314)
# Description
This PR introduces a new `Signals` struct to replace our adhoc passing
around of `ctrlc: Option<Arc<AtomicBool>>`. Doing so has a few benefits:
- We can better enforce when/where resetting or triggering an interrupt
is allowed.
- Consolidates `nu_utils::ctrl_c::was_pressed` and other ad-hoc
re-implementations into a single place: `Signals::check`.
- This allows us to add other types of signals later if we want. E.g.,
exiting or suspension.
- Similarly, we can more easily change the underlying implementation if
we need to in the future.
- Places that used to have a `ctrlc` of `None` now use
`Signals::empty()`, so we can double check these usages for correctness
in the future.
2024-07-07 22:29:01 +00:00
Jack Wright
1f1f581357
Converted perf function to be a macro. Utilized the perf macro within the polars plugin. (#13224)
In this pull request, I converted the `perf` function within `nu_utils`
to a macro. This change facilitates easier usage within plugins by
allowing the use of `env_logger` and setting `RUST_LOG=nu_plugin_polars`
(or another plugin). Without this conversion, the `RUST_LOG` variable
would need to be set to `RUST_LOG=nu_utils::utils`, which is less
intuitive and impossible to narrow the perf results to one plugin.
2024-06-27 18:56:56 -05:00
Darren Schroeder
f241110005
implement autoloading (#13217)
# Description

This PR implements script or module autoloading. It does this by finding
the `$nu.vendor-autoload-dir`, lists the contents and sorts them by file
name. These files are evaluated in that order.

To see what's going on, you can use `--log-level warn`
```
❯ cargo r -- --log-level warn
    Finished dev [unoptimized + debuginfo] target(s) in 0.58s
     Running `target\debug\nu.exe --log-level warn`
2024-06-24 09:23:20.494 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: config.nu, key: config-path, config_file_specified: None
2024-06-24 09:23:20.495 PM [WARN ] nu::config_files: set_config_path() cwd: "C:\\Users\\fdncred\\source\\repos\\nushell", default_config: env.nu, key: env-path, config_file_specified: None
2024-06-24 09:23:20.629 PM [WARN ] nu::config_files: setup_config() config_file_specified: None, env_file_specified: None, login: false
2024-06-24 09:23:20.660 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: true
Hello, from env.nu
2024-06-24 09:23:20.679 PM [WARN ] nu::config_files: read_config_file() config_file_specified: None, is_env_config: false
Hello, from config.nu
Hello, from defs.nu
Activating Microsoft Visual Studio environment.
2024-06-24 09:23:21.398 PM [WARN ] nu::config_files: read_vendor_autoload_files() src\config_files.rs:234:9
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: read_vendor_autoload_files: C:\ProgramData\nushell\vendor\autoload
2024-06-24 09:23:21.399 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\01_get-weather.nu"
2024-06-24 09:23:21.675 PM [WARN ] nu::config_files: AutoLoading: "C:\\ProgramData\\nushell\\vendor\\autoload\\02_temp.nu"
2024-06-24 09:23:21.817 PM [WARN ] nu_cli::repl: Terminal doesn't support use_kitty_protocol config
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-25 18:31:54 -07:00
Devyn Cairns
91d44f15c1
Allow plugins to report their own version and store it in the registry (#12883)
# Description

This allows plugins to report their version (and potentially other
metadata in the future). The version is shown in `plugin list` and in
`version`.

The metadata is stored in the registry file, and reflects whatever was
retrieved on `plugin add`, not necessarily the running binary. This can
help you to diagnose if there's some kind of mismatch with what you
expect. We could potentially use this functionality to show a warning or
error if a plugin being run does not have the same version as what was
in the cache file, suggesting `plugin add` be run again, but I haven't
done that at this point.

It is optional, and it requires the plugin author to make some code
changes if they want to provide it, since I can't automatically
determine the version of the calling crate or anything tricky like that
to do it.

Example:

```
> plugin list | select name version is_running pid
╭───┬────────────────┬─────────┬────────────┬─────╮
│ # │      name      │ version │ is_running │ pid │
├───┼────────────────┼─────────┼────────────┼─────┤
│ 0 │ example        │ 0.93.1  │ false      │     │
│ 1 │ gstat          │ 0.93.1  │ false      │     │
│ 2 │ inc            │ 0.93.1  │ false      │     │
│ 3 │ python_example │ 0.1.0   │ false      │     │
╰───┴────────────────┴─────────┴────────────┴─────╯
```

cc @maxim-uvarov (he asked for it)

# User-Facing Changes

- `plugin list` gets a `version` column
- `version` shows plugin versions when available
- plugin authors *should* add `fn metadata()` to their `impl Plugin`,
but don't have to

# Tests + Formatting

Tested the low level stuff and also the `plugin list` column.

# After Submitting
- [ ] update plugin guide docs
- [ ] update plugin protocol docs (`Metadata` call & response)
- [ ] update plugin template (`fn metadata()` should be easy)
- [ ] release notes
2024-06-21 06:27:09 -05:00
Devyn Cairns
12991cd36f
Change the error style during tests to plain (#13061)
# Description

This fixes issues with trying to run the tests with a terminal that is
small enough to cause errors to wrap around, or in cases where the test
environment might produce strings that are reasonably expected to wrap
around anyway. "Fancy" errors are too fancy for tests to work
predictably 😉

cc @abusch

# User-Facing Changes

- Added `--error-style` option for use with `--commands` (like
`--table-mode`)

# Tests + Formatting

Surprisingly, all of the tests pass, including in small windows! I only
had to make one change to a test for `error make` which was looking for
the box drawing characters miette uses to determine whether the span
label was showing up - but the plain error style output is even better
and easier to match on, so this test is actually more specific now.
2024-06-18 21:37:24 -07:00
Darren Schroeder
0372e8c53c
add $nu.data-dir for completions and $nu.cache-dir for other uses (#13122)
# Description

This PR is an attempt to add a standard location for people to put
completions in. I saw this topic come up again recently and IIRC we
decided to create a standard location. I used the dirs-next crate to
dictate where these locations are. I know some people won't like that
but at least this gets the ball rolling in a direction that has a
standard directory.

This is what the default NU_LIB_DIRS looks like now in the
default_env.nu. It should also be like this when starting nushell with
`nu -n`
```nushell
$env.NU_LIB_DIRS = [
    ($nu.default-config-dir | path join 'scripts') # add <nushell-config-dir>/scripts
    ($nu.data-dir | path join 'completions') # default home for nushell completions
]
```

I also added these default folders to the `$nu` variable so now there is
`$nu.data-path` and `$nu.cache-path`.

## Data Dir Default

![image](https://github.com/nushell/nushell/assets/343840/aeeb7cd6-17b4-43e8-bb6f-986a0c7fce23)

While I was in there, I also decided to add a cache dir

## Cache Dir Default

![image](https://github.com/nushell/nushell/assets/343840/87dead66-4911-4f67-bfb2-acb16f386674)

### This is what the default looks like in Ubuntu.

![image](https://github.com/nushell/nushell/assets/343840/bca8eae8-8c18-47e8-b64f-3efe34f0004f)

### This is what it looks like with XDG_CACHE_HOME and XDG_DATA_HOME
overridden
```nushell
XDG_DATA_HOME=/tmp/data_home XDG_CACHE_HOME=/tmp/cache_home cargo r
```

![image](https://github.com/nushell/nushell/assets/343840/fae86d50-9821-41f1-868e-3814eca3730b)

### This is what the defaults look like in Windows (username scrubbed to
protect the innocent)

![image](https://github.com/nushell/nushell/assets/343840/3ebdb5cd-0150-448c-aff5-c57053e4788a)

How my NU_LIB_DIRS is set in the images above
```nushell
$env.NU_LIB_DIRS = [
    ($nu.default-config-dir | path join 'scripts') # add <nushell-config-dir>/scripts
    '/Users/fdncred/src/nu_scripts'
    ($nu.config-path | path dirname)
    ($nu.data-dir | path join 'completions') # default home for nushell completions
]
```

Let the debate begin.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-11 15:10:31 -04:00
Devyn Cairns
a3bc85bb5f
Add options for filtering the log output from nu (#13044)
# Description

Add `--log-include` and `--log-exclude` options to filter the log output
from `nu` to specific module prefixes. For example,

```nushell
nu --log-level trace --log-exclude '[nu_parser]'
```

This avoids having to scan through parser spam when trying to debug
something else at `TRACE` level, and should make it feel much more
reasonable to add logging, particularly at `TRACE` level, to various
places in the codebase. It can also be used to debug non-Nushell crates
that support the Rust logging infrastructure, as many do.

You can also include a specific module instead of excluding the parser
log output:

```nushell
nu --log-level trace --log-include '[nu_plugin]'
```

Pinging #13041 for reference, but hesitant to outright say that this
closes that. I think it address that concern though. I've also struggled
with debugging plugin stuff with all of the other output, so this will
really help me there.

# User-Facing Changes

- New `nu` option: `--log-include`
- New `nu` option: `--log-exclude`
2024-06-05 16:42:55 +08:00
Darren Schroeder
7d11c28eea
Revert "Remove std::env::set_current_dir() call from EngineState::merge_env()" (#12954)
Reverts nushell/nushell#12922
2024-05-24 11:09:59 -05:00
YizhePKU
6c649809d3
Rewrite run_external.rs (#12921)
This PR is a complete rewrite of `run_external.rs`. The main goal of the
rewrite is improving readability, but it also fixes some bugs related to
argument handling and the PATH variable (fixes
https://github.com/nushell/nushell/issues/6011).

I'll discuss some technical details to make reviewing easier.

## Argument handling

Quoting arguments for external commands is hard. Like, *really* hard.
We've had more than a dozen issues and PRs dedicated to quoting
arguments (see Appendix) but the current implementation is still buggy.

Here's a demonstration of the buggy behavior:

```nu
let foo = "'bar'"
^touch $foo            # This creates a file named `bar`, but it should be `'bar'`
^touch ...[ "'bar'" ]  # Same
```

I'll describe how this PR deals with argument handling.

First, we'll introduce the concept of **bare strings**. Bare strings are
**string literals** that are either **unquoted** or **quoted by
backticks** [^1]. Strings within a list literal are NOT considered bare
strings, even if they are unquoted or quoted by backticks.

When a bare string is used as an argument to external process, we need
to perform tilde-expansion, glob-expansion, and inner-quotes-removal, in
that order. "Inner-quotes-removal" means transforming from
`--option="value"` into `--option=value`.

## `.bat` files and CMD built-ins

On Windows, `.bat` files and `.cmd` files are considered executable, but
they need `CMD.exe` as the interpreter. The Rust standard library
supports running `.bat` files directly and will spawn `CMD.exe` under
the hood (see
[documentation](https://doc.rust-lang.org/std/process/index.html#windows-argument-splitting)).
However, other extensions are not supported [^2].

Nushell also supports a selected number of CMD built-ins. The problem
with CMD is that it uses a different set of quoting rules. Correctly
quoting for CMD requires using
[Command::raw_arg()](https://doc.rust-lang.org/std/os/windows/process/trait.CommandExt.html#tymethod.raw_arg)
and manually quoting CMD special characters, on top of quoting from the
Nushell side. ~~I decided that this is too complex and chose to reject
special characters in CMD built-ins instead [^3]. Hopefully this will
not affact real-world use cases.~~ I've implemented escaping that works
reasonably well.

## `which-support` feature

The `which` crate is now a hard dependency of `nu-command`, making the
`which-support` feature essentially useless. The `which` crate is
already a hard dependency of `nu-cli`, and we should consider removing
the `which-support` feature entirely.

## Appendix

Here's a list of quoting-related issues and PRs in rough chronological
order.

* https://github.com/nushell/nushell/issues/4609
* https://github.com/nushell/nushell/issues/4631
* https://github.com/nushell/nushell/issues/4601
  * https://github.com/nushell/nushell/pull/5846
* https://github.com/nushell/nushell/issues/5978
  * https://github.com/nushell/nushell/pull/6014
* https://github.com/nushell/nushell/issues/6154
  * https://github.com/nushell/nushell/pull/6161
* https://github.com/nushell/nushell/issues/6399
  * https://github.com/nushell/nushell/pull/6420
  * https://github.com/nushell/nushell/pull/6426
* https://github.com/nushell/nushell/issues/6465
* https://github.com/nushell/nushell/issues/6559
  * https://github.com/nushell/nushell/pull/6560

[^1]: The idea that backtick-quoted strings act like bare strings was
introduced by Kubouch and briefly mentioned in [the language
reference](https://www.nushell.sh/lang-guide/chapters/strings_and_text.html#backtick-quotes).

[^2]: The documentation also said "running .bat scripts in this way may
be removed in the future and so should not be relied upon", which is
another reason to move away from this. But again, quoting for CMD is
hard.

[^3]: If anyone wants to try, the best resource I found on the topic is
[this](https://daviddeley.com/autohotkey/parameters/parameters.htm).
2024-05-23 02:05:27 +00:00
YizhePKU
7ede90cba5
Remove std::env::set_current_dir() call from EngineState::merge_env() (#12922)
As discussed in https://github.com/nushell/nushell/pull/12749, we no
longer need to call `std::env::set_current_dir()` to sync `$env.PWD`
with the actual working directory. This PR removes the call from
`EngineState::merge_env()`.
2024-05-22 19:58:27 +03:00
Ian Manske
905e3d0715
Remove dataframes crate and feature (#12889)
# Description
Removes the old `nu-cmd-dataframe` crate in favor of the polars plugin.
As such, this PR also removes the `dataframe` feature, related CI, and
full releases of nushell.
2024-05-20 17:22:08 +00:00
Ian Manske
baeba19b22
Make get_full_help take &dyn Command (#12903)
# Description
Changes `get_full_help` to take a `&dyn Command` instead of multiple
arguments (`&Signature`, `&Examples` `is_parser_keyword`). All of these
arguments can be gathered from a `Command`, so there is no need to pass
the pieces to `get_full_help`.

This PR also fixes an issue where the search terms are not shown if
`--help` is used on a command.
2024-05-19 19:56:33 +02:00
Ian Manske
cc9f41e553
Use CommandType in more places (#12832)
# Description
Kind of a vague title, but this PR does two main things:
1. Rather than overriding functions like `Command::is_parser_keyword`,
this PR instead changes commands to override `Command::command_type`.
The `CommandType` returned by `Command::command_type` is then used to
automatically determine whether `Command::is_parser_keyword` and the
other `is_{type}` functions should return true. These changes allow us
to remove the `CommandType::Other` case and should also guarantee than
only one of the `is_{type}` functions on `Command` will return true.
2. Uses the new, reworked `Command::command_type` function in the `scope
commands` and `which` commands.


# User-Facing Changes
- Breaking change for `scope commands`: multiple columns (`is_builtin`,
`is_keyword`, `is_plugin`, etc.) have been merged into the `type`
column.
- Breaking change: the `which` command can now report `plugin` or
`keyword` instead of `built-in` in the `type` column. It may also now
report `external` instead of `custom` in the `type` column for known
`extern`s.
2024-05-18 23:37:31 +00:00
Ian Manske
6fd854ed9f
Replace ExternalStream with new ByteStream type (#12774)
# Description
This PR introduces a `ByteStream` type which is a `Read`-able stream of
bytes. Internally, it has an enum over three different byte stream
sources:
```rust
pub enum ByteStreamSource {
    Read(Box<dyn Read + Send + 'static>),
    File(File),
    Child(ChildProcess),
}
```

This is in comparison to the current `RawStream` type, which is an
`Iterator<Item = Vec<u8>>` and has to allocate for each read chunk.

Currently, `PipelineData::ExternalStream` serves a weird dual role where
it is either external command output or a wrapper around `RawStream`.
`ByteStream` makes this distinction more clear (via `ByteStreamSource`)
and replaces `PipelineData::ExternalStream` in this PR:
```rust
pub enum PipelineData {
    Empty,
    Value(Value, Option<PipelineMetadata>),
    ListStream(ListStream, Option<PipelineMetadata>),
    ByteStream(ByteStream, Option<PipelineMetadata>),
}
```

The PR is relatively large, but a decent amount of it is just repetitive
changes.

This PR fixes #7017, fixes #10763, and fixes #12369.

This PR also improves performance when piping external commands. Nushell
should, in most cases, have competitive pipeline throughput compared to,
e.g., bash.
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| -------------------------------------------------- | -------------:|
------------:| -----------:|
| `throughput \| rg 'x'` | 3059 | 3744 | 3739 |
| `throughput \| nu --testbin relay o> /dev/null` | 3508 | 8087 | 8136 |

# User-Facing Changes
- This is a breaking change for the plugin communication protocol,
because the `ExternalStreamInfo` was replaced with `ByteStreamInfo`.
Plugins now only have to deal with a single input stream, as opposed to
the previous three streams: stdout, stderr, and exit code.
- The output of `describe` has been changed for external/byte streams.
- Temporary breaking change: `bytes starts-with` no longer works with
byte streams. This is to keep the PR smaller, and `bytes ends-with`
already does not work on byte streams.
- If a process core dumped, then instead of having a `Value::Error` in
the `exit_code` column of the output returned from `complete`, it now is
a `Value::Int` with the negation of the signal number.

# After Submitting
- Update docs and book as necessary
- Release notes (e.g., plugin protocol changes)
- Adapt/convert commands to work with byte streams (high priority is
`str length`, `bytes starts-with`, and maybe `bytes ends-with`).
- Refactor the `tee` code, Devyn has already done some work on this.

---------

Co-authored-by: Devyn Cairns <devyn.cairns@gmail.com>
2024-05-16 07:11:18 -07:00
Darren Schroeder
defed3001d
make it clearer what is being loaded with --log-level info (#12875)
# Description

A common question we get is what config files are loaded when and with
what parameters. It's for this reason that I wrote [this
gist](https://gist.github.com/fdncred/b87b784f04984dc31a150baed9ad2447).
Another way to figure this out is to use `nu --log-level info`. This
will show some performance timings but will also show what is being
loaded when. For the most part the `[INFO]` lines show the performance
timings and the `[WARN]` lines show the files.

This PR tries to make things a little bit clearer when using the
`--log-level info` parameter.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-05-15 09:44:09 -05:00
francesco-gaglione
c4dca5fe03
Merged tests to produce a single binary (#12826)
This PR should close #7147 

# Description
Merged src/tests into /tests to produce a single binary.

![image](https://github.com/nushell/nushell/assets/94604837/84726469-d447-4619-b6d1-2d1415d0f42e)

# User-Facing Changes
No user facing changes

# Tests + Formatting
Moved tests. Tollkit check pr pass.

# After Submitting

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-13 13:37:53 +00:00
Ian Manske
72d3860d05
Refactor the CLI code a bit (#12782)
# Description
Refactors the code in `nu-cli`, `main.rs`, `run.rs`, and few others.
Namely, I added `EngineState::generate_nu_constant` function to
eliminate some duplicate code. Otherwise, I changed a bunch of areas to
return errors instead of calling `std::process::exit`.

# User-Facing Changes
Should be none.
2024-05-10 07:29:27 +08:00
Ian Manske
7271ad7909
Pass Stack ref to Completer::fetch (#12783)
# Description
Adds an additional `&Stack` parameter to `Completer::fetch` so that the
completers don't have to store a `Stack` themselves. I also removed
unnecessary `EngineState`s from the completers, since the same
`EngineState` is available in the `working_set.permanent_state` also
passed to `Completer::fetch`.
2024-05-09 13:38:24 +08:00
YizhePKU
7a86b98f61
Migrate to a new PWD API (part 2) (#12749)
Refer to #12603 for part 1.

We need to be careful when migrating to the new API, because the new API
has slightly different semantics (PWD can contain symlinks). This PR
handles the "obviously safe" part of the migrations. Namely, it handles
two specific use cases:

* Passing PWD into `canonicalize_with()`
* Passing PWD into `EngineState::merge_env()`

The first case is safe because symlinks are canonicalized away. The
second case is safe because `EngineState::merge_env()` only uses PWD to
call `std::env::set_current_dir()`, which shouldn't affact Nushell. The
commit message contains detailed stats on the updated files.

Because these migrations touch a lot of files, I want to keep these PRs
small to avoid merge conflicts.
2024-05-07 18:17:49 +03:00
Wind
97fc190cc5
allow raw string to be used inside subexpression, list, and closure (#12776)
# Description
Fixes: #12744

This pr is moving raw string lex logic into `lex_item` function, so we
can use raw string inside subexpression, list, closure.
```nushell
> [r#'abc'#]
╭───┬─────╮
│ 0 │ abc │
╰───┴─────╯
> (r#'abc'#)
abc
> do {r#'aa'#}
aa
```

# Tests + Formatting
Done

# After Submitting
NaN
2024-05-06 15:53:58 -05:00
YizhePKU
bdb6daa4b5
Migrate to a new PWD API (#12603)
This is the first PR towards migrating to a new `$env.PWD` API that
returns potentially un-canonicalized paths. Refer to PR #12515 for
motivations.

## New API: `EngineState::cwd()`

The goal of the new API is to cover both parse-time and runtime use
case, and avoid unintentional misuse. It takes an `Option<Stack>` as
argument, which if supplied, will search for `$env.PWD` on the stack in
additional to the engine state. I think with this design, there's less
confusion over parse-time and runtime environments. If you have access
to a stack, just supply it; otherwise supply `None`.

## Deprecation of other PWD-related APIs

Other APIs are re-implemented using `EngineState::cwd()` and properly
documented. They're marked deprecated, but their behavior is unchanged.
Unused APIs are deleted, and code that accesses `$env.PWD` directly
without using an API is rewritten.

Deprecated APIs:

* `EngineState::current_work_dir()`
* `StateWorkingSet::get_cwd()`
* `env::current_dir()`
* `env::current_dir_str()`
* `env::current_dir_const()`
* `env::current_dir_str_const()`

Other changes:

* `EngineState::get_cwd()` (deleted)
* `StateWorkingSet::list_env()` (deleted)
* `repl::do_run_cmd()` (rewritten with `env::current_dir_str()`)

## `cd` and `pwd` now use logical paths by default

This pulls the changes from PR #12515. It's currently somewhat broken
because using non-canonicalized paths exposed a bug in our path
normalization logic (Issue #12602). Once that is fixed, this should
work.

## Future plans

This PR needs some tests. Which test helpers should I use, and where
should I put those tests?

I noticed that unquoted paths are expanded within `eval_filepath()` and
`eval_directory()` before they even reach the `cd` command. This means
every paths is expanded twice. Is this intended?

Once this PR lands, the plan is to review all usages of the deprecated
APIs and migrate them to `EngineState::cwd()`. In the meantime, these
usages are annotated with `#[allow(deprecated)]` to avoid breaking CI.

---------

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2024-05-03 14:33:09 +03:00
Darren Schroeder
8ed0d84d6a
add raw-string literal support (#9956)
# Description

This PR adds raw string support by using `r#` at the beginning of single
quoted strings and `#` at the end.

Notice that escapes do not process, even within single quotes,
parentheses don't mean anything, $variables don't mean anything. It's
just a string.
```nushell
❯ echo r#'one\ntwo (blah) ($var)'#
one\ntwo (blah) ($var)
```
Notice how they work without `echo` or `print` and how they work without
carriage returns.
```nushell
❯ r#'adsfa'#
adsfa
❯ r##"asdfa'@qpejq'##
asdfa'@qpejq
❯ r#'asdfasdfasf
∙ foqwejfqo@'23rfjqf'#
```
They also have a special configurable color in the repl. (use single
quotes though)

![image](https://github.com/nushell/nushell/assets/343840/8780e21d-de4c-45b3-9880-2425f5fe10ef)

They should work like rust raw literals and allow `r##`, `r###`,
`r####`, etc, to help with having one or many `#`'s in the middle of
your raw-string.

They should work with `let` as well.

```nushell
r#'some\nraw\nstring'# | str upcase
```

closes https://github.com/nushell/nushell/issues/5091
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect -A clippy::result_large_err` to check that
you're using the standard code style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: WindSoilder <WindSoilder@outlook.com>
Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-05-02 09:36:37 -04:00
Devyn Cairns
0c4d5330ee
Split the plugin crate (#12563)
# Description

This breaks `nu-plugin` up into four crates:

- `nu-plugin-protocol`: just the type definitions for the protocol, no
I/O. If someone wanted to wire up something more bare metal, maybe for
async I/O, they could use this.
- `nu-plugin-core`: the shared stuff between engine/plugin. Less stable
interface.
- `nu-plugin-engine`: everything required for the engine to talk to
plugins. Less stable interface.
- `nu-plugin`: everything required for the plugin to talk to the engine,
what plugin developers use. Should be the most stable interface.

No changes are made to the interface exposed by `nu-plugin` - it should
all still be there. Re-exports from `nu-plugin-protocol` or
`nu-plugin-core` are used as required. Plugins shouldn't ever have to
use those crates directly.

This should be somewhat faster to compile as `nu-plugin-engine` and
`nu-plugin` can compile in parallel, and the engine doesn't need
`nu-plugin` and plugins don't need `nu-plugin-engine` (except for test
support), so that should reduce what needs to be compiled too.

The only significant change here other than splitting stuff up was to
break the `source` out of `PluginCustomValue` and create a new
`PluginCustomValueWithSource` type that contains that instead. One bonus
of that is we get rid of the option and it's now more type-safe, but it
also means that the logic for that stuff (actually running the plugin
for custom value ops) can live entirely within the `nu-plugin-engine`
crate.

# User-Facing Changes
- New crates.
- Added `local-socket` feature for `nu` to try to make it possible to
compile without that support if needed.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-27 12:08:12 -05:00
Ian Manske
f234a0ea33
each signature fix (#12666)
# Description
Removes the second `Int` parameter from the closure in the signature of
`each`. This parameter doesn't exist / isn't supported.
2024-04-25 22:47:30 +02:00
Ian Manske
5b0546cfce
Remove deprecated flags on commandline (#12658)
# Description
Removes the `commandline` flags and API that was deprecated in 0.91.0
with #11877.

# User-Facing Changes
Users need to migrate to the new `commandline` subcommands introduced in
0.91.0.
2024-04-25 14:16:42 +00:00
Devyn Cairns
25cbcb511d
Rename plugin cache file ⇒ plugin registry file (#12634)
# Description
So far this seems like the winner of my poll on what the name should be.
I'll take this off draft once the poll expires, if this is indeed the
winner.
2024-04-24 17:40:39 -05:00
Devyn Cairns
1f4131532d
Deprecate register and add plugin use (#12607)
# Description

Adds a new keyword, `plugin use`. Unlike `register`, this merely loads
the signatures from the plugin cache file. The file is configurable with
the `--plugin-config` option either to `nu` or to `plugin use` itself,
just like the other `plugin` family of commands. At the REPL, one might
do this to replace `register`:

```nushell
> plugin add ~/.cargo/bin/nu_plugin_foo
> plugin use foo
```

This will not work in a script, because `plugin use` is a keyword and
`plugin add` does not evaluate at parse time (intentionally). This means
we no longer run random binaries during parse.

The `--plugins` option has been added to allow running `nu` with certain
plugins in one step. This is used especially for the `nu_with_plugins!`
test macro, but I'd imagine is generally useful. The only weird quirk is
that it has to be a list, and we don't really do this for any of our
other CLI args at the moment.

`register` now prints a deprecation parse warning.

This should fix #11923, as we now have a complete alternative to
`register`.

# User-Facing Changes

- Add `plugin use` command
- Deprecate `register`
- Add `--plugins` option to `nu` to replace a common use of `register`

# Tests + Formatting

I think I've tested it thoroughly enough and every existing test passes.
Testing nu CLI options and alternate config files is a little hairy and
I wish there were some more generic helpers for this, so this will go on
my TODO list for refactoring.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Update plugins sections of book
- [ ] Release notes
2024-04-23 06:37:50 -05:00
Wind
b0acc1d890
Avoid panic when pipe a variable to a custom command which have recursive call (#12491)
# Description
Fixes: #11351

And comment here is also fixed:
https://github.com/nushell/nushell/issues/11351#issuecomment-1996191537

The panic can happened if we pipe a variable to a custom command which
recursively called itself inside another block.

TBH, I think I figure out how it works to panic, but I'm not sure if
there is a potention issue if nushell don't mutate a block in such case.

# User-Facing Changes
Nan

# Tests + Formatting
Done

# After Submitting
Done

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-04-23 06:10:35 +08:00
Devyn Cairns
2595f31541
Overhaul the plugin cache file with a new msgpack+brotli format (#12579)
# Description

- Plugin signatures are now saved to `plugin.msgpackz`, which is
brotli-compressed MessagePack.
- The file is updated incrementally, rather than writing all plugin
commands in the engine every time.
- The file always contains the result of the `Signature` call to the
plugin, even if commands were removed.
- Invalid data for a particular plugin just causes an error to be
reported, but the rest of the plugins can still be parsed

# User-Facing Changes

- The plugin file has a different filename, and it's not a nushell
script.
- The default `plugin.nu` file will be automatically migrated the first
time, but not other plugin config files.
- We don't currently provide any utilities that could help edit this
file, beyond `plugin add` and `plugin rm`
  - `from msgpackz`, `to msgpackz` could also help
- New commands: `plugin add`, `plugin rm`

# Tests + Formatting

Tests added for the format and for the invalid handling.

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

- [ ] Check for documentation changes
- [ ] Definitely needs release notes
2024-04-21 07:36:26 -05:00
Wind
cf8fcef9bf
set the type of default NU_LIB_DIRS and NU_PLUGIN_DIRS to list<string> (#12573)
# Description
Fix: #12489

I believe the issue it's because the default value of `NU_LIB_DIRS` and
`NU_PLUGIN_DIRS` is a string, but it should be a list.

So if users don't set up these values in `env.nu`, we will get a
problem.
2024-04-20 10:04:41 -05:00
YizhePKU
6d2cb4382a
Fix circular source causing Nushell to crash (#12262)
# Description

EngineState now tracks the script currently running, instead of the
parent directory of the script. This also provides an easy way to expose
the current running script to the user (Issue #12195).

Similarly, StateWorkingSet now tracks scripts instead of directories.
`parsed_module_files` and `currently_parsed_pwd` are merged into one
variable, `scripts`, which acts like a stack for tracking the current
running script (which is on the top of the stack).

Circular import check is added for `source` operations, in addition to
module import. A simple testcase is added for circular source.

<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->


<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

It shouldn't have any user facing changes.
2024-04-19 09:38:08 +03:00
Stefan Holderbach
c9e9b138eb
Improve with-env robustness (#12523)
# Description
Work for #7149

- **Error `with-env` given uneven count in list form**
- **Fix `with-env` `CantConvert` to record**
- **Error `with-env` when given protected env vars**
- **Deprecate list/table input of vars to `with-env`**
- **Remove examples for deprecated input**

# User-Facing Changes

## Deprecation of the following forms

```
> with-env [MYENV "my env value"] { $env.MYENV }
my env value

> with-env [X Y W Z] { $env.X }
Y

> with-env [[X W]; [Y Z]] { $env.W }
Z
```

## recommended standardized form

```
# Set by key-value record
> with-env {X: "Y", W: "Z"} { [$env.X $env.W] }
╭───┬───╮
│ 0 │ Y │
│ 1 │ Z │
╰───┴───╯
```

## (Side effect) Repeated definitions in an env shorthand are now
disallowed

```
> FOO=bar FOO=baz $env
Error: nu:🐚:column_defined_twice

  × Record field or table column used twice: FOO
   ╭─[entry #1:1:1]
 1 │ FOO=bar FOO=baz $env
   · ─┬─     ─┬─
   ·  │       ╰── field redefined here
   ·  ╰── field first defined here
   ╰────
```
2024-04-16 19:08:58 +08:00
Marc Schreiber
50b2dac8d7
Set Up Config in LSP Mode (#12454)
# Description

When starting the LSP server, the configuration file and environment
file are used to configure the LSP engine unless --no-config-file is
provided.

This PR provides an improvement that is related to #10794 

CC: @fdncred
2024-04-14 07:32:30 -05:00
Texas Toland
83674909f1
Lex whitespace in input-output types. (#12339)
# Description
Fixes #12264.

# User-Facing Changes


Multiple input-output types can break across lines like command params.

# Tests + Formatting

 E2E parser tests
2024-04-10 16:28:54 +02:00
Wind
18ddf95d44
Force timeit to not capture stdout (#12465)
# Description
Fixes:  #11996

After this change `let t = timeit ^ls` will list current directory to
stdout.
```
❯ let t = timeit ^ls
CODE_OF_CONDUCT.md      Cargo.lock              Cross.toml              README.md               aaa                     benches                 devdocs                 here11                  scripts                 target                  toolkit.nu              wix
CONTRIBUTING.md         Cargo.toml              LICENSE                 a.txt                   assets                  crates                  docker                  rust-toolchain.toml     src                     tests                   typos.toml
```

If user don't want such behavior, he can redirect the stdout to `std
null-stream` easily
```
> use std
> let t = timeit { ^ls o> (std null-device) }
```

# User-Facing Changes
NaN

# Tests + Formatting
Done

# After Submitting
Nan

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-04-10 13:31:29 +00:00
Devyn Cairns
d735607ac8
Isolate tests from user config (#12437)
# Description
This is an attempt to isolate the unit tests from whatever might be in
the user's config. If the
user's config is broken in some way or incompatible with this version
(for example, especially if
there are plugins that aren't built for this version), tests can
spuriously fail.

This makes tests more reliably pass the same way they would on CI even
if the user has config, and
should also make them run faster.

I think this is _good enough_, but I still think we should have a
specific config dir env variable for nushell specifically (rather than
having to use `XDG_CONFIG_HOME`, which would mess with other things) and
then we can just have `nu-test-support` set that to a temporary dir
containing the shipped default config files.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-10 06:27:46 +08:00
Devyn Cairns
14b0ff3f05
Add --no-newline option to nu (#12410)
# Description
I have `nu` set as my shell in my editor, which allows me to easily pipe
selections of text to things like `str pascal-case` or even more complex
string operation pipelines, which I find super handy. However, the only
annoying thing is that I pretty much always have to add `| print -n` at
the end, because `nu` adds a newline when it prints the resulting value.

This adds a `--no-newline` option to stop that from happening, and then
you don't need to pipe to `print -n` anymore, you can just have your
shell command for your editor contain that flag.

# User-Facing Changes
- Add `--no-newline` command line option

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
2024-04-09 10:04:00 -04:00
Yash Thakur
db1dccc762
Don't check if stderr empty in test_xdg_config_symlink (#12435)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes a problem introduced in
https://github.com/nushell/nushell/pull/12420 where one of the config
path tests (`test_xdg_config_symlink`) fails if, on MacOS, a developer
already has config files in the default location for that platform.

This happened because the test is making sure the
`xdg_config_home_invalid` error isn't reported, but to do that, it
asserts that stderr is empty, which it is not if in the case mentioned
above, because Nushell warns that the default location
(`~/.config`/`~/Library/Application Support`) is not empty but
`XDG_CONFIG_HOME` is empty.

If someone with a Mac could test this, that'd be great.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

None, this is for contributors.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-06 18:01:16 -04:00
Ian Manske
03667bdf8c
Fix merging child stack into parent (#12426)
# Description
Fixes #12423 where changes to mutable variables are not properly
persisted after a REPL entry.
2024-04-06 15:03:22 +00:00
Yash Thakur
eb36dbb091
Fix #12416 by canonicalizing XDG_CONFIG_HOME before comparing to config_dir() (#12420)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes #12416. Turns out that in src/main.rs, `XDG_CONFIG_HOME` wasn't
being canonicalized before being compared to `nu_path::config_dir()` to
check if `XDG_CONFIG_HOME` was set to an invalid value. This has been
rectified now.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

Setting `XDG_CONFIG_HOME` to a symlink should work now.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

I manually tested it and the error has disappeared:

New behavior (this branch):

![image](https://github.com/nushell/nushell/assets/45539777/062d1cc5-551c-431c-b138-d3da8de018bd)

Old behavior (main):

![image](https://github.com/nushell/nushell/assets/45539777/22c4b5a3-3fd0-4ab6-9cf0-ae25488645ba)

Thanks to a pointer from Devyn, I've now added tests to make sure the
`xdg_config_home_invalid` error doesn't pop up when `XDG_CONFIG_HOME` is
a symlink (and does when it's actually invalid).

Turns out two of the tests in `test_config_path` tried modifying
`XDG_CONFIG_HOME` using `playground.with_env` but used `nu!`, so the
subprocess didn't actually use the modified value of `XDG_CONFIG_HOME`.
When I added them, I was unaware that the `.with_env` didn't actually
modify the environment. This has now been rectified.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-04-06 09:09:03 -05:00
Darren Schroeder
e97368433b
add a few more logging statements for debugging startup (#12316)
# Description

This PR adds a few more `trace!()` and `perf()` statements that allowed
a deeper understanding of the nushell startup process when used with `nu
-n --no-std-lib --log-level trace`.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-03-28 11:27:12 -05:00
Stefan Holderbach
dfbbacfdf8
Deduplicate nix dependency versions (#12307)
# Description
Now we only use `nix 0.28.0`

Achieved by
- updating `ctrlc` to `3.4.4`
- updating `wl-clipboard-rs` to `0.8.1`
- update our own dependency on `nix` from `0.27` to `0.28`
  - required fixing uses of `nix::unistd::{tcgetpgrp,tcsetpgrp}`
  - now requires an I/O safe file descriptor
  - fake one pointing to `libc::STDIN_FILENO` (we were only accessing
`0` previously, dito for fish)


# User-Facing Changes
Better compile times and less to download as source dependencies
2024-03-27 16:43:37 +01:00
Ian Manske
c747ec75c9
Add command_prelude module (#12291)
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
    record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
    ShellError, Signature, Span, Type, Value,
};
```

This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
    ast::{Call, CellPath},
    engine::{Command, EngineState, Stack},
    record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
    PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```

This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
2024-03-26 21:17:30 +00:00
Auca Coyan
b3721a24fa
🐛 remove 3 backticks messing the hover (#12273)
# Description

The hover was bugged with 3 backticks. I don't understand how it worked
before, but this apparently now works correctly on my machine. This is
really puzzling. My next step is to make a test to assert this will
break a little less. I fixed it 3 times in the past

# Tests + Formatting

Added a test to be sure this doesn't breaks again 😄 (at least from
nushell/nushell side)
2024-03-24 14:15:01 -05:00
Devyn Cairns
ff41cf91ef
Misc doc fixes (#12266)
# Description

Just a bunch of miscellaneous fixes to the Rust documentation that I
found recently while doing
a pass on some things.

# User-Facing Changes
None
2024-03-23 07:26:08 -05:00
dannou812
8237d15683
to json -r not removing whitespaces fix (#11948)
fixes #11900  

# Description
Use `serde_json` instead.

# User-Facing Changes
The problem described in the issue now no longer persists.

No whitespace in the output of `to json --raw`
Output of unicode escape changed to consistent `\uffff`

# Tests + Formatting
I corrected all Tests that were affected by this change.
2024-03-20 22:14:31 +01:00