13d5a15f75
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR makes two changes related to [run-time pipeline input type checking](https://github.com/nushell/nushell/pull/14741): 1. The check which bypasses type checking for commands with only `Type::Nothing` input types has been expanded to work with commands with multiple `Type::Nothing` inputs for different outputs. For example, `ast` has three input/output type pairs, but all of the inputs are `Type::Nothing`: ``` ╭───┬─────────┬────────╮ │ # │ input │ output │ ├───┼─────────┼────────┤ │ 0 │ nothing │ table │ │ 1 │ nothing │ record │ │ 2 │ nothing │ string │ ╰───┴─────────┴────────╯ ``` Before this PR, passing a value (which would otherwise be ignored) to `ast` caused a run-time type error: ``` Error: nu:🐚:only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:6] 1 │ echo 123 | ast -j -f "hi" · ─┬─ ─┬─ · │ ╰── only nothing, nothing, and nothing input data is supported · ╰── input type: int ╰──── ``` After this PR, no error is raised. This doesn't really matter for `ast` (the only other built-in command with a similar input/output type signature is `cal`), but it's more logically consistent. 2. Bypasses input type-checking (parse-time ***and*** run-time) for some (not all, see below) commands which have both a `Type::Nothing` input and some other non-nothing `Type` input. This is accomplished by adding a `Type::Any` input with the same output as the corresponding `Type::Nothing` input/output pair. This is necessary because some commands are intended to operate on an argument with empty pipeline input, or operate on an empty pipeline input with no argument. This causes issues when a value is implicitly passed to one of these commands. I [discovered this issue](https://discord.com/channels/601130461678272522/615962413203718156/1329945784346611712) when working with an example where the `open` command is used in `sort-by` closure: ```nushell ls | sort-by { open -r $in.name | lines | length } ``` Before this PR (but after the run-time input type checking PR), this error is raised: ``` Error: nu:🐚:only_supports_this_input_type × Input type not supported. ╭─[entry #1:1:1] 1 │ ls | sort-by { open -r $in.name | lines | length } · ─┬ ──┬─ · │ ╰── only nothing and string input data is supported · ╰── input type: record<name: string, type: string, size: filesize, modified: date> ╰──── ``` While this error is technically correct, we don't actually want to return an error here since `open` ignores its pipeline input when an argument is passed. This would be a parse-time error as well if the parser was able to infer that the closure input type was a record, but our type inference isn't that robust currently, so this technically incorrect form snuck by type checking until #14741. However, there are some commands with the same kind of type signature where this behavior is actually desirable. This means we can't just bypass type-checking for any command with a `Type::Nothing` input. These commands operate on true `null` values, rather than ignoring their input. For example, `length` returns `0` when passed a `null` value. It's correct, and even desirable, to throw a run-time error when `length` is passed an unexpected type. For example, a string, which should instead be measured with `str length`: ```nushell ["hello" "world"] | sort-by { length } # => Error: nu:🐚:only_supports_this_input_type # => # => × Input type not supported. # => ╭─[entry #32:1:10] # => 1 │ ["hello" "world"] | sort-by { length } # => · ───┬─── ───┬── # => · │ ╰── only list<any>, binary, and nothing input data is supported # => · ╰── input type: string # => ╰──── ``` We need a more robust way for commands to express how they handle the `Type::Nothing` input case. I think a possible solution here is to allow commands to express that they operate on `PipelineData::Empty`, rather than `Value::Nothing`. Then, a command like `open` could have an empty pipeline input type rather than a `Type::Nothing`, and the parse-time and run-time pipeline input type checks know that `open` will safely ignore an incorrectly typed input. That being said, we have a release coming up and the above solution might take a while to implement, so while unfortunate, bypassing input type-checking for these problematic commands serves as a workaround to avoid breaking changes in the release until a more robust solution is implemented. This PR bypasses input type-checking for the following commands: * `load-env`: can take record of envvars as input or argument * `nu-check`: checks input string or filename argument * `open`: can take filename as input or argument * `polars when`: can be used with input, or can be chained with another `polars when` * `stor insert`: data record can be passed as input or argument * `stor update`: data record can be passed as input or argument * `format date`: `--list` ignores input value * `into datetime`: `--list` ignores input value (also added a `Type::Nothing` input which was missing from this command) These commands have a similar input/output signature to the above commands, but are working as intended: * `cd`: The input/output signature was actually incorrect, `cd` always ignores its input. I fixed this in this PR. * `generate` * `get` * `history import` * `interleave` * `into bool` * `length` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> As a temporary workaround, pipeline input type-checking for the following commands has been bypassed to avoid undesirable run-time input type checking errors which were previously not caught at parse-time: * `open` * `load-env` * `format date` * `into datetime` * `nu-check` * `stor insert` * `stor update` * `polars when` # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> CI became green in the time it took me to type the description 😄 # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> N/A
Nushell
A new type of shell.
Table of Contents
- Status
- Learning About Nu
- Installation
- Configuration
- Philosophy
- Goals
- Officially Supported By
- Contributing
- License
Status
This project has reached a minimum-viable-product level of quality. Many people use it as their daily driver, but it may be unstable for some commands. Nu's design is subject to change as it matures.
Learning About Nu
The Nushell book is the primary source of Nushell documentation. You can find a full list of Nu commands in the book, and we have many examples of using Nu in our cookbook.
We're also active on Discord and Twitter; come and chat with us!
Installation
To quickly install Nu:
# Linux and macOS
brew install nushell
# Windows
winget install nushell
To use Nu in GitHub Action, check setup-nu for more detail.
Detailed installation instructions can be found in the installation chapter of the book. Nu is available via many package managers:
For details about which platforms the Nushell team actively supports, see our platform support policy.
Configuration
The default configurations can be found at sample_config which are the configuration files one gets when they startup Nushell for the first time.
It sets all of the default configuration to run Nushell. From here one can then customize this file for their specific needs.
To see where config.nu is located on your system simply type this command.
$nu.config-path
Please see our book for all of the Nushell documentation.
Philosophy
Nu draws inspiration from projects like PowerShell, functional programming languages, and modern CLI tools. Rather than thinking of files and data as raw streams of text, Nu looks at each input as something with structure. For example, when you list the contents of a directory what you get back is a table of rows, where each row represents an item in that directory. These values can be piped through a series of steps, in a series of commands called a 'pipeline'.
Pipelines
In Unix, it's common to pipe between commands to split up a sophisticated command over multiple steps. Nu takes this a step further and builds heavily on the idea of pipelines. As in the Unix philosophy, Nu allows commands to output to stdout and read from stdin. Additionally, commands can output structured data (you can think of this as a third kind of stream). Commands that work in the pipeline fit into one of three categories:
- Commands that produce a stream (e.g.,
ls) - Commands that filter a stream (e.g.,
where type == "dir") - Commands that consume the output of the pipeline (e.g.,
table)
Commands are separated by the pipe symbol (|) to denote a pipeline flowing left to right.
ls | where type == "dir" | table
# => ╭────┬──────────┬──────┬─────────┬───────────────╮
# => │ # │ name │ type │ size │ modified │
# => ├────┼──────────┼──────┼─────────┼───────────────┤
# => │ 0 │ .cargo │ dir │ 0 B │ 9 minutes ago │
# => │ 1 │ assets │ dir │ 0 B │ 2 weeks ago │
# => │ 2 │ crates │ dir │ 4.0 KiB │ 2 weeks ago │
# => │ 3 │ docker │ dir │ 0 B │ 2 weeks ago │
# => │ 4 │ docs │ dir │ 0 B │ 2 weeks ago │
# => │ 5 │ images │ dir │ 0 B │ 2 weeks ago │
# => │ 6 │ pkg_mgrs │ dir │ 0 B │ 2 weeks ago │
# => │ 7 │ samples │ dir │ 0 B │ 2 weeks ago │
# => │ 8 │ src │ dir │ 4.0 KiB │ 2 weeks ago │
# => │ 9 │ target │ dir │ 0 B │ a day ago │
# => │ 10 │ tests │ dir │ 4.0 KiB │ 2 weeks ago │
# => │ 11 │ wix │ dir │ 0 B │ 2 weeks ago │
# => ╰────┴──────────┴──────┴─────────┴───────────────╯
Because most of the time you'll want to see the output of a pipeline, table is assumed.
We could have also written the above:
ls | where type == "dir"
Being able to use the same commands and compose them differently is an important philosophy in Nu.
For example, we could use the built-in ps command to get a list of the running processes, using the same where as above.
ps | where cpu > 0
# => ╭───┬───────┬───────────┬───────┬───────────┬───────────╮
# => │ # │ pid │ name │ cpu │ mem │ virtual │
# => ├───┼───────┼───────────┼───────┼───────────┼───────────┤
# => │ 0 │ 2240 │ Slack.exe │ 16.40 │ 178.3 MiB │ 232.6 MiB │
# => │ 1 │ 16948 │ Slack.exe │ 16.32 │ 205.0 MiB │ 197.9 MiB │
# => │ 2 │ 17700 │ nu.exe │ 3.77 │ 26.1 MiB │ 8.8 MiB │
# => ╰───┴───────┴───────────┴───────┴───────────┴───────────╯
Opening files
Nu can load file and URL contents as raw text or structured data (if it recognizes the format). For example, you can load a .toml file as structured data and explore it:
open Cargo.toml
# => ╭──────────────────┬────────────────────╮
# => │ bin │ [table 1 row] │
# => │ dependencies │ {record 25 fields} │
# => │ dev-dependencies │ {record 8 fields} │
# => │ features │ {record 10 fields} │
# => │ package │ {record 13 fields} │
# => │ patch │ {record 1 field} │
# => │ profile │ {record 3 fields} │
# => │ target │ {record 3 fields} │
# => │ workspace │ {record 1 field} │
# => ╰──────────────────┴────────────────────╯
We can pipe this into a command that gets the contents of one of the columns:
open Cargo.toml | get package
# => ╭───────────────┬────────────────────────────────────╮
# => │ authors │ [list 1 item] │
# => │ default-run │ nu │
# => │ description │ A new type of shell │
# => │ documentation │ https://www.nushell.sh/book/ │
# => │ edition │ 2018 │
# => │ exclude │ [list 1 item] │
# => │ homepage │ https://www.nushell.sh │
# => │ license │ MIT │
# => │ metadata │ {record 1 field} │
# => │ name │ nu │
# => │ repository │ https://github.com/nushell/nushell │
# => │ rust-version │ 1.60 │
# => │ version │ 0.72.0 │
# => ╰───────────────┴────────────────────────────────────╯
And if needed we can drill down further:
open Cargo.toml | get package.version
# => 0.72.0
Plugins
Nu supports plugins that offer additional functionality to the shell and follow the same structured data model that built-in commands use. There are a few examples in the crates/nu_plugins_* directories.
Plugins are binaries that are available in your path and follow a nu_plugin_* naming convention.
These binaries interact with nu via a simple JSON-RPC protocol where the command identifies itself and passes along its configuration, making it available for use.
If the plugin is a filter, data streams to it one element at a time, and it can stream data back in return via stdin/stdout.
If the plugin is a sink, it is given the full vector of final data and is given free reign over stdin/stdout to use as it pleases.
The awesome-nu repo lists a variety of nu-plugins while the showcase repo shows off informative blog posts that have been written about Nushell along with videos that highlight technical topics that have been presented.
Goals
Nu adheres closely to a set of goals that make up its design philosophy. As features are added, they are checked against these goals.
-
First and foremost, Nu is cross-platform. Commands and techniques should work across platforms and Nu has first-class support for Windows, macOS, and Linux.
-
Nu ensures compatibility with existing platform-specific executables.
-
Nu's workflow and tools should have the usability expected of modern software in 2022 (and beyond).
-
Nu views data as either structured or unstructured. It is a structured shell like PowerShell.
-
Finally, Nu views data functionally. Rather than using mutation, pipelines act as a means to load, change, and save data without mutable state.
Officially Supported By
Please submit an issue or PR to be added to this list.
Contributing
See Contributing for details. Thanks to all the people who already contributed!
License
The project is made available under the MIT license. See the LICENSE file for more information.