# Description
Attempt to guess the content type of a file when opening with --raw and
set it in the pipeline metadata.
<img width="644" alt="Screenshot 2024-08-02 at 11 30 10"
src="https://github.com/user-attachments/assets/071f0967-c4dd-405a-b8c8-f7aa073efa98">
# User-Facing Changes
- Content of files can be directly piped into commands like `http post`
with the content type set appropriately when using `--raw`.
# Description
Add `README.md` files to each crate in our workspace (-plugins) and also
include it in the `lib.rs` documentation for <docs.rs> (if there is no
existing `lib.rs` crate documentation)
In all new README I added the defensive comment that the crates are not
considered stable for public consumption. If necessary we can adjust
this if we deem a crate useful for plugin authors.
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
Finishes implementing https://github.com/nushell/nushell/issues/10598,
which asks for a spread operator in lists, in records, and when calling
commands.
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR will allow spreading arguments to commands (both internal and
external). It will also deprecate spreading arguments automatically when
passing to external commands.
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
- Users will be able to use `...` to spread arguments to custom/builtin
commands that have rest parameters or allow unknown arguments, or to any
external command
- If a custom command doesn't have a rest parameter and it doesn't allow
unknown arguments either, the spread operator will not be allowed
- Passing lists to external commands without `...` will work for now but
will cause a deprecation warning saying that it'll stop working in 0.91
(is 2 versions enough time?)
Here's a function to help with demonstrating some behavior:
```nushell
> def foo [ a, b, c?, d?, ...rest ] { [$a $b $c $d $rest] | to nuon }
```
You can pass a list of arguments to fill in the `rest` parameter using
`...`:
```nushell
> foo 1 2 3 4 ...[5 6]
[1, 2, 3, 4, [5, 6]]
```
If you don't use `...`, the list `[5 6]` will be treated as a single
argument:
```nushell
> foo 1 2 3 4 [5 6] # Note the double [[]]
[1, 2, 3, 4, [[5, 6]]]
```
You can omit optional parameters before the spread arguments:
```nushell
> foo 1 2 3 ...[4 5] # d is omitted here
[1, 2, 3, null, [4, 5]]
```
If you have multiple lists, you can spread them all:
```nushell
> foo 1 2 3 ...[4 5] 6 7 ...[8] ...[]
[1, 2, 3, null, [4, 5, 6, 7, 8]]
```
Here's the kind of error you get when you try to spread arguments to a
command with no rest parameter:
And this is the warning you get when you pass a list to an external now
(without `...`):
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Added tests to cover the following cases:
- Spreading arguments to a command that doesn't have a rest parameter
(unexpected spread argument error)
- Spreading arguments to a command that doesn't have a rest parameter
*but* there's also a missing positional argument (missing positional
error)
- Spreading arguments to a command that doesn't have a rest parameter
but does allow unknown arguments, such as `exec` (allowed)
- Spreading a list literal containing arguments of the wrong type (parse
error)
- Spreading a non-list value, both to internal and external commands
- Having named arguments in the middle of rest arguments
- `explain`ing a command call that spreads its arguments
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
# Examples
Suppose you have multiple tables:
```nushell
let people = [[id name age]; [0 alice 100] [1 bob 200] [2 eve 300]]
let evil_twins = [[id name age]; [0 ecila 100] [-1 bob 200] [-2 eve 300]]
```
Maybe you often find yourself needing to merge multiple tables and want
a utility to do that. You could write a function like this:
```nushell
def merge_all [ ...tables ] { $tables | reduce { |it, acc| $acc | merge $it } }
```
Then you can use it like this:
```nushell
> merge_all ...([$people $evil_twins] | each { |$it| $it | select name age })
╭───┬───────┬─────╮
│ # │ name │ age │
├───┼───────┼─────┤
│ 0 │ ecila │ 100 │
│ 1 │ bob │ 200 │
│ 2 │ eve │ 300 │
╰───┴───────┴─────╯
```
Except they had duplicate columns, so now you first want to suffix every
column with a number to tell you which table the column came from. You
can make a command for that:
```nushell
def select_and_merge [ --cols: list<string>, ...tables ] {
let renamed_tables = $tables
| enumerate
| each { |it|
$it.item | select $cols | rename ...($cols | each { |col| $col + ($it.index | into string) })
};
merge_all ...$renamed_tables
}
```
And call it like this:
```nushell
> select_and_merge --cols [name age] $people $evil_twins
╭───┬───────┬──────┬───────┬──────╮
│ # │ name0 │ age0 │ name1 │ age1 │
├───┼───────┼──────┼───────┼──────┤
│ 0 │ alice │ 100 │ ecila │ 100 │
│ 1 │ bob │ 200 │ bob │ 200 │
│ 2 │ eve │ 300 │ eve │ 300 │
╰───┴───────┴──────┴───────┴──────╯
```
---
Suppose someone's made a command to search for APT packages:
```nushell
# The main command
def search-pkgs [
--install # Whether to install any packages it finds
log_level: int # Pretend it's a good idea to make this a required positional parameter
exclude?: list<string> # Packages to exclude
repositories?: list<string> # Which repositories to look in (searches in all if not given)
...pkgs # Package names to search for
] {
{ install: $install, log_level: $log_level, exclude: ($exclude | to nuon), repositories: ($repositories | to nuon), pkgs: ($pkgs | to nuon) }
}
```
It has a lot of parameters to configure it, so you might make your own
helper commands to wrap around it for specific cases. Here's one
example:
```nushell
# Only look for packages locally
def search-pkgs-local [
--install # Whether to install any packages it finds
log_level: int
exclude?: list<string> # Packages to exclude
...pkgs # Package names to search for
] {
# All required and optional positional parameters are given
search-pkgs --install=$install $log_level [] ["<local URI or something>"] ...$pkgs
}
```
And you can run it like this:
```nushell
> search-pkgs-local --install=false 5 ...["python2.7" "vim"]
╭──────────────┬──────────────────────────────╮
│ install │ false │
│ log_level │ 5 │
│ exclude │ [] │
│ repositories │ ["<local URI or something>"] │
│ pkgs │ ["python2.7", vim] │
╰──────────────┴──────────────────────────────╯
```
One thing I realized when writing this was that if we decide to not
allow passing optional arguments using the spread operator, then you can
(mis?)use the spread operator to skip optional parameters. Here, I
didn't want to give `exclude` explicitly, so I used a spread operator to
pass the packages to install. Without it, I would've needed to do
`search-pkgs-local --install=false 5 [] "python2.7" "vim"` (explicitly
pass `[]` (or `null`, in the general case) to `exclude`). There are
probably more idiomatic ways to do this, but I just thought it was
something interesting.
If you're a virologist of the [xkcd](https://xkcd.com/350/) kind,
another helper command you might make is this:
```nushell
# Install any packages it finds
def live-dangerously [ ...pkgs ] {
# One optional argument was given (exclude), while another was not (repositories)
search-pkgs 0 [] ...$pkgs --install # Flags can go after spread arguments
}
```
Running it:
```nushell
> live-dangerously "git" "*vi*" # *vi* because I don't feel like typing out vim and neovim
╭──────────────┬─────────────╮
│ install │ true │
│ log_level │ 0 │
│ exclude │ [] │
│ repositories │ null │
│ pkgs │ [git, *vi*] │
╰──────────────┴─────────────╯
```
Here's an example that uses the spread operator more than once within
the same command call:
```nushell
let extras = [ chrome firefox python java git ]
def search-pkgs-curated [ ...pkgs ] {
(search-pkgs
1
[emacs]
["example.com", "foo.com"]
vim # A must for everyone!
...($pkgs | filter { |p| not ($p | str contains "*") }) # Remove packages with globs
python # Good tool to have
...$extras
--install=false
python3) # I forget, did I already put Python in extras?
}
```
Running it:
```nushell
> search-pkgs-curated "git" "*vi*"
╭──────────────┬───────────────────────────────────────────────────────────────────╮
│ install │ false │
│ log_level │ 1 │
│ exclude │ [emacs] │
│ repositories │ [example.com, foo.com] │
│ pkgs │ [vim, git, python, chrome, firefox, python, java, git, "python3"] │
╰──────────────┴───────────────────────────────────────────────────────────────────╯
```
This PR makes a couple of tweaks to the testing support crate:
Add the `nu` invocation's exit status to the test output so that one
can assert that nu exited with a successful code.
This PR was split off of #10232.
# Description
Its purpose and its limitation around statements are not too obvious but
ubiquituous in our `nu!` tests. Document its behavior as we remove it in
many places for #8670
# User-Facing Changes
None
## Fix `nu-path` usage in `nu!` testing macro
The `nu-path` crate needs to be properly re-exported so the generated
code is valid if `nu-path` is not present among the dependencies of the
using crate.
Usage of crates in `macro_rules!` macros has to follow the
`$crate::symbol_in_crate` path pattern (With an absolute path-spec also
for macros defined in submodules)
## Move `nu-test-support` to devdeps in `nu-protocol`
Also remove the now unnecessary direct dependency on `nu-path`.
`nu!` macro had to be changed to make it a proper transitive dependency.
* Add decimals to int when using `into string --decimals`
* Add tests for `into string` when converting int with `--decimals`
* Apply formatting
* Merge `into_str` test files
* Comment out unused code and add TODOs
* Use decimal separator depending on system locale
* Add test helper to run closure in different locale
* Add tests for int-to-string conversion using different locales
* Add utils function to get system locale
* Add panic message when locking mutex fails
* Catch and resume panic later to prevent Mutex poisoning when test fails
* Move test to `nu-test-support` to keep `nu-utils` free of `nu-*` dependencies
See https://github.com/nushell/nushell/pull/6085#issuecomment-1193131694
* Rename test support fn `with_fake_locale` to `with_locale_override`
* Move `get_system_locale()` to `locale` module
* Allow overriding locale with special env variable (when not in release)
* Use special env var to override locale during testing
* Allow callback to return a value in `with_locale_override()`
* Allow multiple options in `nu!` macro
* Allow to set locale as `nu!` macro option
* Use new `locale` option of `nu!` macro instead of `with_locale_override`
Using the `locale` options does not lock the `LOCALE_OVERRIDE_MUTEX`
mutex in `nu-test-support::locale_override` but instead calls the `nu`
command directly with the `NU_LOCALE_OVERRIDE` environment variable.
This allows for parallel test excecution.
* Fix: Add option identifier for `cwd` in usage of `nu!` macro
* Rely on `Display` trait for formatting `nu!` macro command
- Removed the `DisplayPath` trait
- Implement `Display` for `AbsolutePath`, `RelativePath` and
`AbsoluteFile`
* Default to locale `en_US.UTF-8` for tests when using `nu!` macro
* Add doc comment to `nu!` macro
* Format code using `cargo fmt --all`
* Pass function directly instead of wrapping the call in a closure
https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure
* Pass function to `or_else()` instead of calling it inside `or()`
https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call
* Fix: Add option identifier for `cwd` in usage of `nu!` macro
* Fix failing pipeline()
The `skip(1)` was there likely to remove the welcome message.
* Fix typo
* Fix nu! test macro to enter cwd correctly
Nushell's current working directory is determined primarily by the PWD
environment variable.
* Fix swapped PATH env var separators
* Support pathvar to manipulate other vars than PATH
* Add tests for pathvar and its subcommands
* Fix PATH env name for Windows
Seems like Windows uses PATH as well.
Co-authored-by: Jakub Žádník <jakub.zadnik@tuni.fi>
The autoenv logic mutates environment variables in the running session as
it operates and decides what to do for trusted directories containing `.nu-env`
files. Few of the ways to interact with it were all in a single test function.
We separate out all the ways that were done in the single test function to document
it better. This will greatly help once we start refactoring our way out from setting
environment variables this way to just setting them to `Scope`.
This is part of an on-going effort to keep variables (`PATH` and `ENV`)
in our `Scope` and rely on it for everything related to variables.
We expect to move away from setting (`std::*`) envrironment variables in the current
running process. This is non-trivial since we need to handle cases from vars
coming in from the outside world, prioritize, and also compare to the ones
we have both stored in memory and in configuration files.
Also to send out our in-memory (in `Scope`) variables properly to external
programs once we no longer rely on `std::env` vars from the running process.
Nu has many commands that allow the nuño to customize behavior such
as UI and behavior. Today, coloring can be customized, the line editor,
and other things. The more options there are, the higher the complexity
in managing them.
To mitigate this Nu can store configuration options as nested properties.
But to add and edit them can be taxing. With column path support we can
work with them easier.
* Switch to using `shell`
Switch to using the shell for subprocess to enable more natural shelling out.
* Update external.rs
* This is a test with .shell() for external
* El pollo loco's PR
* co co co
* Attempt to fix windows
* Fmt
* Less is more?
Co-authored-by: Andrés N. Robalino <andres@androbtech.com>
This commit changes the way we shell out externals when using the `"$it"` argument. Also pipes per row to an external's stdin if no `"$it"` argument is present for external commands.
Further separation of logic (preparing the external's command arguments, getting the data for piping, emitting values, spawning processes) will give us a better idea for lower level details regarding external commands until we can find the right abstractions for making them more generic and unify within the pipeline calling logic of Nu internal's and external's.
