Commit Graph

882 Commits

Author SHA1 Message Date
Darren Schroeder
7f2f67238f allow group-by and split-by to work with other values (#14086)
# Description

This PR updates `group-by` and `split-by` to allow other nushell Values
to be used, namely bools.

### Before
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
Error: nu:🐚:cant_convert

  × Can't convert to string.
   ╭─[entry #1:1:2]
 1 │ [false, false, true, false, true, false] | group-by | table -e
   ·  ──┬──
   ·    ╰── can't convert bool to string
   ╰────
```
### After
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
╭───────┬───────────────╮
│       │ ╭───┬───────╮ │
│ false │ │ 0 │ false │ │
│       │ │ 1 │ false │ │
│       │ │ 2 │ false │ │
│       │ │ 3 │ false │ │
│       │ ╰───┴───────╯ │
│       │ ╭───┬──────╮  │
│ true  │ │ 0 │ true │  │
│       │ │ 1 │ true │  │
│       │ ╰───┴──────╯  │
╰───────┴───────────────╯
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-10-20 23:14:11 +02:00
Ian Manske
e911ff4d67
Fix return setting last exit code (#14120)
# Description

Fixes #14113 and #14112.

# Tests + Formatting

Added a test.
2024-10-18 03:05:58 +00:00
Ian Manske
28b6db115a
Revert PRs for 0.99.1 patch (#14119)
# Description

Temporarily reverts PRs merged after the 0.99.1 bump.
2024-10-18 02:51:14 +00:00
Darren Schroeder
e735bd475f
add rendered and json error messages in try/catch (#14082)
# Description

This PR adds a couple more options for dealing with try/catch errors. It
adds a `json` version of the error and a `rendered` version of the
error. It also respects the error_style configuration point.

![image](https://github.com/user-attachments/assets/32574f07-f511-40c0-8b57-de5f6f13a9c4)


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-10-17 20:16:38 -05:00
Darren Schroeder
299d199150
allow group-by and split-by to work with other values (#14086)
# Description

This PR updates `group-by` and `split-by` to allow other nushell Values
to be used, namely bools.

### Before
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
Error: nu:🐚:cant_convert

  × Can't convert to string.
   ╭─[entry #1:1:2]
 1 │ [false, false, true, false, true, false] | group-by | table -e
   ·  ──┬──
   ·    ╰── can't convert bool to string
   ╰────
```
### After
```nushell
❯ [false, false, true, false, true, false] | group-by | table -e
╭───────┬───────────────╮
│       │ ╭───┬───────╮ │
│ false │ │ 0 │ false │ │
│       │ │ 1 │ false │ │
│       │ │ 2 │ false │ │
│       │ │ 3 │ false │ │
│       │ ╰───┴───────╯ │
│       │ ╭───┬──────╮  │
│ true  │ │ 0 │ true │  │
│       │ │ 1 │ true │  │
│       │ ╰───┴──────╯  │
╰───────┴───────────────╯
```

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-10-17 16:14:01 -05:00
Wind
639bd4fc2e
change display_error.exit_code to false (#13873)
The idea comes from @amtoine, I think it would be good to keey
`display_error.exit_code` same value, if user is using default config or
using no config file at all.
2024-10-14 09:57:30 -05:00
Solomon
d83781ddec
support filesize arguments in random binary/chars (#14068)
Closes #13920

# User-Facing Changes

`random binary` and `random chars` now support filesize arguments:

```nushell
random binary 1kb
random chars --length 1kb
```
2024-10-12 14:49:05 +08:00
Ian Manske
de08b68ba8
Fix try printing when it is not the last pipeline element (#13992)
# Description

Fixes #13991. This was done by more clearly separating the case when a
pipeline is drained vs when it is being written (to a file).

I also added an `OutDest::Print` case which might not be strictly
necessary, but is a helpful addition.

# User-Facing Changes

Bug fix.

# Tests + Formatting

Added a test.

# After Submitting

There are still a few redirection bugs that I found, but they require
larger code changes, so I'll leave them until after the release.
2024-10-12 14:37:10 +08:00
Tristan P.
9f714e62cb
[umkdir][tests] get umask instead of assuming it (#14046)
# Description

Contributors to this projects will have a test failure if their `umask`
is not set to `0022`.

Apparently on Debian (at least on my install), it is set to `0002` which
makes my test fail. While `0022` is safer than the value I have, I want
to reduce the amount if issue new contributors could have.

I am making this test not assuming anything and instead, reading the
user umask.

# Related discussion

I see that the `umask` command implementation has been discussed in
#12256 . We could use this and enforce a umask for tests who rely on
this. I believe however (let me know what you think) that hard coded
values are harder to read in the test.



# User-Facing Changes
N/A

# Tests + Formatting
All green on my side after this MR 👍 


# After Submitting
Documentation is not impacted

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-10-11 14:13:42 +02:00
Ian Manske
a95c2198a6
Remove group command (#14056)
# Description

Removes the `group` command that was deprecated back in 0.96.0 with
#13377.

# User-Facing Changes

Breaking change, removed `group` command.
2024-10-11 06:43:12 -05:00
Wind
44be445b57
Revert "fix $env.FILE_PWD and $env.CURRENT_FILE inside use (#13958)" (#14057)
This reverts commit 5002d87af4 from pr
#13958

It seems that something unexpected happened from
[@ealap](https://github.com/ealap)'s report. Thanks!

Reopen: #13425
2024-10-11 14:45:42 +08:00
Wind
5002d87af4
fix $env.FILE_PWD and $env.CURRENT_FILE inside use (#13958)
# Description
Fixes: #13425 

Similar to `source-env`, `use` command should also remove `FILE_PWD` and
`CURRENT_FILE` after evaluating code block in the module file.

And user input can be a directory, in this case, we need to use the
return value of `find_in_dirs_env` carefully, so in case, I renamed
`maybe_file_path` to `maybe_file_path_or_dir` to emphasize it.

# User-Facing Changes
`$env.FILE_PWD` and `$env.CURRENT_FILE` will be more reliable to use.

# Tests + Formatting
Added 2 test cases.

# After Submitting
NaN
2024-10-10 20:54:00 +08:00
132ikl
36c1073441
Rework sorting and add cell path and closure comparators to sort-by (#13154)
# Description

Closes #12535
Implements sort-by functionality of #8322
Fixes sort-by part of #8667

This PR does two main things: add a new cell path and closure parameter
to `sort-by`, and attempt to make Nushell's sorting behavior
well-defined.

## `sort-by` features

The `columns` parameter is replaced with a `comparator` parameter, which
can be a cell path or a closure. Examples are from docs PR.

1. Cell paths

The basic interactive usage of `sort-by` is the same. For example, `ls |
sort-by modified` still works the same as before. It is not quite a
drop-in replacement, see [behavior changes](#behavior-changes).
   
   Here's an example of how the cell path comparator might be useful:
   
   ```nu
   > let cities = [
{name: 'New York', info: { established: 1624, population: 18_819_000 } }
{name: 'Kyoto', info: { established: 794, population: 37_468_000 } }
{name: 'São Paulo', info: { established: 1554, population: 21_650_000 }
}
   ]
   > $cities | sort-by info.established
   ╭───┬───────────┬────────────────────────────╮
   │ # │   name    │            info            │
   ├───┼───────────┼────────────────────────────┤
   │ 0 │ Kyoto     │ ╭─────────────┬──────────╮ │
   │   │           │ │ established │ 794      │ │
   │   │           │ │ population  │ 37468000 │ │
   │   │           │ ╰─────────────┴──────────╯ │
   │ 1 │ São Paulo │ ╭─────────────┬──────────╮ │
   │   │           │ │ established │ 1554     │ │
   │   │           │ │ population  │ 21650000 │ │
   │   │           │ ╰─────────────┴──────────╯ │
   │ 2 │ New York  │ ╭─────────────┬──────────╮ │
   │   │           │ │ established │ 1624     │ │
   │   │           │ │ population  │ 18819000 │ │
   │   │           │ ╰─────────────┴──────────╯ │
   ╰───┴───────────┴────────────────────────────╯
   ```

2. Key closures

You can supply a closure which will transform each value into a sorting
key (without changing the underlying data). Here's an example of a key
closure, where we want to sort a list of assignments by their average
grade:

   ```nu
   > let assignments = [
       {name: 'Homework 1', grades: [97 89 86 92 89] }
       {name: 'Homework 2', grades: [91 100 60 82 91] }
       {name: 'Exam 1', grades: [78 88 78 53 90] }
       {name: 'Project', grades: [92 81 82 84 83] }
   ]
   > $assignments | sort-by { get grades | math avg }
   ╭───┬────────────┬───────────────────────╮
   │ # │    name    │        grades         │
   ├───┼────────────┼───────────────────────┤
   │ 0 │ Exam 1     │ [78, 88, 78, 53, 90]  │
   │ 1 │ Project    │ [92, 81, 82, 84, 83]  │
   │ 2 │ Homework 2 │ [91, 100, 60, 82, 91] │
   │ 3 │ Homework 1 │ [97, 89, 86, 92, 89]  │
   ╰───┴────────────┴───────────────────────╯
   ```

3. Custom sort closure

The `--custom`, or `-c`, flag will tell `sort-by` to interpret closures
as custom sort closures. A custom sort closure has two parameters, and
returns a boolean. The closure should return `true` if the first
parameter comes _before_ the second parameter in the sort order.
   
For a simple example, we could rewrite a cell path sort as a custom sort
(see
[here](https://github.com/nushell/nushell.github.io/pull/1568/files#diff-a7a233e66a361d8665caf3887eb71d4288000001f401670c72b95cc23a948e86R231)
for a more complex example):
   
   ```nu
   > ls | sort-by -c {|a, b| $a.size < $b.size }
   ╭───┬─────────────────────┬──────┬──────────┬────────────────╮
   │ # │        name         │ type │   size   │    modified    │
   ├───┼─────────────────────┼──────┼──────────┼────────────────┤
   │ 0 │ my-secret-plans.txt │ file │    100 B │ 10 minutes ago │
   │ 1 │ shopping_list.txt   │ file │    100 B │ 2 months ago   │
   │ 2 │ myscript.nu         │ file │  1.1 KiB │ 2 weeks ago    │
   │ 3 │ bigfile.img         │ file │ 10.0 MiB │ 3 weeks ago    │
   ╰───┴─────────────────────┴──────┴──────────┴────────────────╯
   ```
   

## Making sort more consistent

I think it's important for something as essential as `sort` to have
well-defined semantics. This PR contains some changes to try to make the
behavior of `sort` and `sort-by` consistent. In addition, after working
with the internals of sorting code, I have a much deeper understanding
of all of the edge cases. Here is my attempt to try to better define
some of the semantics of sorting (if you are just interested in changes,
skip to "User-Facing changes")

- `sort`, `sort -v`, and `sort-by` now all work the same. Each
individual sort implementation has been refactored into two functions in
`sort_utils.rs`: `sort`, and `sort_by`. These can also be used in other
parts of Nushell where values need to be sorted.
  - `sort` and `sort-by` used to handle `-i` and `-n` differently.
- `sort -n` would consider all values which can't be coerced into a
string to be equal
- `sort-by -i` and `sort-by -n` would only work if all values were
strings
- In this PR, insensitive sort only affects comparison between strings,
and natural sort only applies to numbers and strings (see below).
- (not a change) Before and after this PR, `sort` and `sort-by` support
sorting mixed types. There was a lot of discussion about potentially
making `sort` and `sort-by` only work on lists of homogeneous types, but
the general consensus was that `sort` should not error just because its
input contains incompatible types.
- In order to try to make working with data containing `null` values
easier, I changed the PartialOrd order to sort `Nothing` values to the
end of a list, regardless of what other types the list contains. Before,
`null` would be sorted before `Binary`, `CellPath`, and `Custom` values.
- (not a change) When sorted, lists of mixed types will contain sorted
values of each type in order, for the most part
- (not a change) For example, `[0x[1] (date now) "a" ("yesterday" | into
datetime) "b" 0x[0]]` will be sorted as `["a", "b", a day ago, now, [0],
[1]]`, where sorted strings appear first, then sorted datetimes, etc.
- (not a change) The exception to this is `Int`s and `Float`s, which
will intermix, `Strings` and `Glob`s, which will intermix, and `None` as
described above. Additionally, natural sort will intermix strings with
ints and floats (see below).
- Natural sort no longer coerce all inputs to strings.
- I did originally make natural only apply to strings, but @fdncred
pointed out that the previous behavior also allowed you to sort numeric
strings with numbers. This seems like a useful feature if we are trying
to support sorting with mixed types, so I settled on coercing only
numbers (int, float). This can be reverted if people don't like it.
- Here is an example of this behavior in action, which is the same
before and after this PR:
      ```nushell
      $ [1 "4" 3 "2"] | sort --natural
      ╭───┬───╮
      │ 0 │ 1 │
      │ 1 │ 2 │
      │ 2 │ 3 │
      │ 3 │ 4 │
      ╰───┴───╯
      ```



# User-Facing Changes

## New features

- Replaces the `columns` string parameter of `sort-by` with a cell path
or a closure.
  - The cell path parameter works exactly as you would expect
- By default, the `closure` parameter acts as a "key sort"; that is,
each element is transformed by the closure into a sorting key
- With the `--custom` (`-c`) parameter, you can define a comparison
function for completely custom sorting order.

## Behavior changes

<details>
<summary><code>sort -v</code> does not coerce record values to
strings</summary>

This was a bit of a surprising behavior, and is now unified with the
behavior of `sort` and `sort-by`. Here's an example where you can
observe the values being implicitly coerced into strings for sorting, as
they are sorted like strings rather than numbers:

Old behavior:

```nushell
$ {foo: 9 bar: 10} | sort -v
╭─────┬────╮
│ bar │ 10 │
│ foo │ 9  │
╰─────┴────╯
```

New behavior:

```nushell
$ {foo: 9 bar: 10} | sort -v
╭─────┬────╮
│ foo │ 9  │
│ bar │ 10 │
╰─────┴────╯
```

</details>


<details>
<summary>Changed <code>sort-by</code> parameters from
<code>string</code> to <code>cell-path</code> or <code>closure</code>.
Typical interactive usage is the same as before, but if passing a
variable to <code>sort-by</code> it must be a cell path (or closure),
not a string</summary>

Old behavior:

```nushell
$ let sort = "modified"
$ ls | sort-by $sort
╭───┬──────┬──────┬──────┬────────────────╮
│ # │ name │ type │ size │    modified    │
├───┼──────┼──────┼──────┼────────────────┤
│ 0 │ foo  │ file │  0 B │ 10 hours ago   │
│ 1 │ bar  │ file │  0 B │ 35 seconds ago │
╰───┴──────┴──────┴──────┴────────────────╯
```

New behavior:

```nushell
$ let sort = "modified"
$ ls | sort-by $sort
Error: nu:🐚:type_mismatch

  × Type mismatch.
   ╭─[entry #10:1:14]
 1 │ ls | sort-by $sort
   ·              ──┬──
   ·                ╰── Cannot sort using a value which is not a cell path or closure
   ╰────
$ let sort = $."modified"
$ ls | sort-by $sort
╭───┬──────┬──────┬──────┬───────────────╮
│ # │ name │ type │ size │   modified    │
├───┼──────┼──────┼──────┼───────────────┤
│ 0 │ foo  │ file │  0 B │ 10 hours ago  │
│ 1 │ bar  │ file │  0 B │ 2 minutes ago │
╰───┴──────┴──────┴──────┴───────────────╯
```
</details>

<details>
<summary>Insensitve and natural sorting behavior reworked</summary>

Previously, the `-i` and `-n` worked differently for `sort` and
`sort-by` (see "Making sort more consistent"). Here are examples of how
these options result in different sorts now:

1. `sort -n`
- Old behavior (types other than numbers, strings, dates, and binary
sorted incorrectly)
      ```nushell
      $ [2sec 1sec] | sort -n
      ╭───┬──────╮
      │ 0 │ 2sec │
      │ 1 │ 1sec │
      ╰───┴──────╯
      ```
    - New behavior
      ```nushell
      $ [2sec 1sec] | sort -n
      ╭───┬──────╮
      │ 0 │ 1sec │
      │ 1 │ 2sec │
      ╰───┴──────╯
      ```
    
2. `sort-by -i`
- Old behavior (uppercase words appear before lowercase words as they
would in a typical sort, indicating this is not actually an insensitive
sort)
     ```nushell
     $ ["BAR" "bar" "foo" 2 "FOO" 1] | wrap a | sort-by -i a
     ╭───┬─────╮
     │ # │  a  │
     ├───┼─────┤
     │ 0 │   1 │
     │ 1 │   2 │
     │ 2 │ BAR │
     │ 3 │ FOO │
     │ 4 │ bar │
     │ 5 │ foo │
     ╰───┴─────╯
     ```
- New behavior (strings are sorted stably, indicating this is an
insensitive sort)
     ```nushell
     $ ["BAR" "bar" "foo" 2 "FOO" 1] | wrap a | sort-by -i a
     ╭───┬─────╮
     │ # │  a  │
     ├───┼─────┤
     │ 0 │   1 │
     │ 1 │   2 │
     │ 2 │ BAR │
     │ 3 │ bar │
     │ 4 │ foo │
     │ 5 │ FOO │
     ╰───┴─────╯
     ```

3. `sort-by -n`
- Old behavior (natural sort does not work when data contains non-string
values)
     ```nushell
     $ ["10" 8 "9"] | wrap a | sort-by -n a
     ╭───┬────╮
     │ # │ a  │
     ├───┼────┤
     │ 0 │  8 │
     │ 1 │ 10 │
     │ 2 │ 9  │
     ╰───┴────╯
     ```
   - New behavior
     ```nushell
     $ ["10" 8 "9"] | wrap a | sort-by -n a
     ╭───┬────╮
     │ # │ a  │
     ├───┼────┤
     │ 0 │  8 │
     │ 1 │ 9  │
     │ 2 │ 10 │
     ╰───┴────╯
     ```

</details>

<details>
<summary>
Sorting a list of non-record values with a non-existent column/path now
errors instead of sorting the values directly (<code>sort</code> should
be used for this, not <code>sort-by</code>)
</summary>

Old behavior:

```nushell
$ [2 1] | sort-by foo
╭───┬───╮
│ 0 │ 1 │
│ 1 │ 2 │
╰───┴───╯
```

New behavior:

```nushell
$ [2 1] | sort-by foo
Error: nu:🐚:incompatible_path_access

  × Data cannot be accessed with a cell path
   ╭─[entry #29:1:17]
 1 │ [2 1] | sort-by foo
   ·                 ─┬─
   ·                  ╰── int doesn't support cell paths
   ╰────
```

</details>

<details>
<summary><code>sort</code> and <code>sort-by</code> output
<code>List</code> instead of <code>ListStream</code> </summary>

This isn't a meaningful change (unless I misunderstand the purpose of
ListStream), since `sort` and `sort-by` both need to collect in order to
do the sorting anyway, but is user observable.

Old behavior:

```nushell
$ ls | sort | describe -d
╭──────────┬───────────────────╮
│ type     │ stream            │
│ origin   │ nushell           │
│ subtype  │ {record 3 fields} │
│ metadata │ {record 1 field}  │
╰──────────┴───────────────────╯
```

```nushell
$ ls | sort-by name | describe -d
╭──────────┬───────────────────╮
│ type     │ stream            │
│ origin   │ nushell           │
│ subtype  │ {record 3 fields} │
│ metadata │ {record 1 field}  │
╰──────────┴───────────────────╯
```

New behavior:


```nushell
ls | sort | describe -d
╭────────┬─────────────────╮
│ type   │ list            │
│ length │ 22              │
│ values │ [table 22 rows] │
╰────────┴─────────────────╯
```

```nushell
$ ls | sort-by name | describe -d
╭────────┬─────────────────╮
│ type   │ list            │
│ length │ 22              │
│ values │ [table 22 rows] │
╰────────┴─────────────────╯
```

</details>

- `sort` now errors when nothing is piped in (`sort-by` already did
this)

# Tests + Formatting

I added lots of unit tests on the new sort implementation to enforce new
sort behaviors and prevent regressions.

# After Submitting

See [docs PR](https://github.com/nushell/nushell.github.io/pull/1568),
which is ~2/3 finished.

---------

Co-authored-by: NotTheDr01ds <32344964+NotTheDr01ds@users.noreply.github.com>
Co-authored-by: Ian Manske <ian.manske@pm.me>
2024-10-09 19:18:16 -07:00
Justin Ma
2830ec008c
Replace the old encode base64 and decode base64 with new-base64 commands (#14018)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Maybe we can deprecate `encode new-base64` and `decode new-base64`
first, to make the code clean and simple I'd rather remove the old
`encode base64` and `decode base64` and replace them with the `*
new-base64` commands.

Related PR: https://github.com/nushell/nushell/pull/13428

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

- `encode new-base64` --> `encode base64`
- `decode new-base64` --> `decode base64`

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

It's a breaking change
2024-10-08 11:01:43 +08:00
Solomon
071faae772
fix inspect and explore panics on empty records (#13893)
# Description

Fixes a couple panics:

```
> {} | inspect
Error:   x Main thread panicked.
  |-> at crates/nu-command/src/debug/inspect_table.rs:87:15
  `-> attempt to divide by zero
```

```
> {} | explore
# see an empty column, press Down
Error:   x Main thread panicked.
  |-> at crates/nu-explore/src/views/cursor/mod.rs:39:9
  `-> attempt to subtract with overflow
```

# User-Facing Changes

`{} | inspect` now outputs an empty table:

```
╭─────────────┬────────╮
│ description │ record │
├─────────────┴────────┤
│                      │
├──────────────────────┤
```

`{} | explore` opens the help menu.

Both match the empty list behavior.

# Tests

I'm not sure how to test `explore`, as it waits for interaction.
2024-09-25 07:48:16 -05:00
Ian Manske
03ee54a4df
Fix try not working with let, etc. (#13885)
# Description
Partialy addresses #13868. `try` does not catch non-zero exit code
errors from the last command in a pipeline if the result is assigned to
a variable using `let` (or `mut`).

This was fixed by adding a new `OutDest::Value` case. This is used when
the pipeline is in a "value" position. I.e., it will be collected into a
value. This ended up replacing most of the usages of `OutDest::Capture`.
So, this PR also renames `OutDest::Capture` to `OutDest::PipeSeparate`
to better fit the few remaining use cases for it.

# User-Facing Changes
Bug fix.

# Tests + Formatting
Added two tests.
2024-09-23 06:44:25 -05:00
Ian Manske
cd0d0364ec
Fix do -p not waiting for external commands (#13881)
# Description
Similar to #13870 (thanks @WindSoilder), this PR adds a boolean which
determines whether to ignore any errors from an external command. This
is in order to fix #13876. I.e., `do -p` does not wait for externals to
complete before continuing.

# User-Facing Changes
Bug fix.

# Tests + Formatting
Added a test.
2024-09-22 22:26:32 +08:00
nome
5101b5e306
Add --number flag to split column (#13831)
This allows parsing of data (e.g. key-value pairs) where the last column
may contain the delimiter.

- this PR should close #13742

# Description

Adds a `--number (-n)` flag to `split column`, analogous to `split row
--number`.

```
~> ['author: Salina Yoon' r#'title: Where's Ellie?: A Hide-and-Seek Book'#] | split column --number 2 ': ' key value
╭───┬────────┬──────────────────────────────────────╮
│ # │  key   │                value                 │
├───┼────────┼──────────────────────────────────────┤
│ 0 │ author │ Salina Yoon                          │
│ 1 │ title  │ Where's Ellie?: A Hide-and-Seek Book │
╰───┴────────┴──────────────────────────────────────╯
```

# User-Facing Changes
* `split column` gains a `--number` option

# Tests + Formatting
Tests included in strings::split::column::test::test_examples and
commands::split_column::to_column.

# After Submitting
Reference documentation is auto-generated from code. No other
documentation updates necessary.
2024-09-12 07:16:33 -05:00
Wind
217eb4ed70
fix path exists on a non-directory file (#13763)
# Description
Fixes:  #13460

The issue is caused by `try_exists` method on path, it will return
`Err(NotADirectory)` if user tried to check for a path under a regular
file.
To fix it, I think it's ok to use `exists` rather than `try_exists`,
although
[Path::exists()](https://doc.rust-lang.org/std/path/struct.Path.html#method.exists)
only checks whether or not a path was both found and readable. I think
it's ok, and we can add this information under `extra_description`.

# User-Facing Changes
The following code will no longer raise error:
```
touch a; 'a/b' | path exists
```

# Tests + Formatting
Added 1 test.
2024-09-11 12:45:39 -05:00
Ian Manske
493850b1bf
Fix IR for try (#13811)
# Description
Fixes a bug in the IR for `try` to match that of the regular evaluator
(continuing from #13515):
```nushell
# without IR:
try { ^false } catch { 'caught' } # == 'caught'

# with IR:
try { ^false } catch { 'caught' } # error, non-zero exit code
```

In this PR, both now evaluate to `caught`. For the implementation, I had
to add another instruction, and feel free to suggest better
alternatives. In the future, it might be possible to get rid of this
extra instruction.

# User-Facing Changes
Bug fix, `try { ^false } catch { 'caught' }` now works in IR.
2024-09-09 19:44:04 -07:00
Ian Manske
3d008e2c4e
Error on non-zero exit statuses (#13515)
# Description
This PR makes it so that non-zero exit codes and termination by signal
are treated as a normal `ShellError`. Currently, these are silent
errors. That is, if an external command fails, then it's code block is
aborted, but the parent block can sometimes continue execution. E.g.,
see #8569 and this example:
```nushell
[1 2] | each { ^false }
```

Before this would give:
```
╭───┬──╮
│ 0 │  │
│ 1 │  │
╰───┴──╯
```

Now, this shows an error:
```
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
   ╭─[entry #1:1:2]
 1 │ [1 2] | each { ^false }
   ·  ┬
   ·  ╰── source value
   ╰────

Error: nu:🐚:non_zero_exit_code

  × External command had a non-zero exit code
   ╭─[entry #1:1:17]
 1 │ [1 2] | each { ^false }
   ·                 ──┬──
   ·                   ╰── exited with code 1
   ╰────
```

This PR fixes #12874, fixes #5960, fixes #10856, and fixes #5347. This
PR also partially addresses #10633 and #10624 (only the last command of
a pipeline is currently checked). It looks like #8569 is already fixed,
but this PR will make sure it is definitely fixed (fixes #8569).

# User-Facing Changes
- Non-zero exit codes and termination by signal now cause an error to be
thrown.
- The error record value passed to a `catch` block may now have an
`exit_code` column containing the integer exit code if the error was due
to an external command.
- Adds new config values, `display_errors.exit_code` and
`display_errors.termination_signal`, which determine whether an error
message should be printed in the respective error cases. For
non-interactive sessions, these are set to `true`, and for interactive
sessions `display_errors.exit_code` is false (via the default config).

# Tests
Added a few tests.

# After Submitting
- Update docs and book.
- Future work:
- Error if other external commands besides the last in a pipeline exit
with a non-zero exit code. Then, deprecate `do -c` since this will be
the default behavior everywhere.
- Add a better mechanism for exit codes and deprecate
`$env.LAST_EXIT_CODE` (it's buggy).
2024-09-07 06:44:26 +00:00
Jack Wright
2d360fda7f
String values should pass their content-type correctly on http requests with a body (e.g. http post) (#13731)
# Description
The content-type was not being handled appropriately when sending
requests with a string value.

# User-Facing Changes
- Passing a string value through a pipeline with a content-type set as
metadata is handled correctly
- Passing a string value as a parameter with a content-type set as a
flag is handled correctly.
2024-09-05 16:29:50 -07:00
Darren Schroeder
d0c2adabf7
remove config use_grid_icons, move to parameter of grid command (#13788)
# Description

After looking at #13751 I noticed that the config setting
`use_grid_icons` seems out of place. So, I removed it from the config
and made it a parameter to the `grid` command.
2024-09-06 07:25:43 +08:00
Darren Schroeder
870eb2530c
fixed issue with find not working with symbols that should be escaped (#13792)
# Description

Thanks to @weirdan's suggestion, this now works.

![image](https://github.com/user-attachments/assets/34522c7b-b012-4f73-883d-9913142e85b9)

Closes #13789
2024-09-06 07:22:03 +08:00
xonas
4792328d0e
Refactor send_request in client.rs (#13701)
Closes #13687
Closes #13686

# Description
Light refactoring of `send_request `in `client.rs`. In the end there are
more lines but now the logic is more concise and facilitates adding new
conditions in the future. Unit tests ran fine and I tested a few cases
manually.
Cool project btw, I'll be using nushell from now on.
2024-09-04 23:05:39 +02:00
Devyn Cairns
39bda8986e
Make tee work more nicely with non-collections (#13652)
# Description

This changes the behavior of `tee` to be more transparent when given a
value that isn't a list or range. Previously, anything that wasn't a
byte stream would converted to a list stream using the iterator
implementation, which led to some surprising results. Instead, now, if
the value is a string or binary, it will be treated the same way a byte
stream is, and the output of `tee` is a byte stream instead of the
original value. This is done so that we can synchronize with the other
thread on collect, and potentially capture any error produced by the
closure.

For values that can't be converted to streams, the closure is just run
with a clone of the value instead on another thread. Because we can't
wait for the other thread, there is no way to send an error back to the
original thread, so instead it's just written to stderr using
`report_error_new()`.

There are a couple of follow up edge cases I see where byte streams
aren't necessarily treated exactly the same way strings are, but this
should mostly be a good experience.

Fixes #13489.

# User-Facing Changes

Breaking change.

- `tee` now outputs and sends string/binary stream for string/binary
input.
- `tee` now outputs and sends the original value for any other input
other than lists/ranges.

# Tests + Formatting

Added for new behavior.

# After Submitting

- [ ] release notes: breaking change, command change
2024-09-01 19:03:46 +02:00
Andrej Kolčin
3f332bef35
Fix encode/decode todo's (#13683)
Mistakes have been made. I forgot about a bunch of `todo`s in the helper
functions. So, this PR replaces them with proper errors. It also adds
tests for parse-time evaluation, because one `todo` I missed was in a
`run_const` function.
2024-08-24 09:02:02 -05:00
Andrej Kolčin
0560826414
encode/decode for multiple alphabets (#13428)
Based on the discussion in #13419.


## Description

Reworks the `decode`/`encode` commands by adding/changing the following
bases:

- `base32`
- `base32hex`
- `hex`
- `new-base64`

The `hex` base is compatible with the previous version of `hex` out of
the box (it only adds more flags). `base64` isn't, so the PR adds a new
version and deprecates the old one.

All commands have `string -> binary` signature for decoding and `string
| binary -> string` signature for encoding. A few `base64` encodings,
which are not a part of the
[RFC4648](https://datatracker.ietf.org/doc/html/rfc4648#section-6), have
been dropped.


## Example usage

```Nushell
~/fork/nushell> "string" | encode base32 | decode base32 | decode
string
```

```Nushell
~/fork/nushell> "ORSXG5A=" | decode base32
# `decode` always returns a binary value
Length: 4 (0x4) bytes | printable whitespace ascii_other non_ascii
00000000:   74 65 73 74                                          test
```


## User-Facing Changes

- New commands: `encode/decode base32/base32hex`.
- `encode hex` gets a `--lower` flag.
- `encode/decode base64` deprecated in favor of `encode/decode
new-base64`.
2024-08-23 11:18:51 -05:00
Stefan Holderbach
95b78eee25
Change the usage misnomer to "description" (#13598)
# Description
    
The meaning of the word usage is specific to describing how a command
function is *used* and not a synonym for general description. Usage can
be used to describe the SYNOPSIS or EXAMPLES sections of a man page
where the permitted argument combinations are shown or example *uses*
are given.
Let's not confuse people and call it what it is a description.

Our `help` command already creates its own *Usage* section based on the
available arguments and doesn't refer to the description with usage.

# User-Facing Changes

`help commands` and `scope commands` will now use `description` or
`extra_description`
`usage`-> `description`
`extra_usage` -> `extra_description`

Breaking change in the plugin protocol:

In the signature record communicated with the engine.
`usage`-> `description`
`extra_usage` -> `extra_description`

The same rename also takes place for the methods on
`SimplePluginCommand` and `PluginCommand`

# Tests + Formatting
- Updated plugin protocol specific changes
# After Submitting
- [ ] update plugin protocol doc
2024-08-22 12:02:08 +02:00
Devyn Cairns
7a888c9e9b
Change behavior of into record on lists to be more useful (#13637)
# Description

The previous behaviour of `into record` on lists was to create a new
record with each list index as the key. This was not very useful for
creating meaningful records, though, and most people would end up using
commands like `headers` or `transpose` to turn a list of keys and values
into a record.

This PR changes that instead to do what I think the most ergonomic thing
is, and instead:

- A list of records is merged into one record.
- A list of pairs (two element lists) is folded into a record with the
first element of each pair being the key, and the second being the
value.

The former is just generally more useful than having to use `reduce`
with `merge` for such a common operation, and the latter is useful
because it means that `$a | zip $b | into record` *just works* in the
way that seems most obvious.

Example:

```nushell
[[foo bar] [baz quux]] | into record # => {foo: bar, baz: quux}
[{foo: bar} {baz: quux}] | into record # => {foo: bar, baz: quux}
[foo baz] | zip [bar quux] | into record # => {foo: bar, baz: quux}
```

The support for range input has been removed, as it would no longer
reflect the treatment of an equivalent list.

The following is equivalent to the old behavior, in case that's desired:

```
0.. | zip [a b c] | into record # => {0: a, 1: b, 2: c}
```

# User-Facing Changes
- `into record` changed as described above (breaking)
- `into record` no longer supports range input (breaking)

# Tests + Formatting
Examples changed to match, everything works. Some usage in stdlib and
`nu_plugin_nu_example` had to be changed.

# After Submitting
- [ ] release notes (commands, breaking change)
2024-08-22 11:38:43 +02:00
Bruce Weirdan
e690e7aac0
Fallback to extension-based content type detection when parsing Content-Type header fails (#13610)
# Description

Previously when nushell failed to parse the content type header, it
would emit an error instead of returning the response. Now it will fall
back to `text/plain` (which, in turn, will trigger type detection based
on file extension).

May fix (potentially) nushell/nushell#11927

Refs:
https://discord.com/channels/601130461678272522/614593951969574961/1272895236489613366

Supercedes: #13609 

# User-Facing Changes

It's now possible to fetch content even if the server returns an invalid
content type header. Users may need to parse the response manually, but
it's still better than not getting the response at all.

# Tests + Formatting

Added a test for the new behaviour.

# After Submitting
2024-08-14 09:47:01 -05:00
Wind
a55d172e52
make ls -lf outputs full path in symbolic target (#13605)
# Description
Fixes: #13479 

# User-Facing Changes
Given the following setup:
```
cd /tmp
touch src_file.txt
ln -s src_file.txt link1
```
### Before 
```
ls -lf link1 | get target.0  # It outputs src_file.txt
```
### After
```
ls -lf link1 | get target.0  # It outputs /tmp/src_file.txt
```
# Tests + Formatting
Added a test for the change
2024-08-13 06:34:10 -05:00
Devyn Cairns
4e205cd9a7
Add --raw switch to print for binary data (#13597)
# Description

Something I meant to add a long time ago. We currently don't have a
convenient way to print raw binary data intentionally. You can pipe it
through `cat` to turn it into an unknown stream, or write it to a file
and read it again, but we can't really just e.g. generate msgpack and
write it to stdout without this. For example:

```nushell
[abc def] | to msgpack | print --raw
```

This is useful for nushell scripts that will be piped into something
else. It also means that `nu_plugin_nu_example` probably doesn't need to
do this anymore, but I haven't adjusted it yet:

```nushell
def tell_nushell_encoding [] {
  print -n "\u{0004}json"
}
```

This happens to work because 0x04 is a valid UTF-8 character, but it
wouldn't be possible if it were something above 0x80.

`--raw` also formats other things without `table`, I figured the two
things kind of go together. The output is kind of like `to text`.
Debatable whether that should share the same flag, but it was easier
that way and seemed reasonable.

# User-Facing Changes
- `print` new flag: `--raw`

# Tests + Formatting
Added tests.

# After Submitting
- [ ] release notes (command modified)
2024-08-12 17:29:25 +08:00
Qnbie
e530e7d654
Make the math commands const (#13566)
This PR closes [Issue
#13482](https://github.com/nushell/nushell/issues/13482)

# Description
This PR tend to make all math function to be constant. 

# User-Facing Changes
The math commands now can be used as constant methods.

### Some Example
```
> const MODE = [3 3 9 12 12 15] | math mode
> $MODE
╭───┬────╮
│ 0 │  3 │
│ 1 │ 12 │
╰───┴────╯

> const LOG = [16 8 4] | math log 2
> $LOG
╭───┬──────╮
│ 0 │ 4.00 │
│ 1 │ 3.00 │
│ 2 │ 2.00 │
╰───┴──────╯

> const VAR = [1 3 5] | math variance
> $VAR
2.6666666666666665
```

# Tests + Formatting
Tests are added for all of the math command to test there constant
behavior.

I mostly focused on the actual user experience, not the correctness of
the methods and algorithms.

# After Submitting
I think this change don't require any additional documentation. Feel
free to correct me in this topic please.
2024-08-07 22:20:33 +02:00
Bruce Weirdan
2ced9e4d19
Add multipart/form-data uploads (#13532)
Fixes nushell/nushell#11046

# Description
This adds support for `multipart/form-data` (RFC 7578) uploads to
nushell.

Binary data is uploaded as files (`application/octet-stream`),
everything else is uploaded as plain text.

```console
$ http post https://echo.free.beeceptor.com --content-type multipart/form-data {cargo: (open -r Cargo.toml | into binary ), description: "It's some TOML"} | upsert ip "<redacted>"
╭───────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ method            │ POST                                                                                                                                                                    │
│ protocol          │ https                                                                                                                                                                   │
│ host              │ echo.free.beeceptor.com                                                                                                                                                 │
│ path              │ /                                                                                                                                                                       │
│ ip                │ <redacted>                                                                                                                                                              │
│                   │ ╭─────────────────┬────────────────────────────────────────────────────────────────────╮                                                                                │
│ headers           │ │ Host            │ echo.free.beeceptor.com                                            │                                                                                │
│                   │ │ User-Agent      │ nushell                                                            │                                                                                │
│                   │ │ Content-Length  │ 9453                                                               │                                                                                │
│                   │ │ Accept          │ */*                                                                │                                                                                │
│                   │ │ Accept-Encoding │ gzip                                                               │                                                                                │
│                   │ │ Content-Type    │ multipart/form-data; boundary=a15f6a14-5768-4a6a-b3a4-686a112d9e27 │                                                                                │
│                   │ ╰─────────────────┴────────────────────────────────────────────────────────────────────╯                                                                                │
│ parsedQueryParams │ {record 0 fields}                                                                                                                                                       │
│                   │ ╭─────────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │
│ parsedBody        │ │                 │ ╭─────────────┬────────────────╮                                                                                                                  │ │
│                   │ │ textFields      │ │ description │ It's some TOML │                                                                                                                  │ │
│                   │ │                 │ ╰─────────────┴────────────────╯                                                                                                                  │ │
│                   │ │                 │ ╭───┬───────┬──────────┬──────────────────────────┬───────────────────────────┬───────────────────────────────────────────┬────────────────╮      │ │
│                   │ │ files           │ │ # │ name  │ fileName │       Content-Type       │ Content-Transfer-Encoding │            Content-Disposition            │ Content-Length │      │ │
│                   │ │                 │ ├───┼───────┼──────────┼──────────────────────────┼───────────────────────────┼───────────────────────────────────────────┼────────────────┤      │ │
│                   │ │                 │ │ 0 │ cargo │ cargo    │ application/octet-stream │ binary                    │ form-data; name="cargo"; filename="cargo" │ 9101           │      │ │
│                   │ │                 │ ╰───┴───────┴──────────┴──────────────────────────┴───────────────────────────┴───────────────────────────────────────────┴────────────────╯      │ │
│                   │ ╰─────────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ │
╰───────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
```
# User-Facing Changes
`http post --content-type multipart/form-data` now accepts a record
which is uploaded as `multipart/form-data`.
Binary data is uploaded as files (`application/octet-stream`),
everything else is uploaded as plain text.

Previously `http post --content-type multipart/form-data` rejected
records, so there's no BC break.

# Tests + Formatting

Added.

# After Submitting
- [ ] update docs to showcase new functionality
2024-08-06 15:28:38 -05:00
Jack Wright
73e8de9753
Attempt to guess the content type of a file when opening with --raw (#13521)
# Description
Attempt to guess the content type of a file when opening with --raw and
set it in the pipeline metadata.

<img width="644" alt="Screenshot 2024-08-02 at 11 30 10"
src="https://github.com/user-attachments/assets/071f0967-c4dd-405a-b8c8-f7aa073efa98">


# User-Facing Changes
- Content of files can be directly piped into commands like `http post`
with the content type set appropriately when using `--raw`.
2024-08-06 11:36:24 +02:00
Ian Manske
f4c0d9d45b
Path migration part 4: various tests (#13373)
# Description
Part 4 of replacing std::path types with nu_path types added in
https://github.com/nushell/nushell/pull/13115. This PR migrates various
tests throughout the code base.
2024-08-03 10:09:13 +02:00
Stefan Holderbach
42531e017c
Clippy fixes from stable and nightly (#13455)
- **Doccomment style fixes**
- **Forgotten stuff in `nu-pretty-hex`**
- **Don't `for` around an `Option`**
- and more

I think the suggestions here are a net positive, some of the suggestions
moved into #13498 feel somewhat arbitrary, I also raised
https://github.com/rust-lang/rust-clippy/issues/13188 as the nightly
`byte_char_slices` would require either a global allow or otherwise a
ton of granular allows or possibly confusing bytestring literals.
2024-07-31 20:37:40 +02:00
Wind
928c57db41
save: print to stderr for bytestream (#13422)
# Description
Fixes: #13260 

When user run a command like this:
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```

`save` command sinks the output of previous commands' stderr output. I
think it should be `stderr`.

# User-Facing Changes
```nushell
$env.FOO = " New";
$env.BAZ = " New Err";
do -i {nu -n -c 'nu --testbin echo_env FOO; nu --testbin echo_env_stderr BAZ'} | save -a -r save_test_22/log.txt
```
The command will output ` New Err` to stderr.

# Tests + Formatting
Added 2 cases.
2024-07-31 18:19:35 +02:00
Stefan Holderbach
813aac89bd
Clippy fixes for toolchain bump (#13497)
- **Suggested default impl for the new `*Stack`s**
- **Change a hashmap to make clippy happy**
- **Clone from fix**
- **Fix conditional unused in test**
- then **Bump rust toolchain**
2024-07-31 14:49:22 +02:00
Devyn Cairns
8e2917b9ae
Make assignment and const consistent with let/mut (#13385)
# Description

This makes assignment operations and `const` behave the same way `let`
and `mut` do, absorbing the rest of the pipeline.

Changes the lexer to be able to recognize assignment operators as a
separate token, and then makes the lite parser continue to push spans
into the same command regardless of any redirections or pipes if an
assignment operator is encountered. Because the pipeline is no longer
split up by the lite parser at this point, it's trivial to just parse
the right hand side as if it were a subexpression not contained within
parentheses.

# User-Facing Changes
Big breaking change. These are all now possible:

```nushell
const path = 'a' | path join 'b'

mut x = 2
$x = random int
$x = [1 2 3] | math sum

$env.FOO = random chars
```

In the past, these would have led to (an attempt at) bare word string
parsing. So while `$env.FOO = bar` would have previously set the
environment variable `FOO` to the string `"bar"`, it now tries to run
the command named `bar`, hence the major breaking change.

However, this is desirable because it is very consistent - if you see
the `=`, you can just assume it absorbs everything else to the right of
it.

# Tests + Formatting
Added tests for the new behaviour. Adjusted some existing tests that
depended on the right hand side of assignments being parsed as
barewords.

# After Submitting
- [ ] release notes (breaking change!)
2024-07-30 18:55:22 -05:00
Wind
e281c03403
generate: switch the position of <initial> and <closure>, so the closure can have default parameters (#13393)
# Description
Close: #12083 
Close: #12084 

# User-Facing Changes
It's a breaking change because we have switched the position of
`<initial>` and `<closure>`, after the change, initial value will be
optional. So it's possible to do something like this:

```nushell
> let f = {|fib = [0, 1]| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} }
> generate $f | first 5
╭───┬───╮
│ 0 │ 0 │
│ 1 │ 1 │
│ 2 │ 1 │
│ 3 │ 2 │
│ 4 │ 3 │
╰───┴───╯
```

It will also raise error if user don't give initial value, and the
closure don't have default parameter.
```nushell
❯ let f = {|fib| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} }
❯ generate $f
Error:   × The initial value is missing
   ╭─[entry #5:1:1]
 1 │ generate $f
   · ────┬───
   ·     ╰── Missing intial value
   ╰────
  help: Provide <initial> value in generate, or assigning default value to closure parameter

```

# Tests + Formatting
Added some test cases.

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2024-07-19 00:22:28 -07:00
Wind
e8764de3c6
don't allow break/continue in each and items command (#13398)
# Description
Fixes: #11451

# User-Facing Changes
### Before
```nushell
❯ [1 2 3] | each {|e| break; print $e }
╭────────────╮
│ empty list │
╰────────────╯
```

### After
```
❯ [1 2 3] | each {|e| break; print $e }
Error: nu:🐚:eval_block_with_input

  × Eval block failed with pipeline input
   ╭─[entry #9:1:2]
 1 │ [1 2 3] | each {|e| break; print $e }
   ·  ┬
   ·  ╰── source value
   ╰────

Error:   × Break used outside of loop
   ╭─[entry #9:1:21]
 1 │ [1 2 3] | each {|e| break; print $e }
   ·                     ──┬──
   ·                       ╰── used outside of loop
   ╰────
```

# Tests + Formatting
Removes some tests.
2024-07-19 00:21:02 -07:00
Ian Manske
c19944f291
Refactor window (#13401)
# Description
Following from #13377, this PR refactors the related `window` command.
In the case where `window` should behave exactly like `chunks`, it now
reuses the same code that `chunks` does. Otherwise, the other cases have
been rewritten and have resulted in a performance increase:

| window size | stride | old time (ms) | new time (ms) |
| -----------:| ------:| -------------:| -------------:|
| 20          | 1      | 757           | 722           |
| 2           | 1      | 364           | 333           |
| 1           | 1      | 343           | 293           |
| 20          | 20     | 90            | 63            |
| 2           | 2      | 215           | 175           |
| 20          | 30     | 74            | 60            |
| 2           | 4      | 141           | 110           |


# User-Facing Changes
`window` will now error if the window size or stride is 0, which is
technically a breaking change.
2024-07-19 04:16:09 +00:00
Devyn Cairns
aa7d7d0cc3
Overhaul $in expressions (#13357)
# Description

This grew quite a bit beyond its original scope, but I've tried to make
`$in` a bit more consistent and easier to work with.

Instead of the parser generating calls to `collect` and creating
closures, this adds `Expr::Collect` which just evaluates in the same
scope and doesn't require any closure.

When `$in` is detected in an expression, it is replaced with a new
variable (also called `$in`) and wrapped in `Expr::Collect`. During
eval, this expression is evaluated directly, with the input and with
that new variable set to the collected value.

Other than being faster and less prone to gotchas, it also makes it
possible to typecheck the output of an expression containing `$in`,
which is nice. This is a breaking change though, because of the lack of
the closure and because now typechecking will actually happen. Also, I
haven't attempted to typecheck the input yet.

The IR generated now just looks like this:

```gas
collect        %in
clone          %tmp, %in
store-variable $in, %tmp
# %out <- ...expression... <- %in
drop-variable  $in
```

(where `$in` is the local variable created for this collection, and not
`IN_VARIABLE_ID`)

which is a lot better than having to create a closure and call `collect
--keep-env`, dealing with all of the capture gathering and allocation
that entails. Ideally we can also detect whether that input is actually
needed, so maybe we don't have to clone, but I haven't tried to do that
yet. Theoretically now that the variable is a unique one every time, it
should be possible to give it a type - I just don't know how to
determine that yet.

On top of that, I've also reworked how `$in` works in pipeline-initial
position. Previously, it was a little bit inconsistent. For example,
this worked:

```nushell
> 3 | do { let x = $in; let y = $in; print $x $y }
3
3
```

However, this causes a runtime variable not found error on the second
`$in`:

```nushell
> def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
Error: nu:🐚:variable_not_found

  × Variable not found
   ╭─[entry #115:1:35]
 1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
   ·                                   ─┬─
   ·                                    ╰── variable not found
   ╰────
```

I've fixed this by making the first element `$in` detection *always*
happen at the block level, so if you use `$in` in pipeline-initial
position anywhere in a block, it will collect with an implicit
subexpression around the whole thing, and you can then use that `$in`
more than once. In doing this I also rewrote `parse_pipeline()` and
hopefully it's a bit more straightforward and possibly more efficient
too now.

Finally, I've tried to make `let` and `mut` a lot more straightforward
with how they handle the rest of the pipeline, and using a redirection
with `let`/`mut` now does what you'd expect if you assume that they
consume the whole pipeline - the redirection is just processed as
normal. These both work now:

```nushell
let x = ^foo err> err.txt
let y = ^foo out+err>| str length
```

It was previously possible to accomplish this with a subexpression, but
it just seemed like a weird gotcha that you couldn't do it. Intuitively,
`let` and `mut` just seem to take the whole line.

- closes #13137

# User-Facing Changes
- `$in` will behave more consistently with blocks and closures, since
the entire block is now just wrapped to handle it if it appears in the
first pipeline element
- `$in` no longer creates a closure, so what can be done within an
expression containing `$in` is less restrictive
- `$in` containing expressions are now type checked, rather than just
resulting in `any`. However, `$in` itself is still `any`, so this isn't
quite perfect yet
- Redirections are now allowed in `let` and `mut` and behave pretty much
how you'd expect

# Tests + Formatting
Added tests to cover the new behaviour.

# After Submitting
- [ ] release notes (definitely breaking change)
2024-07-17 16:02:42 -05:00
Maxim Zhiburt
5417c89387
Fix issue with truncation when head on border is used (#13389)
close #13365

@fdncred thanks for reproducible

hopefully there won't be issues with it after this one 😅
2024-07-16 16:21:42 -05:00
Jan Christian Grünhage
b66671d339
Switch from dirs_next 2.0 to dirs 5.0 (#13384)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Replaces the `dirs_next` family of crates with `dirs`. `dirs_next` was
born when the `dirs` crates were abandoned three years ago, but they're
being maintained again and most projects depend on `dirs` nowadays.
`dirs_next` has been abandoned since.

This came up while working on
https://github.com/nushell/nushell/pull/13382.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
None.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
Tests and formatter have been run.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-07-16 07:16:26 -05:00
Bruce Weirdan
fcb8e36caa
Remove default list-diving behaviour (#13386)
# Description

Before this change `default` would dive into lists and replace `null`
elements with the default value.
Therefore there was no easy way to process `null|list<null|string>`
input and receive `list<null|string>`
(IOW to apply the default on the top level only).

However it's trivially easy to apply default values to list elements
with the `each` command:
```nushell
[null, "a", null] | each { default "b" }
```

So there's no need to have this behavior in `default` command.

# User-Facing Changes

* `default` no longer dives into lists to replace `null` elements with
the default value.

# Tests + Formatting

Added a couple of tests for the new (and old) behavior.

# After Submitting

* Update docs.
2024-07-16 03:54:24 +00:00
Ian Manske
0918050ac8
Deprecate group in favor of chunks (#13377)
# Description
The name of the `group` command is a little unclear/ambiguous.
Everything I look at it, I think of `group-by`. I think `chunks` more
clearly conveys what the `group` command does. Namely, it divides the
input list into chunks of a certain size. For example,
[`slice::chunks`](https://doc.rust-lang.org/std/primitive.slice.html#method.chunks)
has the same name. So, this PR adds a new `chunks` command to replace
the now deprecated `group` command.

The `chunks` command is a refactored version of `group`. As such, there
is a small performance improvement:
```nushell
# $data is a very large list
> bench { $data | chunks 2 } --rounds 30 | get mean
474ms 921µs 190ns

# deprecation warning was disabled here for fairness
> bench { $data | group 2 } --rounds 30 | get mean
592ms 702µs 440ns



> bench { $data | chunks 200 } --rounds 30 | get mean
374ms 188µs 318ns

> bench { $data | group 200 } --rounds 30 | get mean
481ms 264µs 869ns 



> bench { $data | chunks 1 } --rounds 30 | get mean
642ms 574µs 42ns

> bench { $data | group 1 } --rounds 30 | get mean
981ms 602µs 513ns
```

# User-Facing Changes
- `group` command has been deprecated in favor of new `chunks` command.
- `chunks` errors when given a chunk size of `0` whereas `group` returns
chunks with one element.

# Tests + Formatting
Added tests for `chunks`, since `group` did not have any tests.

# After Submitting
Update book if necessary.
2024-07-16 03:49:00 +00:00
Stefan Holderbach
f5bff8c9c8
Fix select cell path renaming behavior (#13361)
# Description

Fixes #13359

In an attempt to generate names for flat columns resulting from a nested
accesses #3016 generated new column names on nested selection, out of
convenience, that composed the cell path as a string (including `.`) and
then simply replaced all `.` with `_`. As we permit `.` in column names
as long as you quote this surprisingly alters `select`ed columns.


# User-Facing Changes
New columns generated by selection with nested cell paths will for now
be named with a string containing the keys separated by `.` instead of
`_`. We may want to reconsider the semantics for nested access.

# Tests + Formatting
- Alter test to breaking change on nested `select`
2024-07-13 16:54:34 +02:00