Commit Graph

92 Commits

Author SHA1 Message Date
Jack Wright
525acf9d9e
Ability to cast a dataframe's column to a different dtype (#11803)
Provides the ability to cast columns in dataframes, lazy dataframes, and
expressions.

<img width="587" alt="Screenshot 2024-02-14 at 13 53 01"
src="https://github.com/nushell/nushell/assets/56345/b894f746-0e37-472e-9fb0-eb6f71f2bf27">

<img width="616" alt="Screenshot 2024-02-14 at 13 52 37"
src="https://github.com/nushell/nushell/assets/56345/cf10efa7-d89c-4189-ab71-d368b2354d19">

<img width="626" alt="Screenshot 2024-02-14 at 13 54 58"
src="https://github.com/nushell/nushell/assets/56345/cd57cdf0-5096-41dd-8ab5-46e3d1e061b8">

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-02-14 18:15:00 -06:00
nibon7
da4c918392
Bump polars from 0.36 to 0.37 (#11848)
# Description
Bump polars from 0.36 to 0.37

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-13 06:27:30 -06:00
dependabot[bot]
e7f1bf8535
Bump indexmap from 2.1.0 to 2.2.2 (#11746) 2024-02-08 12:31:41 +00:00
Darren Schroeder
08931e976e
bump to dev release of nushell 0.90.2 (#11793)
# Description

Bump nushell version to the dev version of 0.90.2

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-07 16:26:03 -06:00
Jakub Žádník
c2992d5d8b
Bump to 0.90.1 (#11787)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

Merge after https://github.com/nushell/nushell/pull/11786

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-06 16:28:49 -06:00
Jakub Žádník
f5f21aca2d
Bump to 0.90 (#11730)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-06 22:42:43 +02:00
Jakub Žádník
b8d37a7541
Fix panic in rotate; Add safe record creation function (#11718)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

Fixes https://github.com/nushell/nushell/issues/11716

The problem is in our [record creation
API](0d518bf813/crates/nu-protocol/src/value/record.rs (L33))
which panics if the numbers of columns and values are different. I added
a safe variant that returns a `Result` and used it in the `rotate`
command.

## TODO in another PR:

Go through all `from_raw_cols_vals_unchecked()` (this includes the
`record!` macro which uses the unchecked version) and make sure that
either
a) it is guaranteed the number of cols and vals is the same, or
b) convert the call to `from_raw_cols_vals()`

Reason: Nushell should never panic.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-02-03 13:23:16 +02:00
Darren Schroeder
d9b324c5d3
rollback polars 0.37.0 (#11695)
# Description

This PR rolls back the polars updates to 0.37.0 back to 0.36.2 since it
won't compile yet for some reason.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-31 08:39:38 -06:00
dependabot[bot]
c77ec0b667
Bump polars-arrow from 0.36.2 to 0.37.0 (#11660) 2024-01-30 22:26:42 +00:00
dependabot[bot]
6779b248e4
Bump sqlparser from 0.41.0 to 0.43.1 (#11662) 2024-01-30 22:26:31 +00:00
Jack Wright
175dab4898
"[11611] fixing dataframe column comparisons" (#11676)
fixes #11611

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-01-29 17:28:12 -06:00
Jack Wright
f879c00f9d
The ability to specify a schema when using dfr open and dfr into-df (#11634)
# Description

There are times where explicitly specifying a schema for a dataframe is
needed such as:
- Opening CSV and JSON lines files and needing provide more information
to polars to keep it from failing or in a desire to override default
type conversion
- When converting a nushell value to a dataframe and wanting to override
the default conversion behaviors.

This pull requests provides:
- A flag to allow specifying a schema when using dfr into-df
- A flag to allow specifying a schema when using dfr open that works for
CSV and JSON types
- A new command `dfr schema` which displays schema information and will
allow display support schema dtypes

Schema is specified creating a record that has the key value and the
dtype. Examples usages:

```
{a:1, b:{a:2}} | dfr into-df -s {a: u8, b: {a: i32}} | dfr schema
{a: 1, b: {a: [1 2 3]}, c: [a b c]} | dfr into-df -s {a: u8, b: {a: list<u64>}, c: list<str>} | dfr schema
 dfr open -s {pid: i32, ppid: i32, name: str, status: str, cpu: f64, mem: i64, virtual: i64} /tmp/ps.jsonl  | dfr schema
```

Supported dtypes:
null                                                   
bool                                                   
u8                                                     
u16                                                    
u32                                                    
u64                                                    
i8                                                     
i16                                                    
i32                                                    
i64                                                    
f32                                                    
f64                                                    
str                                                    
binary                                                 
date                                                   
datetime[time_unit: (ms, us, ns) timezone (optional)]  
duration[time_unit: (ms, us, ns)]                      
time                                                   
object                                                 
unknown                                                
list[dtype]


structs are also supported but are specified via another record:
{a: u8, b: {d: str}}

Another feature with the dfr schema command is that it returns the data
back in a format that can be passed to provide a valid schema that can
be passed in as schema argument:

<img width="638" alt="Screenshot 2024-01-29 at 10 23 58"
src="https://github.com/nushell/nushell/assets/56345/b49c3bff-5cda-4c86-975a-dfd91d991373">

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-01-29 13:26:04 -06:00
dependabot[bot]
7e37e4bb5f Bump sqlparser from 0.39.0 to 0.41.0
Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.39.0 to 0.41.0.
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.39.0...v0.41.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-01-24 15:49:09 +00:00
nibon7
a44ad949f1
Bump polars from 0.35 to 0.36 (#11624)
# Description
* release notes
 https://github.com/pola-rs/polars/releases/tag/rs-0.36.2

* dependencies 
remove `sysinfo` 0.29.11
add `polars-compute` 0.36.2

# User-Facing Changes
[Change value_counts resulting column name from counts to
count](https://github.com/pola-rs/polars/pull/12506)

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-24 09:27:06 -06:00
Artemiy
1867bb1a88
Fix incorrect handling of boolean flags for builtin commands (#11492)
# Description
Possible fix of #11456
This PR fixes a bug where builtin commands did not respect the logic of
dynamically passed boolean flags. The reason is
[has_flag](6f59abaf43/crates/nu-protocol/src/ast/call.rs (L204C5-L212C6))
method did not evaluate and take into consideration expression used with
flag.

To address this issue a solution is proposed:
1. `has_flag` method is moved to `CallExt` and new logic to evaluate
expression and check if it is a boolean value is added
2. `has_flag_const` method is added to `CallExt` which is a constant
version of `has_flag`
3. `has_named` method is added to `Call` which is basically the old
logic of `has_flag`
4. All usages of `has_flag` in code are updated, mostly to pass
`engine_state` and `stack` to new `has_flag`. In `run_const` commands it
is replaced with `has_flag_const`. And in a few select places: parser,
`to nuon` and `into string` old logic via `has_named` is used.

# User-Facing Changes
Explicit values of boolean flags are now respected in builtin commands.
Before:

![image](https://github.com/nushell/nushell/assets/17511668/f9fbabb2-3cfd-43f9-ba9e-ece76d80043c)
After:

![image](https://github.com/nushell/nushell/assets/17511668/21867596-2075-437f-9c85-45563ac70083)

Another example:
Before:

![image](https://github.com/nushell/nushell/assets/17511668/efdbc5ca-5227-45a4-ac5b-532cdc2bbf5f)
After:

![image](https://github.com/nushell/nushell/assets/17511668/2907d5c5-aa93-404d-af1c-21cdc3d44646)


# Tests + Formatting
Added test reproducing some variants of original issue.
2024-01-11 17:19:48 +02:00
Jakub Žádník
7bb9ee55c4
Bump to dev version 0.89.1 (#11513)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-11 00:19:21 +13:00
Jakub Žádník
2c1560e281
Bump version for 0.89.0 release (#11511)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

- [x] reedline
  - [x] released
  - [x] pinned
- [ ] git dependency check
- [ ] release notes


# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-01-09 22:16:29 +02:00
Stefan Holderbach
7ad0e5541e
Make polars deps optional for cargo test --all (#11415)
As `--workspace/--all` pulls in all crates in the workspace for `cargo
test --workspace` let's make sure that the `polars` family of
dependencies are also feature gated so they only build for `--features
dataframe`. The test modules themselves also depend on the feature.

Should speed up a bare `cargo test --workspace`
2023-12-24 13:12:31 +01:00
Stefan Holderbach
8cfa96b4c0
Construct Records only through checked helpers (#11386)
# Description

Constructing the internals of `Record` without checking the lengths is
bad. (also incompatible with changes to how we store records)

- Use `Record::from_raw_cols_vals` in dataframe code
- Use `record!` macro in dataframe test
- Use `record!` in `nu-color-config` tests
- Stop direct record construction in `nu-command`
- Refactor table construction in `from nuon`

# User-Facing Changes
None

# Tests + Formatting
No new tests, updated tests in equal fashion
2023-12-21 16:48:15 +01:00
Jack Wright
44dc890124
Polars Struct support without unsafe blocks (#11229)
Second attempt at polars Struct support. This version avoid using unsafe
checks by cloning the StructArray and utilizing the into_static to
convert to a StructOwned.

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2023-12-15 11:21:30 +01:00
Stefan Holderbach
c2b684464f
Bump version to 0.88.2 (#11333) 2023-12-14 13:55:48 -06:00
Stefan Holderbach
fd56768fdc
Bump version to 0.88.1 (#11303) 2023-12-14 18:14:47 +01:00
Stefan Holderbach
d43f4253e8
Bump version for 0.88.0 release (#11298)
- [x] reedline
  - [x] released
  - [x] pinned
- [x] git dependency check
- [x] release notes
2023-12-13 06:31:14 +13:00
Eric Hodel
ecb3b3a364
Ensure that command usage starts uppercase and ends period (#11278)
# Description

This repeats #8268 to make all command usage strings start with an
uppercase letter and end with a period per #5056

Adds a test to ensure that commands won't regress

Part of #5066

# User-Facing Changes

Command usage is now consistent

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

Automatic documentation updates
2023-12-10 08:28:54 -06:00
Eric Hodel
a95a4505ef
Convert Shellerror::GenericError to named fields (#11230)
# Description

Replace `.to_string()` used in `GenericError` with `.into()` as
`.into()` seems more popular

Replace `Vec::new()` used in `GenericError` with `vec![]` as `vec![]`
seems more popular

(There are so, so many)
2023-12-07 00:40:03 +01:00
Jack Wright
31146a7591
Upgrading to polars 0.35 (#11241)
Co-authored-by: Jack Wright <jack.wright@disqo.com>
2023-12-05 18:09:34 -06:00
Eric Hodel
67eec92e76
Convert more ShellError variants to named fields (#11222)
# Description

Convert errors to named fields:
* NeedsPositiveValue
* MissingConfigValue
* UnsupportedConfigValue
* DowncastNotPossible
* NonUtf8Custom
* NonUtf8
* DidYouMeanCustom
* DidYouMean
* ReadingFile
* RemoveNotPossible
* ChangedModifiedTimeNotPossible
* ChangedAccessTimeNotPossible

Part of #10700
2023-12-04 10:19:32 +01:00
Stefan Holderbach
112306aab5
Revert "Adding support for Polars structs" (#11171)
Reverts nushell/nushell#10943

The current implementation of `arr_to_value` is unsound, as it allows
casts of arbitrary data to arbitrary types without being marked
`unsafe`.
The full safety requirements to perform both the cast and the following
unchecked access are not as clear that a simple change of `fn
arr_to_value` to `unsafe fn arr_to_value` could be blessed without
further investigation.

cc @ayax79
2023-11-29 16:33:27 +01:00
Eric Hodel
e36f69bf3c
Convert FileNotFoundCustom to named fields (#11123)
# Description

Part of #10700

# User-Facing Changes

None

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A
2023-11-21 17:30:21 -06:00
Stefan Holderbach
adfa4d00c0
Bump version to 0.87.2 (#11114)
Based on the hotfix
https://github.com/nushell/nushell/releases/tag/0.87.1 use this to
disambiguate
2023-11-20 20:31:10 +01:00
nibon7
f41c93b2d3
Apply nightly clippy fixes (#11083)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
Clippy fixes for rust 1.76.0-nightly

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

N/A
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-11-17 09:15:55 -06:00
Stefan Holderbach
2b5f1ee5b3
Bump version to 0.87.1 (#11056) 2023-11-15 23:50:11 +01:00
Stefan Holderbach
77a1c3c7b2
Bump version for 0.87.0 release (#11031)
# Release checklist

- [x] reedline
  - [x] released
  - [x] pinned
- [x] crate graph check
- [x] release notes
- [x] release script update (new crate `nu-lsp`)
- [ ] permission management `nu-lsp` on crates.io
2023-11-14 21:01:19 +01:00
Jack Wright
fe92051bb3
Adding support for Polars structs (#10943)
Provides support for reading Polars structs. This allows opening of
supported files (jsonl, parquet, etc) that contain rows with structured
data.

The following attached json lines
file([receipts.jsonl.gz](https://github.com/nushell/nushell/files/13311476/receipts.jsonl.gz))
contains a customer column with structured data. This json lines file
can now be loaded via `dfr open` and will render as follows:

<img width="525" alt="Screenshot 2023-11-09 at 10 09 18"
src="https://github.com/nushell/nushell/assets/56345/4b26ccdc-c230-43ae-a8d5-8af88a1b72de">


This also addresses some cleanup of date handling and utilizing
timezones where provided.

This pull request only addresses reading data from polars structs. I
will address converting nushell data to polars structs in a future
request as this change is large enough as it is.

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2023-11-09 19:00:59 -06:00
Christopher Durham
0f600bc3f5
Improve case insensitivity consistency (#10884)
# Description

Add an extension trait `IgnoreCaseExt` to nu_utils which adds some case
insensitivity helpers, and use them throughout nu to improve the
handling of case insensitivity. Proper case folding is done via unicase,
which is already a dependency via mime_guess from nu-command.

In actuality a lot of code still does `to_lowercase`, because unicase
only provides immediate comparison and doesn't expose a `to_folded_case`
yet. And since we do a lot of `contains`/`starts_with`/`ends_with`, it's
not sufficient to just have `eq_ignore_case`. But if we get access in
the future, this makes us ready to use it with a change in one place.

Plus, it's clearer what the purpose is at the call site to call
`to_folded_case` instead of `to_lowercase` if it's exclusively for the
purpose of case insensitive comparison, even if it just does
`to_lowercase` still.

# User-Facing Changes

- Some commands that were supposed to be case insensitive remained only
insensitive to ASCII case (a-z), and now are case insensitive w.r.t.
non-ASCII characters as well.

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2023-11-08 23:58:54 +01:00
Eric Hodel
7a3cbf43e8
Convert ShellError::UnsupportedInput to named fields (#10971)
# Description

This is easy to do with rust-analyzer, but I didn't want to just pump
these all out without feedback.

Part of #10700

# User-Facing Changes

None

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

N/A

---------

Co-authored-by: Stefan Holderbach <sholderbach@users.noreply.github.com>
2023-11-07 23:25:32 +01:00
dependabot[bot]
45b02ce2ab
Bump indexmap from 2.0.2 to 2.1.0 (#10969) 2023-11-07 15:36:34 +00:00
Ian Manske
15c22db8f4
Make FromValue take owned Values (#10900)
# Description
Changes `FromValue` to take owned `Value`s instead of borrowed `Value`s.
This eliminates some unnecessary clones (e.g., in `call_ext.rs`).

# User-Facing Changes
Breaking API change for `nu_protocol`.
2023-10-31 19:47:00 +01:00
Stefan Holderbach
4b301710d3
Convert more examples and tests to record! macro (#10840)
# Description
Use `record!` macro instead of defining two separate `vec!` for `cols`
and `vals` when appropriate.
This visually aligns the key with the value.
Further more you don't have to deal with the construction of `Record {
cols, vals }` so we can hide the implementation details in the future.

## State

Not covering all possible commands yet, also some tests/examples are
better expressed by creating cols and vals separately.

# User/Developer-Facing Changes
The examples and tests should read more natural. No relevant functional
change

# Bycatch

Where I noticed it I replaced usage of `Value` constructors with
`Span::test_data()` or `Span::unknown()` to the `Value::test_...`
constructors. This should make things more readable and also simplify
changes to the `Span` system in the future.
2023-10-28 14:52:31 +02:00
Jack Wright
c6016d7659
Dataframe support for small int types (#10828)
Turned features to allow signed and unsigned 8 and 16 bit types.

---------

Co-authored-by: Jack Wright <jack.wright@disqo.com>
2023-10-24 21:25:21 -05:00
Stefan Holderbach
88a87158c2
Bump version to 0.86.1 (#10755)
To dev or to patch that is the question
2023-10-18 13:00:51 -05:00
Stefan Holderbach
5d8763ed1d
Bump version for 0.86.0 release (#10726)
## Release checklist:

- [x] `uu_cp` on crates.io #10725
- [x] new `reedline` released and used nushell/reedline#645
- [x] check of workspace dependency DAG
- [x] release notes ready:
https://github.com/nushell/nushell.github.io/pull/1071
2023-10-18 06:08:20 +13:00
Stefan Holderbach
c925537c48
Update polars to 0.33 (#10672)
# Description
Open question:

Undocumented behavior for the new argument `ambiguous` to the
`as_datetime`
methods. I cheated by passing a default (assuming empty string).
This appears like an API primarily serving the python impl:


https://pola-rs.github.io/polars/py-polars/html/reference/expressions/api/polars.Expr.str.to_datetime.html#polars-expr-str-to-datetime


# User-Facing Changes
Only dependent on breaking changes to the behavior of polars.

# Tests + Formatting
No observed changes to tests

Manually checked `dfr as-datetime`, doesn't seem to panic.
2023-10-11 21:28:18 +02:00
Stefan Holderbach
81ece18d5e
Add a stub dfr command (#10683)
# Description
This will only display the list of subcommands.

Prompted by a question on Discord why completions may be missing.
With standard completion settings getting the subcommands doesn't seem
to be a problem but we could add this command for good measure.

# User-Facing Changes
New command `dfr` that does nothing apart from displaying the
subcommands and hogging a space in the completions

# Tests + Formatting
(-)
2023-10-11 17:51:20 +02:00
Hofer-Julian
7dbda76fad
Add long options for core and dataframes (#10619) 2023-10-06 18:55:29 +02:00
Stefan Holderbach
af15f794b4
Bump to 0.85.1 development version (#10431) 2023-09-20 18:38:42 +12:00
Stefan Holderbach
a6f62e05ae
Bump version for the 0.85 release (#10425) 2023-09-19 21:42:47 +03:00
Darren Schroeder
a9a82de5c4
fix some new chrono warnings (#10384)
# Description

This PR cleans up some warnings on the latest chrono dependency.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2023-09-15 15:46:25 -05:00
Stefan Holderbach
bbf0b45c59
Update internal use of decimal to float (#10333)
# Description
We made the decision that our floating point type should be referred to
as `float` over `decimal`.
Commands were updated by #9979 and #10320

Now make the internal codebase consistent in referring to this data type
as `float`.

Work for #10332

# User-Facing Changes

`decimal` has been removed as a type name/symbol. 

Instead of 
```nushell
def foo [bar: decimal] decimal -> decimal {}
```
use 
```nushell
def foo [bar: float] float -> float {}
```

Potential effect of `SyntaxShape`'s `Display` implementation now also
referring to `float` instead of `decimal`

# Details
- Rename `SyntaxShape::Decimal` to `Float`
- Update `Display for SyntaxShape` to `float`
- Update error message + fn name in dataframe code
- Fix docs in command examples
- Rename tests that are float specific
- Update doccomment on `SyntaxShape`
- Update comment in script

# Tests + Formatting
Updates the names of some tests
2023-09-13 23:53:55 +02:00
Stefan Holderbach
a14e9e0a2e
Invert &Options to Option<&T> (#10315)
Elide the reference for `Copy` type (`usize`)
Use the canonical deref where possible.
* `&Box` -> `&`
* `&String` -> `&str`
* `&PathBuf` -> `&Path`

Skips the ctrl-C handler for now.
2023-09-13 07:00:58 +08:00