mirror of
https://github.com/nushell/nushell.git
synced 2025-07-09 02:48:17 +02:00
main
216 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| 05e570aa71 |
polars: fix datetime type conversion (#16133)
# Description Conversion from `AnyType::DatetimeOwned` was missing, so datetime objects could not be represented in Nushell Values. This adds the conversion. A slight refactor of the `datetime_from_epoch_nanos` was needed. # User-Facing Changes All datetime types can be represented in Nushell. |
|||
| d8255040f1 |
Added flag limit for polars arg-sort (#16132)
# Description Exposes the polars sort option limit for the arg-sort command. ```nu > ❯ : [1 2 2 3 3] | polars into-df | polars arg-sort --limit 2 ╭───┬──────────╮ │ # │ arg_sort │ ├───┼──────────┤ │ 0 │ 0 │ │ 1 │ 1 │ ╰───┴──────────╯ ``` # User-Facing Changes - The `--limit` flag is now available for `polars arg-sort` Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| 2c1b787db5 | build(deps): bump indexmap from 2.9.0 to 2.10.0 (#16087) | |||
| 9da0f41ebb |
Fix easy clippy lints from latest stable (#16053)
1.88.0 was released today, clippy now lints (machine-applicable) against: - format strings with empty braces that could be inlined - easy win - `manual_abs_diff` - returning of a stored result of the last expression. - this can be somewhat contentious but touched only a few places |
|||
| e88a6bff60 |
polars 0.49 upgrade (#16031)
# Description Polars 0.49 upgrade Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| fb691c0da5 |
Allow polars schema --datatype-list to be used without pipeline input (#15964)
# Description Fixes the issue of listing allowed datatypes when not being used with dataframe pipeline input. Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| 7972aea530 |
Make polars last consistent with polars first (#15963)
# Description `polars last` will only return one row by default making it consistent with `polars first` # User-Facing Changes - `polars last` will only return one row by default making it consistent with `polars first` Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| aa710eeb9a |
Add groupby support for polars last (#15953)
# Description
Allows `polars last` to be used with group-by
```nu
> ❯ : [[a b c d]; [1 0.5 true Apple] [2 0.5 true Orange] [2 4 true Apple] [3 10 false Apple] [4 13 false Banana] [5 14 true Banana]] | polars into-df -s {a: u8, b: f32, c: bool, d: str} | polars group-by d | polars last | polars sort-by [a] | polars collect
╭───┬────────┬───┬───────┬───────╮
│ # │ d │ a │ b │ c │
├───┼────────┼───┼───────┼───────┤
│ 0 │ Orange │ 2 │ 0.50 │ true │
│ 1 │ Apple │ 3 │ 10.00 │ false │
│ 2 │ Banana │ 5 │ 14.00 │ true │
╰───┴────────┴───┴───────┴───────╯
```
# User-Facing Changes
- `polars last` can now be used with group-by expressions
Co-authored-by: Jack Wright <jack.wright@nike.com>
|
|||
| ba59f71f20 |
bump to dev version 0.105.2 (#15952)
# Description Bump nushell to development version. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 1fe62ee613 | bump patch version | |||
| ebcdf5a8b1 |
Bump to 0.105.0 (#15930)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description Bump to 0.105.0 |
|||
| 8d46398e13 |
fix(polars): swap out pivot for pivot_stable to suppress warning message (#15913)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> The current implementation of `polars pivot` calls an unsupported version of pivot that throws a warning message in stdout (using println!) stating that "unstable pivot not yet unsupported, using stable pivot." This PR swaps out the call to `pivot` with a call to `pivot_stable`, which is being done in the underlying polars anyways. ```nushell # Current Implementation > [[a b c]; [1 x 10] [1 y 10] [2 x 11] [2 y 11]] | polars into-df | polars pivot -i [a] -o [b] -v [c] unstable pivot not yet supported, using stable pivot ╭───┬───┬────┬────╮ │ # │ a │ x │ y │ ├───┼───┼────┼────┤ │ 0 │ 1 │ 10 │ 10 │ │ 1 │ 2 │ 11 │ 11 │ # Proposed Implementation (no println! statement) > [[a b c]; [1 x 10] [1 y 10] [2 x 11] [2 y 11]] | polars into-df | polars pivot -i [a] -o [b] -v [c] ╭───┬───┬────┬────╮ │ # │ a │ x │ y │ ├───┼───┼────┼────┤ │ 0 │ 1 │ 10 │ 10 │ │ 1 │ 2 │ 11 │ 11 │ ╰───┴───┴────┴────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> None # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Current suite of tests were sufficient # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 21d949207f |
Add regex documentation/examples to polars col (#15898)
# Description Include regular expression example and help documentation to `polars col` Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| 4a9e2ac37b |
Creates col and nth expressions when using paths on lazy frames. (#15891)
# Description Instead of collecting the frame and returning the columns of the collected frame when using paths $df.column_name or $df.0 this creates expressions: ```nu > ❯ : let df = polars open /tmp/foo.parquet > ❯ : $df | polars select $df.pid | polars first 5 | polars collect ╭───┬───────╮ │ # │ pid │ ├───┼───────┤ │ 0 │ 45280 │ │ 1 │ 45252 │ │ 2 │ 45242 │ │ 3 │ 45241 │ │ 4 │ 45207 │ ╰───┴───────╯ > ❯ : $df | polars select $df.0 | polars first 5 | polars collect ╭───┬───────╮ │ # │ pid │ ├───┼───────┤ │ 0 │ 45280 │ │ 1 │ 45252 │ │ 2 │ 45242 │ │ 3 │ 45241 │ │ 4 │ 45207 │ ╰───┴───────╯ ``` --------- Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| bdc7cdbcc4 |
feat(polars): introducing new polars replace (#15706)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR seeks to port the polars command `replace` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.replace.html) and `replace_strict` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.replace_strict.html). See examples below. Consequently, the current `polars replace` and `polars replace-all` have been renamed to `polars str-replace` and `polars str-replace-all` to bring their naming better in-line with `polars str-join` and related str commands. ```nushell Usage: > polars replace {flags} <old> (new) Flags: -h, --help: Display the help message for this command -s, --strict: Require that all values must be replaced or throw an error (ignored if `old` or `new` are expressions). -d, --default <any>: Set values that were not replaced to this value. If no default is specified, (default), an error is raised if any values were not replaced. Accepts expression input. Non-expression inputs are parsed as literals. -t, --return-dtype <string>: Data type of the resulting expression. If set to `null` (default), the data type is determined automatically based on the other inputs. Parameters: old <one_of(record, list<any>)>: Values to be replaced new <list<any>>: Values to replace by (optional) Input/output types: ╭───┬────────────┬────────────╮ │ # │ input │ output │ ├───┼────────────┼────────────┤ │ 0 │ expression │ expression │ ╰───┴────────────┴────────────╯ Examples: Replace column with different values of same type > [[a]; [1] [1] [2] [2]] | polars into-df | polars select (polars col a | polars replace [1 2] [10 20]) | polars collect ╭───┬────╮ │ # │ a │ ├───┼────┤ │ 0 │ 10 │ │ 1 │ 10 │ │ 2 │ 20 │ │ 3 │ 20 │ ╰───┴────╯ Replace column with different values of another type > [[a]; [1] [1] [2] [2]] | polars into-df | polars select (polars col a | polars replace [1 2] [a b] --strict) | polars collect ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ a │ │ 1 │ a │ │ 2 │ b │ │ 3 │ b │ ╰───┴───╯ Replace column with different values based on expressions (cannot be used with strict) > [[a]; [1] [1] [2] [2]] | polars into-df | polars select (polars col a | polars replace [(polars col a | polars max)] [(polars col a | polars max | $in + 5)]) | polars collect ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ 1 │ │ 1 │ 1 │ │ 2 │ 7 │ │ 3 │ 7 │ ╰───┴───╯ Replace column with different values based on expressions with default > [[a]; [1] [1] [2] [3]] | polars into-df | polars select (polars col a | polars replace [1] [10] --default (polars col a | polars max | $in * 100) --strict) | polars collect ╭───┬─────╮ │ # │ a │ ├───┼─────┤ │ 0 │ 10 │ │ 1 │ 10 │ │ 2 │ 300 │ │ 3 │ 300 │ ╰───┴─────╯ Replace column with different values based on expressions with default > [[a]; [1] [1] [2] [3]] | polars into-df | polars select (polars col a | polars replace [1] [10] --default (polars col a | polars max | $in * 100) --strict --return-dtype str) | polars collect ╭───┬─────╮ │ # │ a │ ├───┼─────┤ │ 0 │ 10 │ │ 1 │ 10 │ │ 2 │ 300 │ │ 3 │ 300 │ ╰───┴─────╯ Replace column with different values using a record > [[a]; [1] [1] [2] [2]] | polars into-df | polars select (polars col a | polars replace {1: a, 2: b} --strict --return-dtype str) | polars collect ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ a │ │ 1 │ a │ │ 2 │ b │ │ 3 │ b │ ╰───┴───╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> **BREAKING CHANGE**: `polars replace` and `polars replace-all` have been renamed to `polars str-replace` and `polars str-replace-all`. The new `polars replace` now replaces elements in a series/column rather than patterns within strings. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example tests were added. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 2b524cd861 |
feat(polars): add maintain-order flag to polars group-by and allow expression inputs in polars filter (#15865)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR involves two changes: (1) adding `maintain-order` flag to `polars group-by` for stable sorting when aggregating and (2) allow expression inputs in `polars filter`. The first change was necessary to reliably test the second change, and the two commits are therefore combined in one PR. See example: ```nushell # Filter a single column in a group-by context > [[a b]; [foo 1] [foo 2] [foo 3] [bar 2] [bar 3] [bar 4]] | polars into-df | polars group-by a --maintain-order | polars agg { lt: (polars col b | polars filter ((polars col b) < 2) | polars sum) gte: (polars col b | polars filter ((polars col b) >= 3) | polars sum) } | polars collect ╭───┬─────┬────┬─────╮ │ # │ a │ lt │ gte │ ├───┼─────┼────┼─────┤ │ 0 │ foo │ 1 │ 3 │ │ 1 │ bar │ 0 │ 7 │ ╰───┴─────┴────┴─────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> An example test was added to `polars filter` demonstrating both the stable group-by feature and the expression filtering feature. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 2f74574e35 |
Fix for null handling #15788 (#15857)
Fixes #15788 # Description Fixes null handling. Thanks to @MMesch for reporting and taking a first stab at fixing. Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| 8b9f02246f |
Allow polars first to be used with polars group-by (#15855)
# Description Provides functionality similar to https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.dataframe.group_by.GroupBy.first.html by allowing polars first to be used with a group by ``` > ❯ : [[a b c d]; [1 0.5 true Apple] [2 0.5 true Orange] [2 4 true Apple] [3 10 false Apple] [4 13 false Banana] [5 14 true Banana]] | polars into-df -s {a: u8, b: f32, c: bool, d: str} | polars group-by d | polars first | polars collect ╭───┬────────┬───┬───────┬───────╮ │ # │ d │ a │ b │ c │ ├───┼────────┼───┼───────┼───────┤ │ 0 │ Apple │ 1 │ 0.50 │ true │ │ 1 │ Banana │ 4 │ 13.00 │ false │ │ 2 │ Orange │ 2 │ 0.50 │ true │ ╰───┴────────┴───┴───────┴───────╯ ``` Additionally, I am setting the POLARS_ALLOW_EXTENSION to true to avoid panicking with operations using the dtype object. The conversion will fallback to object when the type cannot be determining, so this could be a common case. # User-Facing Changes - `polars first` can now be used with `polars group-by` --------- Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| d9ecb7da93 |
Polars upgrade (#15852)
# Description Polars 0.48 upgrade # User-Facing Changes - (breaking change) Due to a change in behavior in polars, `polars is-in` now only works as an expression. --------- Co-authored-by: Jack Wright <jack.wright@nike.com> |
|||
| 90afb65329 |
feat(polars): expand polars shift to allow expressions inputs (#15834)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR seeks to expand the `polars shift` command to take expression inputs. See third example below: ```nushell Examples: Shifts the values by a given period > [1 2 2 3 3] | polars into-df | polars shift 2 | polars drop-nulls ╭───┬───╮ │ # │ 0 │ ├───┼───┤ │ 0 │ 1 │ │ 1 │ 2 │ │ 2 │ 2 │ ╰───┴───╯ Shifts the values by a given period, fill absent values with 0 > [1 2 2 3 3] | polars into-lazy | polars shift 2 --fill 0 | polars collect ╭───┬───╮ │ # │ 0 │ ├───┼───┤ │ 0 │ 0 │ │ 1 │ 0 │ │ 2 │ 1 │ │ 3 │ 2 │ │ 4 │ 2 │ ╰───┴───╯ Shift values of a column, fill absent values with 0 > [[a]; [1] [2] [2] [3] [3]] | polars into-lazy | polars with-column {b: (polars col a | polars shift 2 --fill 0)} | polars collect ╭───┬───┬───╮ │ # │ a │ b │ ├───┼───┼───┤ │ 0 │ 1 │ 0 │ │ 1 │ 2 │ 0 │ │ 2 │ 2 │ 1 │ │ 3 │ 3 │ 2 │ │ 4 │ 3 │ 2 │ ╰───┴───┴───╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> An example test was added to `polars shift` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 37bc922a67 |
feat(polars): add polars math expression (#15822)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds a number of math functions under a single `polars math` command that apply to one or more column expressions. Note, `polars math` currently resides in the new module dataframe/command/command/computation/math.rs. I'm open to alternative organization and naming suggestions. ```nushell Collection of math functions to be applied on one or more column expressions This is an incomplete implementation of the available functions listed here: https://docs.pola.rs/api/python/stable/reference/expressions/computation.html. The following functions are currently available: - abs - cos - dot <expression> - exp - log <base; default e> - log1p - sign - sin - sqrt Usage: > polars math <type> ...(args) Flags: -h, --help: Display the help message for this command Parameters: type <string>: Function name. See extra description for full list of accepted values ...args <any>: Extra arguments required by some functions Input/output types: ╭───┬────────────┬────────────╮ │ # │ input │ output │ ├───┼────────────┼────────────┤ │ 0 │ expression │ expression │ ╰───┴────────────┴────────────╯ Examples: Apply function to column expression > [[a]; [0] [-1] [2] [-3] [4]] | polars into-df | polars select [ (polars col a | polars math abs | polars as a_abs) (polars col a | polars math sign | polars as a_sign) (polars col a | polars math exp | polars as a_exp)] | polars collect ╭───┬───────┬────────┬────────╮ │ # │ a_abs │ a_sign │ a_exp │ ├───┼───────┼────────┼────────┤ │ 0 │ 0 │ 0 │ 1.000 │ │ 1 │ 1 │ -1 │ 0.368 │ │ 2 │ 2 │ 1 │ 7.389 │ │ 3 │ 3 │ -1 │ 0.050 │ │ 4 │ 4 │ 1 │ 54.598 │ ╰───┴───────┴────────┴────────╯ Specify arguments for select functions. See description for more information. > [[a]; [0] [1] [2] [4] [8] [16]] | polars into-df | polars select [ (polars col a | polars math log 2 | polars as a_base2)] | polars collect ╭───┬─────────╮ │ # │ a_base2 │ ├───┼─────────┤ │ 0 │ -inf │ │ 1 │ 0.000 │ │ 2 │ 1.000 │ │ 3 │ 2.000 │ │ 4 │ 3.000 │ │ 5 │ 4.000 │ ╰───┴─────────╯ Specify arguments for select functions. See description for more information. > [[a b]; [0 0] [1 1] [2 2] [3 3] [4 4] [5 5]] | polars into-df | polars select [ (polars col a | polars math dot (polars col b) | polars as ab)] | polars collect ╭───┬────────╮ │ # │ ab │ ├───┼────────┤ │ 0 │ 55.000 │ ╰───┴────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example tests were added to `polars math`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| ae51f6d722 |
fix(polars): add Value::Record to NuExpression::can_downcast logic (#15826)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Merged PR #15553 added the ability to provide expressions in the form of records. This PR conforms the `NuExpression::can_downcast` logic to account for the newly allowed records argument type. As such, commands that rely on `can_downcast` in their implementation (e.g., `polars with-column`) will no longer err when provided with a record. See example below: ```nushell # Current error > [[a b]; [1 2] [3 4]] | polars into-df <-- only works if cast as lazyframe | polars with-column { c: ((polars col a) * 2) d: ((polars col a) * 3) } Error: nu:🐚:cant_convert × Can't convert to NuDataFrame, NuLazyFrame, NuExpression, NuLazyGroupBy, NuWhen, │ NuDataType, NuSchema. ╭─[entry #24:3:26] 2 │ | polars into-df 3 │ ╭─▶ | polars with-column { 4 │ │ c: ((polars col a) * 2) 5 │ │ d: ((polars col a) * 3) 6 │ ├─▶ } · ╰──── can't convert record<c: NuExpression, d: NuExpression> to NuDataFrame, NuLazyFrame, NuExpression, NuLazyGroupBy, NuWhen, NuDataType, NuSchema ╰──── # Fixed > [[a b]; [1 2] [3 4]] | polars into-df | polars with-column { c: ((polars col a) * 2) d: ((polars col a) * 3) } | polars collect ╭───┬───┬───┬───┬───╮ │ # │ a │ b │ c │ d │ ├───┼───┼───┼───┼───┤ │ 0 │ 1 │ 2 │ 2 │ 3 │ │ 1 │ 3 │ 4 │ 6 │ 9 │ ╰───┴───┴───┴───┴───╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> An example test was added to `polars with-column`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 6bf955a5a5 |
Fix #15571 panic on write to source parquet file (#15601)
Fixes #15571 # Description Writing to a source `.parquet` (`polars open some_file.parquet | polars save some_file.parquet`) file made the plugin panic, added a guard to check the data_source path as per [this comment](https://github.com/nushell/nushell/issues/15571#issuecomment-2812707161) Example output now: <img width="850" alt="Screenshot 2025-04-18 at 21 10 30" src="https://github.com/user-attachments/assets/40a73cc7-6635-43dc-a423-19c7a0c8f59c" /> # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting - Add 1 test - clippy OK - cargo fmt OK # After Submitting No action required |
|||
| c10e483683 |
Bump dev version to 0.104.2 (#15809)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Bump dev version to 0.104.2 |
|||
| 833471241a |
Refactor: Construct IoError from std::io::Error instead of std::io::ErrorKind (#15777)
|
|||
| 457f162fd9 |
feat(polars): expand polars unique to allow expressions inputs (#15771)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> `polars unique` currently only operates on entire dataframes. This PR seeks to expand this command to handle expressions as well. See examples: ```nushell Returns unique values in a subset of lazyframe columns > [[a]; [2] [1] [2]] | polars into-lazy | polars select (polars col a | polars unique) | polars collect ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ 1 │ │ 1 │ 2 │ ╰───┴───╯ Returns unique values in a subset of lazyframe columns > [[a]; [2] [1] [2]] | polars into-lazy | polars select (polars col a | polars unique --maintain-order) | polars collect ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ 2 │ │ 1 │ 1 │ ╰───┴───╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users have the added option to use `polars unique` in an expressions context. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example tests have been added to `polars unique` # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| c2ac8f730e | Rust 1.85, edition=2024 (#15741) | |||
| 1a0986903f |
Minor DataType refactor (#15728)
# Description This is needed for the enum work. The recent polars changes have broken my enum work, so I am breaking it into smaller pull requests. |
|||
| b0d68c31e8 |
build(deps): bump tokio from 1.44.2 to 1.45.0 (#15710)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.44.2 to 1.45.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/tokio/releases">tokio's releases</a>.</em></p> <blockquote> <h2>Tokio v1.45.0</h2> <h3>Added</h3> <ul> <li>metrics: stabilize <code>worker_total_busy_duration</code>, <code>worker_park_count</code>, and <code>worker_unpark_count</code> (<a href="https://redirect.github.com/tokio-rs/tokio/issues/6899">#6899</a>, <a href="https://redirect.github.com/tokio-rs/tokio/issues/7276">#7276</a>)</li> <li>process: add <code>Command::spawn_with</code> (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7249">#7249</a>)</li> </ul> <h3>Changed</h3> <ul> <li>io: do not require <code>Unpin</code> for some trait impls (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7204">#7204</a>)</li> <li>rt: mark <code>runtime::Handle</code> as unwind safe (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7230">#7230</a>)</li> <li>time: revert internal sharding implementation (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7226">#7226</a>)</li> </ul> <h3>Unstable</h3> <ul> <li>rt: remove alt multi-threaded runtime (<a href="https://redirect.github.com/tokio-rs/tokio/issues/7275">#7275</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/tokio/issues/6899">#6899</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/6899">tokio-rs/tokio#6899</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7276">#7276</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7276">tokio-rs/tokio#7276</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7249">#7249</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7249">tokio-rs/tokio#7249</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7204">#7204</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7204">tokio-rs/tokio#7204</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7230">#7230</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7230">tokio-rs/tokio#7230</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7226">#7226</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7226">tokio-rs/tokio#7226</a> <a href="https://redirect.github.com/tokio-rs/tokio/issues/7275">#7275</a>: <a href="https://redirect.github.com/tokio-rs/tokio/pull/7275">tokio-rs/tokio#7275</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href=" |
|||
| ff8831318d |
Added polars struct-encode-json, providing the ability to encode structs as json (#15678)
# Description
This PR introduces `polars struct-encode-json`. This exposes the ability
to encode struct columns as json strings. This is useful when converting
things to formats like CSV that do not support complex types.
```nushell
> ❯ : [[id person]; [1 {name: "Bob", age: 36}] [2 {name: "Betty", age: 63}]]
| polars into-df -s {id: i64, person: {name: str, age: u8}}
| polars select id (polars col person | polars struct-json-encode | polars as encoded)
| polars collect
╭───┬────┬───────────────────────────╮
│ # │ id │ encoded │
├───┼────┼───────────────────────────┤
│ 0 │ 1 │ {"age":36,"name":"Bob"} │
│ 1 │ 2 │ {"age":63,"name":"Betty"} │
╰───┴────┴───────────────────────────╯
```
# User-Facing Changes
* Added `polars struct-encode-json`, providing the ability to encode
structs as json
|
|||
| ce582cdafb |
feat(polars): add polars horizontal aggregation command (#15656)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR seeks to port over the `*_horizontal` commands in polars rust/python (e.g., https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.sum_horizontal.html), which aggregate across multiple columns (as opposed to rows). See below for several examples. ```nushell # Horizontal sum across two columns (ignore nulls by default) > [[a b]; [1 2] [2 3] [3 4] [4 5] [5 null]] | polars into-df | polars select (polars horizontal sum a b) | polars collect ╭───┬─────╮ │ # │ sum │ ├───┼─────┤ │ 0 │ 3 │ │ 1 │ 5 │ │ 2 │ 7 │ │ 3 │ 9 │ │ 4 │ 5 │ ╰───┴─────╯ # Horizontal sum across two columns while accounting for nulls > [[a b]; [1 2] [2 3] [3 4] [4 5] [5 null]] | polars into-df | polars select (polars horizontal sum a b --nulls) | polars collect ╭───┬─────╮ │ # │ sum │ ├───┼─────┤ │ 0 │ 3 │ │ 1 │ 5 │ │ 2 │ 7 │ │ 3 │ 9 │ │ 4 │ │ ╰───┴─────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users have access to a new command, `polars horizontal`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example tests were added to `polars horizontal`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| eadb8da9f7 |
Bump to 0.104.1 dev version (#15669)
Marks development or hotfix |
|||
| cda15d91dd | Bump version for 0.104.0 release (#15664) | |||
| a1b7574306 |
Renamed join_where to join-where (#15660)
Renames the new `polars join_where` to `polars join-where` so that it conforms to the other Polars commands. |
|||
| d1d6518ece |
feat(polars): enable parsing strings as dates and datetime in polars schema (#15645)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR seeks to add a quality-of-life feature that enables date and datetime parsing of strings in `polars into-df`, `polars into-lazy`, and `polars open`, and avoid the more verbose method of casting each column into date/datetime. Currently, setting the schema to `date` on a `str` column would silently error as a null column. See a comparison of the current and proposed implementations. The proposed implementation assumes a date format "%Y-%m-%d" and a datetime format of "%Y-%m-%d %H:%M:%S" for naive datetimes and "%Y-%m-%d %H:%M:%S%:z" for timezone-aware datetimes. Other formats must be specified via parsing through `polars as-date` and `polars as-datetime`. ```nushell # Current Implementations > [[a]; ["2025-04-01"]] | polars into-df --schema {a: date} ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ │ ╰───┴───╯ > [[a]; ["2025-04-01 01:00:00"]] | polars into-df --schema {a: "datetime<ns,*>"} ╭───┬───╮ │ # │ a │ ├───┼───┤ │ 0 │ │ ╰───┴───╯ # Proposed Implementation > [[a]; ["2025-04-01"]] | polars into-df --schema {a: date} ╭───┬─────────────────────╮ │ # │ a │ ├───┼─────────────────────┤ │ 0 │ 04/01/25 12:00:00AM │ ╰───┴─────────────────────╯ > [[a]; ["2025-04-01 01:00:00"]] | polars into-df --schema {a: "datetime<ns,*>"} ╭───┬─────────────────────╮ │ # │ a │ ├───┼─────────────────────┤ │ 0 │ 04/01/25 01:00:00AM │ ╰───┴─────────────────────╯ > [[a]; ["2025-04-01 01:00:00-04:00"]] | polars into-df --schema {a: "datetime<ns,UTC>"} ╭───┬─────────────────────╮ │ # │ a │ ├───┼─────────────────────┤ │ 0 │ 04/01/25 05:00:00AM │ ╰───┴─────────────────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users have the added option to parse string columns into date/datetimes. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> No tests were added to any examples. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 715b0d90a9 |
fix(polars): conversion from nanoseconds to time_units in Datetime and Duration parsing (#15637)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> The current implementation improperly inverts the conversion from nanoseconds to the specified time units, resulting in nonsensical Datetime and Duration parsing and integer overflows when the specified time unit is not nanoseconds. This PR seeks to correct this conversion by changing the multiplication to an integer division. Below are examples highlighting the current and proposed implementations. ## Current Implementation Specifying a different time unit incorrectly changes the returned value. ```nushell > [[a]; [2024-04-01]] | polars into-df --schema {a: "datetime<ns,UTC>"} ╭───┬───────────────────────╮ │ # │ a │ ├───┼───────────────────────┤ │ 0 │ 04/01/2024 12:00:00AM │ > [[a]; [2024-04-01]] | polars into-df --schema {a: "datetime<ms,UTC>"} ╭───┬───────────────────────╮ │ # │ a │ ├───┼───────────────────────┤ │ 0 │ 06/27/2035 11:22:33PM │ <-- changing the time unit should not change the actual value > [[a]; [1day]] | polars into-df --schema {a: "duration<ns>"} ╭───┬────────────────╮ │ # │ a │ ├───┼────────────────┤ │ 0 │ 86400000000000 │ ╰───┴────────────────╯ > [[a]; [1day]] | polars into-df --schema {a: "duration<ms>"} ╭───┬──────────────────────╮ │ # │ a │ ├───┼──────────────────────┤ │ 0 │ -5833720368547758080 │ <-- i64 overflow ╰───┴──────────────────────╯ ``` ## Proposed Implementation ```nushell > [[a]; [2024-04-01]] | polars into-df --schema {a: "datetime<ns,UTC>"} ╭───┬───────────────────────╮ │ # │ a │ ├───┼───────────────────────┤ │ 0 │ 04/01/2024 12:00:00AM │ ╰───┴───────────────────────╯ > [[a]; [2024-04-01]] | polars into-df --schema {a: "datetime<ms,UTC>"} ╭───┬───────────────────────╮ │ # │ a │ ├───┼───────────────────────┤ │ 0 │ 04/01/2024 12:00:00AM │ ╰───┴───────────────────────╯ > [[a]; [1day]] | polars into-df --schema {a: "duration<ns>"} ╭───┬────────────────╮ │ # │ a │ ├───┼────────────────┤ │ 0 │ 86400000000000 │ ╰───┴────────────────╯ > [[a]; [1day]] | polars into-df --schema {a: "duration<ms>"} ╭───┬──────────╮ │ # │ a │ ├───┼──────────┤ │ 0 │ 86400000 │ ╰───┴──────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No user-facing breaking change. Developer breaking change: to mitigate the silent overflow in nanoseconds conversion functions `nanos_from_timeunit` and `nanos_to_timeunit` (new), the function signatures were changed from `i64` to `Result<i64, ShellError>`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> No additional examples were added, but I'd be happy to add a few if needed. The covering tests just didn't fit well into any examples. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 05c36d1bc7 |
add polars join_where command (#15635)
# Description This adds `polars join_where` which allows joining two dataframes based on a conditions. The command can be used as: ``` ➜ let df_a = [[name cash];[Alice 5] [Bob 10]] | polars into-lazy ➜ let df_b = [[item price];[A 3] [B 7] [C 12]] | polars into-lazy ➜ $df_a | polars join_where $df_b ((polars col cash) > (polars col price)) | polars collect ╭───┬───────┬──────┬──────┬───────╮ │ # │ name │ cash │ item │ price │ ├───┼───────┼──────┼──────┼───────┤ │ 0 │ Bob │ 10 │ B │ 7 │ │ 1 │ Bob │ 10 │ A │ 3 │ │ 2 │ Alice │ 5 │ A │ 3 │ ╰───┴───────┴──────┴──────┴───────╯ ``` # User-Facing Changes - new command `polars join_where` |
|||
| 208ebeefab |
feat(polars): enable parsing decimals in polars schemas (#15632)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR enables the option to set a column type to `decimal` in the `--schema` parameter of `polars into-df` and `polars into-lazy` commands. This option was already available in `polars open`, which used the underlying polars io commands that already accounted for decimal types when specified in the schema. See below for a comparison of the current and proposed implementation. ```nushell # Current Implementation > [[a b]; [1 1.618]]| polars into-df -s {a: u8, b: 'decimal<4,3>'} Error: × Error creating dataframe: Unsupported type: Decimal(Some(4), Some(3)) # Proposed Implementation > [[a b]; [1 1.618]]| polars into-df -s {a: u8, b: 'decimal<4,3>'} | polars schema ╭───┬──────────────╮ │ a │ u8 │ │ b │ decimal<4,3> │ ╰───┴──────────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking change. Users has the new option to specify decimal in `--schema` in `polars into-df` and `polars into-lazy`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> An example in `polars into-df` was modified to showcase the decimal type. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 717081bd2f |
fix mistake in description of polars pivot command (#15621)
Very small change to fix a typo/mistake in the polars pivot command description. |
|||
| 1db4be12d1 |
fix(polars): remove requirement that pivot columns must be same type in polars pivot (#15608)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Contrary to the underlying implementation in polars rust/python, `polars pivot` throws an error if the user tries to pivot on multiple columns of different types. This PR seeks to remove this type-check. See comparison below. ```nushell # Current implementation: throws error when pivoting on multiple values of different types. > [[name subject date test_1 test_2 grade_1 grade_2]; [Cady maths 2025-04-01 98 100 A A] [Cady physics 2025-04-01 99 100 A A] [Karen maths 2025-04-02 61 60 D D] [Karen physics 2025-04-02 58 60 D D]] | polars into-df | polars pivot --on [subject] --index [name] --values [test_1 grade_1] Error: × Merge error ╭─[entry #291:1:271] 1 │ [[name subject date test_1 test_2 grade_1 grade_2]; [Cady maths 2025-04-01 98 100 A A] [Cady physics 2025-04-01 99 100 A A] [Karen maths 2025-04-02 61 60 D D] [Karen physics 2025-04-02 58 60 D D]] | polars into-df | polars pivot --on [subject] --index [name] --values [test_1 grade_1] · ───────┬────── · ╰── found different column types in list ╰──── help: datatypes i64 and str are incompatible # Proposed implementation > [[name subject date test_1 test_2 grade_1 grade_2]; [Cady maths 2025-04-01 98 100 A A] [Cady physics 2025-04-01 99 100 A A] [Karen maths 2025-04-02 61 60 D D] [Karen physics 2025-04-02 58 60 D D]] | polars into-df | polars pivot --on [subject] --index [name] --values [test_1 grade_1] ╭───┬───────┬──────────────┬────────────────┬───────────────┬─────────────────╮ │ # │ name │ test_1_maths │ test_1_physics │ grade_1_maths │ grade_1_physics │ ├───┼───────┼──────────────┼────────────────┼───────────────┼─────────────────┤ │ 0 │ Cady │ 98 │ 99 │ A │ A │ │ 1 │ Karen │ 61 │ 58 │ D │ D │ ╰───┴───────┴──────────────┴────────────────┴───────────────┴─────────────────╯ ``` Additionally, this PR ports over the `separator` parameter in `pivot`, which allows the user to specify how to delimit multiple `values` column names: ```nushell > [[name subject date test_1 test_2 grade_1 grade_2]; [Cady maths 2025-04-01 98 100 A A] [Cady physics 2025-04-01 99 100 A A] [Karen maths 2025-04-02 61 60 D D] [Karen physics 2025-04-02 58 60 D D]] | polars into-df | polars pivot --on [subject] --index [name] --values [test_1 grade_1] --separator / ╭───┬───────┬──────────────┬────────────────┬───────────────┬─────────────────╮ │ # │ name │ test_1/maths │ test_1/physics │ grade_1/maths │ grade_1/physics │ ├───┼───────┼──────────────┼────────────────┼───────────────┼─────────────────┤ │ 0 │ Cady │ 98 │ 99 │ A │ A │ │ 1 │ Karen │ 61 │ 58 │ D │ D │ ╰───┴───────┴──────────────┴────────────────┴───────────────┴─────────────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Soft breaking change: where a user may have previously expected an error (pivoting on multiple columns with different types), no error is thrown. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Examples were added to `polars pivot`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| a2dc3e3b33 |
feat(polars): enable as_date and as_datetime to handle expressions as inputs (#15590)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR is a follow-up to the previous PR #15557 and part of a wider campaign to enable certain polars commands that only operated on the entire dataframe to also operate on expressions. Here, we enable two commands `polars as-date` and `polars as-datetime` to receive expressions as inputs so that they may be used on specific columns in a dataframe with multiple columns of different types. See examples below. ```nushell > [[a b]; ["2025-04-01" 1] ["2025-04-02" 2] ["2025-04-03" 3]] | polars into-df | polars select (polars col a | polars as-date %Y-%m-%d) b | polars collect ╭───┬───────────────────────┬───╮ │ # │ a │ b │ ├───┼───────────────────────┼───┤ │ 0 │ 04/01/2025 12:00:00AM │ 1 │ │ 1 │ 04/02/2025 12:00:00AM │ 2 │ │ 2 │ 04/03/2025 12:00:00AM │ 3 │ ╰───┴───────────────────────┴───╯ > seq date -b 2025-04-01 --periods 4 --increment 25min -o "%Y-%m-%d %H:%M:%S" | polars into-df | polars select (polars col 0 | polars as-datetime "%Y-%m-%d %H:%M:%S") | polars collect ╭───┬───────────────────────╮ │ # │ 0 │ ├───┼───────────────────────┤ │ 0 │ 04/01/2025 12:00:00AM │ │ 1 │ 04/01/2025 12:25:00AM │ │ 2 │ 04/01/2025 12:50:00AM │ │ 3 │ 04/01/2025 01:15:00AM │ ╰───┴───────────────────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users have the additional option to use `polars as-date` and `polars as-datetime` in expressions that operate on specific columns. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Examples have been added to `polars as-date` and `polars as-datetime`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 95998bdd53 |
fix(custom_value) + fix(polars): map // operator to FloorDivide for custom values and in polars (#15599)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR fixes an issue where, for custom values, the `//` operator was incorrectly mapped to `Math::Divide` instead of `Math::FloorDivide`. This PR also fixes the same mis-mapping in the `polars` plugin. ```nushell > [[a b c]; [x 1 1.1] [y 2 2.2] [z 3 3.3]] | polars into-df | polars select {div: ((polars col c) / (polars col b)), floor_div: ((polars col c) // (polars col b))} | polars collect ╭───┬───────┬───────────╮ │ # │ div │ floor_div │ ├───┼───────┼───────────┤ │ 0 │ 1.100 │ 1.000 │ │ 1 │ 1.100 │ 1.000 │ │ 2 │ 1.100 │ 1.000 │ ╰───┴───────┴───────────╯ ``` **Note:** the number of line changes in this PR is inflated because of auto-formatting in `nu_plugin_polars/Cargo.toml`. Substantively, I've only added the `round_series` feature to the polars dependency list. # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Breaking change: users who expected the operator `//` to function the same as `/` for custom values will not get the expected result. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> No tests were yet added, but let me know if we should put something into one of the polars examples. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| bd5de023a1 |
feat(polars): add pow (**) operator for polars expressions (#15598)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR adds the exponent operator ("**") to polars expressions. ```nushell > [[a b]; [6 2] [4 2] [2 2]] | polars into-df | polars select a b {c: ((polars col a) ** 2)} ╭───┬───┬───┬────╮ │ # │ a │ b │ c │ ├───┼───┼───┼────┤ │ 0 │ 6 │ 2 │ 36 │ │ 1 │ 4 │ 2 │ 16 │ │ 2 │ 2 │ 2 │ 4 │ ╰───┴───┴───┴────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users are enabled to use the `**` operator in polars expressions. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> An example in `polars select` was modified to showcase the `**` operator. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 0e9927ea4d |
polars: expand polars col to handle multiple columns and by types (#15570)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR seeks to expand `polars col` functionality to allow selecting multiple columns and columns by type, which is particularly useful when piping to subsequent expressions that should be applied to each column selected (e.g., `polars col int --type | polars sum` as a shorthand for `[(polars col a | polars sum), (polars col b | polars sum)]`). See examples below. ```nushell # Select multiple columns (cannot be used with asterisk wildcard) > [[a b c]; [x 1 1.1] [y 2 2.2] [z 3 3.3]] | polars into-df | polars select (polars col b c | polars sum) | polars collect ╭───┬───┬──────╮ │ # │ b │ c │ ├───┼───┼──────┤ │ 0 │ 6 │ 6.60 │ ╰───┴───┴──────╯ # Select multiple columns by types (cannot be used with asterisk wildcard) > [[a b c]; [x o 1.1] [y p 2.2] [z q 3.3]] | polars into-df | polars select (polars col str f64 --type | polars max) | polars collect ╭───┬───┬───┬──────╮ │ # │ a │ b │ c │ ├───┼───┼───┼──────┤ │ 0 │ z │ q │ 3.30 │ ╰───┴───┴───┴──────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users have the additional capability to select multiple columns in `polars col`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Examples have been added to `polars col`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 2dc5c19b71 |
feat(polars): loosen constraints on accepted expressions in polars group-by (#15583)
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR lifts the constraint that expressions in the `polars group-by`
command must be limited only to the type `Expr::Column` rather than most
`Expr` types, which is what the underlying polars crate allows. This
change enables more complex expressions to group by.
In the example below, we group by even or odd days of column `a`. While
we can reach the same result by creating and grouping by a new column in
two separate steps, integrating these steps in a single group-by allows
for better delegation to the polars optimizer.
```nushell
# Group by an expression and perform an aggregation
> [[a b]; [2025-04-01 1] [2025-04-02 2] [2025-04-03 3] [2025-04-04 4]]
| polars into-lazy
| polars group-by (polars col a | polars get-day | $in mod 2)
| polars agg [
(polars col b | polars min | polars as "b_min")
(polars col b | polars max | polars as "b_max")
(polars col b | polars sum | polars as "b_sum")
]
| polars collect
| polars sort-by a
╭───┬───┬───────┬───────┬───────╮
│ # │ a │ b_min │ b_max │ b_sum │
├───┼───┼───────┼───────┼───────┤
│ 0 │ 0 │ 2 │ 4 │ 6 │
│ 1 │ 1 │ 1 │ 3 │ 4 │
╰───┴───┴───────┴───────┴───────╯
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
No breaking changes. The user is empowered to use more complex
expressions in `polars group-by`
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
An example is added to `polars group-by`.
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
|
|||
| 669b44ad7d |
feat(polars): add polars truncate for rounding datetimes (#15582)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR directly ports the polars function `polars.Expr.dt.truncate` (https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.dt.truncate.html), which rounds a datetime to an arbitrarily specified period length. This function is particularly useful when rounding to variable period lengths such as months or quarters. See below for examples. ```nushell # Truncate a series of dates by period length > seq date -b 2025-01-01 --periods 4 --increment 6wk -o "%Y-%m-%d %H:%M:%S" | polars into-df | polars as-datetime "%F %H:%M:%S" --naive | polars select datetime (polars col datetime | polars truncate 5d37m | polars as truncated) | polars collect ╭───┬───────────────────────┬───────────────────────╮ │ # │ datetime │ truncated │ ├───┼───────────────────────┼───────────────────────┤ │ 0 │ 01/01/2025 12:00:00AM │ 12/30/2024 04:49:00PM │ │ 1 │ 02/12/2025 12:00:00AM │ 02/08/2025 09:45:00PM │ │ 2 │ 03/26/2025 12:00:00AM │ 03/21/2025 02:41:00AM │ │ 3 │ 05/07/2025 12:00:00AM │ 05/05/2025 08:14:00AM │ ╰───┴───────────────────────┴───────────────────────╯ # Truncate based on period length measured in quarters and months > seq date -b 2025-01-01 --periods 4 --increment 6wk -o "%Y-%m-%d %H:%M:%S" | polars into-df | polars as-datetime "%F %H:%M:%S" --naive | polars select datetime (polars col datetime | polars truncate 1q5mo | polars as truncated) | polars collect ╭───┬───────────────────────┬───────────────────────╮ │ # │ datetime │ truncated │ ├───┼───────────────────────┼───────────────────────┤ │ 0 │ 01/01/2025 12:00:00AM │ 09/01/2024 12:00:00AM │ │ 1 │ 02/12/2025 12:00:00AM │ 09/01/2024 12:00:00AM │ │ 2 │ 03/26/2025 12:00:00AM │ 09/01/2024 12:00:00AM │ │ 3 │ 05/07/2025 12:00:00AM │ 05/01/2025 12:00:00AM │ ╰───┴───────────────────────┴───────────────────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. This PR introduces a new command `polars truncate` # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example test was added. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 8f81812ef9 |
fix cannot find issue when performing collect on an eager dataframe (#15577)
# Description Performing a `polars collect` on an eager dataframe should be a no-op operation. However, when used with a pipeline and not saving to a value a cache error occurs. This addresses that cache error. |
|||
| a33650a69e |
fix(polars): cast as date now returns Date type instead of Datetime<ns> (#15574)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR fixes the bug where various commands that cast a column as a `date` type would return `datetime<ns>` rather than the intended type `date`. Affected commands include `polars into-df --schema`, `polars into-lazy --schema`, `polars as-date`, and `polars cast date`. This bug derives from the fact that Nushell uses the `date` type to denote a datetime type whereas polars differentiates between `Date` and `Datetime` types. By default, this PR retains the behavior that a Nushell `date` type will be mapped to a polars `Datetime<ns>` unless otherwise specified. ```nushell # Current (erroneous) implementation > [[a]; [2025-03-20]] | polars into-df --schema {a: "date"} | polars schema ╭───┬──────────────╮ │ a │ datetime<ns> │ ╰───┴──────────────╯ # Fixed implementation > [[a]; [2025-03-20]] | polars into-df --schema {a: "date"} | polars schema ╭───┬──────╮ │ a │ date │ ╰───┴──────╯ # Fixed implementation: by default, Nushell dates map to datetime<ns> > [[a]; [2025-03-20]] | polars into-df | polars schema ╭───┬───────────────────╮ │ a │ datetime<ns, UTC> │ ╰───┴───────────────────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> Soft breaking change: users previously who wanted to cast a date column to type `date` can now expect the output to be type `date` instead of `datetime<ns>`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example test added to `polars as-date` command. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
|||
| 89322f59f2 |
Fix output type of polars schema (#15572)
# Description Output type of `polars schema` signature output type is of dataframe. It should be of type record. # User-Facing Changes - `polars schema` - how has an output type of record |
|||
| 4e307480e4 |
polars: extend NuExpression::extract_exprs to handle records (#15553)
<!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> This PR seeks to simplify the syntax for commands that handle a list of expressions (e.g., `select`, `with-column`, and `agg`) by enabling the user to replace a list of expressions each aliased with `polars as` to a single record where the key is the alias for the value. See below for examples in several contexts. ```nushell # Select a column from a dataframe using a record > [[a b]; [6 2] [4 2] [2 2]] | polars into-df | polars select {c: ((polars col a) * 2)} ╭───┬────╮ │ # │ c │ ├───┼────┤ │ 0 │ 12 │ │ 1 │ 8 │ │ 2 │ 4 │ ╰───┴────╯ # Select a column from a dataframe using a mix of expressions and record of expressions > [[a b]; [6 2] [4 2] [2 2]] | polars into-df | polars select a b {c: ((polars col a) * 2)} ╭───┬───┬───┬────╮ │ # │ a │ b │ c │ ├───┼───┼───┼────┤ │ 0 │ 6 │ 2 │ 12 │ │ 1 │ 4 │ 2 │ 8 │ │ 2 │ 2 │ 2 │ 4 │ ╰───┴───┴───┴────╯ # Add series to the dataframe using a record > [[a b]; [1 2] [3 4]] | polars into-lazy | polars with-column { c: ((polars col a) * 2) d: ((polars col a) * 3) } | polars collect ╭───┬───┬───┬───┬───╮ │ # │ a │ b │ c │ d │ ├───┼───┼───┼───┼───┤ │ 0 │ 1 │ 2 │ 2 │ 3 │ │ 1 │ 3 │ 4 │ 6 │ 9 │ ╰───┴───┴───┴───┴───╯ # Group by and perform an aggregation using a record > [[a b]; [1 2] [1 4] [2 6] [2 4]] | polars into-lazy | polars group-by a | polars agg { b_min: (polars col b | polars min) b_max: (polars col b | polars max) b_sum: (polars col b | polars sum) } | polars collect | polars sort-by a ╭───┬───┬───────┬───────┬───────╮ │ # │ a │ b_min │ b_max │ b_sum │ ├───┼───┼───────┼───────┼───────┤ │ 0 │ 1 │ 2 │ 4 │ 6 │ │ 1 │ 2 │ 4 │ 6 │ 10 │ ╰───┴───┴───────┴───────┴───────╯ ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> No breaking changes. Users now can use a mix of lists of expressions and records of expressions where previously only lists of expressions were accepted (e.g., in `select`, `with-column`, and `agg`). # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> Example tests were added to `select`, `with-column`, and `agg`. # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
