Commit Graph

95 Commits

Author SHA1 Message Date
Jack Wright
2df91e7f92
Removed CustomValue portion of CustomValue type name strings. (#14054)
# Description

This changes the names returned by CustomValue::name() of the various
custom value structs to just say the name of the thing they represent.
For instance "DataFrameCustomValue" is not just "DataFrame".

# User-Facing Changes
- Places such as or errors where NuDataFrameCustomValue would be seen,
now just shows as NuDataFrame.
2024-10-11 06:41:24 -05:00
dependabot[bot]
ba4becc61c
Bump indexmap from 2.5.0 to 2.6.0 (#13983)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.5.0 to
2.6.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.6.0 (2024-10-01)</h2>
<ul>
<li>Implemented <code>Clone</code> for <code>map::IntoIter</code> and
<code>set::IntoIter</code>.</li>
<li>Updated the <code>hashbrown</code> dependency to version 0.15.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bf0362ba25"><code>bf0362b</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/354">#354</a>
from cuviper/release-2.6.0</li>
<li><a
href="bd0b4f7c8c"><code>bd0b4f7</code></a>
Add all release dates</li>
<li><a
href="53400496f4"><code>5340049</code></a>
Release 2.6.0</li>
<li><a
href="7f8022912a"><code>7f80229</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/343">#343</a>
from cuviper/hash_table</li>
<li><a
href="e577bf2556"><code>e577bf2</code></a>
Use <code>hashbrown::HashTable</code> instead of
<code>RawTable</code></li>
<li><a
href="09b48ec3b3"><code>09b48ec</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/353">#353</a>
from cuviper/move_index</li>
<li><a
href="267b83d701"><code>267b83d</code></a>
Add an explicit bounds check in <code>move_index</code></li>
<li><a
href="d74a4daffb"><code>d74a4da</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/349">#349</a>
from waywardmonkeys/improve-doc-formatting</li>
<li><a
href="5b0ed20b87"><code>5b0ed20</code></a>
docs: Improve doc formatting with backticks</li>
<li><a
href="15518f3152"><code>15518f3</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/348">#348</a>
from cuviper/clone-intoiter</li>
<li>Additional commits viewable in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.5.0...2.6.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.5.0&new-version=2.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jack Wright <jack.wright@disqo.com>
2024-10-08 17:07:21 -07:00
Jack Wright
1d6ac16530
polars into-df struct fix (#13977)
# Description
This fixes an issue with converting to a dataframe when specifying a
struct in the schema. Things like the following now work correctly:
```nushell
 [[foo bar]; [{a: "a_0", b:"b_0"} 1] [{a: "a_1", b: "b_1" } 2]] | polars into-df -s {foo: {a: str, b: str}, bar: u8}
```
2024-10-02 05:59:14 -05:00
Jack Wright
bcaef8959c
Expose flag truncate-ragged-lines in polars open (#13939)
# Description
Introduces a new flag `--truncate-ragged-lines` for `polars open` that
will truncate lines that are longer than the schema.

# User-Facing Changes
- Introduction of the flag `--truncate-ragged-lines` for `polars open`
2024-09-27 06:54:46 -05:00
Jack Wright
5bef81a059
Added command polars len for performing count(*) like operations. (#13941)
# Description
This request exposes the prelude::polars::len expression. It is ended
for doing fast select count(*) like operations:

<img width="626" alt="Screenshot 2024-09-26 at 18 14 45"
src="https://github.com/user-attachments/assets/74285fc6-f99c-46e0-9226-9a7d41738d78">

# User-Facing Changes
- Introduction of the `polars len` command
2024-09-27 06:54:28 -05:00
Jack Wright
28a7461057
Added command polars profile for profiling lazy dataframes (#13904)
# Description
Introduce a new command `polars profile` for profiling lazy dataframes:

<img width="965" alt="Screenshot 2024-09-22 at 23 46 18"
src="https://github.com/user-attachments/assets/11402dd3-8256-43df-a986-64241c15354f">

# User-Facing Changes
- Introduces new command `polars profile`
2024-09-23 07:57:59 -05:00
Jack Wright
2541a712e4
Added polars concat to allow concatenation of multiple dataframes (#13879)
# Description
Provides the ability to concatenate multiple dataframes together

# User-Facing Changes
- Introduces new command `polars concat`
2024-09-23 06:43:43 -05:00
Skyler Hawthorne
5fa9d76500
polars: add binary type support (#13830)
# Description
This adds support for reading and writing binary types in the polars
commands.

The `BinaryOffset` type can be read into a Nushell native `Value` type
no problem, but unfortunately this is a lossy conversion, as there's
no Nushell-native semantic equivalent to the fixed size binary type
in Arrow.

# User-Facing Changes

`polars open` and `polars save` now work with binary types.
2024-09-23 06:28:41 -05:00
Wind
1a081c09de
Bump version to 0.98.1 (#13896) 2024-09-22 12:41:44 +08:00
Devyn Cairns
6e1e824473
Bump version to 0.98.0 (#13865) 2024-09-18 00:48:46 -07:00
Jack Wright
af77bc60e2
Improved null handling when converting from nu -> dataframe. (#13855)
# Description
Fixes: #12726 and #13185

Previously converting columns that contained null caused polars to force
a dtype of object even when using a schema.

Now:
1. When using a schema, the type the schema defines for the column will
always be used.
2. When a schema is not used, the previous type is used when a value is
null.

# User-Facing Changes
- The type defined by the schema we be respected when passing in a null
value `[a]; [null] | polars into-df -s {a: str}` will create a df with
an str dtype column with one null value versus a column of type object.
- *BREAKING CHANGE* If you define a schema, all columns must be in the
schema.
2024-09-16 18:07:13 -05:00
Jack Wright
c535c24d03
catch unwrap on panics with polars collect (#13850)
# Description
This resurrects the work from #12866 and fixes #12732. 

Polars panics for a plethora or reasons. While handling panics is
generally frowned upon, in cases like with `polars collect` a panic
cause a lot of work to be lost. Often you might have multiple dataframes
in memory and you are trying one operation and lose all state.

While it possible the panic can leave things a strange state, it is
pretty unlikely as part of a polars pipeline. Most of the time polars
objects are not manipulating dataframes in memory mutability, but rather
creating a new dataframe the operations being applied. This is always
the case with a lazy pipeline. After the collect call, the original
dataframes are intact still and I haven't observed any side effects.
2024-09-15 07:21:02 -05:00
Jack Wright
fb34a4fc6c
Make polars save return an empty pipeline (#13833)
# Description
In order to be more consistent with it's nu counterpart, `polars save`
now returns an empty pipeline instead of a message of the saved file.

# User-Facing Changes
- `polars save` no longer displays a save message, making it consistent
with `save` behavior.
2024-09-12 06:23:40 -05:00
Jack Wright
8d60c0d35d
Migrating polars commands away from macros, removed custom DataFrame comparison. (#13829)
# Description
This PR:
- Removes the lazy_command, expr_command macros and migrates the
commands that were utilizing them.
- Removes the custom logic in DataFrameValues::is_equals to use the
polars DataFrame version of PartialEq
- Adds examples to commands that previously did not have examples or had
inadequate ones.

NOTE: A lot of examples now have a `polars sort` at the end. This is
needed due to the comparison in the result. The new polars version of
equals cares about the ordering. I removed the custom equals logic as it
causes comparisons to lock up when comparing dataframes that contain a
row that contains a list. I discovered this issue when adding examples
to `polars implode`
2024-09-11 10:33:05 -07:00
Jack Wright
1e64f59220
Added documentation explanation explaining how to select all columns with polars col (#13806)
# Description
Previously there were no examples or explanations that you can use '*'
to select all columns. Updated description and added a new example.
2024-09-08 17:12:03 +00:00
Jack Wright
6c1c7f9509
Added expression support for polars cumulative (#13799)
# Description
Provides the ability to use `polars cumulative` as an expression:

<img width="1266" alt="Screenshot 2024-09-06 at 17 47 15"
src="https://github.com/user-attachments/assets/73c11f79-598c-4efa-bfcd-755e536ead66">

# User-Facing Changes
- `polars cumulative` can now be used as an expression.
2024-09-06 22:03:51 -05:00
Jack Wright
f531cc2058
Polars command reorg (#13798)
# Description
House keeping. Restructures polars modules as discussed in:
https://docs.google.com/spreadsheets/d/1gyA58i_yTXKCJ5DbO_RxBNAlK6S7C1M22ppKwVLZltc/edit?usp=sharing
2024-09-06 13:46:37 -07:00
Jack Wright
dc53c20628
Renamed polars concatenate and added expression support. (#13781)
# Description
In order to be more consistent with the nushell terminology and with
polars expression terminology `polars concatenate` is now `polars
str-join`. `polars str-join` can also be used as expression.

<img width="857" alt="Screenshot 2024-09-04 at 12 41 25"
src="https://github.com/user-attachments/assets/8cc5a0c2-194c-49ec-9fe1-65ec4825414d">


# User-Facing Changes
- `polars concatenate` is now `polars str-join`
- `polars str-join` can be used as an expression
2024-09-05 09:28:34 -07:00
Jack Wright
e7c5f83460
Added expression support for polars str-lengths (#13782)
# Description
Allows `polars str-lengths` to be used as an expression:

<img width="826" alt="Screenshot 2024-09-04 at 13 57 45"
src="https://github.com/user-attachments/assets/b74139e0-e8ba-4910-84c2-cf4be4a084b6">

# User-Facing Changes
- `polars str-lengths` can be used as an expression.
- char length is now the default. Use the --bytes flag to get bytes
length.
2024-09-05 09:26:09 -07:00
Jack Wright
f611196373
Expression support for polars str-slice (#13783)
# Description
Provides expression support for `polars str-slice`:
<img width="893" alt="Screenshot 2024-09-04 at 18 03 05"
src="https://github.com/user-attachments/assets/d8a8a2a7-53cf-4c3a-ae7a-dfdaf48a05ee">

# User-Facing Changes
- `polars str-slice` can now be used as an expression
2024-09-05 09:06:37 -07:00
Jack Wright
63b94dbd28
Added expression support for polars contains (#13769)
# Description
Adds the ability to use `polars contains` as an expression:
<img width="785" alt="Screenshot 2024-09-03 at 14 39 03"
src="https://github.com/user-attachments/assets/35c0d4e5-6bef-4974-a31f-463c7203bd03">


# User-Facing Changes
- `polars contains` can now be used with expressions
2024-09-04 12:19:45 +02:00
Jack Wright
eb0de25d19
Expression support for polars strftime (#13767)
# Description
Allows `polars strftime` to be used as an expression:
<img width="849" alt="Screenshot 2024-09-03 at 13 14 11"
src="https://github.com/user-attachments/assets/2a987fdf-cf00-4c57-b6e1-0b706c594c20">

# User-Facing Changes
- `polars strftime` can now be used as an expression.
2024-09-04 12:19:29 +02:00
dependabot[bot]
348c59b740
Bump indexmap from 2.4.0 to 2.5.0 (#13772)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.4.0 to
2.5.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.5.0</h2>
<ul>
<li>Added an <code>insert_before</code> method to <code>IndexMap</code>
and <code>IndexSet</code>, as an
alternative to <code>shift_insert</code> with different behavior on
existing entries.</li>
<li>Added <code>first_entry</code> and <code>last_entry</code> methods
to <code>IndexMap</code>.</li>
<li>Added <code>From</code> implementations between
<code>IndexedEntry</code> and <code>OccupiedEntry</code>.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="48ed49017c"><code>48ed490</code></a>
Release 2.5.0</li>
<li><a
href="139d7addfb"><code>139d7ad</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/340">#340</a>
from cuviper/insert-bounds</li>
<li><a
href="1d9b5e3d03"><code>1d9b5e3</code></a>
Add doc examples for <code>insert_before</code> and
<code>shift_insert</code></li>
<li><a
href="8ca01b0df7"><code>8ca01b0</code></a>
Use <code>insert_before</code> for &quot;new&quot; entries in
<code>insert_sorted</code></li>
<li><a
href="7224def010"><code>7224def</code></a>
Add <code>insert_before</code> as an alternate to
<code>shift_insert</code></li>
<li><a
href="0247a1555d"><code>0247a15</code></a>
Document and assert index bounds in <code>shift_insert</code></li>
<li><a
href="922c6ad1af"><code>922c6ad</code></a>
Update the CI badge</li>
<li><a
href="e482e1768a"><code>e482e17</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/342">#342</a>
from cuviper/btree-like</li>
<li><a
href="b63e4a1556"><code>b63e4a1</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/341">#341</a>
from cuviper/from-entry</li>
<li><a
href="264e5b7304"><code>264e5b7</code></a>
Add doc aliases like <code>BTreeMap</code>/<code>BTreeSet</code></li>
<li>Additional commits viewable in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.4.0...2.5.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.4.0&new-version=2.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-04 10:02:01 +08:00
Jack Wright
0119534f61
Expression support polars replace and polars replace-all (#13726)
# Description
Adds the ability for `polars replace` and `polars replace-all` to work
as expressions.

# User-Facing Changes
- `polars replace` can be used with polars expressions
- `polars replace-all` can be used with polars expressions
2024-08-29 13:59:44 -07:00
Jack Wright
644bebf4c6
Expression support for polars uppercase and polars lowercase (#13724) 2024-08-28 14:08:16 -07:00
Jack Wright
a39e94de8a
Added polars commands for converting string columns to integer and decimal columns (#13711)
# Description
Introduces two new polars commands for converting string columns to
decimal and integer columns:

<img width="740" alt="Screenshot 2024-08-27 at 15 32 28"
src="https://github.com/user-attachments/assets/f9573b6e-48f6-4bbf-8782-39ffb95eb934">

<img width="720" alt="Screenshot 2024-08-27 at 15 33 46"
src="https://github.com/user-attachments/assets/90a66bb5-fa78-4ed3-8b2b-ae05cddd2f3a">

# User-Facing Changes
- Addition of the `polars integer` command
- Addition of the `polars decimal` command
2024-08-28 07:54:31 -05:00
Stefan Holderbach
95b78eee25
Change the usage misnomer to "description" (#13598)
# Description
    
The meaning of the word usage is specific to describing how a command
function is *used* and not a synonym for general description. Usage can
be used to describe the SYNOPSIS or EXAMPLES sections of a man page
where the permitted argument combinations are shown or example *uses*
are given.
Let's not confuse people and call it what it is a description.

Our `help` command already creates its own *Usage* section based on the
available arguments and doesn't refer to the description with usage.

# User-Facing Changes

`help commands` and `scope commands` will now use `description` or
`extra_description`
`usage`-> `description`
`extra_usage` -> `extra_description`

Breaking change in the plugin protocol:

In the signature record communicated with the engine.
`usage`-> `description`
`extra_usage` -> `extra_description`

The same rename also takes place for the methods on
`SimplePluginCommand` and `PluginCommand`

# Tests + Formatting
- Updated plugin protocol specific changes
# After Submitting
- [ ] update plugin protocol doc
2024-08-22 12:02:08 +02:00
Stefan Holderbach
e211b7ba53
Bump version to 0.97.2 (#13666) 2024-08-22 11:36:32 +02:00
Devyn Cairns
60769ac1ba
Bump version to 0.97.1 (#13659)
# Description

Bump version to `0.97.1`, which will be the actual next major release.
(`0.97.0` had a bug.)
2024-08-20 20:21:12 -07:00
Jack Wright
d667b3c0bc
bumped version number to 0.97 (#13655) 2024-08-20 16:28:19 -07:00
Jack Wright
803bc9c63f
Incrementing the eager dataframe cache value before returning it (#13624)
# Description

Fixes issue [12828](https://github.com/nushell/nushell/issues/12828).

When attempting a `polars collect` on an eager dataframe, we return
dataframe as is. However, before this fix I failed to increment the
internal cache reference count. This caused the value to be dropped from
the internal cache when the references were decremented again.

This fix adds a call to cache.get to increment the value before
returning.
2024-08-14 15:38:46 -05:00
dependabot[bot]
e841fce0f9
Bump indexmap from 2.3.0 to 2.4.0 (#13617)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.3.0 to
2.4.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.4.0</h2>
<ul>
<li>Added methods <code>IndexMap::append</code> and
<code>IndexSet::append</code>, moving all items from
one map or set into another, and leaving the original capacity for
reuse.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b66bbfe282"><code>b66bbfe</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/337">#337</a>
from cuviper/append</li>
<li><a
href="a288bf3b3a"><code>a288bf3</code></a>
Add a README note for <code>ordermap</code></li>
<li><a
href="0b2b4b9a78"><code>0b2b4b9</code></a>
Release 2.4.0</li>
<li><a
href="8c0a1cd4be"><code>8c0a1cd</code></a>
Add <code>append</code> methods</li>
<li>See full diff in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.3.0...2.4.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.3.0&new-version=2.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-14 15:21:39 +08:00
Jack Wright
4ff33933dd
Merge polars sink and polars to-* to polars save (#13568)
# Description
This pull request merges `polars sink` and `polars to-*` into one
command `polars save`.


# User-Facing Changes
- `polars to-*` commands have all been replaced with `polars save`. When
saving a lazy frame to a type that supports a polars sink operation, a
sink operation will be performed. Sink operations are much more
performant, performing a collect while streaming to the file system.
2024-08-08 09:46:45 -07:00
Jack Wright
7d4449f021
Added polars sink command, that performs and streaming collect to t… (#13562)
# Description
This exposes the `LazyFrame::sink_*` functionality to allow a streaming
collect directly to the filesystem. This useful when working with data
that is too large to fit into memory.

# User-Facing Changes
- Introduction of the `polars sink` command
2024-08-07 10:59:49 -05:00
Jack Wright
ec3e0e593d
polars first and polars last will now handle lazy frames natively (#13555)
# Description
Prior this pull request `polars first` and `polars last` would collect a
lazy frame into an eager frame before performing operations. Now `polars
first` will to a `LazyFrame::limit` and `polars last` will perform a
`LazyFrame::tail`. This is really useful in working with very large
datasets.
2024-08-07 06:36:52 -05:00
Jack Wright
ff09c7964e
polars open will now open a lazy frame by default (#13556)
# Description
When opening a dataframe the default operation will be to create a lazy
frame if possible. This works much better with large datasets and
supports hive format.

# User-Facing Changes
- `--lazy` is nolonger a valid option. `--eager` must be used to
explicitly open an eager dataframe.
2024-08-07 06:36:08 -05:00
dependabot[bot]
edee2a3c15
Bump indexmap from 2.2.6 to 2.3.0 (#13557)
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.2.6 to
2.3.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.3.0</h2>
<ul>
<li>Added trait <code>MutableEntryKey</code> for opt-in mutable access
to map entry keys.</li>
<li>Added method <code>MutableKeys::iter_mut2</code> for opt-in mutable
iteration of map
keys and values.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="22c0b4e0f3"><code>22c0b4e</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/335">#335</a>
from epage/mut</li>
<li><a
href="39f7cc097a"><code>39f7cc0</code></a>
Release 2.3.0</li>
<li><a
href="6049d518a0"><code>6049d51</code></a>
feat(map): Add MutableKeys::iter_mut2</li>
<li><a
href="65c3c46e37"><code>65c3c46</code></a>
feat(map): Add MutableEntryKey</li>
<li><a
href="7f7d39f734"><code>7f7d39f</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/332">#332</a>
from waywardmonkeys/missing-indentation-in-doc-comment</li>
<li><a
href="8222a59a85"><code>8222a59</code></a>
Fix missing indentation in doc comment.</li>
<li><a
href="1a71dde63d"><code>1a71dde</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/327">#327</a>
from waywardmonkeys/dep-update-dev-dep-itertools</li>
<li><a
href="ac2a8a5a40"><code>ac2a8a5</code></a>
deps(dev): Update <code>itertools</code></li>
<li>See full diff in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.2.6...2.3.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=indexmap&package-manager=cargo&previous-version=2.2.6&new-version=2.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-07 09:02:56 +08:00
suimong
12f57dbc62
Add "--as-columns" flag to polars into-df (#13449)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->
Per discussion on
[Discord](https://discord.com/channels/601130461678272522/864228801851949077/1265718178927870045)
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

To facilitate column-oriented dataframe construction, this PR added a
`--as-columns` flag to `polars into-df` command so that when specified,
and when input shape is record of lists, each list will be treated as a
column rather than a cell value, i.e. `{a: [1 3], b: [2 4]} | polars
into-df --as-columns` returns the same dataframe as `[[a b];[1 2] [3 4]]
| polars into-df`


# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

A new flag `--as-columns`, no change of semantics if this flag is
unspecified.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->

---------

Co-authored-by: Ben Yang <ben@ya.ng>
2024-07-30 08:50:50 -05:00
Devyn Cairns
c31291753c
Bump version to 0.96.2 (#13485)
This should be the new development version. We most likely don't need a
0.96.2 patch release. Should be free to merge PRs after this.
2024-07-29 17:20:55 -07:00
Devyn Cairns
9f90d611e1
Bump version to 0.96.1 (#13439)
(Post-release bump.)
2024-07-25 18:28:18 +08:00
Devyn Cairns
a80dfe8e80
Bump version to 0.96.0 (#13433) 2024-07-23 16:10:35 -07:00
dependabot[bot]
63cea44130
Bump uuid from 1.9.1 to 1.10.0 (#13390)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.9.1 to 1.10.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/uuid-rs/uuid/releases">uuid's
releases</a>.</em></p>
<blockquote>
<h2>1.10.0</h2>
<h2>Deprecations</h2>
<p>This release deprecates and renames the following functions:</p>
<ul>
<li><code>Builder::from_rfc4122_timestamp</code> -&gt;
<code>Builder::from_gregorian_timestamp</code></li>
<li><code>Builder::from_sorted_rfc4122_timestamp</code> -&gt;
<code>Builder::from_sorted_gregorian_timestamp</code></li>
<li><code>Timestamp::from_rfc4122</code> -&gt;
<code>Timestamp::from_gregorian</code></li>
<li><code>Timestamp::to_rfc4122</code> -&gt;
<code>Timestamp::to_gregorian</code></li>
</ul>
<h2>What's Changed</h2>
<ul>
<li>Use const identifier in uuid macro by <a
href="https://github.com/Vrajs16"><code>@​Vrajs16</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/764">uuid-rs/uuid#764</a></li>
<li>Rename most methods referring to RFC4122 by <a
href="https://github.com/Mikopet"><code>@​Mikopet</code></a> / <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/765">uuid-rs/uuid#765</a></li>
<li>prepare for 1.10.0 release by <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/766">uuid-rs/uuid#766</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Vrajs16"><code>@​Vrajs16</code></a> made
their first contribution in <a
href="https://redirect.github.com/uuid-rs/uuid/pull/764">uuid-rs/uuid#764</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0">https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4b4c590ae3"><code>4b4c590</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/766">#766</a> from
uuid-rs/cargo/1.10.0</li>
<li><a
href="68eff32640"><code>68eff32</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/765">#765</a> from
uuid-rs/chore/time-fn-deprecations</li>
<li><a
href="3d5384da4b"><code>3d5384d</code></a>
update docs and deprecation messages for timestamp fns</li>
<li><a
href="de50f2091f"><code>de50f20</code></a>
renaming rfc4122 functions</li>
<li><a
href="4a8841792a"><code>4a88417</code></a>
prepare for 1.10.0 release</li>
<li><a
href="66b4fcef14"><code>66b4fce</code></a>
Merge pull request <a
href="https://redirect.github.com/uuid-rs/uuid/issues/764">#764</a> from
Vrajs16/main</li>
<li><a
href="8896e26c42"><code>8896e26</code></a>
Use expr instead of ident</li>
<li><a
href="09973d6aff"><code>09973d6</code></a>
Added changes</li>
<li><a
href="6edf3e8cd5"><code>6edf3e8</code></a>
Use const identifer in uuid macro</li>
<li>See full diff in <a
href="https://github.com/uuid-rs/uuid/compare/1.9.1...1.10.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=uuid&package-manager=cargo&previous-version=1.9.1&new-version=1.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-17 09:46:59 +08:00
Jack Wright
b68c7cf3fa
Make polars unpivot consistent with polars pivot (#13335)
# Description
Makes `polars unpivot` use the same arguments as `polars pivot` and
makes it consistent with the polars' rust api. Additionally, support for
the polar's streaming engine has been exposed on eager dataframes.
Previously, it would only work with lazy dataframes.


# User-Facing Changes
* `polars unpivot` argument `--columns`|`-c` has been renamed to
`--index`|`-i`
* `polars unpivot` argument `--values`|`-v` has been renamed to
`--on`|`-o`
* `polars unpivot` short argument for `--streamable` is now `-t` to make
it consistent with `polars pivot`. It was made `-t` for `polars pivot`
because `-s` is short for `--short`
2024-07-10 16:36:38 -05:00
Jack Wright
ff27d6a18e
Implemented a command to expose polar's pivot functionality (#13282)
# Description
Implementing pivot support 

The example below is a port of the [python API
example](https://docs.pola.rs/api/python/stable/reference/dataframe/api/polars.DataFrame.pivot.html)

<img width="1079" alt="Screenshot 2024-07-01 at 14 29 27"
src="https://github.com/nushell/nushell/assets/56345/277eb7a2-233b-4070-9d24-c2183805c1b8">

# User-Facing Changes
* Introduction of the `polars pivot` command
2024-07-09 10:17:20 -07:00
Jack Wright
8316a1597e
Polars: Check to see if the cache is empty before enabling GC. More logging (#13286)
There was a bug where anytime the plugin cache remove was called, the
plugin gc was turned back on. This probably happened when I added the
reference counter logic.
2024-07-03 06:44:26 -05:00
Jack Wright
720b4cbd01
Polars 0.41 Upgrade (#13238)
# Description
Upgrading to Polars 0.41

# User-Facing Changes
* `polars melt` has been renamed to `polars unpivot` to match the change
in the polars API. Additionally, it now supports lazy dataframes.
Introduced a `--streamable` option to use the polars streaming engine
for lazy frames.
* The parameter `outer` has been replaced with `full` in `polars join`
to match polars change.
* `polars value-count` now supports the column (rename count column),
parallelize (multithread), sort, and normalize options.

The list of polars changes can be found
[here](https://github.com/pola-rs/polars/releases/tag/rs-0.41.2)
2024-06-28 06:37:45 -05:00
Jack Wright
1f1f581357
Converted perf function to be a macro. Utilized the perf macro within the polars plugin. (#13224)
In this pull request, I converted the `perf` function within `nu_utils`
to a macro. This change facilitates easier usage within plugins by
allowing the use of `env_logger` and setting `RUST_LOG=nu_plugin_polars`
(or another plugin). Without this conversion, the `RUST_LOG` variable
would need to be set to `RUST_LOG=nu_utils::utils`, which is less
intuitive and impossible to narrow the perf results to one plugin.
2024-06-27 18:56:56 -05:00
dependabot[bot]
38ecb6d380
Bump uuid from 1.8.0 to 1.9.1 (#13227) 2024-06-26 06:43:46 +00:00
Jack Wright
0dd35cddcd
Bumping version to 0.95.1 (#13231)
Marks development for hotfix
2024-06-25 18:26:07 -07:00
Jakub Žádník
f93c6680bd
Bump to 0.95.0 (#13221)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-25 21:29:47 +03:00