Compare commits

...

49 Commits

Author SHA1 Message Date
JT
bc682066d8 Bump to 0.35 (#3884) 2021-08-03 20:01:09 +12:00
0a1cdc5107 Add the nu-serde crate (#3878)
* Add the nu-serde crate

nu-serde is a crate that can be used to turn a value implementing
`serde::Serialize` into a `nu-protocol::Value`. This has the potential to
significantly simplify plugin authorship.

This crate was the previously independent
[serde-nu](https://github.com/lily-mara/serde-nu) but the nushell maintainers
expressed an interest in having it added to the mainline nushell repository.

* fixup! Add the nu-serde crate
2021-07-31 22:03:13 -05:00
6984185e61 Better representation in nested dataframes (#3875)
* better dataframe representation in nested df

* Error message correction
2021-07-31 09:02:32 -05:00
5826126284 simple contains arguments (#3874) 2021-07-31 09:01:05 -05:00
762e528ec5 Support equals sign in shorthand environment variable values (#3869)
Some environment variables, such as `RUST_LOG` include equals signs. Nushell
should support this in the shorthand environment variable syntax so that
developers using these variables can control them easily. We accomplish this by
swapping `std::str::split` for `std::str::splitn`, which ensures that we only
consider the first equals sign in the string instead of all of them, which we
did previously.

Closes #3867
2021-07-31 09:10:28 +12:00
JT
c3de9848b4 Revert "Unify use of the surf crate (#3855)" (#3871)
This reverts commit 7f7af2bbaa.
2021-07-31 09:04:01 +12:00
370ae8c20c Add support for mult-doc YAML files (#3870)
Single doc YAML files are returned as before. Multi-doc YAML files are
wrapped in an outer Table. Empty YAML files return Nothing.
2021-07-30 15:45:19 -05:00
69083bfca0 Adding parse fix for power operator error on negative integers and de… (#3821)
* Adding parse fix for power operator error on negative integers and decimal

* Adding correct formatting

* Changed is negative check to follow conventions

* Adding tests

* Added fix for the negatives and added tests

* Removed comments
2021-07-30 07:08:57 -05:00
1e15f26e98 fix interpolated strings when using unicode (#3866)
* fix interpolated strings when using unicode

* added test case
2021-07-29 19:07:34 -05:00
23ba01d89c add performance metrics for measuring startup time (#3854)
* add performance metrics for measureing startup time

* removed some comments

* update so tests pass

* update default.toml for tests, merged main

* fix clippy lints

* wording changes
2021-07-29 18:52:40 -05:00
653cbe651f Going deeper (#3864)
* nuframe in its own type in UntaggedValue

* Removed eager dataframe from enum

* Dataframe created from list of values

* Corrected order in dataframe columns

* Returned tag from stream collection

* Removed series from dataframe commands

* Arithmetic operators

* forced push

* forced push

* Replace all command

* String commands

* appending operations with dfs

* Testing suite for dataframes

* Unit test for dataframe commands

* improved equality for dataframes

* moving all dataframe operations to protocol

* objects in dataframes

* Removed cloning when converting to row
2021-07-30 09:16:30 +12:00
JT
f3e487e829 Add named positionals to all (#3863) 2021-07-30 09:12:24 +12:00
9c016ad479 document compiling without openssl (#3862) 2021-07-30 08:21:20 +12:00
JT
e602647d4d Fix clippy lint and disable broken lint (#3865) 2021-07-30 08:11:47 +12:00
9696e4d315 Improve md5 and sha256 code (#3841)
* Refactor Hash code to simplify md5 and sha256 implementations

Md5 and Sha256 (and other future digests) require less boilerplate code
now. Error reporting includues the name of the hash again.

* Add missing hash sha256 test
2021-07-29 10:22:16 -05:00
7f7af2bbaa Unify use of the surf crate (#3855)
This brings the features used by the `nu_plugin_fetch` and
`nu_plugin_post` in line and drops the default-features, reducing
the number of pulled-in dependencies and avoiding a second round of
compilations.

Retry of #3777 but with different features, post and fetch plugin tested

Signed-off-by: Daniel Egger <daniel@eggers-club.de>
2021-07-29 19:26:38 +12:00
b190051e15 Fix select to insert nulls in sparse tables instead of ignoring absent values (#3857)
Select used to ignore absent values resulting in "squashing" where
columns for multiple rows could be squashed into a single row.

This fixes the problem by inserting null/nothing into absent value, thus
preserving the structure of rows. This makes sure that all selected
columns are present in each row.
2021-07-28 16:39:42 -05:00
83b28cad8d Remove dependencies (#3853)
* nuframe in its own type in UntaggedValue

* Removed eager dataframe from enum

* Dataframe created from list of values

* Corrected order in dataframe columns

* Returned tag from stream collection

* Removed series from dataframe commands

* Arithmetic operators

* forced push

* forced push

* Replace all command

* String commands

* appending operations with dfs

* Testing suite for dataframes

* Unit test for dataframe commands

* improved equality for dataframes

* moving all dataframe operations to protocol
2021-07-27 14:20:06 -05:00
ea42a84a4a Update implementing_a_command.md (#3848)
+  nu-command/src/command.rs has been modified to nu-command/src/mod.rs in nowable version
2021-07-27 09:03:52 -05:00
e4c282f0a6 added the ability to compare time units like 1hr < 2hr (#3845) 2021-07-26 15:19:32 -05:00
d54d7cc431 append dataframes (#3839) 2021-07-26 08:36:09 +12:00
111477aa74 Add sha256 to the hash command (#3836)
Hashers now uses on Rust Crypto Digest trait which makes it trivial to
implement additional hash functions.

The original `md5` crate does not implement the Digest trait and was
replaced by `md-5` crate which does. Sha256 uses already included `sha2`
crate.
2021-07-25 14:08:08 -05:00
JT
226739d13f Bump to 0.34.1 (#3835) 2021-07-25 22:58:33 +12:00
f1ee9113ac All is a DataFrame (#3812)
* nuframe in its own type in UntaggedValue

* Removed eager dataframe from enum

* Dataframe created from list of values

* Corrected order in dataframe columns

* Returned tag from stream collection

* Removed series from dataframe commands

* Arithmetic operators

* forced push

* forced push

* Replace all command

* String commands

* appending operations with dfs

* Testing suite for dataframes

* Unit test for dataframe commands

* improved equality for dataframes
2021-07-25 22:01:54 +12:00
9120a64cfb use chrono_humanize for datetime formatting (#3834)
* use chrono_humanize for datetime formatting

* fix tests
2021-07-25 20:38:45 +12:00
fcd624a722 add date humanize command (#3833)
* add `date humanize` command

* add docs
2021-07-25 17:33:31 +12:00
e6af7f75a1 Read from standard input in rm (#3763)
* Read from standard input in `rm`

With this change, rm falls back to reading from the standard input if no
arguments are supplied. This leads to more intuitive pipes as seen in
https://github.com/nushell/nushell/issues/2824

 ls | get name | sort-by | first 20 | each {rm $it} becomes
 ls | get name | sort-by | first 20 | rm

* [Fix] Run cargo fmt, make files a rest parameter, and fix cargo build warnings

* [Fix] Fix clippy suggestions
2021-07-25 15:01:53 +12:00
27f1e7b60c change describe so it doesn't output colored strings (#3832) 2021-07-25 06:34:11 +12:00
2e24de7f47 Support other variables than PATH in pathvar (2nd attempt) (#3828)
* Fix swapped PATH env var separators

* Support pathvar to manipulate other vars than PATH

* Add tests for pathvar and its subcommands

* Adjust pathvar tests to comply with env quirks

* Make pathvar tests work on non-Windows as well

* Compact the comments in pathvar tests

* Fix last failing test

Co-authored-by: Jakub Žádník <jakub.zadnik@tuni.fi>
2021-07-24 11:44:36 -05:00
e514204db0 Fix wrong path separator (#3829) 2021-07-24 08:42:50 -05:00
0f9e55dac6 Revert "Support other variables than PATH in pathvar (#3791)" (#3827)
This reverts commit f9f39c0a1c.
2021-07-23 09:03:28 -05:00
5d7677dd07 Implement into path conversion (#3811)
This allows converting strings to filepaths without having to use
`path expand` roundtrip.

Filepaths are taken as-is without any validation/conversion.
2021-07-23 19:14:02 +12:00
57073cc6cf port capitalize to engine-p (#3794)
Part of #3390.
2021-07-23 19:13:11 +12:00
f9f39c0a1c Support other variables than PATH in pathvar (#3791)
* Fix swapped PATH env var separators

* Support pathvar to manipulate other vars than PATH

* Add tests for pathvar and its subcommands

* Fix PATH env name for Windows

Seems like Windows uses PATH as well.

Co-authored-by: Jakub Žádník <jakub.zadnik@tuni.fi>
2021-07-23 19:11:56 +12:00
d88d7f26e4 fix typo in release.yml (#3824) 2021-07-22 12:42:11 -05:00
aeaedd2e5e Convert templates to github forms (#3818)
* Create feature_request.yaml

* Rename feature_request.yaml to feature_request.yml

* Update feature_request.yml

* Update feature_request.yml

* Create bug_report.yml

* Update bug_report.yml

* Update bug_report.yml

* Update feature_request.yml

* Delete bug_report.md

* Delete feature_request.md
2021-07-23 05:39:20 +12:00
1f4ef3b606 Add worflow to publish package in winget (#3819)
Remove the job in release workflow to publish to winget.
Trigger winget workflow on published release.

Co-authored-by: Alexandre Nedelec <Alexandre.Nedelec@azeo.com>
2021-07-22 11:38:03 -05:00
9b5db297a6 Replace command (#3823)
* replace command

* cargo fmt

* Signature correction
2021-07-22 08:45:46 -05:00
b7215b5dde Fix expected age of mockup files in example tests (#3808) 2021-07-20 18:05:58 -05:00
411435d68f Dataframe Shape command (#3805)
* size command to get dataframe info

* change command name to shape

* apply lint to file
2021-07-20 07:07:42 -05:00
f656f906ff Update stale.yml
update days-before-issue-stale to 90 days
2021-07-19 15:03:24 -05:00
d0a7363e64 bat theme wasn't getting set properly (#3807) 2021-07-19 14:54:36 -05:00
7401fa2fa5 Fix docs for the config variable completion_type (#3804) 2021-07-19 07:20:56 -05:00
181ee1dade Revert "Unify use of the surf crate (#3777)" (#3783)
This reverts commit 37612345f2.
2021-07-14 20:21:18 -05:00
3645a0f0e4 Updated polars version for faster CSV reader (#3781) 2021-07-14 15:33:21 -05:00
2864eaebae fixed show_hints option to allow hints to be turned off (#3780) 2021-07-14 09:47:33 -05:00
37612345f2 Unify use of the surf crate (#3777)
This brings the features used by the `nu_plugin_fetch` and
`nu_plugin_post` in line and drops the default-features, reducing
the number of pulled-in dependencies and avoiding a second round of
compilations.

Signed-off-by: Daniel Egger <daniel@eggers-club.de>
2021-07-14 08:47:49 -05:00
bb218b824e corrected position of dataframes (#3776) 2021-07-14 08:46:32 -05:00
279329bfaa Add the -s parameter to submit package to winget in pipeline (#3767)
Co-authored-by: Alexandre Nedelec <Alexandre.Nedelec@azeo.com>
2021-07-13 18:35:56 -05:00
209 changed files with 7652 additions and 2407 deletions

View File

@ -49,7 +49,7 @@ steps:
- bash: RUSTFLAGS="-D warnings" cargo test --all
condition: eq(variables['style'], 'unflagged')
displayName: Run tests
- bash: RUSTFLAGS="-D warnings" cargo clippy --all -- -D clippy::unwrap_used
- bash: RUSTFLAGS="-D warnings" cargo clippy --all -- -D clippy::unwrap_used -A clippy::needless_collect
condition: eq(variables['style'], 'unflagged')
displayName: Check clippy lints
- bash: cd samples/wasm && npm install wasm-pack && node ./node_modules/wasm-pack/run.js build

View File

@ -1,47 +0,0 @@
---
name: Bug report
about: Create a report to help us improve
title: ''
labels: ''
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1.
2.
3.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Configuration (please complete the following information):**
Run `version | pivot` and paste the output to show OS, features, etc.
```
> version | pivot
╭───┬────────────────────┬───────────────────────────────────────────────────────────────────────╮
│ # │ Column0 │ Column1 │
├───┼────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ 0 │ version │ 0.24.1 │
│ 1 │ build_os │ macos-x86_64 │
│ 2 │ rust_version │ rustc 1.48.0 │
│ 3 │ cargo_version │ cargo 1.48.0 │
│ 4 │ pkg_version │ 0.24.1 │
│ 5 │ build_time │ 2020-12-18 09:54:09 │
│ 6 │ build_rust_channel │ release │
│ 7 │ features │ ctrlc, default, directories, dirs, git, ichwh, rich-benchmark, │
│ │ │ rustyline, term, uuid, which, zip │
╰───┴────────────────────┴───────────────────────────────────────────────────────────────────────╯
```
**Add any other context about the problem here.**

65
.github/ISSUE_TEMPLATE/bug_report.yml vendored Normal file
View File

@ -0,0 +1,65 @@
name: Bug Report
description: Create a report to help us improve
body:
- type: textarea
id: description
attributes:
label: Describe the Bug
description: A clear and concise description of what the bug is.
validations:
required: true
- type: textarea
id: repro
attributes:
label: To Reproduce
description: Steps to reproduce the behavior
placeholder: |
1.
2.
3.
validations:
required: true
- type: textarea
id: expected
attributes:
label: Expected behavior
description: A clear and concise description of what you expected to happen.
placeholder: I expected nu to...
validations:
required: true
- type: textarea
id: screenshots
attributes:
label: Screenshots
description: Please add any relevant screenshots here, if any
validations:
required: false
- type: textarea
id: config
attributes:
label: Configuration
description: "Please run `> version | pivot` and paste the output to show OS, features, etc"
placeholder: |
> version | pivot
╭───┬────────────────────┬───────────────────────────────────────────────────────────────────────╮
│ # │ Column0 │ Column1 │
├───┼────────────────────┼───────────────────────────────────────────────────────────────────────┤
│ 0 │ version │ 0.24.1 │
│ 1 │ build_os │ macos-x86_64 │
│ 2 │ rust_version │ rustc 1.48.0 │
│ 3 │ cargo_version │ cargo 1.48.0 │
│ 4 │ pkg_version │ 0.24.1 │
│ 5 │ build_time │ 2020-12-18 09:54:09 │
│ 6 │ build_rust_channel │ release │
│ 7 │ features │ ctrlc, default, directories, dirs, git, ichwh, rich-benchmark, │
│ │ │ rustyline, term, uuid, which, zip │
╰───┴────────────────────┴───────────────────────────────────────────────────────────────────────╯
validations:
required: false
- type: textarea
id: context
attributes:
label: Additional Context
description: Add any other context about the problem here.
validations:
required: false

View File

@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: ''
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.

View File

@ -0,0 +1,34 @@
name: Feature Request
description: "When you want a new feature for something that doesn't already exist"
body:
- type: textarea
id: problem
attributes:
label: Related Problem
description: Is your feature request related to a problem? Please describe.
placeholder: |
A clear and concise description of what the problem is.
Example: I am trying to do [...] but [...]
validations:
required: false
- type: textarea
id: desired
attributes:
label: "Describe the solution you'd like"
description: A clear and concise description of what you want to happen.
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: "Describe alternatives you've considered"
description: "A clear and concise description of any alternative solutions or features you've considered."
validations:
required: false
- type: textarea
id: context
attributes:
label: Additional Context and Details
description: Add any other context or screenshots about the feature request here.
validations:
required: false

View File

@ -162,8 +162,6 @@ jobs:
- linux
- macos
- windows
outputs:
version: ${{ steps.info.outputs.version }}
steps:
- name: Check out code
uses: actions/checkout@v2
@ -285,19 +283,4 @@ jobs:
upload_url: ${{ steps.create_release.outputs.upload_url }}
asset_path: ./nushell-windows.msi
asset_name: ${{ steps.info.outputs.windowsdir }}.msi
asset_content_type: applictaion/x-msi
winget:
name: Publish winget package
runs-on: windows-latest
needs: release
steps:
- name: Submit package to Windows Package Manager Community Repository
run: |
iwr https://aka.ms/wingetcreate/latest -OutFile wingetcreate.exe
$nushellVersion="${{ needs.release.outputs.version }}"
$nushellInstallerName="nu_$($nushellVersion -replace '\.','_')_windows.msi"
$installerUrl = "https://github.com/nushell/nushell/releases/download/$nushellVersion/$nushellInstallerName"
.\wingetcreate.exe update Nushell.Nushell -v $nushellVersion -u $installerUrl -t ${{ secrets.NUSHELL_PAT }}
asset_content_type: application/x-msi

View File

@ -19,11 +19,11 @@ jobs:
operations-per-run: 520
enable-statistics: true
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'This issue is being marked stale because it has been open for 30 days without activity. If you feel that this is in error, please comment below and we will keep it marked as active.'
stale-issue-message: 'This issue is being marked stale because it has been open for 90 days without activity. If you feel that this is in error, please comment below and we will keep it marked as active.'
stale-pr-message: 'This PR is being marked stale because it has been open for 45 days without activity. If this PR is still active, please comment below and we will keep it marked as active.'
close-issue-message: 'This issue has been marked stale for more than 10 days without activity. Closing this issue, but if you find that the issue is still valid, please reopen.'
close-pr-message: 'This PR has been marked stale for more than 10 days without activity. Closing this PR, but if you are still working on it, please reopen.'
days-before-issue-stale: 30
days-before-issue-stale: 90
days-before-pr-stale: 45
days-before-issue-close: 10
days-before-pr-close: 10

18
.github/workflows/winget-submission.yml vendored Normal file
View File

@ -0,0 +1,18 @@
name: Submit Nushell package to Windows Package Manager Community Repository
on:
release:
types: [published]
jobs:
winget:
name: Publish winget package
runs-on: windows-latest
steps:
- name: Submit package to Windows Package Manager Community Repository
run: |
iwr https://aka.ms/wingetcreate/latest -OutFile wingetcreate.exe
$github = Get-Content '${{ github.event_path }}' | ConvertFrom-Json
$installerUrl = $github.release.assets | Where-Object -Property name -match 'windows.msi' | Select -ExpandProperty browser_download_url -First 1
.\wingetcreate.exe update Nushell.Nushell -s -v $github.release.tag_name -u $installerUrl -t ${{ secrets.NUSHELL_PAT }}

175
Cargo.lock generated
View File

@ -746,6 +746,15 @@ dependencies = [
"winapi 0.3.9",
]
[[package]]
name = "chrono-humanize"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2eddc119501d583fd930cb92144e605f44e0252c38dd89d9247fffa1993375cb"
dependencies = [
"chrono",
]
[[package]]
name = "chrono-tz"
version = "0.5.3"
@ -1315,6 +1324,15 @@ dependencies = [
"winapi 0.3.9",
]
[[package]]
name = "dirs"
version = "3.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "30baa043103c9d0c2a57cf537cc2f35623889dc0d405e6c3cccfadbc81c71309"
dependencies = [
"dirs-sys",
]
[[package]]
name = "dirs-next"
version = "2.0.0"
@ -2491,6 +2509,21 @@ dependencies = [
"adler32",
]
[[package]]
name = "insta"
version = "1.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4a1b21a2971cea49ca4613c0e9fe8225ecaf5de64090fddc6002284726e9244"
dependencies = [
"console",
"lazy_static 1.4.0",
"serde 1.0.126",
"serde_json",
"serde_yaml",
"similar",
"uuid",
]
[[package]]
name = "instant"
version = "0.1.9"
@ -2812,6 +2845,17 @@ version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ffc5c5338469d4d3ea17d269fa8ea3512ad247247c30bd2df69e68309ed0a08"
[[package]]
name = "md-5"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b5a279bb9607f9f53c22d496eade00d138d1bdcccd07d74650387cf94942a15"
dependencies = [
"block-buffer 0.9.0",
"digest 0.9.0",
"opaque-debug",
]
[[package]]
name = "md5"
version = "0.6.1"
@ -3109,7 +3153,7 @@ dependencies = [
[[package]]
name = "nu"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"ctrlc",
"dunce",
@ -3154,7 +3198,7 @@ dependencies = [
[[package]]
name = "nu-ansi-term"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"doc-comment",
"itertools",
@ -3167,10 +3211,11 @@ dependencies = [
[[package]]
name = "nu-cli"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"ctrlc",
"indexmap",
"lazy_static 1.4.0",
"log",
"nu-ansi-term",
"nu-command",
@ -3192,7 +3237,7 @@ dependencies = [
[[package]]
name = "nu-command"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"Inflector",
"arboard",
@ -3208,6 +3253,7 @@ dependencies = [
"csv",
"ctrlc",
"derive-new",
"digest 0.9.0",
"directories-next",
"dirs-next",
"dtparse",
@ -3226,7 +3272,7 @@ dependencies = [
"itertools",
"lazy_static 1.4.0",
"log",
"md5 0.7.0",
"md-5",
"meval",
"minus",
"nu-ansi-term",
@ -3290,7 +3336,7 @@ dependencies = [
[[package]]
name = "nu-completion"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"dirs-next",
"indexmap",
@ -3308,7 +3354,7 @@ dependencies = [
[[package]]
name = "nu-data"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"byte-unit",
@ -3330,7 +3376,6 @@ dependencies = [
"num-bigint 0.3.2",
"num-format",
"num-traits 0.2.14",
"polars",
"query_interface",
"serde 1.0.126",
"sha2 0.9.5",
@ -3341,7 +3386,7 @@ dependencies = [
[[package]]
name = "nu-engine"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"ansi_term 0.12.1",
"async-recursion",
@ -3396,7 +3441,7 @@ dependencies = [
[[package]]
name = "nu-errors"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"codespan-reporting",
@ -3415,7 +3460,7 @@ dependencies = [
[[package]]
name = "nu-json"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"dunce",
"lazy_static 1.4.0",
@ -3429,7 +3474,7 @@ dependencies = [
[[package]]
name = "nu-parser"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"codespan-reporting",
@ -3452,7 +3497,7 @@ dependencies = [
[[package]]
name = "nu-path"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"dirs-next",
"dunce",
@ -3460,7 +3505,7 @@ dependencies = [
[[package]]
name = "nu-plugin"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"indexmap",
@ -3476,7 +3521,7 @@ dependencies = [
[[package]]
name = "nu-pretty-hex"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"heapless",
"nu-ansi-term",
@ -3485,11 +3530,12 @@ dependencies = [
[[package]]
name = "nu-protocol"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"byte-unit",
"chrono",
"chrono-humanize",
"derive-new",
"getset",
"indexmap",
@ -3507,9 +3553,21 @@ dependencies = [
"toml",
]
[[package]]
name = "nu-serde"
version = "0.35.0"
dependencies = [
"bigdecimal",
"insta",
"nu-protocol",
"nu-source",
"serde 1.0.126",
"thiserror",
]
[[package]]
name = "nu-source"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"derive-new",
"getset",
@ -3520,7 +3578,7 @@ dependencies = [
[[package]]
name = "nu-stream"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"futures 0.3.15",
"nu-errors",
@ -3530,7 +3588,7 @@ dependencies = [
[[package]]
name = "nu-table"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"nu-ansi-term",
"regex",
@ -3539,7 +3597,7 @@ dependencies = [
[[package]]
name = "nu-test-support"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"chrono",
@ -3558,7 +3616,7 @@ dependencies = [
[[package]]
name = "nu-value-ext"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"indexmap",
"itertools",
@ -3570,7 +3628,7 @@ dependencies = [
[[package]]
name = "nu_plugin_binaryview"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"crossterm",
"image 0.22.5",
@ -3586,7 +3644,7 @@ dependencies = [
[[package]]
name = "nu_plugin_chart"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"crossterm",
"nu-data",
@ -3600,7 +3658,7 @@ dependencies = [
[[package]]
name = "nu_plugin_fetch"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"base64",
"futures 0.3.15",
@ -3615,7 +3673,7 @@ dependencies = [
[[package]]
name = "nu_plugin_from_bson"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"bson",
@ -3641,7 +3699,7 @@ dependencies = [
[[package]]
name = "nu_plugin_from_sqlite"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"nu-errors",
@ -3656,7 +3714,7 @@ dependencies = [
[[package]]
name = "nu_plugin_inc"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"nu-errors",
"nu-plugin",
@ -3669,7 +3727,7 @@ dependencies = [
[[package]]
name = "nu_plugin_match"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"nu-errors",
"nu-plugin",
@ -3680,7 +3738,7 @@ dependencies = [
[[package]]
name = "nu_plugin_post"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"base64",
"futures 0.3.15",
@ -3697,7 +3755,7 @@ dependencies = [
[[package]]
name = "nu_plugin_ps"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"futures 0.3.15",
"futures-timer",
@ -3711,7 +3769,7 @@ dependencies = [
[[package]]
name = "nu_plugin_query_json"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"gjson",
"nu-errors",
@ -3723,7 +3781,7 @@ dependencies = [
[[package]]
name = "nu_plugin_s3"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"futures 0.3.15",
"nu-errors",
@ -3735,7 +3793,7 @@ dependencies = [
[[package]]
name = "nu_plugin_selector"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"nipper",
"nu-errors",
@ -3747,7 +3805,7 @@ dependencies = [
[[package]]
name = "nu_plugin_start"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"glob",
"nu-errors",
@ -3761,7 +3819,7 @@ dependencies = [
[[package]]
name = "nu_plugin_sys"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"futures 0.3.15",
"futures-util",
@ -3775,7 +3833,7 @@ dependencies = [
[[package]]
name = "nu_plugin_textview"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bat",
"nu-ansi-term",
@ -3790,7 +3848,7 @@ dependencies = [
[[package]]
name = "nu_plugin_to_bson"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bson",
"nu-errors",
@ -3803,7 +3861,7 @@ dependencies = [
[[package]]
name = "nu_plugin_to_sqlite"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"hex 0.4.3",
"nu-errors",
@ -3818,7 +3876,7 @@ dependencies = [
[[package]]
name = "nu_plugin_tree"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"derive-new",
"nu-errors",
@ -3830,7 +3888,7 @@ dependencies = [
[[package]]
name = "nu_plugin_xpath"
version = "0.34.0"
version = "0.35.0"
dependencies = [
"bigdecimal",
"indexmap",
@ -4419,9 +4477,9 @@ dependencies = [
[[package]]
name = "polars"
version = "0.14.5"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f254b79757346a86a8371ea4a087ce6a56e604c82d61093a1b85bfd0df99aeb"
checksum = "9ed1645f3daba885ecedef8a76839536bfc7263f1e4fc0b67b27282f19325756"
dependencies = [
"polars-core",
"polars-io",
@ -4430,9 +4488,9 @@ dependencies = [
[[package]]
name = "polars-arrow"
version = "0.14.5"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec1ef88e60b660c51644a5b098570519948d95f389b67ef690a0f1187395d7bf"
checksum = "5943252213fbd21e26b62240f5bb73f5d874e1072ed93234c6860781a4c366f2"
dependencies = [
"arrow",
"num 0.4.0",
@ -4441,9 +4499,9 @@ dependencies = [
[[package]]
name = "polars-core"
version = "0.14.5"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9e5e6ee23eb50845501c8c31368051af75801185cf4bedf9e7b3ec945a49af9c"
checksum = "df43d51c6314623deafdd1266ca9a8431f087d9d345fffc8eeb16508aec69fee"
dependencies = [
"ahash",
"anyhow",
@ -4462,21 +4520,23 @@ dependencies = [
"rayon",
"regex",
"serde 1.0.126",
"serde_json",
"thiserror",
"unsafe_unwrap",
]
[[package]]
name = "polars-io"
version = "0.14.5"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94e8719cdb70555e0492dd24e8f09f637cc112bac438be754bad8dca75f466ab"
checksum = "cd998aab494826fb08d9c83763fc2925c9aa4eab2e4b349416bb174b9b300a3a"
dependencies = [
"ahash",
"anyhow",
"arrow",
"csv",
"csv-core",
"dirs 3.0.2",
"fast-float",
"lazy_static 1.4.0",
"lexical",
@ -4488,13 +4548,14 @@ dependencies = [
"polars-core",
"rayon",
"regex",
"simdutf8",
]
[[package]]
name = "polars-lazy"
version = "0.14.5"
version = "0.14.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ca6b2fb59bbe6725a84c48df12f509b4655d173cd113e5fb51f971cff1f93bc"
checksum = "6596b6d1cf1e378976ad9f5d2abe88ba08f112684ca1b67fbe89b97832c9f11d"
dependencies = [
"ahash",
"itertools",
@ -5576,6 +5637,18 @@ dependencies = [
"libc",
]
[[package]]
name = "simdutf8"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c970da16e7c682fa90a261cf0724dee241c9f7831635ecc4e988ae8f3b505559"
[[package]]
name = "similar"
version = "1.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ad1d488a557b235fc46dae55512ffbfc429d2482b08b4d9435ab07384ca8aec"
[[package]]
name = "siphasher"
version = "0.3.5"
@ -5976,7 +6049,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "edd106a334b7657c10b7c540a0106114feadeb4dc314513e97df481d5d966f42"
dependencies = [
"byteorder",
"dirs",
"dirs 1.0.5",
"winapi 0.3.9",
]

View File

@ -10,7 +10,7 @@ license = "MIT"
name = "nu"
readme = "README.md"
repository = "https://github.com/nushell/nushell"
version = "0.34.0"
version = "0.35.0"
[workspace]
members = ["crates/*/"]
@ -18,38 +18,38 @@ members = ["crates/*/"]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
nu-cli = { version = "0.34.0", path="./crates/nu-cli", default-features=false }
nu-command = { version = "0.34.0", path="./crates/nu-command" }
nu-completion = { version = "0.34.0", path="./crates/nu-completion" }
nu-data = { version = "0.34.0", path="./crates/nu-data" }
nu-engine = { version = "0.34.0", path="./crates/nu-engine" }
nu-errors = { version = "0.34.0", path="./crates/nu-errors" }
nu-parser = { version = "0.34.0", path="./crates/nu-parser" }
nu-path = { version = "0.34.0", path="./crates/nu-path" }
nu-plugin = { version = "0.34.0", path="./crates/nu-plugin" }
nu-protocol = { version = "0.34.0", path="./crates/nu-protocol" }
nu-source = { version = "0.34.0", path="./crates/nu-source" }
nu-value-ext = { version = "0.34.0", path="./crates/nu-value-ext" }
nu-cli = { version = "0.35.0", path="./crates/nu-cli", default-features=false }
nu-command = { version = "0.35.0", path="./crates/nu-command" }
nu-completion = { version = "0.35.0", path="./crates/nu-completion" }
nu-data = { version = "0.35.0", path="./crates/nu-data" }
nu-engine = { version = "0.35.0", path="./crates/nu-engine" }
nu-errors = { version = "0.35.0", path="./crates/nu-errors" }
nu-parser = { version = "0.35.0", path="./crates/nu-parser" }
nu-path = { version = "0.35.0", path="./crates/nu-path" }
nu-plugin = { version = "0.35.0", path="./crates/nu-plugin" }
nu-protocol = { version = "0.35.0", path="./crates/nu-protocol" }
nu-source = { version = "0.35.0", path="./crates/nu-source" }
nu-value-ext = { version = "0.35.0", path="./crates/nu-value-ext" }
nu_plugin_binaryview = { version = "0.34.0", path="./crates/nu_plugin_binaryview", optional=true }
nu_plugin_chart = { version = "0.34.0", path="./crates/nu_plugin_chart", optional=true }
nu_plugin_fetch = { version = "0.34.0", path="./crates/nu_plugin_fetch", optional=true }
nu_plugin_from_bson = { version = "0.34.0", path="./crates/nu_plugin_from_bson", optional=true }
nu_plugin_from_sqlite = { version = "0.34.0", path="./crates/nu_plugin_from_sqlite", optional=true }
nu_plugin_inc = { version = "0.34.0", path="./crates/nu_plugin_inc", optional=true }
nu_plugin_match = { version = "0.34.0", path="./crates/nu_plugin_match", optional=true }
nu_plugin_post = { version = "0.34.0", path="./crates/nu_plugin_post", optional=true }
nu_plugin_ps = { version = "0.34.0", path="./crates/nu_plugin_ps", optional=true }
nu_plugin_query_json = { version = "0.34.0", path="./crates/nu_plugin_query_json", optional=true }
nu_plugin_s3 = { version = "0.34.0", path="./crates/nu_plugin_s3", optional=true }
nu_plugin_selector = { version = "0.34.0", path="./crates/nu_plugin_selector", optional=true }
nu_plugin_start = { version = "0.34.0", path="./crates/nu_plugin_start", optional=true }
nu_plugin_sys = { version = "0.34.0", path="./crates/nu_plugin_sys", optional=true }
nu_plugin_textview = { version = "0.34.0", path="./crates/nu_plugin_textview", optional=true }
nu_plugin_to_bson = { version = "0.34.0", path="./crates/nu_plugin_to_bson", optional=true }
nu_plugin_to_sqlite = { version = "0.34.0", path="./crates/nu_plugin_to_sqlite", optional=true }
nu_plugin_tree = { version = "0.34.0", path="./crates/nu_plugin_tree", optional=true }
nu_plugin_xpath = { version = "0.34.0", path="./crates/nu_plugin_xpath", optional=true }
nu_plugin_binaryview = { version = "0.35.0", path="./crates/nu_plugin_binaryview", optional=true }
nu_plugin_chart = { version = "0.35.0", path="./crates/nu_plugin_chart", optional=true }
nu_plugin_fetch = { version = "0.35.0", path="./crates/nu_plugin_fetch", optional=true }
nu_plugin_from_bson = { version = "0.35.0", path="./crates/nu_plugin_from_bson", optional=true }
nu_plugin_from_sqlite = { version = "0.35.0", path="./crates/nu_plugin_from_sqlite", optional=true }
nu_plugin_inc = { version = "0.35.0", path="./crates/nu_plugin_inc", optional=true }
nu_plugin_match = { version = "0.35.0", path="./crates/nu_plugin_match", optional=true }
nu_plugin_post = { version = "0.35.0", path="./crates/nu_plugin_post", optional=true }
nu_plugin_ps = { version = "0.35.0", path="./crates/nu_plugin_ps", optional=true }
nu_plugin_query_json = { version = "0.35.0", path="./crates/nu_plugin_query_json", optional=true }
nu_plugin_s3 = { version = "0.35.0", path="./crates/nu_plugin_s3", optional=true }
nu_plugin_selector = { version = "0.35.0", path="./crates/nu_plugin_selector", optional=true }
nu_plugin_start = { version = "0.35.0", path="./crates/nu_plugin_start", optional=true }
nu_plugin_sys = { version = "0.35.0", path="./crates/nu_plugin_sys", optional=true }
nu_plugin_textview = { version = "0.35.0", path="./crates/nu_plugin_textview", optional=true }
nu_plugin_to_bson = { version = "0.35.0", path="./crates/nu_plugin_to_bson", optional=true }
nu_plugin_to_sqlite = { version = "0.35.0", path="./crates/nu_plugin_to_sqlite", optional=true }
nu_plugin_tree = { version = "0.35.0", path="./crates/nu_plugin_tree", optional=true }
nu_plugin_xpath = { version = "0.35.0", path="./crates/nu_plugin_xpath", optional=true }
# Required to bootstrap the main binary
ctrlc = { version="3.1.7", optional=true }
@ -57,7 +57,7 @@ futures = { version="0.3.12", features=["compat", "io-compat"] }
itertools = "0.10.0"
[dev-dependencies]
nu-test-support = { version = "0.34.0", path="./crates/nu-test-support" }
nu-test-support = { version = "0.35.0", path="./crates/nu-test-support" }
dunce = "1.0.1"
serial_test = "0.5.1"
hamcrest2 = "0.3.0"

View File

@ -9,7 +9,7 @@ description = "Library for ANSI terminal colors and styles (bold, underline)"
edition = "2018"
license = "MIT"
name = "nu-ansi-term"
version = "0.34.0"
version = "0.35.0"
[lib]
doctest = false

View File

@ -4,23 +4,23 @@ description = "CLI for nushell"
edition = "2018"
license = "MIT"
name = "nu-cli"
version = "0.34.0"
version = "0.35.0"
build = "build.rs"
[lib]
doctest = false
[dependencies]
nu-completion = { version = "0.34.0", path="../nu-completion" }
nu-command = { version = "0.34.0", path="../nu-command" }
nu-data = { version = "0.34.0", path="../nu-data" }
nu-engine = { version = "0.34.0", path="../nu-engine" }
nu-errors = { version = "0.34.0", path="../nu-errors" }
nu-parser = { version = "0.34.0", path="../nu-parser" }
nu-protocol = { version = "0.34.0", path="../nu-protocol" }
nu-source = { version = "0.34.0", path="../nu-source" }
nu-stream = { version = "0.34.0", path="../nu-stream" }
nu-ansi-term = { version = "0.34.0", path="../nu-ansi-term" }
nu-completion = { version = "0.35.0", path="../nu-completion" }
nu-command = { version = "0.35.0", path="../nu-command" }
nu-data = { version = "0.35.0", path="../nu-data" }
nu-engine = { version = "0.35.0", path="../nu-engine" }
nu-errors = { version = "0.35.0", path="../nu-errors" }
nu-parser = { version = "0.35.0", path="../nu-parser" }
nu-protocol = { version = "0.35.0", path="../nu-protocol" }
nu-source = { version = "0.35.0", path="../nu-source" }
nu-stream = { version = "0.35.0", path="../nu-stream" }
nu-ansi-term = { version = "0.35.0", path="../nu-ansi-term" }
indexmap ="1.6.1"
log = "0.4.14"
@ -31,6 +31,7 @@ ctrlc = { version="3.1.7", optional=true }
shadow-rs = { version="0.6", default-features=false, optional=true }
serde = { version="1.0.123", features=["derive"] }
serde_yaml = "0.8.16"
lazy_static = "1.4.0"
[build-dependencies]
shadow-rs = "0.6"

View File

@ -1,10 +1,10 @@
mod logger;
mod options;
mod options_parser;
pub mod stopwatch;
pub use options::{CliOptions, NuScript, Options};
use options_parser::{NuParser, OptionsParser};
use self::stopwatch::Stopwatch;
use lazy_static::lazy_static;
use nu_command::{commands::NuSignature as Nu, utils::test_bins as binaries};
use nu_engine::{get_full_help, EvaluationContext};
use nu_errors::ShellError;
@ -12,6 +12,18 @@ use nu_protocol::hir::{Call, Expression, SpannedExpression, Synthetic};
use nu_protocol::{Primitive, UntaggedValue};
use nu_source::{Span, Tag};
use nu_stream::InputStream;
pub use options::{CliOptions, NuScript, Options};
use options_parser::{NuParser, OptionsParser};
use std::sync::Mutex;
lazy_static! {
pub static ref STOPWATCH: Mutex<Stopwatch> = {
let mut sw = Stopwatch::default();
sw.start();
sw.stop();
Mutex::new(sw)
};
}
pub struct App {
parser: Box<dyn OptionsParser>,
@ -32,6 +44,14 @@ impl App {
}
pub fn main(&self, argv: &[String]) -> Result<(), ShellError> {
if self.perf() {
// start the stopwatch running
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
let argv = quote_positionals(argv).join(" ");
if let Err(cause) = self.parse(&argv) {
@ -52,6 +72,21 @@ impl App {
consume(context, stream)?;
if self.perf() {
// stop the stopwatch since we're exiting
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.stop();
eprintln!(
"help {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed()
);
}
std::process::exit(0);
}
@ -93,6 +128,22 @@ impl App {
};
consume(context, stream)?;
if self.perf() {
// stop the stopwatch since we're exiting
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.stop();
eprintln!(
"version {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed()
);
}
std::process::exit(0);
}
@ -113,21 +164,29 @@ impl App {
}
let mut opts = CliOptions::new();
opts.config = self.config().map(std::ffi::OsString::from);
opts.stdin = self.takes_stdin();
opts.save_history = self.save_history();
opts.perf = self.perf();
use logger::{configure, debug_filters, logger, trace_filters};
logger(|builder| {
configure(&self, builder)?;
trace_filters(&self, builder)?;
debug_filters(&self, builder)?;
configure(self, builder)?;
trace_filters(self, builder)?;
debug_filters(self, builder)?;
Ok(())
})?;
if self.perf() {
// start a new split
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start()
}
if let Some(commands) = self.commands() {
let commands = commands?;
let script = NuScript::code(&commands)?;
@ -136,6 +195,21 @@ impl App {
return crate::run_script_file(context, opts);
}
if self.perf() {
// start a new spit
eprintln!(
"commands using -c at launch: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
if let Some(scripts) = self.scripts() {
let mut source_files = vec![];
for script in scripts {
@ -162,12 +236,38 @@ impl App {
return Ok(());
}
if self.perf() {
// start a new split
eprintln!(
"script file(s) passed in: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
let context = crate::create_default_context(true)?;
if !self.skip_plugins() {
let _ = crate::register_plugins(&context);
}
if self.perf() {
// start a new split
eprintln!(
"plugins registered: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
}
#[cfg(feature = "rustyline-support")]
{
crate::cli(context, opts)?;
@ -189,6 +289,13 @@ impl App {
})
}
pub fn perf(&self) -> bool {
self.options
.get("perf")
.map(|v| matches!(v.as_bool(), Ok(true)))
.unwrap_or(false)
}
pub fn help(&self) -> bool {
self.options
.get("help")
@ -298,7 +405,7 @@ impl App {
}
pub fn parse(&self, args: &str) -> Result<(), ShellError> {
self.parser.parse(&args).map(|options| {
self.parser.parse(args).map(|options| {
self.options.swap(&options);
})
}

View File

@ -10,6 +10,7 @@ pub struct CliOptions {
pub stdin: bool,
pub scripts: Vec<NuScript>,
pub save_history: bool,
pub perf: bool,
}
impl Default for CliOptions {
@ -25,6 +26,7 @@ impl CliOptions {
stdin: false,
scripts: vec![],
save_history: true,
perf: false,
}
}
}

View File

@ -111,7 +111,7 @@ impl OptionsParser for NuParser {
.flatten();
if let Some(value) = value {
options.put(&k, value);
options.put(k, value);
}
});
}

View File

@ -0,0 +1,118 @@
#![allow(dead_code)]
use std::default::Default;
use std::fmt;
use std::time::{Duration, Instant};
#[derive(Clone, Copy)]
pub struct Stopwatch {
/// The time the stopwatch was started last, if ever.
start_time: Option<Instant>,
/// The time the stopwatch was split last, if ever.
split_time: Option<Instant>,
/// The time elapsed while the stopwatch was running (between start() and stop()).
elapsed: Duration,
}
impl Default for Stopwatch {
fn default() -> Stopwatch {
Stopwatch {
start_time: None,
split_time: None,
elapsed: Duration::from_secs(0),
}
}
}
impl fmt::Display for Stopwatch {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
return write!(f, "{}ms", self.elapsed_ms());
}
}
impl Stopwatch {
/// Returns a new stopwatch.
pub fn new() -> Stopwatch {
let sw: Stopwatch = Default::default();
sw
}
/// Returns a new stopwatch which will immediately be started.
pub fn start_new() -> Stopwatch {
let mut sw = Stopwatch::new();
sw.start();
sw
}
/// Starts the stopwatch.
pub fn start(&mut self) {
self.start_time = Some(Instant::now());
}
/// Stops the stopwatch.
pub fn stop(&mut self) {
self.elapsed = self.elapsed();
self.start_time = None;
self.split_time = None;
}
/// Resets all counters and stops the stopwatch.
pub fn reset(&mut self) {
self.elapsed = Duration::from_secs(0);
self.start_time = None;
self.split_time = None;
}
/// Resets and starts the stopwatch again.
pub fn restart(&mut self) {
self.reset();
self.start();
}
/// Returns whether the stopwatch is running.
pub fn is_running(&self) -> bool {
self.start_time.is_some()
}
/// Returns the elapsed time since the start of the stopwatch.
pub fn elapsed(&self) -> Duration {
match self.start_time {
// stopwatch is running
Some(t1) => t1.elapsed() + self.elapsed,
// stopwatch is not running
None => self.elapsed,
}
}
/// Returns the elapsed time since the start of the stopwatch in milliseconds.
pub fn elapsed_ms(&self) -> i64 {
let dur = self.elapsed();
(dur.as_secs() * 1000 + dur.subsec_millis() as u64) as i64
}
/// Returns the elapsed time since last split or start/restart.
///
/// If the stopwatch is in stopped state this will always return a zero Duration.
pub fn elapsed_split(&mut self) -> Duration {
match self.start_time {
// stopwatch is running
Some(start) => {
let res = match self.split_time {
Some(split) => split.elapsed(),
None => start.elapsed(),
};
self.split_time = Some(Instant::now());
res
}
// stopwatch is not running
None => Duration::from_secs(0),
}
}
/// Returns the elapsed time since last split or start/restart in milliseconds.
///
/// If the stopwatch is in stopped state this will always return zero.
pub fn elapsed_split_ms(&mut self) -> i64 {
let dur = self.elapsed_split();
(dur.as_secs() * 1000 + dur.subsec_millis() as u64) as i64
}
}

View File

@ -1,3 +1,4 @@
use crate::app::STOPWATCH;
use crate::line_editor::configure_ctrl_c;
use nu_ansi_term::Color;
use nu_engine::{maybe_print_errors, run_block, script::run_script_standalone, EvaluationContext};
@ -97,15 +98,26 @@ pub fn cli(
} else {
load_global_cfg(&context);
}
// Store cmd duration in an env var
context.scope.add_env_var(
"CMD_DURATION_MS",
format!("{}", startup_commands_start_time.elapsed().as_millis()),
);
trace!(
"startup commands took {:?}",
startup_commands_start_time.elapsed()
);
if options.perf {
eprintln!(
"config loaded: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
//Configure rustyline
let mut rl = default_rustyline_editor_configuration();
@ -118,11 +130,39 @@ pub fn cli(
nu_data::config::path::default_history_path()
};
if options.perf {
eprintln!(
"rustyline configuration: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
// Don't load history if it's not necessary
if options.save_history {
let _ = rl.load_history(&history_path);
}
if options.perf {
eprintln!(
"history load: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
//set vars from cfg if present
let (skip_welcome_message, prompt) = if let Some(cfg) = &context.configs().lock().global_config
{
@ -136,6 +176,20 @@ pub fn cli(
(false, None)
};
if options.perf {
eprintln!(
"load custom prompt: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.start();
}
//Check whether dir we start in contains local cfg file and if so load it.
load_local_cfg_if_present(&context);
@ -159,6 +213,20 @@ pub fn cli(
let mut ctrlcbreak = false;
if options.perf {
eprintln!(
"timing stopped. starting run loop: {:?}",
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.elapsed_split()
);
STOPWATCH
.lock()
.expect("unable to lock the stopwatch")
.stop();
}
loop {
if context.ctrl_c().load(Ordering::SeqCst) {
context.ctrl_c().store(false, Ordering::SeqCst);

View File

@ -247,7 +247,7 @@ pub fn rustyline_hinter(
) -> Option<rustyline::hint::HistoryHinter> {
if let Some(line_editor_vars) = config.var("line_editor") {
for (idx, value) in line_editor_vars.row_entries() {
if idx == "show_hints" && value.expect_string() == "false" {
if idx == "show_hints" && value.as_bool() == Ok(false) {
return None;
}
}

View File

@ -90,7 +90,10 @@ impl rustyline::completion::Completer for Helper {
impl rustyline::hint::Hinter for Helper {
type Hint = String;
fn hint(&self, line: &str, pos: usize, ctx: &rustyline::Context<'_>) -> Option<String> {
self.hinter.as_ref().and_then(|h| h.hint(line, pos, ctx))
match &self.hinter {
Some(the_hinter) => the_hinter.hint(line, pos, ctx),
None => Some("".to_string()),
}
}
}

View File

@ -5,27 +5,27 @@ description = "CLI for nushell"
edition = "2018"
license = "MIT"
name = "nu-command"
version = "0.34.0"
version = "0.35.0"
[lib]
doctest = false
[dependencies]
nu-data = { version = "0.34.0", path="../nu-data" }
nu-engine = { version = "0.34.0", path="../nu-engine" }
nu-errors = { version = "0.34.0", path="../nu-errors" }
nu-json = { version = "0.34.0", path="../nu-json" }
nu-path = { version = "0.34.0", path="../nu-path" }
nu-parser = { version = "0.34.0", path="../nu-parser" }
nu-plugin = { version = "0.34.0", path="../nu-plugin" }
nu-protocol = { version = "0.34.0", path="../nu-protocol" }
nu-source = { version = "0.34.0", path="../nu-source" }
nu-stream = { version = "0.34.0", path="../nu-stream" }
nu-table = { version = "0.34.0", path="../nu-table" }
nu-test-support = { version = "0.34.0", path="../nu-test-support" }
nu-value-ext = { version = "0.34.0", path="../nu-value-ext" }
nu-ansi-term = { version = "0.34.0", path="../nu-ansi-term" }
nu-pretty-hex = { version = "0.34.0", path="../nu-pretty-hex" }
nu-data = { version = "0.35.0", path="../nu-data" }
nu-engine = { version = "0.35.0", path="../nu-engine" }
nu-errors = { version = "0.35.0", path="../nu-errors" }
nu-json = { version = "0.35.0", path="../nu-json" }
nu-path = { version = "0.35.0", path="../nu-path" }
nu-parser = { version = "0.35.0", path="../nu-parser" }
nu-plugin = { version = "0.35.0", path="../nu-plugin" }
nu-protocol = { version = "0.35.0", path="../nu-protocol" }
nu-source = { version = "0.35.0", path="../nu-source" }
nu-stream = { version = "0.35.0", path="../nu-stream" }
nu-table = { version = "0.35.0", path="../nu-table" }
nu-test-support = { version = "0.35.0", path="../nu-test-support" }
nu-value-ext = { version = "0.35.0", path="../nu-value-ext" }
nu-ansi-term = { version = "0.35.0", path="../nu-ansi-term" }
nu-pretty-hex = { version = "0.35.0", path="../nu-pretty-hex" }
Inflector = "0.11"
arboard = { version="1.1.0", optional=true }
@ -58,7 +58,7 @@ indexmap = { version="1.7", features=["serde-1"] }
itertools = "0.10.0"
lazy_static = "1.*"
log = "0.4.14"
md5 = "0.7.0"
md-5 = "0.9.1"
meval = "0.2.0"
minus = { version="3.4.0", optional=true, features=["async_std_lib", "search"] }
num-bigint = { version="0.3.1", features=["serde"] }
@ -96,9 +96,10 @@ url = "2.2.0"
uuid_crate = { package="uuid", version="0.8.2", features=["v4"], optional=true }
which = { version="4.1.0", optional=true }
zip = { version="0.5.9", optional=true }
digest = "0.9.0"
[dependencies.polars]
version = "0.14.5"
version = "0.14.8"
optional = true
features = ["parquet", "json", "random", "pivot", "strings", "is_in"]

View File

@ -0,0 +1,126 @@
use std::path::PathBuf;
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{ColumnPath, Primitive, Signature, SyntaxShape, UntaggedValue, Value};
pub struct SubCommand;
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"into path"
}
fn signature(&self) -> Signature {
Signature::build("into path").rest(
SyntaxShape::ColumnPath,
"column paths to convert to filepath (for table input)",
)
}
fn usage(&self) -> &str {
"Convert value to filepath"
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
into_filepath(args)
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Convert string to filepath in table",
example: "echo [[name]; ['/dev/null'] ['C:\\Program Files'] ['../../Cargo.toml']] | into path name",
result: Some(vec![
UntaggedValue::row(indexmap! {
"name".to_string() => UntaggedValue::filepath("/dev/null").into(),
})
.into(),
UntaggedValue::row(indexmap! {
"name".to_string() => UntaggedValue::filepath("C:\\Program Files").into(),
})
.into(),
UntaggedValue::row(indexmap! {
"name".to_string() => UntaggedValue::filepath("../../Cargo.toml").into(),
})
.into(),
]),
},
Example {
description: "Convert string to filepath",
example: "echo 'Cargo.toml' | into path",
result: Some(vec![UntaggedValue::filepath("Cargo.toml").into()]),
},
]
}
}
fn into_filepath(args: CommandArgs) -> Result<OutputStream, ShellError> {
let column_paths: Vec<ColumnPath> = args.rest(0)?;
Ok(args
.input
.map(move |v| {
if column_paths.is_empty() {
action(&v, v.tag())
} else {
let mut ret = v;
for path in &column_paths {
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
)?;
}
Ok(ret)
}
})
.into_input_stream())
}
pub fn action(input: &Value, tag: impl Into<Tag>) -> Result<Value, ShellError> {
let tag = tag.into();
match &input.value {
UntaggedValue::Primitive(prim) => Ok(UntaggedValue::filepath(match prim {
Primitive::String(a_string) => match filepath_from_string(a_string, &tag) {
Ok(n) => n,
Err(e) => {
return Err(e);
}
},
Primitive::FilePath(a_filepath) => a_filepath.clone(),
_ => {
return Err(ShellError::unimplemented(
"'into path' for non-string primitives",
))
}
})
.into_value(&tag)),
UntaggedValue::Row(_) => Err(ShellError::labeled_error(
"specify column name to use, with 'into path COLUMN'",
"found table",
tag,
)),
_ => Err(ShellError::unimplemented(
"'into path' for unsupported type",
)),
}
}
fn filepath_from_string(a_string: &str, _tag: &Tag) -> Result<PathBuf, ShellError> {
Ok(PathBuf::from(a_string))
}
#[cfg(test)]
mod tests {
use super::ShellError;
use super::SubCommand;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -1,9 +1,11 @@
mod binary;
mod command;
mod filepath;
mod int;
pub mod string;
pub use binary::SubCommand as IntoBinary;
pub use command::Command as Into;
pub use filepath::SubCommand as IntoFilepath;
pub use int::SubCommand as IntoInt;
pub use string::SubCommand as IntoString;

View File

@ -31,7 +31,7 @@ pub fn describe(args: CommandArgs) -> Result<ActionStream, ShellError> {
Ok(args
.input
.map(|row| {
let name = value::format_type(&row, 100);
let name = value::plain_type(&row, 100);
ReturnSuccess::value(
UntaggedValue::string(name).into_value(Tag::unknown_anchor(row.tag.span)),
)

View File

@ -63,7 +63,7 @@ pub fn letcmd(args: CommandArgs) -> Result<ActionStream, ShellError> {
.expect("Internal error: type checker should require args");
let var_name = positional[0].var_name()?;
let rhs_raw = evaluate_baseline_expr(&positional[2], &ctx)?;
let rhs_raw = evaluate_baseline_expr(&positional[2], ctx)?;
let tag: Tag = positional[2].span.into();
let rhs: CapturedBlock = FromValue::from_value(&rhs_raw)?;
@ -98,7 +98,7 @@ pub fn letcmd(args: CommandArgs) -> Result<ActionStream, ShellError> {
};
ctx.scope.enter_scope();
let value = evaluate_baseline_expr(expr, &ctx);
let value = evaluate_baseline_expr(expr, ctx);
ctx.scope.exit_scope();
let value = value?;

View File

@ -14,6 +14,7 @@ impl WholeStreamCommand for Command {
.switch("stdin", "redirect stdin", None)
.switch("skip-plugins", "do not load plugins", None)
.switch("no-history", "don't save history", None)
.switch("perf", "show startup performance metrics", None)
.named(
"commands",
SyntaxShape::String,

View File

@ -51,7 +51,7 @@ pub fn source(args: CommandArgs) -> Result<ActionStream, ShellError> {
let contents = std::fs::read_to_string(&expand_path(Cow::Borrowed(Path::new(&filename.item))));
match contents {
Ok(contents) => {
let result = script::run_script_standalone(contents, true, &ctx, false);
let result = script::run_script_standalone(contents, true, ctx, false);
if let Err(err) = result {
ctx.error(err);

View File

@ -2,14 +2,11 @@ use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, PolarsData},
Signature, SyntaxShape, TaggedDictBuilder, UntaggedValue, Value,
dataframe::{Column, FrameStruct, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
use polars::{
frame::groupby::GroupBy,
prelude::{DataType, PolarsError, Series},
};
use polars::{frame::groupby::GroupBy, prelude::PolarsError};
enum Operation {
Mean,
@ -111,17 +108,40 @@ impl WholeStreamCommand for DataFrame {
description: "Aggregate sum by grouping by column a and summing on col b",
example:
"[[a b]; [one 1] [one 2]] | dataframe to-df | dataframe group-by a | dataframe aggregate sum",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::string("one").into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(3).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Aggregate sum in dataframe columns",
example: "[[a b]; [4 1] [5 2]] | dataframe to-df | dataframe aggregate sum",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::int(9).into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(3).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Aggregate sum in series",
example: "[4 1 5 6] | dataframe to-series | dataframe aggregate sum",
result: None,
example: "[4 1 5 6] | dataframe to-df | dataframe aggregate sum",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("0".to_string(), vec![UntaggedValue::int(16).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -139,7 +159,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
})?;
match value.value {
UntaggedValue::DataFrame(PolarsData::GroupBy(nu_groupby)) => {
UntaggedValue::FrameStruct(FrameStruct::GroupBy(nu_groupby)) => {
let groupby = nu_groupby.to_groupby()?;
let res = perform_groupby_aggregation(
@ -152,18 +172,13 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
UntaggedValue::DataFrame(PolarsData::EagerDataFrame(df)) => {
UntaggedValue::DataFrame(df) => {
let df = df.as_ref();
let res = perform_dataframe_aggregation(&df, op, &operation.tag)?;
let res = perform_dataframe_aggregation(df, op, &operation.tag)?;
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
UntaggedValue::DataFrame(PolarsData::Series(series)) => {
let value = perform_series_aggregation(series.as_ref(), op, &operation.tag)?;
Ok(OutputStream::one(value))
}
_ => Err(ShellError::labeled_error(
"No groupby, dataframe or series in stream",
"no groupby, dataframe or series found in input stream",
@ -264,162 +279,15 @@ fn perform_dataframe_aggregation(
}
}
fn perform_series_aggregation(
series: &Series,
operation: Operation,
operation_tag: &Tag,
) -> Result<Value, ShellError> {
match operation {
Operation::Mean => {
let res = match series.mean() {
Some(val) => UntaggedValue::Primitive(val.into()),
None => UntaggedValue::Primitive(0.into()),
};
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
let value = Value {
value: res,
tag: operation_tag.clone(),
};
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
let mut data = TaggedDictBuilder::new(operation_tag.clone());
data.insert_value(series.name(), value);
Ok(data.into_value())
}
Operation::Median => {
let res = match series.median() {
Some(val) => UntaggedValue::Primitive(val.into()),
None => UntaggedValue::Primitive(0.into()),
};
let value = Value {
value: res,
tag: operation_tag.clone(),
};
let mut data = TaggedDictBuilder::new(operation_tag.clone());
data.insert_value(series.name(), value);
Ok(data.into_value())
}
Operation::Sum => {
let untagged = match series.dtype() {
DataType::Int8
| DataType::Int16
| DataType::Int32
| DataType::Int64
| DataType::UInt8
| DataType::UInt16
| DataType::UInt32
| DataType::UInt64 => {
let res: i64 = series.sum().unwrap_or(0);
Ok(UntaggedValue::Primitive(res.into()))
}
DataType::Float32 | DataType::Float64 => {
let res: f64 = series.sum().unwrap_or(0.0);
Ok(UntaggedValue::Primitive(res.into()))
}
_ => Err(ShellError::labeled_error(
"Not valid type",
format!(
"this operation can not be performed with series of type {}",
series.dtype()
),
&operation_tag.span,
)),
}?;
let value = Value {
value: untagged,
tag: operation_tag.clone(),
};
let mut data = TaggedDictBuilder::new(operation_tag.clone());
data.insert_value(series.name(), value);
Ok(data.into_value())
}
Operation::Max => {
let untagged = match series.dtype() {
DataType::Int8
| DataType::Int16
| DataType::Int32
| DataType::Int64
| DataType::UInt8
| DataType::UInt16
| DataType::UInt32
| DataType::UInt64 => {
let res: i64 = series.max().unwrap_or(0);
Ok(UntaggedValue::Primitive(res.into()))
}
DataType::Float32 | DataType::Float64 => {
let res: f64 = series.max().unwrap_or(0.0);
Ok(UntaggedValue::Primitive(res.into()))
}
_ => Err(ShellError::labeled_error(
"Not valid type",
format!(
"this operation can not be performed with series of type {}",
series.dtype()
),
&operation_tag.span,
)),
}?;
let value = Value {
value: untagged,
tag: operation_tag.clone(),
};
let mut data = TaggedDictBuilder::new(operation_tag.clone());
data.insert_value(series.name(), value);
Ok(data.into_value())
}
Operation::Min => {
let untagged = match series.dtype() {
DataType::Int8
| DataType::Int16
| DataType::Int32
| DataType::Int64
| DataType::UInt8
| DataType::UInt16
| DataType::UInt32
| DataType::UInt64 => {
let res: i64 = series.min().unwrap_or(0);
Ok(UntaggedValue::Primitive(res.into()))
}
DataType::Float32 | DataType::Float64 => {
let res: f64 = series.min().unwrap_or(0.0);
Ok(UntaggedValue::Primitive(res.into()))
}
_ => Err(ShellError::labeled_error(
"Not valid type",
format!(
"this operation can not be performed with series of type {}",
series.dtype()
),
&operation_tag.span,
)),
}?;
let value = Value {
value: untagged,
tag: operation_tag.clone(),
};
let mut data = TaggedDictBuilder::new(operation_tag.clone());
data.insert_value(series.name(), value);
Ok(data.into_value())
}
_ => Err(ShellError::labeled_error_with_secondary(
"Not valid operation",
"operation not valid for series",
&operation_tag.span,
"Perhaps you want: mean, median, sum, max, min",
&operation_tag.span,
)),
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,138 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Axis, Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use nu_source::Tagged;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe append"
}
fn usage(&self) -> &str {
"[DataFrame] Appends a new dataframe"
}
fn signature(&self) -> Signature {
Signature::build("dataframe append")
.required_named(
"other",
SyntaxShape::Any,
"dataframe to be appended",
Some('o'),
)
.required_named(
"axis",
SyntaxShape::String,
"row or col axis orientation",
Some('a'),
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Appends a dataframe as new columns",
example: r#"let a = ([[a b]; [1 2] [3 4]] | dataframe to-df);
$a | dataframe append -o $a -a row"#,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
),
Column::new(
"b".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(4).into()],
),
Column::new(
"a_x".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
),
Column::new(
"b_x".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(4).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Appends a dataframe merging at the end of columns",
example: r#"let a = ([[a b]; [1 2] [3 4]] | dataframe to-df);
$a | dataframe append -o $a -a col"#,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(3).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(3).into(),
],
),
Column::new(
"b".to_string(),
vec![
UntaggedValue::int(2).into(),
UntaggedValue::int(4).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(4).into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let other: Value = args.req_named("other")?;
let axis: Tagged<String> = args.req_named("axis")?;
let axis = Axis::try_from_str(axis.item.as_str(), &axis.tag.span)?;
let df_other = match other.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only append a dataframe to a dataframe",
other.tag.span,
)),
}?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let df_new = df.append_df(&df_other, axis, &tag.span)?;
Ok(OutputStream::one(df_new.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,8 +2,8 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, NuSeries},
Signature, SyntaxShape,
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
@ -32,7 +32,15 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "Returns the selected column as series",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe column a",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -41,15 +49,26 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let column: Tagged<String> = args.req(0)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df
.as_ref()
.column(column.item.as_ref())
.map_err(|e| parse_polars_error::<&str>(&e, &column.tag.span, None))?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.clone(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.clone()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use super::utils::{convert_columns, parse_polars_error};
@ -28,7 +31,15 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "drop column a",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe drop a",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"b".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(4).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -39,7 +50,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let columns: Vec<Value> = args.rest(0)?;
let (col_string, col_span) = convert_columns(&columns, &tag)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let new_df = match col_string.get(0) {
Some(col) => df
@ -63,3 +74,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use super::utils::{convert_columns, parse_polars_error};
@ -34,7 +37,21 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "drop duplicates",
example: "[[a b]; [1 2] [3 4] [1 2]] | dataframe to-df | dataframe drop-duplicates",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
),
Column::new(
"b".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(4).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -52,7 +69,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
None => (None, Span::unknown()),
};
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let subset_slice = subset.as_ref().map(|cols| &cols[..]);
@ -63,3 +80,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, NuSeries, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
@ -38,15 +38,45 @@ impl WholeStreamCommand for DataFrame {
example: r#"let df = ([[a b]; [1 2] [3 0] [1 2]] | dataframe to-df);
let res = ($df.b / $df.b);
let df = ($df | dataframe with-column $res --name res);
$df | dataframe drop-nulls
"#,
result: None,
$df | dataframe drop-nulls"#,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(1).into()],
),
Column::new(
"b".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(2).into()],
),
Column::new(
"res".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(1).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "drop null values in dataframe",
example: r#"let s = ([1 2 0 0 3 4] | dataframe to-series);
example: r#"let s = ([1 2 0 0 3 4] | dataframe to-df);
($s / $s) | dataframe drop-nulls"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"div_0_0".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(1).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -60,7 +90,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
})?;
match value.value {
UntaggedValue::DataFrame(PolarsData::EagerDataFrame(df)) => {
UntaggedValue::DataFrame(df) => {
// Extracting the selection columns of the columns to perform the aggregation
let columns: Option<Vec<Value>> = args.opt(0)?;
let (subset, col_span) = match columns {
@ -80,10 +110,6 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
UntaggedValue::DataFrame(PolarsData::Series(series)) => {
let res = series.as_ref().drop_nulls();
Ok(OutputStream::one(NuSeries::series_to_value(res, tag)))
}
_ => Err(ShellError::labeled_error(
"Incorrect type",
"drop nulls cannot be done with this value",
@ -91,3 +117,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, TaggedDictBuilder};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue, Value,
};
pub struct DataFrame;
@ -26,7 +29,27 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "drop column a",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe dtypes",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"column".to_string(),
vec![
UntaggedValue::string("a").into(),
UntaggedValue::string("b").into(),
],
),
Column::new(
"dtype".to_string(),
vec![
UntaggedValue::string("i64").into(),
UntaggedValue::string("i64").into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -35,26 +58,49 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let col_names = df
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let mut dtypes: Vec<Value> = Vec::new();
let names: Vec<Value> = df
.as_ref()
.get_column_names()
.iter()
.map(|v| v.to_string())
.collect::<Vec<String>>();
.map(|v| {
let dtype = df
.as_ref()
.column(v)
.expect("using name from list of names from dataframe")
.dtype();
let values = df
.as_ref()
.dtypes()
.into_iter()
.zip(col_names.into_iter())
.map(move |(dtype, name)| {
let mut data = TaggedDictBuilder::new(tag.clone());
data.insert_value("column", name.as_ref());
data.insert_value("dtype", format!("{}", dtype));
let dtype_str = format!("{}", dtype);
dtypes.push(Value {
value: dtype_str.into(),
tag: Tag::default(),
});
data.into_value()
});
Value {
value: v.to_string().into(),
tag: Tag::default(),
}
})
.collect();
Ok(OutputStream::from_stream(values))
let names_col = Column::new("column".to_string(), names);
let dtypes_col = Column::new("dtype".to_string(), dtypes);
let df = NuDataFrame::try_from_columns(vec![names_col, dtypes_col], &tag.span)?;
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
@ -32,12 +32,70 @@ impl WholeStreamCommand for DataFrame {
Example {
description: "Create new dataframe with dummy variables from a dataframe",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe to-dummies",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a_1".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(0).into()],
),
Column::new(
"a_3".to_string(),
vec![UntaggedValue::int(0).into(), UntaggedValue::int(1).into()],
),
Column::new(
"b_2".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(0).into()],
),
Column::new(
"b_4".to_string(),
vec![UntaggedValue::int(0).into(), UntaggedValue::int(1).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Create new dataframe with dummy variables from a series",
example: "[1 2 2 3 3] | dataframe to-series | dataframe to-dummies",
result: None,
example: "[1 2 2 3 3] | dataframe to-df | dataframe to-dummies",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"0_1".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
],
),
Column::new(
"0_2".to_string(),
vec![
UntaggedValue::int(0).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
],
),
Column::new(
"0_3".to_string(),
vec![
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(1).into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -51,7 +109,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
})?;
match value.value {
UntaggedValue::DataFrame(PolarsData::EagerDataFrame(df)) => {
UntaggedValue::DataFrame(df) => {
let res = df.as_ref().to_dummies().map_err(|e| {
parse_polars_error(
&e,
@ -62,17 +120,6 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
UntaggedValue::DataFrame(PolarsData::Series(series)) => {
let res = series.as_ref().to_dummies().map_err(|e| {
parse_polars_error(
&e,
&tag.span,
Some("The only allowed column types for dummies are String or Int"),
)
})?;
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
_ => Err(ShellError::labeled_error(
"Incorrect type",
"dummies cannot be done with this value",
@ -80,3 +127,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
@ -34,13 +34,21 @@ impl WholeStreamCommand for DataFrame {
vec![
Example {
description: "Filter dataframe using a bool mask",
example: r#"let mask = ([$true $false] | dataframe to-series);
example: r#"let mask = ([$true $false] | dataframe to-df);
[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe filter-with $mask"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::int(1).into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(2).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Filter dataframe by creating a mask from operation",
example: r#"let mask = (([5 6] | dataframe to-series) > 5);
example: r#"let mask = (([5 6] | dataframe to-df) > 5);
[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe filter-with $mask"#,
result: None,
},
@ -53,29 +61,42 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let value: Value = args.req(0)?;
let series_span = value.tag.span;
let series = match value.value {
UntaggedValue::DataFrame(PolarsData::Series(series)) => Ok(series),
let df = match value.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only add a series to a dataframe",
value.tag.span,
)),
}?;
let casted = series.as_ref().bool().map_err(|e| {
let series = df.as_series(&series_span)?;
let casted = series.bool().map_err(|e| {
parse_polars_error(
&e,
&&series_span,
&series_span,
Some("Perhaps you want to use a series with booleans as mask"),
)
})?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df
.as_ref()
.filter(&casted)
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
.filter(casted)
.map_err(|e| parse_polars_error::<&str>(&e, &df_tag.span, None))?;
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
@ -31,8 +34,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create new dataframe with head rows",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe first",
result: None,
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe first 1",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::int(1).into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(2).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -46,8 +57,21 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
None => 5,
};
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df.as_ref().head(Some(rows));
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use super::utils::{convert_columns, parse_polars_error};
pub struct DataFrame;
@ -27,7 +30,15 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "Creates dataframe with selected columns",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe get a",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -38,7 +49,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let (col_string, col_span) = convert_columns(&columns, &tag)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df
.as_ref()
@ -47,3 +58,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, NuGroupBy, PolarsData},
dataframe::{FrameStruct, NuDataFrame, NuGroupBy},
Signature, SyntaxShape, UntaggedValue, Value,
};
@ -43,7 +43,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let by_columns: Vec<Value> = args.rest(0)?;
let (columns_string, col_span) = convert_columns(&by_columns, &tag)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
// This is the expensive part of the groupby; to create the
// groups that will be used for grouping the data in the
@ -57,7 +57,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let groups = groupby.get_groups().to_vec();
let groupby = Value {
tag,
value: UntaggedValue::DataFrame(PolarsData::GroupBy(NuGroupBy::new(
value: UntaggedValue::FrameStruct(FrameStruct::GroupBy(NuGroupBy::new(
NuDataFrame::new(df.as_ref().clone()),
columns_string,
groups,

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
@ -51,20 +51,50 @@ impl WholeStreamCommand for DataFrame {
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "inner join dataframe",
example: r#"let right = ([[a b c]; [1 2 5] [3 4 5] [5 6 6]] | dataframe to-df);
vec![Example {
description: "inner join dataframe",
example: r#"let right = ([[a b c]; [1 2 5] [3 4 5] [5 6 6]] | dataframe to-df);
$right | dataframe join $right -l [a b] -r [a b]"#,
result: None,
},
Example {
description: "right join dataframe",
example: r#"let right = ([[a b c]; [1 2 3] [3 4 5] [5 6 7]] | dataframe to-df);
$right | dataframe join $right -l [a c] -r [a c] -t inner"#,
result: None,
},
]
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(3).into(),
UntaggedValue::int(5).into(),
],
),
Column::new(
"b".to_string(),
vec![
UntaggedValue::int(2).into(),
UntaggedValue::int(4).into(),
UntaggedValue::int(6).into(),
],
),
Column::new(
"c".to_string(),
vec![
UntaggedValue::int(5).into(),
UntaggedValue::int(5).into(),
UntaggedValue::int(6).into(),
],
),
Column::new(
"c_right".to_string(),
vec![
UntaggedValue::int(5).into(),
UntaggedValue::int(5).into(),
UntaggedValue::int(6).into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -97,13 +127,14 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let (l_col_string, l_col_span) = convert_columns(&l_col, &tag)?;
let (r_col_string, r_col_span) = convert_columns(&r_col, &tag)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = match r_df.value {
UntaggedValue::DataFrame(PolarsData::EagerDataFrame(r_df)) => {
UntaggedValue::DataFrame(r_df) => {
// Checking the column types before performing the join
check_column_datatypes(
df.as_ref(),
r_df.as_ref(),
&l_col_string,
&l_col_span,
&r_col_string,
@ -125,7 +156,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
}
fn check_column_datatypes<T: AsRef<str>>(
df: &polars::prelude::DataFrame,
df_l: &polars::prelude::DataFrame,
df_r: &polars::prelude::DataFrame,
l_cols: &[T],
l_col_span: &Span,
r_cols: &[T],
@ -146,13 +178,13 @@ fn check_column_datatypes<T: AsRef<str>>(
}
for (l, r) in l_cols.iter().zip(r_cols.iter()) {
let l_series = df
let l_series = df_l
.column(l.as_ref())
.map_err(|e| parse_polars_error::<&str>(&e, &l_col_span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, l_col_span, None))?;
let r_series = df
let r_series = df_r
.column(r.as_ref())
.map_err(|e| parse_polars_error::<&str>(&e, &r_col_span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, r_col_span, None))?;
if l_series.dtype() != r_series.dtype() {
return Err(ShellError::labeled_error_with_secondary(
@ -171,3 +203,16 @@ fn check_column_datatypes<T: AsRef<str>>(
Ok(())
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
pub struct DataFrame;
@ -30,8 +33,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create new dataframe with last rows",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe last",
result: None,
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe last 1",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::int(3).into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(4).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -45,9 +56,22 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
None => 5,
};
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df.as_ref().tail(Some(rows));
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::PolarsData, Signature, TaggedDictBuilder, UntaggedValue};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue, Value,
};
pub struct DataFrame;
@ -19,46 +22,94 @@ impl WholeStreamCommand for DataFrame {
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
let values = args
let data = args
.context
.scope
.get_vars()
.into_iter()
.filter_map(|(name, value)| {
if let UntaggedValue::DataFrame(PolarsData::EagerDataFrame(df)) = &value.value {
let mut data = TaggedDictBuilder::new(value.tag.clone());
if let UntaggedValue::DataFrame(df) = &value.value {
let rows = Value {
value: (df.as_ref().height() as i64).into(),
tag: Tag::default(),
};
let rows = df.as_ref().height();
let cols = df.as_ref().width();
let cols = Value {
value: (df.as_ref().width() as i64).into(),
tag: Tag::default(),
};
data.insert_value("name", name.as_ref());
data.insert_value("rows", format!("{}", rows));
data.insert_value("columns", format!("{}", cols));
let location = match value.tag.anchor {
Some(AnchorLocation::File(name)) => name,
Some(AnchorLocation::Url(name)) => name,
Some(AnchorLocation::Source(text)) => text.slice(0..text.end).text,
None => "stream".to_string(),
};
match value.tag.anchor {
Some(AnchorLocation::File(name)) => data.insert_value("location", name),
Some(AnchorLocation::Url(name)) => data.insert_value("location", name),
Some(AnchorLocation::Source(text)) => {
let loc_name = text.slice(0..text.end);
data.insert_value("location", loc_name.text)
}
None => data.insert_value("location", "stream"),
}
let location = Value {
value: location.into(),
tag: Tag::default(),
};
Some(data.into_value())
let name = Value {
value: name.into(),
tag: Tag::default(),
};
Some((name, rows, cols, location))
} else {
None
}
});
Ok(OutputStream::from_stream(values))
let mut name = Column::new_empty("name".to_string());
let mut rows = Column::new_empty("rows".to_string());
let mut cols = Column::new_empty("columns".to_string());
let mut location = Column::new_empty("location".to_string());
for tuple in data {
name.push(tuple.0);
rows.push(tuple.1);
cols.push(tuple.2);
location.push(tuple.3);
}
let tag = args.call_info.name_tag;
let df = NuDataFrame::try_from_columns(vec![name, rows, cols, location], &tag.span)?;
Ok(OutputStream::one(df.into_value(tag)))
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Lists loaded dataframes in current scope",
example: "dataframe list",
result: None,
example: "let a = ([[a b];[1 2] [3 4]] | dataframe to-df); dataframe list",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("name".to_string(), vec![UntaggedValue::string("$a").into()]),
Column::new("rows".to_string(), vec![UntaggedValue::int(2).into()]),
Column::new("columns".to_string(), vec![UntaggedValue::int(2).into()]),
Column::new(
"location".to_string(),
vec![UntaggedValue::string("stream").into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use super::utils::convert_columns;
@ -18,8 +21,18 @@ impl WholeStreamCommand for DataFrame {
fn signature(&self) -> Signature {
Signature::build("dataframe melt")
.required("id_columns", SyntaxShape::Table, "Id columns for melting")
.rest(SyntaxShape::Any, "columns used as value columns")
.required_named(
"columns",
SyntaxShape::Table,
"column names for melting",
Some('c'),
)
.required_named(
"values",
SyntaxShape::Table,
"column names used as value columns",
Some('v'),
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
@ -29,8 +42,59 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "melt dataframe",
example: "[[a b]; [a 2] [b 4] [a 6]] | dataframe to-df | dataframe melt a b",
result: None,
example:
"[[a b c d]; [x 1 4 a] [y 2 5 b] [z 3 6 c]] | dataframe to-df | dataframe melt -c [b c] -v [a d]",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"b".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(3).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(3).into(),
],
),
Column::new(
"c".to_string(),
vec![
UntaggedValue::int(4).into(),
UntaggedValue::int(5).into(),
UntaggedValue::int(6).into(),
UntaggedValue::int(4).into(),
UntaggedValue::int(5).into(),
UntaggedValue::int(6).into(),
],
),
Column::new(
"variable".to_string(),
vec![
UntaggedValue::string("a").into(),
UntaggedValue::string("a").into(),
UntaggedValue::string("a").into(),
UntaggedValue::string("d").into(),
UntaggedValue::string("d").into(),
UntaggedValue::string("d").into(),
],
),
Column::new(
"value".to_string(),
vec![
UntaggedValue::string("x").into(),
UntaggedValue::string("y").into(),
UntaggedValue::string("z").into(),
UntaggedValue::string("a").into(),
UntaggedValue::string("b").into(),
UntaggedValue::string("c").into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -38,13 +102,13 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let id_col: Vec<Value> = args.req(0)?;
let val_col: Vec<Value> = args.rest(1)?;
let id_col: Vec<Value> = args.req_named("columns")?;
let val_col: Vec<Value> = args.req_named("values")?;
let (id_col_string, id_col_span) = convert_columns(&id_col, &tag)?;
let (val_col_string, val_col_span) = convert_columns(&val_col, &tag)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
check_column_datatypes(df.as_ref(), &id_col_string, &id_col_span)?;
check_column_datatypes(df.as_ref(), &val_col_string, &val_col_span)?;
@ -75,11 +139,11 @@ fn check_column_datatypes<T: AsRef<str>>(
for w in cols.windows(2) {
let l_series = df
.column(w[0].as_ref())
.map_err(|e| parse_polars_error::<&str>(&e, &col_span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, col_span, None))?;
let r_series = df
.column(w[1].as_ref())
.map_err(|e| parse_polars_error::<&str>(&e, &col_span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, col_span, None))?;
if l_series.dtype() != r_series.dtype() {
return Err(ShellError::labeled_error_with_secondary(
@ -99,3 +163,16 @@ fn check_column_datatypes<T: AsRef<str>>(
Ok(())
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,4 +1,5 @@
pub mod aggregate;
pub mod append;
pub mod column;
pub mod command;
pub mod drop;
@ -18,6 +19,7 @@ pub mod open;
pub mod pivot;
pub mod sample;
pub mod select;
pub mod shape;
pub mod show;
pub mod slice;
pub mod sort;
@ -25,12 +27,12 @@ pub mod take;
pub mod to_csv;
pub mod to_df;
pub mod to_parquet;
pub mod to_series;
pub(crate) mod utils;
pub mod where_;
pub mod with_column;
pub use aggregate::DataFrame as DataFrameAggregate;
pub use append::DataFrame as DataFrameAppend;
pub use column::DataFrame as DataFrameColumn;
pub use command::Command as DataFrame;
pub use drop::DataFrame as DataFrameDrop;
@ -50,6 +52,7 @@ pub use open::DataFrame as DataFrameOpen;
pub use pivot::DataFrame as DataFramePivot;
pub use sample::DataFrame as DataFrameSample;
pub use select::DataFrame as DataFrameSelect;
pub use shape::DataFrame as DataFrameShape;
pub use show::DataFrame as DataFrameShow;
pub use slice::DataFrame as DataFrameSlice;
pub use sort::DataFrame as DataFrameSort;
@ -57,7 +60,6 @@ pub use take::DataFrame as DataFrameTake;
pub use to_csv::DataFrame as DataFrameToCsv;
pub use to_df::DataFrame as DataFrameToDF;
pub use to_parquet::DataFrame as DataFrameToParquet;
pub use to_series::DataFrame as DataFrameToSeries;
pub use where_::DataFrame as DataFrameWhere;
pub use with_column::DataFrame as DataFrameWithColumn;
@ -69,6 +71,8 @@ pub use series::DataFrameArgMin;
pub use series::DataFrameArgSort;
pub use series::DataFrameArgTrue;
pub use series::DataFrameArgUnique;
pub use series::DataFrameConcatenate;
pub use series::DataFrameContains;
pub use series::DataFrameIsDuplicated;
pub use series::DataFrameIsIn;
pub use series::DataFrameIsNotNull;
@ -77,9 +81,15 @@ pub use series::DataFrameIsUnique;
pub use series::DataFrameNNull;
pub use series::DataFrameNUnique;
pub use series::DataFrameNot;
pub use series::DataFrameReplace;
pub use series::DataFrameReplaceAll;
pub use series::DataFrameSeriesRename;
pub use series::DataFrameSet;
pub use series::DataFrameSetWithIdx;
pub use series::DataFrameShift;
pub use series::DataFrameStringLengths;
pub use series::DataFrameStringSlice;
pub use series::DataFrameToLowercase;
pub use series::DataFrameToUppercase;
pub use series::DataFrameUnique;
pub use series::DataFrameValueCounts;

View File

@ -8,7 +8,7 @@ use nu_protocol::{
};
use nu_source::Tagged;
use polars::prelude::{CsvReader, JsonReader, ParquetReader, SerReader};
use polars::prelude::{CsvEncoding, CsvReader, JsonReader, ParquetReader, PolarsError, SerReader};
use std::fs::File;
pub struct DataFrame;
@ -151,7 +151,8 @@ fn from_csv(args: CommandArgs) -> Result<polars::prelude::DataFrame, ShellError>
let columns: Option<Vec<Value>> = args.get_flag("columns")?;
let csv_reader = CsvReader::from_path(&file.item)
.map_err(|e| parse_polars_error::<&str>(&e, &file.tag.span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, &file.tag.span, None))?
.with_encoding(CsvEncoding::LossyUtf8);
let csv_reader = match delimiter {
None => csv_reader,
@ -205,6 +206,15 @@ fn from_csv(args: CommandArgs) -> Result<polars::prelude::DataFrame, ShellError>
match csv_reader.finish() {
Ok(df) => Ok(df),
Err(e) => Err(parse_polars_error::<&str>(&e, &file.tag.span, None)),
Err(e) => match e {
PolarsError::Other(_) => Err(ShellError::labeled_error_with_secondary(
"Schema error",
"Error with the inferred schema",
&file.tag.span,
"You can use the argument 'infer_schema' with a number of rows large enough to better infer the schema",
&file.tag.span,
)),
_ => Err(parse_polars_error::<&str>(&e, &file.tag.span, None)),
},
}
}

View File

@ -73,7 +73,7 @@ impl WholeStreamCommand for DataFrame {
description: "Pivot a dataframe on b and aggregation on col c",
example:
"[[a b c]; [one x 1] [two y 2]] | dataframe to-df | dataframe group-by a | dataframe pivot b c sum",
result: None,
result: None, // No sample because there are nulls in the result dataframe
}]
}
}

View File

@ -42,13 +42,13 @@ impl WholeStreamCommand for DataFrame {
Example {
description: "Sample rows from dataframe",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe sample -r 1",
result: None,
result: None, // No expected value because sampling is random
},
Example {
description: "Shows sample row using fraction and replace",
example:
"[[a b]; [1 2] [3 4] [5 6]] | dataframe to-df | dataframe sample -f 0.5 -e",
result: None,
result: None, // No expected value because sampling is random
},
]
}
@ -61,7 +61,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let fraction: Option<Tagged<f64>> = args.get_flag("fraction")?;
let replace: bool = args.has_flag("replace");
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = match (rows, fraction) {
(Some(rows), None) => df

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use super::utils::{convert_columns, parse_polars_error};
@ -28,7 +31,15 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "Create new dataframe with column a",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe select a",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -40,7 +51,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let (col_string, col_span) = convert_columns(&columns, &tag)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df
.as_ref()
@ -49,3 +60,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature, TaggedDictBuilder, UntaggedValue, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue, Value,
};
pub struct DataFrame;
@ -26,15 +29,31 @@ impl WholeStreamCommand for DataFrame {
vec![
Example {
description: "Returns true if all values are false",
example: "[$false $false $false] | dataframe to-series | dataframe all-false",
result: None,
example: "[$false $false $false] | dataframe to-df | dataframe all-false",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"all_false".to_string(),
vec![UntaggedValue::boolean(true).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Checks the result from a comparison",
example: r#"let s = ([5 6 2 8] | dataframe to-series);
example: r#"let s = ([5 6 2 10] | dataframe to-df);
let res = ($s > 9);
$res | dataframe all-false"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"all_false".to_string(),
vec![UntaggedValue::boolean(false).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -43,9 +62,10 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let bool = series.as_ref().bool().map_err(|e| {
let series = df.as_series(&df_tag.span)?;
let bool = series.bool().map_err(|e| {
parse_polars_error::<&str>(
&e,
&tag.span,
@ -60,8 +80,23 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
tag: tag.clone(),
};
let mut data = TaggedDictBuilder::new(tag);
data.insert_value("all_false", value);
let df = NuDataFrame::try_from_columns(
vec![Column::new("all_false".to_string(), vec![value])],
&tag.span,
)?;
Ok(OutputStream::one(data.into_value()))
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature, TaggedDictBuilder, UntaggedValue, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue, Value,
};
pub struct DataFrame;
@ -26,15 +29,31 @@ impl WholeStreamCommand for DataFrame {
vec![
Example {
description: "Returns true if all values are true",
example: "[$true $true $true] | dataframe to-series | dataframe all-true",
result: None,
example: "[$true $true $true] | dataframe to-df | dataframe all-true",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"all_true".to_string(),
vec![UntaggedValue::boolean(true).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Checks the result from a comparison",
example: r#"let s = ([5 6 2 8] | dataframe to-series);
example: r#"let s = ([5 6 2 8] | dataframe to-df);
let res = ($s > 9);
$res | dataframe all-true"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"all_true".to_string(),
vec![UntaggedValue::boolean(false).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -43,9 +62,10 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let bool = series.as_ref().bool().map_err(|e| {
let series = df.as_series(&df_tag.span)?;
let bool = series.bool().map_err(|e| {
parse_polars_error::<&str>(
&e,
&tag.span,
@ -60,8 +80,23 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
tag: tag.clone(),
};
let mut data = TaggedDictBuilder::new(tag);
data.insert_value("all_true", value);
let df = NuDataFrame::try_from_columns(
vec![Column::new("all_true".to_string(), vec![value])],
&tag.span,
)?;
Ok(OutputStream::one(data.into_value()))
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::{IntoSeries, NewChunkedArray, UInt32Chunked};
@ -27,8 +30,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns index for max value",
example: "[1 3 2] | dataframe to-series | dataframe arg-max",
result: None,
example: "[1 3 2] | dataframe to-df | dataframe arg-max",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"arg_max".to_string(),
vec![UntaggedValue::int(1).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -36,9 +47,10 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let res = series.as_ref().arg_max();
let res = series.arg_max();
let chunked = match res {
Some(index) => UInt32Chunked::new_from_slice("arg_max", &[index as u32]),
@ -46,6 +58,20 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
};
let res = chunked.into_series();
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(NuSeries::series_to_value(res, tag)))
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::{IntoSeries, NewChunkedArray, UInt32Chunked};
@ -27,8 +30,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns index for min value",
example: "[1 3 2] | dataframe to-series | dataframe arg-min",
result: None,
example: "[1 3 2] | dataframe to-df | dataframe arg-min",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"arg_min".to_string(),
vec![UntaggedValue::int(0).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -36,9 +47,9 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series.as_ref().arg_min();
let res = df.as_series(&df_tag.span)?.arg_min();
let chunked = match res {
Some(index) => UInt32Chunked::new_from_slice("arg_min", &[index as u32]),
@ -46,6 +57,20 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
};
let res = chunked.into_series();
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(NuSeries::series_to_value(res, tag)))
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,8 +29,22 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns indexes for a sorted series",
example: "[1 2 2 3 3] | dataframe to-series | dataframe arg-sort",
result: None,
example: "[1 2 2 3 3] | dataframe to-df | dataframe arg-sort",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"arg_sort".to_string(),
vec![
UntaggedValue::int(0).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(3).into(),
UntaggedValue::int(4).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -36,12 +53,24 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let reverse = args.has_flag("reverse");
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series.as_ref().argsort(reverse);
let mut res = df.as_series(&df_tag.span)?.argsort(reverse).into_series();
res.rename("arg_sort");
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,8 +29,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns indexes where values are true",
example: "[$false $true $false] | dataframe to-series | dataframe arg-true",
result: None,
example: "[$false $true $false] | dataframe to-df | dataframe arg-true",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"arg_true".to_string(),
vec![UntaggedValue::int(1).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -35,9 +46,10 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let bool = series.as_ref().bool().map_err(|e| {
let series = df.as_series(&df_tag.span)?;
let bool = series.bool().map_err(|e| {
parse_polars_error::<&str>(
&e,
&tag.span,
@ -46,7 +58,21 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
})?;
let mut res = bool.arg_true().into_series();
res.rename("int");
res.rename("arg_true");
Ok(OutputStream::one(NuSeries::series_to_value(res, tag)))
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,8 +29,20 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns indexes for unique values",
example: "[1 2 2 3 3] | dataframe to-series | dataframe arg-unique",
result: None,
example: "[1 2 2 3 3] | dataframe to-df | dataframe arg-unique",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"arg_unique".to_string(),
vec![
UntaggedValue::int(0).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(3).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -35,15 +50,29 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series
.as_ref()
let mut res = df
.as_series(&df_tag.span)?
.arg_unique()
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?
.into_series();
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
res.rename("arg_unique");
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,107 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe concatenate"
}
fn usage(&self) -> &str {
"[Series] Concatenates strings with other array"
}
fn signature(&self) -> Signature {
Signature::build("dataframe concatenate").required(
"other",
SyntaxShape::Any,
"Other array with string to be concatenated",
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Concatenate string",
example: r#"let other = ([za xs cd] | dataframe to-df);
[abc abc abc] | dataframe to-df | dataframe concatenate $other"#,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("abcza").into(),
UntaggedValue::string("abcxs").into(),
UntaggedValue::string("abccd").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let other: Value = args.req(0)?;
let other_df = match &other.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only concatenate another series",
other.tag.span,
)),
}?;
let other_series = other_df.as_series(&other.tag.span)?;
let other_chunked = other_series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&other.tag.span,
Some("The concatenate command can only be used with string columns"),
)
})?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The concatenate command can only be used with string columns"),
)
})?;
let mut res = chunked.concat(other_chunked);
res.rename(series.name());
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,89 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe contains"
}
fn usage(&self) -> &str {
"[Series] Checks if a patter is contained in a string"
}
fn signature(&self) -> Signature {
Signature::build("dataframe contains").required(
"pattern",
SyntaxShape::String,
"Regex pattern to be searched",
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns boolean indicating if patter was found",
example: "[abc acb acb] | dataframe to-df | dataframe contains ab",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let pattern: Tagged<String> = args.req(0)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The contains command can only be used with string columns"),
)
})?;
let res = chunked
.contains(pattern.as_str())
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,8 +29,24 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create mask indicating duplicated values",
example: "[5 6 6 6 8 8 8] | dataframe to-series | dataframe is-duplicated",
result: None,
example: "[5 6 6 6 8 8 8] | dataframe to-df | dataframe is-duplicated",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"is_duplicated".to_string(),
vec![
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -35,15 +54,29 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series
.as_ref()
let mut res = df
.as_series(&df_tag.span)?
.is_duplicated()
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?
.into_series();
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
res.rename("is_duplicated");
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuSeries, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use polars::prelude::IntoSeries;
@ -29,9 +29,25 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Checks if elements from a series are contained in right series",
example: r#"let other = ([1 3 6] | dataframe to-series);
[5 6 6 6 8 8 8] | dataframe to-series | dataframe is-in $other"#,
result: None,
example: r#"let other = ([1 3 6] | dataframe to-df);
[5 6 6 6 8 8 8] | dataframe to-df | dataframe is-in $other"#,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"is_in".to_string(),
vec![
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -40,8 +56,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let value: Value = args.req(0)?;
let other = match value.value {
UntaggedValue::DataFrame(PolarsData::Series(series)) => Ok(series),
let other_df = match value.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only search in a series",
@ -49,15 +65,31 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}?;
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let other = other_df.as_series(&value.tag.span)?;
let res = series
.as_ref()
.is_in(other.as_ref())
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let mut res = df
.as_series(&df_tag.span)?
.is_in(&other)
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?
.into_series();
res.rename("is_in");
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,10 +29,23 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create mask where values are not null",
example: r#"let s = ([5 6 0 8] | dataframe to-series);
example: r#"let s = ([5 6 0 8] | dataframe to-df);
let res = ($s / $s);
$res | dataframe is-not-null"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"is_not_null".to_string(),
vec![
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(true).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -37,12 +53,23 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series.as_ref().is_not_null();
let res = df.as_series(&df_tag.span)?.is_not_null();
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,10 +29,23 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create mask where values are null",
example: r#"let s = ([5 6 0 8] | dataframe to-series);
example: r#"let s = ([5 6 0 8] | dataframe to-df);
let res = ($s / $s);
$res | dataframe is-null"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"is_null".to_string(),
vec![
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -37,12 +53,23 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series.as_ref().is_null();
let res = df.as_series(&df_tag.span)?.is_null();
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -26,8 +29,24 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Create mask indicating unique values",
example: "[5 6 6 6 8 8 8] | dataframe to-series | dataframe is-unique",
result: None,
example: "[5 6 6 6 8 8 8] | dataframe to-df | dataframe is-unique",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"is_unique".to_string(),
vec![
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(false).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -35,15 +54,29 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series
.as_ref()
let mut res = df
.as_series(&df_tag.span)?
.is_unique()
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?
.into_series();
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
res.rename("is_unique");
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -5,6 +5,8 @@ pub mod arg_min;
pub mod arg_sort;
pub mod arg_true;
pub mod arg_unique;
pub mod concatenate;
pub mod contains;
pub mod is_duplicated;
pub mod is_in;
pub mod is_not_null;
@ -14,9 +16,15 @@ pub mod n_null;
pub mod n_unique;
pub mod not;
pub mod rename;
pub mod replace;
pub mod replace_all;
pub mod set;
pub mod set_with_idx;
pub mod shift;
pub mod str_lengths;
pub mod str_slice;
pub mod to_lowercase;
pub mod to_uppercase;
pub mod unique;
pub mod value_counts;
@ -27,6 +35,8 @@ pub use arg_min::DataFrame as DataFrameArgMin;
pub use arg_sort::DataFrame as DataFrameArgSort;
pub use arg_true::DataFrame as DataFrameArgTrue;
pub use arg_unique::DataFrame as DataFrameArgUnique;
pub use concatenate::DataFrame as DataFrameConcatenate;
pub use contains::DataFrame as DataFrameContains;
pub use is_duplicated::DataFrame as DataFrameIsDuplicated;
pub use is_in::DataFrame as DataFrameIsIn;
pub use is_not_null::DataFrame as DataFrameIsNotNull;
@ -36,8 +46,14 @@ pub use n_null::DataFrame as DataFrameNNull;
pub use n_unique::DataFrame as DataFrameNUnique;
pub use not::DataFrame as DataFrameNot;
pub use rename::DataFrame as DataFrameSeriesRename;
pub use replace::DataFrame as DataFrameReplace;
pub use replace_all::DataFrame as DataFrameReplaceAll;
pub use set::DataFrame as DataFrameSet;
pub use set_with_idx::DataFrame as DataFrameSetWithIdx;
pub use shift::DataFrame as DataFrameShift;
pub use str_lengths::DataFrame as DataFrameStringLengths;
pub use str_slice::DataFrame as DataFrameStringSlice;
pub use to_lowercase::DataFrame as DataFrameToLowercase;
pub use to_uppercase::DataFrame as DataFrameToUppercase;
pub use unique::DataFrame as DataFrameUnique;
pub use value_counts::DataFrame as DataFrameValueCounts;

View File

@ -2,7 +2,8 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::NuSeries, Primitive, Signature, TaggedDictBuilder, UntaggedValue, Value,
dataframe::{Column, NuDataFrame},
Primitive, Signature, UntaggedValue, Value,
};
pub struct DataFrame;
@ -27,9 +28,17 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Counts null values",
example: r#"let s = ([1 1 0 0 3 3 4] | dataframe to-series);
($s / ss) | dataframe count-null"#,
result: None,
example: r#"let s = ([1 1 0 0 3 3 4] | dataframe to-df);
($s / $s) | dataframe count-null"#,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"count_null".to_string(),
vec![UntaggedValue::int(2).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -37,17 +46,32 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series.as_ref().null_count();
let res = df.as_series(&df_tag.span)?.null_count();
let value = Value {
value: UntaggedValue::Primitive(Primitive::Int(res as i64)),
tag: tag.clone(),
};
let mut data = TaggedDictBuilder::new(tag);
data.insert_value("count-null", value);
let df = NuDataFrame::try_from_columns(
vec![Column::new("count_null".to_string(), vec![value])],
&tag.span,
)?;
Ok(OutputStream::one(data.into_value()))
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,8 @@ use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::NuSeries, Primitive, Signature, TaggedDictBuilder, UntaggedValue, Value,
dataframe::{Column, NuDataFrame},
Primitive, Signature, UntaggedValue, Value,
};
pub struct DataFrame;
@ -27,8 +28,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Counts unique values",
example: "[1 1 2 2 3 3 4] | dataframe to-series | dataframe count-unique",
result: None,
example: "[1 1 2 2 3 3 4] | dataframe to-df | dataframe count-unique",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"count_unique".to_string(),
vec![UntaggedValue::int(4).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -36,10 +45,10 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series
.as_ref()
let res = df
.as_series(&df_tag.span)?
.n_unique()
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
@ -48,8 +57,23 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
tag: tag.clone(),
};
let mut data = TaggedDictBuilder::new(tag);
data.insert_value("count-unique", value);
let df = NuDataFrame::try_from_columns(
vec![Column::new("count_unique".to_string(), vec![value])],
&tag.span,
)?;
Ok(OutputStream::one(data.into_value()))
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
use std::ops::Not;
@ -27,8 +30,20 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Inverts boolean mask",
example: "[$true $false $true] | dataframe to-series | dataframe not",
result: None,
example: "[$true $false $true] | dataframe to-df | dataframe not",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::boolean(false).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -36,9 +51,10 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let bool = series.as_ref().bool().map_err(|e| {
let bool = series.bool().map_err(|e| {
parse_polars_error::<&str>(
&e,
&tag.span,
@ -48,8 +64,19 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let res = bool.not();
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature, SyntaxShape};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
pub struct DataFrame;
@ -30,8 +33,21 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Renames a series",
example: "[5 6 7 8] | dataframe to-series | dataframe rename-series new_name",
result: None,
example: "[5 6 7 8] | dataframe to-df | dataframe rename new_name",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"new_name".to_string(),
vec![
UntaggedValue::int(5).into(),
UntaggedValue::int(6).into(),
UntaggedValue::int(7).into(),
UntaggedValue::int(8).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -40,9 +56,25 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let name: Tagged<String> = args.req(0)?;
let mut series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
series.as_mut().rename(name.item.as_ref());
let mut series = df.as_series(&df_tag.span)?;
Ok(OutputStream::one(series.into_value(tag)))
series.rename(name.item.as_ref());
let df = NuDataFrame::try_from_series(vec![series], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,100 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::{Span, Tagged};
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe replace"
}
fn usage(&self) -> &str {
"[Series] Replace the leftmost (sub)string by a regex pattern"
}
fn signature(&self) -> Signature {
Signature::build("dataframe replace")
.required_named(
"pattern",
SyntaxShape::String,
"Regex pattern to be matched",
Some('p'),
)
.required_named(
"replace",
SyntaxShape::String,
"replacing string",
Some('r'),
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Replaces string",
example: "[abc abc abc] | dataframe to-df | dataframe replace -p ab -r AB",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("ABc").into(),
UntaggedValue::string("ABc").into(),
UntaggedValue::string("ABc").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let pattern: Tagged<String> = args.req_named("pattern")?;
let replace: Tagged<String> = args.req_named("replace")?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The replace-all command can only be used with string columns"),
)
})?;
let mut res = chunked
.replace(pattern.as_str(), replace.as_str())
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
res.rename(series.name());
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,100 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe replace-all"
}
fn usage(&self) -> &str {
"[Series] Replace all (sub)strings by a regex pattern"
}
fn signature(&self) -> Signature {
Signature::build("dataframe replace")
.required_named(
"pattern",
SyntaxShape::String,
"Regex pattern to be matched",
Some('p'),
)
.required_named(
"replace",
SyntaxShape::String,
"replacing string",
Some('r'),
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Replaces string",
example: "[abac abac abac] | dataframe to-df | dataframe replace-all -p a -r A",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("AbAc").into(),
UntaggedValue::string("AbAc").into(),
UntaggedValue::string("AbAc").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let pattern: Tagged<String> = args.req_named("pattern")?;
let replace: Tagged<String> = args.req_named("replace")?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The replace command can only be used with string columns"),
)
})?;
let mut res = chunked
.replace_all(pattern.as_str(), replace.as_str())
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
res.rename(series.name());
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Primitive, Signature, SyntaxShape, UntaggedValue, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Primitive, Signature, SyntaxShape, UntaggedValue, Value,
};
use polars::prelude::{ChunkSet, DataType, IntoSeries};
pub struct DataFrame;
@ -33,10 +36,24 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Shifts the values by a given period",
example: r#"let s = ([1 2 2 3 3] | dataframe to-series | dataframe shift 2);
example: r#"let s = ([1 2 2 3 3] | dataframe to-df | dataframe shift 2);
let mask = ($s | dataframe is-null);
$s | dataframe set 0 --mask $mask"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::int(0).into(),
UntaggedValue::int(0).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(2).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -46,20 +63,21 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let value: Value = args.req(0)?;
let mask: Value = args.req_named("mask")?;
let bool_mask = match &mask.value {
UntaggedValue::DataFrame(nu_protocol::dataframe::PolarsData::Series(series)) => {
match series.as_ref().dtype() {
DataType::Boolean => series
.as_ref()
.bool()
.map_err(|e| parse_polars_error::<&str>(&e, &mask.tag.span, None)),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only use bool series as mask",
value.tag.span,
)),
}
}
let mask_df = match &mask.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only use a series as mask",
value.tag.span,
)),
}?;
let mask_series = mask_df.as_series(&mask.tag.span)?;
let bool_mask = match mask_series.dtype() {
DataType::Boolean => mask_series
.bool()
.map_err(|e| parse_polars_error::<&str>(&e, &mask.tag.span, None)),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only use bool series as mask",
@ -67,11 +85,12 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}?;
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
match &value.value {
UntaggedValue::Primitive(Primitive::Int(val)) => {
let chunked = series.as_ref().i64().map_err(|e| {
let chunked = series.i64().map_err(|e| {
parse_polars_error::<&str>(
&e,
&value.tag.span,
@ -83,10 +102,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
.set(bool_mask, Some(*val))
.map_err(|e| parse_polars_error::<&str>(&e, &value.tag.span, None))?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
UntaggedValue::Primitive(Primitive::Decimal(val)) => {
let chunked = series.as_ref().f64().map_err(|e| {
@ -107,10 +124,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)
.map_err(|e| parse_polars_error::<&str>(&e, &value.tag.span, None))?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
UntaggedValue::Primitive(Primitive::String(val)) => {
let chunked = series.as_ref().utf8().map_err(|e| {
@ -128,18 +143,29 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let mut res = res.into_series();
res.rename("string");
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
_ => Err(ShellError::labeled_error(
"Incorrect type",
format!(
"this value cannot be set in a series of type '{}'",
series.as_ref().dtype()
series.dtype()
),
value.tag.span,
)),
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,7 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Primitive, Signature, SyntaxShape, UntaggedValue, Value};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Primitive, Signature, SyntaxShape, UntaggedValue, Value,
};
use polars::prelude::{ChunkSet, DataType, IntoSeries};
pub struct DataFrame;
@ -33,10 +36,25 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Set value in selected rows from series",
example: r#"let series = ([4 1 5 2 4 3] | dataframe to-series);
let indices = ([0 2] | dataframe to-series);
example: r#"let series = ([4 1 5 2 4 3] | dataframe to-df);
let indices = ([0 2] | dataframe to-df);
$series | dataframe set-with-idx 6 -i $indices"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::int(6).into(),
UntaggedValue::int(1).into(),
UntaggedValue::int(6).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(4).into(),
UntaggedValue::int(3).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -47,7 +65,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let indices: Value = args.req_named("indices")?;
let indices = match &indices.value {
UntaggedValue::DataFrame(nu_protocol::dataframe::PolarsData::Series(series)) => Ok(series),
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only use a series for set command",
@ -55,7 +73,9 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}?;
let casted = match indices.as_ref().dtype() {
let indices = indices.as_series(&value.tag.span)?;
let casted = match indices.dtype() {
DataType::UInt32 | DataType::UInt64 | DataType::Int32 | DataType::Int64 => indices
.as_ref()
.cast_with_dtype(&DataType::UInt32)
@ -75,11 +95,12 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
.into_iter()
.filter_map(|val| val.map(|v| v as usize));
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
match &value.value {
UntaggedValue::Primitive(Primitive::Int(val)) => {
let chunked = series.as_ref().i64().map_err(|e| {
let chunked = series.i64().map_err(|e| {
parse_polars_error::<&str>(
&e,
&value.tag.span,
@ -91,10 +112,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
.set_at_idx(indices, Some(*val))
.map_err(|e| parse_polars_error::<&str>(&e, &value.tag.span, None))?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
UntaggedValue::Primitive(Primitive::Decimal(val)) => {
let chunked = series.as_ref().f64().map_err(|e| {
@ -115,10 +134,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)
.map_err(|e| parse_polars_error::<&str>(&e, &value.tag.span, None))?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
UntaggedValue::Primitive(Primitive::String(val)) => {
let chunked = series.as_ref().utf8().map_err(|e| {
@ -136,10 +153,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let mut res = res.into_series();
res.rename("string");
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
_ => Err(ShellError::labeled_error(
"Incorrect type",
@ -151,3 +166,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,9 +1,8 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature, SyntaxShape};
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape};
use nu_source::Tagged;
use polars::prelude::IntoSeries;
pub struct DataFrame;
@ -27,7 +26,7 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Shifts the values by a given period",
example: "[1 2 2 3 3] | dataframe to-series | dataframe shift 2",
example: "[1 2 2 3 3] | dataframe to-df | dataframe shift 2",
result: None,
}]
}
@ -37,12 +36,10 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let period: Tagged<i64> = args.req(0)?;
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series.as_ref().shift(period.item);
let res = df.as_series(&df_tag.span)?.shift(period.item);
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}

View File

@ -0,0 +1,81 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe str-lengths"
}
fn usage(&self) -> &str {
"[Series] Get lengths of all strings"
}
fn signature(&self) -> Signature {
Signature::build("dataframe str-lengths")
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns string lengths",
example: "[a ab abc] | dataframe to-df | dataframe str-lengths",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(3).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The str-lengths command can only be used with string columns"),
)
})?;
let res = chunked.as_ref().str_lengths();
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,92 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe str-slice"
}
fn usage(&self) -> &str {
"[Series] Slices the string from the start position until the selected length"
}
fn signature(&self) -> Signature {
Signature::build("dataframe replace")
.required_named("start", SyntaxShape::Int, "start of slice", Some('s'))
.named("length", SyntaxShape::Int, "optional length", Some('l'))
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Creates slices from the strings",
example: "[abcded abc321 abc123] | dataframe to-df | dataframe str-slice -s 1 -l 2",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("bc").into(),
UntaggedValue::string("bc").into(),
UntaggedValue::string("bc").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let start: Tagged<i64> = args.req_named("start")?;
let length: Option<Tagged<i64>> = args.get_flag("length")?;
let length = length.map(|v| v.item as u64);
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The str-slice command can only be used with string columns"),
)
})?;
let mut res = chunked
.str_slice(start.item, length)
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
res.rename(series.name());
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,82 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe to-lowercase"
}
fn usage(&self) -> &str {
"[Series] Lowercase the strings in the column"
}
fn signature(&self) -> Signature {
Signature::build("dataframe to-lowercase")
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Modifies strings to lowercase",
example: "[Abc aBc abC] | dataframe to-df | dataframe to-lowercase",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("abc").into(),
UntaggedValue::string("abc").into(),
UntaggedValue::string("abc").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The to-lowercase command can only be used with string columns"),
)
})?;
let mut res = chunked.to_lowercase();
res.rename(series.name());
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,82 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use polars::prelude::IntoSeries;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe to-uppercase"
}
fn usage(&self) -> &str {
"[Series] Uppercase the strings in the column"
}
fn signature(&self) -> Signature {
Signature::build("dataframe to-uppercase")
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Modifies strings to uppercase",
example: "[Abc aBc abC] | dataframe to-df | dataframe to-uppercase",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("ABC").into(),
UntaggedValue::string("ABC").into(),
UntaggedValue::string("ABC").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = df.as_series(&df_tag.span)?;
let chunked = series.utf8().map_err(|e| {
parse_polars_error::<&str>(
&e,
&df_tag.span,
Some("The to-uppercase command can only be used with string columns"),
)
})?;
let mut res = chunked.to_uppercase();
res.rename(series.name());
let df = NuDataFrame::try_from_series(vec![res.into_series()], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -1,8 +1,10 @@
use crate::{commands::dataframe::utils::parse_polars_error, prelude::*};
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature};
use polars::prelude::IntoSeries;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
pub struct DataFrame;
@ -26,8 +28,16 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Returns unique values from a series",
example: "[1 2 2 3 3] | dataframe to-series | dataframe unique",
result: None,
example: "[2 2 2 2 2] | dataframe to-df | dataframe unique",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![UntaggedValue::int(2).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -35,15 +45,26 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = series
.as_ref()
let res = df
.as_series(&df_tag.span)?
.unique()
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
Ok(OutputStream::one(NuSeries::series_to_value(
res.into_series(),
tag,
)))
let df = NuDataFrame::try_from_series(vec![res], &tag.span)?;
Ok(OutputStream::one(df.into_value(df_tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,8 +2,8 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, NuSeries},
Signature,
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
use crate::commands::dataframe::utils::parse_polars_error;
@ -30,8 +30,22 @@ impl WholeStreamCommand for DataFrame {
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Calculates value counts",
example: "[5 5 6 6] | dataframe to-series | dataframe value-counts",
result: None,
example: "[5 5 5 5 6 6] | dataframe to-df | dataframe value-counts",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"0".to_string(),
vec![UntaggedValue::int(5).into(), UntaggedValue::int(6).into()],
),
Column::new(
"counts".to_string(),
vec![UntaggedValue::int(4).into(), UntaggedValue::int(2).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -39,12 +53,27 @@ impl WholeStreamCommand for DataFrame {
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let series = NuSeries::try_from_stream(&mut args.input, &tag.span)?;
let (df, df_tag) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let df = series
.as_ref()
let df_new = df
.as_series(&df_tag.span)?
.value_counts()
.map_err(|e| parse_polars_error::<&str>(&e, &tag.span, None))?;
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(df, tag)))
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(
df_new, tag,
)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -0,0 +1,78 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue, Value,
};
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe shape"
}
fn usage(&self) -> &str {
"[DataFrame] Shows column and row size for a dataframe"
}
fn signature(&self) -> Signature {
Signature::build("dataframe shape")
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
command(args)
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Shows row and column shape",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe shape",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("rows".to_string(), vec![UntaggedValue::int(2).into()]),
Column::new("columns".to_string(), vec![UntaggedValue::int(2).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let rows = Value {
value: (df.as_ref().height() as i64).into(),
tag: Tag::default(),
};
let cols = Value {
value: (df.as_ref().width() as i64).into(),
tag: Tag::default(),
};
let rows_col = Column::new("rows".to_string(), vec![rows]);
let cols_col = Column::new("columns".to_string(), vec![cols]);
let df = NuDataFrame::try_from_columns(vec![rows_col, cols_col], &tag.span)?;
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -53,7 +53,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let rows: Option<Tagged<usize>> = args.get_flag("n_rows")?;
let tail: bool = args.has_flag("tail");
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let rows = rows.map(|v| v.item);
let values = if tail { df.tail(rows)? } else { df.head(rows)? };

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature, SyntaxShape};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue,
};
use nu_source::Tagged;
pub struct DataFrame;
@ -29,7 +32,15 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "Create new dataframe from a slice of the rows",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe slice 0 1",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::int(1).into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(2).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -40,8 +51,21 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let offset: Tagged<usize> = args.req(0)?;
let size: Tagged<usize> = args.req(1)?;
let df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let res = df.as_ref().slice(offset.item as i64, size.item);
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, NuSeries, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
@ -33,12 +33,39 @@ impl WholeStreamCommand for DataFrame {
Example {
description: "Create new sorted dataframe",
example: "[[a b]; [3 4] [1 2]] | dataframe to-df | dataframe sort a",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
),
Column::new(
"b".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(4).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Create new sorted series",
example: "[3 4 1 2] | dataframe to-series | dataframe sort",
result: None,
example: "[3 4 1 2] | dataframe to-df | dataframe sort",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),
UntaggedValue::int(3).into(),
UntaggedValue::int(4).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -53,31 +80,38 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let reverse = args.has_flag("reverse");
match value.value {
UntaggedValue::DataFrame(PolarsData::EagerDataFrame(df)) => {
let columns: Vec<Value> = args.rest(0)?;
if !columns.is_empty() {
let (col_string, col_span) = convert_columns(&columns, &tag)?;
match &value.value {
UntaggedValue::DataFrame(df) => {
if df.is_series() {
let columns = df.as_ref().get_column_names();
let res = df
.as_ref()
.sort(&col_string, reverse)
.map_err(|e| parse_polars_error::<&str>(&e, &col_span, None))?;
.sort(columns, reverse)
.map_err(|e| parse_polars_error::<&str>(&e, &value.tag.span, None))?;
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
} else {
Err(ShellError::labeled_error(
"Missing columns",
"missing column name to perform sort",
&tag.span,
))
let columns: Vec<Value> = args.rest(0)?;
if !columns.is_empty() {
let (col_string, col_span) = convert_columns(&columns, &tag)?;
let res = df
.as_ref()
.sort(&col_string, reverse)
.map_err(|e| parse_polars_error::<&str>(&e, &col_span, None))?;
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
} else {
Err(ShellError::labeled_error(
"Missing columns",
"missing column name to perform sort",
&tag.span,
))
}
}
}
UntaggedValue::DataFrame(PolarsData::Series(series)) => {
let res = series.as_ref().sort(reverse);
Ok(OutputStream::one(NuSeries::series_to_value(res, tag)))
}
_ => Err(ShellError::labeled_error(
"Incorrect type",
"sort cannot be done with this value",
@ -85,3 +119,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, NuSeries, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use polars::prelude::DataType;
@ -37,16 +37,38 @@ impl WholeStreamCommand for DataFrame {
Example {
description: "Takes selected rows from dataframe",
example: r#"let df = ([[a b]; [4 1] [5 2] [4 3]] | dataframe to-df);
let indices = ([0 2] | dataframe to-series);
let indices = ([0 2] | dataframe to-df);
$df | dataframe take $indices"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![UntaggedValue::int(4).into(), UntaggedValue::int(4).into()],
),
Column::new(
"b".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Takes selected rows from series",
example: r#"let series = ([4 1 5 2 4 3] | dataframe to-series);
let indices = ([0 2] | dataframe to-series);
example: r#"let series = ([4 1 5 2 4 3] | dataframe to-df);
let indices = ([0 2] | dataframe to-df);
$series | dataframe take $indices"#,
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![UntaggedValue::int(4).into(), UntaggedValue::int(5).into()],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
@ -56,8 +78,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let value: Value = args.req(0)?;
let series = match &value.value {
UntaggedValue::DataFrame(PolarsData::Series(series)) => Ok(series),
let df = match &value.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only use a series for take command",
@ -65,7 +87,9 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}?;
let casted = match series.as_ref().dtype() {
let series = df.as_series(&value.tag.span)?;
let casted = match series.dtype() {
DataType::UInt32 | DataType::UInt64 | DataType::Int32 | DataType::Int64 => series
.as_ref()
.cast_with_dtype(&DataType::UInt32)
@ -88,16 +112,11 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
})?;
match value.value {
UntaggedValue::DataFrame(PolarsData::EagerDataFrame(df)) => {
UntaggedValue::DataFrame(df) => {
let res = df.as_ref().take(indices);
Ok(OutputStream::one(NuDataFrame::dataframe_to_value(res, tag)))
}
UntaggedValue::DataFrame(PolarsData::Series(series)) => {
let res = series.as_ref().take(indices);
Ok(OutputStream::one(NuSeries::series_to_value(res, tag)))
}
_ => Err(ShellError::labeled_error(
"No dataframe or series in stream",
"no dataframe or series found in input stream",
@ -105,3 +124,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -64,7 +64,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let delimiter: Option<Tagged<String>> = args.get_flag("delimiter")?;
let no_header: bool = args.has_flag("no_header");
let mut df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (mut df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let mut file = File::create(&file_name.item).map_err(|e| {
ShellError::labeled_error(

View File

@ -1,7 +1,10 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuDataFrame, Signature};
use nu_protocol::{
dataframe::{Column, NuDataFrame},
Signature, UntaggedValue,
};
pub struct DataFrame;
@ -11,7 +14,7 @@ impl WholeStreamCommand for DataFrame {
}
fn usage(&self) -> &str {
"Converts a pipelined Table or List into a polars dataframe"
"Converts a List, Table or Dictionary into a polars dataframe"
}
fn signature(&self) -> Signature {
@ -27,10 +30,108 @@ impl WholeStreamCommand for DataFrame {
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Takes an input stream and converts it to a polars dataframe",
example: "[[a b];[1 2] [3 4]] | dataframe to-df",
result: None,
}]
vec![
Example {
description: "Takes a dictionary and creates a dataframe",
example: "[[a b];[1 2] [3 4]] | dataframe to-df",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![UntaggedValue::int(1).into(), UntaggedValue::int(3).into()],
),
Column::new(
"b".to_string(),
vec![UntaggedValue::int(2).into(), UntaggedValue::int(4).into()],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Takes a list of tables and creates a dataframe",
example: "[[1 2 a] [3 4 b] [5 6 c]] | dataframe to-df",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"0".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(3).into(),
UntaggedValue::int(5).into(),
],
),
Column::new(
"1".to_string(),
vec![
UntaggedValue::int(2).into(),
UntaggedValue::int(4).into(),
UntaggedValue::int(6).into(),
],
),
Column::new(
"2".to_string(),
vec![
UntaggedValue::string("a").into(),
UntaggedValue::string("b").into(),
UntaggedValue::string("c").into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Takes a list and creates a dataframe",
example: "[a b c] | dataframe to-df",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::string("a").into(),
UntaggedValue::string("b").into(),
UntaggedValue::string("c").into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
Example {
description: "Takes a list of booleans and creates a dataframe",
example: "[$true $true $false] | dataframe to-df",
result: Some(vec![NuDataFrame::try_from_columns(
vec![Column::new(
"0".to_string(),
vec![
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(true).into(),
UntaggedValue::boolean(false).into(),
],
)],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
},
]
}
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -48,7 +48,7 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let file_name: Tagged<PathBuf> = args.req(0)?;
let mut df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let (mut df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let file = File::create(&file_name.item).map_err(|e| {
ShellError::labeled_error(

View File

@ -1,44 +0,0 @@
use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{dataframe::NuSeries, Signature, SyntaxShape};
use nu_source::Tagged;
pub struct DataFrame;
impl WholeStreamCommand for DataFrame {
fn name(&self) -> &str {
"dataframe to-series"
}
fn usage(&self) -> &str {
"Converts a pipelined List into a polars series"
}
fn signature(&self) -> Signature {
Signature::build("dataframe to-series").optional(
"name",
SyntaxShape::String,
"Optional series name",
)
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let name: Option<Tagged<String>> = args.opt(0)?;
let name = name.map(|v| v.item);
let series = NuSeries::try_from_iter(args.input, name)?;
Ok(InputStream::one(series.into_value(tag)))
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Takes an input stream and converts it to a polars series",
example: "[1 2 3 4] | dataframe to-series my-col",
result: None,
}]
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::{evaluate_baseline_expr, WholeStreamCommand};
use nu_errors::ShellError;
use nu_protocol::{
dataframe::NuDataFrame,
dataframe::{Column, NuDataFrame},
hir::{CapturedBlock, ClassifiedCommand, Expression, Literal, Operator, SpannedExpression},
Primitive, Signature, SyntaxShape, UnspannedPathMember, UntaggedValue, Value,
};
@ -37,7 +37,15 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "Filter dataframe based on column a",
example: "[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe where a == 1",
result: None,
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new("a".to_string(), vec![UntaggedValue::int(1).into()]),
Column::new("b".to_string(), vec![UntaggedValue::int(2).into()]),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -91,7 +99,7 @@ fn command(args: CommandArgs) -> Result<OutputStream, ShellError> {
let rhs = evaluate_baseline_expr(&expression.right, &args.context)?;
filter_dataframe(args, &col_name, &col_name_span, &rhs, &expression.op)
filter_dataframe(args, col_name, col_name_span, &rhs, &expression.op)
}
macro_rules! comparison_arm {
@ -143,7 +151,7 @@ fn filter_dataframe(
}?;
let span = args.call_info.name_tag.span;
let df = NuDataFrame::try_from_stream(&mut args.input, &span)?;
let (df, _) = NuDataFrame::try_from_stream(&mut args.input, &span)?;
let col = df
.as_ref()
@ -214,3 +222,16 @@ fn filter_dataframe(
args.call_info.name_tag,
)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -2,7 +2,7 @@ use crate::prelude::*;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{
dataframe::{NuDataFrame, PolarsData},
dataframe::{Column, NuDataFrame},
Signature, SyntaxShape, UntaggedValue, Value,
};
use nu_source::Tagged;
@ -33,8 +33,35 @@ impl WholeStreamCommand for DataFrame {
vec![Example {
description: "Adds a series to the dataframe",
example:
"[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe with-column ([5 6] | dataframe to-series) --name c",
result: None,
"[[a b]; [1 2] [3 4]] | dataframe to-df | dataframe with-column ([5 6] | dataframe to-df) --name c",
result: Some(vec![NuDataFrame::try_from_columns(
vec![
Column::new(
"a".to_string(),
vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(3).into(),
],
),
Column::new(
"b".to_string(),
vec![
UntaggedValue::int(2).into(),
UntaggedValue::int(4).into(),
],
),
Column::new(
"c".to_string(),
vec![
UntaggedValue::int(5).into(),
UntaggedValue::int(6).into(),
],
),
],
&Span::default(),
)
.expect("simple df for test should not fail")
.into_value(Tag::default())]),
}]
}
}
@ -44,8 +71,8 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
let value: Value = args.req(0)?;
let name: Tagged<String> = args.req_named("name")?;
let mut series = match value.value {
UntaggedValue::DataFrame(PolarsData::Series(series)) => Ok(series),
let df = match value.value {
UntaggedValue::DataFrame(df) => Ok(df),
_ => Err(ShellError::labeled_error(
"Incorrect type",
"can only add a series to a dataframe",
@ -53,9 +80,11 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
)),
}?;
let series = series.as_mut().rename(name.item.as_ref()).clone();
let mut series = df.as_series(&value.tag.span)?;
let mut df = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
let series = series.rename(name.item.as_ref()).clone();
let (mut df, _) = NuDataFrame::try_from_stream(&mut args.input, &tag.span)?;
df.as_mut()
.with_column(series)
@ -63,3 +92,16 @@ fn command(mut args: CommandArgs) -> Result<OutputStream, ShellError> {
Ok(OutputStream::one(df.into_value(tag)))
}
#[cfg(test)]
mod tests {
use super::DataFrame;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test_dataframe as test_examples;
test_examples(DataFrame {})
}
}

View File

@ -86,7 +86,7 @@ pub fn set_env(args: CommandArgs) -> Result<ActionStream, ShellError> {
ctx.scope.enter_scope();
ctx.scope.add_vars(&captured.entries);
let value = evaluate_baseline_expr(&expr, &ctx);
let value = evaluate_baseline_expr(&expr, ctx);
ctx.scope.exit_scope();

View File

@ -85,10 +85,10 @@ pub fn load_env(args: CommandArgs) -> Result<ActionStream, ShellError> {
let ctx = &args.context;
if let Some(values) = args.opt::<Vec<Value>>(0)? {
load_env_from_table(values, &ctx)?;
load_env_from_table(values, ctx)?;
}
load_env_from_table(args.input, &ctx)?;
load_env_from_table(args.input, ctx)?;
Ok(ActionStream::empty())
}

View File

@ -109,7 +109,7 @@ fn with_env(args: CommandArgs) -> Result<ActionStream, ShellError> {
context.scope.add_env(env);
context.scope.add_vars(&block.captured.entries);
let result = run_block(&block.block, &context, args.input, external_redirection);
let result = run_block(&block.block, context, args.input, external_redirection);
context.scope.exit_scope();
result.map(|x| x.into_action_stream())

View File

@ -2,7 +2,8 @@ use crate::prelude::*;
use nu_engine::shell::RemoveArgs;
use nu_engine::WholeStreamCommand;
use nu_errors::ShellError;
use nu_protocol::{Signature, SyntaxShape};
use nu_protocol::{Primitive, Signature, SyntaxShape, UntaggedValue};
use nu_source::Tagged;
pub struct Remove;
@ -66,7 +67,7 @@ fn rm(args: CommandArgs) -> Result<ActionStream, ShellError> {
let name = args.call_info.name_tag.clone();
let shell_manager = args.shell_manager();
let args = RemoveArgs {
let mut rm_args = RemoveArgs {
rest: args.rest(0)?,
recursive: args.has_flag("recursive"),
trash: args.has_flag("trash"),
@ -74,7 +75,7 @@ fn rm(args: CommandArgs) -> Result<ActionStream, ShellError> {
force: args.has_flag("force"),
};
if args.trash && args.permanent {
if rm_args.trash && rm_args.permanent {
return Ok(ActionStream::one(Err(ShellError::labeled_error(
"only one of --permanent and --trash can be used",
"conflicting flags",
@ -82,7 +83,19 @@ fn rm(args: CommandArgs) -> Result<ActionStream, ShellError> {
))));
}
shell_manager.rm(args, name)
if rm_args.rest.is_empty() {
let mut input_peek = args.input.peekable();
while let Some(v) = &input_peek.next() {
if let UntaggedValue::Primitive(Primitive::FilePath(path)) = &v.value {
rm_args.rest.push(Tagged {
item: path.to_path_buf(),
tag: args.call_info.name_tag.clone(),
})
};
}
}
shell_manager.rm(rm_args, name)
}
#[cfg(test)]

View File

@ -96,7 +96,11 @@ fn all(args: CommandArgs) -> Result<OutputStream, ShellError> {
let result = args.input.fold(init, move |acc, row| {
let condition = condition.clone();
let ctx = ctx.clone();
ctx.scope.add_var("$it", row);
if let Some(positional) = all_args.predicate.block.params.positional.get(0) {
ctx.scope.add_var(positional.0.name(), row);
} else {
ctx.scope.add_var("$it", row);
}
let condition = evaluate_baseline_expr(&condition, &ctx);

View File

@ -67,7 +67,7 @@ fn collect(args: CommandArgs) -> Result<OutputStream, ShellError> {
let result = run_block(
&block.block,
&context,
context,
InputStream::empty(),
external_redirection,
);

View File

@ -126,11 +126,13 @@ fn first(args: CommandArgs) -> Result<OutputStream, ShellError> {
tag,
)),
#[cfg(all(not(target_arch = "wasm32"), feature = "dataframe"))]
UntaggedValue::DataFrame(_) => Err(ShellError::labeled_error(
"unsure how to handled UntaggedValue::DataFrame",
"found dataframe",
tag,
)),
UntaggedValue::DataFrame(_) | UntaggedValue::FrameStruct(_) => {
Err(ShellError::labeled_error(
"unsure how to handled dataframe struct",
"found dataframe",
tag,
))
}
},
None => Ok(input_peek.take(rows_desired).into_output_stream()),
}

View File

@ -52,7 +52,7 @@ fn merge(args: CommandArgs) -> Result<ActionStream, ShellError> {
context.scope.add_vars(&block.captured.entries);
let result = run_block(
&block.block,
&context,
context,
InputStream::empty(),
ExternalRedirection::Stdout,
);

View File

@ -39,6 +39,8 @@ impl WholeStreamCommand for Reverse {
#[allow(clippy::needless_collect)]
fn reverse(args: CommandArgs) -> Result<ActionStream, ShellError> {
// Clippy warning should be ignored
// This collect is needed to apply rev
let input = args.input.collect::<Vec<_>>();
Ok((input.into_iter().rev().map(ReturnSuccess::value)).into_action_stream())
}

View File

@ -26,7 +26,11 @@ impl WholeStreamCommand for Command {
}
fn run(&self, args: CommandArgs) -> Result<OutputStream, ShellError> {
select(args)
let columns: Vec<ColumnPath> = args.rest(0)?;
let input = args.input;
let name = args.call_info.name_tag;
select(name, columns, input)
}
fn examples(&self) -> Vec<Example> {
@ -45,11 +49,11 @@ impl WholeStreamCommand for Command {
}
}
fn select(args: CommandArgs) -> Result<OutputStream, ShellError> {
let name = args.call_info.name_tag.clone();
let columns: Vec<ColumnPath> = args.rest(0)?;
let input = args.input;
fn select(
name: Tag,
columns: Vec<ColumnPath>,
input: InputStream,
) -> Result<OutputStream, ShellError> {
if columns.is_empty() {
return Err(ShellError::labeled_error(
"Select requires columns to select",
@ -119,7 +123,11 @@ fn select(args: CommandArgs) -> Result<OutputStream, ShellError> {
return Err(reason);
}
bring_back.entry(key.clone()).or_insert(vec![]);
// No value for column 'key' found, insert nothing to make sure all rows contain all keys.
bring_back
.entry(key.clone())
.or_insert(vec![])
.push(UntaggedValue::nothing().into());
}
}
}
@ -154,3 +162,65 @@ fn select(args: CommandArgs) -> Result<OutputStream, ShellError> {
}))
.into_output_stream())
}
#[cfg(test)]
mod tests {
use nu_protocol::ColumnPath;
use nu_source::Span;
use nu_source::SpannedItem;
use nu_source::Tag;
use nu_stream::InputStream;
use nu_test_support::value::nothing;
use nu_test_support::value::row;
use nu_test_support::value::string;
use super::select;
use super::Command;
use super::ShellError;
#[test]
fn examples_work_as_expected() -> Result<(), ShellError> {
use crate::examples::test as test_examples;
test_examples(Command {})
}
#[test]
fn select_using_sparse_table() {
// Create a sparse table with 3 rows:
// col_foo | col_bar
// -----------------
// foo |
// | bar
// foo |
let input = vec![
row(indexmap! {"col_foo".into() => string("foo")}),
row(indexmap! {"col_bar".into() => string("bar")}),
row(indexmap! {"col_foo".into() => string("foo")}),
];
let expected = vec![
row(
indexmap! {"col_none".into() => nothing(), "col_foo".into() => string("foo"), "col_bar".into() => nothing()},
),
row(
indexmap! {"col_none".into() => nothing(), "col_foo".into() => nothing(), "col_bar".into() => string("bar")},
),
row(
indexmap! {"col_none".into() => nothing(), "col_foo".into() => string("foo"), "col_bar".into() => nothing()},
),
];
let actual = select(
Tag::unknown(),
vec![
ColumnPath::build(&"col_none".to_string().spanned(Span::unknown())),
ColumnPath::build(&"col_foo".to_string().spanned(Span::unknown())),
ColumnPath::build(&"col_bar".to_string().spanned(Span::unknown())),
],
input.into(),
);
assert_eq!(Ok(expected), actual.map(InputStream::into_vec));
}
}

View File

@ -166,7 +166,7 @@ fn uniq(args: CommandArgs) -> Result<ActionStream, ShellError> {
))
}
#[cfg(feature = "dataframe")]
UntaggedValue::DataFrame(_) => {
UntaggedValue::DataFrame(_) | UntaggedValue::FrameStruct(_) => {
return Err(ShellError::labeled_error(
"uniq -c cannot operate on data structs",
"source",

View File

@ -69,10 +69,10 @@ fn from_json(args: CommandArgs) -> Result<OutputStream, ShellError> {
let concat_string = args.input.collect_string(name_tag.clone())?;
let string_clone: Vec<_> = concat_string.item.lines().map(|x| x.to_string()).collect();
if objects {
Ok(string_clone
#[allow(clippy::needless_collect)]
let lines: Vec<_> = concat_string.item.lines().map(|x| x.to_string()).collect();
Ok(lines
.into_iter()
.filter_map(move |json_str| {
if json_str.is_empty() {

Some files were not shown because too many files have changed in this diff Show More