Compare commits

...

96 Commits

Author SHA1 Message Date
2a084fc838 Bump to 0.17.0 (#2237) 2020-07-22 06:41:49 +12:00
a36d2a1586 Revert "hopefully the final fix for history (#2222)" (#2235)
This reverts commit 6829ad7a30.
2020-07-21 18:19:04 +12:00
32b875ada9 sample config settigns (#2233) 2020-07-20 21:15:58 -05:00
aaed9c4e8a added ansi example (#2230)
* added ansi example

* added strcollect to example
2020-07-20 18:33:39 -05:00
b9278bdfe1 Char example (#2231)
* added ansi example

* added another example

* changed example

* ansi changes here by mistake
2020-07-20 14:25:38 -05:00
6eb2c94209 Add flag for case-insensitive sort-by (#2225)
* Add flag for case-insensitive sort-by

* Fix test names

* Fix documentation comments
2020-07-21 05:31:58 +12:00
7b1a15b223 Campbell colors (#2219)
* added campbell theme to html colors

* updated test results. had to make change for ci.

* hopefully the last changes for this stupid test :)

* moved tests to html.rs

* remove unnecessary using statement.

* still fighting with tests and tests are winning.
2020-07-20 07:57:29 -05:00
836efd237c fix internal command parsing (args.is_last) (#2224) 2020-07-20 05:49:40 +12:00
aad3cca793 Add benchmark command (#2223) 2020-07-20 05:39:43 +12:00
6829ad7a30 hopefully the final fix for history (#2222) 2020-07-19 07:47:55 -05:00
1f0962eb08 Add some tests for parse_arg (#2220)
* add some tests for parse

* Format

* fix warnings
2020-07-19 19:12:56 +12:00
c65acc174d Add hex pretty print to 'to html' (#2221) 2020-07-19 16:44:15 +12:00
2dea392e40 Add hex pretty print to 'to html' (#2217) 2020-07-19 12:14:40 +12:00
0c43a4d04b Add hex pretty print to 'to html' (#2216) 2020-07-19 10:12:17 +12:00
ebc2d40875 Expose more registry APIs (#2215) 2020-07-19 06:01:05 +12:00
3432078e77 Fix uniq to work with simple values (#2214) 2020-07-19 05:19:03 +12:00
9e5170b3dc Clean up lines command (#2207) 2020-07-19 05:17:56 +12:00
0ae7c5d836 Fix if description (#2204) (#2213) 2020-07-19 05:16:35 +12:00
d0712a00f4 made it easier to change colors (#2212)
* made it easier to change colors
and the beginning of html theming

* fmt
2020-07-18 11:05:45 -05:00
5e722181cb Export more defs from nu-cli (#2205) 2020-07-18 16:47:03 +12:00
ffe3e2c16b Rename calc to math eval and allow it to optionally take an expression as an argument (#2195)
* Rename `calc` to `math eval` and allow it to optionally take the expression as an argument

* Moved calc tests to math eval
Also added 2 tests and changed 1 test

* Move calc docs to math eval
2020-07-18 16:11:19 +12:00
04e8aa31fe update history max size with two different calls. (#2202)
Closes #2193
2020-07-18 15:26:32 +12:00
9d24b440bb Introduce completion abstractions to nushell. (#2198)
* Introduce completion abstractions to nushell.

Currently, we rely on rustyline's completion structures. By abstracting this
away, we are more flexible to introduce someone elses completion engine, or our
own.

* Update value_shell.rs

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-07-18 14:55:10 +12:00
d8594a62c2 Add wasm support (#2199)
* Working towards a PoC for wasm

* Move bson and sqlite to plugins

* proof of concept now working

* tests are green

* Add CI test for --no-default-features

* Fix some tests

* Fix clippy and windows build

* More fixes

* Fix the windows build

* Fix the windows test
2020-07-18 13:59:23 +12:00
dbe0effd67 User error propagation operator (#2201) 2020-07-18 13:12:06 +12:00
b358804904 Auto-Generate Documentation for nushell.com (#2139)
* Very rough idea

* Remove colour codes

* Work on command for generating docs

* Quick comment

* Use nested collapsible markdown

* Refine documentation command

* Clippy and rename docs

* This layout probably seems best

Also moved some code to documentation.rs to avoid making help.rs massive

* Delete summaries.md

* Add usage strings

* Remove static annotations

* get_documentation produces value

Which will be used like
'help generate_docs | save "something"'
The resulting yaml can be passed to a script for generating HTML/MD files in the website

* Fix subcommands

* DRY code

* Address clippy:

* Fix links

* Clippy lints

* Move documentation to more central location
2020-07-18 10:22:43 +12:00
7b02604e6d changed colors as per Jörn's suggestion. (#2200)
* changed colors as per Jörn's suggestion.

* cleaned up old comments
2020-07-17 15:02:54 -05:00
6497421615 Keep until and while as subcommands of keep (#2197) 2020-07-18 07:06:48 +12:00
f26151e36d Silence Rust 1.45 Clippy warnings (#2196)
* Silence Rust 1.45 Clippy warnings dealing with using `map_err()`

* Silence false Clippy warning

* Fix last Clippy error for unnecessary conversion

* Fix `and_then` clippy warnings
2020-07-18 05:57:15 +12:00
0f688d7da7 Use '?' for error propagation, remove match (#2194) 2020-07-17 05:39:51 +12:00
a04dfca63a added ability to supply --dark_bg to to html (#2189)
* added ability to supply --dark_bg to to html

* fmt + fixed tests

* updated other html tests

* fmt
2020-07-16 08:19:29 -05:00
72f6513d2a Keybindings and invocation fix (#2186) 2020-07-15 19:51:59 +12:00
7c0a830d84 Match cleanup (#2184)
* Use `unwrap_or()` to remove `match`

* Use `?` for error propogation, and remove `match`
2020-07-15 19:51:41 +12:00
c299d207f7 Remove unnecessary match (#2183) 2020-07-15 19:50:38 +12:00
42a1adf2e9 Indices are (now) green, bold, right-aligned (#2181)
With https://github.com/nushell/nushell/pull/355 the (numeric) index column of tables was changed to be right-aligned. After the move to `nu-table` the index column is now centered instead of right-aligned. I think this is a copy-paste bug where [this line](71e55541d7/crates/nu-cli/src/commands/table.rs (L190)) has been copied from [this line](71e55541d7/crates/nu-cli/src/commands/table.rs (L207)), since the code is out-of-sync with the comment. This change restores harmony between the description and the function of the code.
2020-07-15 15:48:20 +12:00
b4761f9d8a Remove commands meant for internal use. (#2182) 2020-07-14 21:49:46 -05:00
71e55541d7 Merge skip command varieties into one command with sub commands. (#2179) 2020-07-14 20:44:49 -05:00
5f1075544c Remove unnecessary match statement (#2177) 2020-07-14 20:17:28 -04:00
0934410b38 Use matches!() for true/false returning match statements (#2176) 2020-07-14 20:11:41 -04:00
17e6c53b62 Add str reverse subcommand (#2170)
* Add str reverse subcommand

* rustfmt
2020-07-15 08:47:04 +12:00
80d2a7ee7a Fix autoenv executing scripts multiple times (#2171)
* Fix autoenv executing scripts multiple times

Previously, if the user had only specified entry or exitscripts the scripts
would execute many times. This should be fixed now

* Add tests

* Run exitscripts

* More tests and fixes to existing tests

* Test solution with visited dirs

* Track visited directories

* Comments and fmt
2020-07-15 07:16:50 +12:00
8fd22b61be Add variance and stddev subcommands to math command (#2154)
* add variance (population)
subcommand to math

* impl variance subcommand with spanning errors for invalid types

* add stddev subcommand to math

* rename bytes to filesize

* clippy fix -- use expect instead of unwrap in variance tests
2020-07-15 07:15:02 +12:00
e9313a61af Make str more strict. (#2173) 2020-07-14 10:04:00 -05:00
f2c4d22739 group-by can generate custom grouping key by block evaluation. (#2172) 2020-07-14 08:45:19 -05:00
8551e06d9e Ensure source buffer is cleared after reading in MaybeTextCodec. (#2168) 2020-07-14 11:24:52 +12:00
97cedeb324 Fix str --to-int usages (#2167) 2020-07-13 15:07:34 -04:00
07594222c0 Extend 'Shell' with open and save capabilities (#2165)
* Extend 'Shell' with open and save capabilities

* clippy fix
2020-07-13 21:07:44 +12:00
7a207a673b Update documentation to properly refer to subcommands with spaces (#2164) 2020-07-13 18:39:36 +12:00
78f13407e6 Documentation for autoenv (#2163)
* Documentation

* Somewhat nicer?

* cat
2020-07-13 18:23:19 +12:00
5a34744d8c add --char flag to 'str trim' (#2162) 2020-07-13 17:45:34 +12:00
0456f4a007 To html with color (#2158)
* adding color to html output

* latest changes

* seems to be working now

* WIP - close. Good is the enemy of Great.

* fixed the final issues... hopefully
2020-07-13 17:40:59 +12:00
f3f40df4dd Tests for autoenv (and fixes for bugs the tests found) (#2148)
* add test basic_autoenv_vars_are_added

* Tests

* Entry and exit scripts

* Recursive set and overwrite

* Make sure that overwritten vals are restored

* Move tests to autoenv

* Move tests out of cli crate

* Tests help, apparently. Windows has issues

On windows, .nu-env is not applied immediately after running autoenv trust.
You have to cd out of the directory for it to work.

* Sort paths non-lexicographically

* Sibling dir test

* Revert "Sort paths non-lexicographically"

This reverts commit 72e4b856af.

* Rename test

* Change conditions

* Revert "Revert "Sort paths non-lexicographically""

This reverts commit 71606bc62f.

* Set vars as they are discovered

This means that if a parent directory is untrusted,
the variables in its child directories are still set properly.

* format

* Fix cleanup issues too

* Run commands in their separate functions

* Make everything into one large function like all the cool kids

* Refactoring

* fmt

* Debugging windows path issue

* Canonicalize

* Trim whitespace

* On windows, use echo nul instead of touch to create file in test

* Avoid cloning by using drain()
2020-07-12 16:14:09 +12:00
bdef5d7d72 Add 'str from' subcommand (#2125)
* add human, precision commands

* add 'str from' subcommand (converted from human/precision commands)

move human tests to str from

* add default locale, platform-specific SystemLocale use

* fix platform specific num-format dependency, remove invalid test

* change 'str from' localization to static num_format::Locale::en

* minor cleanup, nudge ci

* re-attempt ci
2020-07-12 15:57:39 +12:00
8d03cf5b02 added more verbose message to assert (#2157)
* added more verbose message to assert

having a -h short is bad for any command since it's already used by --help.

* updated for fmt
2020-07-11 16:49:44 -05:00
3ec0242960 fix the name of 'do' (#2152)
* fix the name of 'do'

* try to fix ci
2020-07-11 17:09:05 +12:00
0bc2e29f99 Rename 'bytes' to 'filesize' (#2153) 2020-07-11 14:17:37 +12:00
1bb6a2d9ed Split key/value in 'config set' (#2151) 2020-07-11 13:15:51 +12:00
e848fc0bbe Updates config to use subcommands (#2146)
* First commit updating `config` to use subcommands (#2119)
    - Implemented `get` subcommand

* Implmented `config set` as a subcommand.

* Implemented `config set_into` as subcommand

* Fixed base `config` command
 - Instead of outputting help, it now outputs the list of all
 configuration parameters.

* Added `config clear` subcommand

* Added `config load` and `config remove` subcommands

* Added `config path` subcommand

* fixed clippy
2020-07-11 12:11:04 +12:00
6820d70e7d make duration pretty print clearer (#2150)
* make duration pretty print clearer

* fix typo and tests

* fix typo and tests
2020-07-11 09:06:52 +12:00
f32ab696d3 Return iter from sort by (#2149) 2020-07-11 06:49:55 +12:00
e07a9e4ee7 1747 add ns to duration (#2128)
* Added nanos to Duration

* Removed unwraps

* Added nanos to Duration

* Removed unwraps

* Fixed errors

* Removed unwraps

* Changed serialization to String

* Fixed Date and Duration comparison
2020-07-11 05:48:11 +12:00
6a89b1b010 Remove duplicate method (retag) (#2147) 2020-07-10 06:21:13 -04:00
b1b93931cb Return an iter from last command (#2143) 2020-07-09 09:07:51 -04:00
1e62a8fb6e Fix variable name (#2142) 2020-07-08 19:55:01 -04:00
ed6f337a48 Refactor Fetch command (No logic changes) (#2131)
* Refactoring unrelated to Streaming...

Mainly keeping code DRY

* Remove cli as dependency
2020-07-09 04:49:27 +12:00
b004236927 Return iter from from vcf (#2137) 2020-07-08 09:20:57 -04:00
0fdb9ac5e2 str substring additions. (#2140) 2020-07-08 04:45:45 -05:00
28be39494c add requoting for completions (#2129) 2020-07-07 17:13:39 -04:00
32f18536e1 Add space to special prompt chars (#2122)
So spaces don't have to be escaped with quotes in the config.toml
2020-07-06 10:30:47 -05:00
34e1e6e426 Add "move column" command. (#2123) 2020-07-06 10:27:01 -05:00
c3ba1e476f Make every stream-able (#2120)
* Make every stream-able

* Make each over ranges stream-able
2020-07-06 20:23:27 +12:00
a1a0710ee6 Return iter from every command (#2118)
* Return iter from `every` command

* Clippy

* Better variable name
2020-07-06 14:25:39 +12:00
455b1ac294 Update config.yml 2020-07-06 08:21:46 +12:00
b2e0dc5b77 Update azure-pipelines.yml 2020-07-06 08:20:38 +12:00
d30c40b40e Bump to 0.16.1 (#2116) 2020-07-06 08:12:44 +12:00
85d848dd7d Stream results of drop command (#2114)
* Stream results of drop command

* When the amount of rows to drop is equal to or greaten than the size of the table, output nothing
2020-07-06 05:46:06 +12:00
74717582ac Slightly nicer "rm" message (#2113)
* maybe this was root issue

* quotes

* formatting
2020-07-06 05:42:37 +12:00
ee18f16378 Autoenv rewrite, security and scripting (#2083)
* Add args in .nurc file to environment

* Working dummy version

* Add add_nurc to sync_env command

* Parse .nurc file

* Delete env vars after leaving directory

* Removing vals not working, strangely

* Refactoring, add comment

* Debugging

* Debug by logging to file

* Add and remove env var behavior appears correct

However, it does not use existing code that well.

* Move work to cli.rs

* Parse config directories

* I am in a state of distress

* Rename .nurc to .nu

* Some notes for me

* Refactoring

* Removing vars works, but not done in a very nice fashion

* Refactor env_vars_to_delete

* Refactor env_vars_to_add()

* Move directory environment code to separate file

* Refactor from_config

* Restore env values

* Working?

* Working?

* Update comments and change var name

* Formatting

* Remove vars after leaving dir

* Remove notes I made

* Rename config function

* Clippy

* Cleanup and handle errors

* cargo fmt

* Better error messages, remove last (?) unwrap

* FORMAT PLZ

* Rename whitelisted_directories to allowed_directories

* Add comment to clarify how overwritten values are restored.

* Change list of allowed dirs to indexmap

* Rewrite starting

* rewrite everything

* Overwritten env values tracks an indexmap instead of vector

* Refactor restore function

* Untrack removed vars properly

* Performance concerns

* Performance concerns

* Error handling

* Clippy

* Add type aliases for String and OsString

* Deletion almost works

* Working?

* Error handling and refactoring

* nicer errors

* Add TODO file

* Move outside of loop

* Error handling

* Reworking adding of vars

* Reworking adding of vars

* Ready for testing

* Refactoring

* Restore overwritten vals code

* todo.org

* Remove overwritten values tracking, as it is not needed

* Cleanup, stop tracking overwritten values as nu takes care of it

* Init autoenv command

* Initialize autoenv and autoenv trust

* autoenv trust toml

* toml

* Use serde for autoenv

* Optional directory arg

* Add autoenv untrust command

* ... actually add autoenv untrust this time

* OsString and paths

* Revert "OsString and paths"

This reverts commit e6eedf8824.

* Fix path

* Fix path

* Autoenv trust and untrust

* Start using autoenv

* Check hashes

* Use trust functionality when setting vars

* Remove unused code

* Clippy

* Nicer errors for autoenv commands

* Non-working errors

* Update error description

* Satisfy fmt

* Errors

* Errors print, but not nicely

* Nicer errors

* fmt

* Delete accidentally added todo.org file

* Rename direnv to autoenv

* Use ShellError instead of Error

* Change tests to pass, danger zone?

* Clippy and errors

* Clippy... again

* Replace match with or_else

* Use sha2 crate for hashing

* parsing and error msg

* Refactoring

* Only apply vars once

* if parent dir

* Delete vars

* Rework exit code

* Adding works

* restore

* Fix possibility of infinite loop

* Refactoring

* Non-working

* Revert "Non-working"

This reverts commit e231b85570.

* Revert "Revert "Non-working""

This reverts commit 804092e46a.

* Autoenv trust works without restart

* Cargo fix

* Script vars

* Serde

* Serde errors

* Entry and exitscripts

* Clippy

* Support windows and handle errors

* Formatting

* Fix infinite loop on windows

* Debugging windows loop

* More windows infinite loop debugging

* Windows loop debugging #3

* windows loop #4

* Don't return err

* Cleanup unused code

* Infinite loop debug

* Loop debugging

* Check if infinite loop is vars_to_add

* env_vars_to_add does not terminate, skip loop as test

* Hypothesis: std::env::current_dir() is messing with something

* Hypothesis: std::env::current_dir() is messing with something

* plz

* make clippy happy

* debugging in env_vars_to_add

* Debbuging env_vars_to_add #2

* clippy

* clippy..

* Fool clippy

* Fix another infinite loop

* Binary search for error location x)

* Binary search #3

* fmt

* Binary search #4

* more searching...

* closing in... maybe

* PLZ

* Cleanup

* Restore commented out functionality

* Handle case when user gives the directory "."

* fmt

* Use fs::canonicalize for paths

* Create optional script section

* fmt

* Add exitscripts even if no entryscripts are defined

* All sections in .nu-env are now optional

* Re-read config file each directory change

* Hot reload after autoenv untrust, don't run exitscripts if untrusted

* Debugging

* Fix issue with recursive adding of vars

* Thank you for finding my issues Mr. Azure

* use std::env
2020-07-06 05:34:00 +12:00
9e82e5a2fa Show entire table if number of rows requested for last is greater than table size (#2112)
* Show entire table if number of rows requested for last is greater than table size

* rustfmt

* Add test
2020-07-05 13:04:17 +12:00
8ea2307815 More informative error messages when "rm" fails (#2109)
* add a nicer error message

* small fix to message and remove log
2020-07-05 13:03:12 +12:00
bbc5a28fe9 Fix buffering in lines command (#2111) 2020-07-05 12:20:58 +12:00
04120e00e4 Oops, fix crash in parser updates (#2108) 2020-07-05 08:56:54 +12:00
efd8a633f2 Align tables in "ls -f" (#2105)
* Stuff column with nothing if we have nothing

* Stuff columns at the very start

Remove unnecessary else clauses.
Add the unix cfg portion

* Added some tests and cfg windows

Not sure how I feel about these tests but it's better than nothing
2020-07-05 08:17:36 +12:00
e75c44c95b If command and touchups (#2106) 2020-07-05 07:40:04 +12:00
0629c896eb Return error for unterminated string in bareword parser (#2103)
* add some tests

* add failing tests

* use some; fix test

* clean up code, flesh out tests

* cargo fmt
2020-07-04 17:14:31 +12:00
eb02c773d0 Add 'str length' command (#2102) 2020-07-04 08:17:44 +12:00
e31e8d1550 Convert open/fetch to stream (#2028)
* Types lined up for open with stream

* Chunking stream

* Maybe I didn't need most of the Stream stuff after all?

* Some clean-up

* Merge weird cargo.lock

* Start moving some encoding logic to MaybeTextCodec

Will we lose the nice table formatting if we Stream? How do we get it back? Collect the Stream at the end?

* Clean-up and small refinements

* Put in auto-convert workaround

* Workaround to make sure bat functionality works

* Handle some easy error cases

* All tests pass

* Remove guessing logic

* Address clippy comments

* Pull latest master and fix MaybeTextCodec usage

* Add tag to enable autoview
2020-07-04 07:53:20 +12:00
8775991c2d Add 'split chars' command (#2101) 2020-07-04 07:09:38 +12:00
de8e2841a0 Numbered each (#2100)
* Add --numbered to each

* Fix example tester and add numbered each
2020-07-03 20:43:55 +12:00
5cafead4a4 Added license and license-for-less to wix build (#2097) 2020-07-03 11:30:00 +12:00
180290f3a8 Remove custom escaping for external args. (#2095)
Our own custom escaping unfortunately is far too simple to cover all cases.
Instead, the parser will now do no transforms on the args passed to an external
command, letting the process spawning library deal with doing the appropriate
escaping.
2020-07-03 11:29:28 +12:00
7813063c93 updated less license to raw gh link (#2088) 2020-07-02 16:25:07 +12:00
ba5d774fe1 Add a histogram example to the random dice documentation (#2087) 2020-07-02 16:24:28 +12:00
7be49e43fd added a few more command chars (#2086)
* added a few more command chars

* forgot my ole' friend clippy

* added unicode name synonums to characters
2020-07-01 17:34:11 -05:00
dcd2227201 Update README.md 2020-07-02 09:00:44 +12:00
2dd28c2909 updated to include less and nushell licenses (#2085) 2020-07-01 10:45:42 +12:00
264 changed files with 9087 additions and 3589 deletions

View File

@ -1,11 +1,14 @@
trigger:
- master
- main
strategy:
matrix:
linux-stable:
image: ubuntu-18.04
style: 'unflagged'
linux-minimal:
image: ubuntu-18.04
style: 'minimal'
macos-stable:
image: macos-10.14
style: 'unflagged'
@ -41,10 +44,10 @@ steps:
rustup component add clippy --toolchain stable-x86_64-apple-darwin
export PATH=$HOME/.cargo/bin:$PATH
fi
rustup update
rustc -Vv
echo "##vso[task.prependpath]$HOME/.cargo/bin"
rustup component add rustfmt
# rustup update
# rustc -Vv
# echo "##vso[task.prependpath]$HOME/.cargo/bin"
# rustup component add rustfmt
displayName: Install Rust
- bash: RUSTFLAGS="-D warnings" cargo test --all --features stable
condition: eq(variables['style'], 'unflagged')
@ -58,6 +61,9 @@ steps:
- bash: NUSHELL_ENABLE_ALL_FLAGS=1 RUSTFLAGS="-D warnings" cargo clippy --all --features=stable -- -D clippy::result_unwrap_used -D clippy::option_unwrap_used
condition: eq(variables['style'], 'canary')
displayName: Check clippy lints
- bash: RUSTFLAGS="-D warnings" cargo test --all --no-default-features
condition: eq(variables['style'], 'minimal')
displayName: Run tests
- bash: cargo fmt --all -- --check
condition: eq(variables['style'], 'fmt')
displayName: Lint

View File

@ -1,3 +1 @@
[build]
#rustflags = ["--cfg", "data_processing_primitives"]

View File

@ -27,7 +27,7 @@ orbs:
workflows:
version: 2.0
# This builds on all pull requests to test, and ignores master
# This builds on all pull requests to test, and ignores main
build_without_deploy:
jobs:
- docker/publish:
@ -39,7 +39,7 @@ workflows:
filters:
branches:
ignore:
- master
- main
before_build:
- pull_cache
after_build:
@ -98,11 +98,11 @@ workflows:
docker push quay.io/nushell/nu
# publish devel to Docker Hub on merge to master (doesn't build --release)
# publish devel to Docker Hub on merge to main (doesn't build --release)
build_with_deploy_devel:
jobs:
# Deploy devel tag on merge to master
# Deploy devel tag on merge to main
- docker/publish:
image: nushell/nu-base
registry: quay.io
@ -113,7 +113,7 @@ workflows:
- pull_cache
filters:
branches:
only: master
only: main
after_build:
- run:
name: Build Multistage (smaller) container
@ -137,7 +137,7 @@ workflows:
filters:
branches:
only:
- master
- main
jobs:
- docker/publish:
image: nushell/nu-base

View File

@ -11,31 +11,39 @@ jobs:
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Install libxcb
run: sudo apt-get install libxcb-composite0-dev
- name: Set up cargo
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Build
uses: actions-rs/cargo@v1
with:
command: build
args: --release --all --features=stable
- name: Create output directory
run: mkdir output
- name: Copy files to output
run: |
cp target/release/nu target/release/nu_plugin_* output/
cp README.build.txt output/README.txt
cp LICENSE output/LICENSE
rm output/*.d
rm output/nu_plugin_core_*
rm output/nu_plugin_stable_*
# Note: If OpenSSL changes, this path will need to be updated
- name: Copy OpenSSL to output
run: cp /usr/lib/x86_64-linux-gnu/libssl.so.1.1 output/
- name: Upload artifact
uses: actions/upload-artifact@v2
with:
@ -48,26 +56,32 @@ jobs:
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Set up cargo
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Build
uses: actions-rs/cargo@v1
with:
command: build
args: --release --all --features=stable
- name: Create output directory
run: mkdir output
- name: Copy files to output
run: |
cp target/release/nu target/release/nu_plugin_* output/
cp README.build.txt output/README.txt
cp LICENSE output/LICENSE
rm output/*.d
rm output/nu_plugin_core_*
rm output/nu_plugin_stable_*
- name: Upload artifact
uses: actions/upload-artifact@v2
with:
@ -80,29 +94,40 @@ jobs:
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Set up cargo
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
override: true
- name: Add cargo-wix subcommand
uses: actions-rs/cargo@v1
with:
command: install
args: cargo-wix
- name: Build
uses: actions-rs/cargo@v1
with:
command: build
args: --release --all --features=stable
- name: Create output directory
run: mkdir output
- name: Download Less Binary
run: Invoke-WebRequest -Uri "https://github.com/jftuga/less-Windows/releases/download/less-v562.1/less.exe" -OutFile "target\release\less.exe"
run: Invoke-WebRequest -Uri "https://github.com/jftuga/less-Windows/releases/download/less-v562.0/less.exe" -OutFile "target\release\less.exe"
- name: Download Less License
run: Invoke-WebRequest -Uri "https://raw.githubusercontent.com/jftuga/less-Windows/master/LICENSE" -OutFile "target\release\LICENSE-for-less.txt"
- name: Copy files to output
run: |
cp target\release\nu.exe output\
cp LICENSE output\
cp target\release\LICENSE-for-less.txt output\
rm target\release\nu_plugin_core_*.exe
rm target\release\nu_plugin_stable_*.exe
cp target\release\nu_plugin_*.exe output\
@ -111,16 +136,19 @@ jobs:
# Note: If the version of `less.exe` needs to be changed, update this URL
# Similarly, if `less.exe` is checked into the repo, copy from the local path here
# moved this stuff down to create wix after we download less
- name: Create msi with wix
uses: actions-rs/cargo@v1
with:
command: wix
args: --no-build --nocapture --output target\wix\nushell-windows.msi
- name: Upload installer
uses: actions/upload-artifact@v2
with:
name: windows-installer
path: target\wix\nushell-windows.msi
- name: Upload zip
uses: actions/upload-artifact@v2
with:
@ -137,6 +165,7 @@ jobs:
steps:
- name: Check out code
uses: actions/checkout@v2
- name: Determine Release Info
id: info
env:
@ -152,6 +181,7 @@ jobs:
echo "::set-output name=macosdir::nu_${MAJOR}_${MINOR}_${PATCH}_macOS"
echo "::set-output name=windowsdir::nu_${MAJOR}_${MINOR}_${PATCH}_windows"
echo "::set-output name=innerdir::nushell-${VERSION}"
- name: Create Release Draft
id: create_release
uses: actions/create-release@v1
@ -161,19 +191,24 @@ jobs:
tag_name: ${{ github.ref }}
release_name: ${{ steps.info.outputs.version }} Release
draft: true
- name: Create Linux Directory
run: mkdir -p ${{ steps.info.outputs.linuxdir }}/${{ steps.info.outputs.innerdir }}
- name: Download Linux Artifacts
uses: actions/download-artifact@v2
with:
name: linux
path: ${{ steps.info.outputs.linuxdir }}/${{ steps.info.outputs.innerdir }}
- name: Restore Linux File Modes
run: |
chmod 755 ${{ steps.info.outputs.linuxdir }}/${{ steps.info.outputs.innerdir }}/nu*
chmod 755 ${{ steps.info.outputs.linuxdir }}/${{ steps.info.outputs.innerdir }}/libssl*
- name: Create Linux tarball
run: tar -zcvf ${{ steps.info.outputs.linuxdir }}.tar.gz ${{ steps.info.outputs.linuxdir }}
- name: Upload Linux Artifact
uses: actions/upload-release-asset@v1
env:
@ -183,17 +218,22 @@ jobs:
asset_path: ./${{ steps.info.outputs.linuxdir }}.tar.gz
asset_name: ${{ steps.info.outputs.linuxdir }}.tar.gz
asset_content_type: application/gzip
- name: Create macOS Directory
run: mkdir -p ${{ steps.info.outputs.macosdir }}/${{ steps.info.outputs.innerdir }}
- name: Download macOS Artifacts
uses: actions/download-artifact@v2
with:
name: macos
path: ${{ steps.info.outputs.macosdir }}/${{ steps.info.outputs.innerdir }}
- name: Restore macOS File Modes
run: chmod 755 ${{ steps.info.outputs.macosdir }}/${{ steps.info.outputs.innerdir }}/nu*
- name: Create macOS Archive
run: zip -r ${{ steps.info.outputs.macosdir }}.zip ${{ steps.info.outputs.macosdir }}
- name: Upload macOS Artifact
uses: actions/upload-release-asset@v1
env:
@ -203,18 +243,22 @@ jobs:
asset_path: ./${{ steps.info.outputs.macosdir }}.zip
asset_name: ${{ steps.info.outputs.macosdir }}.zip
asset_content_type: application/zip
- name: Create Windows Directory
run: mkdir -p ${{ steps.info.outputs.windowsdir }}/${{ steps.info.outputs.innerdir }}
- name: Download Windows zip
uses: actions/download-artifact@v2
with:
name: windows-zip
path: ${{ steps.info.outputs.windowsdir }}/${{ steps.info.outputs.innerdir }}
# TODO: Remove Show
- name: Show Windows Artifacts
run: ls -la ${{ steps.info.outputs.windowsdir }}/${{ steps.info.outputs.innerdir }}
- name: Create macOS Archive
run: zip -r ${{ steps.info.outputs.windowsdir }}.zip ${{ steps.info.outputs.windowsdir }}
- name: Upload Windows zip
uses: actions/upload-release-asset@v1
env:
@ -224,11 +268,13 @@ jobs:
asset_path: ./${{ steps.info.outputs.windowsdir }}.zip
asset_name: ${{ steps.info.outputs.windowsdir }}.zip
asset_content_type: application/zip
- name: Download Windows installer
uses: actions/download-artifact@v2
with:
name: windows-installer
path: ./
- name: Upload Windows installer
uses: actions/upload-release-asset@v1
env:

206
Cargo.lock generated
View File

@ -81,6 +81,15 @@ version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4c527152e37cf757a3f78aae5a06fbeefdb07ccc535c980a3208ee3060dd544"
[[package]]
name = "arrayvec"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cd9fd44efafa8690358b7408d253adf110036b88f55672a933f01d616ad9b1b9"
dependencies = [
"nodrop",
]
[[package]]
name = "arrayvec"
version = "0.5.1"
@ -303,7 +312,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d8fb2d74254a3a0b5cac33ac9f8ed0e44aa50378d9dbb2e5d83bd21ed1dc2c8a"
dependencies = [
"arrayref",
"arrayvec",
"arrayvec 0.5.1",
"constant_time_eq",
]
@ -322,7 +331,16 @@ dependencies = [
"block-padding",
"byte-tools",
"byteorder",
"generic-array",
"generic-array 0.12.3",
]
[[package]]
name = "block-buffer"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4152116fd6e9dadb291ae18fc1ec3575ed6d84c29642d97890f4b4a3417297e4"
dependencies = [
"generic-array 0.14.2",
]
[[package]]
@ -634,6 +652,12 @@ version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3a71ab494c0b5b860bdc8407ae08978052417070c2ced38573a9157ad75b8ac"
[[package]]
name = "cpuid-bool"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6d375c433320f6c5057ae04a04376eef4d04ce2801448cf8863a78da99107be4"
[[package]]
name = "crc32fast"
version = "1.2.0"
@ -866,7 +890,16 @@ version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f3d0c8c8752312f9713efd397ff63acb9f85585afbf179282e720e7704954dd5"
dependencies = [
"generic-array",
"generic-array 0.12.3",
]
[[package]]
name = "digest"
version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3dd60d1080a57a05ab032377049e0591415d2b31afd7028356dbf3cc6dcb066"
dependencies = [
"generic-array 0.14.2",
]
[[package]]
@ -1418,6 +1451,16 @@ dependencies = [
"typenum",
]
[[package]]
name = "generic-array"
version = "0.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac746a5f3bbfdadd6106868134545e684693d54d9d44f6e9588a7d54af0bf980"
dependencies = [
"typenum",
"version_check 0.9.2",
]
[[package]]
name = "gethostname"
version = "0.2.1"
@ -1954,7 +1997,7 @@ version = "0.7.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db65c6da02e61f55dae90a0ae427b2a5f6b3e8db09f58d10efab23af92592616"
dependencies = [
"arrayvec",
"arrayvec 0.5.1",
"bitflags",
"cfg-if",
"ryu",
@ -2358,6 +2401,12 @@ dependencies = [
"void",
]
[[package]]
name = "nodrop"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72ef4a56884ca558e5ddb05a1d1e7e1bfd9a68d9ed024c21704cc98872dae1bb"
[[package]]
name = "nom"
version = "1.2.4"
@ -2396,7 +2445,7 @@ dependencies = [
[[package]]
name = "nu"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"clap",
"crossterm",
@ -2415,6 +2464,8 @@ dependencies = [
"nu-value-ext",
"nu_plugin_binaryview",
"nu_plugin_fetch",
"nu_plugin_from_bson",
"nu_plugin_from_sqlite",
"nu_plugin_inc",
"nu_plugin_match",
"nu_plugin_post",
@ -2422,6 +2473,8 @@ dependencies = [
"nu_plugin_start",
"nu_plugin_sys",
"nu_plugin_textview",
"nu_plugin_to_bson",
"nu_plugin_to_sqlite",
"nu_plugin_tree",
"pretty_env_logger",
"semver 0.10.0",
@ -2434,7 +2487,7 @@ dependencies = [
[[package]]
name = "nu-build"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"lazy_static 1.4.0",
"serde 1.0.114",
@ -2444,7 +2497,7 @@ dependencies = [
[[package]]
name = "nu-cli"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"ansi_term 0.12.1",
"app_dirs",
@ -2452,11 +2505,9 @@ dependencies = [
"async-trait",
"base64 0.12.3",
"bigdecimal",
"bson",
"byte-unit",
"bytes 0.5.5",
"calamine",
"cfg-if",
"chrono",
"clap",
"clipboard",
@ -2495,6 +2546,7 @@ dependencies = [
"nu-test-support",
"nu-value-ext",
"num-bigint",
"num-format",
"num-traits 0.2.12",
"parking_lot 0.11.0",
"pin-utils",
@ -2517,6 +2569,7 @@ dependencies = [
"serde_json",
"serde_urlencoded",
"serde_yaml",
"sha2",
"shellexpand",
"starship",
"strip-ansi-escapes",
@ -2536,7 +2589,7 @@ dependencies = [
[[package]]
name = "nu-errors"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"ansi_term 0.12.1",
"bigdecimal",
@ -2556,7 +2609,7 @@ dependencies = [
[[package]]
name = "nu-parser"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"bigdecimal",
"codespan-reporting",
@ -2575,7 +2628,7 @@ dependencies = [
[[package]]
name = "nu-plugin"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"bigdecimal",
"indexmap",
@ -2591,7 +2644,7 @@ dependencies = [
[[package]]
name = "nu-protocol"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"ansi_term 0.12.1",
"bigdecimal",
@ -2608,6 +2661,7 @@ dependencies = [
"nu-errors",
"nu-source",
"num-bigint",
"num-integer",
"num-traits 0.2.12",
"query_interface",
"serde 1.0.114",
@ -2620,7 +2674,7 @@ dependencies = [
[[package]]
name = "nu-source"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"codespan-reporting",
"derive-new",
@ -2633,7 +2687,7 @@ dependencies = [
[[package]]
name = "nu-table"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"ansi_term 0.12.1",
"unicode-width",
@ -2641,7 +2695,7 @@ dependencies = [
[[package]]
name = "nu-test-support"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"directories 2.0.2",
"dunce",
@ -2657,7 +2711,7 @@ dependencies = [
[[package]]
name = "nu-value-ext"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"indexmap",
"itertools",
@ -2671,7 +2725,7 @@ dependencies = [
[[package]]
name = "nu_plugin_binaryview"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"ansi_term 0.12.1",
"crossterm",
@ -2688,7 +2742,7 @@ dependencies = [
[[package]]
name = "nu_plugin_fetch"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"base64 0.12.3",
"futures 0.3.5",
@ -2701,9 +2755,40 @@ dependencies = [
"url",
]
[[package]]
name = "nu_plugin_from_bson"
version = "0.17.0"
dependencies = [
"bigdecimal",
"bson",
"nu-build",
"nu-errors",
"nu-plugin",
"nu-protocol",
"nu-source",
"nu-value-ext",
"num-traits 0.2.12",
]
[[package]]
name = "nu_plugin_from_sqlite"
version = "0.17.0"
dependencies = [
"bigdecimal",
"nu-build",
"nu-errors",
"nu-plugin",
"nu-protocol",
"nu-source",
"nu-value-ext",
"num-traits 0.2.12",
"rusqlite",
"tempfile",
]
[[package]]
name = "nu_plugin_inc"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"nu-build",
"nu-errors",
@ -2716,7 +2801,7 @@ dependencies = [
[[package]]
name = "nu_plugin_match"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"futures 0.3.5",
"nu-build",
@ -2729,7 +2814,7 @@ dependencies = [
[[package]]
name = "nu_plugin_post"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"base64 0.12.3",
"futures 0.3.5",
@ -2746,7 +2831,7 @@ dependencies = [
[[package]]
name = "nu_plugin_ps"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"futures 0.3.5",
"futures-timer",
@ -2760,7 +2845,7 @@ dependencies = [
[[package]]
name = "nu_plugin_start"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"glob",
"nu-build",
@ -2774,7 +2859,7 @@ dependencies = [
[[package]]
name = "nu_plugin_sys"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"battery",
"futures 0.3.5",
@ -2785,11 +2870,12 @@ dependencies = [
"nu-plugin",
"nu-protocol",
"nu-source",
"num-bigint",
]
[[package]]
name = "nu_plugin_textview"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"ansi_term 0.12.1",
"bat",
@ -2805,9 +2891,39 @@ dependencies = [
"url",
]
[[package]]
name = "nu_plugin_to_bson"
version = "0.17.0"
dependencies = [
"bson",
"nu-build",
"nu-errors",
"nu-plugin",
"nu-protocol",
"nu-source",
"nu-value-ext",
"num-traits 0.2.12",
]
[[package]]
name = "nu_plugin_to_sqlite"
version = "0.17.0"
dependencies = [
"hex 0.4.2",
"nu-build",
"nu-errors",
"nu-plugin",
"nu-protocol",
"nu-source",
"nu-value-ext",
"num-traits 0.2.12",
"rusqlite",
"tempfile",
]
[[package]]
name = "nu_plugin_tree"
version = "0.16.0"
version = "0.17.0"
dependencies = [
"derive-new",
"nu-build",
@ -2830,6 +2946,17 @@ dependencies = [
"serde 1.0.114",
]
[[package]]
name = "num-format"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bafe4179722c2894288ee77a9f044f02811c86af699344c498b0840c698a2465"
dependencies = [
"arrayvec 0.4.12",
"itoa",
"num-bigint",
]
[[package]]
name = "num-integer"
version = "0.1.43"
@ -2969,6 +3096,12 @@ version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2839e79665f131bdb5782e51f2c6c9599c133c6098982a54c794358bf432529c"
[[package]]
name = "opaque-debug"
version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "624a8340c38c1b80fd549087862da4ba43e08858af025b236e509b6649fc13d5"
[[package]]
name = "open"
version = "1.4.0"
@ -3892,10 +4025,23 @@ version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7d94d0bede923b3cea61f3f1ff57ff8cdfd77b400fb8f9998949e0cf04163df"
dependencies = [
"block-buffer",
"digest",
"block-buffer 0.7.3",
"digest 0.8.1",
"fake-simd",
"opaque-debug",
"opaque-debug 0.2.3",
]
[[package]]
name = "sha2"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2933378ddfeda7ea26f48c555bdad8bb446bf8a3d17832dc83e380d444cfb8c1"
dependencies = [
"block-buffer 0.9.0",
"cfg-if",
"cpuid-bool",
"digest 0.9.0",
"opaque-debug 0.3.0",
]
[[package]]

View File

@ -1,16 +1,16 @@
[package]
name = "nu"
version = "0.16.0"
authors = ["The Nu Project Contributors"]
description = "A new type of shell"
license = "MIT"
edition = "2018"
readme = "README.md"
default-run = "nu"
repository = "https://github.com/nushell/nushell"
homepage = "https://www.nushell.sh"
description = "A new type of shell"
documentation = "https://www.nushell.sh/book/"
edition = "2018"
exclude = ["images"]
homepage = "https://www.nushell.sh"
license = "MIT"
name = "nu"
readme = "README.md"
repository = "https://github.com/nushell/nushell"
version = "0.17.0"
[workspace]
members = ["crates/*/"]
@ -18,67 +18,92 @@ members = ["crates/*/"]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
nu-cli = { version = "0.16.0", path = "./crates/nu-cli" }
nu-source = { version = "0.16.0", path = "./crates/nu-source" }
nu-plugin = { version = "0.16.0", path = "./crates/nu-plugin" }
nu-protocol = { version = "0.16.0", path = "./crates/nu-protocol" }
nu-errors = { version = "0.16.0", path = "./crates/nu-errors" }
nu-parser = { version = "0.16.0", path = "./crates/nu-parser" }
nu-value-ext = { version = "0.16.0", path = "./crates/nu-value-ext" }
nu_plugin_binaryview = { version = "0.16.0", path = "./crates/nu_plugin_binaryview", optional=true }
nu_plugin_fetch = { version = "0.16.0", path = "./crates/nu_plugin_fetch", optional=true }
nu_plugin_inc = { version = "0.16.0", path = "./crates/nu_plugin_inc", optional=true }
nu_plugin_match = { version = "0.16.0", path = "./crates/nu_plugin_match", optional=true }
nu_plugin_post = { version = "0.16.0", path = "./crates/nu_plugin_post", optional=true }
nu_plugin_ps = { version = "0.16.0", path = "./crates/nu_plugin_ps", optional=true }
nu_plugin_start = { version = "0.16.0", path = "./crates/nu_plugin_start", optional=true }
nu_plugin_sys = { version = "0.16.0", path = "./crates/nu_plugin_sys", optional=true }
nu_plugin_textview = { version = "0.16.0", path = "./crates/nu_plugin_textview", optional=true }
nu_plugin_tree = { version = "0.16.0", path = "./crates/nu_plugin_tree", optional=true }
nu-cli = {version = "0.17.0", path = "./crates/nu-cli"}
nu-errors = {version = "0.17.0", path = "./crates/nu-errors"}
nu-parser = {version = "0.17.0", path = "./crates/nu-parser"}
nu-plugin = {version = "0.17.0", path = "./crates/nu-plugin"}
nu-protocol = {version = "0.17.0", path = "./crates/nu-protocol"}
nu-source = {version = "0.17.0", path = "./crates/nu-source"}
nu-value-ext = {version = "0.17.0", path = "./crates/nu-value-ext"}
nu_plugin_binaryview = {version = "0.17.0", path = "./crates/nu_plugin_binaryview", optional = true}
nu_plugin_fetch = {version = "0.17.0", path = "./crates/nu_plugin_fetch", optional = true}
nu_plugin_from_bson = {version = "0.17.0", path = "./crates/nu_plugin_from_bson", optional = true}
nu_plugin_from_sqlite = {version = "0.17.0", path = "./crates/nu_plugin_from_sqlite", optional = true}
nu_plugin_inc = {version = "0.17.0", path = "./crates/nu_plugin_inc", optional = true}
nu_plugin_match = {version = "0.17.0", path = "./crates/nu_plugin_match", optional = true}
nu_plugin_post = {version = "0.17.0", path = "./crates/nu_plugin_post", optional = true}
nu_plugin_ps = {version = "0.17.0", path = "./crates/nu_plugin_ps", optional = true}
nu_plugin_start = {version = "0.17.0", path = "./crates/nu_plugin_start", optional = true}
nu_plugin_sys = {version = "0.17.0", path = "./crates/nu_plugin_sys", optional = true}
nu_plugin_textview = {version = "0.17.0", path = "./crates/nu_plugin_textview", optional = true}
nu_plugin_to_bson = {version = "0.17.0", path = "./crates/nu_plugin_to_bson", optional = true}
nu_plugin_to_sqlite = {version = "0.17.0", path = "./crates/nu_plugin_to_sqlite", optional = true}
nu_plugin_tree = {version = "0.17.0", path = "./crates/nu_plugin_tree", optional = true}
crossterm = { version = "0.17.5", optional = true }
semver = { version = "0.10.0", optional = true }
syntect = { version = "4.2", default-features = false, features = ["default-fancy"], optional = true}
url = { version = "2.1.1", optional = true }
crossterm = {version = "0.17.5", optional = true}
semver = {version = "0.10.0", optional = true}
syntect = {version = "4.2", default-features = false, features = ["default-fancy"], optional = true}
url = {version = "2.1.1", optional = true}
clap = "2.33.1"
ctrlc = "3.1.4"
dunce = "1.0.1"
futures = { version = "0.3", features = ["compat", "io-compat"] }
futures = {version = "0.3", features = ["compat", "io-compat"]}
log = "0.4.8"
pretty_env_logger = "0.4.0"
starship = "0.43.0"
[dev-dependencies]
nu-test-support = { version = "0.16.0", path = "./crates/nu-test-support" }
nu-test-support = {version = "0.17.0", path = "./crates/nu-test-support"}
[build-dependencies]
nu-build = {version = "0.17.0", path = "./crates/nu-build"}
serde = {version = "1.0.110", features = ["derive"]}
toml = "0.5.6"
serde = { version = "1.0.110", features = ["derive"] }
nu-build = { version = "0.16.0", path = "./crates/nu-build" }
[features]
default = ["sys", "ps", "textview", "inc"]
stable = ["default", "binaryview", "match", "tree", "post", "fetch", "clipboard-cli", "trash-support", "start"]
default = [
"sys",
"ps",
"textview",
"inc",
"git-support",
"directories-support",
"ctrlc-support",
"which-support",
"ptree-support",
"term-support",
"uuid-support",
]
stable = ["default", "binaryview", "match", "tree", "post", "fetch", "clipboard-cli", "trash-support", "start", "starship-prompt", "bson", "sqlite"]
# Default
textview = ["crossterm", "syntect", "url", "nu_plugin_textview"]
sys = ["nu_plugin_sys"]
ps = ["nu_plugin_ps"]
inc = ["semver", "nu_plugin_inc"]
ps = ["nu_plugin_ps"]
sys = ["nu_plugin_sys"]
textview = ["crossterm", "syntect", "url", "nu_plugin_textview"]
# Stable
binaryview = ["nu_plugin_binaryview"]
bson = ["nu_plugin_from_bson", "nu_plugin_to_bson"]
fetch = ["nu_plugin_fetch"]
match = ["nu_plugin_match"]
post = ["nu_plugin_post"]
sqlite = ["nu_plugin_from_sqlite", "nu_plugin_to_sqlite"]
start = ["nu_plugin_start"]
trace = ["nu-parser/trace"]
tree = ["nu_plugin_tree"]
start = ["nu_plugin_start"]
clipboard-cli = ["nu-cli/clipboard-cli"]
# starship-prompt = ["nu-cli/starship-prompt"]
ctrlc-support = ["nu-cli/ctrlc"]
directories-support = ["nu-cli/directories", "nu-cli/dirs"]
git-support = ["nu-cli/git2"]
ptree-support = ["nu-cli/ptree"]
starship-prompt = ["nu-cli/starship-prompt"]
term-support = ["nu-cli/term"]
trash-support = ["nu-cli/trash-support"]
uuid-support = ["nu-cli/uuid_crate"]
which-support = ["nu-cli/ichwh", "nu-cli/which"]
# Core plugins that ship with `cargo install nu` by default
# Currently, Cargo limits us to installing only one binary

View File

@ -5,6 +5,7 @@
[![Build Status](https://dev.azure.com/nushell/nushell/_apis/build/status/nushell.nushell?branchName=master)](https://dev.azure.com/nushell/nushell/_build/latest?definitionId=2&branchName=master)
[![Discord](https://img.shields.io/discord/601130461678272522.svg?logo=discord)](https://discord.gg/NtAbbGn)
[![The Changelog #363](https://img.shields.io/badge/The%20Changelog-%23363-61c192.svg)](https://changelog.com/podcast/363)
[![@nu_shell](https://img.shields.io/badge/twitter-@nu_shell-1DA1F3?style=flat-square)](https://twitter.com/nu_shell)
## Nu Shell

View File

@ -1,16 +1,16 @@
[package]
name = "nu-build"
version = "0.16.0"
authors = ["The Nu Project Contributors"]
edition = "2018"
description = "Core build system for nushell"
edition = "2018"
license = "MIT"
name = "nu-build"
version = "0.17.0"
[lib]
doctest = false
[dependencies]
serde = { version = "1.0.114", features = ["derive"] }
lazy_static = "1.4.0"
serde = {version = "1.0.114", features = ["derive"]}
serde_json = "1.0.55"
toml = "0.5.6"

View File

@ -1,115 +1,122 @@
[package]
name = "nu-cli"
version = "0.16.0"
authors = ["The Nu Project Contributors"]
description = "CLI for nushell"
edition = "2018"
license = "MIT"
name = "nu-cli"
version = "0.17.0"
[lib]
doctest = false
[dependencies]
nu-source = { version = "0.16.0", path = "../nu-source" }
nu-plugin = { version = "0.16.0", path = "../nu-plugin" }
nu-protocol = { version = "0.16.0", path = "../nu-protocol" }
nu-errors = { version = "0.16.0", path = "../nu-errors" }
nu-parser = { version = "0.16.0", path = "../nu-parser" }
nu-value-ext = { version = "0.16.0", path = "../nu-value-ext" }
nu-test-support = { version = "0.16.0", path = "../nu-test-support" }
nu-table = {version = "0.16.0", path = "../nu-table"}
nu-errors = {version = "0.17.0", path = "../nu-errors"}
nu-parser = {version = "0.17.0", path = "../nu-parser"}
nu-plugin = {version = "0.17.0", path = "../nu-plugin"}
nu-protocol = {version = "0.17.0", path = "../nu-protocol"}
nu-source = {version = "0.17.0", path = "../nu-source"}
nu-table = {version = "0.17.0", path = "../nu-table"}
nu-test-support = {version = "0.17.0", path = "../nu-test-support"}
nu-value-ext = {version = "0.17.0", path = "../nu-value-ext"}
ansi_term = "0.12.1"
app_dirs = "1.2.1"
async-recursion = "0.3.1"
async-trait = "0.1.36"
directories = "2.0.2"
base64 = "0.12.3"
bigdecimal = { version = "0.1.2", features = ["serde"] }
bson = { version = "0.14.1", features = ["decimal128"] }
bigdecimal = {version = "0.1.2", features = ["serde"]}
byte-unit = "3.1.3"
bytes = "0.5.5"
calamine = "0.16"
cfg-if = "0.1"
chrono = { version = "0.4.11", features = ["serde"] }
chrono = {version = "0.4.11", features = ["serde"]}
clap = "2.33.1"
codespan-reporting = "0.9.5"
csv = "1.1"
ctrlc = "3.1.4"
ctrlc = {version = "3.1.4", optional = true}
derive-new = "0.5.8"
dirs = "2.0.2"
directories = {version = "2.0.2", optional = true}
dirs = {version = "2.0.2", optional = true}
dunce = "1.0.1"
eml-parser = "0.1.0"
filesize = "0.2.0"
futures = { version = "0.3", features = ["compat", "io-compat"] }
futures = {version = "0.3", features = ["compat", "io-compat"]}
futures-util = "0.3.5"
futures_codec = "0.4"
getset = "0.1.1"
git2 = { version = "0.13.6", default_features = false }
git2 = {version = "0.13.6", default_features = false, optional = true}
glob = "0.3.0"
hex = "0.4"
htmlescape = "0.3.1"
ical = "0.6.*"
ichwh = "0.3.4"
indexmap = { version = "1.4.0", features = ["serde-1"] }
ichwh = {version = "0.3.4", optional = true}
indexmap = {version = "1.4.0", features = ["serde-1"]}
itertools = "0.9.0"
codespan-reporting = "0.9.5"
log = "0.4.8"
meval = "0.2"
natural = "0.5.0"
num-bigint = { version = "0.2.6", features = ["serde"] }
num-bigint = {version = "0.2.6", features = ["serde"]}
num-format = {version = "0.4", features = ["with-num-bigint"]}
num-traits = "0.2.11"
parking_lot = "0.11.0"
pin-utils = "0.1.0"
pretty-hex = "0.1.1"
pretty_env_logger = "0.4.0"
ptree = {version = "0.2" }
ptree = {version = "0.2", optional = true}
query_interface = "0.3.5"
rand = "0.7"
regex = "1"
roxmltree = "0.13.0"
rustyline = "6.2.0"
serde = { version = "1.0.114", features = ["derive"] }
serde = {version = "1.0.114", features = ["derive"]}
serde-hjson = "0.9.1"
serde_bytes = "0.11.5"
serde_ini = "0.2.0"
serde_json = "1.0.55"
serde_urlencoded = "0.6.1"
serde_yaml = "0.8"
sha2 = "0.9.1"
shellexpand = "2.0.0"
strip-ansi-escapes = "0.1.0"
tempfile = "3.1.0"
term = "0.5.2"
termcolor = "1.1.0"
term = {version = "0.5.2", optional = true}
term_size = "0.3.2"
termcolor = "1.1.0"
toml = "0.5.6"
typetag = "0.1.5"
umask = "1.0.0"
unicode-xid = "0.2.1"
uuid_crate = { package = "uuid", version = "0.8.1", features = ["v4"] }
which = "4.0.1"
uuid_crate = {package = "uuid", version = "0.8.1", features = ["v4"], optional = true}
which = {version = "4.0.1", optional = true}
trash = { version = "1.0.1", optional = true }
clipboard = { version = "0.5", optional = true }
starship = "0.43.0"
rayon = "1.3.1"
clipboard = {version = "0.5", optional = true}
encoding_rs = "0.8.23"
rayon = "1.3.1"
starship = {version = "0.43.0", optional = true}
trash = {version = "1.0.1", optional = true}
[target.'cfg(unix)'.dependencies]
users = "0.10.0"
# TODO this will be possible with new dependency resolver
# (currently on nightly behind -Zfeatures=itarget):
# https://github.com/rust-lang/cargo/issues/7914
#[target.'cfg(not(windows))'.dependencies]
#num-format = {version = "0.4", features = ["with-system-locale"]}
[dependencies.rusqlite]
version = "0.23.1"
features = ["bundled", "blob"]
optional = true
version = "0.23.1"
[build-dependencies]
nu-build = { version = "0.16.0", path = "../nu-build" }
nu-build = {version = "0.17.0", path = "../nu-build"}
[dev-dependencies]
quickcheck = "0.9"
quickcheck_macros = "0.9"
[features]
stable = []
# starship-prompt = ["starship"]
clipboard-cli = ["clipboard"]
stable = []
starship-prompt = ["starship"]
trash-support = ["trash"]

View File

@ -1,5 +1,5 @@
use crate::commands::classified::block::run_block;
use crate::commands::classified::external::{MaybeTextCodec, StringOrBinary};
use crate::commands::classified::maybe_text_codec::{MaybeTextCodec, StringOrBinary};
use crate::commands::plugin::JsonRpc;
use crate::commands::plugin::{PluginCommand, PluginSink};
use crate::commands::whole_stream_command;
@ -12,6 +12,8 @@ use futures_codec::FramedRead;
use nu_errors::{ProximateShellError, ShellDiagnostic, ShellError};
use nu_protocol::hir::{ClassifiedCommand, Expression, InternalCommand, Literal, NamedArguments};
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
#[allow(unused)]
use nu_source::Tagged;
use log::{debug, trace};
use rustyline::error::ReadlineError;
@ -149,10 +151,7 @@ pub fn load_plugins(context: &mut Context) -> Result<(), ShellError> {
.map(|path| {
let bin_name = {
if let Some(name) = path.file_name() {
match name.to_str() {
Some(raw) => raw,
None => "",
}
name.to_str().unwrap_or("")
} else {
""
}
@ -251,6 +250,13 @@ pub fn create_default_context(
whole_stream_command(Remove),
whole_stream_command(Open),
whole_stream_command(Config),
whole_stream_command(ConfigGet),
whole_stream_command(ConfigSet),
whole_stream_command(ConfigSetInto),
whole_stream_command(ConfigClear),
whole_stream_command(ConfigLoad),
whole_stream_command(ConfigRemove),
whole_stream_command(ConfigPath),
whole_stream_command(Help),
whole_stream_command(History),
whole_stream_command(Save),
@ -258,9 +264,8 @@ pub fn create_default_context(
whole_stream_command(Cpy),
whole_stream_command(Date),
whole_stream_command(Cal),
whole_stream_command(Calc),
whole_stream_command(Mkdir),
whole_stream_command(Move),
whole_stream_command(Mv),
whole_stream_command(Kill),
whole_stream_command(Version),
whole_stream_command(Clear),
@ -273,6 +278,7 @@ pub fn create_default_context(
// Statistics
whole_stream_command(Size),
whole_stream_command(Count),
whole_stream_command(Benchmark),
// Metadata
whole_stream_command(Tags),
// Shells
@ -288,6 +294,7 @@ pub fn create_default_context(
whole_stream_command(Split),
whole_stream_command(SplitColumn),
whole_stream_command(SplitRow),
whole_stream_command(SplitChars),
whole_stream_command(Lines),
whole_stream_command(Trim),
whole_stream_command(Echo),
@ -299,15 +306,19 @@ pub fn create_default_context(
whole_stream_command(StrUpcase),
whole_stream_command(StrCapitalize),
whole_stream_command(StrFindReplace),
whole_stream_command(StrFrom),
whole_stream_command(StrSubstring),
whole_stream_command(StrSet),
whole_stream_command(StrToDatetime),
whole_stream_command(StrTrim),
whole_stream_command(StrCollect),
whole_stream_command(StrLength),
whole_stream_command(StrReverse),
whole_stream_command(BuildString),
whole_stream_command(Ansi),
whole_stream_command(Char),
// Column manipulation
whole_stream_command(MoveColumn),
whole_stream_command(Reject),
whole_stream_command(Select),
whole_stream_command(Get),
@ -328,6 +339,7 @@ pub fn create_default_context(
whole_stream_command(Drop),
whole_stream_command(Format),
whole_stream_command(Where),
whole_stream_command(If),
whole_stream_command(Compact),
whole_stream_command(Default),
whole_stream_command(Skip),
@ -342,6 +354,7 @@ pub fn create_default_context(
whole_stream_command(Each),
whole_stream_command(IsEmpty),
// Table manipulation
whole_stream_command(Move),
whole_stream_command(Merge),
whole_stream_command(Shuffle),
whole_stream_command(Wrap),
@ -349,21 +362,24 @@ pub fn create_default_context(
whole_stream_command(Headers),
// Data processing
whole_stream_command(Histogram),
whole_stream_command(Autoenv),
whole_stream_command(AutoenvTrust),
whole_stream_command(AutoenvUnTrust),
whole_stream_command(Math),
whole_stream_command(MathAverage),
whole_stream_command(MathEval),
whole_stream_command(MathMedian),
whole_stream_command(MathMinimum),
whole_stream_command(MathMode),
whole_stream_command(MathMaximum),
whole_stream_command(MathStddev),
whole_stream_command(MathSummation),
whole_stream_command(MathVariance),
// File format output
whole_stream_command(To),
whole_stream_command(ToBSON),
whole_stream_command(ToCSV),
whole_stream_command(ToHTML),
whole_stream_command(ToJSON),
whole_stream_command(ToSQLite),
whole_stream_command(ToDB),
whole_stream_command(ToMarkdown),
whole_stream_command(ToTOML),
whole_stream_command(ToTSV),
@ -376,11 +392,8 @@ pub fn create_default_context(
whole_stream_command(FromTSV),
whole_stream_command(FromSSV),
whole_stream_command(FromINI),
whole_stream_command(FromBSON),
whole_stream_command(FromJSON),
whole_stream_command(FromODS),
whole_stream_command(FromDB),
whole_stream_command(FromSQLite),
whole_stream_command(FromTOML),
whole_stream_command(FromURL),
whole_stream_command(FromXLSX),
@ -395,20 +408,10 @@ pub fn create_default_context(
whole_stream_command(Random),
whole_stream_command(RandomBool),
whole_stream_command(RandomDice),
#[cfg(feature = "uuid_crate")]
whole_stream_command(RandomUUID),
]);
cfg_if::cfg_if! {
if #[cfg(data_processing_primitives)] {
context.add_commands(vec![
whole_stream_command(ReduceBy),
whole_stream_command(EvaluateBy),
whole_stream_command(TSortBy),
whole_stream_command(MapMaxBy),
]);
}
}
#[cfg(feature = "clipboard")]
{
context.add_commands(vec![whole_stream_command(crate::commands::clip::Clip)]);
@ -427,15 +430,18 @@ pub async fn run_vec_of_pipelines(
let _ = crate::load_plugins(&mut context);
let cc = context.ctrl_c.clone();
#[cfg(feature = "ctrlc")]
{
let cc = context.ctrl_c.clone();
ctrlc::set_handler(move || {
cc.store(true, Ordering::SeqCst);
})
.expect("Error setting Ctrl-C handler");
ctrlc::set_handler(move || {
cc.store(true, Ordering::SeqCst);
})
.expect("Error setting Ctrl-C handler");
if context.ctrl_c.load(Ordering::SeqCst) {
context.ctrl_c.store(false, Ordering::SeqCst);
if context.ctrl_c.load(Ordering::SeqCst) {
context.ctrl_c.store(false, Ordering::SeqCst);
}
}
// before we start up, let's run our startup commands
@ -540,6 +546,10 @@ pub async fn cli(
Cmd::Move(Movement::ForwardWord(1, At::AfterEnd, Word::Vi)),
);
if let Err(e) = crate::keybinding::load_keybindings(&mut rl) {
println!("Error loading keybindings: {:?}", e);
}
#[cfg(windows)]
{
let _ = ansi_term::enable_ansi_support();
@ -548,11 +558,15 @@ pub async fn cli(
// we are ok if history does not exist
let _ = rl.load_history(&History::path());
let cc = context.ctrl_c.clone();
ctrlc::set_handler(move || {
cc.store(true, Ordering::SeqCst);
})
.expect("Error setting Ctrl-C handler");
#[cfg(feature = "ctrlc")]
{
let cc = context.ctrl_c.clone();
ctrlc::set_handler(move || {
cc.store(true, Ordering::SeqCst);
})
.expect("Error setting Ctrl-C handler");
}
let mut ctrlcbreak = false;
// before we start up, let's run our startup commands
@ -631,7 +645,9 @@ pub async fn cli(
.map(|i| i.value.expect_int())
.unwrap_or(100_000);
rl.set_max_history_size(max_history_size as usize);
// rl.set_max_history_size(max_history_size as usize);
rustyline::config::Configurer::set_max_history_size(&mut rl, max_history_size as usize);
rustyline::Editor::set_max_history_size(&mut rl, max_history_size as usize);
let key_timeout = config
.get("key_timeout")
@ -653,21 +669,35 @@ pub async fn cli(
let colored_prompt = {
if use_starship {
std::env::set_var("STARSHIP_SHELL", "");
std::env::set_var("PWD", &cwd);
let mut starship_context =
starship::context::Context::new_with_dir(clap::ArgMatches::default(), cwd);
#[cfg(feature = "starship")]
{
std::env::set_var("STARSHIP_SHELL", "");
std::env::set_var("PWD", &cwd);
let mut starship_context =
starship::context::Context::new_with_dir(clap::ArgMatches::default(), cwd);
match starship_context.config.config {
None => {
starship_context.config.config = create_default_starship_config();
}
Some(toml::Value::Table(t)) if t.is_empty() => {
starship_context.config.config = create_default_starship_config();
}
_ => {}
};
starship::print::get_prompt(starship_context)
match starship_context.config.config {
None => {
starship_context.config.config = create_default_starship_config();
}
Some(toml::Value::Table(t)) if t.is_empty() => {
starship_context.config.config = create_default_starship_config();
}
_ => {}
};
starship::print::get_prompt(starship_context)
}
#[cfg(not(feature = "starship"))]
{
format!(
"\x1b[32m{}{}\x1b[m> ",
cwd,
match current_branch() {
Some(s) => format!("({})", s),
None => "".to_string(),
}
)
}
} else if let Some(prompt) = config.get("prompt") {
let prompt_line = prompt.as_string()?;
@ -782,10 +812,7 @@ pub async fn cli(
LineResult::CtrlC => {
let config_ctrlc_exit = config::config(Tag::unknown())?
.get("ctrlc_exit")
.map(|s| match s.value.expect_string() {
"true" => true,
_ => false,
})
.map(|s| s.value.expect_string() == "true")
.unwrap_or(false); // default behavior is to allow CTRL-C spamming similar to other shells
if !config_ctrlc_exit {
@ -823,6 +850,7 @@ fn chomp_newline(s: &str) -> &str {
}
}
#[derive(Debug)]
pub enum LineResult {
Success(String),
Error(String, ShellError),
@ -830,6 +858,36 @@ pub enum LineResult {
Break,
}
pub async fn parse_and_eval(line: &str, ctx: &mut Context) -> Result<String, ShellError> {
let line = if line.ends_with('\n') {
&line[..line.len() - 1]
} else {
line
};
let lite_result = nu_parser::lite_parse(&line, 0)?;
// TODO ensure the command whose examples we're testing is actually in the pipeline
let mut classified_block = nu_parser::classify_block(&lite_result, ctx.registry());
classified_block.block.expand_it_usage();
let input_stream = InputStream::empty();
let env = ctx.get_env();
run_block(
&classified_block.block,
ctx,
input_stream,
&Value::nothing(),
&IndexMap::new(),
&env,
)
.await?
.collect_string(Tag::unknown())
.await
.map(|x| x.item)
}
/// Process the line by parsing the text to turn it into commands, classify those commands so that we understand what is being called in the pipeline, and then run this pipeline
pub async fn process_line(
readline: Result<String, ReadlineError>,
@ -906,7 +964,7 @@ pub async fn process_line(
.unwrap_or(true)
&& canonicalize(ctx.shell_manager.path(), name).is_ok()
&& Path::new(&name).is_dir()
&& which::which(&name).is_err()
&& !crate::commands::classified::external::did_find_command(&name)
{
// Here we work differently if we're in Windows because of the expected Windows behavior
#[cfg(windows)]
@ -952,7 +1010,7 @@ pub async fn process_line(
let input_stream = if redirect_stdin {
let file = futures::io::AllowStdIo::new(std::io::stdin());
let stream = FramedRead::new(file, MaybeTextCodec).map(|line| {
let stream = FramedRead::new(file, MaybeTextCodec::default()).map(|line| {
if let Ok(line) = line {
match line {
StringOrBinary::String(s) => Ok(Value {

View File

@ -8,10 +8,13 @@ pub(crate) mod alias;
pub(crate) mod ansi;
pub(crate) mod append;
pub(crate) mod args;
pub(crate) mod autoenv;
pub(crate) mod autoenv_trust;
pub(crate) mod autoenv_untrust;
pub(crate) mod autoview;
pub(crate) mod benchmark;
pub(crate) mod build_string;
pub(crate) mod cal;
pub(crate) mod calc;
pub(crate) mod cd;
pub(crate) mod char_;
pub(crate) mod classified;
@ -20,6 +23,7 @@ pub(crate) mod clip;
pub(crate) mod command;
pub(crate) mod compact;
pub(crate) mod config;
pub(crate) mod constants;
pub(crate) mod count;
pub(crate) mod cp;
pub(crate) mod date;
@ -31,21 +35,17 @@ pub(crate) mod du;
pub(crate) mod each;
pub(crate) mod echo;
pub(crate) mod enter;
#[allow(unused)]
pub(crate) mod evaluate_by;
pub(crate) mod every;
pub(crate) mod exit;
pub(crate) mod first;
pub(crate) mod format;
pub(crate) mod from;
pub(crate) mod from_bson;
pub(crate) mod from_csv;
pub(crate) mod from_eml;
pub(crate) mod from_ics;
pub(crate) mod from_ini;
pub(crate) mod from_json;
pub(crate) mod from_ods;
pub(crate) mod from_sqlite;
pub(crate) mod from_ssv;
pub(crate) mod from_toml;
pub(crate) mod from_tsv;
@ -61,20 +61,17 @@ pub(crate) mod headers;
pub(crate) mod help;
pub(crate) mod histogram;
pub(crate) mod history;
pub(crate) mod if_;
pub(crate) mod insert;
pub(crate) mod is_empty;
pub(crate) mod keep;
pub(crate) mod keep_until;
pub(crate) mod keep_while;
pub(crate) mod last;
pub(crate) mod lines;
pub(crate) mod ls;
#[allow(unused)]
pub(crate) mod map_max_by;
pub(crate) mod math;
pub(crate) mod merge;
pub(crate) mod mkdir;
pub(crate) mod mv;
pub(crate) mod move_;
pub(crate) mod next;
pub(crate) mod nth;
pub(crate) mod open;
@ -86,8 +83,6 @@ pub(crate) mod prev;
pub(crate) mod pwd;
pub(crate) mod random;
pub(crate) mod range;
#[allow(unused)]
pub(crate) mod reduce_by;
pub(crate) mod reject;
pub(crate) mod rename;
pub(crate) mod reverse;
@ -100,23 +95,17 @@ pub(crate) mod shells;
pub(crate) mod shuffle;
pub(crate) mod size;
pub(crate) mod skip;
pub(crate) mod skip_until;
pub(crate) mod skip_while;
pub(crate) mod sort_by;
pub(crate) mod split;
pub(crate) mod split_by;
pub(crate) mod str_;
#[allow(unused)]
pub(crate) mod t_sort_by;
pub(crate) mod table;
pub(crate) mod tags;
pub(crate) mod to;
pub(crate) mod to_bson;
pub(crate) mod to_csv;
pub(crate) mod to_html;
pub(crate) mod to_json;
pub(crate) mod to_md;
pub(crate) mod to_sqlite;
pub(crate) mod to_toml;
pub(crate) mod to_tsv;
pub(crate) mod to_url;
@ -140,12 +129,17 @@ pub(crate) use command::{
pub(crate) use alias::Alias;
pub(crate) use ansi::Ansi;
pub(crate) use append::Append;
pub(crate) use autoenv::Autoenv;
pub(crate) use autoenv_trust::AutoenvTrust;
pub(crate) use autoenv_untrust::AutoenvUnTrust;
pub(crate) use benchmark::Benchmark;
pub(crate) use build_string::BuildString;
pub(crate) use cal::Cal;
pub(crate) use calc::Calc;
pub(crate) use char_::Char;
pub(crate) use compact::Compact;
pub(crate) use config::Config;
pub(crate) use config::{
Config, ConfigClear, ConfigGet, ConfigLoad, ConfigPath, ConfigRemove, ConfigSet, ConfigSetInto,
};
pub(crate) use count::Count;
pub(crate) use cp::Cpy;
pub(crate) use date::Date;
@ -156,6 +150,7 @@ pub(crate) use drop::Drop;
pub(crate) use du::Du;
pub(crate) use each::Each;
pub(crate) use echo::Echo;
pub(crate) use if_::If;
pub(crate) use is_empty::IsEmpty;
pub(crate) use update::Update;
pub(crate) mod kill;
@ -164,22 +159,17 @@ pub(crate) mod clear;
pub(crate) use clear::Clear;
pub(crate) mod touch;
pub(crate) use enter::Enter;
#[allow(unused_imports)]
pub(crate) use evaluate_by::EvaluateBy;
pub(crate) use every::Every;
pub(crate) use exit::Exit;
pub(crate) use first::First;
pub(crate) use format::Format;
pub(crate) use from::From;
pub(crate) use from_bson::FromBSON;
pub(crate) use from_csv::FromCSV;
pub(crate) use from_eml::FromEML;
pub(crate) use from_ics::FromIcs;
pub(crate) use from_ini::FromINI;
pub(crate) use from_json::FromJSON;
pub(crate) use from_ods::FromODS;
pub(crate) use from_sqlite::FromDB;
pub(crate) use from_sqlite::FromSQLite;
pub(crate) use from_ssv::FromSSV;
pub(crate) use from_toml::FromTOML;
pub(crate) use from_tsv::FromTSV;
@ -197,20 +187,17 @@ pub(crate) use help::Help;
pub(crate) use histogram::Histogram;
pub(crate) use history::History;
pub(crate) use insert::Insert;
pub(crate) use keep::Keep;
pub(crate) use keep_until::KeepUntil;
pub(crate) use keep_while::KeepWhile;
pub(crate) use keep::{Keep, KeepUntil, KeepWhile};
pub(crate) use last::Last;
pub(crate) use lines::Lines;
pub(crate) use ls::Ls;
#[allow(unused_imports)]
pub(crate) use map_max_by::MapMaxBy;
pub(crate) use math::{
Math, MathAverage, MathMaximum, MathMedian, MathMinimum, MathMode, MathSummation,
Math, MathAverage, MathEval, MathMaximum, MathMedian, MathMinimum, MathMode, MathStddev,
MathSummation, MathVariance,
};
pub(crate) use merge::Merge;
pub(crate) use mkdir::Mkdir;
pub(crate) use mv::Move;
pub(crate) use move_::{Move, MoveColumn, Mv};
pub(crate) use next::Next;
pub(crate) use nth::Nth;
pub(crate) use open::Open;
@ -219,10 +206,10 @@ pub(crate) use pivot::Pivot;
pub(crate) use prepend::Prepend;
pub(crate) use prev::Previous;
pub(crate) use pwd::Pwd;
pub(crate) use random::{Random, RandomBool, RandomDice, RandomUUID};
#[cfg(feature = "uuid_crate")]
pub(crate) use random::RandomUUID;
pub(crate) use random::{Random, RandomBool, RandomDice};
pub(crate) use range::Range;
#[allow(unused_imports)]
pub(crate) use reduce_by::ReduceBy;
pub(crate) use reject::Reject;
pub(crate) use rename::Rename;
pub(crate) use reverse::Reverse;
@ -233,30 +220,21 @@ pub(crate) use select::Select;
pub(crate) use shells::Shells;
pub(crate) use shuffle::Shuffle;
pub(crate) use size::Size;
pub(crate) use skip::Skip;
pub(crate) use skip_until::SkipUntil;
pub(crate) use skip_while::SkipWhile;
pub(crate) use skip::{Skip, SkipUntil, SkipWhile};
pub(crate) use sort_by::SortBy;
pub(crate) use split::Split;
pub(crate) use split::SplitColumn;
pub(crate) use split::SplitRow;
pub(crate) use split::{Split, SplitChars, SplitColumn, SplitRow};
pub(crate) use split_by::SplitBy;
pub(crate) use str_::{
Str, StrCapitalize, StrCollect, StrDowncase, StrFindReplace, StrSet, StrSubstring,
StrToDatetime, StrToDecimal, StrToInteger, StrTrim, StrUpcase,
Str, StrCapitalize, StrCollect, StrDowncase, StrFindReplace, StrFrom, StrLength, StrReverse,
StrSet, StrSubstring, StrToDatetime, StrToDecimal, StrToInteger, StrTrim, StrUpcase,
};
#[allow(unused_imports)]
pub(crate) use t_sort_by::TSortBy;
pub(crate) use table::Table;
pub(crate) use tags::Tags;
pub(crate) use to::To;
pub(crate) use to_bson::ToBSON;
pub(crate) use to_csv::ToCSV;
pub(crate) use to_html::ToHTML;
pub(crate) use to_json::ToJSON;
pub(crate) use to_md::ToMarkdown;
pub(crate) use to_sqlite::ToDB;
pub(crate) use to_sqlite::ToSQLite;
pub(crate) use to_toml::ToTOML;
pub(crate) use to_tsv::ToTSV;
pub(crate) use to_url::ToURL;

View File

@ -40,6 +40,14 @@ impl WholeStreamCommand for Ansi {
example: r#"ansi reset"#,
result: Some(vec![Value::from("\u{1b}[0m")]),
},
Example {
description:
"Use ansi to color text (rb = red bold, gb = green bold, pb = purple bold)",
example: r#"echo [$(ansi rb) Hello " " $(ansi gb) Nu " " $(ansi pb) World] | str collect"#,
result: Some(vec![Value::from(
"\u{1b}[1;31mHello \u{1b}[1;32mNu \u{1b}[1;35mWorld",
)]),
},
]
}

View File

@ -0,0 +1,92 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
use serde::Deserialize;
use serde::Serialize;
use sha2::{Digest, Sha256};
use std::io::Read;
use std::path::PathBuf;
pub struct Autoenv;
#[derive(Deserialize, Serialize, Debug, Default)]
pub struct Trusted {
pub files: IndexMap<String, Vec<u8>>,
}
impl Trusted {
pub fn new() -> Self {
Trusted {
files: IndexMap::new(),
}
}
}
pub fn file_is_trusted(nu_env_file: &PathBuf, content: &[u8]) -> Result<bool, ShellError> {
let contentdigest = Sha256::digest(&content).as_slice().to_vec();
let nufile = std::fs::canonicalize(nu_env_file)?;
let trusted = read_trusted()?;
Ok(trusted.files.get(&nufile.to_string_lossy().to_string()) == Some(&contentdigest))
}
pub fn read_trusted() -> Result<Trusted, ShellError> {
let config_path = config::default_path_for(&Some(PathBuf::from("nu-env.toml")))?;
let mut file = std::fs::OpenOptions::new()
.read(true)
.create(true)
.write(true)
.open(config_path)
.map_err(|_| ShellError::untagged_runtime_error("Couldn't open nu-env.toml"))?;
let mut doc = String::new();
file.read_to_string(&mut doc)?;
let allowed = toml::de::from_str(doc.as_str()).unwrap_or_else(|_| Trusted::new());
Ok(allowed)
}
#[async_trait]
impl WholeStreamCommand for Autoenv {
fn name(&self) -> &str {
"autoenv"
}
fn usage(&self) -> &str {
// "Mark a .nu-env file in a directory as trusted. Needs to be re-run after each change to the file or its filepath."
r#"Manage directory specific environment variables and scripts. Create a file called .nu-env in any directory and run 'autoenv trust' to let nushell read it when entering the directory.
The file can contain several optional sections:
env: environment variables to set when visiting the directory. The variables are unset after leaving the directory and any overwritten values are restored.
scriptvars: environment variables that should be set to the return value of a script. After they have been set, they behave in the same way as variables set in the env section.
scripts: scripts to run when entering the directory or leaving it. Note that exitscripts are not run in the directory they are declared in."#
}
fn signature(&self) -> Signature {
Signature::build("autoenv")
}
async fn run(
&self,
_args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
Ok(OutputStream::one(ReturnSuccess::value(
UntaggedValue::string(crate::commands::help::get_help(&Autoenv, &registry))
.into_value(Tag::unknown()),
)))
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Example .nu-env file",
example: r#"cat .nu-env
[env]
mykey = "myvalue"
[scriptvars]
myscript = "echo myval"
[scripts]
entryscripts = ["touch hello.txt", "touch hello2.txt"]
exitscripts = ["touch bye.txt"]"#,
result: None,
}]
}
}

View File

@ -0,0 +1,83 @@
use super::autoenv::read_trusted;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::SyntaxShape;
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
use sha2::{Digest, Sha256};
use std::{fs, path::PathBuf};
pub struct AutoenvTrust;
#[async_trait]
impl WholeStreamCommand for AutoenvTrust {
fn name(&self) -> &str {
"autoenv trust"
}
fn signature(&self) -> Signature {
Signature::build("autoenv trust").optional("dir", SyntaxShape::String, "Directory to allow")
}
fn usage(&self) -> &str {
"Trust a .nu-env file in the current or given directory"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let file_to_trust = match args.call_info.evaluate(registry).await?.args.nth(0) {
Some(Value {
value: UntaggedValue::Primitive(Primitive::String(ref path)),
tag: _,
}) => {
let mut dir = fs::canonicalize(path)?;
dir.push(".nu-env");
dir
}
_ => {
let mut dir = fs::canonicalize(std::env::current_dir()?)?;
dir.push(".nu-env");
dir
}
};
let content = std::fs::read(&file_to_trust)?;
let filename = file_to_trust.to_string_lossy().to_string();
let mut allowed = read_trusted()?;
allowed
.files
.insert(filename, Sha256::digest(&content).as_slice().to_vec());
let config_path = config::default_path_for(&Some(PathBuf::from("nu-env.toml")))?;
let tomlstr = toml::to_string(&allowed).map_err(|_| {
ShellError::untagged_runtime_error("Couldn't serialize allowed dirs to nu-env.toml")
})?;
fs::write(config_path, tomlstr).expect("Couldn't write to toml file");
Ok(OutputStream::one(ReturnSuccess::value(
UntaggedValue::string(".nu-env trusted!").into_value(tag),
)))
}
fn is_binary(&self) -> bool {
false
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Allow .nu-env file in current directory",
example: "autoenv trust",
result: None,
},
Example {
description: "Allow .nu-env file in directory foo",
example: "autoenv trust foo",
result: None,
},
]
}
}

View File

@ -0,0 +1,107 @@
use super::autoenv::Trusted;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::SyntaxShape;
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
use std::io::Read;
use std::{fs, path::PathBuf};
pub struct AutoenvUnTrust;
#[async_trait]
impl WholeStreamCommand for AutoenvUnTrust {
fn name(&self) -> &str {
"autoenv untrust"
}
fn signature(&self) -> Signature {
Signature::build("autoenv untrust").optional(
"dir",
SyntaxShape::String,
"Directory to disallow",
)
}
fn usage(&self) -> &str {
"Untrust a .nu-env file in the current or given directory"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let tag = args.call_info.name_tag.clone();
let file_to_untrust = match args.call_info.evaluate(registry).await?.args.nth(0) {
Some(Value {
value: UntaggedValue::Primitive(Primitive::String(ref path)),
tag: _,
}) => {
let mut dir = fs::canonicalize(path)?;
dir.push(".nu-env");
dir
}
_ => {
let mut dir = std::env::current_dir()?;
dir.push(".nu-env");
dir
}
};
let config_path = config::default_path_for(&Some(PathBuf::from("nu-env.toml")))?;
let mut file = match std::fs::OpenOptions::new()
.read(true)
.create(true)
.write(true)
.open(config_path.clone())
{
Ok(p) => p,
Err(_) => {
return Err(ShellError::untagged_runtime_error(
"Couldn't open nu-env.toml",
));
}
};
let mut doc = String::new();
file.read_to_string(&mut doc)?;
let mut allowed: Trusted = toml::from_str(doc.as_str()).unwrap_or_else(|_| Trusted::new());
let file_to_untrust = file_to_untrust.to_string_lossy().to_string();
if allowed.files.remove(&file_to_untrust).is_none() {
return
Err(ShellError::untagged_runtime_error(
"No .nu-env file to untrust in the given directory. Is it missing, or already untrusted?",
));
}
let tomlstr = toml::to_string(&allowed).map_err(|_| {
ShellError::untagged_runtime_error("Couldn't serialize allowed dirs to nu-env.toml")
})?;
fs::write(config_path, tomlstr).expect("Couldn't write to toml file");
Ok(OutputStream::one(ReturnSuccess::value(
UntaggedValue::string(".nu-env untrusted!").into_value(tag),
)))
}
fn is_binary(&self) -> bool {
false
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Disallow .nu-env file in current directory",
example: "autoenv untrust",
result: None,
},
Example {
description: "Disallow .nu-env file in directory foo",
example: "autoenv untrust foo",
result: None,
},
]
}
}

View File

@ -0,0 +1,80 @@
use crate::commands::classified::block::run_block;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{
hir::Block, Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
};
use chrono::prelude::*;
pub struct Benchmark;
#[derive(Deserialize, Debug)]
struct BenchmarkArgs {
block: Block,
}
#[async_trait]
impl WholeStreamCommand for Benchmark {
fn name(&self) -> &str {
"benchmark"
}
fn signature(&self) -> Signature {
Signature::build("benchmark").required(
"block",
SyntaxShape::Block,
"the block to run and benchmark",
)
}
fn usage(&self) -> &str {
"Runs a block and return the time it took to do execute it. Eg) benchmark { echo $nu.env.NAME }"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
benchmark(args, registry).await
}
}
async fn benchmark(
raw_args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let mut context = Context::from_raw(&raw_args, &registry);
let scope = raw_args.call_info.scope.clone();
let (BenchmarkArgs { block }, input) = raw_args.process(&registry).await?;
let start_time: chrono::DateTime<_> = Utc::now();
let result = run_block(
&block,
&mut context,
input,
&scope.it,
&scope.vars,
&scope.env,
)
.await;
let output = match result {
Ok(mut stream) => {
let _ = stream.drain_vec().await;
let run_duration: chrono::Duration = Utc::now().signed_duration_since(start_time);
context.clear_errors();
Ok(ReturnSuccess::Value(Value {
value: UntaggedValue::Primitive(Primitive::from(run_duration)),
tag: Tag::from(block.span),
}))
}
Err(e) => Err(e),
};
Ok(OutputStream::from(vec![output]))
}

View File

@ -1,88 +0,0 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, UntaggedValue, Value};
pub struct Calc;
#[async_trait]
impl WholeStreamCommand for Calc {
fn name(&self) -> &str {
"calc"
}
fn usage(&self) -> &str {
"Parse a math expression into a number"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
calc(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Calculate math in the pipeline",
example: "echo '10 / 4' | calc",
result: Some(vec![UntaggedValue::decimal(2.5).into()]),
}]
}
}
pub async fn calc(
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let input = args.input;
let name = args.call_info.name_tag.span;
Ok(input
.map(move |input| {
if let Ok(string) = input.as_string() {
match parse(&string, &input.tag) {
Ok(value) => ReturnSuccess::value(value),
Err(err) => Err(ShellError::labeled_error(
"Calculation error",
err,
&input.tag.span,
)),
}
} else {
Err(ShellError::labeled_error(
"Expected a string from pipeline",
"requires string input",
name,
))
}
})
.to_output_stream())
}
pub fn parse(math_expression: &str, tag: impl Into<Tag>) -> Result<Value, String> {
use std::f64;
let num = meval::eval_str(math_expression);
match num {
Ok(num) => {
if num == f64::INFINITY || num == f64::NEG_INFINITY {
return Err(String::from("cannot represent result"));
}
Ok(UntaggedValue::from(Primitive::from(num)).into_value(tag))
}
Err(error) => Err(error.to_string()),
}
}
#[cfg(test)]
mod tests {
use super::Calc;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(Calc {})
}
}

View File

@ -30,11 +30,22 @@ impl WholeStreamCommand for Char {
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Output newline",
example: r#"char newline"#,
result: Some(vec![Value::from("\n")]),
}]
vec![
Example {
description: "Output newline",
example: r#"char newline"#,
result: Some(vec![Value::from("\n")]),
},
Example {
description: "Output prompt character, newline and a hamburger character",
example: r#"echo $(char prompt) $(char newline) $(char hamburger)"#,
result: Some(vec![
UntaggedValue::string("\u{25b6}").into(),
UntaggedValue::string("\n").into(),
UntaggedValue::string("\u{2261}").into(),
]),
},
]
}
async fn run(
@ -64,6 +75,25 @@ fn str_to_character(s: &str) -> Option<String> {
match s {
"newline" | "enter" | "nl" => Some("\n".into()),
"tab" => Some("\t".into()),
"sp" | "space" => Some(" ".into()),
// Unicode names came from https://www.compart.com/en/unicode
// Private Use Area (U+E000-U+F8FF)
"branch" => Some('\u{e0a0}'.to_string()), // 
"segment" => Some('\u{e0b0}'.to_string()), // 
"identical_to" | "hamburger" => Some('\u{2261}'.to_string()), // ≡
"not_identical_to" | "branch_untracked" => Some('\u{2262}'.to_string()), // ≢
"strictly_equivalent_to" | "branch_identical" => Some('\u{2263}'.to_string()), // ≣
"upwards_arrow" | "branch_ahead" => Some('\u{2191}'.to_string()), // ↑
"downwards_arrow" | "branch_behind" => Some('\u{2193}'.to_string()), // ↓
"up_down_arrow" | "branch_ahead_behind" => Some('\u{2195}'.to_string()), // ↕
"black_right_pointing_triangle" | "prompt" => Some('\u{25b6}'.to_string()), // ▶
"vector_or_cross_product" | "failed" => Some('\u{2a2f}'.to_string()), //
"high_voltage_sign" | "elevated" => Some('\u{26a1}'.to_string()), // ⚡
"tilde" | "twiddle" | "squiggly" | "home" => Some("~".into()), // ~
"hash" | "hashtag" | "pound_sign" | "sharp" | "root" => Some("#".into()), // #
_ => None,
}
}

View File

@ -1,3 +1,4 @@
use crate::commands::classified::maybe_text_codec::{MaybeTextCodec, StringOrBinary};
use crate::evaluate::evaluate_baseline_expr;
use crate::futures::ThreadedReceiver;
use crate::prelude::*;
@ -7,9 +8,7 @@ use std::ops::Deref;
use std::process::{Command, Stdio};
use std::sync::mpsc;
use bytes::{BufMut, Bytes, BytesMut};
use futures::executor::block_on_stream;
// use futures::stream::StreamExt;
use futures_codec::FramedRead;
use log::trace;
@ -18,70 +17,6 @@ use nu_protocol::hir::ExternalCommand;
use nu_protocol::{Primitive, Scope, ShellTypeName, UntaggedValue, Value};
use nu_source::Tag;
pub enum StringOrBinary {
String(String),
Binary(Vec<u8>),
}
pub struct MaybeTextCodec;
impl futures_codec::Encoder for MaybeTextCodec {
type Item = StringOrBinary;
type Error = std::io::Error;
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
match item {
StringOrBinary::String(s) => {
dst.reserve(s.len());
dst.put(s.as_bytes());
Ok(())
}
StringOrBinary::Binary(b) => {
dst.reserve(b.len());
dst.put(Bytes::from(b));
Ok(())
}
}
}
}
impl futures_codec::Decoder for MaybeTextCodec {
type Item = StringOrBinary;
type Error = std::io::Error;
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
let v: Vec<u8> = src.to_vec();
match String::from_utf8(v) {
Ok(s) => {
src.clear();
if s.is_empty() {
Ok(None)
} else {
Ok(Some(StringOrBinary::String(s)))
}
}
Err(err) => {
// Note: the longest UTF-8 character per Unicode spec is currently 6 bytes. If we fail somewhere earlier than the last 6 bytes,
// we know that we're failing to understand the string encoding and not just seeing a partial character. When this happens, let's
// fall back to assuming it's a binary buffer.
if src.is_empty() {
Ok(None)
} else if src.len() > 6 && (src.len() - err.utf8_error().valid_up_to() > 6) {
// Fall back to assuming binary
let buf = src.to_vec();
src.clear();
Ok(Some(StringOrBinary::Binary(buf)))
} else {
// Looks like a utf-8 string, so let's assume that
let buf = src.split_to(err.utf8_error().valid_up_to() + 1);
String::from_utf8(buf.to_vec())
.map(|x| Some(StringOrBinary::String(x)))
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
}
}
}
}
}
pub(crate) async fn run_external_command(
command: ExternalCommand,
context: &mut Context,
@ -91,7 +26,7 @@ pub(crate) async fn run_external_command(
) -> Result<InputStream, ShellError> {
trace!(target: "nu::run::external", "-> {}", command.name);
if !did_find_command(&command.name).await {
if !did_find_command(&command.name) {
return Err(ShellError::labeled_error(
"Command not found",
"command not found",
@ -118,35 +53,35 @@ async fn run_with_stdin(
let value =
evaluate_baseline_expr(arg, &context.registry, &scope.it, &scope.vars, &scope.env)
.await?;
// Skip any arguments that don't really exist, treating them as optional
// FIXME: we may want to preserve the gap in the future, though it's hard to say
// what value we would put in its place.
if value.value.is_none() {
continue;
}
// Do the cleanup that we need to do on any argument going out:
let trimmed_value_string = value.as_string()?.trim_end_matches('\n').to_string();
let value_string;
#[cfg(not(windows))]
{
value_string = trimmed_value_string
.replace('$', "\\$")
.replace('"', "\\\"")
.to_string()
}
#[cfg(windows)]
{
value_string = trimmed_value_string
}
command_args.push(value_string);
command_args.push(trimmed_value_string);
}
let process_args = command_args
.iter()
.map(|arg| {
let arg = expand_tilde(arg.deref(), dirs::home_dir);
let home_dir;
#[cfg(feature = "dirs")]
{
home_dir = dirs::home_dir;
}
#[cfg(not(feature = "dirs"))]
{
home_dir = || Some(std::path::PathBuf::from("/"));
}
let arg = expand_tilde(arg.deref(), home_dir);
#[cfg(not(windows))]
{
@ -330,7 +265,7 @@ fn spawn(
};
let file = futures::io::AllowStdIo::new(stdout);
let stream = FramedRead::new(file, MaybeTextCodec);
let stream = FramedRead::new(file, MaybeTextCodec::default());
for line in block_on_stream(stream) {
match line {
@ -384,7 +319,7 @@ fn spawn(
}
let file = futures::io::AllowStdIo::new(stderr);
let err_stream = FramedRead::new(file, MaybeTextCodec);
let err_stream = FramedRead::new(file, MaybeTextCodec::default());
for err_line in block_on_stream(err_stream) {
match err_line {
@ -483,13 +418,19 @@ fn spawn(
}
}
async fn did_find_command(name: &str) -> bool {
#[cfg(not(windows))]
pub fn did_find_command(#[allow(unused)] name: &str) -> bool {
#[cfg(not(feature = "which"))]
{
// we can't perform this check, so just assume it can be found
true
}
#[cfg(all(feature = "which", unix))]
{
which::which(name).is_ok()
}
#[cfg(windows)]
#[cfg(all(feature = "which", windows))]
{
if which::which(name).is_ok() {
true
@ -559,11 +500,17 @@ fn shell_os_paths() -> Vec<std::path::PathBuf> {
mod tests {
use super::{
add_quotes, argument_contains_whitespace, argument_is_quoted, expand_tilde, remove_quotes,
run_external_command, Context, InputStream,
};
#[cfg(feature = "which")]
use super::{run_external_command, Context, InputStream};
#[cfg(feature = "which")]
use futures::executor::block_on;
#[cfg(feature = "which")]
use nu_errors::ShellError;
#[cfg(feature = "which")]
use nu_protocol::Scope;
#[cfg(feature = "which")]
use nu_test_support::commands::ExternalBuilder;
// async fn read(mut stream: OutputStream) -> Option<Value> {
@ -579,6 +526,7 @@ mod tests {
// }
// }
#[cfg(feature = "which")]
async fn non_existent_run() -> Result<(), ShellError> {
let cmd = ExternalBuilder::for_name("i_dont_exist.exe").build();
@ -618,6 +566,7 @@ mod tests {
// block_on(failure_run())
// }
#[cfg(feature = "which")]
#[test]
fn identifies_command_not_found() -> Result<(), ShellError> {
block_on(non_existent_run())

View File

@ -28,6 +28,10 @@ pub(crate) async fn run_internal_command(
let objects: InputStream = trace_stream!(target: "nu::trace_stream::internal", "input" = input);
let internal_command = context.expect_command(&command.name);
if command.name == "autoenv untrust" {
context.user_recently_used_autoenv_untrust = true;
}
let result = {
context
.run_command(

View File

@ -0,0 +1,127 @@
use bytes::{BufMut, Bytes, BytesMut};
use nu_errors::ShellError;
extern crate encoding_rs;
use encoding_rs::{CoderResult, Decoder, Encoding, UTF_8};
#[cfg(not(test))]
const OUTPUT_BUFFER_SIZE: usize = 8192;
#[cfg(test)]
const OUTPUT_BUFFER_SIZE: usize = 4;
#[derive(Debug, Eq, PartialEq)]
pub enum StringOrBinary {
String(String),
Binary(Vec<u8>),
}
pub struct MaybeTextCodec {
decoder: Decoder,
}
impl MaybeTextCodec {
// The constructor takes an Option<&'static Encoding>, because an absence of an encoding indicates that we want BOM sniffing enabled
pub fn new(encoding: Option<&'static Encoding>) -> Self {
let decoder = match encoding {
Some(e) => e.new_decoder_with_bom_removal(),
None => UTF_8.new_decoder(),
};
MaybeTextCodec { decoder }
}
}
impl Default for MaybeTextCodec {
fn default() -> Self {
MaybeTextCodec {
decoder: UTF_8.new_decoder(),
}
}
}
impl futures_codec::Encoder for MaybeTextCodec {
type Item = StringOrBinary;
type Error = std::io::Error;
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
match item {
StringOrBinary::String(s) => {
dst.reserve(s.len());
dst.put(s.as_bytes());
Ok(())
}
StringOrBinary::Binary(b) => {
dst.reserve(b.len());
dst.put(Bytes::from(b));
Ok(())
}
}
}
}
impl futures_codec::Decoder for MaybeTextCodec {
type Item = StringOrBinary;
type Error = ShellError;
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
if src.is_empty() {
return Ok(None);
}
let mut s = String::with_capacity(OUTPUT_BUFFER_SIZE);
let (res, _read, replacements) = self.decoder.decode_to_string(src, &mut s, false);
let result = if replacements {
// If we had to make replacements when converting to utf8, fall back to binary
StringOrBinary::Binary(src.to_vec())
} else {
// If original buffer size is too small, we continue to allocate new Strings and append
// them to the result until the input buffer is smaller than the allocated String
if let CoderResult::OutputFull = res {
let mut buffer = String::with_capacity(OUTPUT_BUFFER_SIZE);
loop {
let (res, _read, _replacements) =
self.decoder
.decode_to_string(&src[s.len()..], &mut buffer, false);
s.push_str(&buffer);
if let CoderResult::InputEmpty = res {
break;
}
buffer.clear();
}
}
StringOrBinary::String(s)
};
src.clear();
Ok(Some(result))
}
}
#[cfg(test)]
mod tests {
use super::{MaybeTextCodec, StringOrBinary};
use bytes::BytesMut;
use futures_codec::Decoder;
// TODO: Write some more tests
#[test]
fn should_consume_all_bytes_from_source_when_temporary_buffer_overflows() {
let mut maybe_text = MaybeTextCodec::new(None);
let mut bytes = BytesMut::from("0123456789");
let text = maybe_text.decode(&mut bytes);
assert_eq!(
Ok(Some(StringOrBinary::String("0123456789".to_string()))),
text
);
assert!(bytes.is_empty());
}
}

View File

@ -3,6 +3,7 @@ mod dynamic;
pub(crate) mod expr;
pub(crate) mod external;
pub(crate) mod internal;
pub(crate) mod maybe_text_codec;
#[allow(unused_imports)]
pub(crate) use dynamic::Command as DynamicCommand;

View File

@ -343,6 +343,10 @@ impl Command {
self.0.usage()
}
pub fn examples(&self) -> Vec<Example> {
self.0.examples()
}
pub async fn run(
&self,
args: CommandArgs,

View File

@ -40,7 +40,7 @@ impl WholeStreamCommand for Compact {
vec![
Example {
description: "Filter out all null entries in a list",
example: "echo [1 2 $null 3 $null $null] | compact target",
example: "echo [1 2 $null 3 $null $null] | compact",
result: Some(vec![
UntaggedValue::int(1).into(),
UntaggedValue::int(2).into(),

View File

@ -1,260 +0,0 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::data::config;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
use std::path::PathBuf;
pub struct Config;
#[derive(Deserialize)]
pub struct ConfigArgs {
load: Option<Tagged<PathBuf>>,
set: Option<(Tagged<String>, Value)>,
set_into: Option<Tagged<String>>,
get: Option<Tagged<String>>,
clear: Tagged<bool>,
remove: Option<Tagged<String>>,
path: Tagged<bool>,
}
#[async_trait]
impl WholeStreamCommand for Config {
fn name(&self) -> &str {
"config"
}
fn signature(&self) -> Signature {
Signature::build("config")
.named(
"load",
SyntaxShape::Path,
"load the config from the path given",
Some('l'),
)
.named(
"set",
SyntaxShape::Any,
"set a value in the config, eg) --set [key value]",
Some('s'),
)
.named(
"set_into",
SyntaxShape::String,
"sets a variable from values in the pipeline",
Some('i'),
)
.named(
"get",
SyntaxShape::Any,
"get a value from the config",
Some('g'),
)
.named(
"remove",
SyntaxShape::Any,
"remove a value from the config",
Some('r'),
)
.switch("clear", "clear the config", Some('c'))
.switch("path", "return the path to the config file", Some('p'))
}
fn usage(&self) -> &str {
"Configuration management."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
config(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "See all config values",
example: "config",
result: None,
},
Example {
description: "Set completion_mode to circular",
example: "config --set [completion_mode circular]",
result: None,
},
Example {
description: "Store the contents of the pipeline as a path",
example: "echo ['/usr/bin' '/bin'] | config --set_into path",
result: None,
},
Example {
description: "Get the current startup commands",
example: "config --get startup",
result: None,
},
Example {
description: "Remove the startup commands",
example: "config --remove startup",
result: None,
},
Example {
description: "Clear the config (be careful!)",
example: "config --clear",
result: None,
},
Example {
description: "Get the path to the current config file",
example: "config --path",
result: None,
},
]
}
}
pub async fn config(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
let name = args.call_info.name_tag.clone();
let registry = registry.clone();
let (
ConfigArgs {
load,
set,
set_into,
get,
clear,
remove,
path,
},
input,
) = args.process(&registry).await?;
let configuration = if let Some(supplied) = load {
Some(supplied.item().clone())
} else {
None
};
let mut result = crate::data::config::read(name_span, &configuration)?;
Ok(if let Some(v) = get {
let key = v.to_string();
let value = result
.get(&key)
.ok_or_else(|| ShellError::labeled_error("Missing key in config", "key", v.tag()))?;
match value {
Value {
value: UntaggedValue::Table(list),
..
} => {
let list: Vec<_> = list
.iter()
.map(|x| ReturnSuccess::value(x.clone()))
.collect();
futures::stream::iter(list).to_output_stream()
}
x => {
let x = x.clone();
OutputStream::one(ReturnSuccess::value(x))
}
}
} else if let Some((key, value)) = set {
result.insert(key.to_string(), value.clone());
config::write(&result, &configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(&value.tag),
))
} else if let Some(v) = set_into {
let rows: Vec<Value> = input.collect().await;
let key = v.to_string();
if rows.is_empty() {
return Err(ShellError::labeled_error(
"No values given for set_into",
"needs value(s) from pipeline",
v.tag(),
));
} else if rows.len() == 1 {
// A single value
let value = &rows[0];
result.insert(key, value.clone());
config::write(&result, &configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
))
} else {
// Take in the pipeline as a table
let value = UntaggedValue::Table(rows).into_value(name.clone());
result.insert(key, value);
config::write(&result, &configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
))
}
} else if let Tagged { item: true, tag } = clear {
result.clear();
config::write(&result, &configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(tag),
))
} else if let Tagged { item: true, tag } = path {
let path = config::default_path_for(&configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Primitive(Primitive::Path(path)).into_value(tag),
))
} else if let Some(v) = remove {
let key = v.to_string();
if result.contains_key(&key) {
result.swap_remove(&key);
config::write(&result, &configuration)?;
futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(v.tag()),
)])
.to_output_stream()
} else {
return Err(ShellError::labeled_error(
"Key does not exist in config",
"key",
v.tag(),
));
}
} else {
futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
)])
.to_output_stream()
})
}
#[cfg(test)]
mod tests {
use super::Config;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(Config {})
}
}

View File

@ -0,0 +1,57 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config clear"
}
fn signature(&self) -> Signature {
Signature::build("config clear")
}
fn usage(&self) -> &str {
"clear the config"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
clear(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Clear the config (be careful!)",
example: "config clear",
result: None,
}]
}
}
pub async fn clear(
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
// NOTE: None because we are not loading a new config file, we just want to read from the
// existing config
let mut result = crate::data::config::read(name_span, &None)?;
result.clear();
config::write(&result, &None)?;
Ok(OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(args.call_info.name_tag),
)))
}

View File

@ -0,0 +1,37 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use crate::{CommandArgs, CommandRegistry, OutputStream};
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
pub struct Command;
#[async_trait]
impl WholeStreamCommand for Command {
fn name(&self) -> &str {
"config"
}
fn signature(&self) -> Signature {
Signature::build("config")
}
fn usage(&self) -> &str {
"Configuration management."
}
async fn run(
&self,
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
let name = args.call_info.name_tag;
let result = crate::data::config::read(name_span, &None)?;
Ok(futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
)])
.to_output_stream())
}
}

View File

@ -0,0 +1,83 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct GetArgs {
get: Tagged<String>,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config get"
}
fn signature(&self) -> Signature {
Signature::build("config get").required(
"get",
SyntaxShape::Any,
"value to get from the config",
)
}
fn usage(&self) -> &str {
"Gets a value from the config"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
get(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Get the current startup commands",
example: "config get startup",
result: None,
}]
}
}
pub async fn get(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
let (GetArgs { get }, _) = args.process(&registry).await?;
// NOTE: None because we are not loading a new config file, we just want to read from the
// existing config
let result = crate::data::config::read(name_span, &None)?;
let key = get.to_string();
let value = result
.get(&key)
.ok_or_else(|| ShellError::labeled_error("Missing key in config", "key", get.tag()))?;
Ok(match value {
Value {
value: UntaggedValue::Table(list),
..
} => {
let list: Vec<_> = list
.iter()
.map(|x| ReturnSuccess::value(x.clone()))
.collect();
futures::stream::iter(list).to_output_stream()
}
x => {
let x = x.clone();
OutputStream::one(ReturnSuccess::value(x))
}
})
}

View File

@ -0,0 +1,59 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue};
use nu_source::Tagged;
use std::path::PathBuf;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct LoadArgs {
load: Tagged<PathBuf>,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config load"
}
fn signature(&self) -> Signature {
Signature::build("config load").required(
"load",
SyntaxShape::Path,
"Path to load the config from",
)
}
fn usage(&self) -> &str {
"Loads the config from the path given"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
set(args, registry).await
}
}
pub async fn set(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name = args.call_info.name_tag.clone();
let name_span = args.call_info.name_tag.clone();
let (LoadArgs { load }, _) = args.process(&registry).await?;
let configuration = load.item().clone();
let result = crate::data::config::read(name_span, &Some(configuration))?;
Ok(futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
)])
.to_output_stream())
}

View File

@ -0,0 +1,17 @@
pub mod clear;
pub mod command;
pub mod get;
pub mod load;
pub mod path;
pub mod remove;
pub mod set;
pub mod set_into;
pub use clear::SubCommand as ConfigClear;
pub use command::Command as Config;
pub use get::SubCommand as ConfigGet;
pub use load::SubCommand as ConfigLoad;
pub use path::SubCommand as ConfigPath;
pub use remove::SubCommand as ConfigRemove;
pub use set::SubCommand as ConfigSet;
pub use set_into::SubCommand as ConfigSetInto;

View File

@ -0,0 +1,49 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue};
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config path"
}
fn signature(&self) -> Signature {
Signature::build("config path")
}
fn usage(&self) -> &str {
"return the path to the config file"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
path(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Get the path to the current config file",
example: "config path",
result: None,
}]
}
}
pub async fn path(
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let path = config::default_path()?;
Ok(OutputStream::one(ReturnSuccess::value(
UntaggedValue::Primitive(Primitive::Path(path)).into_value(args.call_info.name_tag),
)))
}

View File

@ -0,0 +1,75 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue};
use nu_source::Tagged;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct RemoveArgs {
remove: Tagged<String>,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config remove"
}
fn signature(&self) -> Signature {
Signature::build("config remove").required(
"remove",
SyntaxShape::Any,
"remove a value from the config",
)
}
fn usage(&self) -> &str {
"Removes a value from the config"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
remove(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Remove the startup commands",
example: "config --remove startup",
result: None,
}]
}
}
pub async fn remove(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
let (RemoveArgs { remove }, _) = args.process(&registry).await?;
let mut result = crate::data::config::read(name_span, &None)?;
let key = remove.to_string();
if result.contains_key(&key) {
result.swap_remove(&key);
config::write(&result, &None)?;
Ok(futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(remove.tag()),
)])
.to_output_stream())
} else {
Err(ShellError::labeled_error(
"Key does not exist in config",
"key",
remove.tag(),
))
}
}

View File

@ -0,0 +1,67 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct SetArgs {
key: Tagged<String>,
value: Value,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config set"
}
fn signature(&self) -> Signature {
Signature::build("config set")
.required("key", SyntaxShape::String, "variable name to set")
.required("value", SyntaxShape::Any, "value to use")
}
fn usage(&self) -> &str {
"Sets a value in the config"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
set(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Set completion_mode to circular",
example: "config set [completion_mode circular]",
result: None,
}]
}
}
pub async fn set(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
let (SetArgs { key, value }, _) = args.process(&registry).await?;
// NOTE: None because we are not loading a new config file, we just want to read from the
// existing config
let mut result = crate::data::config::read(name_span, &None)?;
result.insert(key.to_string(), value.clone());
config::write(&result, &None)?;
Ok(OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(&value.tag),
)))
}

View File

@ -0,0 +1,98 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct SetIntoArgs {
set_into: Tagged<String>,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"config set_into"
}
fn signature(&self) -> Signature {
Signature::build("config set_into").required(
"set_into",
SyntaxShape::String,
"sets a variable from values in the pipeline",
)
}
fn usage(&self) -> &str {
"Sets a value in the config"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
set_into(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Store the contents of the pipeline as a path",
example: "echo ['/usr/bin' '/bin'] | config set_into path",
result: None,
}]
}
}
pub async fn set_into(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name_span = args.call_info.name_tag.clone();
let name = args.call_info.name_tag.clone();
let (SetIntoArgs { set_into: v }, input) = args.process(&registry).await?;
// NOTE: None because we are not loading a new config file, we just want to read from the
// existing config
let mut result = crate::data::config::read(name_span, &None)?;
// In the original code, this is set to `Some` if the `--load flag is set`
let configuration = None;
let rows: Vec<Value> = input.collect().await;
let key = v.to_string();
Ok(if rows.is_empty() {
return Err(ShellError::labeled_error(
"No values given for set_into",
"needs value(s) from pipeline",
v.tag(),
));
} else if rows.len() == 1 {
// A single value
let value = &rows[0];
result.insert(key, value.clone());
config::write(&result, &configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
))
} else {
// Take in the pipeline as a table
let value = UntaggedValue::Table(rows).into_value(name.clone());
result.insert(key, value);
config::write(&result, &configuration)?;
OutputStream::one(ReturnSuccess::value(
UntaggedValue::Row(result.into()).into_value(name),
))
})
}

View File

@ -0,0 +1,358 @@
pub const BAT_LANGUAGES: &[&str] = &[
"as",
"csv",
"tsv",
"applescript",
"script editor",
"s",
"S",
"adoc",
"asciidoc",
"asc",
"asa",
"yasm",
"nasm",
"asm",
"inc",
"mac",
"awk",
"bat",
"cmd",
"bib",
"sh",
"bash",
"zsh",
".bash_aliases",
".bash_completions",
".bash_functions",
".bash_login",
".bash_logout",
".bash_profile",
".bash_variables",
".bashrc",
".profile",
".textmate_init",
".zshrc",
"PKGBUILD",
".ebuild",
".eclass",
"c",
"h",
"cs",
"csx",
"cpp",
"cc",
"cp",
"cxx",
"c++",
"C",
"h",
"hh",
"hpp",
"hxx",
"h++",
"inl",
"ipp",
"cabal",
"clj",
"cljc",
"cljs",
"edn",
"CMakeLists.txt",
"cmake",
"h.in",
"hh.in",
"hpp.in",
"hxx.in",
"h++.in",
"CMakeCache.txt",
"cr",
"css",
"css.erb",
"css.liquid",
"d",
"di",
"dart",
"diff",
"patch",
"Dockerfile",
"dockerfile",
"ex",
"exs",
"elm",
"erl",
"hrl",
"Emakefile",
"emakefile",
"fs",
"fsi",
"fsx",
"fs",
"fsi",
"fsx",
"fish",
"attributes",
"gitattributes",
".gitattributes",
"COMMIT_EDITMSG",
"MERGE_MSG",
"TAG_EDITMSG",
"gitconfig",
".gitconfig",
".gitmodules",
"exclude",
"gitignore",
".gitignore",
".git",
"gitlog",
"git-rebase-todo",
"go",
"dot",
"DOT",
"gv",
"groovy",
"gvy",
"gradle",
"Jenkinsfile",
"hs",
"hs",
"hsc",
"show-nonprintable",
"html",
"htm",
"shtml",
"xhtml",
"asp",
"html.eex",
"yaws",
"rails",
"rhtml",
"erb",
"html.erb",
"adp",
"twig",
"html.twig",
"ini",
"INI",
"INF",
"reg",
"REG",
"lng",
"cfg",
"CFG",
"desktop",
"url",
"URL",
".editorconfig",
".hgrc",
"hgrc",
"java",
"bsh",
"properties",
"jsp",
"js",
"htc",
"js",
"jsx",
"babel",
"es6",
"js.erb",
"json",
"sublime-settings",
"sublime-menu",
"sublime-keymap",
"sublime-mousemap",
"sublime-theme",
"sublime-build",
"sublime-project",
"sublime-completions",
"sublime-commands",
"sublime-macro",
"sublime-color-scheme",
"ipynb",
"Pipfile.lock",
"jsonnet",
"libsonnet",
"libjsonnet",
"jl",
"kt",
"kts",
"tex",
"ltx",
"less",
"css.less",
"lisp",
"cl",
"clisp",
"l",
"mud",
"el",
"scm",
"ss",
"lsp",
"fasl",
"lhs",
"lua",
"make",
"GNUmakefile",
"makefile",
"Makefile",
"makefile.am",
"Makefile.am",
"makefile.in",
"Makefile.in",
"OCamlMakefile",
"mak",
"mk",
"md",
"mdown",
"markdown",
"markdn",
"matlab",
"build",
"nix",
"m",
"h",
"mm",
"M",
"h",
"ml",
"mli",
"mll",
"mly",
"pas",
"p",
"dpr",
"pl",
"pm",
"pod",
"t",
"PL",
"php",
"php3",
"php4",
"php5",
"php7",
"phps",
"phpt",
"phtml",
"txt",
"ps1",
"psm1",
"psd1",
"proto",
"protodevel",
"pb.txt",
"proto.text",
"textpb",
"pbtxt",
"prototxt",
"pp",
"epp",
"purs",
"py",
"py3",
"pyw",
"pyi",
"pyx",
"pyx.in",
"pxd",
"pxd.in",
"pxi",
"pxi.in",
"rpy",
"cpy",
"SConstruct",
"Sconstruct",
"sconstruct",
"SConscript",
"gyp",
"gypi",
"Snakefile",
"wscript",
"R",
"r",
"s",
"S",
"Rprofile",
"rd",
"re",
"rst",
"rest",
"robot",
"rb",
"Appfile",
"Appraisals",
"Berksfile",
"Brewfile",
"capfile",
"cgi",
"Cheffile",
"config.ru",
"Deliverfile",
"Fastfile",
"fcgi",
"Gemfile",
"gemspec",
"Guardfile",
"irbrc",
"jbuilder",
"Podfile",
"podspec",
"prawn",
"rabl",
"rake",
"Rakefile",
"Rantfile",
"rbx",
"rjs",
"ruby.rail",
"Scanfile",
"simplecov",
"Snapfile",
"thor",
"Thorfile",
"Vagrantfile",
"haml",
"sass",
"rxml",
"builder",
"rs",
"scala",
"sbt",
"sql",
"ddl",
"dml",
"erbsql",
"sql.erb",
"swift",
"log",
"tcl",
"tf",
"tfvars",
"hcl",
"sty",
"cls",
"textile",
"toml",
"tml",
"Cargo.lock",
"Gopkg.lock",
"Pipfile",
"ts",
"tsx",
"varlink",
"vim",
".vimrc",
"xml",
"xsd",
"xslt",
"tld",
"dtml",
"rss",
"opml",
"svg",
"yaml",
"yml",
"sublime-syntax",
];

View File

@ -19,7 +19,7 @@ impl WholeStreamCommand for Do {
}
fn signature(&self) -> Signature {
Signature::build("with-env")
Signature::build("do")
.required("block", SyntaxShape::Block, "the block to run ")
.switch(
"ignore_errors",

View File

@ -35,20 +35,7 @@ impl WholeStreamCommand for Drop {
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let (DropArgs { rows }, input) = args.process(&registry).await?;
let mut v: Vec<_> = input.into_vec().await;
let rows_to_drop = if let Some(quantity) = rows {
*quantity as usize
} else {
1
};
for _ in 0..rows_to_drop {
v.pop();
}
Ok(futures::stream::iter(v).to_output_stream())
drop(args, registry).await
}
fn examples(&self) -> Vec<Example> {
@ -70,6 +57,31 @@ impl WholeStreamCommand for Drop {
}
}
async fn drop(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let (DropArgs { rows }, input) = args.process(&registry).await?;
let v: Vec<_> = input.into_vec().await;
let rows_to_drop = if let Some(quantity) = rows {
*quantity as usize
} else {
1
};
Ok(if rows_to_drop == 0 {
futures::stream::iter(v).to_output_stream()
} else {
let k = if v.len() < rows_to_drop {
0
} else {
v.len() - rows_to_drop
};
let iter = v.into_iter().take(k);
futures::stream::iter(iter).to_output_stream()
})
}
#[cfg(test)]
mod tests {
use super::Drop;

View File

@ -343,7 +343,7 @@ where
let values = vec.into_iter().map(Into::into).collect::<Vec<Value>>();
UntaggedValue::Table(values)
}
.retag(tag)
.into_value(tag)
}
impl From<DirInfo> for Value {
@ -352,17 +352,17 @@ impl From<DirInfo> for Value {
r.insert(
"path".to_string(),
UntaggedValue::path(d.path).retag(&d.tag),
UntaggedValue::path(d.path).into_value(&d.tag),
);
r.insert(
"apparent".to_string(),
UntaggedValue::bytes(d.size).retag(&d.tag),
UntaggedValue::filesize(d.size).into_value(&d.tag),
);
r.insert(
"physical".to_string(),
UntaggedValue::bytes(d.blocks).retag(&d.tag),
UntaggedValue::filesize(d.blocks).into_value(&d.tag),
);
r.insert("directories".to_string(), value_from_vec(d.dirs, &d.tag));
@ -376,7 +376,7 @@ impl From<DirInfo> for Value {
.map(move |e| UntaggedValue::Error(e).into_untagged_value())
.collect::<Vec<Value>>(),
)
.retag(&d.tag);
.into_value(&d.tag);
r.insert("errors".to_string(), v);
}
@ -394,30 +394,33 @@ impl From<FileInfo> for Value {
r.insert(
"path".to_string(),
UntaggedValue::path(f.path).retag(&f.tag),
UntaggedValue::path(f.path).into_value(&f.tag),
);
r.insert(
"apparent".to_string(),
UntaggedValue::bytes(f.size).retag(&f.tag),
UntaggedValue::filesize(f.size).into_value(&f.tag),
);
let b = f
.blocks
.map(UntaggedValue::bytes)
.map(UntaggedValue::filesize)
.unwrap_or_else(UntaggedValue::nothing)
.retag(&f.tag);
.into_value(&f.tag);
r.insert("physical".to_string(), b);
r.insert(
"directories".to_string(),
UntaggedValue::nothing().retag(&f.tag),
UntaggedValue::nothing().into_value(&f.tag),
);
r.insert("files".to_string(), UntaggedValue::nothing().retag(&f.tag));
r.insert(
"files".to_string(),
UntaggedValue::nothing().into_value(&f.tag),
);
UntaggedValue::row(r).retag(&f.tag)
UntaggedValue::row(r).into_value(&f.tag)
}
}

View File

@ -7,14 +7,16 @@ use futures::stream::once;
use nu_errors::ShellError;
use nu_protocol::{
hir::Block, hir::Expression, hir::SpannedExpression, hir::Synthetic, Scope, Signature,
SyntaxShape, UntaggedValue, Value,
SyntaxShape, TaggedDictBuilder, UntaggedValue, Value,
};
use nu_source::Tagged;
pub struct Each;
#[derive(Deserialize)]
pub struct EachArgs {
block: Block,
numbered: Tagged<bool>,
}
#[async_trait]
@ -24,11 +26,13 @@ impl WholeStreamCommand for Each {
}
fn signature(&self) -> Signature {
Signature::build("each").required(
"block",
SyntaxShape::Block,
"the block to run on each row",
)
Signature::build("each")
.required("block", SyntaxShape::Block, "the block to run on each row")
.switch(
"numbered",
"returned a numbered item ($it.index and $it.item)",
Some('n'),
)
}
fn usage(&self) -> &str {
@ -45,6 +49,11 @@ impl WholeStreamCommand for Each {
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Echo the sum of each row",
example: "echo [[1 2] [3 4]] | each { echo $it | math sum }",
result: None,
},
Example {
description: "Echo the square of each integer",
example: "echo [1 2 3] | each { echo $(= $it * $it) }",
@ -55,28 +64,23 @@ impl WholeStreamCommand for Each {
]),
},
Example {
description: "Echo the sum of each row",
example: "echo [[1 2] [3 4]] | each { echo $it | math sum }",
result: Some(vec![
UntaggedValue::int(3).into(),
UntaggedValue::int(7).into(),
]),
description: "Number each item and echo a message",
example:
"echo ['bob' 'fred'] | each --numbered { echo `{{$it.index}} is {{$it.item}}` }",
result: Some(vec![Value::from("0 is bob"), Value::from("1 is fred")]),
},
]
}
}
fn is_expanded_it_usage(head: &SpannedExpression) -> bool {
match &*head {
SpannedExpression {
expr: Expression::Synthetic(Synthetic::String(s)),
..
} if s == "expanded-each" => true,
_ => false,
}
matches!(&*head, SpannedExpression {
expr: Expression::Synthetic(Synthetic::String(s)),
..
} if s == "expanded-each")
}
async fn process_row(
pub async fn process_row(
block: Arc<Block>,
scope: Arc<Scope>,
head: Arc<Box<SpannedExpression>>,
@ -111,21 +115,47 @@ async fn each(
let context = Arc::new(Context::from_raw(&raw_args, &registry));
let (each_args, input): (EachArgs, _) = raw_args.process(&registry).await?;
let block = Arc::new(each_args.block);
Ok(input
.then(move |input| {
let block = block.clone();
let scope = scope.clone();
let head = head.clone();
let context = context.clone();
async {
match process_row(block, scope, head, context, input).await {
Ok(s) => s,
Err(e) => OutputStream::one(Err(e)),
if each_args.numbered.item {
Ok(input
.enumerate()
.then(move |input| {
let block = block.clone();
let scope = scope.clone();
let head = head.clone();
let context = context.clone();
let mut dict = TaggedDictBuilder::new(input.1.tag());
dict.insert_untagged("index", UntaggedValue::int(input.0));
dict.insert_value("item", input.1);
async {
match process_row(block, scope, head, context, dict.into_value()).await {
Ok(s) => s,
Err(e) => OutputStream::one(Err(e)),
}
}
}
})
.flatten()
.to_output_stream())
})
.flatten()
.to_output_stream())
} else {
Ok(input
.then(move |input| {
let block = block.clone();
let scope = scope.clone();
let head = head.clone();
let context = context.clone();
async {
match process_row(block, scope, head, context, input).await {
Ok(s) => s,
Err(e) => OutputStream::one(Err(e)),
}
}
})
.flatten()
.to_output_stream())
}
}
#[cfg(test)]

View File

@ -3,7 +3,7 @@ use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::hir::Operator;
use nu_protocol::{
Primitive, RangeInclusion, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
Primitive, Range, RangeInclusion, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
};
pub struct Echo;
@ -55,60 +55,86 @@ async fn echo(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStr
let registry = registry.clone();
let (args, _): (EchoArgs, _) = args.process(&registry).await?;
let stream = args.rest.into_iter().map(|i| {
match i.as_string() {
Ok(s) => {
OutputStream::one(Ok(ReturnSuccess::Value(
UntaggedValue::string(s).into_value(i.tag.clone()),
)))
}
_ => match i {
Value {
value: UntaggedValue::Table(table),
..
} => {
futures::stream::iter(table.into_iter().map(ReturnSuccess::value)).to_output_stream()
}
Value {
value: UntaggedValue::Primitive(Primitive::Range(range)),
tag
} => {
let mut output_vec = vec![];
let mut current = range.from.0.item;
while current != range.to.0.item {
output_vec.push(Ok(ReturnSuccess::Value(UntaggedValue::Primitive(current.clone()).into_value(&tag))));
current = match crate::data::value::compute_values(Operator::Plus, &UntaggedValue::Primitive(current), &UntaggedValue::int(1)) {
Ok(result) => match result {
UntaggedValue::Primitive(p) => p,
_ => {
return OutputStream::one(Err(ShellError::unimplemented("Internal error: expected a primitive result from increment")));
}
},
Err((left_type, right_type)) => {
return OutputStream::one(Err(ShellError::coerce_error(
left_type.spanned(tag.span),
right_type.spanned(tag.span),
)));
}
}
}
if let RangeInclusion::Inclusive = range.to.1 {
output_vec.push(Ok(ReturnSuccess::Value(UntaggedValue::Primitive(current).into_value(&tag))));
}
futures::stream::iter(output_vec.into_iter()).to_output_stream()
}
_ => {
OutputStream::one(Ok(ReturnSuccess::Value(i.clone())))
}
},
}
let stream = args.rest.into_iter().map(|i| match i.as_string() {
Ok(s) => OutputStream::one(Ok(ReturnSuccess::Value(
UntaggedValue::string(s).into_value(i.tag.clone()),
))),
_ => match i {
Value {
value: UntaggedValue::Table(table),
..
} => futures::stream::iter(table.into_iter().map(ReturnSuccess::value))
.to_output_stream(),
Value {
value: UntaggedValue::Primitive(Primitive::Range(range)),
tag,
} => futures::stream::iter(RangeIterator::new(*range, tag)).to_output_stream(),
_ => OutputStream::one(Ok(ReturnSuccess::Value(i.clone()))),
},
});
Ok(futures::stream::iter(stream).flatten().to_output_stream())
}
struct RangeIterator {
curr: Primitive,
end: Primitive,
tag: Tag,
is_end_inclusive: bool,
is_done: bool,
}
impl RangeIterator {
pub fn new(range: Range, tag: Tag) -> RangeIterator {
RangeIterator {
curr: range.from.0.item,
end: range.to.0.item,
tag,
is_end_inclusive: matches!(range.to.1, RangeInclusion::Inclusive),
is_done: false,
}
}
}
impl Iterator for RangeIterator {
type Item = Result<ReturnSuccess, ShellError>;
fn next(&mut self) -> Option<Self::Item> {
if self.curr != self.end {
let output = UntaggedValue::Primitive(self.curr.clone()).into_value(self.tag.clone());
self.curr = match crate::data::value::compute_values(
Operator::Plus,
&UntaggedValue::Primitive(self.curr.clone()),
&UntaggedValue::int(1),
) {
Ok(result) => match result {
UntaggedValue::Primitive(p) => p,
_ => {
return Some(Err(ShellError::unimplemented(
"Internal error: expected a primitive result from increment",
)));
}
},
Err((left_type, right_type)) => {
return Some(Err(ShellError::coerce_error(
left_type.spanned(self.tag.span),
right_type.spanned(self.tag.span),
)));
}
};
Some(ReturnSuccess::value(output))
} else if self.is_end_inclusive && !self.is_done {
self.is_done = true;
Some(ReturnSuccess::value(
UntaggedValue::Primitive(self.curr.clone()).into_value(self.tag.clone()),
))
} else {
// TODO: add inclusive/exclusive ranges
None
}
}
}
#[cfg(test)]
mod tests {
use super::Echo;

View File

@ -121,21 +121,16 @@ async fn enter(
let full_path = std::path::PathBuf::from(cwd);
let (file_extension, contents, contents_tag) = crate::commands::open::fetch(
let (file_extension, tagged_contents) = crate::commands::open::fetch(
&full_path,
&PathBuf::from(location_clone),
tag.span,
match encoding {
Some(e) => e.to_string(),
_ => "".to_string(),
},
encoding,
)
.await?;
match contents {
match tagged_contents.value {
UntaggedValue::Primitive(Primitive::String(_)) => {
let tagged_contents = contents.into_value(&contents_tag);
if let Some(extension) = file_extension {
let command_name = format!("from {}", extension);
if let Some(converter) = registry.get_command(&command_name) {
@ -156,18 +151,18 @@ async fn enter(
scope: scope.clone(),
},
};
let tag = tagged_contents.tag.clone();
let mut result = converter
.run(new_args.with_input(vec![tagged_contents]), &registry)
.await?;
let result_vec: Vec<Result<ReturnSuccess, ShellError>> =
result.drain_vec().await;
Ok(futures::stream::iter(result_vec.into_iter().map(
move |res| match res {
Ok(ReturnSuccess::Value(Value { value, .. })) => Ok(
ReturnSuccess::Action(CommandAction::EnterValueShell(Value {
value,
tag: contents_tag.clone(),
tag: tag.clone(),
})),
),
x => x,
@ -185,13 +180,9 @@ async fn enter(
)))
}
}
_ => {
let tagged_contents = contents.into_value(contents_tag);
Ok(OutputStream::one(ReturnSuccess::action(
CommandAction::EnterValueShell(tagged_contents),
)))
}
_ => Ok(OutputStream::one(ReturnSuccess::action(
CommandAction::EnterValueShell(tagged_contents),
))),
}
}
}

View File

@ -1,83 +0,0 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use crate::utils::data_processing::{evaluate, fetch};
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::{SpannedItem, Tagged};
use nu_value_ext::ValueExt;
pub struct EvaluateBy;
#[derive(Deserialize)]
pub struct EvaluateByArgs {
evaluate_with: Option<Tagged<String>>,
}
#[async_trait]
impl WholeStreamCommand for EvaluateBy {
fn name(&self) -> &str {
"evaluate-by"
}
fn signature(&self) -> Signature {
Signature::build("evaluate-by").named(
"evaluate_with",
SyntaxShape::String,
"the name of the column to evaluate by",
Some('w'),
)
}
fn usage(&self) -> &str {
"Creates a new table with the data from the tables rows evaluated by the column given."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
evaluate_by(args, registry).await
}
}
pub async fn evaluate_by(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let name = args.call_info.name_tag.clone();
let (EvaluateByArgs { evaluate_with }, mut input) = args.process(&registry).await?;
let values: Vec<Value> = input.collect().await;
if values.is_empty() {
Err(ShellError::labeled_error(
"Expected table from pipeline",
"requires a table input",
name,
))
} else {
let evaluate_with = if let Some(evaluator) = evaluate_with {
Some(evaluator.item().clone())
} else {
None
};
match evaluate(&values[0], evaluate_with, name) {
Ok(evaluated) => Ok(OutputStream::one(ReturnSuccess::value(evaluated))),
Err(err) => Err(err),
}
}
}
#[cfg(test)]
mod tests {
use super::EvaluateBy;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(EvaluateBy {})
}
}

View File

@ -71,25 +71,23 @@ impl WholeStreamCommand for Every {
async fn every(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let (EveryArgs { stride, skip }, input) = args.process(&registry).await?;
let v: Vec<_> = input.into_vec().await;
let stride_desired = if stride.item < 1 { 1 } else { stride.item } as usize;
let stride = stride.item;
let skip = skip.item;
let mut values_vec_deque = VecDeque::new();
Ok(input
.enumerate()
.filter_map(move |(i, value)| async move {
let stride_desired = if stride < 1 { 1 } else { stride } as usize;
let should_include = skip == (i % stride_desired != 0);
for (i, x) in v.iter().enumerate() {
let should_include = if skip.item {
i % stride_desired != 0
} else {
i % stride_desired == 0
};
if should_include {
values_vec_deque.push_back(ReturnSuccess::value(x.clone()));
}
}
Ok(futures::stream::iter(values_vec_deque).to_output_stream())
if should_include {
Some(ReturnSuccess::value(value))
} else {
None
}
})
.to_output_stream())
}
#[cfg(test)]

View File

@ -5,7 +5,6 @@ use ical::parser::vcard::component::*;
use ical::property::Property;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue, Value};
use std::io::BufReader;
pub struct FromVcf;
@ -42,28 +41,20 @@ async fn from_vcf(
let input = args.input;
let input_string = input.collect_string(tag.clone()).await?.item;
let input_bytes = input_string.as_bytes();
let buf_reader = BufReader::new(input_bytes);
let parser = ical::VcardParser::new(buf_reader);
let input_bytes = input_string.into_bytes();
let cursor = std::io::Cursor::new(input_bytes);
let parser = ical::VcardParser::new(cursor);
let mut values_vec_deque = VecDeque::new();
let iter = parser.map(move |contact| match contact {
Ok(c) => ReturnSuccess::value(contact_to_value(c, tag.clone())),
Err(_) => Err(ShellError::labeled_error(
"Could not parse as .vcf",
"input cannot be parsed as .vcf",
tag.clone(),
)),
});
for contact in parser {
match contact {
Ok(c) => {
values_vec_deque.push_back(ReturnSuccess::value(contact_to_value(c, tag.clone())))
}
Err(_) => {
return Err(ShellError::labeled_error(
"Could not parse as .vcf",
"input cannot be parsed as .vcf",
tag.clone(),
))
}
}
}
Ok(futures::stream::iter(values_vec_deque).to_output_stream())
Ok(futures::stream::iter(iter).to_output_stream())
}
fn contact_to_value(contact: VcardContact, tag: Tag) -> Value {

View File

@ -10,7 +10,7 @@ pub struct GroupBy;
#[derive(Deserialize)]
pub struct GroupByArgs {
column_name: Option<Tagged<String>>,
grouper: Option<Value>,
}
#[async_trait]
@ -21,14 +21,14 @@ impl WholeStreamCommand for GroupBy {
fn signature(&self) -> Signature {
Signature::build("group-by").optional(
"column_name",
SyntaxShape::String,
"the name of the column to group by",
"grouper",
SyntaxShape::Any,
"the grouper value to use",
)
}
fn usage(&self) -> &str {
"Creates a new table with the data from the table rows grouped by the column given."
"create a new table grouped."
}
async fn run(
@ -42,12 +42,17 @@ impl WholeStreamCommand for GroupBy {
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Group items by type",
description: "group items by column named \"type\"",
example: r#"ls | group-by type"#,
result: None,
},
Example {
description: "Group items by their value",
description: "blocks can be used for generating a grouping key (same as above)",
example: r#"ls | group-by { get type }"#,
result: None,
},
Example {
description: "you can also group by raw values by leaving out the argument",
example: "echo [1 3 1 3 2 1 1] | group-by",
result: Some(vec![UntaggedValue::row(indexmap! {
"1".to_string() => UntaggedValue::Table(vec![
@ -68,26 +73,95 @@ impl WholeStreamCommand for GroupBy {
})
.into()]),
},
Example {
description: "write pipelines for a more involved grouping key",
example:
"echo [1 3 1 3 2 1 1] | group-by { echo `({{$it}} - 1) % 3` | calc | str from }",
result: None,
},
]
}
}
enum Grouper {
ByColumn(Option<Tagged<String>>),
ByBlock,
}
pub async fn group_by(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let name = args.call_info.name_tag.clone();
let (GroupByArgs { column_name }, input) = args.process(&registry).await?;
let registry = registry.clone();
let head = Arc::new(args.call_info.args.head.clone());
let scope = Arc::new(args.call_info.scope.clone());
let context = Arc::new(Context::from_raw(&args, &registry));
let (GroupByArgs { grouper }, input) = args.process(&registry).await?;
let values: Vec<Value> = input.collect().await;
let mut keys: Vec<Result<String, ShellError>> = vec![];
let mut group_strategy = Grouper::ByColumn(None);
match grouper {
Some(Value {
value: UntaggedValue::Block(block_given),
..
}) => {
let block = Arc::new(block_given);
let error_key = "error";
for value in values.iter() {
let run = block.clone();
let scope = scope.clone();
let head = head.clone();
let context = context.clone();
match crate::commands::each::process_row(run, scope, head, context, value.clone())
.await
{
Ok(mut s) => {
let collection: Vec<Result<ReturnSuccess, ShellError>> =
s.drain_vec().await;
if collection.len() > 1 {
return Err(ShellError::labeled_error(
"expected one value from the block",
"requires a table with one value for grouping",
&name,
));
}
let value = match collection.get(0) {
Some(Ok(return_value)) => {
return_value.raw_value().unwrap_or_else(|| {
UntaggedValue::string(error_key).into_value(&name)
})
}
Some(Err(_)) | None => {
UntaggedValue::string(error_key).into_value(&name)
}
};
keys.push(as_string(&value));
}
Err(_) => {
keys.push(Ok(error_key.into()));
}
}
}
group_strategy = Grouper::ByBlock;
}
Some(other) => {
group_strategy = Grouper::ByColumn(Some(as_string(&other)?.tagged(&name)));
}
_ => {}
}
if values.is_empty() {
return Err(ShellError::labeled_error(
"Expected table from pipeline",
"expected table from pipeline",
"requires a table input",
name,
));
@ -95,9 +169,25 @@ pub async fn group_by(
let values = UntaggedValue::table(&values).into_value(&name);
match group(&column_name, &values, name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(reason) => Err(reason),
match group_strategy {
Grouper::ByBlock => {
let map = keys.clone();
let block = Box::new(move |idx: usize, row: &Value| match map.get(idx) {
Some(Ok(key)) => Ok(key.clone()),
Some(Err(reason)) => Err(reason.clone()),
None => as_string(row),
});
match crate::utils::data::group(&values, &Some(block), &name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(reason) => Err(reason),
}
}
Grouper::ByColumn(column_name) => match group(&column_name, &values, name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(reason) => Err(reason),
},
}
}
@ -141,7 +231,7 @@ pub fn group(
match grouper {
Grouper::ByColumn(Some(column_name)) => {
let block = Box::new(move |row: &Value| {
let block = Box::new(move |_, row: &Value| {
match row.get_data_by_key(column_name.borrow_spanned()) {
Some(group_key) => Ok(as_string(&group_key)?),
None => Err(suggestions(column_name.borrow_tagged(), &row)),
@ -151,13 +241,16 @@ pub fn group(
crate::utils::data::group(&values, &Some(block), &name)
}
Grouper::ByColumn(None) => {
let block = Box::new(move |row: &Value| match as_string(row) {
let block = Box::new(move |_, row: &Value| match as_string(row) {
Ok(group_key) => Ok(group_key),
Err(reason) => Err(reason),
});
crate::utils::data::group(&values, &Some(block), &name)
}
Grouper::ByBlock => Err(ShellError::unimplemented(
"Block not implemented: This should never happen.",
)),
}
}

View File

@ -34,7 +34,7 @@ impl WholeStreamCommand for GroupByDate {
}
fn usage(&self) -> &str {
"Creates a new table with the data from the table rows grouped by the column given."
"creates a table grouped by date."
}
async fn run(
@ -98,17 +98,14 @@ pub async fn group_by_date(
Grouper::ByDate(None)
};
match (grouper_date, grouper_column) {
let value_result = match (grouper_date, grouper_column) {
(Grouper::ByDate(None), GroupByColumn::Name(None)) => {
let block = Box::new(move |row: &Value| row.format("%Y-%b-%d"));
let block = Box::new(move |_, row: &Value| row.format("%Y-%b-%d"));
match crate::utils::data::group(&values, &Some(block), &name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(err) => Err(err),
}
crate::utils::data::group(&values, &Some(block), &name)
}
(Grouper::ByDate(None), GroupByColumn::Name(Some(column_name))) => {
let block = Box::new(move |row: &Value| {
let block = Box::new(move |_, row: &Value| {
let group_key = match row.get_data_by_key(column_name.borrow_spanned()) {
Some(group_key) => Ok(group_key),
None => Err(suggestions(column_name.borrow_tagged(), &row)),
@ -117,21 +114,15 @@ pub async fn group_by_date(
group_key?.format("%Y-%b-%d")
});
match crate::utils::data::group(&values, &Some(block), &name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(err) => Err(err),
}
crate::utils::data::group(&values, &Some(block), &name)
}
(Grouper::ByDate(Some(fmt)), GroupByColumn::Name(None)) => {
let block = Box::new(move |row: &Value| row.format(&fmt));
let block = Box::new(move |_, row: &Value| row.format(&fmt));
match crate::utils::data::group(&values, &Some(block), &name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(err) => Err(err),
}
crate::utils::data::group(&values, &Some(block), &name)
}
(Grouper::ByDate(Some(fmt)), GroupByColumn::Name(Some(column_name))) => {
let block = Box::new(move |row: &Value| {
let block = Box::new(move |_, row: &Value| {
let group_key = match row.get_data_by_key(column_name.borrow_spanned()) {
Some(group_key) => Ok(group_key),
None => Err(suggestions(column_name.borrow_tagged(), &row)),
@ -140,12 +131,11 @@ pub async fn group_by_date(
group_key?.format(&fmt)
});
match crate::utils::data::group(&values, &Some(block), &name) {
Ok(grouped) => Ok(OutputStream::one(ReturnSuccess::value(grouped))),
Err(err) => Err(err),
}
crate::utils::data::group(&values, &Some(block), &name)
}
}
};
Ok(OutputStream::one(ReturnSuccess::value(value_result?)))
}
}

View File

@ -1,12 +1,10 @@
use crate::commands::WholeStreamCommand;
use crate::data::command_dict;
use crate::documentation::{generate_docs, get_documentation, DocumentationConfig};
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{
NamedType, PositionalType, ReturnSuccess, Signature, SyntaxShape, TaggedDictBuilder,
UntaggedValue,
};
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, TaggedDictBuilder, UntaggedValue};
use nu_source::{SpannedItem, Tagged};
use nu_value_ext::get_data_by_key;
@ -98,6 +96,10 @@ async fn help(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStr
}))
.to_output_stream(),
)
} else if rest[0].item == "generate_docs" {
Ok(OutputStream::one(ReturnSuccess::value(generate_docs(
&registry,
))))
} else if rest.len() == 2 {
// Check for a subcommand
let command_name = format!("{} {}", rest[0].item, rest[1].item);
@ -150,165 +152,8 @@ You can also learn more at https://www.nushell.sh/book/"#;
}
}
#[allow(clippy::cognitive_complexity)]
pub fn get_help(cmd: &dyn WholeStreamCommand, registry: &CommandRegistry) -> String {
let cmd_name = cmd.name();
let signature = cmd.signature();
let mut long_desc = String::new();
long_desc.push_str(&cmd.usage());
long_desc.push_str("\n");
let mut subcommands = String::new();
for name in registry.names() {
if name.starts_with(&format!("{} ", cmd_name)) {
let subcommand = registry.get_command(&name).expect("This shouldn't happen");
subcommands.push_str(&format!(" {} - {}\n", name, subcommand.usage()));
}
}
let mut one_liner = String::new();
one_liner.push_str(&signature.name);
one_liner.push_str(" ");
for positional in &signature.positional {
match &positional.0 {
PositionalType::Mandatory(name, _m) => {
one_liner.push_str(&format!("<{}> ", name));
}
PositionalType::Optional(name, _o) => {
one_liner.push_str(&format!("({}) ", name));
}
}
}
if signature.rest_positional.is_some() {
one_liner.push_str(" ...args");
}
if !subcommands.is_empty() {
one_liner.push_str("<subcommand> ");
}
if !signature.named.is_empty() {
one_liner.push_str("{flags} ");
}
long_desc.push_str(&format!("\nUsage:\n > {}\n", one_liner));
if !subcommands.is_empty() {
long_desc.push_str("\nSubcommands:\n");
long_desc.push_str(&subcommands);
}
if !signature.positional.is_empty() || signature.rest_positional.is_some() {
long_desc.push_str("\nParameters:\n");
for positional in &signature.positional {
match &positional.0 {
PositionalType::Mandatory(name, _m) => {
long_desc.push_str(&format!(" <{}> {}\n", name, positional.1));
}
PositionalType::Optional(name, _o) => {
long_desc.push_str(&format!(" ({}) {}\n", name, positional.1));
}
}
}
if let Some(rest_positional) = &signature.rest_positional {
long_desc.push_str(&format!(" ...args: {}\n", rest_positional.1));
}
}
if !signature.named.is_empty() {
long_desc.push_str(&get_flags_section(&signature))
}
let palette = crate::shell::palette::DefaultPalette {};
let examples = cmd.examples();
if !examples.is_empty() {
long_desc.push_str("\nExamples:");
}
for example in examples {
long_desc.push_str("\n");
long_desc.push_str(" ");
long_desc.push_str(example.description);
let colored_example =
crate::shell::helper::Painter::paint_string(example.example, registry, &palette);
long_desc.push_str(&format!("\n > {}\n", colored_example));
}
long_desc.push_str("\n");
long_desc
}
fn get_flags_section(signature: &Signature) -> String {
let mut long_desc = String::new();
long_desc.push_str("\nFlags:\n");
for (flag, ty) in &signature.named {
let msg = match ty.0 {
NamedType::Switch(s) => {
if let Some(c) = s {
format!(
" -{}, --{}{} {}\n",
c,
flag,
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
} else {
format!(
" --{}{} {}\n",
flag,
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
}
}
NamedType::Mandatory(s, m) => {
if let Some(c) = s {
format!(
" -{}, --{} <{}> (required parameter){} {}\n",
c,
flag,
m.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
} else {
format!(
" --{} <{}> (required parameter){} {}\n",
flag,
m.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
}
}
NamedType::Optional(s, o) => {
if let Some(c) = s {
format!(
" -{}, --{} <{}>{} {}\n",
c,
flag,
o.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
} else {
format!(
" --{} <{}>{} {}\n",
flag,
o.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
}
}
};
long_desc.push_str(&msg);
}
long_desc
get_documentation(cmd, registry, &DocumentationConfig::default())
}
#[cfg(test)]

View File

@ -0,0 +1,188 @@
use crate::commands::classified::block::run_block;
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::evaluate::evaluate_baseline_expr;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{hir::Block, hir::ClassifiedCommand, Signature, SyntaxShape, UntaggedValue};
pub struct If;
#[derive(Deserialize)]
pub struct IfArgs {
condition: Block,
then_case: Block,
else_case: Block,
}
#[async_trait]
impl WholeStreamCommand for If {
fn name(&self) -> &str {
"if"
}
fn signature(&self) -> Signature {
Signature::build("if")
.required(
"condition",
SyntaxShape::Math,
"the condition that must match",
)
.required(
"then_case",
SyntaxShape::Block,
"block to run if condition is true",
)
.required(
"else_case",
SyntaxShape::Block,
"block to run if condition is false",
)
}
fn usage(&self) -> &str {
"Run blocks if a condition is true or false."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
if_command(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Run a block if a condition is true",
example: "echo 10 | if $it > 5 { echo 'greater than 5' } { echo 'less than or equal to 5' }",
result: Some(vec![UntaggedValue::string("greater than 5").into()]),
},
Example {
description: "Run a block if a condition is false",
example: "echo 1 | if $it > 5 { echo 'greater than 5' } { echo 'less than or equal to 5' }",
result: Some(vec![UntaggedValue::string("less than or equal to 5").into()]),
},
]
}
}
async fn if_command(
raw_args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = Arc::new(registry.clone());
let scope = Arc::new(raw_args.call_info.scope.clone());
let tag = raw_args.call_info.name_tag.clone();
let context = Arc::new(Context::from_raw(&raw_args, &registry));
let (
IfArgs {
condition,
then_case,
else_case,
},
input,
) = raw_args.process(&registry).await?;
let condition = {
if condition.block.len() != 1 {
return Err(ShellError::labeled_error(
"Expected a condition",
"expected a condition",
tag,
));
}
match condition.block[0].list.get(0) {
Some(item) => match item {
ClassifiedCommand::Expr(expr) => expr.clone(),
_ => {
return Err(ShellError::labeled_error(
"Expected a condition",
"expected a condition",
tag,
));
}
},
None => {
return Err(ShellError::labeled_error(
"Expected a condition",
"expected a condition",
tag,
));
}
}
};
Ok(input
.then(move |input| {
let condition = condition.clone();
let then_case = then_case.clone();
let else_case = else_case.clone();
let registry = registry.clone();
let scope = scope.clone();
let mut context = context.clone();
async move {
//FIXME: should we use the scope that's brought in as well?
let condition =
evaluate_baseline_expr(&condition, &*registry, &input, &scope.vars, &scope.env)
.await;
match condition {
Ok(condition) => match condition.as_bool() {
Ok(b) => {
if b {
match run_block(
&then_case,
Arc::make_mut(&mut context),
InputStream::empty(),
&input,
&scope.vars,
&scope.env,
)
.await
{
Ok(stream) => stream.to_output_stream(),
Err(e) => futures::stream::iter(vec![Err(e)].into_iter())
.to_output_stream(),
}
} else {
match run_block(
&else_case,
Arc::make_mut(&mut context),
InputStream::empty(),
&input,
&scope.vars,
&scope.env,
)
.await
{
Ok(stream) => stream.to_output_stream(),
Err(e) => futures::stream::iter(vec![Err(e)].into_iter())
.to_output_stream(),
}
}
}
Err(e) => {
futures::stream::iter(vec![Err(e)].into_iter()).to_output_stream()
}
},
Err(e) => futures::stream::iter(vec![Err(e)].into_iter()).to_output_stream(),
}
}
})
.flatten()
.to_output_stream())
}
#[cfg(test)]
mod tests {
use super::If;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(If {})
}
}

View File

@ -56,10 +56,9 @@ async fn insert(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputS
Value {
value: UntaggedValue::Row(_),
..
} => match row.insert_data_at_column_path(&column, value.clone()) {
Ok(v) => Ok(ReturnSuccess::Value(v)),
Err(err) => Err(err),
},
} => Ok(ReturnSuccess::Value(
row.insert_data_at_column_path(&column, value.clone())?,
)),
Value { tag, .. } => Err(ShellError::labeled_error(
"Unrecognized type in stream",

View File

@ -5,15 +5,15 @@ use nu_errors::ShellError;
use nu_protocol::{Signature, SyntaxShape, UntaggedValue};
use nu_source::Tagged;
pub struct Keep;
pub struct Command;
#[derive(Deserialize)]
pub struct KeepArgs {
pub struct Arguments {
rows: Option<Tagged<usize>>,
}
#[async_trait]
impl WholeStreamCommand for Keep {
impl WholeStreamCommand for Command {
fn name(&self) -> &str {
"keep"
}
@ -22,7 +22,7 @@ impl WholeStreamCommand for Keep {
Signature::build("keep").optional(
"rows",
SyntaxShape::Int,
"starting from the front, the number of rows to keep",
"Starting from the front, the number of rows to keep",
)
}
@ -61,7 +61,7 @@ impl WholeStreamCommand for Keep {
async fn keep(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let (KeepArgs { rows }, input) = args.process(&registry).await?;
let (Arguments { rows }, input) = args.process(&registry).await?;
let rows_desired = if let Some(quantity) = rows {
*quantity
} else {
@ -73,12 +73,12 @@ async fn keep(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStr
#[cfg(test)]
mod tests {
use super::Keep;
use super::Command;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(Keep {})
test_examples(Command {})
}
}

View File

@ -0,0 +1,7 @@
mod command;
mod until;
mod while_;
pub use command::Command as Keep;
pub use until::SubCommand as KeepUntil;
pub use while_::SubCommand as KeepWhile;

View File

@ -5,20 +5,20 @@ use log::trace;
use nu_errors::ShellError;
use nu_protocol::{hir::ClassifiedCommand, Signature, SyntaxShape, UntaggedValue, Value};
pub struct KeepUntil;
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for KeepUntil {
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"keep-until"
"keep until"
}
fn signature(&self) -> Signature {
Signature::build("keep-until")
Signature::build("keep until")
.required(
"condition",
SyntaxShape::Math,
"the condition that must be met to stop keeping rows",
"The condition that must be met to stop keeping rows",
)
.filter()
}
@ -100,10 +100,7 @@ impl WholeStreamCommand for KeepUntil {
.await;
trace!("RESULT = {:?}", result);
match result {
Ok(ref v) if v.is_true() => false,
_ => true,
}
!matches!(result, Ok(ref v) if v.is_true())
}
})
.to_output_stream())
@ -112,12 +109,12 @@ impl WholeStreamCommand for KeepUntil {
#[cfg(test)]
mod tests {
use super::KeepUntil;
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(KeepUntil {})
test_examples(SubCommand {})
}
}

View File

@ -5,20 +5,20 @@ use log::trace;
use nu_errors::ShellError;
use nu_protocol::{hir::ClassifiedCommand, Signature, SyntaxShape, UntaggedValue, Value};
pub struct KeepWhile;
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for KeepWhile {
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"keep-while"
"keep while"
}
fn signature(&self) -> Signature {
Signature::build("keep-while")
Signature::build("keep while")
.required(
"condition",
SyntaxShape::Math,
"the condition that must be met to keep rows",
"The condition that must be met to keep rows",
)
.filter()
}
@ -100,10 +100,7 @@ impl WholeStreamCommand for KeepWhile {
.await;
trace!("RESULT = {:?}", result);
match result {
Ok(ref v) if v.is_true() => true,
_ => false,
}
matches!(result, Ok(ref v) if v.is_true())
}
})
.to_output_stream())
@ -112,12 +109,12 @@ impl WholeStreamCommand for KeepWhile {
#[cfg(test)]
mod tests {
use super::KeepWhile;
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(KeepWhile {})
test_examples(SubCommand {})
}
}

View File

@ -2,7 +2,7 @@ use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_protocol::{Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
pub struct Last;
@ -63,25 +63,21 @@ async fn last(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStr
let (LastArgs { rows }, input) = args.process(&registry).await?;
let v: Vec<_> = input.into_vec().await;
let rows_desired = if let Some(quantity) = rows {
*quantity
let end_rows_desired = if let Some(quantity) = rows {
*quantity as usize
} else {
1
};
let mut values_vec_deque = VecDeque::new();
let beginning_rows_to_skip = if end_rows_desired < v.len() {
v.len() - end_rows_desired
} else {
0
};
let count = rows_desired as usize;
let iter = v.into_iter().skip(beginning_rows_to_skip);
if count < v.len() {
let k = v.len() - count;
for x in v[k..].iter() {
values_vec_deque.push_back(ReturnSuccess::value(x.clone()));
}
}
Ok(futures::stream::iter(values_vec_deque).to_output_stream())
Ok(futures::stream::iter(iter).to_output_stream())
}
#[cfg(test)]

View File

@ -2,6 +2,7 @@ use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
use parking_lot::Mutex;
pub struct Lines;
@ -47,8 +48,7 @@ fn ends_with_line_ending(st: &str) -> bool {
}
async fn lines(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let leftover = Arc::new(vec![]);
let leftover_string = Arc::new(String::new());
let leftover_string = Arc::new(Mutex::new(String::new()));
let registry = registry.clone();
let args = args.evaluate_once(&registry).await?;
let tag = args.name_tag();
@ -62,88 +62,47 @@ async fn lines(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputSt
.input
.chain(eos)
.map(move |item| {
let mut leftover = leftover.clone();
let mut leftover_string = leftover_string.clone();
let leftover_string = leftover_string.clone();
match item {
Value {
value: UntaggedValue::Primitive(Primitive::String(st)),
..
} => {
let st = (&*leftover_string).clone() + &st;
if let Some(leftover) = Arc::get_mut(&mut leftover) {
leftover.clear();
}
let mut lines: Vec<String> = st.lines().map(|x| x.to_string()).collect();
if !ends_with_line_ending(&st) {
if let Some(last) = lines.pop() {
if let Some(leftover_string) = Arc::get_mut(&mut leftover_string) {
leftover_string.clear();
leftover_string.push_str(&last);
}
} else if let Some(leftover_string) = Arc::get_mut(&mut leftover_string) {
leftover_string.clear();
}
} else if let Some(leftover_string) = Arc::get_mut(&mut leftover_string) {
leftover_string.clear();
}
let success_lines: Vec<_> = lines
.iter()
.map(|x| ReturnSuccess::value(UntaggedValue::line(x).into_untagged_value()))
.collect();
futures::stream::iter(success_lines)
}
Value {
| Value {
value: UntaggedValue::Primitive(Primitive::Line(st)),
..
} => {
let mut leftover_string = leftover_string.lock();
let st = (&*leftover_string).clone() + &st;
if let Some(leftover) = Arc::get_mut(&mut leftover) {
leftover.clear();
}
let mut lines: Vec<String> = st.lines().map(|x| x.to_string()).collect();
leftover_string.clear();
if !ends_with_line_ending(&st) {
if let Some(last) = lines.pop() {
if let Some(leftover_string) = Arc::get_mut(&mut leftover_string) {
leftover_string.clear();
leftover_string.push_str(&last);
}
} else if let Some(leftover_string) = Arc::get_mut(&mut leftover_string) {
leftover_string.clear();
leftover_string.push_str(&last);
}
} else if let Some(leftover_string) = Arc::get_mut(&mut leftover_string) {
leftover_string.clear();
}
let success_lines: Vec<_> = lines
.iter()
.map(|x| ReturnSuccess::value(UntaggedValue::line(x).into_untagged_value()))
.collect();
futures::stream::iter(success_lines)
}
Value {
value: UntaggedValue::Primitive(Primitive::EndOfStream),
..
} => {
if !leftover.is_empty() {
let mut st = (&*leftover_string).clone();
if let Ok(extra) = String::from_utf8((&*leftover).clone()) {
st.push_str(&extra);
}
// futures::stream::iter(vec![ReturnSuccess::value(
// UntaggedValue::string(st).into_untagged_value(),
// )])
if !st.is_empty() {
futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::string(&*leftover_string).into_untagged_value(),
)])
} else {
futures::stream::iter(vec![])
}
let st = (&*leftover_string).lock().clone();
if !st.is_empty() {
futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::string(st).into_untagged_value(),
)])
} else {
futures::stream::iter(vec![])
}

View File

@ -1,84 +0,0 @@
use crate::commands::WholeStreamCommand;
use crate::data::value;
use crate::prelude::*;
use crate::utils::data_processing::map_max;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
use num_traits::cast::ToPrimitive;
pub struct MapMaxBy;
#[derive(Deserialize)]
pub struct MapMaxByArgs {
column_name: Option<Tagged<String>>,
}
#[async_trait]
impl WholeStreamCommand for MapMaxBy {
fn name(&self) -> &str {
"map-max-by"
}
fn signature(&self) -> Signature {
Signature::build("map-max-by").named(
"column_name",
SyntaxShape::String,
"the name of the column to map-max the table's rows",
Some('c'),
)
}
fn usage(&self) -> &str {
"Creates a new table with the data from the tables rows maxed by the column given."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
map_max_by(args, registry).await
}
}
pub async fn map_max_by(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let name = args.call_info.name_tag.clone();
let (MapMaxByArgs { column_name }, mut input) = args.process(&registry).await?;
let values: Vec<Value> = input.collect().await;
if values.is_empty() {
Err(ShellError::labeled_error(
"Expected table from pipeline",
"requires a table input",
name,
))
} else {
let map_by_column = if let Some(column_to_map) = column_name {
Some(column_to_map.item().clone())
} else {
None
};
match map_max(&values[0], map_by_column, name) {
Ok(table_maxed) => Ok(OutputStream::one(ReturnSuccess::value(table_maxed))),
Err(err) => Err(err),
}
}
}
#[cfg(test)]
mod tests {
use super::MapMaxBy;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(MapMaxBy {})
}
}

View File

@ -71,7 +71,7 @@ pub fn average(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
match total {
Value {
value: UntaggedValue::Primitive(Primitive::Bytes(num)),
value: UntaggedValue::Primitive(Primitive::Filesize(num)),
..
} => {
let left = UntaggedValue::from(Primitive::Int(num.into()));
@ -81,7 +81,7 @@ pub fn average(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
Ok(UntaggedValue::Primitive(Primitive::Decimal(result))) => {
let number = Number::Decimal(result);
let number = convert_number_to_u64(&number);
Ok(UntaggedValue::bytes(number).into_value(name))
Ok(UntaggedValue::filesize(number).into_value(name))
}
Ok(_) => Err(ShellError::labeled_error(
"could not calculate average of non-integer or unrelated types",

View File

@ -35,12 +35,13 @@ impl WholeStreamCommand for Command {
mod tests {
use super::*;
use crate::commands::math::{
avg::average, max::maximum, median::median, min::minimum, mode::mode, sum::summation,
utils::calculate, utils::MathFunction,
avg::average, max::maximum, median::median, min::minimum, mode::mode, stddev::stddev,
sum::summation, utils::calculate, utils::MathFunction, variance::variance,
};
use nu_plugin::row;
use nu_plugin::test_helpers::value::{decimal, int, table};
use nu_protocol::Value;
use std::str::FromStr;
#[test]
fn examples_work_as_expected() {
@ -75,7 +76,9 @@ mod tests {
Ok(int(10)),
Ok(int(10)),
Ok(table(&[int(10)])),
Ok(decimal(0)),
Ok(int(10)),
Ok(decimal(0)),
],
},
TestCase {
@ -88,7 +91,9 @@ mod tests {
Ok(int(30)),
Ok(int(20)),
Ok(table(&[int(10), int(20), int(30)])),
Ok(decimal(BigDecimal::from_str("8.164965809277260327324280249019637973219824935522233761442308557503201258191050088466198110348800783").expect("Could not convert to decimal from string"))),
Ok(int(60)),
Ok(decimal(BigDecimal::from_str("66.66666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666667").expect("Could not convert to decimal from string"))),
],
},
TestCase {
@ -101,7 +106,9 @@ mod tests {
Ok(decimal(26.5)),
Ok(decimal(26.5)),
Ok(table(&[decimal(26.5)])),
Ok(decimal(BigDecimal::from_str("7.77817459305202276840928798315333943213319531457321440247173855894902863154158871367713143880202865").expect("Could not convert to decimal from string"))),
Ok(decimal(63)),
Ok(decimal(60.5)),
],
},
TestCase {
@ -114,7 +121,9 @@ mod tests {
Ok(int(10)),
Ok(int(-11)),
Ok(table(&[int(-14), int(-11), int(10)])),
Ok(decimal(BigDecimal::from_str("10.67707825203131121081152396559571062628228776946058011397810604284900898365140801704064843595778374").expect("Could not convert to decimal from string"))),
Ok(int(-15)),
Ok(decimal(114)),
],
},
TestCase {
@ -127,7 +136,9 @@ mod tests {
Ok(int(10)),
Ok(decimal(-11.5)),
Ok(table(&[decimal(-13.5), decimal(-11.5), int(10)])),
Ok(decimal(BigDecimal::from_str("10.63798226482196513098036125801342585449179971588207816421068645273754903468375890632981926875247027").expect("Could not convert to decimal from string"))),
Ok(decimal(-15)),
Ok(decimal(BigDecimal::from_str("113.1666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666667").expect("Could not convert to decimal from string"))),
],
},
TestCase {
@ -147,8 +158,13 @@ mod tests {
Ok(row![
"col1".to_owned() => table(&[int(1), int(2), int(3), int(4)]),
"col2".to_owned() => table(&[int(5), int(6), int(7), int(8)])
]),
Ok(row![
"col1".to_owned() => decimal(BigDecimal::from_str("1.118033988749894848204586834365638117720309179805762862135448622705260462818902449707207204189391137").expect("Could not convert to decimal from string")),
"col2".to_owned() => decimal(BigDecimal::from_str("1.118033988749894848204586834365638117720309179805762862135448622705260462818902449707207204189391137").expect("Could not convert to decimal from string"))
]),
Ok(row!["col1".to_owned() => int(10), "col2".to_owned() => int(26)]),
Ok(row!["col1".to_owned() => decimal(1.25), "col2".to_owned() => decimal(1.25)]),
],
},
// TODO-Uncomment once Issue: https://github.com/nushell/nushell/issues/1883 is resolved
@ -162,8 +178,9 @@ mod tests {
let test_tag = Tag::unknown();
for tc in tt.iter() {
let tc: &TestCase = tc; // Just for type annotations
let math_functions: Vec<MathFunction> =
vec![average, minimum, maximum, median, mode, summation];
let math_functions: Vec<MathFunction> = vec![
average, minimum, maximum, median, mode, stddev, summation, variance,
];
let results = math_functions
.into_iter()
.map(|mf| calculate(&tc.values, &test_tag, mf))

View File

@ -0,0 +1,107 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct SubCommandArgs {
expression: Option<Tagged<String>>,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"math eval"
}
fn usage(&self) -> &str {
"Evaluate a math expression into a number"
}
fn signature(&self) -> Signature {
Signature::build("math eval").desc(self.usage()).optional(
"math expression",
SyntaxShape::String,
"the math expression to evaluate",
)
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
eval(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Evalulate math in the pipeline",
example: "echo '10 / 4' | math eval",
result: Some(vec![UntaggedValue::decimal(2.5).into()]),
}]
}
}
pub async fn eval(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name = args.call_info.name_tag.span;
let (SubCommandArgs { expression }, input) = args.process(registry).await?;
Ok(input
.map(move |x| {
if let Some(Tagged {
tag,
item: expression,
}) = &expression
{
UntaggedValue::string(expression).into_value(tag)
} else {
x
}
})
.map(move |input| {
if let Ok(string) = input.as_string() {
match parse(&string, &input.tag) {
Ok(value) => ReturnSuccess::value(value),
Err(err) => Err(ShellError::labeled_error(
"Math evaluation error",
err,
&input.tag.span,
)),
}
} else {
Err(ShellError::labeled_error(
"Expected a string from pipeline",
"requires string input",
name,
))
}
})
.to_output_stream())
}
pub fn parse<T: Into<Tag>>(math_expression: &str, tag: T) -> Result<Value, String> {
match meval::eval_str(math_expression) {
Ok(num) if num.is_infinite() || num.is_nan() => Err("cannot represent result".to_string()),
Ok(num) => Ok(UntaggedValue::from(Primitive::from(num)).into_value(tag)),
Err(error) => Err(error.to_string().to_lowercase()),
}
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -73,7 +73,7 @@ pub fn median(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
sorted.push(item.clone());
}
crate::commands::sort_by::sort(&mut sorted, &[], name)?;
crate::commands::sort_by::sort(&mut sorted, &[], name, false)?;
match take {
Pick::Median => {
@ -134,7 +134,7 @@ fn compute_average(values: &[Value], name: impl Into<Tag>) -> Result<Value, Shel
match total {
Value {
value: UntaggedValue::Primitive(Primitive::Bytes(num)),
value: UntaggedValue::Primitive(Primitive::Filesize(num)),
..
} => {
let left = UntaggedValue::from(Primitive::Int(num.into()));
@ -144,7 +144,7 @@ fn compute_average(values: &[Value], name: impl Into<Tag>) -> Result<Value, Shel
Ok(UntaggedValue::Primitive(Primitive::Decimal(result))) => {
let number = Number::Decimal(result);
let number = convert_number_to_u64(&number);
Ok(UntaggedValue::bytes(number).into_value(name))
Ok(UntaggedValue::filesize(number).into_value(name))
}
Ok(_) => Err(ShellError::labeled_error(
"could not calculate median of non-numeric or unrelated types",

View File

@ -1,16 +1,22 @@
pub mod avg;
pub mod command;
pub mod eval;
pub mod max;
pub mod median;
pub mod min;
pub mod mode;
pub mod stddev;
pub mod sum;
pub mod utils;
pub mod variance;
pub use avg::SubCommand as MathAverage;
pub use command::Command as Math;
pub use eval::SubCommand as MathEval;
pub use max::SubCommand as MathMaximum;
pub use median::SubCommand as MathMedian;
pub use min::SubCommand as MathMinimum;
pub use mode::SubCommand as MathMode;
pub use stddev::SubCommand as MathStddev;
pub use sum::SubCommand as MathSummation;
pub use variance::SubCommand as MathVariance;

View File

@ -77,7 +77,7 @@ pub fn mode(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
}
}
crate::commands::sort_by::sort(&mut modes, &[], name)?;
crate::commands::sort_by::sort(&mut modes, &[], name, false)?;
Ok(UntaggedValue::Table(modes).into_value(name))
}

View File

@ -0,0 +1,87 @@
use super::variance::variance;
use crate::commands::math::utils::run_with_function;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, Signature, UntaggedValue, Value};
use std::str::FromStr;
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"math stddev"
}
fn signature(&self) -> Signature {
Signature::build("math stddev")
}
fn usage(&self) -> &str {
"Finds the stddev of a list of numbers or tables"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
run_with_function(
RunnableContext {
input: args.input,
registry: registry.clone(),
shell_manager: args.shell_manager,
host: args.host,
ctrl_c: args.ctrl_c,
current_errors: args.current_errors,
name: args.call_info.name_tag,
raw_input: args.raw_input,
},
stddev,
)
.await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Get the stddev of a list of numbers",
example: "echo [1 2 3 4 5] | math stddev",
result: Some(vec![UntaggedValue::decimal(BigDecimal::from_str("1.414213562373095048801688724209698078569671875376948073176679737990732478462107038850387534327641573").expect("Could not convert to decimal from string")).into()]),
}]
}
}
pub fn stddev(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
let variance = variance(values, name)?.as_primitive()?;
let sqrt_var = match variance {
Primitive::Decimal(var) => var.sqrt(),
_ => {
return Err(ShellError::labeled_error(
"Could not take square root of variance",
"error occured here",
name.span,
))
}
};
match sqrt_var {
Some(stddev) => Ok(UntaggedValue::from(Primitive::Decimal(stddev)).into_value(name)),
None => Err(ShellError::labeled_error(
"Could not calculate stddev",
"error occured here",
name.span,
)),
}
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -50,12 +50,7 @@ pub fn calculate(values: &[Value], name: &Tag, mf: MathFunction) -> Result<Value
// The mathematical function operates over the columns of the table
let mut column_totals = IndexMap::new();
for (col_name, col_vals) in column_values {
match mf(&col_vals, &name) {
Ok(result) => {
column_totals.insert(col_name, result);
}
Err(err) => return Err(err),
}
column_totals.insert(col_name, mf(&col_vals, &name)?);
}
Ok(UntaggedValue::Row(Dictionary {

View File

@ -0,0 +1,162 @@
use crate::commands::math::utils::run_with_function;
use crate::commands::WholeStreamCommand;
use crate::data::value::compute_values;
use crate::prelude::*;
use bigdecimal::{FromPrimitive, Zero};
use nu_errors::ShellError;
use nu_protocol::{hir::Operator, Primitive, Signature, UntaggedValue, Value};
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"math variance"
}
fn signature(&self) -> Signature {
Signature::build("math variance")
}
fn usage(&self) -> &str {
"Finds the variance of a list of numbers or tables"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
run_with_function(
RunnableContext {
input: args.input,
registry: registry.clone(),
shell_manager: args.shell_manager,
host: args.host,
ctrl_c: args.ctrl_c,
current_errors: args.current_errors,
name: args.call_info.name_tag,
raw_input: args.raw_input,
},
variance,
)
.await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Get the variance of a list of numbers",
example: "echo [1 2 3 4 5] | math variance",
result: Some(vec![UntaggedValue::decimal(2).into()]),
}]
}
}
fn sum_of_squares(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
let n = BigDecimal::from_usize(values.len()).ok_or_else(|| {
ShellError::labeled_error(
"could not convert to big decimal",
"could not convert to big decimal",
&name.span,
)
})?;
let mut sum_x = Value::zero();
let mut sum_x2 = Value::zero();
for value in values {
let v = match value {
Value {
value: UntaggedValue::Primitive(Primitive::Filesize(num)),
..
} => {
UntaggedValue::from(Primitive::Int(num.clone().into()))
},
Value {
value: UntaggedValue::Primitive(num),
..
} => {
UntaggedValue::from(num.clone())
},
_ => {
return Err(ShellError::labeled_error(
"Attempted to compute the sum of squared values of a value that cannot be summed or squared.",
"value appears here",
value.tag.span,
))
}
};
let v_squared = compute_values(Operator::Multiply, &v, &v);
match v_squared {
// X^2
Ok(x2) => sum_x2 = sum_x2 + x2.into_untagged_value(),
Err((left_type, right_type)) => {
return Err(ShellError::coerce_error(
left_type.spanned(value.tag.span),
right_type.spanned(value.tag.span),
))
}
};
sum_x = sum_x + v.into_untagged_value();
}
let sum_x_squared = match compute_values(Operator::Multiply, &sum_x, &sum_x) {
Ok(v) => v.into_untagged_value(),
Err((left_type, right_type)) => {
return Err(ShellError::coerce_error(
left_type.spanned(name.span),
right_type.spanned(name.span),
))
}
};
let sum_x_squared_div_n = match compute_values(Operator::Divide, &sum_x_squared, &n.into()) {
Ok(v) => v.into_untagged_value(),
Err((left_type, right_type)) => {
return Err(ShellError::coerce_error(
left_type.spanned(name.span),
right_type.spanned(name.span),
))
}
};
let ss = match compute_values(Operator::Minus, &sum_x2, &sum_x_squared_div_n) {
Ok(v) => v.into_untagged_value(),
Err((left_type, right_type)) => {
return Err(ShellError::coerce_error(
left_type.spanned(name.span),
right_type.spanned(name.span),
))
}
};
Ok(ss)
}
pub fn variance(values: &[Value], name: &Tag) -> Result<Value, ShellError> {
let ss = sum_of_squares(values, name)?;
let n = BigDecimal::from_usize(values.len()).ok_or_else(|| {
ShellError::labeled_error(
"could not convert to big decimal",
"could not convert to big decimal",
&name.span,
)
})?;
let variance = compute_values(Operator::Divide, &ss, &n.into());
match variance {
Ok(value) => Ok(value.into_value(name)),
Err((_, _)) => Err(ShellError::labeled_error(
"could not calculate variance of non-integer or unrelated types",
"source",
name,
)),
}
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -0,0 +1,335 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::data::base::select_fields;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ColumnPath, ReturnSuccess, Signature, SyntaxShape, Value};
use nu_source::span_for_spanned_list;
pub struct SubCommand;
#[derive(Deserialize)]
pub struct Arguments {
rest: Vec<ColumnPath>,
after: Option<ColumnPath>,
before: Option<ColumnPath>,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"move column"
}
fn signature(&self) -> Signature {
Signature::build("move column")
.rest(SyntaxShape::ColumnPath, "the columns to move")
.named(
"after",
SyntaxShape::ColumnPath,
"the column that will precede the columns moved",
None,
)
.named(
"before",
SyntaxShape::ColumnPath,
"the column that will be next the columns moved",
None,
)
}
fn usage(&self) -> &str {
"Move columns."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
operate(args, registry).await
}
}
async fn operate(
raw_args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name = raw_args.call_info.name_tag.clone();
let registry = registry.clone();
let (
Arguments {
rest: mut columns,
before,
after,
},
input,
) = raw_args.process(&registry).await?;
if columns.is_empty() {
return Err(ShellError::labeled_error(
"expected columns",
"expected columns",
name,
));
}
if columns.iter().any(|c| c.members().len() > 1) {
return Err(ShellError::labeled_error(
"expected columns",
"expected columns",
name,
));
}
if vec![&after, &before]
.iter()
.map(|o| if o.is_some() { 1 } else { 0 })
.sum::<usize>()
> 1
{
return Err(ShellError::labeled_error(
"can't move column(s)",
"pick exactly one (before, after)",
name,
));
}
if let Some(after) = after {
let member = columns.remove(0);
Ok(input
.map(move |item| {
let member = vec![member.clone()];
let column_paths = vec![&member, &columns]
.into_iter()
.flatten()
.collect::<Vec<&ColumnPath>>();
let after_span = span_for_spanned_list(after.members().iter().map(|p| p.span));
if after.members().len() == 1 {
let keys = column_paths
.iter()
.filter_map(|c| c.last())
.map(|c| c.as_string())
.collect::<Vec<_>>();
if let Some(column) = after.last() {
if !keys.contains(&column.as_string()) {
ReturnSuccess::value(move_after(&item, &keys, &after, &name)?)
} else {
let msg =
format!("can't move column {} after itself", column.as_string());
Err(ShellError::labeled_error(
"can't move column",
msg,
after_span,
))
}
} else {
Err(ShellError::labeled_error(
"expected column",
"expected column",
after_span,
))
}
} else {
Err(ShellError::labeled_error(
"expected column",
"expected column",
after_span,
))
}
})
.to_output_stream())
} else if let Some(before) = before {
let member = columns.remove(0);
Ok(input
.map(move |item| {
let member = vec![member.clone()];
let column_paths = vec![&member, &columns]
.into_iter()
.flatten()
.collect::<Vec<&ColumnPath>>();
let before_span = span_for_spanned_list(before.members().iter().map(|p| p.span));
if before.members().len() == 1 {
let keys = column_paths
.iter()
.filter_map(|c| c.last())
.map(|c| c.as_string())
.collect::<Vec<_>>();
if let Some(column) = before.last() {
if !keys.contains(&column.as_string()) {
ReturnSuccess::value(move_before(&item, &keys, &before, &name)?)
} else {
let msg =
format!("can't move column {} before itself", column.as_string());
Err(ShellError::labeled_error(
"can't move column",
msg,
before_span,
))
}
} else {
Err(ShellError::labeled_error(
"expected column",
"expected column",
before_span,
))
}
} else {
Err(ShellError::labeled_error(
"expected column",
"expected column",
before_span,
))
}
})
.to_output_stream())
} else {
Err(ShellError::labeled_error(
"no columns given",
"no columns given",
name,
))
}
}
fn move_after(
table: &Value,
columns: &[String],
from: &ColumnPath,
tag: impl Into<Tag>,
) -> Result<Value, ShellError> {
let tag = tag.into();
let from_fields = span_for_spanned_list(from.members().iter().map(|p| p.span));
let from = if let Some((last, _)) = from.split_last() {
last.as_string()
} else {
return Err(ShellError::labeled_error(
"unknown column",
"unknown column",
from_fields,
));
};
let columns_moved = table
.data_descriptors()
.into_iter()
.map(|name| {
if columns.contains(&name) {
None
} else {
Some(name)
}
})
.collect::<Vec<_>>();
let mut reordered_columns = vec![];
let mut insert = false;
let mut inserted = false;
for name in columns_moved.into_iter() {
if let Some(name) = name {
reordered_columns.push(Some(name.clone()));
if !inserted && name == from {
insert = true;
}
} else {
reordered_columns.push(None);
}
if insert {
for column in columns {
reordered_columns.push(Some(column.clone()));
}
inserted = true;
}
}
Ok(select_fields(
table,
&reordered_columns
.into_iter()
.filter_map(|v| v)
.collect::<Vec<_>>(),
&tag,
))
}
fn move_before(
table: &Value,
columns: &[String],
from: &ColumnPath,
tag: impl Into<Tag>,
) -> Result<Value, ShellError> {
let tag = tag.into();
let from_fields = span_for_spanned_list(from.members().iter().map(|p| p.span));
let from = if let Some((last, _)) = from.split_last() {
last.as_string()
} else {
return Err(ShellError::labeled_error(
"unknown column",
"unknown column",
from_fields,
));
};
let columns_moved = table
.data_descriptors()
.into_iter()
.map(|name| {
if columns.contains(&name) {
None
} else {
Some(name)
}
})
.collect::<Vec<_>>();
let mut reordered_columns = vec![];
let mut inserted = false;
for name in columns_moved.into_iter() {
if let Some(name) = name {
if !inserted && name == from {
for column in columns {
reordered_columns.push(Some(column.clone()));
}
inserted = true;
}
reordered_columns.push(Some(name.clone()));
} else {
reordered_columns.push(None);
}
}
Ok(select_fields(
table,
&reordered_columns
.into_iter()
.filter_map(|v| v)
.collect::<Vec<_>>(),
&tag,
))
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -0,0 +1,46 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
#[derive(Clone)]
pub struct Command;
#[async_trait]
impl WholeStreamCommand for Command {
fn name(&self) -> &str {
"move"
}
fn signature(&self) -> Signature {
Signature::build("move")
}
fn usage(&self) -> &str {
"moves across desired subcommand."
}
async fn run(
&self,
_args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
Ok(OutputStream::one(Ok(ReturnSuccess::Value(
UntaggedValue::string(crate::commands::help::get_help(&Command, &registry))
.into_value(Tag::unknown()),
))))
}
}
#[cfg(test)]
mod tests {
use super::Command;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(Command {})
}
}

View File

@ -0,0 +1,7 @@
mod column;
mod command;
pub mod mv;
pub use column::SubCommand as MoveColumn;
pub use command::Command as Move;
pub use mv::Mv;

View File

@ -6,16 +6,16 @@ use nu_protocol::{Signature, SyntaxShape};
use nu_source::Tagged;
use std::path::PathBuf;
pub struct Move;
pub struct Mv;
#[derive(Deserialize)]
pub struct MoveArgs {
pub struct Arguments {
pub src: Tagged<PathBuf>,
pub dst: Tagged<PathBuf>,
}
#[async_trait]
impl WholeStreamCommand for Move {
impl WholeStreamCommand for Mv {
fn name(&self) -> &str {
"mv"
}
@ -78,12 +78,12 @@ async fn mv(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStrea
#[cfg(test)]
mod tests {
use super::Move;
use super::Mv;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(Move {})
test_examples(Mv {})
}
}

View File

@ -70,16 +70,20 @@ async fn nth(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStre
let row_numbers = vec![vec![row_number], and_rows]
.into_iter()
.flatten()
.collect::<Vec<Tagged<u64>>>();
.map(|x| x.item)
.collect::<Vec<u64>>();
let max_row_number = row_numbers
.iter()
.max()
.expect("Internal error: should be > 0 row numbers");
Ok(input
.take(*max_row_number as usize + 1)
.enumerate()
.filter_map(move |(idx, item)| {
futures::future::ready(
if row_numbers
.iter()
.any(|requested| requested.item == idx as u64)
{
if row_numbers.iter().any(|requested| *requested == idx as u64) {
Some(ReturnSuccess::value(item))
} else {
None

View File

@ -1,15 +1,14 @@
use crate::commands::classified::maybe_text_codec::StringOrBinary;
use crate::commands::constants::BAT_LANGUAGES;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use encoding_rs::{Encoding, UTF_8};
use futures_util::StreamExt;
use log::debug;
use nu_errors::ShellError;
use nu_protocol::{CommandAction, ReturnSuccess, Signature, SyntaxShape, UntaggedValue};
use nu_protocol::{CommandAction, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::{AnchorLocation, Span, Tagged};
use std::path::{Path, PathBuf};
extern crate encoding_rs;
use encoding_rs::*;
use std::fs::File;
use std::io::BufWriter;
use std::io::Read;
use std::io::Write;
use std::path::PathBuf;
pub struct Open;
@ -81,24 +80,27 @@ documentation link at https://docs.rs/encoding_rs/0.8.23/encoding_rs/#statics"#
}
}
pub fn get_encoding(opt: Option<String>) -> &'static Encoding {
pub fn get_encoding(opt: Option<Tagged<String>>) -> Result<&'static Encoding, ShellError> {
match opt {
None => UTF_8,
Some(label) => match Encoding::for_label((&label).as_bytes()) {
None => {
//print!("{} is not a known encoding label. Trying UTF-8.", label);
//std::process::exit(-2);
get_encoding(Some("utf-8".to_string()))
}
Some(encoding) => encoding,
None => Ok(UTF_8),
Some(label) => match Encoding::for_label((&label.item).as_bytes()) {
None => Err(ShellError::labeled_error(
format!(
r#"{} is not a valid encoding, refer to https://docs.rs/encoding_rs/0.8.23/encoding_rs/#statics for a valid list of encodings"#,
label.item
),
"invalid encoding",
label.span(),
)),
Some(encoding) => Ok(encoding),
},
}
}
async fn open(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let cwd = PathBuf::from(args.shell_manager.path());
let full_path = cwd;
let registry = registry.clone();
let shell_manager = args.shell_manager.clone();
let (
OpenArgs {
@ -108,329 +110,134 @@ async fn open(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStr
},
_,
) = args.process(&registry).await?;
let enc = match encoding {
Some(e) => e.to_string(),
_ => "".to_string(),
};
let result = fetch(&full_path, &path.item, path.tag.span, enc).await;
let (file_extension, contents, contents_tag) = result?;
// TODO: Remove once Streams are supported everywhere!
// As a short term workaround for getting AutoConvert and Bat functionality (Those don't currently support Streams)
let file_extension = if raw.item {
// Check if the extension has a "from *" command OR "bat" supports syntax highlighting
// AND the user doesn't want the raw output
// In these cases, we will collect the Stream
let ext = if raw.item {
None
} else {
// If the extension could not be determined via mimetype, try to use the path
// extension. Some file types do not declare their mimetypes (such as bson files).
file_extension.or_else(|| path.extension().map(|x| x.to_string_lossy().to_string()))
path.extension()
.map(|name| name.to_string_lossy().to_string())
};
let tagged_contents = contents.into_value(&contents_tag);
if let Some(extension) = file_extension {
Ok(OutputStream::one(ReturnSuccess::action(
CommandAction::AutoConvert(tagged_contents, extension),
)))
} else {
Ok(OutputStream::one(ReturnSuccess::value(tagged_contents)))
if let Some(ext) = ext {
// Check if we have a conversion command
if let Some(_command) = registry.get_command(&format!("from {}", ext)) {
let (_, tagged_contents) = crate::commands::open::fetch(
&cwd,
&PathBuf::from(&path.item),
path.tag.span,
encoding,
)
.await?;
return Ok(OutputStream::one(ReturnSuccess::action(
CommandAction::AutoConvert(tagged_contents, ext),
)));
}
// Check if bat does syntax highlighting
if BAT_LANGUAGES.contains(&ext.as_ref()) {
let (_, tagged_contents) = crate::commands::open::fetch(
&cwd,
&PathBuf::from(&path.item),
path.tag.span,
encoding,
)
.await?;
return Ok(OutputStream::one(ReturnSuccess::value(tagged_contents)));
}
}
// Normal Streaming operation
let with_encoding = if encoding.is_none() {
None
} else {
Some(get_encoding(encoding)?)
};
let sob_stream = shell_manager.open(&path.item, path.tag.span, with_encoding)?;
let final_stream = sob_stream.map(move |x| {
// The tag that will used when returning a Value
let file_tag = Tag {
span: path.tag.span,
anchor: Some(AnchorLocation::File(path.to_string_lossy().to_string())),
};
match x {
Ok(StringOrBinary::String(s)) => {
ReturnSuccess::value(UntaggedValue::string(s).into_value(file_tag))
}
Ok(StringOrBinary::Binary(b)) => ReturnSuccess::value(
UntaggedValue::binary(b.into_iter().collect()).into_value(file_tag),
),
Err(se) => Err(se),
}
});
Ok(OutputStream::new(final_stream))
}
// Note that we do not output a Stream in "fetch" since it is only used by "enter" command
// Which we expect to use a concrete Value a not a Stream
pub async fn fetch(
cwd: &PathBuf,
location: &PathBuf,
span: Span,
encoding: String,
) -> Result<(Option<String>, UntaggedValue, Tag), ShellError> {
encoding_choice: Option<Tagged<String>>,
) -> Result<(Option<String>, Value), ShellError> {
// TODO: I don't understand the point of this? Maybe for better error reporting
let mut cwd = cwd.clone();
let output_encoding: &Encoding = get_encoding(Some("utf-8".to_string()));
let input_encoding: &Encoding = get_encoding(Some(encoding.clone()));
let mut decoder = input_encoding.new_decoder();
let mut encoder = output_encoding.new_encoder();
let mut _file: File;
let buf = Vec::new();
let mut bufwriter = BufWriter::new(buf);
cwd.push(Path::new(location));
if let Ok(cwd) = dunce::canonicalize(&cwd) {
if !encoding.is_empty() {
// use the encoding string
match File::open(&Path::new(&cwd)) {
Ok(mut _file) => {
convert_via_utf8(
&mut decoder,
&mut encoder,
&mut _file,
&mut bufwriter,
false,
);
//bufwriter.flush()?;
Ok((
cwd.extension()
.map(|name| name.to_string_lossy().to_string()),
UntaggedValue::string(String::from_utf8_lossy(&bufwriter.buffer())),
Tag {
span,
anchor: Some(AnchorLocation::File(cwd.to_string_lossy().to_string())),
},
))
}
Err(_) => Err(ShellError::labeled_error(
format!("Cannot open {:?} for reading.", &cwd),
"file not found",
span,
)),
}
} else {
// Do the old stuff
match std::fs::read(&cwd) {
Ok(bytes) => match std::str::from_utf8(&bytes) {
Ok(s) => Ok((
cwd.extension()
.map(|name| name.to_string_lossy().to_string()),
UntaggedValue::string(s),
Tag {
span,
anchor: Some(AnchorLocation::File(cwd.to_string_lossy().to_string())),
},
)),
Err(_) => {
//Non utf8 data.
match (bytes.get(0), bytes.get(1)) {
(Some(x), Some(y)) if *x == 0xff && *y == 0xfe => {
// Possibly UTF-16 little endian
let utf16 = read_le_u16(&bytes[2..]);
if let Some(utf16) = utf16 {
match std::string::String::from_utf16(&utf16) {
Ok(s) => Ok((
cwd.extension()
.map(|name| name.to_string_lossy().to_string()),
UntaggedValue::string(s),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
)),
Err(_) => Ok((
None,
UntaggedValue::binary(bytes),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
)),
}
} else {
Ok((
None,
UntaggedValue::binary(bytes),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
))
}
}
(Some(x), Some(y)) if *x == 0xfe && *y == 0xff => {
// Possibly UTF-16 big endian
let utf16 = read_be_u16(&bytes[2..]);
if let Some(utf16) = utf16 {
match std::string::String::from_utf16(&utf16) {
Ok(s) => Ok((
cwd.extension()
.map(|name| name.to_string_lossy().to_string()),
UntaggedValue::string(s),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
)),
Err(_) => Ok((
None,
UntaggedValue::binary(bytes),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
)),
}
} else {
Ok((
None,
UntaggedValue::binary(bytes),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
))
}
}
_ => Ok((
None,
UntaggedValue::binary(bytes),
Tag {
span,
anchor: Some(AnchorLocation::File(
cwd.to_string_lossy().to_string(),
)),
},
)),
}
}
},
Err(_) => Err(ShellError::labeled_error(
format!("Cannot open {:?} for reading.", &cwd),
"file not found",
span,
)),
}
}
} else {
Err(ShellError::labeled_error(
format!("Cannot open {:?} for reading.", &cwd),
"file not found",
cwd.push(location);
let nice_location = dunce::canonicalize(&cwd).map_err(|e| {
ShellError::labeled_error(
format!("Cannot canonicalize file {:?} because {:?}", &cwd, e),
"Cannot canonicalize",
span,
))
}
}
)
})?;
fn convert_via_utf8(
decoder: &mut Decoder,
encoder: &mut Encoder,
read: &mut dyn Read,
write: &mut dyn Write,
last: bool,
) {
let mut input_buffer = [0u8; 2048];
let mut intermediate_buffer_bytes = [0u8; 4096];
// Is there a safe way to create a stack-allocated &mut str?
let mut intermediate_buffer: &mut str =
//unsafe { std::mem::transmute(&mut intermediate_buffer_bytes[..]) };
std::str::from_utf8_mut(&mut intermediate_buffer_bytes[..]).expect("error with from_utf8_mut");
let mut output_buffer = [0u8; 4096];
let mut current_input_ended = false;
while !current_input_ended {
match read.read(&mut input_buffer) {
Err(_) => {
print!("Error reading input.");
//std::process::exit(-5);
}
Ok(decoder_input_end) => {
current_input_ended = decoder_input_end == 0;
let input_ended = last && current_input_ended;
let mut decoder_input_start = 0usize;
loop {
let (decoder_result, decoder_read, decoder_written, _) = decoder.decode_to_str(
&input_buffer[decoder_input_start..decoder_input_end],
&mut intermediate_buffer,
input_ended,
);
decoder_input_start += decoder_read;
// The extension may be used in AutoConvert later on
let ext = location
.extension()
.map(|name| name.to_string_lossy().to_string());
let last_output = if input_ended {
match decoder_result {
CoderResult::InputEmpty => true,
CoderResult::OutputFull => false,
}
} else {
false
};
// The tag that will used when returning a Value
let file_tag = Tag {
span,
anchor: Some(AnchorLocation::File(
nice_location.to_string_lossy().to_string(),
)),
};
// Regardless of whether the intermediate buffer got full
// or the input buffer was exhausted, let's process what's
// in the intermediate buffer.
let res = std::fs::read(location)?;
if encoder.encoding() == UTF_8 {
// If the target is UTF-8, optimize out the encoder.
if write
.write_all(&intermediate_buffer.as_bytes()[..decoder_written])
.is_err()
{
print!("Error writing output.");
//std::process::exit(-7);
}
} else {
let mut encoder_input_start = 0usize;
loop {
let (encoder_result, encoder_read, encoder_written, _) = encoder
.encode_from_utf8(
&intermediate_buffer[encoder_input_start..decoder_written],
&mut output_buffer,
last_output,
);
encoder_input_start += encoder_read;
if write.write_all(&output_buffer[..encoder_written]).is_err() {
print!("Error writing output.");
//std::process::exit(-6);
}
match encoder_result {
CoderResult::InputEmpty => {
break;
}
CoderResult::OutputFull => {
continue;
}
}
}
}
// Now let's see if we should read again or process the
// rest of the current input buffer.
match decoder_result {
CoderResult::InputEmpty => {
break;
}
CoderResult::OutputFull => {
continue;
}
}
}
}
}
}
}
fn read_le_u16(input: &[u8]) -> Option<Vec<u16>> {
if input.len() % 2 != 0 || input.len() < 2 {
None
// If no encoding is provided we try to guess the encoding to read the file with
let encoding = if encoding_choice.is_none() {
UTF_8
} else {
let mut result = vec![];
let mut pos = 0;
while pos < input.len() {
result.push(u16::from_le_bytes([input[pos], input[pos + 1]]));
pos += 2;
}
get_encoding(encoding_choice.clone())?
};
Some(result)
}
}
fn read_be_u16(input: &[u8]) -> Option<Vec<u16>> {
if input.len() % 2 != 0 || input.len() < 2 {
None
// If the user specified an encoding, then do not do BOM sniffing
let decoded_res = if encoding_choice.is_some() {
let (cow_res, _replacements) = encoding.decode_with_bom_removal(&res);
cow_res
} else {
let mut result = vec![];
let mut pos = 0;
while pos < input.len() {
result.push(u16::from_be_bytes([input[pos], input[pos + 1]]));
pos += 2;
// Otherwise, use the default UTF-8 encoder with BOM sniffing
let (cow_res, actual_encoding, replacements) = encoding.decode(&res);
// If we had to use replacement characters then fallback to binary
if replacements {
return Ok((ext, UntaggedValue::binary(res).into_value(file_tag)));
}
Some(result)
}
debug!("Decoded using {:?}", actual_encoding);
cow_res
};
let v = UntaggedValue::string(decoded_res.to_string()).into_value(file_tag);
Ok((ext, v))
}
#[cfg(test)]

View File

@ -2,10 +2,12 @@ pub mod command;
pub mod bool;
pub mod dice;
#[cfg(feature = "uuid_crate")]
pub mod uuid;
pub use command::Command as Random;
pub use self::bool::SubCommand as RandomBool;
pub use dice::SubCommand as RandomDice;
#[cfg(feature = "uuid_crate")]
pub use uuid::SubCommand as RandomUUID;

View File

@ -1,83 +0,0 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use crate::utils::data_processing::reduce;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
use num_traits::cast::ToPrimitive;
pub struct ReduceBy;
#[derive(Deserialize)]
pub struct ReduceByArgs {
reduce_with: Option<Tagged<String>>,
}
#[async_trait]
impl WholeStreamCommand for ReduceBy {
fn name(&self) -> &str {
"reduce-by"
}
fn signature(&self) -> Signature {
Signature::build("reduce-by").named(
"reduce_with",
SyntaxShape::String,
"the command to reduce by with",
Some('w'),
)
}
fn usage(&self) -> &str {
"Creates a new table with the data from the tables rows reduced by the command given."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
reduce_by(args, registry).await
}
}
pub async fn reduce_by(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let name = args.call_info.name_tag.clone();
let (ReduceByArgs { reduce_with }, mut input) = args.process(&registry).await?;
let values: Vec<Value> = input.collect().await;
if values.is_empty() {
return Err(ShellError::labeled_error(
"Expected table from pipeline",
"requires a table input",
name,
));
}
let reduce_with = if let Some(reducer) = reduce_with {
Some(reducer.item().clone())
} else {
None
};
match reduce(&values[0], reduce_with, name) {
Ok(reduced) => Ok(OutputStream::one(ReturnSuccess::value(reduced))),
Err(err) => Err(err),
}
}
#[cfg(test)]
mod tests {
use super::ReduceBy;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(ReduceBy {})
}
}

View File

@ -75,6 +75,7 @@ impl WholeStreamCommand for RunExternalCommand {
Context {
registry: registry.clone(),
host: args.host.clone(),
user_recently_used_autoenv_untrust: false,
shell_manager: args.shell_manager.clone(),
ctrl_c: args.ctrl_c.clone(),
current_errors: Arc::new(Mutex::new(vec![])),
@ -88,6 +89,7 @@ impl WholeStreamCommand for RunExternalCommand {
{
Context {
registry: registry.clone(),
user_recently_used_autoenv_untrust: false,
host: args.host.clone(),
shell_manager: args.shell_manager.clone(),
ctrl_c: args.ctrl_c.clone(),
@ -155,7 +157,7 @@ async fn maybe_autocd_dir<'a>(cmd: &ExternalCommand, ctx: &mut Context) -> Optio
|| (cmd.args.is_empty()
&& PathBuf::from(name).is_dir()
&& dunce::canonicalize(name).is_ok()
&& which::which(&name).is_err())
&& !crate::commands::classified::external::did_find_command(&name))
{
Some(name)
} else {

View File

@ -170,7 +170,7 @@ async fn save(
let host = raw_args.host.clone();
let ctrl_c = raw_args.ctrl_c.clone();
let current_errors = raw_args.current_errors.clone();
let shell_manager = raw_args.shell_manager.clone();
let mut shell_manager = raw_args.shell_manager.clone();
let head = raw_args.call_info.args.head.clone();
let (
@ -219,7 +219,7 @@ async fn save(
host,
ctrl_c,
current_errors,
shell_manager,
shell_manager: shell_manager.clone(),
call_info: UnevaluatedCallInfo {
args: nu_protocol::hir::Call {
head,
@ -252,14 +252,7 @@ async fn save(
};
match content {
Ok(save_data) => match std::fs::write(full_path, save_data) {
Ok(_) => Ok(OutputStream::empty()),
Err(e) => Err(ShellError::labeled_error(
e.to_string(),
"IO error while saving",
name,
)),
},
Ok(save_data) => shell_manager.save(&full_path, &save_data, name.span),
Err(e) => Err(e),
}
}

View File

@ -5,21 +5,21 @@ use nu_errors::ShellError;
use nu_protocol::{Signature, SyntaxShape, UntaggedValue};
use nu_source::Tagged;
pub struct Skip;
pub struct Command;
#[derive(Deserialize)]
pub struct SkipArgs {
pub struct Arguments {
rows: Option<Tagged<usize>>,
}
#[async_trait]
impl WholeStreamCommand for Skip {
impl WholeStreamCommand for Command {
fn name(&self) -> &str {
"skip"
}
fn signature(&self) -> Signature {
Signature::build("skip").optional("rows", SyntaxShape::Int, "how many rows to skip")
Signature::build("skip").optional("rows", SyntaxShape::Int, "How many rows to skip")
}
fn usage(&self) -> &str {
@ -48,7 +48,7 @@ impl WholeStreamCommand for Skip {
async fn skip(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let (SkipArgs { rows }, input) = args.process(&registry).await?;
let (Arguments { rows }, input) = args.process(&registry).await?;
let rows_desired = if let Some(quantity) = rows {
*quantity
} else {
@ -60,12 +60,12 @@ async fn skip(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStr
#[cfg(test)]
mod tests {
use super::Skip;
use super::Command;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(Skip {})
test_examples(Command {})
}
}

View File

@ -0,0 +1,7 @@
mod command;
mod until;
mod while_;
pub use command::Command as Skip;
pub use until::SubCommand as SkipUntil;
pub use while_::SubCommand as SkipWhile;

View File

@ -5,20 +5,20 @@ use log::trace;
use nu_errors::ShellError;
use nu_protocol::{hir::ClassifiedCommand, Signature, SyntaxShape, UntaggedValue, Value};
pub struct SkipUntil;
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SkipUntil {
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"skip-until"
"skip until"
}
fn signature(&self) -> Signature {
Signature::build("skip-until")
Signature::build("skip until")
.required(
"condition",
SyntaxShape::Math,
"the condition that must be met to stop skipping",
"The condition that must be met to stop skipping",
)
.filter()
}
@ -99,10 +99,7 @@ impl WholeStreamCommand for SkipUntil {
.await;
trace!("RESULT = {:?}", result);
match result {
Ok(ref v) if v.is_true() => false, // stop skipping
_ => true,
}
!matches!(result, Ok(ref v) if v.is_true())
}
})
.to_output_stream())
@ -111,12 +108,12 @@ impl WholeStreamCommand for SkipUntil {
#[cfg(test)]
mod tests {
use super::SkipUntil;
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SkipUntil {})
test_examples(SubCommand {})
}
}

View File

@ -5,20 +5,20 @@ use log::trace;
use nu_errors::ShellError;
use nu_protocol::{hir::ClassifiedCommand, Signature, SyntaxShape, UntaggedValue, Value};
pub struct SkipWhile;
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SkipWhile {
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"skip-while"
"skip while"
}
fn signature(&self) -> Signature {
Signature::build("skip-while")
Signature::build("skip while")
.required(
"condition",
SyntaxShape::Math,
"the condition that must be met to continue skipping",
"The condition that must be met to continue skipping",
)
.filter()
}
@ -99,10 +99,7 @@ impl WholeStreamCommand for SkipWhile {
.await;
trace!("RESULT = {:?}", result);
match result {
Ok(ref v) if v.is_true() => true,
_ => false,
}
matches!(result, Ok(ref v) if v.is_true())
}
})
.to_output_stream())
@ -111,12 +108,12 @@ impl WholeStreamCommand for SkipWhile {
#[cfg(test)]
mod tests {
use super::SkipWhile;
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SkipWhile {})
test_examples(SubCommand {})
}
}

View File

@ -11,6 +11,7 @@ pub struct SortBy;
#[derive(Deserialize)]
pub struct SortByArgs {
rest: Vec<Tagged<String>>,
insensitive: bool,
}
#[async_trait]
@ -20,7 +21,13 @@ impl WholeStreamCommand for SortBy {
}
fn signature(&self) -> Signature {
Signature::build("sort-by").rest(SyntaxShape::String, "the column(s) to sort by")
Signature::build("sort-by")
.switch(
"insensitive",
"Sort string-based columns case insensitively",
Some('i'),
)
.rest(SyntaxShape::String, "the column(s) to sort by")
}
fn usage(&self) -> &str {
@ -57,6 +64,24 @@ impl WholeStreamCommand for SortBy {
example: "ls | sort-by type size",
result: None,
},
Example {
description: "Sort strings (case sensitive)",
example: "echo [airplane Truck Car] | sort-by",
result: Some(vec![
UntaggedValue::string("Car").into(),
UntaggedValue::string("Truck").into(),
UntaggedValue::string("airplane").into(),
]),
},
Example {
description: "Sort strings (case insensitive)",
example: "echo [airplane Truck Car] | sort-by -i",
result: Some(vec![
UntaggedValue::string("airplane").into(),
UntaggedValue::string("Car").into(),
UntaggedValue::string("Truck").into(),
]),
},
]
}
}
@ -68,24 +93,19 @@ async fn sort_by(
let registry = registry.clone();
let tag = args.call_info.name_tag.clone();
let (SortByArgs { rest }, mut input) = args.process(&registry).await?;
let (SortByArgs { rest, insensitive }, mut input) = args.process(&registry).await?;
let mut vec = input.drain_vec().await;
sort(&mut vec, &rest, &tag)?;
sort(&mut vec, &rest, &tag, insensitive)?;
let mut values_vec_deque: VecDeque<Value> = VecDeque::new();
for item in vec {
values_vec_deque.push_back(item);
}
Ok(futures::stream::iter(values_vec_deque).to_output_stream())
Ok(futures::stream::iter(vec.into_iter()).to_output_stream())
}
pub fn sort(
vec: &mut [Value],
keys: &[Tagged<String>],
tag: impl Into<Tag>,
insensitive: bool,
) -> Result<(), ShellError> {
let tag = tag.into();
@ -113,12 +133,38 @@ pub fn sort(
value: UntaggedValue::Primitive(_),
..
} => {
vec.sort_by(|a, b| coerce_compare(a, b).expect("Unimplemented BUG: What about primitives that don't have an order defined?").compare());
let should_sort_case_insensitively = insensitive && vec.iter().all(|x| x.is_string());
vec.sort_by(|a, b| {
if should_sort_case_insensitively {
let lowercase_a_string = a.expect_string().to_ascii_lowercase();
let lowercase_b_string = b.expect_string().to_ascii_lowercase();
lowercase_a_string.cmp(&lowercase_b_string)
} else {
coerce_compare(a, b).expect("Unimplemented BUG: What about primitives that don't have an order defined?").compare()
}
});
}
_ => {
let calc_key = |item: &Value| {
keys.iter()
.map(|f| get_data_by_key(item, f.borrow_spanned()))
.map(|f| {
let mut value_option = get_data_by_key(item, f.borrow_spanned());
if insensitive {
if let Some(value) = &value_option {
if let Ok(string_value) = value.as_string() {
value_option = Some(
UntaggedValue::string(string_value.to_ascii_lowercase())
.into_value(value.tag.clone()),
)
}
}
}
value_option
})
.collect::<Vec<Option<Value>>>()
};
vec.sort_by_cached_key(calc_key);

View File

@ -0,0 +1,84 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, Value};
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"split chars"
}
fn signature(&self) -> Signature {
Signature::build("split chars")
}
fn usage(&self) -> &str {
"splits a string's characters into separate rows"
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
split_chars(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Split the string's characters into separate rows",
example: "echo 'hello' | split chars",
result: Some(vec![
Value::from("h"),
Value::from("e"),
Value::from("l"),
Value::from("l"),
Value::from("o"),
]),
}]
}
}
async fn split_chars(
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let name = args.call_info.name_tag.clone();
let input = args.input;
Ok(input
.flat_map(move |v| {
if let Ok(s) = v.as_string() {
futures::stream::iter(
s.chars()
.collect::<Vec<_>>()
.into_iter()
.map(move |x| ReturnSuccess::value(Value::from(x.to_string()))),
)
.to_output_stream()
} else {
OutputStream::one(Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name.span,
"value originates from here",
v.tag.span,
)))
}
})
.to_output_stream())
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -1,7 +1,9 @@
pub mod chars;
pub mod column;
pub mod command;
pub mod row;
pub use chars::SubCommand as SplitChars;
pub use column::SubCommand as SplitColumn;
pub use command::Command as Split;
pub use row::SubCommand as SplitRow;

View File

@ -81,7 +81,7 @@ pub fn split(
match grouper {
Grouper::ByColumn(Some(column_name)) => {
let block = Box::new(move |row: &Value| {
let block = Box::new(move |_, row: &Value| {
match row.get_data_by_key(column_name.borrow_spanned()) {
Some(group_key) => Ok(as_string(&group_key)?),
None => Err(suggestions(column_name.borrow_tagged(), &row)),
@ -91,7 +91,7 @@ pub fn split(
crate::utils::data::split(&values, &Some(block), &name)
}
Grouper::ByColumn(None) => {
let block = Box::new(move |row: &Value| match as_string(row) {
let block = Box::new(move |_, row: &Value| match as_string(row) {
Ok(group_key) => Ok(group_key),
Err(reason) => Err(reason),
});

View File

@ -62,25 +62,15 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => return Err(err),
}
)?;
}
ReturnSuccess::value(ret)

View File

@ -62,27 +62,15 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)

View File

@ -80,29 +80,17 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, &options, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, &options, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let options = options.clone();
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, &options, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)

View File

@ -0,0 +1,252 @@
use crate::commands::str_::trim::trim_char;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{
ColumnPath, Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
};
use nu_source::Tagged;
use num_bigint::{BigInt, BigUint, ToBigInt};
// TODO num_format::SystemLocale once platform-specific dependencies are stable (see Cargo.toml)
use num_format::{Locale, ToFormattedString};
use num_traits::{Pow, Signed};
use std::iter;
pub struct SubCommand;
#[derive(Deserialize)]
struct Arguments {
rest: Vec<ColumnPath>,
decimals: Option<Tagged<u64>>,
#[serde(rename(deserialize = "group-digits"))]
group_digits: bool,
}
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"str from"
}
fn signature(&self) -> Signature {
Signature::build("str from")
.rest(
SyntaxShape::ColumnPath,
"optionally convert to string by column paths",
)
.named(
"decimals",
SyntaxShape::Int,
"decimal digits to which to round",
Some('d'),
)
.switch(
"group-digits",
// TODO according to system localization
"group digits, currently by thousand with commas",
Some('g'),
)
}
fn usage(&self) -> &str {
"Converts numeric types to strings. Trims trailing zeros unless decimals parameter is specified."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
operate(args, registry).await
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "round to nearest integer",
example: "= 1.7 | str from -d 0",
result: Some(vec![UntaggedValue::string("2").into_untagged_value()]),
},
Example {
description: "format large number with localized digit grouping",
example: "= 1000000.2 | str from -g",
result: Some(vec![
UntaggedValue::string("1,000,000.2").into_untagged_value()
]),
},
]
}
}
async fn operate(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let (
Arguments {
decimals,
group_digits,
rest: column_paths,
},
input,
) = args.process(&registry.clone()).await?;
let digits = decimals.map(|tagged| tagged.item);
Ok(input
.map(move |v| {
if column_paths.is_empty() {
ReturnSuccess::value(action(&v, v.tag(), digits, group_digits)?)
} else {
let mut ret = v;
for path in &column_paths {
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag(), digits, group_digits)),
)?;
}
ReturnSuccess::value(ret)
}
})
.to_output_stream())
}
// TODO If you're using the with-system-locale feature and you're on Windows, Clang 3.9 or higher is also required.
fn action(
input: &Value,
tag: impl Into<Tag>,
digits: Option<u64>,
group_digits: bool,
) -> Result<Value, ShellError> {
match &input.value {
UntaggedValue::Primitive(prim) => Ok(UntaggedValue::string(match prim {
Primitive::Int(int) => {
if group_digits {
format_bigint(int) // int.to_formatted_string(*locale)
} else {
int.to_string()
}
}
Primitive::Decimal(dec) => format_decimal(dec.clone(), digits, group_digits),
_ => {
return Err(ShellError::unimplemented(
"str from for non-numeric primitives",
))
}
})
.into_value(tag)),
UntaggedValue::Row(_) => Err(ShellError::labeled_error(
"specify column to use 'str from'",
"found table",
input.tag.clone(),
)),
_ => Err(ShellError::unimplemented(
"str from for non-primitive, non-table types",
)),
}
}
fn format_bigint(int: &BigInt) -> String {
int.to_formatted_string(&Locale::en)
// TODO once platform-specific dependencies are stable (see Cargo.toml)
// #[cfg(windows)]
// {
// int.to_formatted_string(&Locale::en)
// }
// #[cfg(not(windows))]
// {
// match SystemLocale::default() {
// Ok(locale) => int.to_formatted_string(&locale),
// Err(_) => int.to_formatted_string(&Locale::en),
// }
// }
}
fn format_decimal(mut decimal: BigDecimal, digits: Option<u64>, group_digits: bool) -> String {
if let Some(n) = digits {
decimal = round_decimal(&decimal, n)
}
if decimal.is_integer() && (digits.is_none() || digits == Some(0)) {
let int = decimal
.to_bigint()
.expect("integer BigDecimal should convert to BigInt");
return if group_digits {
int.to_string()
} else {
format_bigint(&int)
};
}
let (int, exp) = decimal.as_bigint_and_exponent();
let factor = BigInt::from(10).pow(BigUint::from(exp as u64)); // exp > 0 for non-int decimal
let int_part = &int / &factor;
let dec_part = (&int % &factor)
.abs()
.to_biguint()
.expect("BigInt::abs should always produce positive signed BigInt and thus BigUInt")
.to_str_radix(10);
let dec_str = if let Some(n) = digits {
dec_part
.chars()
.chain(iter::repeat('0'))
.take(n as usize)
.collect()
} else {
trim_char(&dec_part, '0', false, true)
};
let format_default_loc = |int_part: BigInt| {
let loc = Locale::en;
let (int_str, sep) = (
int_part.to_formatted_string(&loc),
String::from(loc.decimal()),
);
format!("{}{}{}", int_str, sep, dec_str)
};
format_default_loc(int_part)
// TODO once platform-specific dependencies are stable (see Cargo.toml)
// #[cfg(windows)]
// {
// format_default_loc(int_part)
// }
// #[cfg(not(windows))]
// {
// match SystemLocale::default() {
// Ok(sys_loc) => {
// let int_str = int_part.to_formatted_string(&sys_loc);
// let sep = String::from(sys_loc.decimal());
// format!("{}{}{}", int_str, sep, dec_str)
// }
// Err(_) => format_default_loc(int_part),
// }
// }
}
fn round_decimal(decimal: &BigDecimal, mut digits: u64) -> BigDecimal {
let mut mag = decimal.clone();
while mag >= BigDecimal::from(1) {
mag = mag / 10;
digits += 1;
}
decimal.with_prec(digits)
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -0,0 +1,65 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"str length"
}
fn signature(&self) -> Signature {
Signature::build("str length")
}
fn usage(&self) -> &str {
"outputs the lengths of the strings in the pipeline"
}
async fn run(
&self,
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
Ok(args
.input
.map(move |x| match x.as_string() {
Ok(s) => ReturnSuccess::value(UntaggedValue::int(s.len()).into_untagged_value()),
Err(err) => Err(err),
})
.to_output_stream())
}
fn examples(&self) -> Vec<Example> {
vec![
Example {
description: "Return the lengths of multiple strings",
example: "echo 'hello' | str length",
result: Some(vec![UntaggedValue::int(5).into_untagged_value()]),
},
Example {
description: "Return the lengths of multiple strings",
example: "echo 'hi' 'there' | str length",
result: Some(vec![
UntaggedValue::int(2).into_untagged_value(),
UntaggedValue::int(5).into_untagged_value(),
]),
},
]
}
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -3,6 +3,9 @@ mod collect;
mod command;
mod downcase;
mod find_replace;
mod from;
mod length;
mod reverse;
mod set;
mod substring;
mod to_datetime;
@ -16,6 +19,9 @@ pub use collect::SubCommand as StrCollect;
pub use command::Command as Str;
pub use downcase::SubCommand as StrDowncase;
pub use find_replace::SubCommand as StrFindReplace;
pub use from::SubCommand as StrFrom;
pub use length::SubCommand as StrLength;
pub use reverse::SubCommand as StrReverse;
pub use set::SubCommand as StrSet;
pub use substring::SubCommand as StrSubstring;
pub use to_datetime::SubCommand as StrToDatetime;

View File

@ -0,0 +1,58 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
pub struct SubCommand;
#[async_trait]
impl WholeStreamCommand for SubCommand {
fn name(&self) -> &str {
"str reverse"
}
fn signature(&self) -> Signature {
Signature::build("str reverse")
}
fn usage(&self) -> &str {
"outputs the reversals of the strings in the pipeline"
}
async fn run(
&self,
args: CommandArgs,
_registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
Ok(args
.input
.map(move |x| match x.as_string() {
Ok(s) => ReturnSuccess::value(
UntaggedValue::string(s.chars().rev().collect::<String>())
.into_untagged_value(),
),
Err(err) => Err(err),
})
.to_output_stream())
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Return the reversals of multiple strings",
example: "echo 'Nushell' | str reverse",
result: Some(vec![UntaggedValue::string("llehsuN").into_untagged_value()]),
}]
}
}
#[cfg(test)]
mod tests {
use super::SubCommand;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(SubCommand {})
}
}

View File

@ -73,29 +73,17 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, &options, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, &options, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let options = options.clone();
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, &options, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)

View File

@ -5,14 +5,16 @@ use nu_protocol::ShellTypeName;
use nu_protocol::{
ColumnPath, Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
};
use nu_source::{Tag, Tagged};
use nu_value_ext::ValueExt;
use nu_source::Tag;
use nu_value_ext::{as_string, ValueExt};
use std::cmp;
use std::cmp::Ordering;
use std::convert::TryInto;
#[derive(Deserialize)]
struct Arguments {
range: Tagged<String>,
range: Value,
rest: Vec<ColumnPath>,
}
@ -28,8 +30,8 @@ impl WholeStreamCommand for SubCommand {
Signature::build("str substring")
.required(
"range",
SyntaxShape::String,
"the indexes to substring \"start, end\"",
SyntaxShape::Any,
"the indexes to substring [start end]",
)
.rest(
SyntaxShape::ColumnPath,
@ -53,9 +55,19 @@ impl WholeStreamCommand for SubCommand {
vec![
Example {
description: "Get a substring from the text",
example: "echo 'good nushell' | str substring [5 12]",
result: Some(vec![Value::from("nushell")]),
},
Example {
description: "Alternatively, you can use the form",
example: "echo 'good nushell' | str substring '5,12'",
result: Some(vec![Value::from("nushell")]),
},
Example {
description: "Get the last characters from the string",
example: "echo 'good nushell' | str substring ',-5'",
result: Some(vec![Value::from("shell")]),
},
Example {
description: "Get the remaining characters from a starting index",
example: "echo 'good nushell' | str substring '5,'",
@ -71,7 +83,15 @@ impl WholeStreamCommand for SubCommand {
}
#[derive(Clone)]
struct Substring(usize, usize);
struct Substring(isize, isize);
impl From<(isize, isize)> for Substring {
fn from(input: (isize, isize)) -> Substring {
Substring(input.0, input.1)
}
}
struct SubstringText(String, String);
async fn operate(
args: CommandArgs,
@ -82,68 +102,23 @@ async fn operate(
let (Arguments { range, rest }, input) = args.process(&registry).await?;
let v: Vec<&str> = range.item.split(',').collect();
let start = match v[0] {
"" => 0,
_ => v[0].trim().parse().map_err(|_| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?,
};
let end = match v[1] {
"" => usize::max_value(),
_ => v[1].trim().parse().map_err(|_| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?,
};
if start > end {
return Err(ShellError::labeled_error(
"End must be greater than or equal to Start",
"End must be greater than or equal to Start",
name.span,
));
}
let options = Substring(start, end);
let column_paths: Vec<_> = rest;
let options = process_arguments(range, name)?.into();
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, &options, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, &options, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let options = options.clone();
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, &options, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)
@ -153,35 +128,180 @@ async fn operate(
}
fn action(input: &Value, options: &Substring, tag: impl Into<Tag>) -> Result<Value, ShellError> {
let tag = tag.into();
match &input.value {
UntaggedValue::Primitive(Primitive::Line(s))
| UntaggedValue::Primitive(Primitive::String(s)) => {
let start = options.0;
let end: usize = cmp::min(options.1, s.len());
let len: isize = s.len().try_into().map_err(|_| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
tag.span,
)
})?;
let out = {
if start > s.len() - 1 {
UntaggedValue::string("")
} else {
UntaggedValue::string(
s.chars().skip(start).take(end - start).collect::<String>(),
)
let start: isize = options.0;
let end: isize = options.1;
if start < len && end >= 0 {
match start.cmp(&end) {
Ordering::Equal => Ok(UntaggedValue::string("").into_value(tag)),
Ordering::Greater => Err(ShellError::labeled_error(
"End must be greater than or equal to Start",
"End must be greater than or equal to Start",
tag.span,
)),
Ordering::Less => {
let end: isize = cmp::min(options.1, len);
Ok(UntaggedValue::string(
s.chars()
.skip(start as usize)
.take((end - start) as usize)
.collect::<String>(),
)
.into_value(tag))
}
}
};
} else if start >= 0 && end <= 0 {
let end = options.1.abs();
let reversed = s
.chars()
.skip(start as usize)
.take((len - start) as usize)
.collect::<String>();
Ok(out.into_value(tag))
let reversed = if start == 0 {
reversed
} else {
s.chars().take(start as usize).collect::<String>()
};
let reversed = reversed
.chars()
.rev()
.take(end as usize)
.collect::<String>();
Ok(
UntaggedValue::string(reversed.chars().rev().collect::<String>())
.into_value(tag),
)
} else {
Ok(UntaggedValue::string("").into_value(tag))
}
}
other => {
let got = format!("got {}", other.type_name());
Err(ShellError::labeled_error(
"value is not string",
got,
tag.into().span,
tag.span,
))
}
}
}
fn process_arguments(range: Value, name: impl Into<Tag>) -> Result<(isize, isize), ShellError> {
let name = name.into();
let search = match &range.value {
UntaggedValue::Table(indexes) => {
if indexes.len() > 2 {
Err(ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
))
} else {
let idx: Vec<String> = indexes
.iter()
.map(|v| as_string(v).unwrap_or_else(|_| String::from("")))
.collect();
let start = idx
.get(0)
.ok_or_else(|| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?
.to_string();
let end = idx
.get(1)
.ok_or_else(|| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?
.to_string();
Ok(SubstringText(start, end))
}
}
UntaggedValue::Primitive(Primitive::String(indexes)) => {
let idx: Vec<&str> = indexes.split(',').collect();
let start = idx
.get(0)
.ok_or_else(|| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?
.to_string();
let end = idx
.get(1)
.ok_or_else(|| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?
.to_string();
Ok(SubstringText(start, end))
}
_ => Err(ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)),
}?;
let start = match &search {
SubstringText(start, _) if start == "" || start == "_" => 0,
SubstringText(start, _) => start.trim().parse().map_err(|_| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?,
};
let end = match &search {
SubstringText(_, end) if end == "" || end == "_" => isize::max_value(),
SubstringText(_, end) => end.trim().parse().map_err(|_| {
ShellError::labeled_error(
"could not perform substring",
"could not perform substring",
name.span,
)
})?,
};
Ok((start, end))
}
#[cfg(test)]
mod tests {
use super::{action, SubCommand, Substring};
@ -195,14 +315,55 @@ mod tests {
test_examples(SubCommand {})
}
struct Expectation<'a> {
options: (isize, isize),
expected: &'a str,
}
impl Expectation<'_> {
fn options(&self) -> Substring {
Substring(self.options.0, self.options.1)
}
}
fn expectation(word: &str, indexes: (isize, isize)) -> Expectation {
Expectation {
options: indexes,
expected: word,
}
}
#[test]
fn given_start_and_end_indexes() {
let word = string("andresS");
let expected = string("andres");
fn substrings_indexes() {
let word = string("andres");
let substring_options = Substring(0, 6);
let cases = vec![
expectation("a", (0, 1)),
expectation("an", (0, 2)),
expectation("and", (0, 3)),
expectation("andr", (0, 4)),
expectation("andre", (0, 5)),
expectation("andres", (0, 6)),
expectation("andres", (0, -6)),
expectation("ndres", (0, -5)),
expectation("dres", (0, -4)),
expectation("res", (0, -3)),
expectation("es", (0, -2)),
expectation("s", (0, -1)),
expectation("", (6, 0)),
expectation("s", (6, -1)),
expectation("es", (6, -2)),
expectation("res", (6, -3)),
expectation("dres", (6, -4)),
expectation("ndres", (6, -5)),
expectation("andres", (6, -6)),
];
let actual = action(&word, &substring_options, Tag::unknown()).unwrap();
assert_eq!(actual, expected);
for expectation in cases.iter() {
let expected = expectation.expected;
let actual = action(&word, &expectation.options(), Tag::unknown()).unwrap();
assert_eq!(actual, string(expected));
}
}
}

View File

@ -81,28 +81,17 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, &options, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, &options, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let options = options.clone();
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, &options, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)
@ -123,7 +112,13 @@ fn action(
let out = match DateTime::parse_from_str(s, dt) {
Ok(d) => UntaggedValue::date(d),
Err(_) => UntaggedValue::string(s),
Err(reason) => {
return Err(ShellError::labeled_error(
"could not parse as datetime",
reason.to_string(),
tag.into().span,
))
}
};
Ok(out.into_value(tag))
@ -166,4 +161,15 @@ mod tests {
_ => panic!("Didn't convert to date"),
}
}
#[test]
fn communicates_parsing_error_given_an_invalid_datetimelike_string() {
let date_str = string("16.11.1984 8:00 am Oops0000");
let fmt_options = DatetimeFormat("%d.%m.%Y %H:%M %P %z".to_string());
let actual = action(&date_str, &fmt_options, Tag::unknown());
assert!(actual.is_err());
}
}

View File

@ -65,27 +65,15 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)
@ -101,7 +89,13 @@ fn action(input: &Value, tag: impl Into<Tag>) -> Result<Value, ShellError> {
let other = s.trim();
let out = match BigDecimal::from_str(other) {
Ok(v) => UntaggedValue::decimal(v),
Err(_) => UntaggedValue::string(s),
Err(reason) => {
return Err(ShellError::labeled_error(
"could not parse as decimal",
reason.to_string(),
tag.into().span,
))
}
};
Ok(out.into_value(tag))
}
@ -138,4 +132,13 @@ mod tests {
let actual = action(&word, Tag::unknown()).unwrap();
assert_eq!(actual, expected);
}
#[test]
fn communicates_parsing_error_given_an_invalid_decimallike_string() {
let decimal_str = string("11.6anra");
let actual = action(&decimal_str, Tag::unknown());
assert!(actual.is_err());
}
}

View File

@ -65,27 +65,15 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)
@ -101,7 +89,13 @@ fn action(input: &Value, tag: impl Into<Tag>) -> Result<Value, ShellError> {
let other = s.trim();
let out = match BigInt::from_str(other) {
Ok(v) => UntaggedValue::int(v),
Err(_) => UntaggedValue::string(s),
Err(reason) => {
return Err(ShellError::labeled_error(
"could not parse as an integer",
reason.to_string(),
tag.into().span,
))
}
};
Ok(out.into_value(tag))
}
@ -137,4 +131,13 @@ mod tests {
let actual = action(&word, Tag::unknown()).unwrap();
assert_eq!(actual, expected);
}
#[test]
fn communicates_parsing_error_given_an_invalid_integerlike_string() {
let integer_str = string("36anra");
let actual = action(&integer_str, Tag::unknown());
assert!(actual.is_err());
}
}

View File

@ -5,12 +5,14 @@ use nu_protocol::ShellTypeName;
use nu_protocol::{
ColumnPath, Primitive, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
};
use nu_source::Tag;
use nu_source::{Tag, Tagged};
use nu_value_ext::ValueExt;
#[derive(Deserialize)]
struct Arguments {
rest: Vec<ColumnPath>,
#[serde(rename(deserialize = "char"))]
char_: Option<Tagged<char>>,
}
pub struct SubCommand;
@ -22,10 +24,17 @@ impl WholeStreamCommand for SubCommand {
}
fn signature(&self) -> Signature {
Signature::build("str trim").rest(
SyntaxShape::ColumnPath,
"optionally trim text by column paths",
)
Signature::build("str trim")
.rest(
SyntaxShape::ColumnPath,
"optionally trim text by column paths",
)
.named(
"char",
SyntaxShape::String,
"character to trim (default: whitespace)",
Some('c'),
)
}
fn usage(&self) -> &str {
@ -41,11 +50,18 @@ impl WholeStreamCommand for SubCommand {
}
fn examples(&self) -> Vec<Example> {
vec![Example {
description: "Trim contents",
example: "echo 'Nu shell ' | str trim",
result: Some(vec![Value::from("Nu shell")]),
}]
vec![
Example {
description: "Trim whitespace",
example: "echo 'Nu shell ' | str trim",
result: Some(vec![Value::from("Nu shell")]),
},
Example {
description: "Trim a specific character",
example: "echo '=== Nu shell ===' | str trim -c '=' | str trim",
result: Some(vec![Value::from("Nu shell")]),
},
]
}
}
@ -55,34 +71,23 @@ async fn operate(
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let (Arguments { rest }, input) = args.process(&registry).await?;
let (Arguments { rest, char_ }, input) = args.process(&registry).await?;
let column_paths: Vec<_> = rest;
let to_trim = char_.map(|tagged| tagged.item);
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, v.tag(), to_trim)?)
} else {
let mut ret = v;
for path in &column_paths {
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
Box::new(move |old| action(old, old.tag(), to_trim)),
)?;
}
ReturnSuccess::value(ret)
@ -91,11 +96,15 @@ async fn operate(
.to_output_stream())
}
fn action(input: &Value, tag: impl Into<Tag>) -> Result<Value, ShellError> {
fn action(input: &Value, tag: impl Into<Tag>, char_: Option<char>) -> Result<Value, ShellError> {
match &input.value {
UntaggedValue::Primitive(Primitive::Line(s))
| UntaggedValue::Primitive(Primitive::String(s)) => {
Ok(UntaggedValue::string(s.trim()).into_value(tag))
Ok(UntaggedValue::string(match char_ {
None => String::from(s.trim()),
Some(ch) => trim_char(s, ch, true, true),
})
.into_value(tag))
}
other => {
let got = format!("got {}", other.type_name());
@ -108,6 +117,33 @@ fn action(input: &Value, tag: impl Into<Tag>) -> Result<Value, ShellError> {
}
}
pub fn trim_char(from: &str, to_trim: char, leading: bool, trailing: bool) -> String {
let mut trimmed = String::from("");
let mut backlog = String::from("");
let mut at_left = true;
from.chars().for_each(|ch| match ch {
c if c == to_trim => {
if !(leading && at_left) {
if trailing {
backlog.push(c)
} else {
trimmed.push(c)
}
}
}
other => {
at_left = false;
if trailing {
trimmed.push_str(backlog.as_str());
backlog = String::from("");
}
trimmed.push(other);
}
});
trimmed
}
#[cfg(test)]
mod tests {
use super::{action, SubCommand};
@ -126,7 +162,7 @@ mod tests {
let word = string("andres ");
let expected = string("andres");
let actual = action(&word, Tag::unknown()).unwrap();
let actual = action(&word, Tag::unknown(), None).unwrap();
assert_eq!(actual, expected);
}
}

View File

@ -62,27 +62,15 @@ async fn operate(
Ok(input
.map(move |v| {
if column_paths.is_empty() {
match action(&v, v.tag()) {
Ok(out) => ReturnSuccess::value(out),
Err(err) => Err(err),
}
ReturnSuccess::value(action(&v, v.tag())?)
} else {
let mut ret = v;
for path in &column_paths {
let swapping = ret.swap_data_by_column_path(
ret = ret.swap_data_by_column_path(
path,
Box::new(move |old| action(old, old.tag())),
);
match swapping {
Ok(new_value) => {
ret = new_value;
}
Err(err) => {
return Err(err);
}
}
)?;
}
ReturnSuccess::value(ret)

View File

@ -1,113 +0,0 @@
use crate::commands::WholeStreamCommand;
use crate::data::TaggedListBuilder;
use crate::prelude::*;
use crate::utils::data_processing::{columns_sorted, t_sort};
use chrono::{DateTime, NaiveDate, Utc};
use nu_errors::ShellError;
use nu_protocol::{
Primitive, ReturnSuccess, Signature, SyntaxShape, TaggedDictBuilder, UntaggedValue, Value,
};
use nu_source::Tagged;
use nu_value_ext::get_data_by_key;
pub struct TSortBy;
#[derive(Deserialize)]
pub struct TSortByArgs {
#[serde(rename(deserialize = "show-columns"))]
show_columns: bool,
group_by: Option<Tagged<String>>,
#[allow(unused)]
split_by: Option<String>,
}
#[async_trait]
impl WholeStreamCommand for TSortBy {
fn name(&self) -> &str {
"t-sort-by"
}
fn signature(&self) -> Signature {
Signature::build("t-sort-by")
.switch(
"show-columns",
"Displays the column names sorted",
Some('c'),
)
.named(
"group_by",
SyntaxShape::String,
"the name of the column to group by",
Some('g'),
)
.named(
"split_by",
SyntaxShape::String,
"the name of the column within the grouped by table to split by",
Some('s'),
)
}
fn usage(&self) -> &str {
"Sort by the given columns."
}
async fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
t_sort_by(args, registry).await
}
}
async fn t_sort_by(
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
let registry = registry.clone();
let name = args.call_info.name_tag.clone();
let (
TSortByArgs {
show_columns,
group_by,
..
},
mut input,
) = args.process(&registry).await?;
let values: Vec<Value> = input.collect().await;
let column_grouped_by_name = if let Some(grouped_by) = group_by {
Some(grouped_by)
} else {
None
};
if show_columns {
Ok(futures::stream::iter(
columns_sorted(column_grouped_by_name, &values[0], &name)
.into_iter()
.map(move |label| {
ReturnSuccess::value(UntaggedValue::string(label.item).into_value(label.tag))
}),
)
.to_output_stream())
} else {
match t_sort(column_grouped_by_name, None, &values[0], name) {
Ok(sorted) => Ok(OutputStream::one(ReturnSuccess::value(sorted))),
Err(err) => Ok(OutputStream::one(Err(err))),
}
}
}
#[cfg(test)]
mod tests {
use super::TSortBy;
#[test]
fn examples_work_as_expected() {
use crate::examples::test as test_examples;
test_examples(TSortBy {})
}
}

View File

@ -187,7 +187,7 @@ fn values_to_entries(
StyledString::new(
(starting_idx + idx).to_string(),
TextStyle {
alignment: Alignment::Center,
alignment: Alignment::Right,
color: Some(ansi_term::Color::Green),
is_bold: true,
},

View File

@ -115,8 +115,8 @@ pub fn clone_tagged_value(v: &Value) -> Value {
UntaggedValue::Primitive(Primitive::Path(x)) => {
UntaggedValue::Primitive(Primitive::Path(x.clone()))
}
UntaggedValue::Primitive(Primitive::Bytes(b)) => {
UntaggedValue::Primitive(Primitive::Bytes(*b))
UntaggedValue::Primitive(Primitive::Filesize(b)) => {
UntaggedValue::Primitive(Primitive::Filesize(*b))
}
UntaggedValue::Primitive(Primitive::Date(d)) => {
UntaggedValue::Primitive(Primitive::Date(*d))
@ -134,7 +134,7 @@ fn to_string_tagged_value(v: &Value) -> Result<String, ShellError> {
match &v.value {
UntaggedValue::Primitive(Primitive::String(_))
| UntaggedValue::Primitive(Primitive::Line(_))
| UntaggedValue::Primitive(Primitive::Bytes(_))
| UntaggedValue::Primitive(Primitive::Filesize(_))
| UntaggedValue::Primitive(Primitive::Boolean(_))
| UntaggedValue::Primitive(Primitive::Decimal(_))
| UntaggedValue::Primitive(Primitive::Path(_))

Some files were not shown because too many files have changed in this diff Show More