Compare commits

...

134 Commits

Author SHA1 Message Date
791f7dd9c3 Bump to 0.12.0 (#1538) 2020-04-01 06:25:21 +13:00
a4c1b092ba Add configurations for table headers (#1537)
* Add configurations for table headers

* lint

Co-authored-by: Amanita-Muscaria <nope>
2020-03-31 12:19:48 +13:00
6e71c1008d Change get to remove blanks (#1534)
Remove blank values when getting a column of values
2020-03-30 15:36:21 +13:00
906d0b920f A few improvements to du implementation: (#1533)
1. Fixed a bug where `--all` wasn't showing files at the root directories.
2. More use of `Result`'s `map` and `map_err` methods.
3. Making tables be homogeneous so one can, for example, `get directories`.
2020-03-29 21:16:09 -04:00
efbf4f48c6 Fix poor message for executable that user doesn't have permissi… (#1535)
Previously, if the user didn't have the appropriate permissions to execute the
binary/script, they would see "command not found", which is confusing.

This commit eliminates the `which` crate in favour of `ichwh`, which deals
better with permissions by not dealing with them at all! This is closer to the
behaviour of `which` in many shells. Permission checks are then left up to the
caller to deal with.
2020-03-29 21:15:55 -04:00
2ddab3e8ce Some small improvements to du readability.
Mostly, making more use of `map` and `map_err` in `Result`. One benefit
is that at least one location had duplicated logic for how to map the
error, which is no longer the case after this commit.
2020-03-29 17:03:01 -04:00
35dc7438a5 Make use of interruptible stream in various places 2020-03-29 17:03:01 -04:00
2a54ee0c54 Introduce InterruptibleStream type.
An interruptible stream can query an `AtomicBool. If that bool is true,
the stream will no longer produce any values.

Also introducing the `Interruptible` trait, which extends any `Stream`
with the `interruptible` function, to simplify the construction and
allow chaining.
2020-03-29 17:03:01 -04:00
cad2741e9e Split input and output streams into separate modules 2020-03-29 17:03:01 -04:00
ae5f3c8210 WIP: 1486/first row as headers (#1530)
* headers plugin

* Remove plugin

* Add non-functioning headers command

* Add ability to extract headers from first row

* Refactor header extraction

* Rebuild indexmap with proper headers

* Rebuild result properly

* Compiling, probably wrapped too much?

* Refactoring

* Deal with case of empty header cell

* Deal with case of empty header cell

* Fix formatting

* Fix linting, attempt 2.

* Move whole_stream_command(Headers) to more appropriate section

* ... more linting

* Return Err(ShellError...) instead of panic, yield each row instead of entire table

* Insert Column[index] if no header info is found.

* Update error description

* Add initial test

* Add tests for headers command

* Lint test cases in headers

* Change ShellError for headers, Add sample_headers file to utils.rs

* Add empty sheet to test file

* Revert "Add empty sheet to test file"

This reverts commit a4bf38a31d.

* Show error message when given empty table
2020-03-29 15:05:57 +13:00
a5e97ca549 Respect CARGO_TARGET_DIR when set (#1528)
This makes the `binaries` function respect the `CARGO_TARGET_DIR` environment variable when set. If it's not present it falls back to the regular target directory used by Cargo.
2020-03-27 17:13:59 -04:00
06f87cfbe8 Add support for removing multiple files at once (#1526) 2020-03-25 16:19:01 -04:00
d4e78c6f47 Improve the rotated row wrap (#1524) 2020-03-25 06:27:16 +13:00
3653400ebc testing fix to matrix to define all variables (#1522)
there is currently a bug with invalid syntax for some of the
docker build steps, and I think this is because there are build
variables in the matrix that are not defined. This PR will
attempt to resolve this issue by defining all missing variables
for each row in the matrix.

Signed-off-by: vsoch <vsochat@stanford.edu>
2020-03-24 16:20:39 +13:00
81a48d6d0e Fix '/' and '..' not being valid mv targets (#1519)
* Fix '/' and '..' not being valid mv targets

If `/` or `../` is specified as the destination for `mv`, it will fail with an error message saying it's not a valid destination. This fixes it to account for the fact that `Path::file_name` return `None` when the file name evaluates to `/` or `..`. It will only take the slow(er) path if `Path::file_name` returns `None` in its initial check.

Fixes #1291

* Add test
2020-03-24 14:00:48 +13:00
f030ab3f12 Add experimental auto-rotate (#1516) 2020-03-23 09:55:30 +13:00
0dc0c6a10a Add quickstart option to Docker section in README (#1515) 2020-03-23 09:18:50 +13:00
53c8185af3 Fixes the crash for ps --full in Windows (#1514)
* Fixes the crash for `ps --full` in Windows

* Update ps.rs
2020-03-23 08:28:02 +13:00
36b5d063c1 Simplify and improve listing for which. (#1510)
* Simplified implementation
* Show executables, even if the current user doesn't have permissions to
  execute them.
2020-03-22 09:11:39 -04:00
a7ec00a037 Add documentation for from-ics and from-vcf (#1509) 2020-03-21 14:50:13 +13:00
918822ae0d Fix numeric comparison with nothing (#1508) 2020-03-21 11:02:49 +13:00
ab5e24a0e7 WIP: Add vcard/ical support (#1504)
* Initial from-ical implementation

* Initial from-vcard implementation

* Rename from-ics and from-vcf for autoconvert

* Remove redundant clones

* Add from-vcf and from-ics tests

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-03-21 08:35:09 +13:00
b5ea522f0e Add a --full mode to ps (#1507)
* Add a --full mode to ps

* Use a slightly older heim
2020-03-20 20:53:49 +13:00
afa963fd50 Add is_dir check to auto-cd (#1506)
* Add markdown output

* Add is_dir() check
2020-03-20 16:57:36 +13:00
1e343ff00c Add markdown output (#1503) 2020-03-20 08:18:24 +13:00
21a543a901 Make sum plugin as internal command. (#1501) 2020-03-18 18:46:00 -05:00
390deb4ff7 Windows needs to remember auto-cd paths when changing drives (#1500)
* Windows needs to remember auto-cd paths when changing drives

* Windows needs to remember auto-cd paths when changing drives
2020-03-18 15:10:45 +13:00
1c4cb30d64 Add documentation for skip and skip-while (#1499) 2020-03-18 14:22:35 +13:00
1ec2ec72b5 Add automatic change directory (#1496)
* Allow automatic cd in cli mode

* Set correct priority for auto-cd and add test
2020-03-18 07:13:38 +13:00
0d244a9701 Open fails silently, fix #1493 (#1495)
* Fix #1493

The error was wrongfully discarded

* Run cargo fmt
2020-03-17 17:40:04 +13:00
b36d21e76f Infer types from regular delimited plain text unstructured files. (#1494)
* Infer types from regular delimited plain text unstructured files.

* Nothing resolves to an empty string.
2020-03-16 15:50:45 -05:00
d8c4565413 Csv errors (#1490)
* Add error message for csv parsing failures

* Add csv error prettyfier

* Improve readability of the error

Line 2: error is easier to understand than:
Line 2, error

* Remove unnecessary use of the format! macro

Replacing it with .to_string() fixes a clippy warning

* Improve consistency with JSON parsing errors
2020-03-16 12:32:02 -05:00
22ba4c2a2f Add svg support to to-html (#1492) 2020-03-16 20:19:18 +13:00
8d19b21b9f Custom canonicalize method on Filesystem Shell. (#1485)
* Custom canonicalize method for FilesystemShell.

* Use custom canonicalize method.
Fixed missing import.

* Move function body to already impl body.

* Create test that aims to resolve.
2020-03-16 19:28:18 +13:00
45a3afdc79 Update Cargo.toml 2020-03-16 06:12:28 +13:00
2d078849cb Add simple to-html output and bump version (#1487) 2020-03-15 16:04:44 +13:00
b6363f3ce1 Added new flag '--all/-a' for Ls, also refactor some code (#1483)
* Utility function to detect hidden folders.
Implemented for Unix and Windows.

* Rename function argument.

* Revert "Rename function argument."

This reverts commit e7ab70f0f0.

* Add flag '--all/-a' to Ls

* Rename function argument.

* Check if flag '--all/-a' is present and path is hidden.
Replace match with map_err for glob result.
Remove redundancy in stream body.
Included comments on new stream body.
Replace async_stream::stream with async_stream::try_stream.
Minor tweaks to is_empty_dir.
Fix and refactor is_hidden_dir.

* Fix "implicit" bool coerse

* Fixed clippy errors
2020-03-14 06:27:04 +13:00
5ca9e12b7f Fix whitespace and typos (#1481)
* Remove EOL whitespace in files other than docs

* Break paragraphs into lines

See http://rhodesmill.org/brandon/2012/one-sentence-per-line/ for the rationale

* Fix various typos

* Remove EOL whitespace in docs/commands/*.md
2020-03-14 06:23:41 +13:00
5b0b2f1ddd Fixes #1204 : sys | get host.users displays the same user (#1480)
account twice while only one exists (macOS)

- renamed host.users to host.sessions
2020-03-12 14:01:55 +13:00
3afb53b8ce fix typo in calc command documentation (#1477)
minimumum -> minimum
2020-03-11 11:20:22 -04:00
b40d16310c More relaxed file modes for now. (#1476) 2020-03-11 13:19:15 +13:00
d3718d00db Merge shuffle nu plugin as core command. (#1475) 2020-03-10 17:00:08 -05:00
f716f61fc1 Update Cargo.lock 2020-03-11 08:53:26 +13:00
b2ce669791 Update Cargo.toml 2020-03-11 08:51:53 +13:00
cd155f63e1 Update Cargo.toml 2020-03-11 08:51:17 +13:00
9eaa6877f3 Update Cargo.toml 2020-03-11 08:50:51 +13:00
a6b6afbca9 Update Cargo.toml 2020-03-11 08:08:13 +13:00
62666bebc9 Bump to 0.11.0 (#1474) 2020-03-11 06:34:19 +13:00
d1fcce0cd3 Fixes #1427: Prints help message with -h switch (#1454)
For some commands like `which` -h flag would trigger an error asking for
missing required parameters instead of printing the help message as it
does with --help. This commit adds a check in the command parser to
avoid that.
2020-03-11 05:59:50 +13:00
a2443fbe02 Remove unused parsing logic. (#1473)
* Remove unused parsing logic.

* Run tokens iteration tests baseline.

* Pass lint.

* lifetimes can be elided without being explicit.
2020-03-10 04:31:42 -05:00
db16b56fe1 Columnpath support when passing fields for formatting. (#1472) 2020-03-10 01:55:03 -05:00
54bf671a50 Fix deleting / showing ls named pipes and other fs objects no… (#1461)
* Fix deleting named pipes
* Use std::os::unix::fs::FileTypeExt to show correct type for unix-specific fs objects; Fix formatting

Co-authored-by: Linards Kalvāns <linards.kalvans@twino.eu>
2020-03-09 09:02:53 -04:00
755d0e648b Eliminate some compiler warnings (#1468)
- Unnecessary parentheses
- Deprecated `description()` method
2020-03-09 08:19:07 +13:00
e440d8c939 Bump some deps (#1467) 2020-03-09 08:18:44 +13:00
01dd358a18 Don't emit a newline in autoview. (#1466)
The extra newline character makes it hard to use nu as part of an
external processing pipeline, since the extra character could taint the
results. For example:

```
$ nu -c 'echo test | xxd'
00000000: 7465 7374                                test
```

versus

```
nu -c 'echo test' | xxd
00000000: 7465 7374 0a                             test.
```
2020-03-09 08:18:24 +13:00
50fb97f6b7 Merge env into $nu and simplify table/get (#1463) 2020-03-08 18:33:30 +13:00
ebf139f5e5 Auto-detect string / binary in save command (#1459)
* Auto-detect string / binary in save command

* Linter
2020-03-08 07:33:29 +13:00
8925ca5da3 Move to bytes/string hybrid codec (#1457)
* WIP: move to bytes codec

* Progress on adding collect helpers

* Progress on adding collect helpers

* Add in line splitting back to lines

* Lines outputting line primitives

* Close to ready?

* Finish fixing lines

* clippy fixes

* fmt fixes

* removed unused code

* Cleanup a few bits

* Cleanup a few bits

* Cleanup a few more bits

* Fix failing test with corrected test case
2020-03-07 05:06:39 +13:00
287652573b Fix and refactor cd for Filesystem Shell. (#1452)
* Fix and refactor cd for Filesystem Shell.
Reorder check conditions, don't check existence twice.
If building for unix check exec bit on folder.

* Import PermissionsExt only on unix target.

* It seems that this is the correct way?
2020-03-06 20:13:47 +13:00
db24ad8f36 Add --num parameter to limit the number of output lines (#1455)
Add `--num` parameter to limit the numer of returned elements
2020-03-05 05:26:46 -05:00
f88674f353 Nu internals are logged under nu filter. (#1451) 2020-03-05 05:18:53 -05:00
59cb0ba381 Color appropiately commands. (#1453) 2020-03-04 23:22:42 -05:00
c4cfab5e16 Make feature options available downstream to nu-cli subcrate. (#1450) 2020-03-04 15:31:12 -05:00
b2c5af457e Move most of the root package into a subcrate. (#1445)
This improves incremental build time when working on what was previously
the root package. For example, previously all plugins would be rebuilt
with a change to `src/commands/classified/external.rs`, but now only
`nu-cli` will have to be rebuilt (and anything that depends on it).
2020-03-04 13:58:20 -05:00
c731a5b628 Columns can be renamed. (#1447) 2020-03-03 16:01:24 -05:00
f97f9d4af3 Update deps locklfile (#1446)
Update deps lockfile
2020-03-03 15:34:22 -05:00
ed7d3fed66 Add shuffle plugin (#1443)
* Add shuffle plugin

see #1437

* Change plugin to integrate into nu structure and build system
2020-03-03 08:44:12 +13:00
7304d06c0b Use threads to avoid blocking reads/writes in externals. (#1440)
In particular, one thing that we can't (properly) do before this commit
is consuming an infinite input stream. For example:

```
yes | grep y | head -n10
```

will give 10 "y"s in most shells, but blocks indefinitely in nu. This PR
resolves that by doing blocking I/O in threads, and reducing the `await`
calls we currently have in our pipeline code.
2020-03-02 06:19:09 +13:00
ca615d9389 Bump to 0.10.1 (#1442) 2020-03-01 20:59:13 +13:00
6d096206b6 Add support for compound shorthand flags (#1414)
* Break multicharacter shorthand flags into single character flags

* Remove shorthand flag test
2020-03-01 13:20:42 +13:00
2a8cb24309 Add support for downloading unsupported mime types (#1441) 2020-03-01 13:14:36 +13:00
8d38743e27 Add docs for debug (#1438)
* Add docs for `debug`

* Put debug docs in right folder
Also fixed minor spacing problem
2020-03-01 04:09:28 +13:00
eabfa2de54 Let ls ignore permission errors (#1435)
* Create a function to create an empty directory entry

* Print an empty directory entry if permission is denied

* Fix rustfmt whitespace issues.

* Made metadata optional for `dir_entry_dict`.

Removed `empty_dir_entry_dict` as its not needed anymore.
2020-02-29 14:33:52 +13:00
a86a0abb90 Plugin documentation (#1431)
* Add very basic documentation. Need to play with rest of the api to figure out what it does

* Add some documentation to more of the Plugin API methods

* fmt
2020-02-24 15:28:46 +13:00
adcda450d5 Update LICENSE 2020-02-21 10:49:46 +13:00
147b9d4436 Add Better-TOML (#1417) 2020-02-19 16:59:42 -05:00
c43a58d9d6 Fix incorrect display for zero-size files (#1422) 2020-02-19 09:57:58 -05:00
e38442782e Command documentation for du (#1416) 2020-02-19 09:55:22 +13:00
b98f893217 add a touch command (#1399) 2020-02-19 09:54:32 +13:00
bd6556eee1 Use proper file extension for uniq command docs (#1411) 2020-02-18 09:37:46 -05:00
18d988d4c8 Restrict short-hand flag detection to exact match. (#1406) 2020-02-18 01:58:30 -05:00
0f7c723672 Bump version to 0.10.0 (#1403) 2020-02-18 16:56:09 +13:00
afce2fd0f9 Revert "Display rows in the same table regardless of their column order given they are equal. (#1392)" (#1401)
This reverts commit 4fd9974204.
2020-02-17 17:34:37 -08:00
4fd9974204 Display rows in the same table regardless of their column order given they are equal. (#1392) 2020-02-16 20:35:01 -05:00
71615f77a7 Fix minor typo in calc command error (#1395) 2020-02-16 16:02:41 -05:00
9bc5022c9c Force a \n at the end of a stdout stream (#1391)
* Force a \n at the end of a stdout stream

* clippy
2020-02-14 18:15:32 -08:00
552848b8b9 Leave raw mode correctly. (#1388)
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-02-14 17:31:21 -05:00
8ae8ebd107 Add support for multiline script files (#1386)
* Add support for multiline script files

* clippy
2020-02-13 21:24:18 -08:00
473e9f9422 Tiny improvement to sys (#1385) 2020-02-13 08:33:55 -08:00
96985aa692 Fix invalid shorthand flag (#1384) 2020-02-13 07:47:34 -08:00
0961da406d Add string to datetime to str plugin (#1381)
* Add string to datetime to str plugin

* Test string to date/time conversion
2020-02-13 07:47:04 -08:00
84927d52b5 Refuse internal command execution given unexpected arguments. (#1383) 2020-02-13 02:34:43 -05:00
73312b506f Finer grained parsing and coloring command tail. (#1382) 2020-02-12 20:20:19 -05:00
c1bec3b443 Return error on a divide by zero (#1376)
Return error on a divide by zero
2020-02-12 08:38:04 -05:00
c0be02a434 Short-hand flags (#1378)
* typo fixes

* Change signature to take in short-hand flags

* update help information

* Parse short-hand flags as their long counterparts

* lints

* Modified a couple tests to use shorthand flags
2020-02-11 18:24:31 -08:00
2ab8d035e6 External it and nu variable column path fetch support. (#1379) 2020-02-11 18:25:56 -05:00
24094acee9 Allow switch flags anywhere in the pipeline. (#1375) 2020-02-11 03:49:00 -05:00
0b2be52bb5 Only add quotes if not in Windows (which adds its own?) (#1374)
* Only add quotes if not in Windows (which adds its own?)

* Only add quotes if not in Windows (which adds its own?)
2020-02-10 23:07:44 -08:00
6a371802b4 Add block size to du (#1341)
* Add block size to du

* Change blocks to physical size

* Use path instead of strings for file/directory names

* Why don't I just use paths instead of strings anyway?

* shorten physical size and apparent size to physical and apparent resp.
2020-02-10 12:32:18 -08:00
29ccb9f5cd Ensure stable plugins get installed. (#1373) 2020-02-10 15:32:10 -05:00
20ab125861 bump version (#1370) 2020-02-10 09:18:00 -08:00
fb532f3f4e Prototype shebang support (#1368)
* Add shebang support to nu.

* Move test file

* Add test for scripts

Co-authored-by: Jason Gedge <jason.gedge@shopify.com>
2020-02-10 08:49:45 -08:00
a29d52158e Do not panic when failing to decode lines from external stdout (#1364) 2020-02-10 07:37:48 -08:00
dc50e61f26 Switch stdin redirect to manual. Add test (#1367) 2020-02-09 22:55:07 -08:00
a2668e3327 Add some nu_source docs for meta.rs (#1366)
* Add some docs for meta.rs

* add better explanation for Span merging

* Add some doc tests - not sure how to get them to run

* get rid of doc comments for the temporary method

* add doc test for is_unknown

* fmt
2020-02-09 18:08:14 -08:00
e606407d79 Add error codes to -c (#1361) 2020-02-08 20:04:53 -08:00
5f4fae5b06 Pipeline sink refactor (#1359)
* Refactor pipeline ahead of block changes. Add '-c' commandline option

* Update pipelining an error value

* Fmt

* Clippy

* Add stdin redirect for -c flag

* Add stdin redirect for -c flag
2020-02-08 18:24:33 -08:00
3687603799 Only spawn external once when no $it argument (#1358) 2020-02-08 17:57:05 -08:00
643b532537 Fixed mv not throwing error when the source path was invalid (#1351)
* Fixed mv not throwing error when the source path was invalid

* Fixed failing test

* Fixed another lint error

* Fix $PATH conflicts in .gitpod.Dockerfile (#1349)

- Use the correct user for gitpod Dockerfile.
- Remove unneeded packages (curl, rustc) from gitpod Dockerfile.

* Added test to check for the error

* Fixed linting error

* Fixed mv not moving files on Windows. (#1342)

Move files correctly in windows.

* Fixed mv not throwing error when the source path was invalid

* Fixed failing test

* Fixed another lint error

* Added test to check for the error

* Fixed linting error

* Changed error message

* Typo and fixed test

Co-authored-by: Sean Hellum <seanhellum45@gmail.com>
2020-02-07 12:40:48 -05:00
ed86b1fbe8 Fixed mv not moving files on Windows. (#1342)
Move files correctly in windows.
2020-02-07 11:24:01 -05:00
44a114111e Fix $PATH conflicts in .gitpod.Dockerfile (#1349)
- Use the correct user for gitpod Dockerfile.
- Remove unneeded packages (curl, rustc) from gitpod Dockerfile.
2020-02-06 15:20:18 -05:00
812a76d588 Update more futures-preview to futures (#1346) 2020-02-05 20:28:42 -08:00
e3be849c2a Futures v0.3 upgrade (#1344)
* Upgrade futures, async-stream, and futures_codec

These were the last three dependencies on futures-preview. `nu` itself
is now fully dependent on `futures@0.3`, as opposed to `futures-preview`
alpha.

Because the update to `futures` from `0.3.0-alpha.19` to `0.3.0` removed
the `Stream` implementation of `VecDeque` ([changelog][changelog]), most
commands that convert a `VecDeque` to an `OutputStream` broke and had to
be fixed.

The current solution is to now convert `VecDeque`s to a `Stream` via
`futures::stream::iter`. However, it may be useful for `futures` to
create an `IntoStream` trait, implemented on the `std::collections` (or
really any `IntoIterator`). If something like this happends, it may be
worthwhile to update the trait implementations on `OutputStream` and
refactor these commands again.

While upgrading `futures_codec`, we remove a custom implementation of
`LinesCodec`, as one has been added to the library. There's also a small
refactor to make the stream output more idiomatic.

[changelog]: https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md#030---2019-11-5

* Upgrade sys & ps plugin dependencies

They were previously dependent on `futures-preview`, and `nu_plugin_ps`
was dependent on an old version of `futures-timer`.

* Remove dependency on futures-timer from nu

* Update Cargo.lock

* Fix formatting

* Revert fmt regressions

CI is still on 1.40.0, but the latest rustfmt v1.41.0 has changes to the
`val @ pattern` syntax, causing the linting job to fail.

* Fix clippy warnings
2020-02-05 19:46:48 -08:00
ba1b67c072 Attempt rustup update on each PR (#1345)
* Attempt update on each PR

* Update fmt
2020-02-05 19:28:49 -08:00
fa910b95b7 Have from-ssv not fail for header-only inputs (#1334) 2020-02-05 11:54:14 -08:00
427bde83f7 Allow cp to overwrite existing files (#1339) 2020-02-05 01:54:05 -05:00
7a0bc6bc46 Opt-out unused heim features from sys/ps plugins. (#1335) 2020-02-04 01:51:14 -05:00
c6da56949c Add support for plugin names containing numbers (#1321)
* Add ability to have numbers in plugin name. Plugin must start with alphabetic char

* remove the first character as alphabetic requirement

* Update cli.rs

Going ahead and changing to plus to prevent issue notryanb found

* Update cli.rs

Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
2020-02-01 22:08:38 -08:00
5b398d2ed2 Adding cross-platform kill command (#1326)
* Adding kill command, unclean code

* Removing old comments

* Added quiet option, supports variable number of ids

* Made it per_item_command, calling commands directly without the shell
2020-02-01 10:46:28 -08:00
dcdfa2a866 Improve tests and labeling in FilesystemShell (#1305)
Additional `ls` command tests and better FilesystemShell error and label messages.
2020-02-01 03:34:34 -05:00
9474fa1ea5 Improved code in du command (#1320)
Made the code a little easier to read
2020-02-01 03:32:06 -05:00
49a1385543 Make tests work from directory names with spaces (#1325) 2020-01-31 22:12:56 -08:00
6427ea2331 Update Cargo.lock for ichwh fix (#1312)
`ichwh@0.3.1` fixes a bug that causes path searches to fail. We update
`Cargo.lock` to fix this.

Resolves #1207
2020-01-31 22:11:42 -08:00
3610baa227 Default plugins are independent and called from Nu. (#1322) 2020-01-31 17:45:33 -05:00
4e201d20ca Paths from Nu config take priority over external paths. (#1319) 2020-01-31 14:19:47 -05:00
1fa21ff056 Exclude images to reduce crate by 3MB (#1316)
Maybe there are more candidates for exclusion, but 'images/'
seemed obviously unnecessary.

Something I started realizing lately is that cargo puts most
of the root directory into the crate archive, causing huge
crates to appear on crates.io.

Now that I am in China, I do seem to notice every kilobyte.
2020-01-31 10:38:26 -05:00
0bbd12e37f Improve the default help message (#1313) 2020-01-30 20:13:14 -08:00
7df8fdfb28 Rename the now-deprecated add command docs into insert comm… (#1307) 2020-01-30 08:15:20 -05:00
6a39cd8546 Add docs for the calc command (#1290) 2020-01-29 08:34:54 -05:00
dc3370b103 Make a calc command (#1280) 2020-01-29 08:34:36 -05:00
ac5ad45783 Pretty Nu print default, pretty print regular secondary as raw flag. (#1302) 2020-01-29 02:46:54 -05:00
8ef5c47515 Update cargo flags (#1295)
* Update cargo flags

See https://github.com/nushell/nushell.github.io/issues/29

* Update to suggested flag
2020-01-28 22:44:49 -08:00
5b19bebe7d Isolate environment state changes into Host. (#1296)
Moves the state changes for setting and removing environment variables
into the context's host as opposed to calling `std::env::*` directly
from anywhere else.

Introduced FakeHost to prevent environemnt state changes leaking
between unit tests and cause random test failures.
2020-01-29 00:40:06 -05:00
2c529cd849 Fix bug where --with-symlink-targets would not display the targets column (#1300) 2020-01-28 21:36:20 -08:00
409 changed files with 11940 additions and 8282 deletions

View File

@ -38,8 +38,8 @@ steps:
if [ "$(uname)" == "Darwin" ]; then
curl https://sh.rustup.rs -sSf | sh -s -- -y --no-modify-path --default-toolchain "stable"
export PATH=$HOME/.cargo/bin:$PATH
rustup update
fi
rustup update
rustc -Vv
echo "##vso[task.prependpath]$HOME/.cargo/bin"
rustup component add rustfmt
@ -59,4 +59,3 @@ steps:
- bash: cargo fmt --all -- --check
condition: eq(variables['style'], 'fmt')
displayName: Lint

View File

@ -1,3 +1,3 @@
[build]
#rustflags = ["--cfg", "coloring_in_tokens"]
#rustflags = ["--cfg", "data_processing_primitives"]

1
.dockerignore Normal file
View File

@ -0,0 +1 @@
target

View File

@ -52,15 +52,15 @@ jobs:
- glibc
- musl
include:
- { tag: alpine, base-image: alpine, arch: x86_64-unknown-linux-musl, plugin: true }
- { tag: slim, base-image: 'debian:stable-slim', arch: x86_64-unknown-linux-gnu, plugin: true }
- { tag: debian, base-image: debian, arch: x86_64-unknown-linux-gnu, plugin: true }
- { tag: glibc-busybox, base-image: 'busybox:glibc', arch: x86_64-unknown-linux-gnu, use-patch: true }
- { tag: musl-busybox, base-image: 'busybox:musl', arch: x86_64-unknown-linux-musl, }
- { tag: musl-distroless, base-image: 'gcr.io/distroless/static', arch: x86_64-unknown-linux-musl, }
- { tag: glibc-distroless, base-image: 'gcr.io/distroless/cc', arch: x86_64-unknown-linux-gnu, use-patch: true }
- { tag: glibc, base-image: scratch, arch: x86_64-unknown-linux-gnu, }
- { tag: musl, base-image: scratch, arch: x86_64-unknown-linux-musl, }
- { tag: alpine, base-image: alpine, arch: x86_64-unknown-linux-musl, plugin: true, use-patch: false}
- { tag: slim, base-image: 'debian:stable-slim', arch: x86_64-unknown-linux-gnu, plugin: true, use-patch: false}
- { tag: debian, base-image: debian, arch: x86_64-unknown-linux-gnu, plugin: true, use-patch: false}
- { tag: glibc-busybox, base-image: 'busybox:glibc', arch: x86_64-unknown-linux-gnu, plugin: false, use-patch: true }
- { tag: musl-busybox, base-image: 'busybox:musl', arch: x86_64-unknown-linux-musl, plugin: false, use-patch: false}
- { tag: musl-distroless, base-image: 'gcr.io/distroless/static', arch: x86_64-unknown-linux-musl, plugin: false, use-patch: false}
- { tag: glibc-distroless, base-image: 'gcr.io/distroless/cc', arch: x86_64-unknown-linux-gnu, plugin: false, use-patch: true }
- { tag: glibc, base-image: scratch, arch: x86_64-unknown-linux-gnu, plugin: false, use-patch: false}
- { tag: musl, base-image: scratch, arch: x86_64-unknown-linux-musl, plugin: false, use-patch: false}
steps:
- uses: actions/checkout@v1
- uses: actions/download-artifact@master

1
.gitignore vendored
View File

@ -4,6 +4,7 @@
history.txt
tests/fixtures/nuplayground
crates/*/target
# Debian/Ubuntu
debian/.debhelper/
debian/debhelper-build-stamp

12
.gitpod.Dockerfile vendored
View File

@ -1,7 +1,9 @@
FROM gitpod/workspace-full
USER root
RUN apt-get update && apt-get install -y libssl-dev \
USER gitpod
RUN sudo apt-get update && \
sudo apt-get install -y \
libssl-dev \
libxcb-composite0-dev \
pkg-config \
curl \
rustc
pkg-config

View File

@ -26,3 +26,4 @@ vscode:
- serayuzgur.crates@0.4.7:HMkoguLcXp9M3ud7ac3eIw==
- belfz.search-crates-io@1.2.1:kSLnyrOhXtYPjQpKnMr4eQ==
- vadimcn.vscode-lldb@1.4.5:lwHCNwtm0kmOBXeQUIPGMQ==
- bungcip.better-toml@0.3.2:3QfgGxxYtGHfJKQU7H0nEw==

1428
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,8 +1,8 @@
[package]
name = "nu"
version = "0.9.0"
authors = ["Yehuda Katz <wycats@gmail.com>", "Jonathan Turner <jonathan.d.turner@gmail.com>", "Andrés N. Robalino <andres@androbtech.com>"]
description = "A shell for the GitHub era"
version = "0.12.0"
authors = ["The Nu Project Contributors"]
description = "A new kind of shell"
license = "MIT"
edition = "2018"
readme = "README.md"
@ -10,146 +10,68 @@ default-run = "nu"
repository = "https://github.com/nushell/nushell"
homepage = "https://www.nushell.sh"
documentation = "https://www.nushell.sh/book/"
exclude = ["images"]
[workspace]
members = [
"crates/nu-macros",
"crates/nu-errors",
"crates/nu-source",
"crates/nu_plugin_average",
"crates/nu_plugin_binaryview",
"crates/nu_plugin_fetch",
"crates/nu_plugin_inc",
"crates/nu_plugin_match",
"crates/nu_plugin_post",
"crates/nu_plugin_ps",
"crates/nu_plugin_str",
"crates/nu_plugin_sum",
"crates/nu_plugin_sys",
"crates/nu_plugin_textview",
"crates/nu_plugin_tree",
"crates/nu-protocol",
"crates/nu-plugin",
"crates/nu-parser",
"crates/nu-value-ext",
"crates/nu-build"
]
members = ["crates/*/"]
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
nu-source = {version = "0.9.0", path = "./crates/nu-source"}
nu-plugin = {version = "0.9.0", path = "./crates/nu-plugin"}
nu-protocol = {version = "0.9.0", path = "./crates/nu-protocol"}
nu-errors = {version = "0.9.0", path = "./crates/nu-errors"}
nu-parser = {version = "0.9.0", path = "./crates/nu-parser"}
nu-value-ext = {version = "0.9.0", path = "./crates/nu-value-ext"}
nu_plugin_average = {version = "0.9.0", path = "./crates/nu_plugin_average", optional=true}
nu_plugin_binaryview = {version = "0.9.0", path = "./crates/nu_plugin_binaryview", optional=true}
nu_plugin_fetch = {version = "0.9.0", path = "./crates/nu_plugin_fetch", optional=true}
nu_plugin_inc = {version = "0.9.0", path = "./crates/nu_plugin_inc", optional=true}
nu_plugin_match = {version = "0.9.0", path = "./crates/nu_plugin_match", optional=true}
nu_plugin_post = {version = "0.9.0", path = "./crates/nu_plugin_post", optional=true}
nu_plugin_ps = {version = "0.9.0", path = "./crates/nu_plugin_ps", optional=true}
nu_plugin_str = {version = "0.9.0", path = "./crates/nu_plugin_str", optional=true}
nu_plugin_sum = {version = "0.9.0", path = "./crates/nu_plugin_sum", optional=true}
nu_plugin_sys = {version = "0.9.0", path = "./crates/nu_plugin_sys", optional=true}
nu_plugin_textview = {version = "0.9.0", path = "./crates/nu_plugin_textview", optional=true}
nu_plugin_tree = {version = "0.9.0", path = "./crates/nu_plugin_tree", optional=true}
nu-macros = { version = "0.9.0", path = "./crates/nu-macros" }
nu-cli = { version = "0.12.0", path = "./crates/nu-cli" }
nu-source = { version = "0.12.0", path = "./crates/nu-source" }
nu-plugin = { version = "0.12.0", path = "./crates/nu-plugin" }
nu-protocol = { version = "0.12.0", path = "./crates/nu-protocol" }
nu-errors = { version = "0.12.0", path = "./crates/nu-errors" }
nu-parser = { version = "0.12.0", path = "./crates/nu-parser" }
nu-value-ext = { version = "0.12.0", path = "./crates/nu-value-ext" }
nu_plugin_average = { version = "0.12.0", path = "./crates/nu_plugin_average", optional=true }
nu_plugin_binaryview = { version = "0.12.0", path = "./crates/nu_plugin_binaryview", optional=true }
nu_plugin_fetch = { version = "0.12.0", path = "./crates/nu_plugin_fetch", optional=true }
nu_plugin_inc = { version = "0.12.0", path = "./crates/nu_plugin_inc", optional=true }
nu_plugin_match = { version = "0.12.0", path = "./crates/nu_plugin_match", optional=true }
nu_plugin_post = { version = "0.12.0", path = "./crates/nu_plugin_post", optional=true }
nu_plugin_ps = { version = "0.12.0", path = "./crates/nu_plugin_ps", optional=true }
nu_plugin_str = { version = "0.12.0", path = "./crates/nu_plugin_str", optional=true }
nu_plugin_sys = { version = "0.12.0", path = "./crates/nu_plugin_sys", optional=true }
nu_plugin_textview = { version = "0.12.0", path = "./crates/nu_plugin_textview", optional=true }
nu_plugin_tree = { version = "0.12.0", path = "./crates/nu_plugin_tree", optional=true }
nu-macros = { version = "0.12.0", path = "./crates/nu-macros" }
query_interface = "0.3.5"
typetag = "0.1.4"
rustyline = "6.0.0"
chrono = { version = "0.4.10", features = ["serde"] }
derive-new = "0.5.8"
prettytable-rs = "0.8.0"
itertools = "0.8.2"
ansi_term = "0.12.1"
nom = "5.0.1"
dunce = "1.0.0"
indexmap = { version = "1.3.1", features = ["serde-1"] }
byte-unit = "3.0.3"
base64 = "0.11"
futures-preview = { version = "=0.3.0-alpha.19", features = ["compat", "io-compat"] }
async-stream = "0.1.2"
futures_codec = "0.2.5"
num-traits = "0.2.11"
term = "0.5.2"
bytes = "0.4.12"
log = "0.4.8"
pretty_env_logger = "0.3.1"
serde = { version = "1.0.104", features = ["derive"] }
bson = { version = "0.14.0", features = ["decimal128"] }
serde_json = "1.0.44"
serde-hjson = "0.9.1"
serde_yaml = "0.8"
serde_bytes = "0.11.3"
getset = "0.0.9"
language-reporting = "0.4.0"
app_dirs = "1.2.1"
csv = "1.1"
toml = "0.5.6"
clap = "2.33.0"
git2 = { version = "0.11.0", default_features = false }
dirs = "2.0.2"
glob = "0.3.0"
ctrlc = "3.1.3"
roxmltree = "0.9.0"
nom_locate = "1.0.0"
nom-tracable = "0.4.1"
unicode-xid = "0.2.0"
serde_ini = "0.2.0"
pretty-hex = "0.1.1"
hex = "0.4"
tempfile = "3.1.0"
which = "3.1.0"
ichwh = "0.3"
textwrap = {version = "0.11.0", features = ["term_size"]}
shellexpand = "1.1.1"
pin-utils = "0.1.0-alpha.4"
num-bigint = { version = "0.2.5", features = ["serde"] }
bigdecimal = { version = "0.1.0", features = ["serde"] }
serde_urlencoded = "0.6.1"
trash = "1.0.0"
regex = "1"
cfg-if = "0.1"
strip-ansi-escapes = "0.1.0"
calamine = "0.16"
umask = "0.1"
futures-util = "0.3.1"
termcolor = "1.1.0"
natural = "0.3.0"
parking_lot = "0.10.0"
futures-timer = "1.0.2"
clipboard = {version = "0.5", optional = true }
ptree = {version = "0.2" }
starship = { version = "0.33.1", optional = true}
heim = {version = "0.0.9", optional = true}
battery = {version = "0.7.5", optional = true}
syntect = {version = "3.2.0", optional = true }
crossterm = { version = "0.16.0", optional = true }
onig_sys = { version = "=69.1.0", optional = true }
crossterm = {version = "0.14.2", optional = true}
url = {version = "2.1.1", optional = true}
semver = { version = "0.9.0", optional = true }
syntect = { version = "3.2.0", optional = true }
url = { version = "2.1.1", optional = true }
[target.'cfg(unix)'.dependencies]
users = "0.9"
clap = "2.33.0"
ctrlc = "3.1.4"
dunce = "1.0.0"
futures = { version = "0.3", features = ["compat", "io-compat"] }
log = "0.4.8"
pretty_env_logger = "0.4.0"
[dev-dependencies]
pretty_assertions = "0.6.1"
nu-test-support = { version = "0.12.0", path = "./crates/nu-test-support" }
[build-dependencies]
toml = "0.5.6"
serde = { version = "1.0.105", features = ["derive"] }
nu-build = { version = "0.12.0", path = "./crates/nu-build" }
[features]
# Test executables
test-bins = []
default = ["sys", "ps", "textview", "inc", "str"]
stable = ["default", "starship-prompt", "binaryview", "match", "tree", "average", "sum", "post", "fetch", "clipboard"]
stable = ["default", "starship-prompt", "binaryview", "match", "tree", "average", "post", "fetch", "clipboard-cli"]
# Default
sys = ["heim", "battery"]
ps = ["heim"]
textview = ["crossterm", "syntect", "onig_sys", "url"]
inc = ["nu_plugin_inc"]
textview = ["crossterm", "syntect", "onig_sys", "url", "nu_plugin_textview"]
sys = ["nu_plugin_sys"]
ps = ["nu_plugin_ps"]
inc = ["semver", "nu_plugin_inc"]
str = ["nu_plugin_str"]
# Stable
@ -158,28 +80,11 @@ binaryview = ["nu_plugin_binaryview"]
fetch = ["nu_plugin_fetch"]
match = ["nu_plugin_match"]
post = ["nu_plugin_post"]
starship-prompt = ["starship"]
sum = ["nu_plugin_sum"]
trace = ["nu-parser/trace"]
tree = ["nu_plugin_tree"]
[dependencies.rusqlite]
version = "0.20.0"
features = ["bundled", "blob"]
[dev-dependencies]
pretty_assertions = "0.6.1"
nu-test-support = { version = "0.9.0", path = "./crates/nu-test-support" }
[build-dependencies]
toml = "0.5.6"
serde = { version = "1.0.104", features = ["derive"] }
nu-build = { version = "0.9.0", path = "./crates/nu-build" }
[lib]
name = "nu"
doctest = false
path = "src/lib.rs"
clipboard-cli = ["nu-cli/clipboard-cli"]
starship-prompt = ["nu-cli/starship-prompt"]
[[bin]]
name = "fail"
@ -196,6 +101,16 @@ name = "cococo"
path = "crates/nu-test-support/src/bins/cococo.rs"
required-features = ["test-bins"]
[[bin]]
name = "nonu"
path = "crates/nu-test-support/src/bins/nonu.rs"
required-features = ["test-bins"]
[[bin]]
name = "iecho"
path = "crates/nu-test-support/src/bins/iecho.rs"
required-features = ["test-bins"]
# Core plugins that ship with `cargo install nu` by default
# Currently, Cargo limits us to installing only one binary
# unless we use [[bin]], so we use this as a workaround
@ -224,6 +139,37 @@ name = "nu_plugin_core_sys"
path = "src/plugins/nu_plugin_core_sys.rs"
required-features = ["sys"]
# Stable plugins
[[bin]]
name = "nu_plugin_stable_average"
path = "src/plugins/nu_plugin_stable_average.rs"
required-features = ["average"]
[[bin]]
name = "nu_plugin_stable_fetch"
path = "src/plugins/nu_plugin_stable_fetch.rs"
required-features = ["fetch"]
[[bin]]
name = "nu_plugin_stable_binaryview"
path = "src/plugins/nu_plugin_stable_binaryview.rs"
required-features = ["binaryview"]
[[bin]]
name = "nu_plugin_stable_match"
path = "src/plugins/nu_plugin_stable_match.rs"
required-features = ["match"]
[[bin]]
name = "nu_plugin_stable_post"
path = "src/plugins/nu_plugin_stable_post.rs"
required-features = ["post"]
[[bin]]
name = "nu_plugin_stable_tree"
path = "src/plugins/nu_plugin_stable_tree.rs"
required-features = ["tree"]
# Main nu binary
[[bin]]
name = "nu"

View File

@ -1,6 +1,6 @@
MIT License
Copyright (c) 2019 Yehuda Katz, Jonathan Turner
Copyright (c) 2019 - 2020 Yehuda Katz, Jonathan Turner
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@ -13,15 +13,22 @@ A new type of shell.
# Status
This project has reached a minimum-viable product level of quality. While contributors dogfood it as their daily driver, it may be unstable for some commands. Future releases will work to fill out missing features and improve stability. Its design is also subject to change as it matures.
This project has reached a minimum-viable product level of quality.
While contributors dogfood it as their daily driver, it may be unstable for some commands.
Future releases will work to fill out missing features and improve stability.
Its design is also subject to change as it matures.
Nu comes with a set of built-in commands (listed below). If a command is unknown, the command will shell-out and execute it (using cmd on Windows or bash on Linux and macOS), correctly passing through stdin, stdout, and stderr, so things like your daily git workflows and even `vim` will work just fine.
Nu comes with a set of built-in commands (listed below).
If a command is unknown, the command will shell-out and execute it (using cmd on Windows or bash on Linux and macOS), correctly passing through stdin, stdout, and stderr, so things like your daily git workflows and even `vim` will work just fine.
# Learning more
There are a few good resources to learn about Nu. There is a [book](https://www.nushell.sh/book/) about Nu that is currently in progress. The book focuses on using Nu and its core concepts.
There are a few good resources to learn about Nu.
There is a [book](https://www.nushell.sh/book/) about Nu that is currently in progress.
The book focuses on using Nu and its core concepts.
If you're a developer who would like to contribute to Nu, we're also working on a [book for developers](https://www.nushell.sh/contributor-book/) to help you get started. There are also [good first issues](https://github.com/nushell/nushell/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22) to help you dive in.
If you're a developer who would like to contribute to Nu, we're also working on a [book for developers](https://www.nushell.sh/contributor-book/) to help you get started.
There are also [good first issues](https://github.com/nushell/nushell/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22) to help you dive in.
We also have an active [Discord](https://discord.gg/NtAbbGn) and [Twitter](https://twitter.com/nu_shell) if you'd like to come and chat with us.
@ -58,11 +65,21 @@ cargo install nu
You can also build Nu yourself with all the bells and whistles (be sure to have installed the [dependencies](https://www.nushell.sh/book/en/installation.html#dependencies) for your platform), once you have checked out this repo with git:
```
cargo build --all --features=stable
cargo build --workspace --features=stable
```
## Docker
### Quickstart
Want to try Nu right away? Execute the following to get started.
```bash
docker run -it quay.io/nushell/nu:latest
```
### Guide
If you want to pull a pre-built container, you can browse tags for the [nushell organization](https://quay.io/organization/nushell)
on Quay.io. Pulling a container would come down to:
@ -107,11 +124,18 @@ The second container is a bit smaller if the size is important to you.
# Philosophy
Nu draws inspiration from projects like PowerShell, functional programming languages, and modern CLI tools. Rather than thinking of files and services as raw streams of text, Nu looks at each input as something with structure. For example, when you list the contents of a directory, what you get back is a table of rows, where each row represents an item in that directory. These values can be piped through a series of steps, in a series of commands called a 'pipeline'.
Nu draws inspiration from projects like PowerShell, functional programming languages, and modern CLI tools.
Rather than thinking of files and services as raw streams of text, Nu looks at each input as something with structure.
For example, when you list the contents of a directory, what you get back is a table of rows, where each row represents an item in that directory.
These values can be piped through a series of steps, in a series of commands called a 'pipeline'.
## Pipelines
In Unix, it's common to pipe between commands to split up a sophisticated command over multiple steps. Nu takes this a step further and builds heavily on the idea of _pipelines_. Just as the Unix philosophy, Nu allows commands to output from stdout and read from stdin. Additionally, commands can output structured data (you can think of this as a third kind of stream). Commands that work in the pipeline fit into one of three categories:
In Unix, it's common to pipe between commands to split up a sophisticated command over multiple steps.
Nu takes this a step further and builds heavily on the idea of _pipelines_.
Just as the Unix philosophy, Nu allows commands to output from stdout and read from stdin.
Additionally, commands can output structured data (you can think of this as a third kind of stream).
Commands that work in the pipeline fit into one of three categories:
* Commands that produce a stream (eg, `ls`)
* Commands that filter a stream (eg, `where type == "Directory"`)
@ -135,13 +159,15 @@ Commands are separated by the pipe symbol (`|`) to denote a pipeline flowing lef
────┴───────────┴───────────┴──────────┴────────┴──────────────┴────────────────
```
Because most of the time you'll want to see the output of a pipeline, `autoview` is assumed. We could have also written the above:
Because most of the time you'll want to see the output of a pipeline, `autoview` is assumed.
We could have also written the above:
```
/home/jonathan/Source/nushell(master)> ls | where type == Directory
```
Being able to use the same commands and compose them differently is an important philosophy in Nu. For example, we could use the built-in `ps` command as well to get a list of the running processes, using the same `where` as above.
Being able to use the same commands and compose them differently is an important philosophy in Nu.
For example, we could use the built-in `ps` command as well to get a list of the running processes, using the same `where` as above.
```text
/home/jonathan/Source/nushell(master)> ps | where cpu > 0
@ -157,7 +183,8 @@ Being able to use the same commands and compose them differently is an important
## Opening files
Nu can load file and URL contents as raw text or as structured data (if it recognizes the format). For example, you can load a .toml file as structured data and explore it:
Nu can load file and URL contents as raw text or as structured data (if it recognizes the format).
For example, you can load a .toml file as structured data and explore it:
```
/home/jonathan/Source/nushell(master)> open Cargo.toml
@ -210,19 +237,26 @@ To set one of these variables, you can use `config --set`. For example:
## Shells
Nu will work inside of a single directory and allow you to navigate around your filesystem by default. Nu also offers a way of adding additional working directories that you can jump between, allowing you to work in multiple directories at the same time.
Nu will work inside of a single directory and allow you to navigate around your filesystem by default.
Nu also offers a way of adding additional working directories that you can jump between, allowing you to work in multiple directories at the same time.
To do so, use the `enter` command, which will allow you create a new "shell" and enter it at the specified path. You can toggle between this new shell and the original shell with the `p` (for previous) and `n` (for next), allowing you to navigate around a ring buffer of shells. Once you're done with a shell, you can `exit` it and remove it from the ring buffer.
To do so, use the `enter` command, which will allow you create a new "shell" and enter it at the specified path.
You can toggle between this new shell and the original shell with the `p` (for previous) and `n` (for next), allowing you to navigate around a ring buffer of shells.
Once you're done with a shell, you can `exit` it and remove it from the ring buffer.
Finally, to get a list of all the current shells, you can use the `shells` command.
## Plugins
Nu supports plugins that offer additional functionality to the shell and follow the same structured data model that built-in commands use. This allows you to extend nu for your needs.
Nu supports plugins that offer additional functionality to the shell and follow the same structured data model that built-in commands use.
This allows you to extend nu for your needs.
There are a few examples in the `plugins` directory.
Plugins are binaries that are available in your path and follow a `nu_plugin_*` naming convention. These binaries interact with nu via a simple JSON-RPC protocol where the command identifies itself and passes along its configuration, which then makes it available for use. If the plugin is a filter, data streams to it one element at a time, and it can stream data back in return via stdin/stdout. If the plugin is a sink, it is given the full vector of final data and is given free reign over stdin/stdout to use as it pleases.
Plugins are binaries that are available in your path and follow a `nu_plugin_*` naming convention.
These binaries interact with nu via a simple JSON-RPC protocol where the command identifies itself and passes along its configuration, which then makes it available for use.
If the plugin is a filter, data streams to it one element at a time, and it can stream data back in return via stdin/stdout.
If the plugin is a sink, it is given the full vector of final data and is given free reign over stdin/stdout to use as it pleases.
# Goals
@ -244,5 +278,5 @@ You can find a list of Nu commands, complete with documentation, in [quick comma
# License
The project is made available under the MIT license. See "LICENSE" for more information.
The project is made available under the MIT license. See the `LICENSE` file for more information.

View File

@ -1,6 +1,6 @@
[package]
name = "nu-build"
version = "0.9.0"
version = "0.12.0"
authors = ["Yehuda Katz <wycats@gmail.com>", "Jonathan Turner <jonathan.d.turner@gmail.com>", "Andrés N. Robalino <andres@androbtech.com>"]
edition = "2018"
description = "Core build system for nushell"

110
crates/nu-cli/Cargo.toml Normal file
View File

@ -0,0 +1,110 @@
[package]
name = "nu-cli"
version = "0.12.0"
authors = ["Yehuda Katz <wycats@gmail.com>", "Jonathan Turner <jonathan.d.turner@gmail.com>", "Andrés N. Robalino <andres@androbtech.com>"]
description = "CLI for nushell"
edition = "2018"
license = "MIT"
[lib]
doctest = false
[dependencies]
nu-source = { version = "0.12.0", path = "../nu-source" }
nu-plugin = { version = "0.12.0", path = "../nu-plugin" }
nu-protocol = { version = "0.12.0", path = "../nu-protocol" }
nu-errors = { version = "0.12.0", path = "../nu-errors" }
nu-parser = { version = "0.12.0", path = "../nu-parser" }
nu-value-ext = { version = "0.12.0", path = "../nu-value-ext" }
nu-macros = { version = "0.12.0", path = "../nu-macros" }
nu-test-support = { version = "0.12.0", path = "../nu-test-support" }
ansi_term = "0.12.1"
app_dirs = "1.2.1"
async-stream = "0.2"
base64 = "0.12.0"
bigdecimal = { version = "0.1.0", features = ["serde"] }
bson = { version = "0.14.1", features = ["decimal128"] }
byte-unit = "3.0.3"
bytes = "0.5.4"
calamine = "0.16"
cfg-if = "0.1"
chrono = { version = "0.4.11", features = ["serde"] }
clap = "2.33.0"
csv = "1.1"
ctrlc = "3.1.4"
derive-new = "0.5.8"
dirs = "2.0.2"
dunce = "1.0.0"
filesize = "0.1.0"
futures = { version = "0.3", features = ["compat", "io-compat"] }
futures-util = "0.3.4"
futures_codec = "0.4"
getset = "0.1.0"
git2 = { version = "0.13.0", default_features = false }
glob = "0.3.0"
hex = "0.4"
htmlescape = "0.3.1"
ical = "0.6.*"
ichwh = "0.3"
indexmap = { version = "1.3.2", features = ["serde-1"] }
itertools = "0.9.0"
language-reporting = "0.4.0"
log = "0.4.8"
meval = "0.2"
natural = "0.5.0"
nom = "5.0.1"
nom-tracable = "0.4.1"
nom_locate = "1.0.0"
num-bigint = { version = "0.2.6", features = ["serde"] }
num-traits = "0.2.11"
parking_lot = "0.10.0"
pin-utils = "0.1.0-alpha.4"
pretty-hex = "0.1.1"
pretty_env_logger = "0.4.0"
prettytable-rs = "0.8.0"
ptree = {version = "0.2" }
query_interface = "0.3.5"
rand = "0.7"
regex = "1"
roxmltree = "0.10.0"
rustyline = "6.0.0"
serde = { version = "1.0.105", features = ["derive"] }
serde-hjson = "0.9.1"
serde_bytes = "0.11.3"
serde_ini = "0.2.0"
serde_json = "1.0.48"
serde_urlencoded = "0.6.1"
serde_yaml = "0.8"
shellexpand = "2.0.0"
strip-ansi-escapes = "0.1.0"
tempfile = "3.1.0"
term = "0.5.2"
termcolor = "1.1.0"
textwrap = {version = "0.11.0", features = ["term_size"]}
toml = "0.5.6"
trash = "1.0.0"
typetag = "0.1.4"
umask = "0.1"
unicode-xid = "0.2.0"
clipboard = { version = "0.5", optional = true }
starship = { version = "0.38.0", optional = true }
[target.'cfg(unix)'.dependencies]
users = "0.9"
[dependencies.rusqlite]
version = "0.21.0"
features = ["bundled", "blob"]
[dev-dependencies]
pretty_assertions = "0.6.1"
[build-dependencies]
nu-build = { version = "0.12.0", path = "../nu-build" }
[features]
stable = []
starship-prompt = ["starship"]
clipboard-cli = ["clipboard"]

View File

@ -1,3 +1,4 @@
use crate::commands::classified::external::{MaybeTextCodec, StringOrBinary};
use crate::commands::classified::pipeline::run_pipeline;
use crate::commands::plugin::JsonRpc;
use crate::commands::plugin::{PluginCommand, PluginSink};
@ -6,13 +7,14 @@ use crate::context::Context;
#[cfg(not(feature = "starship-prompt"))]
use crate::git::current_branch;
use crate::prelude::*;
use futures_codec::FramedRead;
use nu_errors::ShellError;
use nu_parser::hir::Expression;
use nu_parser::{
hir, ClassifiedCommand, ClassifiedPipeline, InternalCommand, PipelineShape, SpannedToken,
ClassifiedCommand, ClassifiedPipeline, ExternalCommand, PipelineShape, SpannedToken,
TokensIterator,
};
use nu_protocol::{Signature, Value};
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
use log::{debug, log_enabled, trace};
use rustyline::error::ReadlineError;
@ -96,48 +98,31 @@ fn load_plugin(path: &std::path::Path, context: &mut Context) -> Result<(), Shel
}
fn search_paths() -> Vec<std::path::PathBuf> {
use std::env;
let mut search_paths = Vec::new();
#[cfg(debug_assertions)]
{
// Use our debug plugins in debug mode
let mut path = std::path::PathBuf::from(".");
path.push("target");
path.push("debug");
if path.exists() {
search_paths.push(path);
// Automatically add path `nu` is in as a search path
if let Ok(exe_path) = env::current_exe() {
if let Some(exe_dir) = exe_path.parent() {
search_paths.push(exe_dir.to_path_buf());
}
}
#[cfg(not(debug_assertions))]
{
use std::env;
match env::var_os("PATH") {
Some(paths) => {
search_paths = env::split_paths(&paths).collect::<Vec<_>>();
search_paths.extend(env::split_paths(&paths).collect::<Vec<_>>());
}
None => println!("PATH is not defined in the environment."),
}
// Use our release plugins in release mode
let mut path = std::path::PathBuf::from(".");
path.push("target");
path.push("release");
if path.exists() {
search_paths.push(path);
}
}
// permit Nu finding and picking up development plugins
// if there are any first.
search_paths.reverse();
search_paths
}
fn load_plugins(context: &mut Context) -> Result<(), ShellError> {
pub fn load_plugins(context: &mut Context) -> Result<(), ShellError> {
let opts = glob::MatchOptions {
case_sensitive: false,
require_literal_separator: false,
@ -147,7 +132,7 @@ fn load_plugins(context: &mut Context) -> Result<(), ShellError> {
for path in search_paths() {
let mut pattern = path.to_path_buf();
pattern.push(std::path::Path::new("nu_plugin_[a-z]*"));
pattern.push(std::path::Path::new("nu_plugin_[a-z0-9][a-z0-9]*"));
match glob::glob_with(&pattern.to_string_lossy(), opts) {
Err(_) => {}
@ -173,14 +158,14 @@ fn load_plugins(context: &mut Context) -> Result<(), ShellError> {
{
bin_name
.chars()
.all(|c| c.is_ascii_alphabetic() || c == '_' || c == '.')
.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '.')
}
#[cfg(not(windows))]
{
bin_name
.chars()
.all(|c| c.is_ascii_alphabetic() || c == '_')
.all(|c| c.is_ascii_alphanumeric() || c == '_')
}
};
@ -240,10 +225,9 @@ fn create_default_starship_config() -> Option<toml::Value> {
Some(toml::Value::Table(map))
}
/// The entry point for the CLI. Will register all known internal commands, load experimental commands, load plugins, then prepare the prompt and line reader for input.
pub async fn cli() -> Result<(), Box<dyn Error>> {
let mut syncer = crate::env::environment_syncer::EnvironmentSyncer::new();
pub fn create_default_context(
syncer: &mut crate::env::environment_syncer::EnvironmentSyncer,
) -> Result<Context, Box<dyn Error>> {
syncer.load_environment();
let mut context = Context::basic()?;
@ -259,17 +243,19 @@ pub async fn cli() -> Result<(), Box<dyn Error>> {
per_item_command(Ls),
per_item_command(Du),
whole_stream_command(Cd),
whole_stream_command(Env),
per_item_command(Remove),
per_item_command(Open),
whole_stream_command(Config),
per_item_command(Help),
per_item_command(History),
whole_stream_command(Save),
per_item_command(Touch),
per_item_command(Cpy),
whole_stream_command(Date),
per_item_command(Calc),
per_item_command(Mkdir),
per_item_command(Move),
per_item_command(Kill),
whole_stream_command(Version),
whole_stream_command(Clear),
whole_stream_command(What),
@ -319,18 +305,24 @@ pub async fn cli() -> Result<(), Box<dyn Error>> {
whole_stream_command(Default),
whole_stream_command(SkipWhile),
whole_stream_command(Range),
whole_stream_command(Rename),
whole_stream_command(Uniq),
// Table manipulation
whole_stream_command(Shuffle),
whole_stream_command(Wrap),
whole_stream_command(Pivot),
whole_stream_command(Headers),
// Data processing
whole_stream_command(Histogram),
whole_stream_command(Sum),
// File format output
whole_stream_command(ToBSON),
whole_stream_command(ToCSV),
whole_stream_command(ToHTML),
whole_stream_command(ToJSON),
whole_stream_command(ToSQLite),
whole_stream_command(ToDB),
whole_stream_command(ToMarkdown),
whole_stream_command(ToTOML),
whole_stream_command(ToTSV),
whole_stream_command(ToURL),
@ -351,6 +343,8 @@ pub async fn cli() -> Result<(), Box<dyn Error>> {
whole_stream_command(FromXML),
whole_stream_command(FromYAML),
whole_stream_command(FromYML),
whole_stream_command(FromIcs),
whole_stream_command(FromVcf),
]);
cfg_if::cfg_if! {
@ -372,6 +366,55 @@ pub async fn cli() -> Result<(), Box<dyn Error>> {
}
}
Ok(context)
}
pub async fn run_pipeline_standalone(
pipeline: String,
redirect_stdin: bool,
context: &mut Context,
) -> Result<(), Box<dyn Error>> {
let line = process_line(Ok(pipeline), context, redirect_stdin, false).await;
match line {
LineResult::Success(line) => {
let error_code = {
let errors = context.current_errors.clone();
let errors = errors.lock();
if errors.len() > 0 {
1
} else {
0
}
};
context.maybe_print_errors(Text::from(line));
if error_code != 0 {
std::process::exit(error_code);
}
}
LineResult::Error(line, err) => {
context.with_host(|host| {
print_err(err, host, &Text::from(line.clone()));
});
context.maybe_print_errors(Text::from(line));
std::process::exit(1);
}
_ => {}
}
Ok(())
}
/// The entry point for the CLI. Will register all known internal commands, load experimental commands, load plugins, then prepare the prompt and line reader for input.
pub async fn cli() -> Result<(), Box<dyn Error>> {
let mut syncer = crate::env::environment_syncer::EnvironmentSyncer::new();
let mut context = create_default_context(&mut syncer)?;
let _ = load_plugins(&mut context);
let config = Config::builder().color_mode(ColorMode::Forced).build();
@ -480,7 +523,7 @@ pub async fn cli() -> Result<(), Box<dyn Error>> {
initial_command = None;
}
let line = process_line(readline, &mut context).await;
let line = process_line(readline, &mut context, false, true).await;
// Check the config to see if we need to update the path
// TODO: make sure config is cached so we don't path this load every call
@ -559,7 +602,12 @@ enum LineResult {
}
/// Process the line by parsing the text to turn it into commands, classify those commands so that we understand what is being called in the pipeline, and then run this pipeline
async fn process_line(readline: Result<String, ReadlineError>, ctx: &mut Context) -> LineResult {
async fn process_line(
readline: Result<String, ReadlineError>,
ctx: &mut Context,
redirect_stdin: bool,
cli_mode: bool,
) -> LineResult {
match &readline {
Ok(line) if line.trim() == "" => LineResult::Success(line.clone()),
@ -577,37 +625,131 @@ async fn process_line(readline: Result<String, ReadlineError>, ctx: &mut Context
debug!("=== Parsed ===");
debug!("{:#?}", result);
let mut pipeline = classify_pipeline(&result, ctx, &Text::from(line));
let pipeline = classify_pipeline(&result, &ctx, &Text::from(line));
if let Some(failure) = pipeline.failed {
return LineResult::Error(line.to_string(), failure.into());
}
let should_push = match pipeline.commands.list.last() {
Some(ClassifiedCommand::External(_)) => false,
_ => true,
};
// There's a special case to check before we process the pipeline:
// If we're giving a path by itself
// ...and it's not a command in the path
// ...and it doesn't have any arguments
// ...and we're in the CLI
// ...then change to this directory
if cli_mode && pipeline.commands.list.len() == 1 {
if let ClassifiedCommand::External(ExternalCommand {
ref name, ref args, ..
}) = pipeline.commands.list[0]
{
if dunce::canonicalize(name).is_ok()
&& PathBuf::from(name).is_dir()
&& ichwh::which(name).await.unwrap_or(None).is_none()
&& args.list.is_empty()
{
// Here we work differently if we're in Windows because of the expected Windows behavior
#[cfg(windows)]
{
if name.ends_with(':') {
// This looks like a drive shortcut. We need to a) switch drives and b) go back to the previous directory we were viewing on that drive
// But first, we need to save where we are now
let current_path = ctx.shell_manager.path();
if should_push {
pipeline
.commands
.list
.push(ClassifiedCommand::Internal(InternalCommand {
name: "autoview".to_string(),
name_tag: Tag::unknown(),
args: hir::Call::new(
Box::new(
Expression::synthetic_string("autoview").into_expr(Span::unknown()),
),
None,
None,
Span::unknown(),
),
}));
let split_path: Vec<_> = current_path.split(':').collect();
if split_path.len() > 1 {
ctx.windows_drives_previous_cwd
.lock()
.insert(split_path[0].to_string(), current_path);
}
match run_pipeline(pipeline, ctx, None, line).await {
Ok(_) => LineResult::Success(line.to_string()),
let name = name.to_uppercase();
let new_drive: Vec<_> = name.split(':').collect();
if let Some(val) =
ctx.windows_drives_previous_cwd.lock().get(new_drive[0])
{
ctx.shell_manager.set_path(val.to_string());
return LineResult::Success(line.to_string());
} else {
ctx.shell_manager.set_path(name.to_string());
return LineResult::Success(line.to_string());
}
} else {
ctx.shell_manager.set_path(name.to_string());
return LineResult::Success(line.to_string());
}
}
#[cfg(not(windows))]
{
ctx.shell_manager.set_path(name.to_string());
return LineResult::Success(line.to_string());
}
}
}
}
let input_stream = if redirect_stdin {
let file = futures::io::AllowStdIo::new(std::io::stdin());
let stream = FramedRead::new(file, MaybeTextCodec).map(|line| {
if let Ok(line) = line {
match line {
StringOrBinary::String(s) => Ok(Value {
value: UntaggedValue::Primitive(Primitive::String(s)),
tag: Tag::unknown(),
}),
StringOrBinary::Binary(b) => Ok(Value {
value: UntaggedValue::Primitive(Primitive::Binary(
b.into_iter().collect(),
)),
tag: Tag::unknown(),
}),
}
} else {
panic!("Internal error: could not read lines of text from stdin")
}
});
Some(stream.to_input_stream())
} else {
None
};
match run_pipeline(pipeline, ctx, input_stream, line).await {
Ok(Some(input)) => {
// Running a pipeline gives us back a stream that we can then
// work through. At the top level, we just want to pull on the
// values to compute them.
use futures::stream::TryStreamExt;
let context = RunnableContext {
input,
shell_manager: ctx.shell_manager.clone(),
host: ctx.host.clone(),
ctrl_c: ctx.ctrl_c.clone(),
commands: ctx.registry.clone(),
name: Tag::unknown(),
source: Text::from(String::new()),
};
if let Ok(mut output_stream) = crate::commands::autoview::autoview(context) {
loop {
match output_stream.try_next().await {
Ok(Some(ReturnSuccess::Value(Value {
value: UntaggedValue::Error(e),
..
}))) => return LineResult::Error(line.to_string(), e),
Ok(Some(_item)) => {
if ctx.ctrl_c.load(Ordering::SeqCst) {
break;
}
}
Ok(None) => break,
Err(e) => return LineResult::Error(line.to_string(), e),
}
}
}
LineResult::Success(line.to_string())
}
Ok(None) => LineResult::Success(line.to_string()),
Err(err) => LineResult::Error(line.to_string(), err),
}
}

View File

@ -7,6 +7,7 @@ mod to_delimited_data;
pub(crate) mod append;
pub(crate) mod args;
pub(crate) mod autoview;
pub(crate) mod calc;
pub(crate) mod cd;
pub(crate) mod classified;
pub(crate) mod clip;
@ -22,7 +23,6 @@ pub(crate) mod du;
pub(crate) mod echo;
pub(crate) mod edit;
pub(crate) mod enter;
pub(crate) mod env;
#[allow(unused)]
pub(crate) mod evaluate_by;
pub(crate) mod exit;
@ -30,6 +30,7 @@ pub(crate) mod first;
pub(crate) mod format;
pub(crate) mod from_bson;
pub(crate) mod from_csv;
pub(crate) mod from_ics;
pub(crate) mod from_ini;
pub(crate) mod from_json;
pub(crate) mod from_ods;
@ -38,11 +39,13 @@ pub(crate) mod from_ssv;
pub(crate) mod from_toml;
pub(crate) mod from_tsv;
pub(crate) mod from_url;
pub(crate) mod from_vcf;
pub(crate) mod from_xlsx;
pub(crate) mod from_xml;
pub(crate) mod from_yaml;
pub(crate) mod get;
pub(crate) mod group_by;
pub(crate) mod headers;
pub(crate) mod help;
pub(crate) mod histogram;
pub(crate) mod history;
@ -68,10 +71,12 @@ pub(crate) mod range;
#[allow(unused)]
pub(crate) mod reduce_by;
pub(crate) mod reject;
pub(crate) mod rename;
pub(crate) mod reverse;
pub(crate) mod rm;
pub(crate) mod save;
pub(crate) mod shells;
pub(crate) mod shuffle;
pub(crate) mod size;
pub(crate) mod skip;
pub(crate) mod skip_while;
@ -79,13 +84,16 @@ pub(crate) mod sort_by;
pub(crate) mod split_by;
pub(crate) mod split_column;
pub(crate) mod split_row;
pub(crate) mod sum;
#[allow(unused)]
pub(crate) mod t_sort_by;
pub(crate) mod table;
pub(crate) mod tags;
pub(crate) mod to_bson;
pub(crate) mod to_csv;
pub(crate) mod to_html;
pub(crate) mod to_json;
pub(crate) mod to_md;
pub(crate) mod to_sqlite;
pub(crate) mod to_toml;
pub(crate) mod to_tsv;
@ -102,11 +110,12 @@ pub(crate) mod wrap;
pub(crate) use autoview::Autoview;
pub(crate) use cd::Cd;
pub(crate) use command::{
per_item_command, whole_stream_command, Command, PerItemCommand, RawCommandArgs,
UnevaluatedCallInfo, WholeStreamCommand,
per_item_command, whole_stream_command, Command, PerItemCommand, UnevaluatedCallInfo,
WholeStreamCommand,
};
pub(crate) use append::Append;
pub(crate) use calc::Calc;
pub(crate) use compact::Compact;
pub(crate) use config::Config;
pub(crate) use count::Count;
@ -117,10 +126,12 @@ pub(crate) use default::Default;
pub(crate) use du::Du;
pub(crate) use echo::Echo;
pub(crate) use edit::Edit;
pub(crate) mod kill;
pub(crate) use kill::Kill;
pub(crate) mod clear;
pub(crate) use clear::Clear;
pub(crate) mod touch;
pub(crate) use enter::Enter;
pub(crate) use env::Env;
#[allow(unused_imports)]
pub(crate) use evaluate_by::EvaluateBy;
pub(crate) use exit::Exit;
@ -128,6 +139,7 @@ pub(crate) use first::First;
pub(crate) use format::Format;
pub(crate) use from_bson::FromBSON;
pub(crate) use from_csv::FromCSV;
pub(crate) use from_ics::FromIcs;
pub(crate) use from_ini::FromINI;
pub(crate) use from_json::FromJSON;
pub(crate) use from_ods::FromODS;
@ -137,12 +149,14 @@ pub(crate) use from_ssv::FromSSV;
pub(crate) use from_toml::FromTOML;
pub(crate) use from_tsv::FromTSV;
pub(crate) use from_url::FromURL;
pub(crate) use from_vcf::FromVcf;
pub(crate) use from_xlsx::FromXLSX;
pub(crate) use from_xml::FromXML;
pub(crate) use from_yaml::FromYAML;
pub(crate) use from_yaml::FromYML;
pub(crate) use get::Get;
pub(crate) use group_by::GroupBy;
pub(crate) use headers::Headers;
pub(crate) use help::Help;
pub(crate) use histogram::Histogram;
pub(crate) use history::History;
@ -167,10 +181,12 @@ pub(crate) use range::Range;
#[allow(unused_imports)]
pub(crate) use reduce_by::ReduceBy;
pub(crate) use reject::Reject;
pub(crate) use rename::Rename;
pub(crate) use reverse::Reverse;
pub(crate) use rm::Remove;
pub(crate) use save::Save;
pub(crate) use shells::Shells;
pub(crate) use shuffle::Shuffle;
pub(crate) use size::Size;
pub(crate) use skip::Skip;
pub(crate) use skip_while::SkipWhile;
@ -178,19 +194,23 @@ pub(crate) use sort_by::SortBy;
pub(crate) use split_by::SplitBy;
pub(crate) use split_column::SplitColumn;
pub(crate) use split_row::SplitRow;
pub(crate) use sum::Sum;
#[allow(unused_imports)]
pub(crate) use t_sort_by::TSortBy;
pub(crate) use table::Table;
pub(crate) use tags::Tags;
pub(crate) use to_bson::ToBSON;
pub(crate) use to_csv::ToCSV;
pub(crate) use to_html::ToHTML;
pub(crate) use to_json::ToJSON;
pub(crate) use to_md::ToMarkdown;
pub(crate) use to_sqlite::ToDB;
pub(crate) use to_sqlite::ToSQLite;
pub(crate) use to_toml::ToTOML;
pub(crate) use to_tsv::ToTSV;
pub(crate) use to_url::ToURL;
pub(crate) use to_yaml::ToYAML;
pub(crate) use touch::Touch;
pub(crate) use trim::Trim;
pub(crate) use uniq::Uniq;
pub(crate) use version::Version;

View File

@ -43,6 +43,7 @@ fn append(
) -> Result<OutputStream, ShellError> {
let mut after: VecDeque<Value> = VecDeque::new();
after.push_back(row);
let after = futures::stream::iter(after);
Ok(OutputStream::from_input(input.values.chain(after)))
}

View File

@ -0,0 +1,296 @@
use crate::commands::UnevaluatedCallInfo;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_parser::{hir, hir::Expression, hir::Literal, hir::SpannedExpression};
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
use std::sync::atomic::AtomicBool;
use std::sync::atomic::Ordering;
pub struct Autoview;
impl WholeStreamCommand for Autoview {
fn name(&self) -> &str {
"autoview"
}
fn signature(&self) -> Signature {
Signature::build("autoview")
}
fn usage(&self) -> &str {
"View the contents of the pipeline as a table or list."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
autoview(RunnableContext {
input: args.input,
commands: registry.clone(),
shell_manager: args.shell_manager,
host: args.host,
source: args.call_info.source,
ctrl_c: args.ctrl_c,
name: args.call_info.name_tag,
})
}
}
pub struct RunnableContextWithoutInput {
pub shell_manager: ShellManager,
pub host: Arc<parking_lot::Mutex<Box<dyn Host>>>,
pub source: Text,
pub ctrl_c: Arc<AtomicBool>,
pub commands: CommandRegistry,
pub name: Tag,
}
impl RunnableContextWithoutInput {
pub fn convert(context: RunnableContext) -> (InputStream, RunnableContextWithoutInput) {
let new_context = RunnableContextWithoutInput {
shell_manager: context.shell_manager,
host: context.host,
source: context.source,
ctrl_c: context.ctrl_c,
commands: context.commands,
name: context.name,
};
(context.input, new_context)
}
}
pub fn autoview(context: RunnableContext) -> Result<OutputStream, ShellError> {
let binary = context.get_command("binaryview");
let text = context.get_command("textview");
let table = context.get_command("table");
Ok(OutputStream::new(async_stream! {
let (mut input_stream, context) = RunnableContextWithoutInput::convert(context);
match input_stream.next().await {
Some(x) => {
match input_stream.next().await {
Some(y) => {
let ctrl_c = context.ctrl_c.clone();
let stream = async_stream! {
yield Ok(x);
yield Ok(y);
loop {
match input_stream.next().await {
Some(z) => {
if ctrl_c.load(Ordering::SeqCst) {
break;
}
yield Ok(z);
}
_ => break,
}
}
};
let stream = stream.to_input_stream();
if let Some(table) = table {
let command_args = create_default_command_args(&context).with_input(stream);
let result = table.run(command_args, &context.commands);
result.collect::<Vec<_>>().await;
}
}
_ => {
match x {
Value {
value: UntaggedValue::Primitive(Primitive::String(ref s)),
tag: Tag { anchor, span },
} if anchor.is_some() => {
if let Some(text) = text {
let mut stream = VecDeque::new();
stream.push_back(UntaggedValue::string(s).into_value(Tag { anchor, span }));
let command_args = create_default_command_args(&context).with_input(stream);
let result = text.run(command_args, &context.commands);
result.collect::<Vec<_>>().await;
} else {
out!("{}", s);
}
}
Value {
value: UntaggedValue::Primitive(Primitive::String(s)),
..
} => {
out!("{}", s);
}
Value {
value: UntaggedValue::Primitive(Primitive::Line(ref s)),
tag: Tag { anchor, span },
} if anchor.is_some() => {
if let Some(text) = text {
let mut stream = VecDeque::new();
stream.push_back(UntaggedValue::string(s).into_value(Tag { anchor, span }));
let command_args = create_default_command_args(&context).with_input(stream);
let result = text.run(command_args, &context.commands);
result.collect::<Vec<_>>().await;
} else {
out!("{}\n", s);
}
}
Value {
value: UntaggedValue::Primitive(Primitive::Line(s)),
..
} => {
out!("{}\n", s);
}
Value {
value: UntaggedValue::Primitive(Primitive::Path(s)),
..
} => {
out!("{}", s.display());
}
Value {
value: UntaggedValue::Primitive(Primitive::Int(n)),
..
} => {
out!("{}", n);
}
Value {
value: UntaggedValue::Primitive(Primitive::Decimal(n)),
..
} => {
out!("{}", n);
}
Value { value: UntaggedValue::Primitive(Primitive::Binary(ref b)), .. } => {
if let Some(binary) = binary {
let mut stream = VecDeque::new();
stream.push_back(x);
let command_args = create_default_command_args(&context).with_input(stream);
let result = binary.run(command_args, &context.commands);
result.collect::<Vec<_>>().await;
} else {
use pretty_hex::*;
out!("{:?}", b.hex_dump());
}
}
Value { value: UntaggedValue::Error(e), .. } => {
yield Err(e);
}
Value { value: UntaggedValue::Row(row), ..} => {
use prettytable::format::{FormatBuilder, LinePosition, LineSeparator};
use prettytable::{color, Attr, Cell, Row, Table};
use crate::data::value::{format_leaf, style_leaf};
use textwrap::fill;
let termwidth = std::cmp::max(textwrap::termwidth(), 20);
enum TableMode {
Light,
Normal,
}
let mut table = Table::new();
let table_mode = crate::data::config::config(Tag::unknown());
let table_mode = if let Some(s) = table_mode?.get("table_mode") {
match s.as_string() {
Ok(typ) if typ == "light" => TableMode::Light,
_ => TableMode::Normal,
}
} else {
TableMode::Normal
};
match table_mode {
TableMode::Light => {
table.set_format(
FormatBuilder::new()
.separator(LinePosition::Title, LineSeparator::new('─', '─', ' ', ' '))
.padding(1, 1)
.build(),
);
}
_ => {
table.set_format(
FormatBuilder::new()
.column_separator('│')
.separator(LinePosition::Top, LineSeparator::new('─', '┬', ' ', ' '))
.separator(LinePosition::Title, LineSeparator::new('─', '┼', ' ', ' '))
.separator(LinePosition::Bottom, LineSeparator::new('─', '┴', ' ', ' '))
.padding(1, 1)
.build(),
);
}
}
let mut max_key_len = 0;
for (key, _) in row.entries.iter() {
max_key_len = std::cmp::max(max_key_len, key.chars().count());
}
if max_key_len > (termwidth/2 - 1) {
max_key_len = termwidth/2 - 1;
}
let max_val_len = termwidth - max_key_len - 5;
for (key, value) in row.entries.iter() {
table.add_row(Row::new(vec![Cell::new(&fill(&key, max_key_len)).with_style(Attr::ForegroundColor(color::GREEN)).with_style(Attr::Bold),
Cell::new(&fill(&format_leaf(value).plain_string(100_000), max_val_len))]));
}
table.printstd();
// table.print_term(&mut *context.host.lock().out_terminal().ok_or_else(|| ShellError::untagged_runtime_error("Could not open terminal for output"))?)
// .map_err(|_| ShellError::untagged_runtime_error("Internal error: could not print to terminal (for unix systems check to make sure TERM is set)"))?;
}
Value { value: ref item, .. } => {
if let Some(table) = table {
let mut stream = VecDeque::new();
stream.push_back(x);
let command_args = create_default_command_args(&context).with_input(stream);
let result = table.run(command_args, &context.commands);
result.collect::<Vec<_>>().await;
} else {
out!("{:?}", item);
}
}
}
}
}
}
_ => {
//out!("<no results>");
}
}
// Needed for async_stream to type check
if false {
yield ReturnSuccess::value(UntaggedValue::nothing().into_untagged_value());
}
}))
}
fn create_default_command_args(context: &RunnableContextWithoutInput) -> RawCommandArgs {
let span = context.name.span;
RawCommandArgs {
host: context.host.clone(),
ctrl_c: context.ctrl_c.clone(),
shell_manager: context.shell_manager.clone(),
call_info: UnevaluatedCallInfo {
args: hir::Call {
head: Box::new(SpannedExpression::new(
Expression::Literal(Literal::String(span)),
span,
)),
positional: None,
named: None,
span,
},
source: context.source.clone(),
name_tag: context.name.clone(),
},
}
}

View File

@ -0,0 +1,63 @@
use crate::commands::PerItemCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{CallInfo, Primitive, ReturnSuccess, UntaggedValue, Value};
pub struct Calc;
impl PerItemCommand for Calc {
fn name(&self) -> &str {
"calc"
}
fn usage(&self) -> &str {
"Parse a math expression into a number"
}
fn run(
&self,
_call_info: &CallInfo,
_registry: &CommandRegistry,
raw_args: &RawCommandArgs,
input: Value,
) -> Result<OutputStream, ShellError> {
calc(input, raw_args)
}
}
fn calc(input: Value, args: &RawCommandArgs) -> Result<OutputStream, ShellError> {
let name_span = &args.call_info.name_tag.span;
let output = if let Ok(string) = input.as_string() {
match parse(&string, &input.tag) {
Ok(value) => ReturnSuccess::value(value),
Err(err) => Err(ShellError::labeled_error(
"Calculation error",
err,
&input.tag.span,
)),
}
} else {
Err(ShellError::labeled_error(
"Expected a string from pipeline",
"requires string input",
name_span,
))
};
Ok(vec![output].into())
}
pub fn parse(math_expression: &str, tag: impl Into<Tag>) -> Result<Value, String> {
use std::f64;
let num = meval::eval_str(math_expression);
match num {
Ok(num) => {
if num == f64::INFINITY || num == f64::NEG_INFINITY {
return Err(String::from("cannot represent result"));
}
Ok(UntaggedValue::from(Primitive::from(num)).into_value(tag))
}
Err(error) => Err(error.to_string()),
}
}

View File

@ -0,0 +1,853 @@
use crate::futures::ThreadedReceiver;
use crate::prelude::*;
use bytes::{BufMut, Bytes, BytesMut};
use futures::executor::block_on_stream;
use futures::stream::StreamExt;
use futures_codec::FramedRead;
use log::trace;
use nu_errors::ShellError;
use nu_parser::commands::classified::external::ExternalArg;
use nu_parser::ExternalCommand;
use nu_protocol::{ColumnPath, Primitive, ShellTypeName, UntaggedValue, Value};
use nu_source::{Tag, Tagged};
use nu_value_ext::as_column_path;
use std::io::Write;
use std::ops::Deref;
use std::process::{Command, Stdio};
use std::sync::mpsc;
pub enum StringOrBinary {
String(String),
Binary(Vec<u8>),
}
pub struct MaybeTextCodec;
impl futures_codec::Encoder for MaybeTextCodec {
type Item = StringOrBinary;
type Error = std::io::Error;
fn encode(&mut self, item: Self::Item, dst: &mut BytesMut) -> Result<(), Self::Error> {
match item {
StringOrBinary::String(s) => {
dst.reserve(s.len());
dst.put(s.as_bytes());
Ok(())
}
StringOrBinary::Binary(b) => {
dst.reserve(b.len());
dst.put(Bytes::from(b));
Ok(())
}
}
}
}
impl futures_codec::Decoder for MaybeTextCodec {
type Item = StringOrBinary;
type Error = std::io::Error;
fn decode(&mut self, src: &mut BytesMut) -> Result<Option<Self::Item>, Self::Error> {
let v: Vec<u8> = src.to_vec();
match String::from_utf8(v) {
Ok(s) => {
src.clear();
if s.is_empty() {
Ok(None)
} else {
Ok(Some(StringOrBinary::String(s)))
}
}
Err(err) => {
// Note: the longest UTF-8 character per Unicode spec is currently 6 bytes. If we fail somewhere earlier than the last 6 bytes,
// we know that we're failing to understand the string encoding and not just seeing a partial character. When this happens, let's
// fall back to assuming it's a binary buffer.
if src.is_empty() {
Ok(None)
} else if src.len() > 6 && (src.len() - err.utf8_error().valid_up_to() > 6) {
// Fall back to assuming binary
let buf = src.to_vec();
src.clear();
Ok(Some(StringOrBinary::Binary(buf)))
} else {
// Looks like a utf-8 string, so let's assume that
let buf = src.split_to(err.utf8_error().valid_up_to() + 1);
String::from_utf8(buf.to_vec())
.map(|x| Some(StringOrBinary::String(x)))
.map_err(|e| std::io::Error::new(std::io::ErrorKind::InvalidData, e))
}
}
}
}
}
pub fn nu_value_to_string(command: &ExternalCommand, from: &Value) -> Result<String, ShellError> {
match &from.value {
UntaggedValue::Primitive(Primitive::Int(i)) => Ok(i.to_string()),
UntaggedValue::Primitive(Primitive::String(s))
| UntaggedValue::Primitive(Primitive::Line(s)) => Ok(s.clone()),
UntaggedValue::Primitive(Primitive::Path(p)) => Ok(p.to_string_lossy().to_string()),
unsupported => Err(ShellError::labeled_error(
format!("needs string data (given: {})", unsupported.type_name()),
"expected a string",
&command.name_tag,
)),
}
}
pub(crate) async fn run_external_command(
command: ExternalCommand,
context: &mut Context,
input: Option<InputStream>,
is_last: bool,
) -> Result<Option<InputStream>, ShellError> {
trace!(target: "nu::run::external", "-> {}", command.name);
if !did_find_command(&command.name).await {
return Err(ShellError::labeled_error(
"Command not found",
"command not found",
&command.name_tag,
));
}
if command.has_it_argument() || command.has_nu_argument() {
run_with_iterator_arg(command, context, input, is_last)
} else {
run_with_stdin(command, context, input, is_last)
}
}
fn prepare_column_path_for_fetching_it_variable(
argument: &ExternalArg,
) -> Result<Tagged<ColumnPath>, ShellError> {
// We have "$it.[contents of interest]"
// and start slicing from "$it.[member+]"
// ^ here.
let key = nu_source::Text::from(argument.deref()).slice(4..argument.len());
to_column_path(&key, &argument.tag)
}
fn prepare_column_path_for_fetching_nu_variable(
argument: &ExternalArg,
) -> Result<Tagged<ColumnPath>, ShellError> {
// We have "$nu.[contents of interest]"
// and start slicing from "$nu.[member+]"
// ^ here.
let key = nu_source::Text::from(argument.deref()).slice(4..argument.len());
to_column_path(&key, &argument.tag)
}
fn to_column_path(
path_members: &str,
tag: impl Into<Tag>,
) -> Result<Tagged<ColumnPath>, ShellError> {
let tag = tag.into();
as_column_path(
&UntaggedValue::Table(
path_members
.split('.')
.map(|x| {
let member = match x.parse::<u64>() {
Ok(v) => UntaggedValue::int(v),
Err(_) => UntaggedValue::string(x),
};
member.into_value(&tag)
})
.collect(),
)
.into_value(&tag),
)
}
fn run_with_iterator_arg(
command: ExternalCommand,
context: &mut Context,
input: Option<InputStream>,
is_last: bool,
) -> Result<Option<InputStream>, ShellError> {
let path = context.shell_manager.path();
let mut inputs: InputStream = if let Some(input) = input {
trace_stream!(target: "nu::trace_stream::external::it", "input" = input)
} else {
InputStream::empty()
};
let stream = async_stream! {
while let Some(value) = inputs.next().await {
let name = command.name.clone();
let name_tag = command.name_tag.clone();
let home_dir = dirs::home_dir();
let path = &path;
let args = command.args.clone();
let it_replacement = {
if command.has_it_argument() {
let empty_arg = ExternalArg {
arg: "".to_string(),
tag: name_tag.clone()
};
let key = args.iter()
.find(|arg| arg.looks_like_it())
.unwrap_or_else(|| &empty_arg);
if args.iter().all(|arg| !arg.is_it()) {
let key = match prepare_column_path_for_fetching_it_variable(&key) {
Ok(keypath) => keypath,
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
}
};
match crate::commands::get::get_column_path(&key, &value) {
Ok(field) => {
match nu_value_to_string(&command, &field) {
Ok(val) => Some(val),
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
},
}
},
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
}
}
} else {
match nu_value_to_string(&command, &value) {
Ok(val) => Some(val),
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
},
}
}
} else {
None
}
};
let nu_replacement = {
if command.has_nu_argument() {
let empty_arg = ExternalArg {
arg: "".to_string(),
tag: name_tag.clone()
};
let key = args.iter()
.find(|arg| arg.looks_like_nu())
.unwrap_or_else(|| &empty_arg);
let nu_var = match crate::evaluate::variables::nu(&name_tag) {
Ok(variables) => variables,
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
}
};
if args.iter().all(|arg| !arg.is_nu()) {
let key = match prepare_column_path_for_fetching_nu_variable(&key) {
Ok(keypath) => keypath,
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
}
};
match crate::commands::get::get_column_path(&key, &nu_var) {
Ok(field) => {
match nu_value_to_string(&command, &field) {
Ok(val) => Some(val),
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
},
}
},
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
}
}
} else {
match nu_value_to_string(&command, &nu_var) {
Ok(val) => Some(val),
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
},
}
}
} else {
None
}
};
let process_args = args.iter().filter_map(|arg| {
if arg.chars().all(|c| c.is_whitespace()) {
None
} else {
let arg = if arg.looks_like_it() {
if let Some(mut value) = it_replacement.to_owned() {
let mut value = expand_tilde(&value, || home_dir.as_ref()).as_ref().to_string();
#[cfg(not(windows))]
{
value = {
if argument_contains_whitespace(&value) && !argument_is_quoted(&value) {
add_quotes(&value)
} else {
value
}
};
}
Some(value)
} else {
None
}
} else if arg.looks_like_nu() {
if let Some(mut value) = nu_replacement.to_owned() {
#[cfg(not(windows))]
{
value = {
if argument_contains_whitespace(&value) && !argument_is_quoted(&value) {
add_quotes(&value)
} else {
value
}
};
}
Some(value)
} else {
None
}
} else {
Some(arg.to_string())
};
arg
}
}).collect::<Vec<String>>();
match spawn(&command, &path, &process_args[..], None, is_last) {
Ok(res) => {
if let Some(mut res) = res {
while let Some(item) = res.next().await {
yield Ok(item)
}
}
}
Err(reason) => {
yield Ok(Value {
value: UntaggedValue::Error(reason),
tag: name_tag
});
return;
}
}
}
};
Ok(Some(stream.to_input_stream()))
}
fn run_with_stdin(
command: ExternalCommand,
context: &mut Context,
input: Option<InputStream>,
is_last: bool,
) -> Result<Option<InputStream>, ShellError> {
let path = context.shell_manager.path();
let input = input
.map(|input| trace_stream!(target: "nu::trace_stream::external::stdin", "input" = input));
let process_args = command
.args
.iter()
.map(|arg| {
let arg = expand_tilde(arg.deref(), dirs::home_dir);
#[cfg(not(windows))]
{
if argument_contains_whitespace(&arg) && argument_is_quoted(&arg) {
if let Some(unquoted) = remove_quotes(&arg) {
format!(r#""{}""#, unquoted)
} else {
arg.as_ref().to_string()
}
} else {
arg.as_ref().to_string()
}
}
#[cfg(windows)]
{
if let Some(unquoted) = remove_quotes(&arg) {
unquoted.to_string()
} else {
arg.as_ref().to_string()
}
}
})
.collect::<Vec<String>>();
spawn(&command, &path, &process_args[..], input, is_last)
}
fn spawn(
command: &ExternalCommand,
path: &str,
args: &[String],
input: Option<InputStream>,
is_last: bool,
) -> Result<Option<InputStream>, ShellError> {
let command = command.clone();
let mut process = {
#[cfg(windows)]
{
let mut process = Command::new("cmd");
process.arg("/c");
process.arg(&command.name);
for arg in args {
process.arg(&arg);
}
process
}
#[cfg(not(windows))]
{
let cmd_with_args = vec![command.name.clone(), args.join(" ")].join(" ");
let mut process = Command::new("sh");
process.arg("-c").arg(cmd_with_args);
process
}
};
process.current_dir(path);
trace!(target: "nu::run::external", "cwd = {:?}", &path);
// We want stdout regardless of what
// we are doing ($it case or pipe stdin)
if !is_last {
process.stdout(Stdio::piped());
trace!(target: "nu::run::external", "set up stdout pipe");
}
// open since we have some contents for stdin
if input.is_some() {
process.stdin(Stdio::piped());
trace!(target: "nu::run::external", "set up stdin pipe");
}
trace!(target: "nu::run::external", "built command {:?}", process);
// TODO Switch to async_std::process once it's stabilized
if let Ok(mut child) = process.spawn() {
let (tx, rx) = mpsc::sync_channel(0);
let mut stdin = child.stdin.take();
let stdin_write_tx = tx.clone();
let stdout_read_tx = tx;
let stdin_name_tag = command.name_tag.clone();
let stdout_name_tag = command.name_tag;
std::thread::spawn(move || {
if let Some(input) = input {
let mut stdin_write = stdin
.take()
.expect("Internal error: could not get stdin pipe for external command");
for value in block_on_stream(input) {
match &value.value {
UntaggedValue::Primitive(Primitive::Nothing) => continue,
UntaggedValue::Primitive(Primitive::String(s))
| UntaggedValue::Primitive(Primitive::Line(s)) => {
if let Err(e) = stdin_write.write(s.as_bytes()) {
let message = format!("Unable to write to stdin (error = {})", e);
let _ = stdin_write_tx.send(Ok(Value {
value: UntaggedValue::Error(ShellError::labeled_error(
message,
"application may have closed before completing pipeline",
&stdin_name_tag,
)),
tag: stdin_name_tag,
}));
return Err(());
}
}
UntaggedValue::Primitive(Primitive::Binary(b)) => {
if let Err(e) = stdin_write.write(b) {
let message = format!("Unable to write to stdin (error = {})", e);
let _ = stdin_write_tx.send(Ok(Value {
value: UntaggedValue::Error(ShellError::labeled_error(
message,
"application may have closed before completing pipeline",
&stdin_name_tag,
)),
tag: stdin_name_tag,
}));
return Err(());
}
}
unsupported => {
let _ = stdin_write_tx.send(Ok(Value {
value: UntaggedValue::Error(ShellError::labeled_error(
format!(
"Received unexpected type from pipeline ({})",
unsupported.type_name()
),
"expected a string",
stdin_name_tag.clone(),
)),
tag: stdin_name_tag,
}));
return Err(());
}
};
}
}
Ok(())
});
std::thread::spawn(move || {
if !is_last {
let stdout = if let Some(stdout) = child.stdout.take() {
stdout
} else {
let _ = stdout_read_tx.send(Ok(Value {
value: UntaggedValue::Error(ShellError::labeled_error(
"Can't redirect the stdout for external command",
"can't redirect stdout",
&stdout_name_tag,
)),
tag: stdout_name_tag,
}));
return Err(());
};
let file = futures::io::AllowStdIo::new(stdout);
let stream = FramedRead::new(file, MaybeTextCodec);
for line in block_on_stream(stream) {
match line {
Ok(line) => match line {
StringOrBinary::String(s) => {
let result = stdout_read_tx.send(Ok(Value {
value: UntaggedValue::Primitive(Primitive::String(s.clone())),
tag: stdout_name_tag.clone(),
}));
if result.is_err() {
break;
}
}
StringOrBinary::Binary(b) => {
let result = stdout_read_tx.send(Ok(Value {
value: UntaggedValue::Primitive(Primitive::Binary(
b.into_iter().collect(),
)),
tag: stdout_name_tag.clone(),
}));
if result.is_err() {
break;
}
}
},
Err(_) => {
let _ = stdout_read_tx.send(Ok(Value {
value: UntaggedValue::Error(ShellError::labeled_error(
"Unable to read from stdout.",
"unable to read from stdout",
&stdout_name_tag,
)),
tag: stdout_name_tag.clone(),
}));
break;
}
}
}
}
// We can give an error when we see a non-zero exit code, but this is different
// than what other shells will do.
if child.wait().is_err() {
let cfg = crate::data::config::config(Tag::unknown());
if let Ok(cfg) = cfg {
if cfg.contains_key("nonzero_exit_errors") {
let _ = stdout_read_tx.send(Ok(Value {
value: UntaggedValue::Error(ShellError::labeled_error(
"External command failed",
"command failed",
&stdout_name_tag,
)),
tag: stdout_name_tag,
}));
}
}
}
Ok(())
});
let stream = ThreadedReceiver::new(rx);
Ok(Some(stream.to_input_stream()))
} else {
Err(ShellError::labeled_error(
"Failed to spawn process",
"failed to spawn",
&command.name_tag,
))
}
}
async fn did_find_command(name: &str) -> bool {
#[cfg(not(windows))]
{
ichwh::which(name).await.unwrap_or(None).is_some()
}
#[cfg(windows)]
{
if ichwh::which(name).await.unwrap_or(None).is_some() {
true
} else {
let cmd_builtins = [
"call", "cls", "color", "date", "dir", "echo", "find", "hostname", "pause",
"start", "time", "title", "ver", "copy", "mkdir", "rename", "rd", "rmdir", "type",
];
cmd_builtins.contains(&name)
}
}
}
fn expand_tilde<SI: ?Sized, P, HD>(input: &SI, home_dir: HD) -> std::borrow::Cow<str>
where
SI: AsRef<str>,
P: AsRef<std::path::Path>,
HD: FnOnce() -> Option<P>,
{
shellexpand::tilde_with_context(input, home_dir)
}
#[allow(unused)]
pub fn argument_contains_whitespace(argument: &str) -> bool {
argument.chars().any(|c| c.is_whitespace())
}
fn argument_is_quoted(argument: &str) -> bool {
if argument.len() < 2 {
return false;
}
(argument.starts_with('"') && argument.ends_with('"'))
|| (argument.starts_with('\'') && argument.ends_with('\''))
}
#[allow(unused)]
fn add_quotes(argument: &str) -> String {
format!("\"{}\"", argument)
}
fn remove_quotes(argument: &str) -> Option<&str> {
if !argument_is_quoted(argument) {
return None;
}
let size = argument.len();
Some(&argument[1..size - 1])
}
#[allow(unused)]
fn shell_os_paths() -> Vec<std::path::PathBuf> {
let mut original_paths = vec![];
if let Some(paths) = std::env::var_os("PATH") {
original_paths = std::env::split_paths(&paths).collect::<Vec<_>>();
}
original_paths
}
#[cfg(test)]
mod tests {
use super::{
add_quotes, argument_contains_whitespace, argument_is_quoted, expand_tilde, remove_quotes,
run_external_command, Context,
};
use futures::executor::block_on;
use nu_errors::ShellError;
use nu_test_support::commands::ExternalBuilder;
// async fn read(mut stream: OutputStream) -> Option<Value> {
// match stream.try_next().await {
// Ok(val) => {
// if let Some(val) = val {
// val.raw_value()
// } else {
// None
// }
// }
// Err(_) => None,
// }
// }
async fn non_existent_run() -> Result<(), ShellError> {
let cmd = ExternalBuilder::for_name("i_dont_exist.exe").build();
let mut ctx = Context::basic().expect("There was a problem creating a basic context.");
assert!(run_external_command(cmd, &mut ctx, None, false)
.await
.is_err());
Ok(())
}
// async fn failure_run() -> Result<(), ShellError> {
// let cmd = ExternalBuilder::for_name("fail").build();
// let mut ctx = Context::basic().expect("There was a problem creating a basic context.");
// let stream = run_external_command(cmd, &mut ctx, None, false)
// .await?
// .expect("There was a problem running the external command.");
// match read(stream.into()).await {
// Some(Value {
// value: UntaggedValue::Error(_),
// ..
// }) => {}
// None | _ => panic!("Command didn't fail."),
// }
// Ok(())
// }
// #[test]
// fn identifies_command_failed() -> Result<(), ShellError> {
// block_on(failure_run())
// }
#[test]
fn identifies_command_not_found() -> Result<(), ShellError> {
block_on(non_existent_run())
}
#[test]
fn checks_contains_whitespace_from_argument_to_be_passed_in() {
assert_eq!(argument_contains_whitespace("andrés"), false);
assert_eq!(argument_contains_whitespace("and rés"), true);
assert_eq!(argument_contains_whitespace(r#"and\ rés"#), true);
}
#[test]
fn checks_quotes_from_argument_to_be_passed_in() {
assert_eq!(argument_is_quoted(""), false);
assert_eq!(argument_is_quoted("'"), false);
assert_eq!(argument_is_quoted("'a"), false);
assert_eq!(argument_is_quoted("a"), false);
assert_eq!(argument_is_quoted("a'"), false);
assert_eq!(argument_is_quoted("''"), true);
assert_eq!(argument_is_quoted(r#"""#), false);
assert_eq!(argument_is_quoted(r#""a"#), false);
assert_eq!(argument_is_quoted(r#"a"#), false);
assert_eq!(argument_is_quoted(r#"a""#), false);
assert_eq!(argument_is_quoted(r#""""#), true);
assert_eq!(argument_is_quoted("'andrés"), false);
assert_eq!(argument_is_quoted("andrés'"), false);
assert_eq!(argument_is_quoted(r#""andrés"#), false);
assert_eq!(argument_is_quoted(r#"andrés""#), false);
assert_eq!(argument_is_quoted("'andrés'"), true);
assert_eq!(argument_is_quoted(r#""andrés""#), true);
}
#[test]
fn adds_quotes_to_argument_to_be_passed_in() {
assert_eq!(add_quotes("andrés"), "\"andrés\"");
//assert_eq!(add_quotes("\"andrés\""), "\"andrés\"");
}
#[test]
fn strips_quotes_from_argument_to_be_passed_in() {
assert_eq!(remove_quotes(""), None);
assert_eq!(remove_quotes("'"), None);
assert_eq!(remove_quotes("'a"), None);
assert_eq!(remove_quotes("a"), None);
assert_eq!(remove_quotes("a'"), None);
assert_eq!(remove_quotes("''"), Some(""));
assert_eq!(remove_quotes(r#"""#), None);
assert_eq!(remove_quotes(r#""a"#), None);
assert_eq!(remove_quotes(r#"a"#), None);
assert_eq!(remove_quotes(r#"a""#), None);
assert_eq!(remove_quotes(r#""""#), Some(""));
assert_eq!(remove_quotes("'andrés"), None);
assert_eq!(remove_quotes("andrés'"), None);
assert_eq!(remove_quotes(r#""andrés"#), None);
assert_eq!(remove_quotes(r#"andrés""#), None);
assert_eq!(remove_quotes("'andrés'"), Some("andrés"));
assert_eq!(remove_quotes(r#""andrés""#), Some("andrés"));
}
#[test]
fn expands_tilde_if_starts_with_tilde_character() {
assert_eq!(
expand_tilde("~", || Some(std::path::Path::new("the_path_to_nu_light"))),
"the_path_to_nu_light"
);
}
#[test]
fn does_not_expand_tilde_if_tilde_is_not_first_character() {
assert_eq!(
expand_tilde("1~1", || Some(std::path::Path::new("the_path_to_nu_light"))),
"1~1"
);
}
}

View File

@ -5,7 +5,7 @@ use nu_errors::ShellError;
use nu_parser::InternalCommand;
use nu_protocol::{CommandAction, Primitive, ReturnSuccess, UntaggedValue, Value};
pub(crate) async fn run_internal_command(
pub(crate) fn run_internal_command(
command: InternalCommand,
context: &mut Context,
input: Option<InputStream>,
@ -138,6 +138,14 @@ pub(crate) async fn run_internal_command(
}
},
Ok(ReturnSuccess::Value(Value {
value: UntaggedValue::Error(err),
..
})) => {
context.error(err);
break;
}
Ok(ReturnSuccess::Value(v)) => {
yielded = true;
yield Ok(v);

View File

@ -1,19 +1,17 @@
use crate::commands::classified::external::run_external_command;
use crate::commands::classified::internal::run_internal_command;
use crate::context::Context;
use crate::stream::{InputStream, OutputStream};
use crate::stream::InputStream;
use nu_errors::ShellError;
use nu_parser::{ClassifiedCommand, ClassifiedPipeline};
use nu_protocol::{ReturnSuccess, UntaggedValue, Value};
use nu_source::Text;
use std::sync::atomic::Ordering;
pub(crate) async fn run_pipeline(
pipeline: ClassifiedPipeline,
ctx: &mut Context,
mut input: Option<InputStream>,
line: &str,
) -> Result<(), ShellError> {
) -> Result<Option<InputStream>, ShellError> {
let mut iter = pipeline.commands.list.into_iter().peekable();
loop {
@ -33,7 +31,7 @@ pub(crate) async fn run_pipeline(
(_, Some(ClassifiedCommand::Error(err))) => return Err(err.clone().into()),
(Some(ClassifiedCommand::Internal(left)), _) => {
run_internal_command(left, ctx, input, Text::from(line)).await?
run_internal_command(left, ctx, input, Text::from(line))?
}
(Some(ClassifiedCommand::External(left)), None) => {
@ -48,26 +46,5 @@ pub(crate) async fn run_pipeline(
};
}
use futures::stream::TryStreamExt;
if let Some(input) = input {
let mut output_stream: OutputStream = input.into();
loop {
match output_stream.try_next().await {
Ok(Some(ReturnSuccess::Value(Value {
value: UntaggedValue::Error(e),
..
}))) => return Err(e),
Ok(Some(_item)) => {
if ctx.ctrl_c.load(Ordering::SeqCst) {
break;
}
}
_ => {
break;
}
}
}
}
Ok(())
Ok(input)
}

View File

@ -525,9 +525,11 @@ impl Command {
match call_info {
Ok(call_info) => match command.run(&call_info, &registry, &raw_args, x) {
Ok(o) => o,
Err(e) => VecDeque::from(vec![ReturnValue::Err(e)]).to_output_stream(),
Err(e) => {
futures::stream::iter(vec![ReturnValue::Err(e)]).to_output_stream()
}
},
Err(e) => VecDeque::from(vec![ReturnValue::Err(e)]).to_output_stream(),
Err(e) => futures::stream::iter(vec![ReturnValue::Err(e)]).to_output_stream(),
}
})
.flatten();

View File

@ -31,21 +31,34 @@ impl WholeStreamCommand for Config {
"load",
SyntaxShape::Path,
"load the config from the path give",
Some('l'),
)
.named(
"set",
SyntaxShape::Any,
"set a value in the config, eg) --set [key value]",
Some('s'),
)
.named(
"set_into",
SyntaxShape::Member,
"sets a variable from values in the pipeline",
Some('i'),
)
.named("get", SyntaxShape::Any, "get a value from the config")
.named("remove", SyntaxShape::Any, "remove a value from the config")
.switch("clear", "clear the config")
.switch("path", "return the path to the config file")
.named(
"get",
SyntaxShape::Any,
"get a value from the config",
Some('g'),
)
.named(
"remove",
SyntaxShape::Any,
"remove a value from the config",
Some('r'),
)
.switch("clear", "clear the config", Some('c'))
.switch("path", "return the path to the config file", Some('p'))
}
fn usage(&self) -> &str {

View File

@ -24,7 +24,11 @@ impl PerItemCommand for Cpy {
Signature::build("cp")
.required("src", SyntaxShape::Pattern, "the place to copy from")
.required("dst", SyntaxShape::Path, "the place to copy to")
.switch("recursive", "copy recursively through subdirectories")
.switch(
"recursive",
"copy recursively through subdirectories",
Some('r'),
)
}
fn usage(&self) -> &str {

View File

@ -18,8 +18,8 @@ impl WholeStreamCommand for Date {
fn signature(&self) -> Signature {
Signature::build("date")
.switch("utc", "use universal time (UTC)")
.switch("local", "use the local time")
.switch("utc", "use universal time (UTC)", Some('u'))
.switch("local", "use the local time", Some('l'))
}
fn usage(&self) -> &str {
@ -91,5 +91,5 @@ pub fn date(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStrea
date_out.push_back(value);
Ok(date_out.to_output_stream())
Ok(futures::stream::iter(date_out).to_output_stream())
}

View File

@ -6,7 +6,9 @@ use nu_protocol::{ReturnSuccess, Signature, UntaggedValue};
pub struct Debug;
#[derive(Deserialize)]
pub struct DebugArgs {}
pub struct DebugArgs {
raw: bool,
}
impl WholeStreamCommand for Debug {
fn name(&self) -> &str {
@ -14,7 +16,7 @@ impl WholeStreamCommand for Debug {
}
fn signature(&self) -> Signature {
Signature::build("debug")
Signature::build("debug").switch("raw", "Prints the raw value representation.", Some('r'))
}
fn usage(&self) -> &str {
@ -31,13 +33,19 @@ impl WholeStreamCommand for Debug {
}
fn debug_value(
_args: DebugArgs,
DebugArgs { raw }: DebugArgs,
RunnableContext { input, .. }: RunnableContext,
) -> Result<impl ToOutputStream, ShellError> {
Ok(input
.values
.map(|v| {
ReturnSuccess::value(UntaggedValue::string(format!("{:#?}", v)).into_untagged_value())
.map(move |v| {
if raw {
ReturnSuccess::value(
UntaggedValue::string(format!("{:#?}", v)).into_untagged_value(),
)
} else {
ReturnSuccess::debug_value(v)
}
})
.to_output_stream())
}

View File

@ -67,7 +67,8 @@ fn default(
} else {
result.push_back(ReturnSuccess::value(item));
}
result
futures::stream::iter(result)
})
.flatten();

View File

@ -0,0 +1,376 @@
extern crate filesize;
use crate::commands::command::RunnablePerItemContext;
use crate::prelude::*;
use filesize::file_real_size_fast;
use glob::*;
use indexmap::map::IndexMap;
use nu_errors::ShellError;
use nu_protocol::{CallInfo, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
use std::path::PathBuf;
const NAME: &str = "du";
const GLOB_PARAMS: MatchOptions = MatchOptions {
case_sensitive: true,
require_literal_separator: true,
require_literal_leading_dot: false,
};
pub struct Du;
#[derive(Deserialize, Clone)]
pub struct DuArgs {
path: Option<Tagged<PathBuf>>,
all: bool,
deref: bool,
exclude: Option<Tagged<String>>,
#[serde(rename = "max-depth")]
max_depth: Option<Tagged<u64>>,
#[serde(rename = "min-size")]
min_size: Option<Tagged<u64>>,
}
impl PerItemCommand for Du {
fn name(&self) -> &str {
NAME
}
fn signature(&self) -> Signature {
Signature::build(NAME)
.optional("path", SyntaxShape::Pattern, "starting directory")
.switch(
"all",
"Output file sizes as well as directory sizes",
Some('a'),
)
.switch(
"deref",
"Dereference symlinks to their targets for size",
Some('r'),
)
.named(
"exclude",
SyntaxShape::Pattern,
"Exclude these file names",
Some('x'),
)
.named(
"max-depth",
SyntaxShape::Int,
"Directory recursion limit",
Some('d'),
)
.named(
"min-size",
SyntaxShape::Int,
"Exclude files below this size",
Some('m'),
)
}
fn usage(&self) -> &str {
"Find disk usage sizes of specified items"
}
fn run(
&self,
call_info: &CallInfo,
_registry: &CommandRegistry,
raw_args: &RawCommandArgs,
_input: Value,
) -> Result<OutputStream, ShellError> {
call_info
.process(&raw_args.shell_manager, raw_args.ctrl_c.clone(), du)?
.run()
}
}
fn du(args: DuArgs, ctx: &RunnablePerItemContext) -> Result<OutputStream, ShellError> {
let tag = ctx.name.clone();
let exclude = args.exclude.map_or(Ok(None), move |x| {
Pattern::new(&x.item)
.map(Option::Some)
.map_err(|e| ShellError::labeled_error(e.msg, "glob error", x.tag.clone()))
})?;
let include_files = args.all;
let paths = match args.path {
Some(p) => {
let p = p.item.to_str().expect("Why isn't this encoded properly?");
glob::glob_with(p, GLOB_PARAMS)
}
None => glob::glob_with("*", GLOB_PARAMS),
}
.map_err(|e| ShellError::labeled_error(e.msg, "glob error", tag.clone()))?
.filter(move |p| {
if include_files {
true
} else {
match p {
Ok(f) if f.is_dir() => true,
Err(e) if e.path().is_dir() => true,
_ => false,
}
}
})
.map(|v| v.map_err(glob_err_into));
let ctrl_c = ctx.ctrl_c.clone();
let all = args.all;
let deref = args.deref;
let max_depth = args.max_depth.map(|f| f.item);
let min_size = args.min_size.map(|f| f.item);
let params = DirBuilder {
tag: tag.clone(),
min: min_size,
deref,
exclude,
all,
};
let stream = futures::stream::iter(paths)
.interruptible(ctrl_c)
.map(move |path| match path {
Ok(p) => {
if p.is_dir() {
Ok(ReturnSuccess::Value(
DirInfo::new(p, &params, max_depth).into(),
))
} else {
FileInfo::new(p, deref, tag.clone()).map(|v| ReturnSuccess::Value(v.into()))
}
}
Err(e) => Err(e),
});
Ok(stream.to_output_stream())
}
struct DirBuilder {
tag: Tag,
min: Option<u64>,
deref: bool,
exclude: Option<Pattern>,
all: bool,
}
struct DirInfo {
dirs: Vec<DirInfo>,
files: Vec<FileInfo>,
errors: Vec<ShellError>,
size: u64,
blocks: u64,
path: PathBuf,
tag: Tag,
}
struct FileInfo {
path: PathBuf,
size: u64,
blocks: Option<u64>,
tag: Tag,
}
impl FileInfo {
fn new(path: impl Into<PathBuf>, deref: bool, tag: Tag) -> Result<Self, ShellError> {
let path = path.into();
let m = if deref {
std::fs::metadata(&path)
} else {
std::fs::symlink_metadata(&path)
};
match m {
Ok(d) => {
let block_size = file_real_size_fast(&path, &d).ok();
Ok(FileInfo {
path,
blocks: block_size,
size: d.len(),
tag,
})
}
Err(e) => Err(e.into()),
}
}
}
impl DirInfo {
fn new(path: impl Into<PathBuf>, params: &DirBuilder, depth: Option<u64>) -> Self {
let path = path.into();
let mut s = Self {
dirs: Vec::new(),
errors: Vec::new(),
files: Vec::new(),
size: 0,
blocks: 0,
tag: params.tag.clone(),
path,
};
match std::fs::read_dir(&s.path) {
Ok(d) => {
for f in d {
match f {
Ok(i) => match i.file_type() {
Ok(t) if t.is_dir() => s = s.add_dir(i.path(), depth, &params),
Ok(_t) => s = s.add_file(i.path(), &params),
Err(e) => s = s.add_error(e.into()),
},
Err(e) => s = s.add_error(e.into()),
}
}
}
Err(e) => s = s.add_error(e.into()),
}
s
}
fn add_dir(
mut self,
path: impl Into<PathBuf>,
mut depth: Option<u64>,
params: &DirBuilder,
) -> Self {
if let Some(current) = depth {
if let Some(new) = current.checked_sub(1) {
depth = Some(new);
} else {
return self;
}
}
let d = DirInfo::new(path, &params, depth);
self.size += d.size;
self.blocks += d.blocks;
self.dirs.push(d);
self
}
fn add_file(mut self, f: impl Into<PathBuf>, params: &DirBuilder) -> Self {
let f = f.into();
let include = params
.exclude
.as_ref()
.map_or(true, |x| !x.matches_path(&f));
if include {
match FileInfo::new(f, params.deref, self.tag.clone()) {
Ok(file) => {
let inc = params.min.map_or(true, |s| file.size >= s);
if inc {
self.size += file.size;
self.blocks += file.blocks.unwrap_or(0);
if params.all {
self.files.push(file);
}
}
}
Err(e) => self = self.add_error(e),
}
}
self
}
fn add_error(mut self, e: ShellError) -> Self {
self.errors.push(e);
self
}
}
fn glob_err_into(e: GlobError) -> ShellError {
let e = e.into_error();
ShellError::from(e)
}
fn value_from_vec<V>(vec: Vec<V>, tag: &Tag) -> Value
where
V: Into<Value>,
{
if vec.is_empty() {
UntaggedValue::nothing()
} else {
let values = vec.into_iter().map(Into::into).collect::<Vec<Value>>();
UntaggedValue::Table(values)
}
.retag(tag)
}
impl From<DirInfo> for Value {
fn from(d: DirInfo) -> Self {
let mut r: IndexMap<String, Value> = IndexMap::new();
r.insert(
"path".to_string(),
UntaggedValue::path(d.path).retag(&d.tag),
);
r.insert(
"apparent".to_string(),
UntaggedValue::bytes(d.size).retag(&d.tag),
);
r.insert(
"physical".to_string(),
UntaggedValue::bytes(d.blocks).retag(&d.tag),
);
r.insert("directories".to_string(), value_from_vec(d.dirs, &d.tag));
r.insert("files".to_string(), value_from_vec(d.files, &d.tag));
if !d.errors.is_empty() {
let v = UntaggedValue::Table(
d.errors
.into_iter()
.map(move |e| UntaggedValue::Error(e).into_untagged_value())
.collect::<Vec<Value>>(),
)
.retag(&d.tag);
r.insert("errors".to_string(), v);
}
Value {
value: UntaggedValue::row(r),
tag: d.tag,
}
}
}
impl From<FileInfo> for Value {
fn from(f: FileInfo) -> Self {
let mut r: IndexMap<String, Value> = IndexMap::new();
r.insert(
"path".to_string(),
UntaggedValue::path(f.path).retag(&f.tag),
);
r.insert(
"apparent".to_string(),
UntaggedValue::bytes(f.size).retag(&f.tag),
);
let b = f
.blocks
.map(UntaggedValue::bytes)
.unwrap_or_else(UntaggedValue::nothing)
.retag(&f.tag);
r.insert("physical".to_string(), b);
r.insert(
"directories".to_string(),
UntaggedValue::nothing().retag(&f.tag),
);
r.insert("files".to_string(), UntaggedValue::nothing().retag(&f.tag));
UntaggedValue::row(r).retag(&f.tag)
}
}

View File

@ -60,7 +60,8 @@ fn run(
}
}
let stream = VecDeque::from(output);
// TODO: This whole block can probably be replaced with `.map()`
let stream = futures::stream::iter(output);
Ok(stream.to_output_stream())
}

View File

@ -42,11 +42,13 @@ impl PerItemCommand for Edit {
let replacement = call_info.args.expect_nth(1)?.tagged_unknown();
let stream = match value {
obj @ Value {
obj
@
Value {
value: UntaggedValue::Row(_),
..
} => match obj.replace_data_at_column_path(&field, replacement.item.clone()) {
Some(v) => VecDeque::from(vec![Ok(ReturnSuccess::Value(v))]),
Some(v) => futures::stream::iter(vec![Ok(ReturnSuccess::Value(v))]),
None => {
return Err(ShellError::labeled_error(
"edit could not find place to insert column",

View File

@ -23,6 +23,7 @@ impl WholeStreamCommand for EvaluateBy {
"evaluate_with",
SyntaxShape::String,
"the name of the column to evaluate by",
Some('w'),
)
}

View File

@ -12,7 +12,7 @@ impl WholeStreamCommand for Exit {
}
fn signature(&self) -> Signature {
Signature::build("exit").switch("now", "exit out of the shell immediately")
Signature::build("exit").switch("now", "exit out of the shell immediately", Some('n'))
}
fn usage(&self) -> &str {

View File

@ -9,7 +9,7 @@ pub struct First;
#[derive(Deserialize)]
pub struct FirstArgs {
rows: Option<Tagged<u64>>,
rows: Option<Tagged<usize>>,
}
impl WholeStreamCommand for First {

View File

@ -2,7 +2,11 @@ use crate::commands::PerItemCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{CallInfo, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_protocol::{
CallInfo, ColumnPath, ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value,
};
use nu_source::Tagged;
use nu_value_ext::{as_column_path, get_data_by_column_path};
use std::borrow::Borrow;
use nom::{
@ -45,16 +49,18 @@ impl PerItemCommand for Format {
ShellError::labeled_error(
"Could not create format pattern",
"could not create format pattern",
pattern_tag,
&pattern_tag,
)
})?;
let commands = format_pattern.1;
let output = if let Value {
value: UntaggedValue::Row(dict),
let output = match value {
value
@
Value {
value: UntaggedValue::Row(_),
..
} = value
{
} => {
let mut output = String::new();
for command in &commands {
@ -63,8 +69,17 @@ impl PerItemCommand for Format {
output.push_str(s);
}
FormatCommand::Column(c) => {
if let Some(c) = dict.entries.get(c) {
output.push_str(&value::format_leaf(c.borrow()).plain_string(100_000))
let key = to_column_path(&c, &pattern_tag)?;
let fetcher = get_data_by_column_path(
&value,
&key,
Box::new(move |(_, _, error)| error),
);
if let Ok(c) = fetcher {
output
.push_str(&value::format_leaf(c.borrow()).plain_string(100_000))
}
// That column doesn't match, so don't emit anything
}
@ -72,11 +87,11 @@ impl PerItemCommand for Format {
}
output
} else {
String::new()
}
_ => String::new(),
};
Ok(VecDeque::from(vec![ReturnSuccess::value(
Ok(futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::string(output).into_untagged_value(),
)])
.to_output_stream())
@ -116,3 +131,27 @@ fn format(input: &str) -> IResult<&str, Vec<FormatCommand>> {
Ok((loop_input, output))
}
fn to_column_path(
path_members: &str,
tag: impl Into<Tag>,
) -> Result<Tagged<ColumnPath>, ShellError> {
let tag = tag.into();
as_column_path(
&UntaggedValue::Table(
path_members
.split('.')
.map(|x| {
let member = match x.parse::<u64>() {
Ok(v) => UntaggedValue::int(v),
Err(_) => UntaggedValue::string(x),
};
member.into_value(&tag)
})
.collect(),
)
.into_value(&tag),
)
}

View File

@ -205,13 +205,9 @@ fn from_bson(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStre
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let bytes = input.collect_binary(tag.clone()).await?;
for value in values {
let value_tag = &value.tag;
match value.value {
UntaggedValue::Primitive(Primitive::Binary(vb)) =>
match from_bson_bytes_to_value(vb, tag.clone()) {
match from_bson_bytes_to_value(bytes.item, tag.clone()) {
Ok(x) => yield ReturnSuccess::value(x),
Err(_) => {
yield Err(ShellError::labeled_error_with_secondary(
@ -219,20 +215,10 @@ fn from_bson(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStre
"input cannot be parsed as BSON",
tag.clone(),
"value originates from here",
value_tag,
bytes.tag,
))
}
}
_ => yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
tag.clone(),
"value originates from here",
value_tag,
)),
}
}
};
Ok(stream.to_output_stream())

View File

@ -23,8 +23,13 @@ impl WholeStreamCommand for FromCSV {
"separator",
SyntaxShape::String,
"a character to separate columns, defaults to ','",
Some('s'),
)
.switch(
"headerless",
"don't treat the first row as column names",
None,
)
.switch("headerless", "don't treat the first row as column names")
}
fn usage(&self) -> &str {

View File

@ -0,0 +1,169 @@
use crate::prelude::*;
use nu_errors::ShellError;
use nu_parser::hir::syntax_shape::{ExpandContext, SignatureRegistry};
use nu_parser::utils::{parse_line_with_separator as parse, LineSeparatedShape};
use nu_parser::TokensIterator;
use nu_protocol::{ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue, Value};
use nu_source::nom_input;
use derive_new::new;
pub fn from_delimited_data(
headerless: bool,
sep: char,
format_name: &'static str,
RunnableContext { input, name, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let name_tag = name;
let stream = async_stream! {
let concat_string = input.collect_string(name_tag.clone()).await?;
match from_delimited_string_to_value(concat_string.item, headerless, sep, name_tag.clone()) {
Ok(rows) => {
for row in rows {
match row {
Value { value: UntaggedValue::Table(list), .. } => {
for l in list {
yield ReturnSuccess::value(l);
}
}
x => yield ReturnSuccess::value(x),
}
}
},
Err(err) => {
let line_one = format!("Could not parse as {}", format_name);
let line_two = format!("input cannot be parsed as {}", format_name);
yield Err(ShellError::labeled_error_with_secondary(
line_one,
line_two,
name_tag.clone(),
"value originates from here",
concat_string.tag,
))
} ,
}
};
Ok(stream.to_output_stream())
}
#[derive(Debug, Clone, new)]
pub struct EmptyRegistry {
#[new(default)]
signatures: indexmap::IndexMap<String, Signature>,
}
impl EmptyRegistry {}
impl SignatureRegistry for EmptyRegistry {
fn has(&self, _name: &str) -> bool {
false
}
fn get(&self, _name: &str) -> Option<Signature> {
None
}
fn clone_box(&self) -> Box<dyn SignatureRegistry> {
Box::new(self.clone())
}
}
fn from_delimited_string_to_value(
s: String,
headerless: bool,
sep: char,
tag: impl Into<Tag>,
) -> Result<Vec<Value>, ShellError> {
let tag = tag.into();
let mut entries = s.lines();
let mut fields = vec![];
let mut out = vec![];
if let Some(first_entry) = entries.next() {
let tokens = match parse(&sep.to_string(), nom_input(first_entry)) {
Ok((_, tokens)) => tokens,
Err(err) => return Err(ShellError::parse_error(err)),
};
let tokens_span = tokens.span;
let source: nu_source::Text = tokens_span.slice(&first_entry).into();
if !headerless {
fields = tokens
.item
.iter()
.filter(|token| !token.is_separator())
.map(|field| field.source(&source).to_string())
.collect::<Vec<_>>();
}
let registry = Box::new(EmptyRegistry::new());
let ctx = ExpandContext::new(registry, &source, None);
let mut iterator = TokensIterator::new(&tokens.item, ctx, tokens_span);
let (results, tokens_identified) = iterator.expand(LineSeparatedShape);
let results = results?;
let mut row = TaggedDictBuilder::new(&tag);
if headerless {
let fallback_columns = (1..=tokens_identified)
.map(|i| format!("Column{}", i))
.collect::<Vec<String>>();
for (idx, field) in results.into_iter().enumerate() {
let key = if headerless {
&fallback_columns[idx]
} else {
&fields[idx]
};
row.insert_value(key, field.into_value(&tag));
}
out.push(row.into_value())
}
}
for entry in entries {
let tokens = match parse(&sep.to_string(), nom_input(entry)) {
Ok((_, tokens)) => tokens,
Err(err) => return Err(ShellError::parse_error(err)),
};
let tokens_span = tokens.span;
let source: nu_source::Text = tokens_span.slice(&entry).into();
let registry = Box::new(EmptyRegistry::new());
let ctx = ExpandContext::new(registry, &source, None);
let mut iterator = TokensIterator::new(&tokens.item, ctx, tokens_span);
let (results, tokens_identified) = iterator.expand(LineSeparatedShape);
let results = results?;
let mut row = TaggedDictBuilder::new(&tag);
let fallback_columns = (1..=tokens_identified)
.map(|i| format!("Column{}", i))
.collect::<Vec<String>>();
for (idx, field) in results.into_iter().enumerate() {
let key = if headerless {
&fallback_columns[idx]
} else {
match fields.get(idx) {
Some(key) => key,
None => &fallback_columns[idx],
}
};
row.insert_value(key, field.into_value(&tag));
}
out.push(row.into_value())
}
Ok(out)
}

View File

@ -0,0 +1,240 @@
extern crate ical;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use ical::parser::ical::component::*;
use ical::property::Property;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue, Value};
use std::io::BufReader;
pub struct FromIcs;
impl WholeStreamCommand for FromIcs {
fn name(&self) -> &str {
"from-ics"
}
fn signature(&self) -> Signature {
Signature::build("from-ics")
}
fn usage(&self) -> &str {
"Parse text as .ics and create table."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
from_ics(args, registry)
}
}
fn from_ics(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let input = args.input;
let stream = async_stream! {
let input_string = input.collect_string(tag.clone()).await?.item;
let input_bytes = input_string.as_bytes();
let buf_reader = BufReader::new(input_bytes);
let parser = ical::IcalParser::new(buf_reader);
for calendar in parser {
match calendar {
Ok(c) => yield ReturnSuccess::value(calendar_to_value(c, tag.clone())),
Err(_) => yield Err(ShellError::labeled_error(
"Could not parse as .ics",
"input cannot be parsed as .ics",
tag.clone()
)),
}
}
};
Ok(stream.to_output_stream())
}
fn calendar_to_value(calendar: IcalCalendar, tag: Tag) -> Value {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(calendar.properties, tag.clone()),
);
row.insert_untagged("events", events_to_value(calendar.events, tag.clone()));
row.insert_untagged("alarms", alarms_to_value(calendar.alarms, tag.clone()));
row.insert_untagged("to-Dos", todos_to_value(calendar.todos, tag.clone()));
row.insert_untagged(
"journals",
journals_to_value(calendar.journals, tag.clone()),
);
row.insert_untagged(
"free-busys",
free_busys_to_value(calendar.free_busys, tag.clone()),
);
row.insert_untagged("timezones", timezones_to_value(calendar.timezones, tag));
row.into_value()
}
fn events_to_value(events: Vec<IcalEvent>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&events
.into_iter()
.map(|event| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(event.properties, tag.clone()),
);
row.insert_untagged("alarms", alarms_to_value(event.alarms, tag.clone()));
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn alarms_to_value(alarms: Vec<IcalAlarm>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&alarms
.into_iter()
.map(|alarm| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(alarm.properties, tag.clone()),
);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn todos_to_value(todos: Vec<IcalTodo>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&todos
.into_iter()
.map(|todo| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(todo.properties, tag.clone()),
);
row.insert_untagged("alarms", alarms_to_value(todo.alarms, tag.clone()));
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn journals_to_value(journals: Vec<IcalJournal>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&journals
.into_iter()
.map(|journal| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(journal.properties, tag.clone()),
);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn free_busys_to_value(free_busys: Vec<IcalFreeBusy>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&free_busys
.into_iter()
.map(|free_busy| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(free_busy.properties, tag.clone()),
);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn timezones_to_value(timezones: Vec<IcalTimeZone>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&timezones
.into_iter()
.map(|timezone| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(timezone.properties, tag.clone()),
);
row.insert_untagged(
"transitions",
timezone_transitions_to_value(timezone.transitions, tag.clone()),
);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn timezone_transitions_to_value(
transitions: Vec<IcalTimeZoneTransition>,
tag: Tag,
) -> UntaggedValue {
UntaggedValue::table(
&transitions
.into_iter()
.map(|transition| {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged(
"properties",
properties_to_value(transition.properties, tag.clone()),
);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn properties_to_value(properties: Vec<Property>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&properties
.into_iter()
.map(|prop| {
let mut row = TaggedDictBuilder::new(tag.clone());
let name = UntaggedValue::string(prop.name);
let value = match prop.value {
Some(val) => UntaggedValue::string(val),
None => UntaggedValue::Primitive(Primitive::Nothing),
};
let params = match prop.params {
Some(param_list) => params_to_value(param_list, tag.clone()).into(),
None => UntaggedValue::Primitive(Primitive::Nothing),
};
row.insert_untagged("name", name);
row.insert_untagged("value", value);
row.insert_untagged("params", params);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn params_to_value(params: Vec<(String, Vec<String>)>, tag: Tag) -> Value {
let mut row = TaggedDictBuilder::new(tag);
for (param_name, param_values) in params {
let values: Vec<Value> = param_values.into_iter().map(|val| val.into()).collect();
let values = UntaggedValue::table(&values);
row.insert_untagged(param_name, values);
}
row.into_value()
}

View File

@ -66,32 +66,12 @@ pub fn from_ini_string_to_value(
fn from_ini(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let span = tag.span;
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let concat_string = input.collect_string(tag.clone()).await?;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
for value in values {
latest_tag = Some(value.tag.clone());
let value_span = value.tag.span;
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
} else {
yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
span,
"value originates from here",
value_span,
))
}
}
match from_ini_string_to_value(concat_string, tag.clone()) {
match from_ini_string_to_value(concat_string.item, tag.clone()) {
Ok(x) => match x {
Value { value: UntaggedValue::Table(list), .. } => {
for l in list {
@ -100,15 +80,15 @@ fn from_ini(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStrea
}
x => yield ReturnSuccess::value(x),
},
Err(_) => if let Some(last_tag) = latest_tag {
Err(_) => {
yield Err(ShellError::labeled_error_with_secondary(
"Could not parse as INI",
"input cannot be parsed as INI",
&tag,
"value originates from here",
last_tag,
concat_string.tag,
))
} ,
}
}
};

View File

@ -16,7 +16,11 @@ impl WholeStreamCommand for FromJSON {
}
fn signature(&self) -> Signature {
Signature::build("from-json").switch("objects", "treat each line as a separate value")
Signature::build("from-json").switch(
"objects",
"treat each line as a separate value",
Some('o'),
)
}
fn usage(&self) -> &str {
@ -70,35 +74,13 @@ fn from_json(
FromJSONArgs { objects }: FromJSONArgs,
RunnableContext { input, name, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let name_span = name.span;
let name_tag = name;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
for value in values {
latest_tag = Some(value.tag.clone());
let value_span = value.tag.span;
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
} else {
yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name_span,
"value originates from here",
value_span,
))
}
}
let concat_string = input.collect_string(name_tag.clone()).await?;
if objects {
for json_str in concat_string.lines() {
for json_str in concat_string.item.lines() {
if json_str.is_empty() {
continue;
}
@ -107,7 +89,6 @@ fn from_json(
Ok(x) =>
yield ReturnSuccess::value(x),
Err(e) => {
if let Some(ref last_tag) = latest_tag {
let mut message = "Could not parse as JSON (".to_string();
message.push_str(&e.to_string());
message.push_str(")");
@ -117,13 +98,12 @@ fn from_json(
"input cannot be parsed as JSON",
&name_tag,
"value originates from here",
last_tag))
}
concat_string.tag.clone()))
}
}
}
} else {
match from_json_string_to_value(concat_string, name_tag.clone()) {
match from_json_string_to_value(concat_string.item, name_tag.clone()) {
Ok(x) =>
match x {
Value { value: UntaggedValue::Table(list), .. } => {
@ -134,7 +114,6 @@ fn from_json(
x => yield ReturnSuccess::value(x),
}
Err(e) => {
if let Some(last_tag) = latest_tag {
let mut message = "Could not parse as JSON (".to_string();
message.push_str(&e.to_string());
message.push_str(")");
@ -144,8 +123,7 @@ fn from_json(
"input cannot be parsed as JSON",
name_tag,
"value originates from here",
last_tag))
}
concat_string.tag))
}
}
}

View File

@ -0,0 +1,98 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use crate::TaggedListBuilder;
use calamine::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue};
use std::io::Cursor;
pub struct FromODS;
#[derive(Deserialize)]
pub struct FromODSArgs {
headerless: bool,
}
impl WholeStreamCommand for FromODS {
fn name(&self) -> &str {
"from-ods"
}
fn signature(&self) -> Signature {
Signature::build("from-ods").switch(
"headerless",
"don't treat the first row as column names",
None,
)
}
fn usage(&self) -> &str {
"Parse OpenDocument Spreadsheet(.ods) data and create table."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
args.process(registry, from_ods)?.run()
}
}
fn from_ods(
FromODSArgs {
headerless: _headerless,
}: FromODSArgs,
runnable_context: RunnableContext,
) -> Result<OutputStream, ShellError> {
let input = runnable_context.input;
let tag = runnable_context.name;
let stream = async_stream! {
let bytes = input.collect_binary(tag.clone()).await?;
let mut buf: Cursor<Vec<u8>> = Cursor::new(bytes.item);
let mut ods = Ods::<_>::new(buf).map_err(|_| ShellError::labeled_error(
"Could not load ods file",
"could not load ods file",
&tag))?;
let mut dict = TaggedDictBuilder::new(&tag);
let sheet_names = ods.sheet_names().to_owned();
for sheet_name in &sheet_names {
let mut sheet_output = TaggedListBuilder::new(&tag);
if let Some(Ok(current_sheet)) = ods.worksheet_range(sheet_name) {
for row in current_sheet.rows() {
let mut row_output = TaggedDictBuilder::new(&tag);
for (i, cell) in row.iter().enumerate() {
let value = match cell {
DataType::Empty => UntaggedValue::nothing(),
DataType::String(s) => UntaggedValue::string(s),
DataType::Float(f) => UntaggedValue::decimal(*f),
DataType::Int(i) => UntaggedValue::int(*i),
DataType::Bool(b) => UntaggedValue::boolean(*b),
_ => UntaggedValue::nothing(),
};
row_output.insert_untagged(&format!("Column{}", i), value);
}
sheet_output.push_untagged(row_output.into_untagged_value());
}
dict.insert_untagged(sheet_name, sheet_output.into_untagged_value());
} else {
yield Err(ShellError::labeled_error(
"Could not load sheet",
"could not load sheet",
&tag));
}
}
yield ReturnSuccess::value(dict.into_value());
};
Ok(stream.to_output_stream())
}

View File

@ -138,13 +138,8 @@ fn from_sqlite(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputSt
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
for value in values {
let value_tag = &value.tag;
match value.value {
UntaggedValue::Primitive(Primitive::Binary(vb)) =>
match from_sqlite_bytes_to_value(vb, tag.clone()) {
let bytes = input.collect_binary(tag.clone()).await?;
match from_sqlite_bytes_to_value(bytes.item, tag.clone()) {
Ok(x) => match x {
Value { value: UntaggedValue::Table(list), .. } => {
for l in list {
@ -160,20 +155,10 @@ fn from_sqlite(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputSt
"input cannot be parsed as SQLite",
&tag,
"value originates from here",
value_tag,
bytes.tag,
))
}
}
_ => yield Err(ShellError::labeled_error_with_secondary(
"Expected binary data from pipeline",
"requires binary data input",
&tag,
"value originates from here",
value_tag,
)),
}
}
};
Ok(stream.to_output_stream())

View File

@ -27,12 +27,17 @@ impl WholeStreamCommand for FromSSV {
fn signature(&self) -> Signature {
Signature::build(STRING_REPRESENTATION)
.switch("headerless", "don't treat the first row as column names")
.switch("aligned-columns", "assume columns are aligned")
.switch(
"headerless",
"don't treat the first row as column names",
None,
)
.switch("aligned-columns", "assume columns are aligned", Some('a'))
.named(
"minimum-spaces",
SyntaxShape::Int,
"the mininum spaces to separate columns",
"the minimum spaces to separate columns",
Some('m'),
)
}
@ -197,15 +202,17 @@ fn string_to_table(
headerless: bool,
aligned_columns: bool,
split_at: usize,
) -> Option<Vec<Vec<(String, String)>>> {
) -> Vec<Vec<(String, String)>> {
let mut lines = s.lines().filter(|l| !l.trim().is_empty());
let separator = " ".repeat(std::cmp::max(split_at, 1));
let (ls, header_options) = if headerless {
(lines, HeaderOptions::WithoutHeaders)
} else {
let headers = lines.next()?;
(lines, HeaderOptions::WithHeaders(headers))
match lines.next() {
Some(header) => (lines, HeaderOptions::WithHeaders(header)),
None => return vec![],
}
};
let f = if aligned_columns {
@ -214,11 +221,7 @@ fn string_to_table(
parse_separated_columns
};
let parsed = f(ls, header_options, &separator);
match parsed.len() {
0 => None,
_ => Some(parsed),
}
f(ls, header_options, &separator)
}
fn from_ssv_string_to_value(
@ -229,7 +232,7 @@ fn from_ssv_string_to_value(
tag: impl Into<Tag>,
) -> Option<Value> {
let tag = tag.into();
let rows = string_to_table(s, headerless, aligned_columns, split_at)?
let rows = string_to_table(s, headerless, aligned_columns, split_at)
.iter()
.map(|row| {
let mut tagged_dict = TaggedDictBuilder::new(&tag);
@ -256,45 +259,26 @@ fn from_ssv(
RunnableContext { input, name, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
let concat_string = input.collect_string(name.clone()).await?;
let split_at = match minimum_spaces {
Some(number) => number.item,
None => DEFAULT_MINIMUM_SPACES
};
for value in values {
let value_tag = value.tag.clone();
latest_tag = Some(value_tag.clone());
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
}
else {
yield Err(ShellError::labeled_error_with_secondary (
"Expected a string from pipeline",
"requires string input",
&name,
"value originates from here",
&value_tag
))
}
}
match from_ssv_string_to_value(&concat_string, headerless, aligned_columns, split_at, name.clone()) {
match from_ssv_string_to_value(&concat_string.item, headerless, aligned_columns, split_at, name.clone()) {
Some(x) => match x {
Value { value: UntaggedValue::Table(list), ..} => {
for l in list { yield ReturnSuccess::value(l) }
}
x => yield ReturnSuccess::value(x)
},
None => if let Some(tag) = latest_tag {
None => {
yield Err(ShellError::labeled_error_with_secondary(
"Could not parse as SSV",
"input cannot be parsed ssv",
&name,
"value originates from here",
&tag,
&concat_string.tag,
))
},
}
@ -323,10 +307,10 @@ mod tests {
let result = string_to_table(input, false, true, 1);
assert_eq!(
result,
Some(vec![
vec![
vec![owned("a", "1"), owned("b", "2")],
vec![owned("a", "3"), owned("b", "4")]
])
]
);
}
@ -338,10 +322,7 @@ mod tests {
2
"#;
let result = string_to_table(input, false, true, 1);
assert_eq!(
result,
Some(vec![vec![owned("a", "1")], vec![owned("a", "2")]])
);
assert_eq!(result, vec![vec![owned("a", "1")], vec![owned("a", "2")]]);
}
#[test]
@ -354,21 +335,14 @@ mod tests {
let result = string_to_table(input, true, true, 1);
assert_eq!(
result,
Some(vec![
vec![
vec![owned("Column1", "a"), owned("Column2", "b")],
vec![owned("Column1", "1"), owned("Column2", "2")],
vec![owned("Column1", "3"), owned("Column2", "4")]
])
]
);
}
#[test]
fn it_returns_none_given_an_empty_string() {
let input = "";
let result = string_to_table(input, true, true, 1);
assert!(result.is_none());
}
#[test]
fn it_allows_a_predefined_number_of_spaces() {
let input = r#"
@ -380,18 +354,18 @@ mod tests {
let result = string_to_table(input, false, true, 3);
assert_eq!(
result,
Some(vec![
vec![
vec![
owned("column a", "entry 1"),
owned("column b", "entry number 2")
],
vec![owned("column a", "3"), owned("column b", "four")]
])
]
);
}
#[test]
fn it_trims_remaining_separator_space() -> Result<(), ShellError> {
fn it_trims_remaining_separator_space() {
let input = r#"
colA colB colC
val1 val2 val3
@ -399,17 +373,14 @@ mod tests {
let trimmed = |s: &str| s.trim() == s;
let result = string_to_table(input, false, true, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let result = string_to_table(input, false, true, 2);
assert!(result
.iter()
.all(|row| row.iter().all(|(a, b)| trimmed(a) && trimmed(b))));
Ok(())
}
#[test]
fn it_keeps_empty_columns() -> Result<(), ShellError> {
fn it_keeps_empty_columns() {
let input = r#"
colA col B col C
val2 val3
@ -417,8 +388,7 @@ mod tests {
val7 val8
"#;
let result = string_to_table(input, false, true, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let result = string_to_table(input, false, true, 2);
assert_eq!(
result,
vec![
@ -439,38 +409,43 @@ mod tests {
],
]
);
Ok(())
}
#[test]
fn it_uses_the_full_final_column() -> Result<(), ShellError> {
fn it_can_produce_an_empty_stream_for_header_only_input() {
let input = "colA col B";
let result = string_to_table(input, false, true, 2);
let expected: Vec<Vec<(String, String)>> = vec![];
assert_eq!(expected, result);
}
#[test]
fn it_uses_the_full_final_column() {
let input = r#"
colA col B
val1 val2 trailing value that should be included
"#;
let result = string_to_table(input, false, true, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let result = string_to_table(input, false, true, 2);
assert_eq!(
result,
vec![vec![
owned("colA", "val1"),
owned("col B", "val2 trailing value that should be included"),
],]
]]
);
Ok(())
}
#[test]
fn it_handles_empty_values_when_headerless_and_aligned_columns() -> Result<(), ShellError> {
fn it_handles_empty_values_when_headerless_and_aligned_columns() {
let input = r#"
a multi-word value b d
1 3-3 4
last
"#;
let result = string_to_table(input, true, true, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let result = string_to_table(input, true, true, 2);
assert_eq!(
result,
vec![
@ -497,27 +472,21 @@ mod tests {
],
]
);
Ok(())
}
#[test]
fn input_is_parsed_correctly_if_either_option_works() -> Result<(), ShellError> {
fn input_is_parsed_correctly_if_either_option_works() {
let input = r#"
docker-registry docker-registry=default docker-registry=default 172.30.78.158 5000/TCP
kubernetes component=apiserver,provider=kubernetes <none> 172.30.0.2 443/TCP
kubernetes-ro component=apiserver,provider=kubernetes <none> 172.30.0.1 80/TCP
"#;
let aligned_columns_headerless = string_to_table(input, true, true, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let separator_headerless = string_to_table(input, true, false, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let aligned_columns_with_headers = string_to_table(input, false, true, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let separator_with_headers = string_to_table(input, false, false, 2)
.ok_or_else(|| ShellError::unexpected("table couldn't be parsed"))?;
let aligned_columns_headerless = string_to_table(input, true, true, 2);
let separator_headerless = string_to_table(input, true, false, 2);
let aligned_columns_with_headers = string_to_table(input, false, true, 2);
let separator_with_headers = string_to_table(input, false, false, 2);
assert_eq!(aligned_columns_headerless, separator_headerless);
assert_eq!(aligned_columns_with_headers, separator_with_headers);
Ok(())
}
}

View File

@ -69,33 +69,11 @@ pub fn from_toml(
) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let name_span = tag.span;
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
for value in values {
latest_tag = Some(value.tag.clone());
let value_span = value.tag.span;
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
}
else {
yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name_span,
"value originates from here",
value_span,
))
}
}
match from_toml_string_to_value(concat_string, tag.clone()) {
let concat_string = input.collect_string(tag.clone()).await?;
match from_toml_string_to_value(concat_string.item, tag.clone()) {
Ok(x) => match x {
Value { value: UntaggedValue::Table(list), .. } => {
for l in list {
@ -104,15 +82,15 @@ pub fn from_toml(
}
x => yield ReturnSuccess::value(x),
},
Err(_) => if let Some(last_tag) = latest_tag {
Err(_) => {
yield Err(ShellError::labeled_error_with_secondary(
"Could not parse as TOML",
"input cannot be parsed as TOML",
&tag,
"value originates from here",
last_tag,
concat_string.tag,
))
} ,
}
}
};

View File

@ -17,8 +17,11 @@ impl WholeStreamCommand for FromTSV {
}
fn signature(&self) -> Signature {
Signature::build("from-tsv")
.switch("headerless", "don't treat the first row as column names")
Signature::build("from-tsv").switch(
"headerless",
"don't treat the first row as column names",
None,
)
}
fn usage(&self) -> &str {

View File

@ -1,7 +1,7 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue, Value};
use nu_protocol::{ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue};
pub struct FromURL;
@ -30,32 +30,12 @@ impl WholeStreamCommand for FromURL {
fn from_url(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let name_span = tag.span;
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let concat_string = input.collect_string(tag.clone()).await?;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
for value in values {
latest_tag = Some(value.tag.clone());
let value_span = value.tag.span;
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
} else {
yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name_span,
"value originates from here",
value_span,
))
}
}
let result = serde_urlencoded::from_str::<Vec<(String, String)>>(&concat_string);
let result = serde_urlencoded::from_str::<Vec<(String, String)>>(&concat_string.item);
match result {
Ok(result) => {
@ -68,17 +48,15 @@ fn from_url(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStrea
yield ReturnSuccess::value(row.into_value());
}
_ => {
if let Some(last_tag) = latest_tag {
yield Err(ShellError::labeled_error_with_secondary(
"String not compatible with url-encoding",
"input not url-encoded",
tag,
"value originates from here",
last_tag,
concat_string.tag,
));
}
}
}
};
Ok(stream.to_output_stream())

View File

@ -0,0 +1,102 @@
extern crate ical;
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use ical::parser::vcard::component::*;
use ical::property::Property;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue, Value};
use std::io::BufReader;
pub struct FromVcf;
impl WholeStreamCommand for FromVcf {
fn name(&self) -> &str {
"from-vcf"
}
fn signature(&self) -> Signature {
Signature::build("from-vcf")
}
fn usage(&self) -> &str {
"Parse text as .vcf and create table."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
from_vcf(args, registry)
}
}
fn from_vcf(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let input = args.input;
let stream = async_stream! {
let input_string = input.collect_string(tag.clone()).await?.item;
let input_bytes = input_string.as_bytes();
let buf_reader = BufReader::new(input_bytes);
let parser = ical::VcardParser::new(buf_reader);
for contact in parser {
match contact {
Ok(c) => yield ReturnSuccess::value(contact_to_value(c, tag.clone())),
Err(_) => yield Err(ShellError::labeled_error(
"Could not parse as .vcf",
"input cannot be parsed as .vcf",
tag.clone()
)),
}
}
};
Ok(stream.to_output_stream())
}
fn contact_to_value(contact: VcardContact, tag: Tag) -> Value {
let mut row = TaggedDictBuilder::new(tag.clone());
row.insert_untagged("properties", properties_to_value(contact.properties, tag));
row.into_value()
}
fn properties_to_value(properties: Vec<Property>, tag: Tag) -> UntaggedValue {
UntaggedValue::table(
&properties
.into_iter()
.map(|prop| {
let mut row = TaggedDictBuilder::new(tag.clone());
let name = UntaggedValue::string(prop.name);
let value = match prop.value {
Some(val) => UntaggedValue::string(val),
None => UntaggedValue::Primitive(Primitive::Nothing),
};
let params = match prop.params {
Some(param_list) => params_to_value(param_list, tag.clone()).into(),
None => UntaggedValue::Primitive(Primitive::Nothing),
};
row.insert_untagged("name", name);
row.insert_untagged("value", value);
row.insert_untagged("params", params);
row.into_value()
})
.collect::<Vec<Value>>(),
)
}
fn params_to_value(params: Vec<(String, Vec<String>)>, tag: Tag) -> Value {
let mut row = TaggedDictBuilder::new(tag);
for (param_name, param_values) in params {
let values: Vec<Value> = param_values.into_iter().map(|val| val.into()).collect();
let values = UntaggedValue::table(&values);
row.insert_untagged(param_name, values);
}
row.into_value()
}

View File

@ -0,0 +1,99 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use crate::TaggedListBuilder;
use calamine::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, TaggedDictBuilder, UntaggedValue};
use std::io::Cursor;
pub struct FromXLSX;
#[derive(Deserialize)]
pub struct FromXLSXArgs {
headerless: bool,
}
impl WholeStreamCommand for FromXLSX {
fn name(&self) -> &str {
"from-xlsx"
}
fn signature(&self) -> Signature {
Signature::build("from-xlsx").switch(
"headerless",
"don't treat the first row as column names",
None,
)
}
fn usage(&self) -> &str {
"Parse binary Excel(.xlsx) data and create table."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
args.process(registry, from_xlsx)?.run()
}
}
fn from_xlsx(
FromXLSXArgs {
headerless: _headerless,
}: FromXLSXArgs,
runnable_context: RunnableContext,
) -> Result<OutputStream, ShellError> {
let input = runnable_context.input;
let tag = runnable_context.name;
let stream = async_stream! {
let value = input.collect_binary(tag.clone()).await?;
let mut buf: Cursor<Vec<u8>> = Cursor::new(value.item);
let mut xls = Xlsx::<_>::new(buf).map_err(|_| {
ShellError::labeled_error("Could not load xlsx file", "could not load xlsx file", &tag)
})?;
let mut dict = TaggedDictBuilder::new(&tag);
let sheet_names = xls.sheet_names().to_owned();
for sheet_name in &sheet_names {
let mut sheet_output = TaggedListBuilder::new(&tag);
if let Some(Ok(current_sheet)) = xls.worksheet_range(sheet_name) {
for row in current_sheet.rows() {
let mut row_output = TaggedDictBuilder::new(&tag);
for (i, cell) in row.iter().enumerate() {
let value = match cell {
DataType::Empty => UntaggedValue::nothing(),
DataType::String(s) => UntaggedValue::string(s),
DataType::Float(f) => UntaggedValue::decimal(*f),
DataType::Int(i) => UntaggedValue::int(*i),
DataType::Bool(b) => UntaggedValue::boolean(*b),
_ => UntaggedValue::nothing(),
};
row_output.insert_untagged(&format!("Column{}", i), value);
}
sheet_output.push_untagged(row_output.into_untagged_value());
}
dict.insert_untagged(sheet_name, sheet_output.into_untagged_value());
} else {
yield Err(ShellError::labeled_error(
"Could not load sheet",
"could not load sheet",
&tag,
));
}
}
yield ReturnSuccess::value(dict.into_value());
};
Ok(stream.to_output_stream())
}

View File

@ -101,34 +101,12 @@ pub fn from_xml_string_to_value(s: String, tag: impl Into<Tag>) -> Result<Value,
fn from_xml(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let name_span = tag.span;
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let concat_string = input.collect_string(tag.clone()).await?;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
for value in values {
latest_tag = Some(value.tag.clone());
let value_span = value.tag.span;
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
}
else {
yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name_span,
"value originates from here",
value_span,
))
}
}
match from_xml_string_to_value(concat_string, tag.clone()) {
match from_xml_string_to_value(concat_string.item, tag.clone()) {
Ok(x) => match x {
Value { value: UntaggedValue::Table(list), .. } => {
for l in list {
@ -137,13 +115,13 @@ fn from_xml(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStrea
}
x => yield ReturnSuccess::value(x),
},
Err(_) => if let Some(last_tag) = latest_tag {
Err(_) => {
yield Err(ShellError::labeled_error_with_secondary(
"Could not parse as XML",
"input cannot be parsed as XML",
&tag,
"value originates from here",
&last_tag,
&concat_string.tag,
))
} ,
}

View File

@ -121,34 +121,12 @@ pub fn from_yaml_string_to_value(s: String, tag: impl Into<Tag>) -> Result<Value
fn from_yaml(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let name_span = tag.span;
let input = args.input;
let stream = async_stream! {
let values: Vec<Value> = input.values.collect().await;
let concat_string = input.collect_string(tag.clone()).await?;
let mut concat_string = String::new();
let mut latest_tag: Option<Tag> = None;
for value in values {
latest_tag = Some(value.tag.clone());
let value_span = value.tag.span;
if let Ok(s) = value.as_string() {
concat_string.push_str(&s);
}
else {
yield Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name_span,
"value originates from here",
value_span,
))
}
}
match from_yaml_string_to_value(concat_string, tag.clone()) {
match from_yaml_string_to_value(concat_string.item, tag.clone()) {
Ok(x) => match x {
Value { value: UntaggedValue::Table(list), .. } => {
for l in list {
@ -157,15 +135,15 @@ fn from_yaml(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStre
}
x => yield ReturnSuccess::value(x),
},
Err(_) => if let Some(last_tag) = latest_tag {
Err(_) => {
yield Err(ShellError::labeled_error_with_secondary(
"Could not parse as YAML",
"input cannot be parsed as YAML",
&tag,
"value originates from here",
&last_tag,
&concat_string.tag,
))
} ,
}
}
};

View File

@ -1,13 +1,11 @@
use crate::commands::WholeStreamCommand;
use crate::data::base::shape::Shapes;
use crate::prelude::*;
use futures_util::pin_mut;
use indexmap::set::IndexSet;
use log::trace;
use nu_errors::ShellError;
use nu_protocol::{
did_you_mean, ColumnPath, PathMember, ReturnSuccess, ReturnValue, Signature, SyntaxShape,
UnspannedPathMember, UntaggedValue, Value,
did_you_mean, ColumnPath, PathMember, Primitive, ReturnSuccess, ReturnValue, Signature,
SyntaxShape, UnspannedPathMember, UntaggedValue, Value,
};
use nu_source::span_for_spanned_list;
use nu_value_ext::get_data_by_column_path;
@ -180,23 +178,15 @@ pub fn get_column_path(path: &ColumnPath, obj: &Value) -> Result<Value, ShellErr
pub fn get(
GetArgs { rest: mut fields }: GetArgs,
RunnableContext { input, .. }: RunnableContext,
RunnableContext { mut input, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
if fields.is_empty() {
let stream = async_stream! {
let values = input.values;
pin_mut!(values);
let mut vec = input.drain_vec().await;
let mut shapes = Shapes::new();
let mut index = 0;
while let Some(row) = values.next().await {
shapes.add(&row, index);
index += 1;
}
for row in shapes.to_values() {
yield ReturnSuccess::value(row);
let descs = nu_protocol::merge_descriptors(&vec);
for desc in descs {
yield ReturnSuccess::value(desc);
}
};
@ -231,6 +221,10 @@ pub fn get(
result.push_back(ReturnSuccess::value(item.clone()));
}
}
Value {
value: UntaggedValue::Primitive(Primitive::Nothing),
..
} => {}
other => result.push_back(ReturnSuccess::value(other.clone())),
},
Err(reason) => result.push_back(ReturnSuccess::value(
@ -239,7 +233,7 @@ pub fn get(
}
}
result
futures::stream::iter(result)
})
.flatten();

View File

@ -0,0 +1,80 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use futures::stream::StreamExt;
use indexmap::IndexMap;
use nu_errors::ShellError;
use nu_protocol::Dictionary;
use nu_protocol::{ReturnSuccess, Signature, UntaggedValue, Value};
pub struct Headers;
#[derive(Deserialize)]
pub struct HeadersArgs {}
impl WholeStreamCommand for Headers {
fn name(&self) -> &str {
"headers"
}
fn signature(&self) -> Signature {
Signature::build("headers")
}
fn usage(&self) -> &str {
"Use the first row of the table as column names"
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
args.process(registry, headers)?.run()
}
}
pub fn headers(
HeadersArgs {}: HeadersArgs,
RunnableContext { input, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let stream = async_stream! {
let rows: Vec<Value> = input.values.collect().await;
if rows.len() < 1 {
yield Err(ShellError::untagged_runtime_error("Couldn't find headers, was the input a properly formatted, non-empty table?"));
}
//the headers are the first row in the table
let headers: Vec<String> = match &rows[0].value {
UntaggedValue::Row(d) => {
Ok(d.entries.iter().map(|(k, v)| {
match v.as_string() {
Ok(s) => s,
Err(_) => { //If a cell that should contain a header name is empty, we name the column Column[index]
match d.entries.get_full(k) {
Some((index, _, _)) => format!("Column{}", index),
None => "unknownColumn".to_string()
}
}
}
}).collect())
}
_ => Err(ShellError::unexpected_eof("Could not get headers, is the table empty?", rows[0].tag.span))
}?;
//Each row is a dictionary with the headers as keys
for r in rows.iter().skip(1) {
match &r.value {
UntaggedValue::Row(d) => {
let mut i = 0;
let mut entries = IndexMap::new();
for (_, v) in d.entries.iter() {
entries.insert(headers[i].clone(), v.clone());
i += 1;
}
yield Ok(ReturnSuccess::Value(UntaggedValue::Row(Dictionary{entries}).into_value(r.tag.clone())))
}
_ => yield Err(ShellError::unexpected_eof("Couldn't iterate through rows, was the input a properly formatted table?", r.tag.span))
}
}
};
Ok(stream.to_output_stream())
}

View File

@ -77,6 +77,7 @@ impl PerItemCommand for Help {
get_help(&command.name(), &command.usage(), command.signature()).into(),
);
}
let help = futures::stream::iter(help);
Ok(help.to_output_stream())
}
_ => {
@ -86,13 +87,25 @@ Here are some tips to help you get started.
* help commands - list all available commands
* help <command name> - display help about a particular command
Nushell works on the idea of a "pipeline". Pipelines are commands connected with the '|' character.
Each stage in the pipeline works together to load, parse, and display information to you.
[Examples]
List the files in the current directory, sorted by size:
ls | sort-by size
Get information about the current system:
sys | get host
Get the processes on your system actively using CPU:
ps | where cpu > 0
You can also learn more at https://www.nushell.sh/book/"#;
let mut output_stream = VecDeque::new();
output_stream.push_back(ReturnSuccess::value(
let output_stream = futures::stream::iter(vec![ReturnSuccess::value(
UntaggedValue::string(msg).into_value(tag),
));
)]);
Ok(output_stream.to_output_stream())
}
@ -158,34 +171,67 @@ pub(crate) fn get_help(
if !signature.named.is_empty() {
long_desc.push_str("\nflags:\n");
for (flag, ty) in signature.named {
match ty.0 {
NamedType::Switch => {
long_desc.push_str(&format!(
let msg = match ty.0 {
NamedType::Switch(s) => {
if let Some(c) = s {
format!(
" -{}, --{}{} {}\n",
c,
flag,
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
} else {
format!(
" --{}{} {}\n",
flag,
if !ty.1.is_empty() { ":" } else { "" },
ty.1
));
)
}
NamedType::Mandatory(m) => {
long_desc.push_str(&format!(
}
NamedType::Mandatory(s, m) => {
if let Some(c) = s {
format!(
" -{}, --{} <{}> (required parameter){} {}\n",
c,
flag,
m.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
} else {
format!(
" --{} <{}> (required parameter){} {}\n",
flag,
m.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
));
)
}
NamedType::Optional(o) => {
long_desc.push_str(&format!(
}
NamedType::Optional(s, o) => {
if let Some(c) = s {
format!(
" -{}, --{} <{}>{} {}\n",
c,
flag,
o.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
)
} else {
format!(
" --{} <{}>{} {}\n",
flag,
o.display(),
if !ty.1.is_empty() { ":" } else { "" },
ty.1
));
)
}
}
};
long_desc.push_str(&msg);
}
}

View File

@ -42,11 +42,13 @@ impl PerItemCommand for Insert {
let replacement = call_info.args.expect_nth(1)?.tagged_unknown();
let stream = match value {
obj @ Value {
obj
@
Value {
value: UntaggedValue::Row(_),
..
} => match obj.insert_data_at_column_path(&field, replacement.item.clone()) {
Ok(v) => VecDeque::from(vec![Ok(ReturnSuccess::Value(v))]),
Ok(v) => futures::stream::iter(vec![Ok(ReturnSuccess::Value(v))]),
Err(err) => return Err(err),
},

View File

@ -0,0 +1,104 @@
use crate::commands::command::RunnablePerItemContext;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{CallInfo, Signature, SyntaxShape, Value};
use nu_source::Tagged;
use std::process::{Command, Stdio};
pub struct Kill;
#[derive(Deserialize)]
pub struct KillArgs {
pub pid: Tagged<u64>,
pub rest: Vec<Tagged<u64>>,
pub force: Tagged<bool>,
pub quiet: Tagged<bool>,
}
impl PerItemCommand for Kill {
fn name(&self) -> &str {
"kill"
}
fn signature(&self) -> Signature {
Signature::build("kill")
.required(
"pid",
SyntaxShape::Int,
"process id of process that is to be killed",
)
.rest(SyntaxShape::Int, "rest of processes to kill")
.switch("force", "forcefully kill the process", Some('f'))
.switch("quiet", "won't print anything to the console", Some('q'))
}
fn usage(&self) -> &str {
"Kill a process using the process id."
}
fn run(
&self,
call_info: &CallInfo,
_registry: &CommandRegistry,
raw_args: &RawCommandArgs,
_input: Value,
) -> Result<OutputStream, ShellError> {
call_info
.process(&raw_args.shell_manager, raw_args.ctrl_c.clone(), kill)?
.run()
}
}
fn kill(
KillArgs {
pid,
rest,
force,
quiet,
}: KillArgs,
_context: &RunnablePerItemContext,
) -> Result<OutputStream, ShellError> {
let mut cmd = if cfg!(windows) {
let mut cmd = Command::new("taskkill");
if *force {
cmd.arg("/F");
}
cmd.arg("/PID");
cmd.arg(pid.item().to_string());
// each pid must written as `/PID 0` otherwise
// taskkill will act as `killall` unix command
for id in &rest {
cmd.arg("/PID");
cmd.arg(id.item().to_string());
}
cmd
} else {
let mut cmd = Command::new("kill");
if *force {
cmd.arg("-9");
}
cmd.arg(pid.item().to_string());
cmd.args(rest.iter().map(move |id| id.item().to_string()));
cmd
};
// pipe everything to null
if *quiet {
cmd.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null());
}
cmd.status().expect("failed to execute shell command");
Ok(OutputStream::empty())
}

View File

@ -0,0 +1,116 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{Primitive, ReturnSuccess, Signature, UntaggedValue, Value};
pub struct Lines;
impl WholeStreamCommand for Lines {
fn name(&self) -> &str {
"lines"
}
fn signature(&self) -> Signature {
Signature::build("lines")
}
fn usage(&self) -> &str {
"Split single string into rows, one per line."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
lines(args, registry)
}
}
fn ends_with_line_ending(st: &str) -> bool {
let mut temp = st.to_string();
let last = temp.pop();
if let Some(c) = last {
c == '\n'
} else {
false
}
}
fn lines(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream, ShellError> {
let args = args.evaluate_once(registry)?;
let tag = args.name_tag();
let name_span = tag.span;
let mut input = args.input;
let mut leftover = vec![];
let mut leftover_string = String::new();
let stream = async_stream! {
loop {
match input.values.next().await {
Some(Value { value: UntaggedValue::Primitive(Primitive::String(st)), ..}) => {
let mut st = leftover_string.clone() + &st;
leftover.clear();
let mut lines: Vec<String> = st.lines().map(|x| x.to_string()).collect();
if !ends_with_line_ending(&st) {
if let Some(last) = lines.pop() {
leftover_string = last;
} else {
leftover_string.clear();
}
} else {
leftover_string.clear();
}
let success_lines: Vec<_> = lines.iter().map(|x| ReturnSuccess::value(UntaggedValue::line(x).into_untagged_value())).collect();
yield futures::stream::iter(success_lines)
}
Some(Value { value: UntaggedValue::Primitive(Primitive::Line(st)), ..}) => {
let mut st = leftover_string.clone() + &st;
leftover.clear();
let mut lines: Vec<String> = st.lines().map(|x| x.to_string()).collect();
if !ends_with_line_ending(&st) {
if let Some(last) = lines.pop() {
leftover_string = last;
} else {
leftover_string.clear();
}
} else {
leftover_string.clear();
}
let success_lines: Vec<_> = lines.iter().map(|x| ReturnSuccess::value(UntaggedValue::line(x).into_untagged_value())).collect();
yield futures::stream::iter(success_lines)
}
Some( Value { tag: value_span, ..}) => {
yield futures::stream::iter(vec![Err(ShellError::labeled_error_with_secondary(
"Expected a string from pipeline",
"requires string input",
name_span,
"value originates from here",
value_span,
))]);
}
None => {
if !leftover.is_empty() {
let mut st = leftover_string.clone();
if let Ok(extra) = String::from_utf8(leftover) {
st.push_str(&extra);
}
yield futures::stream::iter(vec![ReturnSuccess::value(UntaggedValue::string(st).into_untagged_value())])
}
break;
}
}
}
if !leftover_string.is_empty() {
yield futures::stream::iter(vec![ReturnSuccess::value(UntaggedValue::string(leftover_string).into_untagged_value())]);
}
}
.flatten();
Ok(stream.to_output_stream())
}

View File

@ -10,6 +10,7 @@ pub struct Ls;
#[derive(Deserialize)]
pub struct LsArgs {
pub path: Option<Tagged<PathBuf>>,
pub all: bool,
pub full: bool,
#[serde(rename = "short-names")]
pub short_names: bool,
@ -29,11 +30,21 @@ impl PerItemCommand for Ls {
SyntaxShape::Pattern,
"a path to get the directory contents from",
)
.switch("full", "list all available columns for each entry")
.switch("short-names", "only print the file names and not the path")
.switch("all", "also show hidden files", Some('a'))
.switch(
"full",
"list all available columns for each entry",
Some('f'),
)
.switch(
"short-names",
"only print the file names and not the path",
Some('s'),
)
.switch(
"with-symlink-targets",
"display the paths to the target files that symlinks point to",
Some('w'),
)
}

View File

@ -24,6 +24,7 @@ impl WholeStreamCommand for MapMaxBy {
"column_name",
SyntaxShape::String,
"the name of the column to map-max the table's rows",
Some('c'),
)
}

View File

@ -68,7 +68,7 @@ fn nth(
result.push_back(ReturnSuccess::value(item));
}
result
futures::stream::iter(result)
})
.flatten();

View File

@ -20,7 +20,11 @@ impl PerItemCommand for Open {
SyntaxShape::Path,
"the file path to load values from",
)
.switch("raw", "load content as a string insead of a table")
.switch(
"raw",
"load content as a string instead of a table",
Some('r'),
)
}
fn usage(&self) -> &str {

View File

@ -136,6 +136,6 @@ impl PerItemCommand for Parse {
} else {
VecDeque::new()
};
Ok(output.to_output_stream())
Ok(output.into())
}
}

View File

@ -80,7 +80,7 @@ fn pick(
"No data to fetch.",
format!("Couldn't pick column \"{}\"", column),
path_member_tried.span,
format!("How about exploring it with \"get\"? Check the input is appropiate originating from here"),
format!("How about exploring it with \"get\"? Check the input is appropriate originating from here"),
obj_source.tag.span)
}

View File

@ -1,7 +1,9 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, TaggedDictBuilder, UntaggedValue, Value};
use nu_protocol::{
merge_descriptors, ReturnSuccess, Signature, SyntaxShape, TaggedDictBuilder, UntaggedValue,
};
use nu_source::{SpannedItem, Tagged};
use nu_value_ext::get_data_by_key;
@ -23,8 +25,16 @@ impl WholeStreamCommand for Pivot {
fn signature(&self) -> Signature {
Signature::build("pivot")
.switch("header-row", "treat the first row as column names")
.switch("ignore-titles", "don't pivot the column names into values")
.switch(
"header-row",
"treat the first row as column names",
Some('r'),
)
.switch(
"ignore-titles",
"don't pivot the column names into values",
Some('i'),
)
.rest(
SyntaxShape::String,
"the names to give columns once pivoted",
@ -44,18 +54,6 @@ impl WholeStreamCommand for Pivot {
}
}
fn merge_descriptors(values: &[Value]) -> Vec<String> {
let mut ret = vec![];
for value in values {
for desc in value.data_descriptors() {
if !ret.contains(&desc) {
ret.push(desc);
}
}
}
ret
}
pub fn pivot(args: PivotArgs, context: RunnableContext) -> Result<OutputStream, ShellError> {
let stream = async_stream! {
let input = context.input.into_vec().await;

View File

@ -84,9 +84,11 @@ pub fn filter_plugin(
let mut bos: VecDeque<Value> = VecDeque::new();
bos.push_back(UntaggedValue::Primitive(Primitive::BeginningOfStream).into_untagged_value());
let bos = futures::stream::iter(bos);
let mut eos: VecDeque<Value> = VecDeque::new();
eos.push_back(UntaggedValue::Primitive(Primitive::EndOfStream).into_untagged_value());
let eos = futures::stream::iter(eos);
let call_info = args.call_info.clone();
@ -294,6 +296,7 @@ pub fn filter_plugin(
}
}
})
.map(futures::stream::iter) // convert to a stream
.flatten();
Ok(stream.to_output_stream())

View File

@ -41,8 +41,7 @@ fn prepend(
PrependArgs { row }: PrependArgs,
RunnableContext { input, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let mut prepend: VecDeque<Value> = VecDeque::new();
prepend.push_back(row);
let prepend = futures::stream::iter(vec![row]);
Ok(OutputStream::from_input(prepend.chain(input.values)))
}

View File

@ -47,7 +47,10 @@ fn range(
let (from, _) = range.from;
let (to, _) = range.to;
let from = *from as usize;
let to = *to as usize;
Ok(OutputStream::from_input(
input.values.skip(*from).take(*to - *from + 1),
input.values.skip(from).take(to - from + 1),
))
}

View File

@ -23,6 +23,7 @@ impl WholeStreamCommand for ReduceBy {
"reduce_with",
SyntaxShape::String,
"the command to reduce by with",
Some('w'),
)
}

View File

@ -0,0 +1,97 @@
use crate::commands::WholeStreamCommand;
use crate::prelude::*;
use indexmap::IndexMap;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, Signature, SyntaxShape, UntaggedValue, Value};
use nu_source::Tagged;
pub struct Rename;
#[derive(Deserialize)]
pub struct Arguments {
column_name: Tagged<String>,
rest: Vec<Tagged<String>>,
}
impl WholeStreamCommand for Rename {
fn name(&self) -> &str {
"rename"
}
fn signature(&self) -> Signature {
Signature::build("rename")
.required(
"column_name",
SyntaxShape::String,
"the name of the column to rename for",
)
.rest(
SyntaxShape::Member,
"Additional column name(s) to rename for",
)
}
fn usage(&self) -> &str {
"Creates a new table with columns renamed."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
args.process(registry, rename)?.run()
}
}
pub fn rename(
Arguments { column_name, rest }: Arguments,
RunnableContext { input, name, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let mut new_column_names = vec![vec![column_name]];
new_column_names.push(rest);
let new_column_names = new_column_names.into_iter().flatten().collect::<Vec<_>>();
let stream = input
.values
.map(move |item| {
let mut result = VecDeque::new();
if let Value {
value: UntaggedValue::Row(row),
tag,
} = item
{
let mut renamed_row = IndexMap::new();
for (idx, (key, value)) in row.entries.iter().enumerate() {
let key = if idx < new_column_names.len() {
&new_column_names[idx].item
} else {
key
};
renamed_row.insert(key.clone(), value.clone());
}
let out = UntaggedValue::Row(renamed_row.into()).into_value(tag);
result.push_back(ReturnSuccess::value(out));
} else {
result.push_back(ReturnSuccess::value(
UntaggedValue::Error(ShellError::labeled_error(
"no column names available",
"can't rename",
&name,
))
.into_untagged_value(),
));
}
futures::stream::iter(result)
})
.flatten();
Ok(stream.to_output_stream())
}

View File

@ -32,11 +32,11 @@ fn reverse(args: CommandArgs, registry: &CommandRegistry) -> Result<OutputStream
let args = args.evaluate_once(registry)?;
let (input, _args) = args.parts();
let output = input.values.collect::<Vec<_>>();
let input = input.values.collect::<Vec<_>>();
let output = output.map(move |mut vec| {
let output = input.map(move |mut vec| {
vec.reverse();
vec.into_iter().collect::<VecDeque<_>>()
futures::stream::iter(vec)
});
Ok(output.flatten_stream().from_input_stream())

View File

@ -10,7 +10,7 @@ pub struct Remove;
#[derive(Deserialize)]
pub struct RemoveArgs {
pub target: Tagged<PathBuf>,
pub rest: Vec<Tagged<PathBuf>>,
pub recursive: Tagged<bool>,
pub trash: Tagged<bool>,
}
@ -22,16 +22,17 @@ impl PerItemCommand for Remove {
fn signature(&self) -> Signature {
Signature::build("rm")
.required("path", SyntaxShape::Pattern, "the file path to remove")
.switch(
"trash",
"use the platform's recycle bin instead of permanently deleting",
Some('t'),
)
.switch("recursive", "delete subdirectories recursively")
.switch("recursive", "delete subdirectories recursively", Some('r'))
.rest(SyntaxShape::Pattern, "the file path(s) to remove")
}
fn usage(&self) -> &str {
"Remove a file"
"Remove file(s)"
}
fn run(

View File

@ -7,6 +7,22 @@ use std::path::{Path, PathBuf};
pub struct Save;
macro_rules! process_unknown {
($scope:tt, $input:ident, $name_tag:ident) => {{
if $input.len() > 0 {
match $input[0] {
Value {
value: UntaggedValue::Primitive(Primitive::Binary(_)),
..
} => process_binary!($scope, $input, $name_tag),
_ => process_string!($scope, $input, $name_tag),
}
} else {
process_string!($scope, $input, $name_tag)
}
}};
}
macro_rules! process_string {
($scope:tt, $input:ident, $name_tag:ident) => {{
let mut result_string = String::new();
@ -31,6 +47,32 @@ macro_rules! process_string {
}};
}
macro_rules! process_binary {
($scope:tt, $input:ident, $name_tag:ident) => {{
let mut result_binary: Vec<u8> = Vec::new();
for res in $input {
match res {
Value {
value: UntaggedValue::Primitive(Primitive::Binary(b)),
..
} => {
for u in b.into_iter() {
result_binary.push(u);
}
}
_ => {
break $scope Err(ShellError::labeled_error(
"Save could not successfully save",
"unexpected data during binary save",
$name_tag,
));
}
}
}
Ok(result_binary)
}};
}
macro_rules! process_string_return_success {
($scope:tt, $result_vec:ident, $name_tag:ident) => {{
let mut result_string = String::new();
@ -98,6 +140,7 @@ impl WholeStreamCommand for Save {
.switch(
"raw",
"treat values as-is rather than auto-converting based on file extension",
Some('r'),
)
}
@ -203,10 +246,10 @@ fn save(
process_string_return_success!('scope, result_vec, name_tag)
}
} else {
process_string!('scope, input, name_tag)
process_unknown!('scope, input, name_tag)
}
} else {
process_string!('scope, input, name_tag)
process_unknown!('scope, input, name_tag)
}
} else {
Ok(string_from(&input).into_bytes())

View File

@ -46,5 +46,5 @@ fn shells(args: CommandArgs, _registry: &CommandRegistry) -> Result<OutputStream
shells_out.push_back(dict.into_value());
}
Ok(shells_out.to_output_stream())
Ok(shells_out.into())
}

View File

@ -0,0 +1,69 @@
use crate::commands::WholeStreamCommand;
use crate::context::CommandRegistry;
use crate::prelude::*;
use nu_errors::ShellError;
use nu_protocol::{ReturnSuccess, ReturnValue, Signature, SyntaxShape, Value};
use nu_source::Tagged;
use rand::seq::SliceRandom;
use rand::thread_rng;
pub struct Shuffle;
#[derive(Deserialize)]
pub struct Arguments {
#[serde(rename = "num")]
limit: Option<Tagged<u64>>,
}
impl WholeStreamCommand for Shuffle {
fn name(&self) -> &str {
"shuffle"
}
fn signature(&self) -> Signature {
Signature::build("shuffle").named(
"num",
SyntaxShape::Int,
"Limit `num` number of rows",
Some('n'),
)
}
fn usage(&self) -> &str {
"Shuffle rows randomly."
}
fn run(
&self,
args: CommandArgs,
registry: &CommandRegistry,
) -> Result<OutputStream, ShellError> {
args.process(registry, shuffle)?.run()
}
}
fn shuffle(
Arguments { limit }: Arguments,
RunnableContext { input, .. }: RunnableContext,
) -> Result<OutputStream, ShellError> {
let stream = async_stream! {
let mut values: Vec<Value> = input.values.collect().await;
let out = if let Some(n) = limit {
let (shuffled, _) = values.partial_shuffle(&mut thread_rng(), *n as usize);
shuffled.to_vec()
} else {
values.shuffle(&mut thread_rng());
values.clone()
};
for val in out.into_iter() {
yield ReturnSuccess::value(val);
}
};
let stream: BoxStream<'static, ReturnValue> = stream.boxed();
Ok(stream.to_output_stream())
}

View File

@ -9,7 +9,7 @@ pub struct Skip;
#[derive(Deserialize)]
pub struct SkipArgs {
rows: Option<Tagged<u64>>,
rows: Option<Tagged<usize>>,
}
impl WholeStreamCommand for Skip {

View File

@ -27,7 +27,7 @@ impl WholeStreamCommand for SplitBy {
}
fn usage(&self) -> &str {
"Creates a new table with the data from the inner tables splitted by the column given."
"Creates a new table with the data from the inner tables split by the column given."
}
fn run(

View File

@ -29,7 +29,7 @@ impl WholeStreamCommand for SplitColumn {
SyntaxShape::Any,
"the character that denotes what separates columns",
)
.switch("collapse-empty", "remove empty columns")
.switch("collapse-empty", "remove empty columns", Some('c'))
.rest(SyntaxShape::Member, "column names to give the new columns")
}

View File

@ -58,7 +58,7 @@ fn split_row(
UntaggedValue::Primitive(Primitive::String(s.into())).into_value(&v.tag),
));
}
result
futures::stream::iter(result)
} else {
let mut result = VecDeque::new();
result.push_back(Err(ShellError::labeled_error_with_secondary(
@ -68,7 +68,7 @@ fn split_row(
"value originates from here",
v.tag.span,
)));
result
futures::stream::iter(result)
}
})
.flatten();

Some files were not shown because too many files have changed in this diff Show More