Commit Graph

5828 Commits

Author SHA1 Message Date
Andrew Barnes
a64e0956cd
Support binary data to stdin of run-external (#4984)
* Add test for passing binary data through externals

This change adds an ignored test to confirm that binary data is passed
correctly between externals to be enabled in a later commit along with
the fix.

To assist in platform agnostic testing of binary data a couple of
additional testbins were added to allow testing on `Value::Binary` inside
`ExternalStream`.

* Support binary data to stdin of run-external

Prior to this change, any pipeline producing binary data (not detected
as string) then feed into an external would be ignored due to
run-external only supporting `Value::String` on stdin.

This change adds binary stdin support for externals allowing something
like this for example:

  〉^cat /dev/urandom | ^head -c 1MiB | ^pv -b | ignore
  1.00MiB

This would previously output `0.00 B [0.00 B/s]` due to the data not
being pushed to stdin at each stage.
2022-03-27 15:35:59 +13:00
Andrew Barnes
91e17d2f9f
Limit mem usage + back-pressure via bounded channels (#4986)
Prior to this change, a pipeline of externals would result in high memory
usage if any of the producers in the chain, produced data faster than
the consumers.

For example a pipeline:

  > fast-producer | slow-consumer

Would cause a build up of `Value::{String,Binary}`'s in the mpsc channels
between each command as values are added to the channels faster than they
are consumed, eventually OOM'ing depnding on system resources, the volume
of data and speed diff. between fast v's slow.

This change replaces the unbounded channels with bounded channels
to limit the number of values that can build up and providing
back-pressure to limit ram usage.
2022-03-27 15:34:34 +13:00
Darren Schroeder
56a546e73d
fix ls when file is a socket on mac (#4983) 2022-03-26 21:26:39 -05:00
JT
cf88c8eef3
Improve escaping in string interpolation (#4982) 2022-03-27 12:52:09 +13:00
Jakub Žádník
3484e0defd
Add parser keyword note to help and $nu.scope (#4978) 2022-03-26 21:22:45 +02:00
Jakub Žádník
79e4d35f01
Remove is_private from $nu.scope.commands (#4979) 2022-03-26 21:22:35 +02:00
Scott Boggs
71dd857926
Termux/Android target support for v0.60.0 (#4956)
* Add android as target os for procfs-based ps

* Turn off code for dealing with trash on platforms which are known to not support a standard trash protocol

* Update lib.rs

* Update lib.rs

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2022-03-27 07:21:19 +13:00
Hristo Filaretov
7a789d68a2
Don't include trailing separator when expanding tilde (#4974)
* Fix path when expanding tilde

Expanding tilde with no other relative paths would result in:
`$HOME/` instead of `$HOME`. This occurs when users run `cd` with
no extra arguments. In that case, the user's PWD would include the
trailing separator. This does not happen when explicitly passing
a value, such as `cd ~`, because in that case, the path would be
canonicalized.

This happens because std::path::PathBuf::push always adds a separator,
even if adding an empty path, which is what happens when `cd` is
invoked.

* Add test

* Fix test on Windows

Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
2022-03-27 06:28:31 +13:00
Hristo Filaretov
8a9cc33aac
Fix alias import (#4968)
* Fix alias import

Alias importing was registering the alias id as a decl instead of alias.
This caused issues when hiding and then reimporting the alias.

* Clippy format

Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
2022-03-25 17:56:40 -05:00
JT
66087b01e6
Improve the 'use' and 'source' errors (#4966)
* Improve the 'use' and 'source' errors

* Add register
2022-03-26 10:43:46 +13:00
JT
19fa41b114
Fix single quote environment values (#4960)
* Fix single quote external values

* Try to fix windows

* fix test

* fix test
2022-03-26 09:14:48 +13:00
JT
91cd1717e9
Add escapes to 'to nuon' (#4964) 2022-03-26 08:35:37 +13:00
JT
12b85beecc
Fix path join on streams (#4959) 2022-03-26 07:46:48 +13:00
Michael Angerman
2252833917
bump cargo crate sys-locale to the latest version (#4957) 2022-03-25 10:00:35 -07:00
JT
4e9c1067fb
Fix 4946 (#4951)
* Fix reject

* test

* clippy
2022-03-25 20:48:01 +13:00
Michael Angerman
e505e57a7a
align all of the serde_json crates to the same version (#4949) 2022-03-25 18:54:49 +13:00
JT
d122827a30
Fix operator precedence parser (#4947) 2022-03-25 16:23:08 +13:00
Justin Ma
b007290a4e
Fix #4942, and add a table sorting example for sort-by command (#4948)
* Fix #4942, and add a table sorting example for `sort-by` command

* ci skip
2022-03-25 16:22:57 +13:00
Darren Schroeder
7c92791eed
add .mailmap to .gitignore 2022-03-24 20:36:52 -05:00
Randy Barlow
80769b7197
Set the minimum Rust version to 1.59 (#4940)
nushell uses the strip option in two of its profiles in Cargo.toml.
This option is new in Rust 1.59[0], so this commit adjusts Cargo.toml to
mark 1.59 as the minimum supported Rust version[1].

[0] https://blog.rust-lang.org/2022/02/24/Rust-1.59.0.html
[1] https://doc.rust-lang.org/cargo/reference/manifest.html#the-rust-version-field

Signed-off-by: Randy Barlow <randy@electronsweatshop.com>
2022-03-25 08:08:34 +13:00
Michael Angerman
9b5dff828d
bump csv crate to the latest 1.1.6 (#4939) 2022-03-24 12:59:27 -05:00
Tomoki Aonuma
90013295aa
Fix parse_string_strict() to detect unbalanced quotes properly (#4928) 2022-03-25 05:57:03 +13:00
Jonathan Moore
ea7c8c237e
CantConvert improvements (#4926)
* CantConvert improvements

* cargo fmt
2022-03-24 07:04:31 -05:00
JT
5d5b02d8dc
Don't assume external ls (#4925) 2022-03-24 16:42:41 +13:00
Vishal Sodani
00b67d338d
added missing metadata for drop and uniq #4763 (#4908)
* added missing metadata for drop and uniq #4763

* added missing metadata for keep #4763

* added missing metadata for append #4763

* added missing metadata for shuffle #4763
2022-03-24 07:27:01 +13:00
Darren Schroeder
d32e878868
rename export def to export alias (#4912)
copy-n-paste error
2022-03-23 07:53:10 -05:00
Darren Schroeder
41af2e4b30
update link (#4915) 2022-03-23 07:52:49 -05:00
Justin Ma
e9f9aab79f
chore: Update default register examples (#4904) 2022-03-23 20:41:58 +13:00
LebsterFace
e826540037
Pass /D flag to cmd.exe to disable AutoRun (#4903)
* Pass `/D` flag to `cmd.exe` to disable AutoRun

* Pass `/D` flag before `/c`

This avoids running the command '/D <&self.name.item>' in cmd
2022-03-23 19:05:06 +13:00
Darren Schroeder
a435a9924c
add ability to execute on demand (#4896) 2022-03-22 21:09:58 -05:00
JT
02ed15b932
Update Cargo.toml 2022-03-23 09:44:24 +13:00
JT
81e269c483
Update Cargo.toml 2022-03-23 09:44:03 +13:00
JT
eceae26b0a
Update Cargo.toml 2022-03-23 09:39:03 +13:00
JT
ec5fd62f9f
Add licenses (#4893)
* Add licenses

* Add licenses
2022-03-23 09:25:38 +13:00
JT
74af31a42f
Update release.yml 2022-03-23 08:07:11 +13:00
JT
1c964cdfe7
Bump to 0.60 (#4892)
* WIP

* semi-revert metadata change
2022-03-23 07:32:03 +13:00
Darren Schroeder
352cf31db2
update comments, add other targets (#4891)
keep these other targets in case we need them in the future
2022-03-22 08:27:48 -05:00
JT
66e736dab4
Externals shouldn't expand aliases (#4889) 2022-03-22 11:57:48 +13:00
Darren Schroeder
18067138aa
created an alternate way to determine line count (#4887) 2022-03-21 11:56:14 -05:00
Darren Schroeder
bd7a506897
update size command to be more accurate (#4885) 2022-03-20 17:09:30 -05:00
Homa Wong
1d38ff071e
fix: typo (#4882)
Fix a typo in the default config
2022-03-20 07:49:00 -05:00
JT
e6a5011fdb
Allow 'error make' to make simple errors (#4881)
* Allow 'error make' to make simple errors

* Add example
2022-03-20 16:25:45 +13:00
JT
d5f23ab592
Put completions in their own module (#4880) 2022-03-20 12:03:58 +13:00
JT
bd5778fa24
remove the boolean vars (#4879) 2022-03-20 08:12:10 +13:00
JT
f3bb1d11d3
Add export alias and export extern (#4878)
* export alias

* export extern
2022-03-20 07:58:01 +13:00
Darren Schroeder
285f91e67a
add module name to $nu.scope.commands info (#4877) 2022-03-19 10:58:56 -05:00
Darren Schroeder
01c1e5e8b0
commands are either custom or builtin, not both (#4876)
* commands are either custom or builtin, not both

* clippy
2022-03-19 09:52:50 -05:00
Fernando Herrera
d6669d3f33
Polars update (#4875)
* update to polars 0.20

* add to date parser for series
2022-03-19 11:13:34 +00:00
Jakub Žádník
3db608eb5c
Re-enable virtualenv tests (#4755) 2022-03-19 00:36:38 +02:00
JT
b293282e9b
Add insert/update to lists (#4873) 2022-03-19 10:12:54 +13:00