Commit Graph

98 Commits

Author SHA1 Message Date
319930a1b9 Add streaming support to save for ExternalStream data (#4985)
* Add streaming support to save for ExternalStream data

Prior to this change, save would collect data from an ExternalStream (data
originating from externals) consuming memory for the full amount of data piped
to it,

This change adds streaming support for ExternalStream allowing saving of
arbitrarily large files and bounding memory usage.

* Remove broken save test

This test passes but not for the right reasons, since this test was
written filename has become a required parameter.  The parser outputs
an error but the test still passes as is checking the original un-mutated
file assuming save has re-written the contents.

This change removes the test.

```
running 1 test
=== stderr
Error: nu::parser::missing_positional (https://docs.rs/nu-parser/0.60.0/nu-parser/enum.ParseError.html#variant.MissingPositional)

  × Missing required positional argument.
   ╭─[source:1:1]
 1 │ open save_test_1/cargo_sample.toml | save
   ·                                          ▲
   ·                                          ╰── missing filename
   ╰────
  help: Usage: save {flags} <filename>

test commands::save::figures_out_intelligently_where_to_write_out_with_metadata ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 515 filtered out; finished in 0.10s
```
2022-03-27 15:39:27 +13:00
56a546e73d fix ls when file is a socket on mac (#4983) 2022-03-26 21:26:39 -05:00
71dd857926 Termux/Android target support for v0.60.0 (#4956)
* Add android as target os for procfs-based ps

* Turn off code for dealing with trash on platforms which are known to not support a standard trash protocol

* Update lib.rs

* Update lib.rs

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2022-03-27 07:21:19 +13:00
10d805c1fa feat: fix and update some examples (#4844) 2022-03-14 07:41:09 -05:00
JT
ff3dffd813 Nu glob (#4818)
* Fork glob. Normalise license holder

* Fix more licenses

* unwraps

* bad doc test
2022-03-13 11:30:27 -07:00
f3626f7c3a Update docs for open and decode command, regenerate all docs (#4815)
* Update docs for open and decode command, regenerate all docs

* Update open.rs

* Update open.md

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2022-03-11 05:39:54 -05:00
JT
299fea8538 Fix external extra (#4777)
* Fix empty table from externals

* Fix empty table from externals
2022-03-07 20:17:33 -05:00
4aa9a18c63 Allow save to accept a list of strings (#4743) 2022-03-05 15:56:04 -06:00
1527b34d9c Add back --append flag to save command (#4744) 2022-03-05 13:36:58 -06:00
1157fcf372 fix typo, update some examples and regenerate docs (#4718) 2022-03-04 06:10:09 -06:00
b09acdb7f9 Fix unsupported type message for some math related commands (#4672)
* Fix unsupported type message of some math related commands

* changing the error form for UnsupportedInput

* cargo fmt
2022-02-28 10:14:33 -05:00
JT
0c3ea636fb Add support for stderr and exit code (#4647) 2022-02-25 14:51:31 -05:00
JT
676457acd3 Better ls paths (#4612)
* Fix ls paths... again

* Fix ls paths... again

* Always expand paths inside of glob_from

* Expand in ls before we check for directory info
2022-02-23 10:54:47 -05:00
968427c4e9 feat: update: #4518, Add example for register,source,save,shuffle and from tsv (#4577) 2022-02-21 06:25:41 -06:00
JT
fd22211737 Add nuon format for fun (#4401)
* Add nuon format for fun

* more fun

* More nuon fixes, allow comments, improve errors
2022-02-20 16:26:41 -05:00
ac99ac003a Add example for cd,transpose,detect columns,split column and split row (#4549) 2022-02-19 09:24:48 -06:00
1235d516a5 Add examples for env,let-env,rm,touch and date list-timezone (#4531)
* feat: update #4518, add examples for env,let-env,rm,touch and date list-timezone

* fix typo

* update example for `date list-timezone` command
2022-02-18 18:19:37 +02:00
dd11be03be feat: update #4518, add command examples for def, do, cp, mv, mkdir and ls (#4528) 2022-02-18 08:30:16 -06:00
JT
393cb7ca6f Treat ls for absolute paths as-is (#4513)
* Absolute paths in ls are treated as-in

* Better fix
2022-02-17 12:49:20 -05:00
926177235c Added quiet flag rm command #4423 (#4430)
* rm now uses -f flag to not print anything

* changed quiet flag to q not f

* Changed value passed to Value::Nothing in rm command
2022-02-12 01:22:40 +02:00
85d1a681c7 Remove stringification for binary values in save command (#4428)
* Remove stringification for binary values in `save`

* Fix typo and clippy warning
2022-02-11 14:26:36 -05:00
e1f98c1bfd Fix trash-support feature flag (#4394)
Pass it through to be inclued with `--all-features`

Make clippy without `--all-features` happy
2022-02-09 18:20:46 -05:00
JT
43850bf20e Re-port filesystem commands (#4387)
* Re-port the filesystem commands

* Remove commented out section
2022-02-09 09:56:27 -05:00
8204cc4f28 fix ls and ls tests (#931)
* fix `ls` and ls tests

* tweak to ls so it doesn't scream on empty dirs

* clippy

* reworked `ls` to put in what was left out
2022-02-04 14:32:13 -06:00
JT
a008f1aa80 Command tests (#922)
* WIP command tests

* Finish marking todo tests

* update

* update

* Windows cd test ignoring
2022-02-03 21:01:45 -05:00
e1c28cf06b add --du to ls command (#917) 2022-02-03 13:58:32 -06:00
JT
e91d8655c6 Only trim prompt (#876)
* Only trim the output for prompts

* Only remove the last newline
2022-01-28 18:22:09 -05:00
JT
4c029d2545 Automatically trim ends of stdin/stdout strings (#874) 2022-01-28 16:59:00 -05:00
JT
86eeb4a5e7 Fix a bad slice into erroring utf-8 buffer (#872) 2022-01-28 15:32:46 -05:00
JT
020ad24b25 "maybe text codec" version 2 (#871)
* Add a RawStream that can be binary or string

* Finish up updating the into's
2022-01-28 13:32:33 -05:00
JT
8ee619954d Start support for commandline args to nu itself (#851)
* cmdline args wip

* WIP

* redirect working

* Add help and examples

* Only show flags in signature of more than help
2022-01-27 01:42:39 +11:00
JT
988a873466 Allow open to read its filename from input (#841)
* Allow `open` to read its filename from input

* Add examples
2022-01-25 08:04:28 +11:00
JT
62e9698b11 Allow external args to expand globs (#839)
* Allow external args to expand globs

* WIP

* A bit of cleanups and refactor to glob_from

* oops, add file
2022-01-25 05:26:56 +11:00
4e171203cc Fix cd-ing into a file (#831)
* Add custom error for path not being a directory

* Fix cd issue with cd-ing into a file

* Keep formatting style as before

* Check if path is not a directory and return error if that's the case
2022-01-23 15:02:12 +02:00
JT
b4e61a056c add cd - support (#787) 2022-01-21 07:51:44 +11:00
bfe3c50dce Fix empty entry in ls (#759) 2022-01-17 02:40:00 +11:00
3b4baa31b6 Fix ls relative path & command argument path expansion (#757)
* Switch to short-names when the path is a relative_path (a dir) and exit with an error if the path does not exist

* Remove debugging print line

* Show relative filenames... It does not work yet for ls ../

* Try something else to fix relative paths... it works, but the ../ code part is not very pretty

* Add canonicalize check and remove code clones

* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested

* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested

* Add single-dot expansion to nu-path

* Move value path expansion from parser to eval

Fixes #745

* Remove single dot expansion from parser

It is not necessary since it will get expanded anyway in the eval.

* Fix ls to display globs with relative paths

* Use pathdiff crate to get relative paths for ls

Co-authored-by: Stefan Stanciulescu <contact@stefanstanciulescu.com>
2022-01-17 00:55:56 +11:00
JT
b78924c777 Add support for load-env (#752) 2022-01-15 18:50:11 -05:00
JT
75db4a75bc Save (#750)
* Add support for save

* Add support for binary filetypes
2022-01-16 07:44:20 +11:00
21a7278259 Revert "Fix ls relative path and erroring on fake dir (#697)" (#744)
This reverts commit bee5ba3deb.
2022-01-15 12:58:24 +02:00
bee5ba3deb Fix ls relative path and erroring on fake dir (#697)
* Switch to short-names when the path is a relative_path (a dir) and exit with an error if the path does not exist

* Remove debugging print line

* Show relative filenames... It does not work yet for ls ../

* Try something else to fix relative paths... it works, but the ../ code part is not very pretty

* Add canonicalize check and remove code clones

* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested

* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested
2022-01-15 12:30:39 +02:00
d0c280f6cc Fixes how environment is cloned inside tight loops (#678)
* Improve cd IO error

* Fix environment cloning in loops

* Remove debug print

* Fmt
2022-01-06 09:21:26 +11:00
JT
14cd798f00 Make ls more forgiving (#681) 2022-01-06 09:21:15 +11:00
JT
058738c48c More shell fixes (#673) 2022-01-05 17:36:42 +11:00
JT
affb9696c7 Fix directory change lag (#672) 2022-01-05 16:50:27 +11:00
74dcd91cc3 Use only $nu.env.PWD for getting the current directory (#587)
* Use only $nu.env.PWD for getting current directory

Because setting and reading to/from std::env changes the global state
shich is problematic if we call `cd` from multiple threads (e.g., in a
`par-each` block).

With this change, when engine-q starts, it will either inherit existing
PWD env var, or create a new one from `std::env::current_dir()`.
Otherwise, everything that needs the current directory will get it from
`$nu.env.PWD`. Each spawned external command will get its current
directory per-process which should be thread-safe.

One thing left to do is to patch nu-path for this as well since it uses
`std::env::current_dir()` in its expansions.

* Rename nu-path functions

*_with is not *_relative which should be more descriptive and frees
"with" for use in a followup commit.

* Clone stack every each iter; Fix some commands

Cloning the stack each iteration of `each` makes sure we're not reusing
PWD between iterations.

Some fixes in commands to make them use the new PWD.

* Post-rebase cleanup, fmt, clippy

* Change back _relative to _with in nu-path funcs

Didn't use the idea I had for the new "_with".

* Remove leftover current_dir from rebase

* Add cwd sync at merge_delta()

This makes sure the parser and completer always have up-to-date cwd.

* Always pass absolute path to glob in ls

* Do not allow PWD a relative path; Allow recovery

Makes it possible to recover PWD by proceeding with the REPL cycle.

* Clone stack in each also for byte/string stream

* (WIP) Start moving env variables to engine state

* (WIP) Move env vars to engine state (ugly)

Quick and dirty code.

* (WIP) Remove unused mut and args; Fmt

* (WIP) Fix dataframe tests

* (WIP) Fix missing args after rebase

* (WIP) Clone only env vars, not the whole stack

* (WIP) Add env var clone to `for` loop as well

* Minor edits

* Refactor merge_delta() to include stack merging.

Less error-prone than doing it manually.

* Clone env for each `update` command iteration

* Mark env var hidden only when found in eng. state

* Fix clippt warnings

* Add TODO about env var reading

* Do not clone empty environment in loops

* Remove extra cwd collection

* Split current_dir() into str and path; Fix autocd

* Make completions respect PWD env var
2022-01-05 09:30:34 +11:00
7faa4fbff4 revert file_types to lowercase (#623)
* revert file_types to lowercase

* fix test
2021-12-29 21:16:50 -06:00
JT
de30236f38 Fix ls listing (#593) 2021-12-27 12:46:32 +11:00
JT
89a000a572 Fix some 'open' signature stuff (#583) 2021-12-26 09:13:43 +11:00
JT
a811eee6b8 Add support for 'open' (#573) 2021-12-25 06:24:55 +11:00