* updated to reedline generic menus
* help menu with examples
* generic menus in the engine
* description menu template
* list of menus in config
* default value for menu
* menu from block
* generic menus examples
* change to reedline git path
* cargo fmt
* menu name typo
* remove commas from default file
* added error message
* Remove panic from BlockCommands run function
Instead of panicing, the run method now returns an error to prevent
nushell from unexpected termination.
* Add ability to open command to run with blocks
The open command tries to parse the content of the file
if there is a command called 'from (file ending)'. This works
fine if the command was 'built in' because the run method doesn't
fail in this case. It did fail on a BlockCommand, though.
This change will first probe if the command contains a block and
evaluate it, if this is the case. If there is no block, it will run
the command the same way as before.
* Add test open files with BlockCommands
* Update open.rs
* Adjust file type on open with BlockCommand parser
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
Enables the use of some features on reedline
- Keeping the line when clearing the screen with `Ctrl-L`
- Using the internal cut buffer between lines
- Submitting external commands via keybinding and keeping the line
Additional effect:
Keep the history around and do basic syncs (performance improvement
minimal as session changes have to be read and written)
Additional change:
Give the option to defer writing/rereading the history file to the
closing of the session ($config.sync_history_on_enter)
* fix(filters): pass metadata for select
* fix(filters): pass metadata for group, window
* fix(filters): pass metadata for each, every
* fix(filters): pass metadata for collect, compact, flatten
* fix(filters): pass metadata for get
* fix: get rid of necessary `.clone()``
* style: rename closure call to be consistent w/ others
* fix(filters): pass metadata for par-each, prepend
* fix(filters): pass metadata for range
* fix(filters): pass metadata for reject
* fix(filters): pass metadata for more commands
* style: run cargo fmt
* Add search terms to command
* Rename Signature desc to usage
To be named uniformly with extra_usage
* Throw in foldl search term for reduce
* Add missing usage to post
* Add search terms to signature
* Try to add capnp Signature serialization
* nu-completer with suggestions
* help menu with scrolling
* updates description rows based on space
* configuration for help menu
* update nu-ansi-term
* corrected test for update cells
* changed keybinding
* Add streaming support to save for ExternalStream data
Prior to this change, save would collect data from an ExternalStream (data
originating from externals) consuming memory for the full amount of data piped
to it,
This change adds streaming support for ExternalStream allowing saving of
arbitrarily large files and bounding memory usage.
* Remove broken save test
This test passes but not for the right reasons, since this test was
written filename has become a required parameter. The parser outputs
an error but the test still passes as is checking the original un-mutated
file assuming save has re-written the contents.
This change removes the test.
```
running 1 test
=== stderr
Error: nu::parser::missing_positional (https://docs.rs/nu-parser/0.60.0/nu-parser/enum.ParseError.html#variant.MissingPositional)
× Missing required positional argument.
╭─[source:1:1]
1 │ open save_test_1/cargo_sample.toml | save
· ▲
· ╰── missing filename
╰────
help: Usage: save {flags} <filename>
test commands::save::figures_out_intelligently_where_to_write_out_with_metadata ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 515 filtered out; finished in 0.10s
```
* Add test for passing binary data through externals
This change adds an ignored test to confirm that binary data is passed
correctly between externals to be enabled in a later commit along with
the fix.
To assist in platform agnostic testing of binary data a couple of
additional testbins were added to allow testing on `Value::Binary` inside
`ExternalStream`.
* Support binary data to stdin of run-external
Prior to this change, any pipeline producing binary data (not detected
as string) then feed into an external would be ignored due to
run-external only supporting `Value::String` on stdin.
This change adds binary stdin support for externals allowing something
like this for example:
〉^cat /dev/urandom | ^head -c 1MiB | ^pv -b | ignore
1.00MiB
This would previously output `0.00 B [0.00 B/s]` due to the data not
being pushed to stdin at each stage.
Prior to this change, a pipeline of externals would result in high memory
usage if any of the producers in the chain, produced data faster than
the consumers.
For example a pipeline:
> fast-producer | slow-consumer
Would cause a build up of `Value::{String,Binary}`'s in the mpsc channels
between each command as values are added to the channels faster than they
are consumed, eventually OOM'ing depnding on system resources, the volume
of data and speed diff. between fast v's slow.
This change replaces the unbounded channels with bounded channels
to limit the number of values that can build up and providing
back-pressure to limit ram usage.
* Add android as target os for procfs-based ps
* Turn off code for dealing with trash on platforms which are known to not support a standard trash protocol
* Update lib.rs
* Update lib.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Fix path when expanding tilde
Expanding tilde with no other relative paths would result in:
`$HOME/` instead of `$HOME`. This occurs when users run `cd` with
no extra arguments. In that case, the user's PWD would include the
trailing separator. This does not happen when explicitly passing
a value, such as `cd ~`, because in that case, the path would be
canonicalized.
This happens because std::path::PathBuf::push always adds a separator,
even if adding an empty path, which is what happens when `cd` is
invoked.
* Add test
* Fix test on Windows
Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
* Fix alias import
Alias importing was registering the alias id as a decl instead of alias.
This caused issues when hiding and then reimporting the alias.
* Clippy format
Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
* added missing metadata for drop and uniq #4763
* added missing metadata for keep #4763
* added missing metadata for append #4763
* added missing metadata for shuffle #4763
* add a display of what the colors look like in `ansi --list`
* change 'color' to 'preview' - add the ability to turn it off via config with use_ansi_coloring
To simplify use of nu-ansi-term in both nushell/nushell and
nushell/reedline remove it from the workspace to have a separate
progression of version numbers.
This allows reedline to use the latest published version and nushells
workspace to use the same most recent version
Changes the `Cargo.toml`s to use reedline from crates.io
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Refactor usage of is_perf_true to be a parameter passed around
* Move repl loop and command/script execution to nu_cli
* Move config setup out of nu_cli
* Update config_files.rs
* Update main.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Refactor & fix which
Instead of fetching all definitions / aliases, only show the one that is
visible.
* Fix $nu.scope to show only visible definitions
* Add missing tests file; Rename one which test
* Do not pass non-string env vars to externals
Also misc cleanup
* Add note to default config
* Add a test
* Ensure PATH/Path conversion list <-> string
* Update docs for open and decode command, regenerate all docs
* Update open.rs
* Update open.md
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
At the moment `crossterm` apparently has a regression decoding certain important key combinations on Windows.
Thus reedline reverted to the previous version.
Some changes are necessary to remove the need for `crossterm` in the use of `lscolors`.
Introduces two local conversion traits.
Additionally update the `Highlighter` API to support the cursor
position.
This will enable brace/statement match highlighting.
Previously, the parser tried to look up the predecl also in the
permanent state and if a definition with that name already existed, it
would try to update it, which is illegal.
* Handle string->value env conv. with env. var.
Also adds the environment variable for Path/PATH and removes it from
config.
* Simplify getting the string->value conversion
* Refactor env conversion into its own function
* Use env var for to_string conversion; Remove conf
* Fix indentation in default config
This restores a basic version of the `post` command.
Some source types have been omitted from this first take.
I copied from `fetch` and from `post`@0.40.0.
Part of #4356
* Fix failing pipeline()
The `skip(1)` was there likely to remove the welcome message.
* Fix typo
* Fix nu! test macro to enter cwd correctly
Nushell's current working directory is determined primarily by the PWD
environment variable.
* Add `date format --list`, and make format string optional (providing default)
* Make DRY
`into datetime --list` now uses `generate_strfttime_list` from `date format --list`
* refactor strftime to use current datetime
* Fix formatting of specification descriptions
Fixes issues caused when copying directly from docs.rs
* Change default format to rfc2822
Perhaps to make it more DRY, functions from `into datetime` can be used. However, currently `into datetime` is a bit tricky to use as it needs a separate time zone argument.
* Tweak in-shell docs to match modified behavior
* Show %#z format specifier in `into datetime --list` only
* cargo fmt
* Satisfy clippy
* Add different features combinations
* Specify styles manually
* Fix args
* Fix typo
* Let other CI jobs finish if one fails
* Fix unused symbols without plugin feature
* Put "which" tests behind "which" feature
* Add Python virtualenv job
* Oops forgot git command
* Install Nushell in virtualenv tests
* Add names to steps; Test v.env in separate step
* cd into virtualenv
* Do not run on Python 2.7
* Build Nushell after formatting and clippy checks
* Add alias interning
Now, AliasId is used to reference aliases stored in EngineState, similar
to decls, blocks, etc.
* Fix wrong message
* Fix using decl instead of alias
* Extend also alias id visibility
* Merge also aliases from delta
* Add alias hiding code
Does not work yet but passes tests at least.
* Fix wrong alias lookup and visibility appending
* Add hide alias tests
* Fmt & Clippy
* Fix random clippy warnings in "which" command
* Fix "index out of bounds" when input to group-by is empty #4369
* Fix formatting #4369
* Adds test for empty input
Co-authored-by: Ray Henry <ray.henry@thermofisher.com>
Same rationale as in #4378
Also accelerate `grid`
before:
```
Command being timed: "./eager_nu -c for i in 0..100000 { echo whatever } | grid"
User time (seconds): 0.21
System time (seconds): 0.05
Percent of CPU this job got: 36%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.71
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 48112
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 10580
Voluntary context switches: 266
Involuntary context switches: 2595
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
after:
```
Command being timed: "./lazy_nu -c for i in 0..100000 { echo whatever } | grid"
User time (seconds): 0.14
System time (seconds): 0.05
Percent of CPU this job got: 33%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.60
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 48272
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1
Minor (reclaiming a frame) page faults: 10582
Voluntary context switches: 286
Involuntary context switches: 831
Swaps: 0
File system inputs: 56
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
For the width calculations for table layout the `strip_ansi` function
has to be called frequently. By checking for the ASCII control chars
(0x00 to 0x1f) except `\n` that are stripped by `strip_ansi_escapes` the number of
necessary allocations can be reduced significantly for the simple case
of text not containing ANSI escapes.
**Benchmark:**
```
nu -c "for i in 0..1000 { ls } | flatten | table"
```
**Allocation reduction**
Running on the nushell repo root as the directory, this change reduces the
allocation volume by approximately 400 MB
(Measured run via KDE heaptrack)
**Speed improvement to output**
Measured via `/usr/bin/time -v`
*before*
```
Command being timed: "./eager_nu -c for i in 0..1000 {ls} | flatten | table"
User time (seconds): 0.87
System time (seconds): 0.14
Percent of CPU this job got: 87%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.16
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 18888
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 4809
Voluntary context switches: 38
Involuntary context switches: 14
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
*after*
```
Command being timed: "./lazy_nu -c for i in 0..1000 {ls} | flatten | table"
User time (seconds): 0.63
System time (seconds): 0.14
Percent of CPU this job got: 80%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.97
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 18660
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 5149
Voluntary context switches: 24
Involuntary context switches: 5
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
* Allow range in 'drop nth'
* Unit tests for drop nth range
* Add range case to the description
* Fix description 2
* format fixes
* Fix example and some refactoring
* clippy fixes
* fix into filesize tests and filesize
* tweaks
* added span back for like the 10th time
* Update filesize.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* query command with json, web, xml
* query xml now working
* clippy
* comment out web tests
* Initial work on query web
For now we can query everything except tables
* Support for querying tables
Now we can query multiple tables just like before, now the only thing
missing is the test coverage
* Revert "Query plugin"
* augment `ps -l` on windows to display more info
Co-authored-by: Luccas Mateus de Medeiros Gomes <luccasmmg@gmail.com>
* query command with json, web, xml
* query xml now working
* clippy
* comment out web tests
* Initial work on query web
For now we can query everything except tables
* Support for querying tables
Now we can query multiple tables just like before, now the only thing
missing is the test coverage
* finish off
* comment out web test
Co-authored-by: Luccas Mateus de Medeiros Gomes <luccasmmg@gmail.com>
* Port update cells command
Clean up, nicer match statements in UpdateCellsIterator
Return columns flag into HashSet errors
Add FIXME: for update cell behavior on nested lists
* Fix: process cells for Record when no columns are specified
* Fix: address clippy lints for unwrap and into_iter
* Fix: don't step into lists and don't bind $it var
* Maybe solve the none bug?
* cargo fmt
* use nothing, not string
* check at last
* I check it at last
* Use error which has span
* use not found error
* fix error
* use a empty value length?
* * Add commit about what I change and fmt
Now all test passed, but I do not know if it is right
* update the test
* check if it is nothing
* update commit
* Rename test
Co-authored-by: Jakub Žádník <kubouch@gmail.com>
* Add rotate command
* Add rotate counter clockwise
* Fix comments in the code
* Fix clippy warnings
* Fix comment
* Fix wrong step for non even table sizes
* Fix comment for moving through array
* Refactor rotate and have only one command with a --ccw flag for counter-clockwise rotation. By default, rotate is clockwise
* Update usage description
* Add `Find` command
* Complete rustdoc for test `Value` constructors
* Use `Option::unwrap_or` instead of match
* Add `Value::test_filesize` constructor
* Handle searching for terms in `find`
* Fix `find` command signature
* Return multiple elements when `find`ing by predicate
* Do not accept rest parameter with predicate
* Handle `CellPath` in `r#in` and `not_in` for `Find`
* Use `PipelineData::filter`
* Add custom error for path not being a directory
* Fix cd issue with cd-ing into a file
* Keep formatting style as before
* Check if path is not a directory and return error if that's the case
* Add example test to zip
* Port merge command from Nushell
On top of the original merge, this one should not collect a stream
returned from the merged block and allows merging records.
Ansi-cut expects ranges of character numbers (of the non-ansi control
sequence characters) instead of byte indices.
This fixes the panics when wrapping of non-unicode lines (which exceed
the demanded number of characters as byte indices).
Also rectifies some wrong wrapping of unicdoe containing lines that
don't panic
* Switch to short-names when the path is a relative_path (a dir) and exit with an error if the path does not exist
* Remove debugging print line
* Show relative filenames... It does not work yet for ls ../
* Try something else to fix relative paths... it works, but the ../ code part is not very pretty
* Add canonicalize check and remove code clones
* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested
* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested
* Add single-dot expansion to nu-path
* Move value path expansion from parser to eval
Fixes#745
* Remove single dot expansion from parser
It is not necessary since it will get expanded anyway in the eval.
* Fix ls to display globs with relative paths
* Use pathdiff crate to get relative paths for ls
Co-authored-by: Stefan Stanciulescu <contact@stefanstanciulescu.com>
* Switch to short-names when the path is a relative_path (a dir) and exit with an error if the path does not exist
* Remove debugging print line
* Show relative filenames... It does not work yet for ls ../
* Try something else to fix relative paths... it works, but the ../ code part is not very pretty
* Add canonicalize check and remove code clones
* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested
* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested
* Make env var eval order during "use" deterministic
Fixes#726.
* Merge delta after getting config
To make sure env vars are all in the engine state and not in the stack.
* Move fetch to extra
* Move byte stream code to a function instead of copying it twice
* Fix formatting issues
* Make fetch a default command
* Fix formatting
* Use only $nu.env.PWD for getting current directory
Because setting and reading to/from std::env changes the global state
shich is problematic if we call `cd` from multiple threads (e.g., in a
`par-each` block).
With this change, when engine-q starts, it will either inherit existing
PWD env var, or create a new one from `std::env::current_dir()`.
Otherwise, everything that needs the current directory will get it from
`$nu.env.PWD`. Each spawned external command will get its current
directory per-process which should be thread-safe.
One thing left to do is to patch nu-path for this as well since it uses
`std::env::current_dir()` in its expansions.
* Rename nu-path functions
*_with is not *_relative which should be more descriptive and frees
"with" for use in a followup commit.
* Clone stack every each iter; Fix some commands
Cloning the stack each iteration of `each` makes sure we're not reusing
PWD between iterations.
Some fixes in commands to make them use the new PWD.
* Post-rebase cleanup, fmt, clippy
* Change back _relative to _with in nu-path funcs
Didn't use the idea I had for the new "_with".
* Remove leftover current_dir from rebase
* Add cwd sync at merge_delta()
This makes sure the parser and completer always have up-to-date cwd.
* Always pass absolute path to glob in ls
* Do not allow PWD a relative path; Allow recovery
Makes it possible to recover PWD by proceeding with the REPL cycle.
* Clone stack in each also for byte/string stream
* (WIP) Start moving env variables to engine state
* (WIP) Move env vars to engine state (ugly)
Quick and dirty code.
* (WIP) Remove unused mut and args; Fmt
* (WIP) Fix dataframe tests
* (WIP) Fix missing args after rebase
* (WIP) Clone only env vars, not the whole stack
* (WIP) Add env var clone to `for` loop as well
* Minor edits
* Refactor merge_delta() to include stack merging.
Less error-prone than doing it manually.
* Clone env for each `update` command iteration
* Mark env var hidden only when found in eng. state
* Fix clippt warnings
* Add TODO about env var reading
* Do not clone empty environment in loops
* Remove extra cwd collection
* Split current_dir() into str and path; Fix autocd
* Make completions respect PWD env var
* Port fetch to engine-q
* Fix check for path as a string
* Add a timeout flag and fix some span issues
* Add a temporary fetch command that returns byte streams. Got rid of async stuff as we're using the blocking feature of tokio
* More tweaks for the bytestream
* Rewrite fetch using ByteStreams
* buffer read on bytes directly
Co-authored-by: Stefan Stanciulescu <contact@stefanstanciulescu.com>
* get_columns is working in the columns command
* the new location of the get_columns method is nu-protocol/src/column.rs
* reference the new location of the get_columns method
* move get_columns to nu-engine
