* nu-cli/completions: fix paths with special chars
* add backticks
* fix replace
* added single quotes to check list
* check escape using fold
* fix clippy errors
* fix comment line
* fix conflicts
* change to vec
* skip sort checking
* removed invalid windows path
* remove comment
* added tests for escape function
* fix fn import
* fix fn import error
* test windows issue fix
* fix windows backslash path in the tests
* show expected path on error
* skip test for windows
* Added search terms to math commands
* Attempts to add ~user.
From: // Extend this to work with "~user" style of home paths
* Clippy recommendation
* clippy suggestions, again.
* fixing non-compilation on windows and macos
* fmt apparently does not like my imports
* even more clippy issues.
* less expect(), single conversion, match. Should work for MacOS too.
* Attempted to add functionality for windows: all it does is take the home path of current user, and replace the username.
* silly mistake in Windows version of user_home_dir()
* Update tilde.rs
* user_home_dir now returns a path instead of a string - should be smoother with no conversions to string
* clippy warnings
* clippy warnings 2
* Changed user_home_dir to return PathBuf now.
* Changed user_home_dir to return PathBuf now.
* forgot to fmt
* fixed windows build errors from modifying pathbuf but not returning it
* fixed windows clippy errors from returning () instead of pathbuf
* forgot to fmt
* borrowed path did not live long enough.
* previously, path.push did not work because rest_of_path started with "/" - it was not relative. Removing the / makes it a relative path again.
* Issue fixed.
* Update tilde.rs
* fmt.
* There is now a zero chance of panic. All expect()s have been removed.
* Patched join_path_relative to accommodate ~user paths. Previously, /some/path/~user might have been passed on; now, ~user is taken as absolute.
* fmt
* clippy errors
* Added search terms to math commands
* Attempts to add ~user.
From: // Extend this to work with "~user" style of home paths
* Clippy recommendation
* clippy suggestions, again.
* fixing non-compilation on windows and macos
* fmt apparently does not like my imports
* even more clippy issues.
* less expect(), single conversion, match. Should work for MacOS too.
* Attempted to add functionality for windows: all it does is take the home path of current user, and replace the username.
* silly mistake in Windows version of user_home_dir()
* Update tilde.rs
* user_home_dir now returns a path instead of a string - should be smoother with no conversions to string
* clippy warnings
* clippy warnings 2
* Changed user_home_dir to return PathBuf now.
* Changed user_home_dir to return PathBuf now.
* forgot to fmt
* fixed windows build errors from modifying pathbuf but not returning it
* fixed windows clippy errors from returning () instead of pathbuf
* forgot to fmt
* borrowed path did not live long enough.
* previously, path.push did not work because rest_of_path started with "/" - it was not relative. Removing the / makes it a relative path again.
* Issue fixed.
* Update tilde.rs
* fmt.
* There is now a zero chance of panic. All expect()s have been removed.
* database commands
* db commands
* filesystem opens sqlite file
* clippy error
* corrected error in ci file
* removes matrix flag from ci
* flax matrix for clippy
* add conditional compile for tests
* add conditional compile for tests
* correct order of command
* correct error msg
* correct typo
* Pass completion options to each fetch() call
* Add MatchAlgorithm to CompletionOptions
* Add unit test for MatchAlgorithm
* Pass completion options to directory completer
Changes to reedline since `v0.4.0`:
- vi normal mode `I` for inserting at line beginning
- `InsertNewline` edit command that can be bound to `Alt-Enter` if
desired to have line breaks without relying on the `Validator`
- `ClearScreen` will directly clear the visible screen. `Signal::CtrlL` has been
removed.
- `ClearScrollback` will clear the screen and scrollback. Can be used to
mimic macOS `Cmd-K` screen clearing. Helps with #5089
* nu-cli: added tests for file completions
* test adding extra sort
* Feature/refactor completion options (#5228)
* Copy completion filter to custom completions
* Remove filter function from completer
This function was a no-op for FileCompletion and CommandCompletion.
Flag- and VariableCompletion just filters with `starts_with` which
happens in both completers anyway and should therefore also be a no-op.
The remaining use case in CustomCompletion was moved into the
CustomCompletion source file.
Filtering should probably happen immediately while fetching completions
to avoid unnecessary memory allocations.
* Add get_sort_by() to Completer trait
* Remove CompletionOptions from Completer::fetch()
* Fix clippy lints
* Apply Completer changes to DotNuCompletion
* add os to $nu based on rust's understanding (#5243)
* add os to $nu based on rust's understanding
* add a few more constants
Co-authored-by: Richard <Tropid@users.noreply.github.com>
Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
* Copy completion filter to custom completions
* Remove filter function from completer
This function was a no-op for FileCompletion and CommandCompletion.
Flag- and VariableCompletion just filters with `starts_with` which
happens in both completers anyway and should therefore also be a no-op.
The remaining use case in CustomCompletion was moved into the
CustomCompletion source file.
Filtering should probably happen immediately while fetching completions
to avoid unnecessary memory allocations.
* Add get_sort_by() to Completer trait
* Remove CompletionOptions from Completer::fetch()
* Fix clippy lints
* Apply Completer changes to DotNuCompletion
Prior to this change we would recover the names for known
externals by looking up the span in the engine state. This would fail
when using an alias for two reasons:
1. In cases where we don't have a subcommand, like this:
```
>>> extern bat [filename: string]
>>> alias b = bat
>>> bat some_file
'b' is not recognized as an internal or external command,
operable program or batch file.
```
The problem is that after alias expansion, we replace the span of the
expanded name with the original alias (this is done to alleviate
non-related issues). The span contents we look up therefore contain `b`,
the alias, instead of the expanded command name.
2. In cases where there's a subcommand:
```
>>> alias g = git
>>> g push
thread 'main' panicked at 'internal error: span missing in file contents cache', crates\nu-protocol\src\engine\engine_state.rs:474:9
note: run with `RUST_BACKTRACE=1` environment variable to display a
backtrace
```
In this case, the span in call starts where the expansion for the `g`
alias is defined and end after `push` on the last command entered. This
is not a proper span and causes a panic when we try to look it up. Note
that this is the case for all expanded aliases that involve a
subcommand, but we never actually try to retrieve the contents for that
span in other cases.
Anyway, the new way of looking up the name is arguably cleaner
regardless of the issues mentioned above. But it's nice that it fixes
them too.
Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
This commit renders ANSI chars in order to provide shell integrations
such Kitty's opening feature that captures the output of the last
command in a pager such as less.
Fixes#5138
- With a change to reedline hints can now be hidden. This is useful when
no ANSI coloring is available as hints become indistinguishable from the
actual buffer
- remove commented out code
- order the logging calls according to the implementation
I don't personally agree with this; I'd prefer less magic,
and not expanding _anything_ except `~` as an initial path element
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
To remove a default keybinding for a particular edit mode, set the `event: null`:
e.g. to disable screen clearing with Ctrl-L
```
let $config = {keybindings: [{
modifier: control
keycode: char_l
mode: [emacs, vi_normal, vi_insert]
event: null
} ]}
```
* Remove unused packages from base Cargo.toml
* Remove unused crossterm_winapi from nu-cli
* Remove unused dependencies from nu-system
* Remove unused dependencies from nu-test-support
* Initial implementation of ordered call args
* Run cargo fmt
* Fix some clippy lints
* Add positional len and nth
* Cargo fmt
* Remove more old nth calls
* Good ole rustfmt
* Add named len
Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
* Include license text in all crates
Three crates already have license texts, so I'm keeping them, but
symlinking the `LICENSE` from the top level to the rest of the crate
directories. This works as long as `cargo publish` is done on a Unix-y
system and not Windows.
Also bump the copyright year to end in 2022.
Signed-off-by: Michel Alexandre Salim <salimma@fedoraproject.org>
* Replace symlinks
Co-authored-by: sholderbach <sholderbach@users.noreply.github.com>
* Add timestamp flag to `touch` command
* Add modify flag to `touch` command
* Add date flag to `touch` command
* Remove unnecessary `touch` test and fix tests setups
* Change `touch` flags descriptions
* Update `touch` example
* Add reference flag to `touch` command
* Add access flag to `touch` command
* Add no-create flag to `touch` command
* Replace `unwrap` with `expect`
* updated to reedline generic menus
* help menu with examples
* generic menus in the engine
* description menu template
* list of menus in config
* default value for menu
* menu from block
* generic menus examples
* change to reedline git path
* cargo fmt
* menu name typo
* remove commas from default file
* added error message
* Remove panic from BlockCommands run function
Instead of panicing, the run method now returns an error to prevent
nushell from unexpected termination.
* Add ability to open command to run with blocks
The open command tries to parse the content of the file
if there is a command called 'from (file ending)'. This works
fine if the command was 'built in' because the run method doesn't
fail in this case. It did fail on a BlockCommand, though.
This change will first probe if the command contains a block and
evaluate it, if this is the case. If there is no block, it will run
the command the same way as before.
* Add test open files with BlockCommands
* Update open.rs
* Adjust file type on open with BlockCommand parser
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
Enables the use of some features on reedline
- Keeping the line when clearing the screen with `Ctrl-L`
- Using the internal cut buffer between lines
- Submitting external commands via keybinding and keeping the line
Additional effect:
Keep the history around and do basic syncs (performance improvement
minimal as session changes have to be read and written)
Additional change:
Give the option to defer writing/rereading the history file to the
closing of the session ($config.sync_history_on_enter)
* fix(filters): pass metadata for select
* fix(filters): pass metadata for group, window
* fix(filters): pass metadata for each, every
* fix(filters): pass metadata for collect, compact, flatten
* fix(filters): pass metadata for get
* fix: get rid of necessary `.clone()``
* style: rename closure call to be consistent w/ others
* fix(filters): pass metadata for par-each, prepend
* fix(filters): pass metadata for range
* fix(filters): pass metadata for reject
* fix(filters): pass metadata for more commands
* style: run cargo fmt
* Add search terms to command
* Rename Signature desc to usage
To be named uniformly with extra_usage
* Throw in foldl search term for reduce
* Add missing usage to post
* Add search terms to signature
* Try to add capnp Signature serialization
* nu-completer with suggestions
* help menu with scrolling
* updates description rows based on space
* configuration for help menu
* update nu-ansi-term
* corrected test for update cells
* changed keybinding
* Add streaming support to save for ExternalStream data
Prior to this change, save would collect data from an ExternalStream (data
originating from externals) consuming memory for the full amount of data piped
to it,
This change adds streaming support for ExternalStream allowing saving of
arbitrarily large files and bounding memory usage.
* Remove broken save test
This test passes but not for the right reasons, since this test was
written filename has become a required parameter. The parser outputs
an error but the test still passes as is checking the original un-mutated
file assuming save has re-written the contents.
This change removes the test.
```
running 1 test
=== stderr
Error: nu::parser::missing_positional (https://docs.rs/nu-parser/0.60.0/nu-parser/enum.ParseError.html#variant.MissingPositional)
× Missing required positional argument.
╭─[source:1:1]
1 │ open save_test_1/cargo_sample.toml | save
· ▲
· ╰── missing filename
╰────
help: Usage: save {flags} <filename>
test commands::save::figures_out_intelligently_where_to_write_out_with_metadata ... ok
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 515 filtered out; finished in 0.10s
```
* Add test for passing binary data through externals
This change adds an ignored test to confirm that binary data is passed
correctly between externals to be enabled in a later commit along with
the fix.
To assist in platform agnostic testing of binary data a couple of
additional testbins were added to allow testing on `Value::Binary` inside
`ExternalStream`.
* Support binary data to stdin of run-external
Prior to this change, any pipeline producing binary data (not detected
as string) then feed into an external would be ignored due to
run-external only supporting `Value::String` on stdin.
This change adds binary stdin support for externals allowing something
like this for example:
〉^cat /dev/urandom | ^head -c 1MiB | ^pv -b | ignore
1.00MiB
This would previously output `0.00 B [0.00 B/s]` due to the data not
being pushed to stdin at each stage.
Prior to this change, a pipeline of externals would result in high memory
usage if any of the producers in the chain, produced data faster than
the consumers.
For example a pipeline:
> fast-producer | slow-consumer
Would cause a build up of `Value::{String,Binary}`'s in the mpsc channels
between each command as values are added to the channels faster than they
are consumed, eventually OOM'ing depnding on system resources, the volume
of data and speed diff. between fast v's slow.
This change replaces the unbounded channels with bounded channels
to limit the number of values that can build up and providing
back-pressure to limit ram usage.
* Add android as target os for procfs-based ps
* Turn off code for dealing with trash on platforms which are known to not support a standard trash protocol
* Update lib.rs
* Update lib.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Fix path when expanding tilde
Expanding tilde with no other relative paths would result in:
`$HOME/` instead of `$HOME`. This occurs when users run `cd` with
no extra arguments. In that case, the user's PWD would include the
trailing separator. This does not happen when explicitly passing
a value, such as `cd ~`, because in that case, the path would be
canonicalized.
This happens because std::path::PathBuf::push always adds a separator,
even if adding an empty path, which is what happens when `cd` is
invoked.
* Add test
* Fix test on Windows
Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
* Fix alias import
Alias importing was registering the alias id as a decl instead of alias.
This caused issues when hiding and then reimporting the alias.
* Clippy format
Co-authored-by: Hristo Filaretov <h.filaretov@protonmail.com>
* added missing metadata for drop and uniq #4763
* added missing metadata for keep #4763
* added missing metadata for append #4763
* added missing metadata for shuffle #4763
* add a display of what the colors look like in `ansi --list`
* change 'color' to 'preview' - add the ability to turn it off via config with use_ansi_coloring
To simplify use of nu-ansi-term in both nushell/nushell and
nushell/reedline remove it from the workspace to have a separate
progression of version numbers.
This allows reedline to use the latest published version and nushells
workspace to use the same most recent version
Changes the `Cargo.toml`s to use reedline from crates.io
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Refactor usage of is_perf_true to be a parameter passed around
* Move repl loop and command/script execution to nu_cli
* Move config setup out of nu_cli
* Update config_files.rs
* Update main.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* Refactor & fix which
Instead of fetching all definitions / aliases, only show the one that is
visible.
* Fix $nu.scope to show only visible definitions
* Add missing tests file; Rename one which test
* Do not pass non-string env vars to externals
Also misc cleanup
* Add note to default config
* Add a test
* Ensure PATH/Path conversion list <-> string
* Update docs for open and decode command, regenerate all docs
* Update open.rs
* Update open.md
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
At the moment `crossterm` apparently has a regression decoding certain important key combinations on Windows.
Thus reedline reverted to the previous version.
Some changes are necessary to remove the need for `crossterm` in the use of `lscolors`.
Introduces two local conversion traits.
Additionally update the `Highlighter` API to support the cursor
position.
This will enable brace/statement match highlighting.
Previously, the parser tried to look up the predecl also in the
permanent state and if a definition with that name already existed, it
would try to update it, which is illegal.
* Handle string->value env conv. with env. var.
Also adds the environment variable for Path/PATH and removes it from
config.
* Simplify getting the string->value conversion
* Refactor env conversion into its own function
* Use env var for to_string conversion; Remove conf
* Fix indentation in default config
This restores a basic version of the `post` command.
Some source types have been omitted from this first take.
I copied from `fetch` and from `post`@0.40.0.
Part of #4356
* Fix failing pipeline()
The `skip(1)` was there likely to remove the welcome message.
* Fix typo
* Fix nu! test macro to enter cwd correctly
Nushell's current working directory is determined primarily by the PWD
environment variable.
* Add `date format --list`, and make format string optional (providing default)
* Make DRY
`into datetime --list` now uses `generate_strfttime_list` from `date format --list`
* refactor strftime to use current datetime
* Fix formatting of specification descriptions
Fixes issues caused when copying directly from docs.rs
* Change default format to rfc2822
Perhaps to make it more DRY, functions from `into datetime` can be used. However, currently `into datetime` is a bit tricky to use as it needs a separate time zone argument.
* Tweak in-shell docs to match modified behavior
* Show %#z format specifier in `into datetime --list` only
* cargo fmt
* Satisfy clippy
* Add different features combinations
* Specify styles manually
* Fix args
* Fix typo
* Let other CI jobs finish if one fails
* Fix unused symbols without plugin feature
* Put "which" tests behind "which" feature
* Add Python virtualenv job
* Oops forgot git command
* Install Nushell in virtualenv tests
* Add names to steps; Test v.env in separate step
* cd into virtualenv
* Do not run on Python 2.7
* Build Nushell after formatting and clippy checks
* Add alias interning
Now, AliasId is used to reference aliases stored in EngineState, similar
to decls, blocks, etc.
* Fix wrong message
* Fix using decl instead of alias
* Extend also alias id visibility
* Merge also aliases from delta
* Add alias hiding code
Does not work yet but passes tests at least.
* Fix wrong alias lookup and visibility appending
* Add hide alias tests
* Fmt & Clippy
* Fix random clippy warnings in "which" command
* Fix "index out of bounds" when input to group-by is empty #4369
* Fix formatting #4369
* Adds test for empty input
Co-authored-by: Ray Henry <ray.henry@thermofisher.com>
Same rationale as in #4378
Also accelerate `grid`
before:
```
Command being timed: "./eager_nu -c for i in 0..100000 { echo whatever } | grid"
User time (seconds): 0.21
System time (seconds): 0.05
Percent of CPU this job got: 36%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.71
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 48112
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 10580
Voluntary context switches: 266
Involuntary context switches: 2595
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
after:
```
Command being timed: "./lazy_nu -c for i in 0..100000 { echo whatever } | grid"
User time (seconds): 0.14
System time (seconds): 0.05
Percent of CPU this job got: 33%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.60
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 48272
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 1
Minor (reclaiming a frame) page faults: 10582
Voluntary context switches: 286
Involuntary context switches: 831
Swaps: 0
File system inputs: 56
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
For the width calculations for table layout the `strip_ansi` function
has to be called frequently. By checking for the ASCII control chars
(0x00 to 0x1f) except `\n` that are stripped by `strip_ansi_escapes` the number of
necessary allocations can be reduced significantly for the simple case
of text not containing ANSI escapes.
**Benchmark:**
```
nu -c "for i in 0..1000 { ls } | flatten | table"
```
**Allocation reduction**
Running on the nushell repo root as the directory, this change reduces the
allocation volume by approximately 400 MB
(Measured run via KDE heaptrack)
**Speed improvement to output**
Measured via `/usr/bin/time -v`
*before*
```
Command being timed: "./eager_nu -c for i in 0..1000 {ls} | flatten | table"
User time (seconds): 0.87
System time (seconds): 0.14
Percent of CPU this job got: 87%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:01.16
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 18888
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 4809
Voluntary context switches: 38
Involuntary context switches: 14
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
*after*
```
Command being timed: "./lazy_nu -c for i in 0..1000 {ls} | flatten | table"
User time (seconds): 0.63
System time (seconds): 0.14
Percent of CPU this job got: 80%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.97
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 18660
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 5149
Voluntary context switches: 24
Involuntary context switches: 5
Swaps: 0
File system inputs: 0
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0
Signals delivered: 0
Page size (bytes): 4096
Exit status: 0
```
* Allow range in 'drop nth'
* Unit tests for drop nth range
* Add range case to the description
* Fix description 2
* format fixes
* Fix example and some refactoring
* clippy fixes
* fix into filesize tests and filesize
* tweaks
* added span back for like the 10th time
* Update filesize.rs
Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
* query command with json, web, xml
* query xml now working
* clippy
* comment out web tests
* Initial work on query web
For now we can query everything except tables
* Support for querying tables
Now we can query multiple tables just like before, now the only thing
missing is the test coverage
* Revert "Query plugin"
* augment `ps -l` on windows to display more info
Co-authored-by: Luccas Mateus de Medeiros Gomes <luccasmmg@gmail.com>
* query command with json, web, xml
* query xml now working
* clippy
* comment out web tests
* Initial work on query web
For now we can query everything except tables
* Support for querying tables
Now we can query multiple tables just like before, now the only thing
missing is the test coverage
* finish off
* comment out web test
Co-authored-by: Luccas Mateus de Medeiros Gomes <luccasmmg@gmail.com>
* Port update cells command
Clean up, nicer match statements in UpdateCellsIterator
Return columns flag into HashSet errors
Add FIXME: for update cell behavior on nested lists
* Fix: process cells for Record when no columns are specified
* Fix: address clippy lints for unwrap and into_iter
* Fix: don't step into lists and don't bind $it var
* Maybe solve the none bug?
* cargo fmt
* use nothing, not string
* check at last
* I check it at last
* Use error which has span
* use not found error
* fix error
* use a empty value length?
* * Add commit about what I change and fmt
Now all test passed, but I do not know if it is right
* update the test
* check if it is nothing
* update commit
* Rename test
Co-authored-by: Jakub Žádník <kubouch@gmail.com>
* Add rotate command
* Add rotate counter clockwise
* Fix comments in the code
* Fix clippy warnings
* Fix comment
* Fix wrong step for non even table sizes
* Fix comment for moving through array
* Refactor rotate and have only one command with a --ccw flag for counter-clockwise rotation. By default, rotate is clockwise
* Update usage description
* Add `Find` command
* Complete rustdoc for test `Value` constructors
* Use `Option::unwrap_or` instead of match
* Add `Value::test_filesize` constructor
* Handle searching for terms in `find`
* Fix `find` command signature
* Return multiple elements when `find`ing by predicate
* Do not accept rest parameter with predicate
* Handle `CellPath` in `r#in` and `not_in` for `Find`
* Use `PipelineData::filter`
* Add custom error for path not being a directory
* Fix cd issue with cd-ing into a file
* Keep formatting style as before
* Check if path is not a directory and return error if that's the case
* Add example test to zip
* Port merge command from Nushell
On top of the original merge, this one should not collect a stream
returned from the merged block and allows merging records.
