* Types lined up for open with stream
* Chunking stream
* Maybe I didn't need most of the Stream stuff after all?
* Some clean-up
* Merge weird cargo.lock
* Start moving some encoding logic to MaybeTextCodec
Will we lose the nice table formatting if we Stream? How do we get it back? Collect the Stream at the end?
* Clean-up and small refinements
* Put in auto-convert workaround
* Workaround to make sure bat functionality works
* Handle some easy error cases
* All tests pass
* Remove guessing logic
* Address clippy comments
* Pull latest master and fix MaybeTextCodec usage
* Add tag to enable autoview
Our own custom escaping unfortunately is far too simple to cover all cases.
Instead, the parser will now do no transforms on the args passed to an external
command, letting the process spawning library deal with doing the appropriate
escaping.
* .get() already checks for the argument, don't need to use .has() as well
* Supplying the month-names flag should also cause the months column to show up, it should not require the -m flag first
* Add bool subcommand to random
* Fix function name copy paste error
* Fix issue 2062: allow deserialization of a decimal
* Add bias flag to `random bool`
* Update calculate to return a table when Value is a table
* impl mode subcommand for math
* add tests for math mode subcommand
* add table/row tests for math mode subcommand
* fix formatting
* uniq: Add counting option (WIP!)
Usage:
fetch https://raw.githubusercontent.com/timbray/topfew/master/test/data/access-1k | lines | wrap item | uniq | sort-by count | last 10
* uniq: Add first test
* uniq: Re-enable the non-counting variant.
* uniq: Also handle primitive lines.
* uniq: Update documentation
* uniq: Final comment about error handling. Let's get some feedback
* uniq: Address review comments.
Not happy with the way I create a TypeError. There must be a cleaner
way. Anyway, good for shipping.
* uniq: Use Labeled_error as suggested by jturner in chat.
* uniq: Return error directly.
Co-authored-by: Christoph Siedentop <christoph@siedentop.name>
* simplify textview match code
* Math median tests and documentation additions (#2018)
* Add math median example and unit tests
* Update output of other all math ls command examples to keep consistent with math median output
* Fix output of math max example
* Update output of other math commands using pwd examples to keep data consistent
* Add math median example and unit tests
* Update output of other all math ls command examples to keep consistent with math median output
* Fix output of math max example
* Update output of other math commands using pwd examples to keep data consistent
* WIP - changes to support bat config
* added bat configuration
* removed debug info
* clippy fix
* changed [bat] to [textview]
Co-authored-by: Darren Schroeder <fdncred@hotmail.com>
* Move sum tests into math directory
* Move sum documentation over to math documentation
One sum example already existed in the math examples and a few of the others were outdated and didn't work, so I only moved one over, and updated their output
* Remove no-longer-in-use mod statement
* Add support to allow the week day start in cal to be configurable
* Fix variable name
* Use a flag instead of a configuration setting for specifying the starting day of the week
* WIP - Modified textview to use bat crate
* use input_from_bytes_with_name instead of input_file
* removed old paging
added prettyprint on else blocks
duplicated too much code
hard coded defaults
Co-authored-by: Darren Schroeder <fdncred@hotmail.com>
* Add 'every' command
* Add --skip option to 'every' command
This option skips instead of selects every nth row
* Fix descriptions for 'every' command
* Add docummentation for 'every' command
* Check actual filenames in 'every' command tests
* Adding math docs
* Add some comments to calculate
* Remove redudant message
Message already shows up in subcommands list
* Added not working example
Accidentantly
* Remove table
* Converting average.rs to math.rs
* Making some progress towards math
Examples and unit tests failing, also think I found a bug with passing in strings
* Fix typos
* Found issue with negative numbers
* Add some comments
* Split commands like in split and str_ but do not register?
* register commands in cli
* Address clippy warnings
* Fix bad examples
* Make the example failure message more helpful
* Replace unwraps
* Use compare_values to avoid coercion issues
* Remove unneeded code
* WIP - not compiling
* compiling but panicing
* still broken
* nearly working
* reverted deserializer_string changes
updated enter.rs and open.rs to use Option<Tagged<String>>
Accepted Clippy suggestions
Accepted fmt suggestions
Left original code from open.rs
We may want to use some of it and only fallback to encoding.
* Don't exit when there is an unknown encoding.
* When encoding is unknown default to utf-8.
* only do encoding if the user says to it
* merged some conflicts on open
* made error messages consistent
* Updated unwrap with expect
* updated open test to pass with more descriptive err
updated enter test to not fail
* change _location to location
* changed _visitor to visitor
* Added a more verbose usage statement for encoding
Linked to docs.rs/encoding_rs for details
Co-authored-by: Darren Schroeder <fdncred@hotmail.com>
* Add args in .nurc file to environment
* Working dummy version
* Add add_nurc to sync_env command
* Parse .nurc file
* Delete env vars after leaving directory
* Removing vals not working, strangely
* Refactoring, add comment
* Debugging
* Debug by logging to file
* Add and remove env var behavior appears correct
However, it does not use existing code that well.
* Move work to cli.rs
* Parse config directories
* I am in a state of distress
* Rename .nurc to .nu
* Some notes for me
* Refactoring
* Removing vars works, but not done in a very nice fashion
* Refactor env_vars_to_delete
* Refactor env_vars_to_add()
* Move directory environment code to separate file
* Refactor from_config
* Restore env values
* Working?
* Working?
* Update comments and change var name
* Formatting
* Remove vars after leaving dir
* Remove notes I made
* Rename config function
* Clippy
* Cleanup and handle errors
* cargo fmt
* Better error messages, remove last (?) unwrap
* FORMAT PLZ
* Rename whitelisted_directories to allowed_directories
* Add comment to clarify how overwritten values are restored.
* Do not allow invalid top-level toml
Move recursive toml conversion into a helper func
* Forgot to format
* Forgot to use helper inside collect values
Added some additional tests
* Add `rm_always_trash` option to config
* Add `--permanent` flag to `rm`
* `rm`: error if both `-t` and `-p` are present
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
* Possible implementation of globbing for start command
* Whoops forgot to remove Error used for debugging
* Use string lossy
* Run clippy
* Pin glob
* Better error messages
* Remove unneeded comment
Primarily, instead of building a parse pattern enum, we just build the regex
directly, with the appropriate capture group names so that the column name
codepaths can be shared between simple and `--regex` patterns.
Also removed capture group count compared to column name count. I don't think
this codepath can possibly be reached with the regex we now use for the
simplified capture form.
* Show error when trying to sort by invalid column
* Added test for changes
* Addressed comments, updated test
* Removed unnecessary mutable keyword
* Changed split-column to solt column after rebase from upstream
* added helper to convert data to strings
added ability to auto-rotate single row output
if row will be greater than terminal width
* Added pivot_to_fit config value
* Added ColumnPath to convert_to_string helper
* Figured out I had to run `cargo fmt --all -- --check`
Co-authored-by: Darren Schroeder <fdncred@hotmail.com>
* figuring out error with lines
* make progress in printing of block
* support for external commands; fix some tiny bugs in formatting
* basic printing of block; going to experiment with bubbling raw input to the command itself to avoid potential edge cases
* remove fmt::Display impls for hir structs; bubbled raw_input to command args
* compiling checkpoint :)
* process raw input alias to remove save flag; do duplicates stored
* fix warnings; run clippy
* removed tmux log file
* fix bug in looking for same alias; changed unwraps to safe unwraps
* Adding iniitial draft for the addition of the count column on the histogram command
* Update histogram documentation
* Add count column test to histogram command
* Fix error in histogram documentation
* Calculate the quarter directly
* Group some data together, remove attribute to ignore Clippy warning
* Group items into structs and add methods
* Updates to cal command
* Update cal.rs
* Update cal.rs
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
This changeset contains everything that a separate binary needs to
register its own commands (including the new help function). It is
very possible that this commit misses other pub use exports, but
the contained ones work for our use cases so far.
* Don't include year and month by default, adds an option to display the quarters of the year
* Add a test for cal that checks that year requested is in the return
* rustfmt the cal tests
* Add cal command
* Fix docmentation to show commands on two lines
* Use bullet points on flag documentation for cal
* Dereference day before calling string method
* Silence Clippy warning regarding a function with too many arguments
* Update cal flag descriptions and documentation
* Add some tests for the cal command
* mvp for start command
* modified the signature of the start command
* parse filenames
* working model for macos is done
* refactored to read from pipes
* start command works well on macos; manual testing reveals need of --args flag support
* implemented start error; color printing of warning and errors
* ran clippy and fixed warnings
* fix a clippy lint that was caught in pipeline
* fix dead code clippy lint for windows
* add cfg annotation to import
Both to and from without a subcommand only print the helptext. Expand the usage line a bit, so a glance at `help commands` indicates the existance of the subcommands and mentions some common formats.
Ref a9968046ed
Ref #1708
* Changes to allow plugins to be loaded in a multi-threaded manner in order to decrease startup time.
* Ran rust fmt and clippy to find and fix first pass errors.
Updated launch.jason to make debugging easier in vscode.
Also added tasks.json so tasks like clippy can be ran easily.
* ran fmt again
* Delete launch.json
Remove IDE settings file
* Remove IDE settings file
* Ignore vscode IDE settings
* Cloned the context instead of Arc/Mutexing it.
Co-authored-by: Darren Schroeder <fdncred@hotmail.com>
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
* Fix: the symlink target column will only dispaly if either the `full` or `with_symlink_targets` options are given
* If the metadata for every item in the size column is None, do not show the size column
* Do not show the symlink target column if the metadata is None for all the items in the table
* fix: absolutize path against its parent if it was a symlink.
On Linux this happens because Rust calls readlink but doesn't canonicalize the resultant path.
* feat: playground function to create symlinks
* fix: use playground dirs
* feat: test for #1631, shift tests names
* Creation of FilesystemShell with custom location may fail
* Replace ichwh with which
* Creation of FilesystemShell with custom location may fail
* Replace ichwh with which
* fix: add ichwh again since it cannot be completely replaced
* fix: replace one more use of which
For example, when running the following:
crates/nu-cli/src
nushell currently parses this as an external command. Before running the command, we check to see if
it's a directory. If it is, we "auto cd" into that directory, otherwise we go through normal
external processing.
If we put a trailing slash on it though, shells typically interpret that as "user is explicitly
referencing directory". So
crates/nu-cli/src/
should not be interpreted as "run an external command". We intercept a trailing slash in the head
position of a command in a pipeline as such, and inject a `cd` internal command.
* make trim apply to strings contained in tables (also at deeper nesting
levels), not just top-level strings
* remove unnecessary clone (thanks clippy)
* from-eml initial ver
* Adding tests for `from-eml`
* Add eml to prepares_and_decorates_filesystem_source_files
* Sort the file order
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
* fix: absolutize path against its parent if it was a symlink.
On Linux this happens because Rust calls readlink but doesn't canonicalize the resultant path.
* feat: playground function to create symlinks
* fix: use playground dirs
* feat: test for #1631, shift tests names
* refactor: expand_path and expand_ndots now work for any string.
* refactor: refactor test and add new ones.
* refactor: convert expanded to owned string
* feat: pub export of expand_ndots
* feat: add completion for ndots in fs-shell
* Fix --headerless option of to-csv and to-tsv
Before to-csv --headerless split the "headerfull" output on lines,
skipped the first, and then concatenated them together. That meant
that all remaining output would be put on the same line, without
any delimiter, making it unusable. Now we replace the range of the
first line with the empty string, maintaining the rest of the
output unchanged.
* Remove extra space in indentation of to_delimited_data
* Add --separator <string> argument to to-csv
This functionaliy has been present before, but wasn't exposed
to the command.
* Refactor InputStream and affected commands.
First, making `values` private and leaning on the `Stream` implementation makes
consumes of `InputStream` less likely to have to change in the future, if we
change what an `InputStream` is internally.
Second, we're dropping `Option<InputStream>` as the input to pipelines,
internals, and externals. Instead, `InputStream.is_empty` can be used to check
for "emptiness". Empty streams are typically only ever used as the first input
to a pipeline.
* Add run_external internal command.
We want to push external commands closer to internal commands, eventually
eliminating the concept of "external" completely. This means we can consolidate
a couple of things:
- Variable evaluation (for example, `$it`, `$nu`, alias vars)
- Behaviour of whole stream vs per-item external execution
It should also make it easier for us to start introducing argument signatures
for external commands,
* Update run_external.rs
* Update run_external.rs
* Update run_external.rs
* Update run_external.rs
Co-authored-by: Jonathan Turner <jonathandturner@users.noreply.github.com>
* Better message error
* Use custom canonicalize in FileStructure build
* Better glob error in ls
* Use custom canonicalize, remove some duplicate code in cd.
* Enable recursive copying with patterns.
* Change test to fit new error message
* Test recursive with glob pattern
* Show that not matches were found in cp
* Fix typo in message error
* Change old canonicalize usage, follow newest changes
* Expand n dots early where tilde was also expanded.
* Remove normalize, not needed.
New function absolutize, doesn't follow links neither checks existence.
Renamed canonicalize_existing to canonicalize, works as expected.
* Remove normalize usages, change canonicalize.
* Treat strings as paths
* Making Commands match what UntaggedValue needs
* WIP
* WIP
* WIP
* Moved to expressions for conditions
* Add 'each' command to use command blocks
* More cleanup
* Add test for 'each'
* Instead use an expression block
* New 'path' module under nu-cli.
Added normalize and canonicalize method.
Added some unit tests.
* Replace old usages of normalize and canonicalize.
* Fix reading symlinks and existence logic.
* Better explained
1. Fixed a bug where `--all` wasn't showing files at the root directories.
2. More use of `Result`'s `map` and `map_err` methods.
3. Making tables be homogeneous so one can, for example, `get directories`.
Previously, if the user didn't have the appropriate permissions to execute the
binary/script, they would see "command not found", which is confusing.
This commit eliminates the `which` crate in favour of `ichwh`, which deals
better with permissions by not dealing with them at all! This is closer to the
behaviour of `which` in many shells. Permission checks are then left up to the
caller to deal with.
Mostly, making more use of `map` and `map_err` in `Result`. One benefit
is that at least one location had duplicated logic for how to map the
error, which is no longer the case after this commit.
An interruptible stream can query an `AtomicBool. If that bool is true,
the stream will no longer produce any values.
Also introducing the `Interruptible` trait, which extends any `Stream`
with the `interruptible` function, to simplify the construction and
allow chaining.
* headers plugin
* Remove plugin
* Add non-functioning headers command
* Add ability to extract headers from first row
* Refactor header extraction
* Rebuild indexmap with proper headers
* Rebuild result properly
* Compiling, probably wrapped too much?
* Refactoring
* Deal with case of empty header cell
* Deal with case of empty header cell
* Fix formatting
* Fix linting, attempt 2.
* Move whole_stream_command(Headers) to more appropriate section
* ... more linting
* Return Err(ShellError...) instead of panic, yield each row instead of entire table
* Insert Column[index] if no header info is found.
* Update error description
* Add initial test
* Add tests for headers command
* Lint test cases in headers
* Change ShellError for headers, Add sample_headers file to utils.rs
* Add empty sheet to test file
* Revert "Add empty sheet to test file"
This reverts commit a4bf38a31d.
* Show error message when given empty table
This makes the `binaries` function respect the `CARGO_TARGET_DIR` environment variable when set. If it's not present it falls back to the regular target directory used by Cargo.
* Fix '/' and '..' not being valid mv targets
If `/` or `../` is specified as the destination for `mv`, it will fail with an error message saying it's not a valid destination. This fixes it to account for the fact that `Path::file_name` return `None` when the file name evaluates to `/` or `..`. It will only take the slow(er) path if `Path::file_name` returns `None` in its initial check.
Fixes#1291
* Add test
* Add error message for csv parsing failures
* Add csv error prettyfier
* Improve readability of the error
Line 2: error is easier to understand than:
Line 2, error
* Remove unnecessary use of the format! macro
Replacing it with .to_string() fixes a clippy warning
* Improve consistency with JSON parsing errors
* Custom canonicalize method for FilesystemShell.
* Use custom canonicalize method.
Fixed missing import.
* Move function body to already impl body.
* Create test that aims to resolve.
* Utility function to detect hidden folders.
Implemented for Unix and Windows.
* Rename function argument.
* Revert "Rename function argument."
This reverts commit e7ab70f0f0.
* Add flag '--all/-a' to Ls
* Rename function argument.
* Check if flag '--all/-a' is present and path is hidden.
Replace match with map_err for glob result.
Remove redundancy in stream body.
Included comments on new stream body.
Replace async_stream::stream with async_stream::try_stream.
Minor tweaks to is_empty_dir.
Fix and refactor is_hidden_dir.
* Fix "implicit" bool coerse
* Fixed clippy errors
For some commands like `which` -h flag would trigger an error asking for
missing required parameters instead of printing the help message as it
does with --help. This commit adds a check in the command parser to
avoid that.
* Fix deleting named pipes
* Use std::os::unix::fs::FileTypeExt to show correct type for unix-specific fs objects; Fix formatting
Co-authored-by: Linards Kalvāns <linards.kalvans@twino.eu>
The extra newline character makes it hard to use nu as part of an
external processing pipeline, since the extra character could taint the
results. For example:
```
$ nu -c 'echo test | xxd'
00000000: 7465 7374 test
```
versus
```
nu -c 'echo test' | xxd
00000000: 7465 7374 0a test.
```
* WIP: move to bytes codec
* Progress on adding collect helpers
* Progress on adding collect helpers
* Add in line splitting back to lines
* Lines outputting line primitives
* Close to ready?
* Finish fixing lines
* clippy fixes
* fmt fixes
* removed unused code
* Cleanup a few bits
* Cleanup a few bits
* Cleanup a few more bits
* Fix failing test with corrected test case
* Fix and refactor cd for Filesystem Shell.
Reorder check conditions, don't check existence twice.
If building for unix check exec bit on folder.
* Import PermissionsExt only on unix target.
* It seems that this is the correct way?
This improves incremental build time when working on what was previously
the root package. For example, previously all plugins would be rebuilt
with a change to `src/commands/classified/external.rs`, but now only
`nu-cli` will have to be rebuilt (and anything that depends on it).
In particular, one thing that we can't (properly) do before this commit
is consuming an infinite input stream. For example:
```
yes | grep y | head -n10
```
will give 10 "y"s in most shells, but blocks indefinitely in nu. This PR
resolves that by doing blocking I/O in threads, and reducing the `await`
calls we currently have in our pipeline code.
* Add very basic documentation. Need to play with rest of the api to figure out what it does
* Add some documentation to more of the Plugin API methods
* fmt
* typo fixes
* Change signature to take in short-hand flags
* update help information
* Parse short-hand flags as their long counterparts
* lints
* Modified a couple tests to use shorthand flags
* Add some docs for meta.rs
* add better explanation for Span merging
* Add some doc tests - not sure how to get them to run
* get rid of doc comments for the temporary method
* add doc test for is_unknown
* fmt
* Refactor pipeline ahead of block changes. Add '-c' commandline option
* Update pipelining an error value
* Fmt
* Clippy
* Add stdin redirect for -c flag
* Add stdin redirect for -c flag
* Upgrade futures, async-stream, and futures_codec
These were the last three dependencies on futures-preview. `nu` itself
is now fully dependent on `futures@0.3`, as opposed to `futures-preview`
alpha.
Because the update to `futures` from `0.3.0-alpha.19` to `0.3.0` removed
the `Stream` implementation of `VecDeque` ([changelog][changelog]), most
commands that convert a `VecDeque` to an `OutputStream` broke and had to
be fixed.
The current solution is to now convert `VecDeque`s to a `Stream` via
`futures::stream::iter`. However, it may be useful for `futures` to
create an `IntoStream` trait, implemented on the `std::collections` (or
really any `IntoIterator`). If something like this happends, it may be
worthwhile to update the trait implementations on `OutputStream` and
refactor these commands again.
While upgrading `futures_codec`, we remove a custom implementation of
`LinesCodec`, as one has been added to the library. There's also a small
refactor to make the stream output more idiomatic.
[changelog]: https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md#030---2019-11-5
* Upgrade sys & ps plugin dependencies
They were previously dependent on `futures-preview`, and `nu_plugin_ps`
was dependent on an old version of `futures-timer`.
* Remove dependency on futures-timer from nu
* Update Cargo.lock
* Fix formatting
* Revert fmt regressions
CI is still on 1.40.0, but the latest rustfmt v1.41.0 has changes to the
`val @ pattern` syntax, causing the linting job to fail.
* Fix clippy warnings
* add some notes into README for more elaboration
* rewrite the overview
* remove unused first line
* add last part about tracing and debugging
* change the wording to make it easier to read
* Add example of metadata system
* Add contact information as other helpful links
