mirror of
https://github.com/nushell/nushell.git
synced 2025-01-25 07:40:26 +01:00
7e055810b1
17 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Devyn Cairns
|
8e2917b9ae
|
Make assignment and const consistent with let/mut (#13385)
# Description This makes assignment operations and `const` behave the same way `let` and `mut` do, absorbing the rest of the pipeline. Changes the lexer to be able to recognize assignment operators as a separate token, and then makes the lite parser continue to push spans into the same command regardless of any redirections or pipes if an assignment operator is encountered. Because the pipeline is no longer split up by the lite parser at this point, it's trivial to just parse the right hand side as if it were a subexpression not contained within parentheses. # User-Facing Changes Big breaking change. These are all now possible: ```nushell const path = 'a' | path join 'b' mut x = 2 $x = random int $x = [1 2 3] | math sum $env.FOO = random chars ``` In the past, these would have led to (an attempt at) bare word string parsing. So while `$env.FOO = bar` would have previously set the environment variable `FOO` to the string `"bar"`, it now tries to run the command named `bar`, hence the major breaking change. However, this is desirable because it is very consistent - if you see the `=`, you can just assume it absorbs everything else to the right of it. # Tests + Formatting Added tests for the new behaviour. Adjusted some existing tests that depended on the right hand side of assignments being parsed as barewords. # After Submitting - [ ] release notes (breaking change!) |
||
|
Devyn Cairns
|
aa7d7d0cc3
|
Overhaul $in expressions (#13357)
# Description
This grew quite a bit beyond its original scope, but I've tried to make
`$in` a bit more consistent and easier to work with.
Instead of the parser generating calls to `collect` and creating
closures, this adds `Expr::Collect` which just evaluates in the same
scope and doesn't require any closure.
When `$in` is detected in an expression, it is replaced with a new
variable (also called `$in`) and wrapped in `Expr::Collect`. During
eval, this expression is evaluated directly, with the input and with
that new variable set to the collected value.
Other than being faster and less prone to gotchas, it also makes it
possible to typecheck the output of an expression containing `$in`,
which is nice. This is a breaking change though, because of the lack of
the closure and because now typechecking will actually happen. Also, I
haven't attempted to typecheck the input yet.
The IR generated now just looks like this:
```gas
collect %in
clone %tmp, %in
store-variable $in, %tmp
# %out <- ...expression... <- %in
drop-variable $in
```
(where `$in` is the local variable created for this collection, and not
`IN_VARIABLE_ID`)
which is a lot better than having to create a closure and call `collect
--keep-env`, dealing with all of the capture gathering and allocation
that entails. Ideally we can also detect whether that input is actually
needed, so maybe we don't have to clone, but I haven't tried to do that
yet. Theoretically now that the variable is a unique one every time, it
should be possible to give it a type - I just don't know how to
determine that yet.
On top of that, I've also reworked how `$in` works in pipeline-initial
position. Previously, it was a little bit inconsistent. For example,
this worked:
```nushell
> 3 | do { let x = $in; let y = $in; print $x $y }
3
3
```
However, this causes a runtime variable not found error on the second
`$in`:
```nushell
> def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
Error: nu:🐚:variable_not_found
× Variable not found
╭─[entry #115:1:35]
1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo
· ─┬─
· ╰── variable not found
╰────
```
I've fixed this by making the first element `$in` detection *always*
happen at the block level, so if you use `$in` in pipeline-initial
position anywhere in a block, it will collect with an implicit
subexpression around the whole thing, and you can then use that `$in`
more than once. In doing this I also rewrote `parse_pipeline()` and
hopefully it's a bit more straightforward and possibly more efficient
too now.
Finally, I've tried to make `let` and `mut` a lot more straightforward
with how they handle the rest of the pipeline, and using a redirection
with `let`/`mut` now does what you'd expect if you assume that they
consume the whole pipeline - the redirection is just processed as
normal. These both work now:
```nushell
let x = ^foo err> err.txt
let y = ^foo out+err>| str length
```
It was previously possible to accomplish this with a subexpression, but
it just seemed like a weird gotcha that you couldn't do it. Intuitively,
`let` and `mut` just seem to take the whole line.
- closes #13137
# User-Facing Changes
- `$in` will behave more consistently with blocks and closures, since
the entire block is now just wrapped to handle it if it appears in the
first pipeline element
- `$in` no longer creates a closure, so what can be done within an
expression containing `$in` is less restrictive
- `$in` containing expressions are now type checked, rather than just
resulting in `any`. However, `$in` itself is still `any`, so this isn't
quite perfect yet
- Redirections are now allowed in `let` and `mut` and behave pretty much
how you'd expect
# Tests + Formatting
Added tests to cover the new behaviour.
# After Submitting
- [ ] release notes (definitely breaking change)
|
||
Ian Manske
|
6012af2412
|
Fix panic when redirecting nothing (#12970)
# Description Fixes #12969 where the parser can panic if a redirection is applied to nothing / an empty command. # Tests + Formatting Added a test. |
||
|
Ian Manske
|
c747ec75c9
|
Add command_prelude module (#12291)
# Description
When implementing a `Command`, one must also import all the types
present in the function signatures for `Command`. This makes it so that
we often import the same set of types in each command implementation
file. E.g., something like this:
```rust
use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, PipelineData,
ShellError, Signature, Span, Type, Value,
};
```
This PR adds the `nu_engine::command_prelude` module which contains the
necessary and commonly used types to implement a `Command`:
```rust
// command_prelude.rs
pub use crate::CallExt;
pub use nu_protocol::{
ast::{Call, CellPath},
engine::{Command, EngineState, Stack},
record, Category, Example, IntoInterruptiblePipelineData, IntoPipelineData, IntoSpanned,
PipelineData, Record, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
};
```
This should reduce the boilerplate needed to implement a command and
also gives us a place to track the breadth of the `Command` API. I tried
to be conservative with what went into the prelude modules, since it
might be hard/annoying to remove items from the prelude in the future.
Let me know if something should be included or excluded.
|
||
|
Ian Manske
|
b6c7656194
|
IO and redirection overhaul (#11934)
# Description The PR overhauls how IO redirection is handled, allowing more explicit and fine-grain control over `stdout` and `stderr` output as well as more efficient IO and piping. To summarize the changes in this PR: - Added a new `IoStream` type to indicate the intended destination for a pipeline element's `stdout` and `stderr`. - The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to avoid adding 6 additional arguments to every eval function and `Command::run`. The `stdout` and `stderr` streams can be temporarily overwritten through functions on `Stack` and these functions will return a guard that restores the original `stdout` and `stderr` when dropped. - In the AST, redirections are now directly part of a `PipelineElement` as a `Option<Redirection>` field instead of having multiple different `PipelineElement` enum variants for each kind of redirection. This required changes to the parser, mainly in `lite_parser.rs`. - `Command`s can also set a `IoStream` override/redirection which will apply to the previous command in the pipeline. This is used, for example, in `ignore` to allow the previous external command to have its stdout redirected to `Stdio::null()` at spawn time. In contrast, the current implementation has to create an os pipe and manually consume the output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`, etc.) have precedence over overrides from commands. This PR improves piping and IO speed, partially addressing #10763. Using the `throughput` command from that issue, this PR gives the following speedup on my setup for the commands below: | Command | Before (MB/s) | After (MB/s) | Bash (MB/s) | | --------------------------- | -------------:| ------------:| -----------:| | `throughput o> /dev/null` | 1169 | 52938 | 54305 | | `throughput \| ignore` | 840 | 55438 | N/A | | `throughput \| null` | Error | 53617 | N/A | | `throughput \| rg 'x'` | 1165 | 3049 | 3736 | | `(throughput) \| rg 'x'` | 810 | 3085 | 3815 | (Numbers above are the median samples for throughput) This PR also paves the way to refactor our `ExternalStream` handling in the various commands. For example, this PR already fixes the following code: ```nushell ^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world" ``` This returns an empty list on 0.90.1 and returns a highlighted "hello world" on this PR. Since the `stdout` and `stderr` `IoStream`s are available to commands when they are run, then this unlocks the potential for more convenient behavior. E.g., the `find` command can disable its ansi highlighting if it detects that the output `IoStream` is not the terminal. Knowing the output streams will also allow background job output to be redirected more easily and efficiently. # User-Facing Changes - External commands returned from closures will be collected (in most cases): ```nushell 1..2 | each {|_| nu -c "print a" } ``` This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n" and then return an empty list. ```nushell 1..2 | each {|_| nu -c "print -e a" } ``` This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used to return an empty list and print "a\na\n" to stderr. - Trailing new lines are always trimmed for external commands when piping into internal commands or collecting it as a value. (Failure to decode the output as utf-8 will keep the trailing newline for the last binary value.) In the current nushell version, the following three code snippets differ only in parenthesis placement, but they all also have different outputs: 1. `1..2 | each { ^echo a }` ``` a a ╭────────────╮ │ empty list │ ╰────────────╯ ``` 2. `1..2 | each { (^echo a) }` ``` ╭───┬───╮ │ 0 │ a │ │ 1 │ a │ ╰───┴───╯ ``` 3. `1..2 | (each { ^echo a })` ``` ╭───┬───╮ │ 0 │ a │ │ │ │ │ 1 │ a │ │ │ │ ╰───┴───╯ ``` But in this PR, the above snippets will all have the same output: ``` ╭───┬───╮ │ 0 │ a │ │ 1 │ a │ ╰───┴───╯ ``` - All existing flags on `run-external` are now deprecated. - File redirections now apply to all commands inside a code block: ```nushell (nu -c "print -e a"; nu -c "print -e b") e> test.out ``` This gives "a\nb\n" in `test.out` and prints nothing. The same result would happen when printing to stdout and using a `o>` file redirection. - External command output will (almost) never be ignored, and ignoring output must be explicit now: ```nushell (^echo a; ^echo b) ``` This prints "a\nb\n", whereas this used to print only "b\n". This only applies to external commands; values and internal commands not in return position will not print anything (e.g., `(echo a; echo b)` still only prints "b"). - `complete` now always captures stderr (`do` is not necessary). # After Submitting The language guide and other documentation will need to be updated. |
||
Wind
|
58c6fea60b
|
Support redirect stderr and stdout+stderr with a pipe (#11708)
# Description Close: #9673 Close: #8277 Close: #10944 This pr introduces the following syntax: 1. `e>|`, pipe stderr to next command. Example: `$env.FOO=bar nu --testbin echo_env_stderr FOO e>| str length` 2. `o+e>|` and `e+o>|`, pipe both stdout and stderr to next command, example: `$env.FOO=bar nu --testbin echo_env_mixed out-err FOO FOO e+o>| str length` Note: it only works for external commands. ~There is no different for internal commands, that is, the following three commands do the same things:~ Edit: it raises errors if we want to pipes for internal commands ``` ❯ ls e>| str length Error: × `e>|` only works with external streams ╭─[entry #1:1:1] 1 │ ls e>| str length · ─┬─ · ╰── `e>|` only works on external streams ╰──── ❯ ls e+o>| str length Error: × `o+e>|` only works with external streams ╭─[entry #2:1:1] 1 │ ls e+o>| str length · ──┬── · ╰── `o+e>|` only works on external streams ╰──── ``` This can help us to avoid some strange issues like the following: `$env.FOO=bar (nu --testbin echo_env_stderr FOO) e>| str length` Which is hard to understand and hard to explain to users. # User-Facing Changes Nan # Tests + Formatting To be done # After Submitting Maybe update documentation about these syntax. |
||
|
WindSoilder
|
077d1c8125
|
Support o>>, e>>, o+e>> to append output to an external file (#10764)
# Description Close: #10278 This pr introduces `o>>`, `e>>`, `o+e>>` to allow redirection to append to a file. Examples: ```nushell echo abc o>> a.txt echo abc o>> a.txt cat asdf e>> a.txt cat asdf e>> a.txt cat asdf o+e>> a.txt ``` ~~TODO:~~ ~~1. currently internal commands with `o+e>` redirect to a variable is broken: `let x = "a.txt"; echo abc o+e> $x`, not sure when it was introduced...~~ ~~2. redirect stdout and stderr with append mode doesn't supported yet: `cat asdf o>>a.txt e>>b.ext`~~ ~~For these 2 items, I'd like to fix them in different prs.~~ Already done in this pr |
||
|
WindSoilder
|
57808ca7cc
|
Redirect: support redirect stderr with piping stdout to next commands. (#10851)
# Description Fixes: #10271 Given the following script: ```shell # test.sh echo aaaaa echo bbbbb 1>&2 echo cc ``` This pr makes the following command possible: ```nushell bash test.sh err> /dev/null | lines | each {|line| $line | str length} ``` ## General idea behind the change: When nushell redirect stderr message to external file 1. it take stdout of external stream, and pass this stream to next command, so it won't block next pipeline command from running. 2. relative stderr stream are handled by `save` command These two streams are handled separately, so we need to delegate a thread to `save` command, or else we'll have a chance to hang nushell, we have meet a similar before: #5625. ### One case to consider What if we're failed to save to an external stream? (Like we don't have a permission to save to a file)? In this case nushell will just print a waning message, and don't stop the following scripts from running. # User-Facing Changes ## Before ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} aaaaa cc ``` ## After ```nushell ❯ bash test2.sh err> /dev/null | lines | each {|line| $line | str length} ╭───┬───╮ │ 0 │ 5 │ │ 1 │ 2 │ ╰───┴───╯ ``` BTY, after this pr, the following commands are impossible either, it's important to make sure that the implementation doesn't introduce too much costs: ```nushell ❯ echo a e> a.txt e> a.txt Error: × Can't make stderr redirection twice ╭─[entry #1:1:1] 1 │ echo a e> a.txt e> a.txt · ─┬ · ╰── try to remove one ╰──── ❯ echo a o> a.txt o> a.txt Error: × Can't make stdout redirection twice ╭─[entry #2:1:1] 1 │ echo a o> a.txt o> a.txt · ─┬ · ╰── try to remove one ╰──── ``` |
||
|
WindSoilder
|
f043a8a8ff
|
redirect should have a target (#10835)
# Description Fixes: #10830 The issue happened during lite-parsing, when we want to put a `LiteElement` to a `LitePipeline`, we do nothing if relative redirection target is empty. So the command `echo aaa o> | ignore` will be interpreted to `echo aaa | ignore`. This pr is going to check and return an error if redirection target is empty. # User-Facing Changes ## Before ``` ❯ echo aaa o> | ignore # nothing happened ``` ## After ```nushell ❯ echo aaa o> | ignore Error: nu::parser::parse_mismatch × Parse mismatch during operation. ╭─[entry #1:1:1] 1 │ echo aaa o> | ignore · ─┬ · ╰── expected redirection target ╰──── ``` |
||
|
WindSoilder
|
d204defb68
|
Refactor: remove duplication to simplify lite_parsing logic. (#10735)
When looking into `lite_parse` function, I found that it contains some duplicate code, and they can be expressed as an action called `push_command_to(pipeline)`. And I believe it will make our life easier to support something like `o>> a.txt`, `e>> a.txt`. |
||
Horasal
|
b943cbedff
|
skip comments and eols while parsing pipeline (#10149)
This pr - fixes https://github.com/nushell/nushell/issues/10143 - fixes https://github.com/nushell/nushell/issues/5559 # Description Current `lite_parse` does not handle multiple line comments and eols in pipeline. When parsing the following tokens: | `"abcdefg"` | ` \|` | `# foobar` | ` \n` | `split chars` | | ------------- | ------------- |------------- |------------- |------------- | | [Command] | [Pipe] | [Comment] | [Eol] | [Command] | | | | Last Token |Current Token | | `TokenContent::Eol` handler only checks if `last_token` is `Pipe` but it will be broken if there exist any other thing, e.g. extra `[Comment]` in this example. This pr make the following change: - While parsing `[Eol]`, try to find the last non-comment token as `last_token` - Comment is supposed as `[Comment]+` or `([Comment] [Eol])+` - `[Eol]+` is still parsed just like current nu (i.e. generates `nothing`). Notice that this pr is just a quick patch if more comment/eol related issue occures, `lite_parser` may need a rewrite. # User-Facing Changes Now the following pipeline works: ```bash 1 | # comment each { |it| $it + 2 } | # comment math sum ``` Comment will not end the pipeline in interactive mode: ```bash ❯ 1 | # comment (now enter multiple line mode instead of end) ▶▶ # foo ▶▶ 2 ``` # Tests + Formatting - 🟢 `toolkit fmt` - 🟢 `toolkit clippy` - 🟢 `toolkit test` - 🟢 `toolkit test stdlib` # After Submitting None --------- Co-authored-by: Horasal <horsal@horsal.dev> |
||
JT
|
5d9e2455f7
|
Let with pipeline (#9589)
# Description This changes the default behaviour of `let` to be able to take a pipeline as its initial value. For example: ``` > let x = "hello world" | str length ``` This is a change from the existing behaviour, where the right hand side is assumed to be an expression. Pipelines are more general, and can be more powerful. My google foo is failing me, but this also fixes this issue: ``` let x = foo ``` Currently, this reads `foo` as a bareword that gets converted to a string rather than running the `foo` command. In practice, this is really annoying and is a really hard to spot bug in a script. # User-Facing Changes BREAKING CHANGE BREAKING CHANGE `let` gains the power to be assigned via a pipeline. However, this changes the behaviour of `let x = foo` from assigning the string "foo" to `$x` to being "run the command `foo` and give the result to `$x`" # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> |
||
|
WindSoilder
|
b150f9f5d8
|
Avoid blocking when o+e> redirects too much stderr message (#8784)
# Description Fixes: #8565 Here is another pr #7240 tried to address the issue, but it works in a wrong way. After this change `o+e>` won't redirect all stdout message then stderr message and it works more like how bash does. # User-Facing Changes For the given python code: ```python # test.py import sys print('aa'*300, flush=True) print('bb'*999999, file=sys.stderr, flush=True) print('cc'*300, flush=True) ``` Running `python test.py out+err> a.txt` shoudn't hang nushell, and `a.txt` keeps output in the same order ## About the change The core idea is that when doing lite-parsing, introduce a new variant `LiteElement::SameTargetRedirection` if we meet `out+err>` redirection token(which is generated by lex function), During converting from lite block to block, LiteElement::SameTargetRedirection will be converted to PipelineElement::SameTargetRedirection. Then in the block eval process, if we get PipelineElement::SameTargetRedirection, we'll invoke `run-external` with `--redirect-combine` flag, then pipe the result into save command ## What happened internally? Take the following command as example: `^ls o+e> log.txt` lex parsing result(`Tokens`) are not changed, but `LiteBlock` and `Block` is changed after this pr. ### LiteBlock before ```rust LiteBlock { block: [ LitePipeline { commands: [ Command(None, LiteCommand { comments: [], parts: [Span { start: 39041, end: 39044 }] }), // actually the span of first Redirection is wrong too.. Redirection(Span { start: 39058, end: 39062 }, StdoutAndStderr, LiteCommand { comments: [], parts: [Span { start: 39050, end: 39057 }] }), ] }] } ``` ### LiteBlock after ```rust LiteBlock { block: [ LitePipeline { commands: [ SameTargetRedirection { cmd: (None, LiteCommand { comments: [], parts: [Span { start: 147945, end: 147948}]}), redirection: (Span { start: 147949, end: 147957 }, LiteCommand { comments: [], parts: [Span { start: 147958, end: 147965 }]}) } ] } ] } ``` ### Block before ```rust Pipeline { elements: [ Expression(None, Expression { expr: ExternalCall(Expression { expr: String("ls"), span: Span { start: 39042, end: 39044 }, ty: String, custom_completion: None }, [], false), span: Span { start: 39041, end: 39044 }, ty: Any, custom_completion: None }), Redirection(Span { start: 39058, end: 39062 }, StdoutAndStderr, Expression { expr: String("out.txt"), span: Span { start: 39050, end: 39057 }, ty: String, custom_completion: None })] } ``` ### Block after ```rust Pipeline { elements: [ SameTargetRedirection { cmd: (None, Expression { expr: ExternalCall(Expression { expr: String("ls"), span: Span { start: 147946, end: 147948 }, ty: String, custom_completion: None}, [], false), span: Span { start: 147945, end: 147948}, ty: Any, custom_completion: None }), redirection: (Span { start: 147949, end: 147957}, Expression {expr: String("log.txt"), span: Span { start: 147958, end: 147965 },ty: String,custom_completion: None} } ] } ``` # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. |
||
|
JT
|
aded2c1937
|
Refactor to support multiple parse errors (#8765)
# Description This is a pretty heavy refactor of the parser to support multiple parser errors. It has a few issues we should address before landing: - [x] In some cases, error quality has gotten worse `1 / "bob"` for example - [x] if/else isn't currently parsing correctly - probably others # User-Facing Changes This may have error quality degradation as we adjust to the new error reporting mechanism. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. |
||
|
WindSoilder
|
4f57c5d56e
|
Fix multi-line redirection inside a block (#7808)
# Description Fixes: #7786 The issue is because the lite block is wrong while converting from lex tokens # What happened internally? Take the following as example: ``` ❯ def foobar [] { 'hello' out> /tmp/output.1 'world' out> /tmp/output.2 } ``` ## Before: ``` LiteBlock { block: [ LitePipeline { commands: [ Command(None, LiteCommand { comments: [], parts: [Span { start: 40900, end:40907 }] }), Redirection(Span { start: 40908, end: 40912 }, Stdout, LiteCommand { comments: [], parts: [Span { start: 40913, end: 40926 }] })] }, LitePipeline { commands: [ Redirection(Span { start: 40908, end: 40912 }, Stdout, LiteCommand { comments: [], parts: [Span { start: 40929, end: 40936 }] }), // this is wrong, should be command. Redirection(Span { start: 40937, end: 40941 }, Stdout, LiteCommand { comments: [], parts: [Span { start: 40942, end: 40955 }] })] }] } ``` ## After: ``` LiteBlock { block: [ LitePipeline { commands: [ Command(None, LiteCommand { comments: [], parts: [Span { start: 40824, end: 40831 }] }), Redirection(Span { start: 40832, end: 40836 }, Stdout, LiteCommand { comments: [], parts: [Span { start: 40837, end: 40850 }] })] }, LitePipeline { commands: [ Command(None, LiteCommand { comments: [], parts: [Span { start: 40854, end: 40861 }] }), Redirection(Span { start: 40862, end: 40866 }, Stdout, LiteCommand { comments: [], parts: [Span { start: 40867, end: 40880 }] })] } ] } ``` # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. |
||
|
WindSoilder
|
8aa2632661
|
Support redirect err and out to different streams (#7685)
# Description Closes: #7364 # User-Facing Changes Given the following shell script: ```bash x=$(printf '=%.0s' {1..100}) echo $x echo $x 1>&2 ``` It supports the following command: ``` bash test.sh out> out.txt err> err.txt ``` Then both `out.txt` and `err.txt` will contain `=`(100 times) ## About the change The core idea is that when doing lite-parsing, introduce a new variant `LiteElement::SeparateRedirection` if we meet two Redirection token(which is generated by `lex` function), During converting from lite block to block, `LiteElement::SeparateRedirection` will be converted to `PipelineElement::SeparateRedirection`. Then in the block eval process, if we get `PipelineElement::SeparateRedirection`, we invoke `save` command with `--stderr` arguments to acthive our behavior. ## What happened internally? Take the following command as example: ``` ^ls out> out.txt err> err.txt ``` lex parsing result(`Tokens`) are not changed, but `LiteBlock` and `Block` is changed after this pr. ### LiteBlock before ```rust LiteBlock { block: [ LitePipeline { commands: [ Command(None, LiteCommand { comments: [], parts: [Span { start: 39041, end: 39044 }] }), // actually the span of first Redirection is wrong too.. Redirection(Span { start: 39058, end: 39062 }, Stdout, LiteCommand { comments: [], parts: [Span { start: 39050, end: 39057 }] }), Redirection(Span { start: 39058, end: 39062 }, Stderr, LiteCommand { comments: [], parts: [Span { start: 39063, end: 39070 }] }) ] }] } ``` ### LiteBlock after ```rust LiteBlock { block: [ LitePipeline { commands: [ Command( None, LiteCommand { comments: [], parts: [Span { start: 38525, end: 38528 }] }), // new one! two Redirection merged into one SeparateRedirection. SeparateRedirection { out: (Span { start: 38529, end: 38533 }, LiteCommand { comments: [], parts: [Span { start: 38534, end: 38541 }] }), err: (Span { start: 38542, end: 38546 }, LiteCommand { comments: [], parts: [Span { start: 38547, end: 38554 }] }) } ] }] } ``` ### Block before ```rust Pipeline { elements: [ Expression(None, Expression { expr: ExternalCall(Expression { expr: String("ls"), span: Span { start: 39042, end: 39044 }, ty: String, custom_completion: None }, [], false), span: Span { start: 39041, end: 39044 }, ty: Any, custom_completion: None }), Redirection(Span { start: 39058, end: 39062 }, Stdout, Expression { expr: String("out.txt"), span: Span { start: 39050, end: 39057 }, ty: String, custom_completion: None }), Redirection(Span { start: 39058, end: 39062 }, Stderr, Expression { expr: String("err.txt"), span: Span { start: 39063, end: 39070 }, ty: String, custom_completion: None })] } ``` ### Block after ```rust Pipeline { elements: [ Expression(None, Expression { expr: ExternalCall(Expression { expr: String("ls"), span: Span { start: 38526, end: 38528 }, ty: String, custom_completion: None }, [], false), span: Span { start: 38525, end: 38528 }, ty: Any, custom_completion: None }), // new one! SeparateRedirection SeparateRedirection { out: (Span { start: 38529, end: 38533 }, Expression { expr: String("out.txt"), span: Span { start: 38534, end: 38541 }, ty: String, custom_completion: None }), err: (Span { start: 38542, end: 38546 }, Expression { expr: String("err.txt"), span: Span { start: 38547, end: 38554 }, ty: String, custom_completion: None }) } ] } ``` # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. |
||
Jakub Žádník
|
d8a2e0e9a3
|
Small parser refactors (#7568) |