nushell/crates/nu-protocol/src/ast/pipeline.rs
WindSoilder b150f9f5d8
Avoid blocking when o+e> redirects too much stderr message (#8784)
# Description

Fixes: #8565

Here is another pr #7240 tried to address the issue, but it works in a
wrong way.

After this change `o+e>` won't redirect all stdout message then stderr
message and it works more like how bash does.

# User-Facing Changes

For the given python code:
```python
# test.py
import sys

print('aa'*300, flush=True)
print('bb'*999999, file=sys.stderr, flush=True)
print('cc'*300, flush=True)
```

Running `python test.py out+err> a.txt` shoudn't hang nushell, and
`a.txt` keeps output in the same order

## About the change
The core idea is that when doing lite-parsing, introduce a new variant
`LiteElement::SameTargetRedirection` if we meet `out+err>` redirection
token(which is generated by lex function),

During converting from lite block to block,
LiteElement::SameTargetRedirection will be converted to
PipelineElement::SameTargetRedirection.

Then in the block eval process, if we get
PipelineElement::SameTargetRedirection, we'll invoke `run-external` with
`--redirect-combine` flag, then pipe the result into save command

## What happened internally?

Take the following command as example:
`^ls o+e> log.txt`

lex parsing result(`Tokens`) are not changed, but `LiteBlock` and
`Block` is changed after this pr.
### LiteBlock before
```rust
LiteBlock {
    block: [
        LitePipeline { commands: [
            Command(None, LiteCommand { comments: [], parts: [Span { start: 39041, end: 39044 }] }),
            // actually the span of first Redirection is wrong too..
            Redirection(Span { start: 39058, end: 39062 }, StdoutAndStderr, LiteCommand { comments: [], parts: [Span { start: 39050, end: 39057 }] }),
        ]
    }]
}
```
### LiteBlock after
```rust
LiteBlock { 
    block: [
        LitePipeline {
            commands: [
                SameTargetRedirection {
                    cmd: (None, LiteCommand { comments: [], parts: [Span { start: 147945, end: 147948}]}),
                    redirection: (Span { start: 147949, end: 147957 }, LiteCommand { comments: [], parts: [Span { start: 147958, end: 147965 }]})
                }
            ]
        }
    ]
}
```
### Block before
```rust
Pipeline {
    elements: [
        Expression(None, Expression {
            expr: ExternalCall(Expression { expr: String("ls"), span: Span { start: 39042, end: 39044 }, ty: String, custom_completion: None }, [], false),
            span: Span { start: 39041, end: 39044 },
            ty: Any, custom_completion: None 
        }),
        Redirection(Span { start: 39058, end: 39062 }, StdoutAndStderr, Expression { expr: String("out.txt"), span: Span { start: 39050, end: 39057 }, ty: String, custom_completion: None })] }
```
### Block after
```rust
Pipeline {
    elements: [
        SameTargetRedirection { 
            cmd: (None, Expression {
                expr: ExternalCall(Expression { expr: String("ls"), span: Span { start: 147946, end: 147948 }, ty: String, custom_completion: None}, [], false),
                span: Span { start: 147945, end: 147948},
                ty: Any, custom_completion: None
            }),
            redirection: (Span { start: 147949, end: 147957}, Expression {expr: String("log.txt"), span: Span { start: 147958, end: 147965 },ty: String,custom_completion: None}
        }
    ]
}
```

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-05-17 17:47:03 -05:00

185 lines
5.7 KiB
Rust

use crate::{ast::Expression, engine::StateWorkingSet, Span, VarId};
use serde::{Deserialize, Serialize};
use std::ops::{Index, IndexMut};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum Redirection {
Stdout,
Stderr,
StdoutAndStderr,
}
// Note: Span in the below is for the span of the connector not the whole element
#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum PipelineElement {
Expression(Option<Span>, Expression),
Redirection(Span, Redirection, Expression),
SeparateRedirection {
out: (Span, Expression),
err: (Span, Expression),
},
SameTargetRedirection {
cmd: (Option<Span>, Expression),
redirection: (Span, Expression),
},
And(Span, Expression),
Or(Span, Expression),
}
impl PipelineElement {
pub fn expression(&self) -> &Expression {
match self {
PipelineElement::Expression(_, expression) => expression,
PipelineElement::Redirection(_, _, expression) => expression,
PipelineElement::SeparateRedirection {
out: (_, expression),
..
} => expression,
PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression,
PipelineElement::And(_, expression) => expression,
PipelineElement::Or(_, expression) => expression,
}
}
pub fn span(&self) -> Span {
match self {
PipelineElement::Expression(None, expression)
| PipelineElement::SameTargetRedirection {
cmd: (None, expression),
..
} => expression.span,
PipelineElement::Expression(Some(span), expression)
| PipelineElement::Redirection(span, _, expression)
| PipelineElement::SeparateRedirection {
out: (span, expression),
..
}
| PipelineElement::And(span, expression)
| PipelineElement::Or(span, expression)
| PipelineElement::SameTargetRedirection {
cmd: (Some(span), expression),
..
} => Span {
start: span.start,
end: expression.span.end,
},
}
}
pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool {
match self {
PipelineElement::Expression(_, expression)
| PipelineElement::Redirection(_, _, expression)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression.has_in_variable(working_set),
PipelineElement::SeparateRedirection {
out: (_, out_expr),
err: (_, err_expr),
} => out_expr.has_in_variable(working_set) || err_expr.has_in_variable(working_set),
}
}
pub fn replace_in_variable(&mut self, working_set: &mut StateWorkingSet, new_var_id: VarId) {
match self {
PipelineElement::Expression(_, expression)
| PipelineElement::Redirection(_, _, expression)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression.replace_in_variable(working_set, new_var_id),
PipelineElement::SeparateRedirection {
out: (_, out_expr),
err: (_, err_expr),
} => {
if out_expr.has_in_variable(working_set) {
out_expr.replace_in_variable(working_set, new_var_id)
}
if err_expr.has_in_variable(working_set) {
err_expr.replace_in_variable(working_set, new_var_id)
}
}
}
}
pub fn replace_span(
&mut self,
working_set: &mut StateWorkingSet,
replaced: Span,
new_span: Span,
) {
match self {
PipelineElement::Expression(_, expression)
| PipelineElement::Redirection(_, _, expression)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
}
| PipelineElement::SeparateRedirection {
out: (_, expression),
..
} => expression.replace_span(working_set, replaced, new_span),
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Pipeline {
pub elements: Vec<PipelineElement>,
}
impl Default for Pipeline {
fn default() -> Self {
Self::new()
}
}
impl Pipeline {
pub fn new() -> Self {
Self { elements: vec![] }
}
pub fn from_vec(expressions: Vec<Expression>) -> Pipeline {
Self {
elements: expressions
.into_iter()
.enumerate()
.map(|(idx, x)| {
PipelineElement::Expression(if idx == 0 { None } else { Some(x.span) }, x)
})
.collect(),
}
}
pub fn len(&self) -> usize {
self.elements.len()
}
pub fn is_empty(&self) -> bool {
self.elements.is_empty()
}
}
impl Index<usize> for Pipeline {
type Output = PipelineElement;
fn index(&self, index: usize) -> &Self::Output {
&self.elements[index]
}
}
impl IndexMut<usize> for Pipeline {
fn index_mut(&mut self, index: usize) -> &mut Self::Output {
&mut self.elements[index]
}
}