extern/nushell
1
0
Fork 1
mirror of https://github.com/nushell/nushell.git synced 2025-01-09 07:48:14 +01:00
Code Issues Packages Projects Releases Wiki Activity

IO and redirection overhaul (#11934)

Browse Source 
# Description
The PR overhauls how IO redirection is handled, allowing more explicit
and fine-grain control over `stdout` and `stderr` output as well as more
efficient IO and piping.

To summarize the changes in this PR:
- Added a new `IoStream` type to indicate the intended destination for a
pipeline element's `stdout` and `stderr`.
- The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to
avoid adding 6 additional arguments to every eval function and
`Command::run`. The `stdout` and `stderr` streams can be temporarily
overwritten through functions on `Stack` and these functions will return
a guard that restores the original `stdout` and `stderr` when dropped.
- In the AST, redirections are now directly part of a `PipelineElement`
as a `Option<Redirection>` field instead of having multiple different
`PipelineElement` enum variants for each kind of redirection. This
required changes to the parser, mainly in `lite_parser.rs`.
- `Command`s can also set a `IoStream` override/redirection which will
apply to the previous command in the pipeline. This is used, for
example, in `ignore` to allow the previous external command to have its
stdout redirected to `Stdio::null()` at spawn time. In contrast, the
current implementation has to create an os pipe and manually consume the
output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`,
etc.) have precedence over overrides from commands.

This PR improves piping and IO speed, partially addressing #10763. Using
the `throughput` command from that issue, this PR gives the following
speedup on my setup for the commands below:
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| --------------------------- | -------------:| ------------:|
-----------:|
| `throughput o> /dev/null` | 1169 | 52938 | 54305 |
| `throughput \| ignore` | 840 | 55438 | N/A |
| `throughput \| null` | Error | 53617 | N/A |
| `throughput \| rg 'x'` | 1165 | 3049 | 3736 |
| `(throughput) \| rg 'x'` | 810 | 3085 | 3815 |

(Numbers above are the median samples for throughput)

This PR also paves the way to refactor our `ExternalStream` handling in
the various commands. For example, this PR already fixes the following
code:
```nushell
^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world"
```
This returns an empty list on 0.90.1 and returns a highlighted "hello
world" on this PR.

Since the `stdout` and `stderr` `IoStream`s are available to commands
when they are run, then this unlocks the potential for more convenient
behavior. E.g., the `find` command can disable its ansi highlighting if
it detects that the output `IoStream` is not the terminal. Knowing the
output streams will also allow background job output to be redirected
more easily and efficiently.

# User-Facing Changes
- External commands returned from closures will be collected (in most
cases):
  ```nushell
  1..2 | each {|_| nu -c "print a" }
  ```
This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n"
and then return an empty list.

  ```nushell
  1..2 | each {|_| nu -c "print -e a" }
  ```
This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used
to return an empty list and print "a\na\n" to stderr.

- Trailing new lines are always trimmed for external commands when
piping into internal commands or collecting it as a value. (Failure to
decode the output as utf-8 will keep the trailing newline for the last
binary value.) In the current nushell version, the following three code
snippets differ only in parenthesis placement, but they all also have
different outputs:

  1. `1..2 | each { ^echo a }`
     ```
     a
     a
     ╭────────────╮
     │ empty list │
     ╰────────────╯
     ```
  2. `1..2 | each { (^echo a) }`
     ```
     ╭───┬───╮
     │ 0 │ a │
     │ 1 │ a │
     ╰───┴───╯
     ```
  3. `1..2 | (each { ^echo a })`
     ```
     ╭───┬───╮
     │ 0 │ a │
     │   │   │
     │ 1 │ a │
     │   │   │
     ╰───┴───╯
     ```

  But in this PR, the above snippets will all have the same output:
  ```
  ╭───┬───╮
  │ 0 │ a │
  │ 1 │ a │
  ╰───┴───╯
  ```

- All existing flags on `run-external` are now deprecated.

- File redirections now apply to all commands inside a code block:
  ```nushell
  (nu -c "print -e a"; nu -c "print -e b") e> test.out
  ```
This gives "a\nb\n" in `test.out` and prints nothing. The same result
would happen when printing to stdout and using a `o>` file redirection.

- External command output will (almost) never be ignored, and ignoring
output must be explicit now:
  ```nushell
  (^echo a; ^echo b)
  ```
This prints "a\nb\n", whereas this used to print only "b\n". This only
applies to external commands; values and internal commands not in return
position will not print anything (e.g., `(echo a; echo b)` still only
prints "b").

- `complete` now always captures stderr (`do` is not necessary).

# After Submitting
The language guide and other documentation will need to be updated.
This commit is contained in:
Ian Manske 2024-03-14 20:51:55 +00:00 committed by GitHub
parent e2907e7e3a
commit b6c7656194
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
113 changed files with 3272 additions and 4022 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

530
crates/nu-cli/src/completions/completer.rs
View File

@ -4,10 +4,9 @@ use crate::completions::{
}; };
use nu_color_config::{color_record_to_nustyle, lookup_ansi_color_style}; use nu_color_config::{color_record_to_nustyle, lookup_ansi_color_style};
use nu_engine::eval_block; use nu_engine::eval_block;
use nu_parser::{flatten_expression, parse, FlatShape}; use nu_parser::{flatten_pipeline_element, parse, FlatShape};
use nu_protocol::debugger::WithoutDebug; use nu_protocol::debugger::WithoutDebug;
use nu_protocol::{ use nu_protocol::{
ast::PipelineElement,
engine::{EngineState, Stack, StateWorkingSet}, engine::{EngineState, Stack, StateWorkingSet},
BlockId, PipelineData, Span, Value, BlockId, PipelineData, Span, Value,
}; };
@ -25,7 +24,7 @@ impl NuCompleter {
pub fn new(engine_state: Arc<EngineState>, stack: Stack) -> Self { pub fn new(engine_state: Arc<EngineState>, stack: Stack) -> Self {
Self { Self {
engine_state, engine_state,
stack, stack: stack.reset_stdio().capture(),
} }
} }
@ -64,9 +63,10 @@ impl NuCompleter {
offset: usize, offset: usize,
span: Span, span: Span,
) -> Option<Vec<Suggestion>> { ) -> Option<Vec<Suggestion>> {
let stack = self.stack.clone();
let block = self.engine_state.get_block(block_id); let block = self.engine_state.get_block(block_id);
let mut callee_stack = stack.gather_captures(&self.engine_state, &block.captures); let mut callee_stack = self
.stack
.gather_captures(&self.engine_state, &block.captures);
// Line // Line
if let Some(pos_arg) = block.signature.required_positional.first() { if let Some(pos_arg) = block.signature.required_positional.first() {
@ -89,8 +89,6 @@ impl NuCompleter {
&mut callee_stack, &mut callee_stack,
block, block,
PipelineData::empty(), PipelineData::empty(),
true,
true,
); );
match result { match result {
@ -128,281 +126,265 @@ impl NuCompleter {
for pipeline in output.pipelines.into_iter() { for pipeline in output.pipelines.into_iter() {
for pipeline_element in pipeline.elements { for pipeline_element in pipeline.elements {
match pipeline_element { let flattened = flatten_pipeline_element(&working_set, &pipeline_element);
PipelineElement::Expression(_, expr) let mut spans: Vec<String> = vec![];
| PipelineElement::ErrPipedExpression(_, expr)
| PipelineElement::OutErrPipedExpression(_, expr)
| PipelineElement::Redirection(_, _, expr, _)
| PipelineElement::And(_, expr)
| PipelineElement::Or(_, expr)
| PipelineElement::SameTargetRedirection { cmd: (_, expr), .. }
| PipelineElement::SeparateRedirection {
out: (_, expr, _), ..
} => {
let flattened: Vec<_> = flatten_expression(&working_set, &expr);
let mut spans: Vec<String> = vec![];
for (flat_idx, flat) in flattened.iter().enumerate() { for (flat_idx, flat) in flattened.iter().enumerate() {
let is_passthrough_command = spans let is_passthrough_command = spans
.first() .first()
.filter(|content| { .filter(|content| content.as_str() == "sudo" || content.as_str() == "doas")
content.as_str() == "sudo" || content.as_str() == "doas" .is_some();
}) // Read the current spam to string
.is_some(); let current_span = working_set.get_span_contents(flat.0).to_vec();
// Read the current spam to string let current_span_str = String::from_utf8_lossy(&current_span);
let current_span = working_set.get_span_contents(flat.0).to_vec();
let current_span_str = String::from_utf8_lossy(&current_span);
let is_last_span = pos >= flat.0.start && pos < flat.0.end; let is_last_span = pos >= flat.0.start && pos < flat.0.end;
// Skip the last 'a' as span item // Skip the last 'a' as span item
if is_last_span { if is_last_span {
let offset = pos - flat.0.start; let offset = pos - flat.0.start;
if offset == 0 { if offset == 0 {
spans.push(String::new()) spans.push(String::new())
} else { } else {
let mut current_span_str = current_span_str.to_string(); let mut current_span_str = current_span_str.to_string();
current_span_str.remove(offset); current_span_str.remove(offset);
spans.push(current_span_str); spans.push(current_span_str);
} }
} else { } else {
spans.push(current_span_str.to_string()); spans.push(current_span_str.to_string());
}
// Complete based on the last span
if is_last_span {
// Context variables
let most_left_var =
most_left_variable(flat_idx, &working_set, flattened.clone());
// Create a new span
let new_span = Span::new(flat.0.start, flat.0.end - 1);
// Parses the prefix. Completion should look up to the cursor position, not after.
let mut prefix = working_set.get_span_contents(flat.0).to_vec();
let index = pos - flat.0.start;
prefix.drain(index..);
// Variables completion
if prefix.starts_with(b"$") || most_left_var.is_some() {
let mut completer = VariableCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
most_left_var.unwrap_or((vec![], vec![])),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
// Flags completion
if prefix.starts_with(b"-") {
// Try to complete flag internally
let mut completer = FlagCompletion::new(pipeline_element.expr.clone());
let result = self.process_completion(
&mut completer,
&working_set,
prefix.clone(),
new_span,
fake_offset,
pos,
);
if !result.is_empty() {
return result;
} }
// Complete based on the last span // We got no results for internal completion
if is_last_span { // now we can check if external completer is set and use it
// Context variables if let Some(block_id) = config.external_completer {
let most_left_var = if let Some(external_result) = self.external_completion(
most_left_variable(flat_idx, &working_set, flattened.clone()); block_id,
&spans,
// Create a new span fake_offset,
let new_span = Span::new(flat.0.start, flat.0.end - 1); new_span,
) {
// Parses the prefix. Completion should look up to the cursor position, not after. return external_result;
let mut prefix = working_set.get_span_contents(flat.0).to_vec();
let index = pos - flat.0.start;
prefix.drain(index..);
// Variables completion
if prefix.starts_with(b"$") || most_left_var.is_some() {
let mut completer = VariableCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
most_left_var.unwrap_or((vec![], vec![])),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
} }
// Flags completion
if prefix.starts_with(b"-") {
// Try to complete flag internally
let mut completer = FlagCompletion::new(expr.clone());
let result = self.process_completion(
&mut completer,
&working_set,
prefix.clone(),
new_span,
fake_offset,
pos,
);
if !result.is_empty() {
return result;
}
// We got no results for internal completion
// now we can check if external completer is set and use it
if let Some(block_id) = config.external_completer {
if let Some(external_result) = self.external_completion(
block_id,
&spans,
fake_offset,
new_span,
) {
return external_result;
}
}
}
// specially check if it is currently empty - always complete commands
if (is_passthrough_command && flat_idx == 1)
|| (flat_idx == 0
&& working_set.get_span_contents(new_span).is_empty())
{
let mut completer = CommandCompletion::new(
self.engine_state.clone(),
&working_set,
flattened.clone(),
// flat_idx,
FlatShape::String,
true,
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
// Completions that depends on the previous expression (e.g: use, source-env)
if (is_passthrough_command && flat_idx > 1) || flat_idx > 0 {
if let Some(previous_expr) = flattened.get(flat_idx - 1) {
// Read the content for the previous expression
let prev_expr_str =
working_set.get_span_contents(previous_expr.0).to_vec();
// Completion for .nu files
if prev_expr_str == b"use"
|| prev_expr_str == b"overlay use"
|| prev_expr_str == b"source-env"
{
let mut completer = DotNuCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
} else if prev_expr_str == b"ls" {
let mut completer = FileCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
}
}
// Match other types
match &flat.1 {
FlatShape::Custom(decl_id) => {
let mut completer = CustomCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
*decl_id,
initial_line,
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
FlatShape::Directory => {
let mut completer = DirectoryCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
FlatShape::Filepath | FlatShape::GlobPattern => {
let mut completer = FileCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
flat_shape => {
let mut completer = CommandCompletion::new(
self.engine_state.clone(),
&working_set,
flattened.clone(),
// flat_idx,
flat_shape.clone(),
false,
);
let mut out: Vec<_> = self.process_completion(
&mut completer,
&working_set,
prefix.clone(),
new_span,
fake_offset,
pos,
);
if !out.is_empty() {
return out;
}
// Try to complete using an external completer (if set)
if let Some(block_id) = config.external_completer {
if let Some(external_result) = self.external_completion(
block_id,
&spans,
fake_offset,
new_span,
) {
return external_result;
}
}
// Check for file completion
let mut completer = FileCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
out = self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
if !out.is_empty() {
return out;
}
}
};
} }
} }
// specially check if it is currently empty - always complete commands
if (is_passthrough_command && flat_idx == 1)
|| (flat_idx == 0 && working_set.get_span_contents(new_span).is_empty())
{
let mut completer = CommandCompletion::new(
self.engine_state.clone(),
&working_set,
flattened.clone(),
// flat_idx,
FlatShape::String,
true,
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
// Completions that depends on the previous expression (e.g: use, source-env)
if (is_passthrough_command && flat_idx > 1) || flat_idx > 0 {
if let Some(previous_expr) = flattened.get(flat_idx - 1) {
// Read the content for the previous expression
let prev_expr_str =
working_set.get_span_contents(previous_expr.0).to_vec();
// Completion for .nu files
if prev_expr_str == b"use"
|| prev_expr_str == b"overlay use"
|| prev_expr_str == b"source-env"
{
let mut completer = DotNuCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
} else if prev_expr_str == b"ls" {
let mut completer = FileCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
}
}
// Match other types
match &flat.1 {
FlatShape::Custom(decl_id) => {
let mut completer = CustomCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
*decl_id,
initial_line,
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
FlatShape::Directory => {
let mut completer = DirectoryCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
FlatShape::Filepath | FlatShape::GlobPattern => {
let mut completer = FileCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
return self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
}
flat_shape => {
let mut completer = CommandCompletion::new(
self.engine_state.clone(),
&working_set,
flattened.clone(),
// flat_idx,
flat_shape.clone(),
false,
);
let mut out: Vec<_> = self.process_completion(
&mut completer,
&working_set,
prefix.clone(),
new_span,
fake_offset,
pos,
);
if !out.is_empty() {
return out;
}
// Try to complete using an external completer (if set)
if let Some(block_id) = config.external_completer {
if let Some(external_result) = self.external_completion(
block_id,
&spans,
fake_offset,
new_span,
) {
return external_result;
}
}
// Check for file completion
let mut completer = FileCompletion::new(
self.engine_state.clone(),
self.stack.clone(),
);
out = self.process_completion(
&mut completer,
&working_set,
prefix,
new_span,
fake_offset,
pos,
);
if !out.is_empty() {
return out;
}
}
};
} }
} }
} }

4
crates/nu-cli/src/completions/custom_completions.rs
View File

@ -25,7 +25,7 @@ impl CustomCompletion {
pub fn new(engine_state: Arc<EngineState>, stack: Stack, decl_id: usize, line: String) -> Self { pub fn new(engine_state: Arc<EngineState>, stack: Stack, decl_id: usize, line: String) -> Self {
Self { Self {
engine_state, engine_state,
stack, stack: stack.reset_stdio().capture(),
decl_id, decl_id,
line, line,
sort_by: SortBy::None, sort_by: SortBy::None,
@ -67,8 +67,6 @@ impl Completer for CustomCompletion {
custom_completion: None, custom_completion: None,
}), }),
], ],
redirect_stdout: true,
redirect_stderr: true,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}, },
PipelineData::empty(), PipelineData::empty(),

32
crates/nu-cli/src/eval_cmds.rs
View File

@ -56,27 +56,21 @@ pub fn evaluate_commands(
} }
// Run the block // Run the block
let exit_code = let exit_code = match eval_block::<WithoutDebug>(engine_state, stack, &block, input) {
match eval_block::<WithoutDebug>(engine_state, stack, &block, input, false, false) { Ok(pipeline_data) => {
Ok(pipeline_data) => { let mut config = engine_state.get_config().clone();
let mut config = engine_state.get_config().clone(); if let Some(t_mode) = table_mode {
if let Some(t_mode) = table_mode { config.table_mode = t_mode.coerce_str()?.parse().unwrap_or_default();
config.table_mode = t_mode.coerce_str()?.parse().unwrap_or_default();
}
crate::eval_file::print_table_or_error(
engine_state,
stack,
pipeline_data,
&mut config,
)
} }
Err(err) => { crate::eval_file::print_table_or_error(engine_state, stack, pipeline_data, &mut config)
let working_set = StateWorkingSet::new(engine_state); }
Err(err) => {
let working_set = StateWorkingSet::new(engine_state);
report_error(&working_set, &err); report_error(&working_set, &err);
std::process::exit(1); std::process::exit(1);
} }
}; };
info!("evaluate {}:{}:{}", file!(), line!(), column!()); info!("evaluate {}:{}:{}", file!(), line!(), column!());

13
crates/nu-cli/src/eval_file.rs
View File

@ -131,14 +131,8 @@ pub fn evaluate_file(
if engine_state.find_decl(b"main", &[]).is_some() { if engine_state.find_decl(b"main", &[]).is_some() {
let args = format!("main {}", args.join(" ")); let args = format!("main {}", args.join(" "));
let pipeline_data = eval_block::<WithoutDebug>( let pipeline_data =
engine_state, eval_block::<WithoutDebug>(engine_state, stack, &block, PipelineData::empty());
stack,
&block,
PipelineData::empty(),
false,
false,
);
let pipeline_data = match pipeline_data { let pipeline_data = match pipeline_data {
Err(ShellError::Return { .. }) => { Err(ShellError::Return { .. }) => {
// allows early exists before `main` is run. // allows early exists before `main` is run.
@ -214,8 +208,7 @@ pub(crate) fn print_table_or_error(
print_or_exit(pipeline_data, engine_state, config); print_or_exit(pipeline_data, engine_state, config);
} else { } else {
// The final call on table command, it's ok to set redirect_output to false. // The final call on table command, it's ok to set redirect_output to false.
let mut call = Call::new(Span::new(0, 0)); let call = Call::new(Span::new(0, 0));
call.redirect_stdout = false;
let table = command.run(engine_state, stack, &call, pipeline_data); let table = command.run(engine_state, stack, &call, pipeline_data);
match table { match table {

11
crates/nu-cli/src/menus/menu_completions.rs
View File

@ -28,7 +28,7 @@ impl NuMenuCompleter {
Self { Self {
block_id, block_id,
span, span,
stack, stack: stack.reset_stdio().capture(),
engine_state, engine_state,
only_buffer_difference, only_buffer_difference,
} }
@ -57,14 +57,7 @@ impl Completer for NuMenuCompleter {
let input = Value::nothing(self.span).into_pipeline_data(); let input = Value::nothing(self.span).into_pipeline_data();
let res = eval_block::<WithoutDebug>( let res = eval_block::<WithoutDebug>(&self.engine_state, &mut self.stack, block, input);
&self.engine_state,
&mut self.stack,
block,
input,
false,
false,
);
if let Ok(values) = res { if let Ok(values) = res {
let values = values.into_value(self.span); let values = values.into_value(self.span);

12
crates/nu-cli/src/reedline_config.rs
View File

@ -109,17 +109,9 @@ pub(crate) fn add_menus(
(output, working_set.render()) (output, working_set.render())
}; };
let mut temp_stack = Stack::new(); let mut temp_stack = Stack::new().capture();
let input = PipelineData::Empty; let input = PipelineData::Empty;
let res = eval_block::<WithoutDebug>(&engine_state, &mut temp_stack, &block, input)?;
let res = eval_block::<WithoutDebug>(
&engine_state,
&mut temp_stack,
&block,
input,
false,
false,
)?;
if let PipelineData::Value(value, None) = res { if let PipelineData::Value(value, None) = res {
for menu in create_menus(&value)? { for menu in create_menus(&value)? {

48
crates/nu-cli/src/syntax_highlight.rs
View File

@ -3,7 +3,7 @@ use nu_ansi_term::Style;
use nu_color_config::{get_matching_brackets_style, get_shape_color}; use nu_color_config::{get_matching_brackets_style, get_shape_color};
use nu_engine::env; use nu_engine::env;
use nu_parser::{flatten_block, parse, FlatShape}; use nu_parser::{flatten_block, parse, FlatShape};
use nu_protocol::ast::{Argument, Block, Expr, Expression, PipelineElement, RecordItem}; use nu_protocol::ast::{Argument, Block, Expr, Expression, PipelineRedirection, RecordItem};
use nu_protocol::engine::{EngineState, Stack, StateWorkingSet}; use nu_protocol::engine::{EngineState, Stack, StateWorkingSet};
use nu_protocol::{Config, Span}; use nu_protocol::{Config, Span};
use reedline::{Highlighter, StyledText}; use reedline::{Highlighter, StyledText};
@ -262,26 +262,38 @@ fn find_matching_block_end_in_block(
) -> Option<usize> { ) -> Option<usize> {
for p in &block.pipelines { for p in &block.pipelines {
for e in &p.elements { for e in &p.elements {
match e { if e.expr.span.contains(global_cursor_offset) {
PipelineElement::Expression(_, e) if let Some(pos) = find_matching_block_end_in_expr(
| PipelineElement::ErrPipedExpression(_, e) line,
| PipelineElement::OutErrPipedExpression(_, e) working_set,
| PipelineElement::Redirection(_, _, e, _) &e.expr,
| PipelineElement::And(_, e) global_span_offset,
| PipelineElement::Or(_, e) global_cursor_offset,
| PipelineElement::SameTargetRedirection { cmd: (_, e), .. } ) {
| PipelineElement::SeparateRedirection { out: (_, e, _), .. } => { return Some(pos);
if e.span.contains(global_cursor_offset) { }
if let Some(pos) = find_matching_block_end_in_expr( }
line,
working_set, if let Some(redirection) = e.redirection.as_ref() {
e, match redirection {
global_span_offset, PipelineRedirection::Single { target, .. }
global_cursor_offset, | PipelineRedirection::Separate { out: target, .. }
) { | PipelineRedirection::Separate { err: target, .. }
if target.span().contains(global_cursor_offset) =>
{
if let Some(pos) = target.expr().and_then(|expr| {
find_matching_block_end_in_expr(
line,
working_set,
expr,
global_span_offset,
global_cursor_offset,
)
}) {
return Some(pos); return Some(pos);
} }
} }
_ => {}
} }
} }
} }

11
crates/nu-cli/src/util.rs
View File

@ -241,16 +241,9 @@ pub fn eval_source(
} }
let b = if allow_return { let b = if allow_return {
eval_block_with_early_return::<WithoutDebug>( eval_block_with_early_return::<WithoutDebug>(engine_state, stack, &block, input)
engine_state,
stack,
&block,
input,
false,
false,
)
} else { } else {
eval_block::<WithoutDebug>(engine_state, stack, &block, input, false, false) eval_block::<WithoutDebug>(engine_state, stack, &block, input)
}; };
match b { match b {

4
crates/nu-cli/tests/support/completions_helpers.rs
View File

@ -200,9 +200,7 @@ pub fn merge_input(
engine_state, engine_state,
stack, stack,
&block, &block,
PipelineData::Value(Value::nothing(Span::unknown(),), None), PipelineData::Value(Value::nothing(Span::unknown()), None),
false,
false,
) )
.is_ok()); .is_ok());

17
crates/nu-cmd-base/src/hook.rs
View File

@ -116,7 +116,7 @@ pub fn eval_hook(
}) })
.collect(); .collect();
match eval_block::<WithoutDebug>(engine_state, stack, &block, input, false, false) { match eval_block::<WithoutDebug>(engine_state, stack, &block, input) {
Ok(pipeline_data) => { Ok(pipeline_data) => {
output = pipeline_data; output = pipeline_data;
} }
@ -244,14 +244,7 @@ pub fn eval_hook(
}) })
.collect(); .collect();
match eval_block::<WithoutDebug>( match eval_block::<WithoutDebug>(engine_state, stack, &block, input) {
engine_state,
stack,
&block,
input,
false,
false,
) {
Ok(pipeline_data) => { Ok(pipeline_data) => {
output = pipeline_data; output = pipeline_data;
} }
@ -327,7 +320,9 @@ fn run_hook_block(
let input = optional_input.unwrap_or_else(PipelineData::empty); let input = optional_input.unwrap_or_else(PipelineData::empty);
let mut callee_stack = stack.gather_captures(engine_state, &block.captures); let mut callee_stack = stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
for (idx, PositionalArg { var_id, .. }) in for (idx, PositionalArg { var_id, .. }) in
block.signature.required_positional.iter().enumerate() block.signature.required_positional.iter().enumerate()
@ -349,8 +344,6 @@ fn run_hook_block(
&mut callee_stack, &mut callee_stack,
block, block,
input, input,
false,
false,
)?; )?;
if let PipelineData::Value(Value::Error { error, .. }, _) = pipeline_data { if let PipelineData::Value(Value::Error { error, .. }, _) = pipeline_data {

16
crates/nu-cmd-dataframe/src/dataframe/test_dataframe.rs
View File

@ -79,18 +79,12 @@ pub fn test_dataframe_example(engine_state: &mut Box<EngineState>, example: &Exa
.merge_delta(delta) .merge_delta(delta)
.expect("Error merging delta"); .expect("Error merging delta");
let mut stack = Stack::new(); let mut stack = Stack::new().capture();
let result = eval_block::<WithoutDebug>( let result =
engine_state, eval_block::<WithoutDebug>(engine_state, &mut stack, &block, PipelineData::empty())
&mut stack, .unwrap_or_else(|err| panic!("test eval error in `{}`: {:?}", example.example, err))
&block, .into_value(Span::test_data());
PipelineData::empty(),
true,
true,
)
.unwrap_or_else(|err| panic!("test eval error in `{}`: {:?}", example.example, err))
.into_value(Span::test_data());
println!("input: {}", example.example); println!("input: {}", example.example);
println!("result: {result:?}"); println!("result: {result:?}");

8
crates/nu-cmd-extra/src/extra/filters/each_while.rs
View File

@ -83,8 +83,6 @@ impl Command for EachWhile {
let orig_env_vars = stack.env_vars.clone(); let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone(); let orig_env_hidden = stack.env_hidden.clone();
let span = call.head; let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state);
match input { match input {
@ -111,8 +109,6 @@ impl Command for EachWhile {
&mut stack, &mut stack,
&block, &block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => { Ok(v) => {
let value = v.into_value(span); let value = v.into_value(span);
@ -155,8 +151,6 @@ impl Command for EachWhile {
&mut stack, &mut stack,
&block, &block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => { Ok(v) => {
let value = v.into_value(span); let value = v.into_value(span);
@ -185,8 +179,6 @@ impl Command for EachWhile {
&mut stack, &mut stack,
&block, &block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) )
} }
} }

22
crates/nu-cmd-extra/src/extra/filters/update_cells.rs
View File

@ -105,9 +105,6 @@ impl Command for UpdateCells {
let block: Block = engine_state.get_block(block.block_id).clone(); let block: Block = engine_state.get_block(block.block_id).clone();
let eval_block_fn = get_eval_block(&engine_state); let eval_block_fn = get_eval_block(&engine_state);
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let span = call.head; let span = call.head;
stack.with_env(&orig_env_vars, &orig_env_hidden); stack.with_env(&orig_env_vars, &orig_env_hidden);
@ -130,8 +127,6 @@ impl Command for UpdateCells {
stack, stack,
block, block,
columns, columns,
redirect_stdout,
redirect_stderr,
span, span,
eval_block_fn, eval_block_fn,
} }
@ -146,8 +141,6 @@ struct UpdateCellIterator {
engine_state: EngineState, engine_state: EngineState,
stack: Stack, stack: Stack,
block: Block, block: Block,
redirect_stdout: bool,
redirect_stderr: bool,
eval_block_fn: EvalBlockFn, eval_block_fn: EvalBlockFn,
span: Span, span: Span,
} }
@ -177,8 +170,6 @@ impl Iterator for UpdateCellIterator {
&self.engine_state, &self.engine_state,
&mut self.stack, &mut self.stack,
&self.block, &self.block,
self.redirect_stdout,
self.redirect_stderr,
span, span,
self.eval_block_fn, self.eval_block_fn,
), ),
@ -192,8 +183,6 @@ impl Iterator for UpdateCellIterator {
&self.engine_state, &self.engine_state,
&mut self.stack, &mut self.stack,
&self.block, &self.block,
self.redirect_stdout,
self.redirect_stderr,
self.span, self.span,
self.eval_block_fn, self.eval_block_fn,
)), )),
@ -210,8 +199,6 @@ fn process_cell(
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
block: &Block, block: &Block,
redirect_stdout: bool,
redirect_stderr: bool,
span: Span, span: Span,
eval_block_fn: EvalBlockFn, eval_block_fn: EvalBlockFn,
) -> Value { ) -> Value {
@ -221,14 +208,7 @@ fn process_cell(
} }
} }
match eval_block_fn( match eval_block_fn(engine_state, stack, block, val.into_pipeline_data()) {
engine_state,
stack,
block,
val.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) {
Ok(pd) => pd.into_value(span), Ok(pd) => pd.into_value(span),
Err(e) => Value::error(e, span), Err(e) => Value::error(e, span),
} }

2
crates/nu-cmd-extra/src/extra/strings/format/command.rs
View File

@ -295,7 +295,7 @@ fn format_record(
} }
} }
FormatOperation::ValueNeedEval(_col_name, span) => { FormatOperation::ValueNeedEval(_col_name, span) => {
let exp = parse_expression(working_set, &[*span], false); let exp = parse_expression(working_set, &[*span]);
match working_set.parse_errors.first() { match working_set.parse_errors.first() {
None => { None => {
let parsed_result = eval_expression(engine_state, stack, &exp); let parsed_result = eval_expression(engine_state, stack, &exp);

5
crates/nu-cmd-lang/src/core_commands/collect.rs
View File

@ -45,7 +45,8 @@ impl Command for Collect {
let capture_block: Closure = call.req(engine_state, stack, 0)?; let capture_block: Closure = call.req(engine_state, stack, 0)?;
let block = engine_state.get_block(capture_block.block_id).clone(); let block = engine_state.get_block(capture_block.block_id).clone();
let mut stack_captures = stack.captures_to_stack(capture_block.captures.clone()); let mut stack_captures =
stack.captures_to_stack_preserve_stdio(capture_block.captures.clone());
let metadata = input.metadata(); let metadata = input.metadata();
let input: Value = input.into_value(call.head); let input: Value = input.into_value(call.head);
@ -65,8 +66,6 @@ impl Command for Collect {
&mut stack_captures, &mut stack_captures,
&block, &block,
input.into_pipeline_data(), input.into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
) )
.map(|x| x.set_metadata(metadata)); .map(|x| x.set_metadata(metadata));

19
crates/nu-cmd-lang/src/core_commands/do_.rs
View File

@ -5,8 +5,8 @@ use nu_protocol::ast::Call;
use nu_protocol::engine::{Closure, Command, EngineState, Stack}; use nu_protocol::engine::{Closure, Command, EngineState, Stack};
use nu_protocol::{ use nu_protocol::{
Category, Example, IntoSpanned, ListStream, PipelineData, RawStream, ShellError, Signature, Category, Example, IntoSpanned, IoStream, ListStream, PipelineData, RawStream, ShellError,
Span, SyntaxShape, Type, Value, Signature, Span, SyntaxShape, Type, Value,
}; };
#[derive(Clone)] #[derive(Clone)]
@ -79,19 +79,12 @@ impl Command for Do {
let capture_errors = call.has_flag(engine_state, caller_stack, "capture-errors")?; let capture_errors = call.has_flag(engine_state, caller_stack, "capture-errors")?;
let has_env = call.has_flag(engine_state, caller_stack, "env")?; let has_env = call.has_flag(engine_state, caller_stack, "env")?;
let mut callee_stack = caller_stack.captures_to_stack(block.captures); let mut callee_stack = caller_stack.captures_to_stack_preserve_stdio(block.captures);
let block = engine_state.get_block(block.block_id); let block = engine_state.get_block(block.block_id);
bind_args_to(&mut callee_stack, &block.signature, rest, call.head)?; bind_args_to(&mut callee_stack, &block.signature, rest, call.head)?;
let eval_block_with_early_return = get_eval_block_with_early_return(engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(engine_state);
let result = eval_block_with_early_return( let result = eval_block_with_early_return(engine_state, &mut callee_stack, block, input);
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stdout,
);
if has_env { if has_env {
// Merge the block's environment to the current stack // Merge the block's environment to the current stack
@ -204,7 +197,9 @@ impl Command for Do {
span, span,
metadata, metadata,
trim_end_newline, trim_end_newline,
}) if ignore_program_errors && !call.redirect_stdout => { }) if ignore_program_errors
&& !matches!(caller_stack.stdout(), IoStream::Pipe | IoStream::Capture) =>
{
Ok(PipelineData::ExternalStream { Ok(PipelineData::ExternalStream {
stdout, stdout,
stderr, stderr,

50
crates/nu-cmd-lang/src/core_commands/echo.rs
View File

@ -2,8 +2,8 @@ use nu_engine::CallExt;
use nu_protocol::ast::Call; use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack}; use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::{ use nu_protocol::{
Category, Example, ListStream, PipelineData, ShellError, Signature, Span, SyntaxShape, Type, Category, Example, IntoPipelineData, PipelineData, ShellError, Signature, Span, SyntaxShape,
Value, Type, Value,
}; };
#[derive(Clone)] #[derive(Clone)]
@ -38,8 +38,13 @@ little reason to use this over just writing the values as-is."#
call: &Call, call: &Call,
_input: PipelineData, _input: PipelineData,
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
let args = call.rest(engine_state, stack, 0); let mut args = call.rest(engine_state, stack, 0)?;
run(engine_state, args, stack, call) let value = match args.len() {
0 => Value::string("", call.head),
1 => args.pop().expect("one element"),
_ => Value::list(args, call.head),
};
Ok(value.into_pipeline_data())
} }
fn examples(&self) -> Vec<Example> { fn examples(&self) -> Vec<Example> {
@ -62,43 +67,6 @@ little reason to use this over just writing the values as-is."#
} }
} }
fn run(
engine_state: &EngineState,
args: Result<Vec<Value>, ShellError>,
stack: &mut Stack,
call: &Call,
) -> Result<PipelineData, ShellError> {
let result = args.map(|to_be_echoed| {
let n = to_be_echoed.len();
match n.cmp(&1usize) {
// More than one value is converted in a stream of values
std::cmp::Ordering::Greater => PipelineData::ListStream(
ListStream::from_stream(to_be_echoed.into_iter(), engine_state.ctrlc.clone()),
None,
),
// But a single value can be forwarded as it is
std::cmp::Ordering::Equal => PipelineData::Value(to_be_echoed[0].clone(), None),
// When there are no elements, we echo the empty string
std::cmp::Ordering::Less => PipelineData::Value(Value::string("", call.head), None),
}
});
// If echo is not redirected, then print to the screen (to behave in a similar way to other shells)
if !call.redirect_stdout {
match result {
Ok(pipeline) => {
pipeline.print(engine_state, stack, false, false)?;
Ok(PipelineData::Empty)
}
Err(err) => Err(err),
}
} else {
result
}
}
#[cfg(test)] #[cfg(test)]
mod test { mod test {
#[test] #[test]

32
crates/nu-cmd-lang/src/core_commands/for_.rs
View File

@ -84,8 +84,8 @@ impl Command for For {
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
let block = engine_state.get_block(block.block_id).clone(); let block = engine_state.get_block(block.block_id).clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr; let stack = &mut stack.push_redirection(None, None);
match values { match values {
Value::List { vals, .. } => { Value::List { vals, .. } => {
@ -109,14 +109,7 @@ impl Command for For {
}, },
); );
match eval_block( match eval_block(&engine_state, stack, &block, PipelineData::empty()) {
&engine_state,
stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
) {
Err(ShellError::Break { .. }) => { Err(ShellError::Break { .. }) => {
break; break;
} }
@ -154,14 +147,7 @@ impl Command for For {
}, },
); );
match eval_block( match eval_block(&engine_state, stack, &block, PipelineData::empty()) {
&engine_state,
stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
) {
Err(ShellError::Break { .. }) => { Err(ShellError::Break { .. }) => {
break; break;
} }
@ -185,15 +171,7 @@ impl Command for For {
x => { x => {
stack.add_var(var_id, x); stack.add_var(var_id, x);
eval_block( eval_block(&engine_state, stack, &block, PipelineData::empty())?.into_value(head);
&engine_state,
stack,
&block,
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)?
.into_value(head);
} }
} }
Ok(PipelineData::empty()) Ok(PipelineData::empty())

40
crates/nu-cmd-lang/src/core_commands/if_.rs
View File

@ -115,47 +115,19 @@ impl Command for If {
Value::Bool { val, .. } => { Value::Bool { val, .. } => {
if *val { if *val {
let block = engine_state.get_block(then_block.block_id); let block = engine_state.get_block(then_block.block_id);
eval_block( eval_block(engine_state, stack, block, input)
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
} else if let Some(else_case) = else_case { } else if let Some(else_case) = else_case {
if let Some(else_expr) = else_case.as_keyword() { if let Some(else_expr) = else_case.as_keyword() {
if let Some(block_id) = else_expr.as_block() { if let Some(block_id) = else_expr.as_block() {
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
eval_block( eval_block(engine_state, stack, block, input)
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
} else { } else {
eval_expression_with_input( eval_expression_with_input(engine_state, stack, else_expr, input)
engine_state, .map(|res| res.0)
stack,
else_expr,
input,
call.redirect_stdout,
call.redirect_stderr,
)
.map(|res| res.0)
} }
} else { } else {
eval_expression_with_input( eval_expression_with_input(engine_state, stack, else_case, input)
engine_state, .map(|res| res.0)
stack,
else_case,
input,
call.redirect_stdout,
call.redirect_stderr,
)
.map(|res| res.0)
} }
} else { } else {
Ok(PipelineData::empty()) Ok(PipelineData::empty())

8
crates/nu-cmd-lang/src/core_commands/ignore.rs
View File

@ -1,6 +1,8 @@
use nu_protocol::ast::Call; use nu_protocol::ast::Call;
use nu_protocol::engine::{Command, EngineState, Stack, StateWorkingSet}; use nu_protocol::engine::{Command, EngineState, Stack, StateWorkingSet};
use nu_protocol::{Category, Example, PipelineData, ShellError, Signature, Span, Type, Value}; use nu_protocol::{
Category, Example, IoStream, PipelineData, ShellError, Signature, Span, Type, Value,
};
#[derive(Clone)] #[derive(Clone)]
pub struct Ignore; pub struct Ignore;
@ -56,6 +58,10 @@ impl Command for Ignore {
result: Some(Value::nothing(Span::test_data())), result: Some(Value::nothing(Span::test_data())),
}] }]
} }
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Null), None)
}
} }
#[cfg(test)] #[cfg(test)]

6
crates/nu-cmd-lang/src/core_commands/lazy_make.rs
View File

@ -86,10 +86,12 @@ impl Command for LazyMake {
} }
} }
let stack = stack.clone().reset_stdio().capture();
Ok(Value::lazy_record( Ok(Value::lazy_record(
Box::new(NuLazyRecord { Box::new(NuLazyRecord {
engine_state: engine_state.clone(), engine_state: engine_state.clone(),
stack: Arc::new(Mutex::new(stack.clone())), stack: Arc::new(Mutex::new(stack)),
columns: columns.into_iter().map(|s| s.item).collect(), columns: columns.into_iter().map(|s| s.item).collect(),
get_value, get_value,
span, span,
@ -152,8 +154,6 @@ impl<'a> LazyRecord<'a> for NuLazyRecord {
&mut stack, &mut stack,
block, block,
PipelineData::Value(column_value, None), PipelineData::Value(column_value, None),
false,
false,
); );
pipeline_result.map(|data| match data { pipeline_result.map(|data| match data {

3
crates/nu-cmd-lang/src/core_commands/let_.rs
View File

@ -64,7 +64,8 @@ impl Command for Let {
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let pipeline_data = eval_block(engine_state, stack, block, input, true, false)?; let stack = &mut stack.start_capture();
let pipeline_data = eval_block(engine_state, stack, block, input)?;
let mut value = pipeline_data.into_value(call.head); let mut value = pipeline_data.into_value(call.head);
// if given variable type is Glob, and our result is string // if given variable type is Glob, and our result is string

11
crates/nu-cmd-lang/src/core_commands/loop_.rs
View File

@ -36,6 +36,8 @@ impl Command for Loop {
let block: Block = call.req(engine_state, stack, 0)?; let block: Block = call.req(engine_state, stack, 0)?;
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let stack = &mut stack.push_redirection(None, None);
loop { loop {
if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) { if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) {
break; break;
@ -43,14 +45,7 @@ impl Command for Loop {
let block = engine_state.get_block(block.block_id); let block = engine_state.get_block(block.block_id);
match eval_block( match eval_block(engine_state, stack, block, PipelineData::empty()) {
engine_state,
stack,
block,
PipelineData::empty(),
call.redirect_stdout,
call.redirect_stderr,
) {
Err(ShellError::Break { .. }) => { Err(ShellError::Break { .. }) => {
break; break;
} }

20
crates/nu-cmd-lang/src/core_commands/match_.rs
View File

@ -70,24 +70,10 @@ impl Command for Match {
if guard_matches { if guard_matches {
return if let Some(block_id) = match_.1.as_block() { return if let Some(block_id) = match_.1.as_block() {
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
eval_block( eval_block(engine_state, stack, block, input)
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
} else { } else {
eval_expression_with_input( eval_expression_with_input(engine_state, stack, &match_.1, input)
engine_state, .map(|x| x.0)
stack,
&match_.1,
input,
call.redirect_stdout,
call.redirect_stderr,
)
.map(|x| x.0)
}; };
} }
} }

11
crates/nu-cmd-lang/src/core_commands/mut_.rs
View File

@ -65,15 +65,8 @@ impl Command for Mut {
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let stack = &mut stack.start_capture();
let pipeline_data = eval_block( let pipeline_data = eval_block(engine_state, stack, block, input)?;
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)?;
let mut value = pipeline_data.into_value(call.head); let mut value = pipeline_data.into_value(call.head);
// if given variable type is Glob, and our result is string // if given variable type is Glob, and our result is string

14
crates/nu-cmd-lang/src/core_commands/overlay/use_.rs
View File

@ -124,7 +124,9 @@ impl Command for OverlayUse {
)?; )?;
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures); let mut callee_stack = caller_stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
if let Some(path) = &maybe_path { if let Some(path) = &maybe_path {
// Set the currently evaluated directory, if the argument is a valid path // Set the currently evaluated directory, if the argument is a valid path
@ -142,15 +144,7 @@ impl Command for OverlayUse {
} }
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let _ = eval_block(engine_state, &mut callee_stack, block, input);
let _ = eval_block(
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
);
// The export-env block should see the env vars *before* activating this overlay // The export-env block should see the env vars *before* activating this overlay
caller_stack.add_overlay(overlay_name); caller_stack.add_overlay(overlay_name);

4
crates/nu-cmd-lang/src/core_commands/try_.rs
View File

@ -50,7 +50,7 @@ impl Command for Try {
let try_block = engine_state.get_block(try_block.block_id); let try_block = engine_state.get_block(try_block.block_id);
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let result = eval_block(engine_state, stack, try_block, input, false, false); let result = eval_block(engine_state, stack, try_block, input);
match result { match result {
Err(error) => { Err(error) => {
@ -117,8 +117,6 @@ fn handle_catch(
catch_block, catch_block,
// Make the error accessible with $in, too // Make the error accessible with $in, too
err_value.into_pipeline_data(), err_value.into_pipeline_data(),
false,
false,
) )
} else { } else {
Ok(PipelineData::empty()) Ok(PipelineData::empty())

13
crates/nu-cmd-lang/src/core_commands/use_.rs
View File

@ -105,7 +105,9 @@ This command is a parser keyword. For details, check:
.as_ref() .as_ref()
.and_then(|path| path.parent().map(|p| p.to_path_buf())); .and_then(|path| path.parent().map(|p| p.to_path_buf()));
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures); let mut callee_stack = caller_stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
// If so, set the currently evaluated directory (file-relative PWD) // If so, set the currently evaluated directory (file-relative PWD)
if let Some(parent) = maybe_parent { if let Some(parent) = maybe_parent {
@ -121,14 +123,7 @@ This command is a parser keyword. For details, check:
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
// Run the block (discard the result) // Run the block (discard the result)
let _ = eval_block( let _ = eval_block(engine_state, &mut callee_stack, block, input)?;
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)?;
// Merge the block's environment to the current stack // Merge the block's environment to the current stack
redirect_env(engine_state, caller_stack, &callee_stack); redirect_env(engine_state, caller_stack, &callee_stack);

11
crates/nu-cmd-lang/src/core_commands/while_.rs
View File

@ -48,6 +48,8 @@ impl Command for While {
let eval_expression = get_eval_expression(engine_state); let eval_expression = get_eval_expression(engine_state);
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let stack = &mut stack.push_redirection(None, None);
loop { loop {
if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) { if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) {
break; break;
@ -60,14 +62,7 @@ impl Command for While {
if *val { if *val {
let block = engine_state.get_block(block.block_id); let block = engine_state.get_block(block.block_id);
match eval_block( match eval_block(engine_state, stack, block, PipelineData::empty()) {
engine_state,
stack,
block,
PipelineData::empty(),
call.redirect_stdout,
call.redirect_stderr,
) {
Err(ShellError::Break { .. }) => { Err(ShellError::Break { .. }) => {
break; break;
} }

7
crates/nu-cmd-lang/src/example_support.rs
View File

@ -112,12 +112,11 @@ pub fn eval_block(
.merge_delta(delta) .merge_delta(delta)
.expect("Error merging delta"); .expect("Error merging delta");
let mut stack = Stack::new(); let mut stack = Stack::new().capture();
stack.add_env_var("PWD".to_string(), Value::test_string(cwd.to_string_lossy())); stack.add_env_var("PWD".to_string(), Value::test_string(cwd.to_string_lossy()));
match nu_engine::eval_block::<WithoutDebug>(engine_state, &mut stack, &block, input, true, true) match nu_engine::eval_block::<WithoutDebug>(engine_state, &mut stack, &block, input) {
{
Err(err) => panic!("test eval error in `{}`: {:?}", "TODO", err), Err(err) => panic!("test eval error in `{}`: {:?}", "TODO", err),
Ok(result) => result.into_value(Span::test_data()), Ok(result) => result.into_value(Span::test_data()),
} }
@ -128,7 +127,7 @@ pub fn check_example_evaluates_to_expected_output(
cwd: &std::path::Path, cwd: &std::path::Path,
engine_state: &mut Box<EngineState>, engine_state: &mut Box<EngineState>,
) { ) {
let mut stack = Stack::new(); let mut stack = Stack::new().capture();
// Set up PWD // Set up PWD
stack.add_env_var("PWD".to_string(), Value::test_string(cwd.to_string_lossy())); stack.add_env_var("PWD".to_string(), Value::test_string(cwd.to_string_lossy()));

2
crates/nu-color-config/src/style_computer.rs
View File

@ -78,8 +78,6 @@ impl<'a> StyleComputer<'a> {
&mut stack, &mut stack,
&block, &block,
value.clone().into_pipeline_data(), value.clone().into_pipeline_data(),
false,
false,
) { ) {
Ok(v) => { Ok(v) => {
let value = v.into_value(span); let value = v.into_value(span);

5
crates/nu-command/src/debug/explain.rs
View File

@ -73,8 +73,9 @@ pub fn get_pipeline_elements(
let mut i = 0; let mut i = 0;
while i < pipeline.elements.len() { while i < pipeline.elements.len() {
let pipeline_element = &pipeline.elements[i]; let pipeline_element = &pipeline.elements[i];
let pipeline_expression = pipeline_element.expression().clone(); let pipeline_expression = &pipeline_element.expr;
let pipeline_span = pipeline_element.span(); let pipeline_span = pipeline_element.expr.span;
let element_str = let element_str =
String::from_utf8_lossy(engine_state.get_span_contents(pipeline_span)); String::from_utf8_lossy(engine_state.get_span_contents(pipeline_span));
let value = Value::string(element_str.to_string(), pipeline_span); let value = Value::string(element_str.to_string(), pipeline_span);

2
crates/nu-command/src/debug/profile.rs
View File

@ -133,8 +133,6 @@ confusing the id/parent_id hierarchy. The --expr flag is helpful for investigati
&mut callee_stack, &mut callee_stack,
block, block,
input, input,
call.redirect_stdout,
call.redirect_stdout,
); );
// TODO: See eval_source() // TODO: See eval_source()

20
crates/nu-command/src/debug/timeit.rs
View File

@ -55,25 +55,11 @@ impl Command for TimeIt {
if let Some(block_id) = command_to_run.as_block() { if let Some(block_id) = command_to_run.as_block() {
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
eval_block( eval_block(engine_state, stack, block, input)?
engine_state,
stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)?
} else { } else {
let eval_expression_with_input = get_eval_expression_with_input(engine_state); let eval_expression_with_input = get_eval_expression_with_input(engine_state);
eval_expression_with_input( eval_expression_with_input(engine_state, stack, command_to_run, input)
engine_state, .map(|res| res.0)?
stack,
command_to_run,
input,
call.redirect_stdout,
call.redirect_stderr,
)
.map(|res| res.0)?
} }
} else { } else {
PipelineData::empty() PipelineData::empty()

8
crates/nu-command/src/env/config/utils.rs vendored
View File

@ -1,7 +1,7 @@
use std::collections::HashMap; use std::collections::HashMap;
use std::path::PathBuf; use std::path::PathBuf;
use nu_protocol::{Span, Spanned}; use nu_protocol::{IoStream, Span, Spanned};
use crate::ExternalCommand; use crate::ExternalCommand;
@ -32,10 +32,8 @@ pub(crate) fn gen_command(
name, name,
args, args,
arg_keep_raw: vec![false; number_of_args], arg_keep_raw: vec![false; number_of_args],
redirect_stdout: false, out: IoStream::Inherit,
redirect_stderr: false, err: IoStream::Inherit,
redirect_combine: false,
env_vars: env_vars_str, env_vars: env_vars_str,
trim_end_newline: false,
} }
} }

13
crates/nu-command/src/env/export_env.rs vendored
View File

@ -38,18 +38,13 @@ impl Command for ExportEnv {
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
let capture_block: Closure = call.req(engine_state, caller_stack, 0)?; let capture_block: Closure = call.req(engine_state, caller_stack, 0)?;
let block = engine_state.get_block(capture_block.block_id); let block = engine_state.get_block(capture_block.block_id);
let mut callee_stack = caller_stack.captures_to_stack(capture_block.captures); let mut callee_stack = caller_stack
.captures_to_stack(capture_block.captures)
.reset_pipes();
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let _ = eval_block( let _ = eval_block(engine_state, &mut callee_stack, block, input);
engine_state,
&mut callee_stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
);
redirect_env(engine_state, caller_stack, &callee_stack); redirect_env(engine_state, caller_stack, &callee_stack);

13
crates/nu-command/src/env/source_env.rs vendored
View File

@ -76,18 +76,13 @@ impl Command for SourceEnv {
// Evaluate the block // Evaluate the block
let block = engine_state.get_block(block_id as usize).clone(); let block = engine_state.get_block(block_id as usize).clone();
let mut callee_stack = caller_stack.gather_captures(engine_state, &block.captures); let mut callee_stack = caller_stack
.gather_captures(engine_state, &block.captures)
.reset_pipes();
let eval_block_with_early_return = get_eval_block_with_early_return(engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(engine_state);
let result = eval_block_with_early_return( let result = eval_block_with_early_return(engine_state, &mut callee_stack, &block, input);
engine_state,
&mut callee_stack,
&block,
input,
call.redirect_stdout,
call.redirect_stderr,
);
// Merge the block's environment to the current stack // Merge the block's environment to the current stack
redirect_env(engine_state, caller_stack, &callee_stack); redirect_env(engine_state, caller_stack, &callee_stack);

12
crates/nu-command/src/env/with_env.rs vendored
View File

@ -81,12 +81,11 @@ fn with_env(
call: &Call, call: &Call,
input: PipelineData, input: PipelineData,
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
// let external_redirection = args.call_info.args.external_redirection;
let variable: Value = call.req(engine_state, stack, 0)?; let variable: Value = call.req(engine_state, stack, 0)?;
let capture_block: Closure = call.req(engine_state, stack, 1)?; let capture_block: Closure = call.req(engine_state, stack, 1)?;
let block = engine_state.get_block(capture_block.block_id); let block = engine_state.get_block(capture_block.block_id);
let mut stack = stack.captures_to_stack(capture_block.captures); let mut stack = stack.captures_to_stack_preserve_stdio(capture_block.captures);
let mut env: HashMap<String, Value> = HashMap::new(); let mut env: HashMap<String, Value> = HashMap::new();
@ -145,14 +144,7 @@ fn with_env(
stack.add_env_var(k, v); stack.add_env_var(k, v);
} }
eval_block::<WithoutDebug>( eval_block::<WithoutDebug>(engine_state, &mut stack, block, input)
engine_state,
&mut stack,
block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
} }
#[cfg(test)] #[cfg(test)]

2
crates/nu-command/src/filesystem/open.rs
View File

@ -194,7 +194,7 @@ impl Command for Open {
let decl = engine_state.get_decl(converter_id); let decl = engine_state.get_decl(converter_id);
let command_output = if let Some(block_id) = decl.get_block_id() { let command_output = if let Some(block_id) = decl.get_block_id() {
let block = engine_state.get_block(block_id); let block = engine_state.get_block(block_id);
eval_block(engine_state, stack, block, file_contents, false, false) eval_block(engine_state, stack, block, file_contents)
} else { } else {
decl.run(engine_state, stack, &Call::new(call_span), file_contents) decl.run(engine_state, stack, &Call::new(call_span), file_contents)
}; };

165
crates/nu-command/src/filesystem/save.rs
View File

@ -5,7 +5,7 @@ use nu_protocol::ast::{Call, Expr, Expression};
use nu_protocol::engine::{Command, EngineState, Stack}; use nu_protocol::engine::{Command, EngineState, Stack};
use nu_protocol::IntoSpanned; use nu_protocol::IntoSpanned;
use nu_protocol::{ use nu_protocol::{
Category, DataSource, Example, PipelineData, PipelineMetadata, RawStream, ShellError, Category, DataSource, Example, IoStream, PipelineData, PipelineMetadata, RawStream, ShellError,
Signature, Span, Spanned, SyntaxShape, Type, Value, Signature, Span, Spanned, SyntaxShape, Type, Value,
}; };
use std::fs::File; use std::fs::File;
@ -104,16 +104,7 @@ impl Command for Save {
}); });
match input { match input {
PipelineData::ExternalStream { stdout: None, .. } => { PipelineData::ExternalStream { stdout, stderr, .. } => {
// Open files to possibly truncate them
let _ = get_files(&path, stderr_path.as_ref(), append, false, false, force)?;
Ok(PipelineData::empty())
}
PipelineData::ExternalStream {
stdout: Some(stream),
stderr,
..
} => {
let (file, stderr_file) = get_files( let (file, stderr_file) = get_files(
&path, &path,
stderr_path.as_ref(), stderr_path.as_ref(),
@ -123,35 +114,42 @@ impl Command for Save {
force, force,
)?; )?;
// delegate a thread to redirect stderr to result. match (stdout, stderr) {
let handler = stderr (Some(stdout), stderr) => {
.map(|stderr_stream| match stderr_file { // delegate a thread to redirect stderr to result.
Some(stderr_file) => thread::Builder::new() let handler = stderr
.name("stderr redirector".to_string()) .map(|stderr| match stderr_file {
.spawn(move || { Some(stderr_file) => thread::Builder::new()
stream_to_file(stderr_stream, stderr_file, span, progress) .name("stderr redirector".to_string())
}), .spawn(move || {
None => thread::Builder::new() stream_to_file(stderr, stderr_file, span, progress)
.name("stderr redirector".to_string()) }),
.spawn(move || { None => thread::Builder::new()
let _ = stderr_stream.into_bytes(); .name("stderr redirector".to_string())
Ok(PipelineData::empty()) .spawn(move || stderr.drain()),
}), })
}) .transpose()
.transpose() .map_err(|e| e.into_spanned(span))?;
.map_err(|e| e.into_spanned(span))?;
let res = stream_to_file(stream, file, span, progress); let res = stream_to_file(stdout, file, span, progress);
if let Some(h) = handler { if let Some(h) = handler {
h.join().map_err(|err| ShellError::ExternalCommand { h.join().map_err(|err| ShellError::ExternalCommand {
label: "Fail to receive external commands stderr message".to_string(), label: "Fail to receive external commands stderr message"
help: format!("{err:?}"), .to_string(),
span, help: format!("{err:?}"),
})??; span,
res })??;
} else { }
res res?;
} }
(None, Some(stderr)) => match stderr_file {
Some(stderr_file) => stream_to_file(stderr, stderr_file, span, progress)?,
None => stderr.drain()?,
},
(None, None) => {}
};
Ok(PipelineData::Empty)
} }
PipelineData::ListStream(ls, pipeline_metadata) PipelineData::ListStream(ls, pipeline_metadata)
if raw || prepare_path(&path, append, force)?.0.extension().is_none() => if raw || prepare_path(&path, append, force)?.0.extension().is_none() =>
@ -265,6 +263,10 @@ impl Command for Save {
}, },
] ]
} }
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Capture), Some(IoStream::Capture))
}
} }
/// Convert [`PipelineData`] bytes to write in file, possibly converting /// Convert [`PipelineData`] bytes to write in file, possibly converting
@ -430,13 +432,13 @@ fn get_files(
fn stream_to_file( fn stream_to_file(
mut stream: RawStream, mut stream: RawStream,
file: File, mut file: File,
span: Span, span: Span,
progress: bool, progress: bool,
) -> Result<PipelineData, ShellError> { ) -> Result<(), ShellError> {
// https://github.com/nushell/nushell/pull/9377 contains the reason // https://github.com/nushell/nushell/pull/9377 contains the reason
// for not using BufWriter<File> // for not using BufWriter<File>
let mut writer = file; let writer = &mut file;
let mut bytes_processed: u64 = 0; let mut bytes_processed: u64 = 0;
let bytes_processed_p = &mut bytes_processed; let bytes_processed_p = &mut bytes_processed;
@ -456,47 +458,45 @@ fn stream_to_file(
(None, None) (None, None)
}; };
let result = stream stream.try_for_each(move |result| {
.try_for_each(move |result| { let buf = match result {
let buf = match result { Ok(v) => match v {
Ok(v) => match v { Value::String { val, .. } => val.into_bytes(),
Value::String { val, .. } => val.into_bytes(), Value::Binary { val, .. } => val,
Value::Binary { val, .. } => val, // Propagate errors by explicitly matching them before the final case.
// Propagate errors by explicitly matching them before the final case. Value::Error { error, .. } => return Err(*error),
Value::Error { error, .. } => return Err(*error), other => {
other => { return Err(ShellError::OnlySupportsThisInputType {
return Err(ShellError::OnlySupportsThisInputType { exp_input_type: "string or binary".into(),
exp_input_type: "string or binary".into(), wrong_type: other.get_type().to_string(),
wrong_type: other.get_type().to_string(), dst_span: span,
dst_span: span, src_span: other.span(),
src_span: other.span(), });
});
}
},
Err(err) => {
*process_failed_p = true;
return Err(err);
} }
}; },
Err(err) => {
// If the `progress` flag is set then
if progress {
// Update the total amount of bytes that has been saved and then print the progress bar
*bytes_processed_p += buf.len() as u64;
if let Some(bar) = &mut bar_opt {
bar.update_bar(*bytes_processed_p);
}
}
if let Err(err) = writer.write(&buf) {
*process_failed_p = true; *process_failed_p = true;
return Err(ShellError::IOError { return Err(err);
msg: err.to_string(),
});
} }
Ok(()) };
})
.map(|_| PipelineData::empty()); // If the `progress` flag is set then
if progress {
// Update the total amount of bytes that has been saved and then print the progress bar
*bytes_processed_p += buf.len() as u64;
if let Some(bar) = &mut bar_opt {
bar.update_bar(*bytes_processed_p);
}
}
if let Err(err) = writer.write_all(&buf) {
*process_failed_p = true;
return Err(ShellError::IOError {
msg: err.to_string(),
});
}
Ok(())
})?;
// If the `progress` flag is set then // If the `progress` flag is set then
if progress { if progress {
@ -508,6 +508,7 @@ fn stream_to_file(
} }
} }
// And finally return the stream result. file.flush()?;
result
Ok(())
} }

2
crates/nu-command/src/filesystem/watch.rs
View File

@ -217,8 +217,6 @@ impl Command for Watch {
stack, stack,
&block, &block,
Value::nothing(call.span()).into_pipeline_data(), Value::nothing(call.span()).into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
); );
match eval_result { match eval_result {

10
crates/nu-command/src/filters/each.rs
View File

@ -129,8 +129,6 @@ with 'transpose' first."#
let orig_env_vars = stack.env_vars.clone(); let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone(); let orig_env_hidden = stack.env_hidden.clone();
let span = call.head; let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
match input { match input {
PipelineData::Empty => Ok(PipelineData::Empty), PipelineData::Empty => Ok(PipelineData::Empty),
@ -157,10 +155,6 @@ with 'transpose' first."#
&mut stack, &mut stack,
&block, &block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
// WithoutDebug,
// &None,
) { ) {
Ok(v) => Some(v.into_value(span)), Ok(v) => Some(v.into_value(span)),
Err(ShellError::Continue { span }) => Some(Value::nothing(span)), Err(ShellError::Continue { span }) => Some(Value::nothing(span)),
@ -205,8 +199,6 @@ with 'transpose' first."#
&mut stack, &mut stack,
&block, &block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => Some(v.into_value(span)), Ok(v) => Some(v.into_value(span)),
Err(ShellError::Continue { span }) => Some(Value::nothing(span)), Err(ShellError::Continue { span }) => Some(Value::nothing(span)),
@ -232,8 +224,6 @@ with 'transpose' first."#
&mut stack, &mut stack,
&block, &block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) )
} }
} }

8
crates/nu-command/src/filters/filter.rs
View File

@ -62,8 +62,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
let orig_env_vars = stack.env_vars.clone(); let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone(); let orig_env_hidden = stack.env_hidden.clone();
let span = call.head; let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state); let eval_block = get_eval_block(&engine_state);
match input { match input {
@ -92,8 +90,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
&block, &block,
// clone() is used here because x is given to Ok() below. // clone() is used here because x is given to Ok() below.
x.clone().into_pipeline_data(), x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => { Ok(v) => {
if v.into_value(span).is_true() { if v.into_value(span).is_true() {
@ -136,8 +132,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
&block, &block,
// clone() is used here because x is given to Ok() below. // clone() is used here because x is given to Ok() below.
x.clone().into_pipeline_data(), x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => { Ok(v) => {
if v.into_value(span).is_true() { if v.into_value(span).is_true() {
@ -171,8 +165,6 @@ a variable. On the other hand, the "row condition" syntax is not supported."#
&block, &block,
// clone() is used here because x is given to Ok() below. // clone() is used here because x is given to Ok() below.
x.clone().into_pipeline_data(), x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => { Ok(v) => {
if v.into_value(span).is_true() { if v.into_value(span).is_true() {

5
crates/nu-command/src/filters/group_by.rs
View File

@ -171,7 +171,7 @@ pub fn group_by(
Value::CellPath { val, .. } => group_cell_path(val, values)?, Value::CellPath { val, .. } => group_cell_path(val, values)?,
Value::Block { .. } | Value::Closure { .. } => { Value::Block { .. } | Value::Closure { .. } => {
let block: Option<Closure> = call.opt(engine_state, stack, 0)?; let block: Option<Closure> = call.opt(engine_state, stack, 0)?;
group_closure(values, span, block, stack, engine_state, call)? group_closure(values, span, block, stack, engine_state)?
} }
_ => { _ => {
@ -234,7 +234,6 @@ fn group_closure(
block: Option<Closure>, block: Option<Closure>,
stack: &mut Stack, stack: &mut Stack,
engine_state: &EngineState, engine_state: &EngineState,
call: &Call,
) -> Result<IndexMap<String, Vec<Value>>, ShellError> { ) -> Result<IndexMap<String, Vec<Value>>, ShellError> {
let error_key = "error"; let error_key = "error";
let mut groups: IndexMap<String, Vec<Value>> = IndexMap::new(); let mut groups: IndexMap<String, Vec<Value>> = IndexMap::new();
@ -251,8 +250,6 @@ fn group_closure(
&mut stack, &mut stack,
block, block,
value.clone().into_pipeline_data(), value.clone().into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
); );
let group_key = match pipeline { let group_key = match pipeline {

28
crates/nu-command/src/filters/insert.rs
View File

@ -130,9 +130,6 @@ fn insert(
let cell_path: CellPath = call.req(engine_state, stack, 0)?; let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let replacement: Value = call.req(engine_state, stack, 1)?; let replacement: Value = call.req(engine_state, stack, 1)?;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
@ -153,8 +150,6 @@ fn insert(
span, span,
engine_state, engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
block, block,
&cell_path.members, &cell_path.members,
false, false,
@ -168,8 +163,6 @@ fn insert(
replacement, replacement,
engine_state, engine_state,
stack, stack,
redirect_stdout,
redirect_stderr,
&cell_path.members, &cell_path.members,
matches!(first, Some(PathMember::Int { .. })), matches!(first, Some(PathMember::Int { .. })),
eval_block, eval_block,
@ -225,8 +218,6 @@ fn insert(
&mut stack, &mut stack,
block, block,
value.clone().into_pipeline_data(), value.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)?; )?;
pre_elems.push(output.into_value(span)); pre_elems.push(output.into_value(span));
@ -243,8 +234,6 @@ fn insert(
replacement, replacement,
engine_state, engine_state,
stack, stack,
redirect_stdout,
redirect_stderr,
path, path,
true, true,
eval_block, eval_block,
@ -282,8 +271,6 @@ fn insert(
replacement_span, replacement_span,
&engine_state, &engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
&block, &block,
&cell_path.members, &cell_path.members,
false, false,
@ -330,8 +317,6 @@ fn insert_value_by_closure(
span: Span, span: Span,
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
block: &Block, block: &Block,
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
@ -356,14 +341,7 @@ fn insert_value_by_closure(
.map(IntoPipelineData::into_pipeline_data) .map(IntoPipelineData::into_pipeline_data)
.unwrap_or(PipelineData::Empty); .unwrap_or(PipelineData::Empty);
let output = eval_block_fn( let output = eval_block_fn(engine_state, stack, block, input_at_path)?;
engine_state,
stack,
block,
input_at_path,
redirect_stdout,
redirect_stderr,
)?;
value.insert_data_at_cell_path(cell_path, output.into_value(span), span) value.insert_data_at_cell_path(cell_path, output.into_value(span), span)
} }
@ -374,8 +352,6 @@ fn insert_single_value_by_closure(
replacement: Value, replacement: Value,
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
eval_block_fn: EvalBlockFn, eval_block_fn: EvalBlockFn,
@ -390,8 +366,6 @@ fn insert_single_value_by_closure(
span, span,
engine_state, engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
block, block,
cell_path, cell_path,
first_path_member_int, first_path_member_int,

9
crates/nu-command/src/filters/interleave.rs
View File

@ -128,14 +128,7 @@ interleave
// Evaluate the closure on this thread // Evaluate the closure on this thread
let block = engine_state.get_block(closure.block_id); let block = engine_state.get_block(closure.block_id);
let mut stack = stack.captures_to_stack(closure.captures); let mut stack = stack.captures_to_stack(closure.captures);
eval_block_with_early_return( eval_block_with_early_return(engine_state, &mut stack, block, PipelineData::Empty)
engine_state,
&mut stack,
block,
PipelineData::Empty,
true,
false,
)
})) }))
.try_for_each(|stream| { .try_for_each(|stream| {
stream.and_then(|stream| { stream.and_then(|stream| {

3
crates/nu-command/src/filters/items.rs
View File

@ -54,7 +54,6 @@ impl Command for Items {
let orig_env_vars = stack.env_vars.clone(); let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone(); let orig_env_hidden = stack.env_hidden.clone();
let span = call.head; let span = call.head;
let redirect_stderr = call.redirect_stderr;
let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state);
let input_span = input.span().unwrap_or(call.head); let input_span = input.span().unwrap_or(call.head);
@ -81,8 +80,6 @@ impl Command for Items {
&mut stack, &mut stack,
&block, &block,
PipelineData::empty(), PipelineData::empty(),
true,
redirect_stderr,
) { ) {
Ok(v) => Some(v.into_value(span)), Ok(v) => Some(v.into_value(span)),
Err(ShellError::Break { .. }) => None, Err(ShellError::Break { .. }) => None,

12
crates/nu-command/src/filters/par_each.rs
View File

@ -129,8 +129,6 @@ impl Command for ParEach {
let block_id = capture_block.block_id; let block_id = capture_block.block_id;
let mut stack = stack.captures_to_stack(capture_block.captures); let mut stack = stack.captures_to_stack(capture_block.captures);
let span = call.head; let span = call.head;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
// A helper function sorts the output if needed // A helper function sorts the output if needed
let apply_order = |mut vec: Vec<(usize, Value)>| { let apply_order = |mut vec: Vec<(usize, Value)>| {
@ -173,8 +171,6 @@ impl Command for ParEach {
&mut stack, &mut stack,
block, block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => v.into_value(span), Ok(v) => v.into_value(span),
Err(error) => Value::error( Err(error) => Value::error(
@ -213,8 +209,6 @@ impl Command for ParEach {
&mut stack, &mut stack,
block, block,
x.clone().into_pipeline_data(), x.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => v.into_value(span), Ok(v) => v.into_value(span),
Err(error) => Value::error( Err(error) => Value::error(
@ -252,8 +246,6 @@ impl Command for ParEach {
&mut stack, &mut stack,
block, block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => v.into_value(span), Ok(v) => v.into_value(span),
Err(error) => Value::error( Err(error) => Value::error(
@ -297,8 +289,6 @@ impl Command for ParEach {
&mut stack, &mut stack,
block, block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
Ok(v) => v.into_value(span), Ok(v) => v.into_value(span),
Err(error) => Value::error(error, span), Err(error) => Value::error(error, span),
@ -326,8 +316,6 @@ impl Command for ParEach {
&mut stack, &mut stack,
block, block,
x.into_pipeline_data(), x.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) )
} }
} }

10
crates/nu-command/src/filters/reduce.rs
View File

@ -107,9 +107,6 @@ impl Command for Reduce {
let orig_env_vars = stack.env_vars.clone(); let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone(); let orig_env_hidden = stack.env_hidden.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
// To enumerate over the input (for the index argument), // To enumerate over the input (for the index argument),
// it must be converted into an iterator using into_iter(). // it must be converted into an iterator using into_iter().
let mut input_iter = input.into_iter(); let mut input_iter = input.into_iter();
@ -130,9 +127,7 @@ impl Command for Reduce {
let mut acc = start_val; let mut acc = start_val;
let mut input_iter = input_iter.peekable(); for x in input_iter {
while let Some(x) = input_iter.next() {
// with_env() is used here to ensure that each iteration uses // with_env() is used here to ensure that each iteration uses
// a different set of environment variables. // a different set of environment variables.
// Hence, a 'cd' in the first loop won't affect the next loop. // Hence, a 'cd' in the first loop won't affect the next loop.
@ -157,9 +152,6 @@ impl Command for Reduce {
&mut stack, &mut stack,
block, block,
PipelineData::empty(), PipelineData::empty(),
// redirect stdout until its the last input value
redirect_stdout || input_iter.peek().is_some(),
redirect_stderr,
)? )?
.into_value(span); .into_value(span);

4
crates/nu-command/src/filters/rename.rs
View File

@ -140,8 +140,6 @@ fn rename(
} }
None => None, None => None,
}; };
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let block_info = let block_info =
if let Some(capture_block) = call.get_flag::<Closure>(engine_state, stack, "block")? { if let Some(capture_block) = call.get_flag::<Closure>(engine_state, stack, "block")? {
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
@ -185,8 +183,6 @@ fn rename(
&mut stack, &mut stack,
&block, &block,
Value::string(c.clone(), span).into_pipeline_data(), Value::string(c.clone(), span).into_pipeline_data(),
redirect_stdout,
redirect_stderr,
); );
match eval_result { match eval_result {

18
crates/nu-command/src/filters/skip/skip_until.rs
View File

@ -92,9 +92,6 @@ impl Command for SkipUntil {
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state); let eval_block = get_eval_block(&engine_state);
Ok(input Ok(input
@ -104,17 +101,10 @@ impl Command for SkipUntil {
stack.add_var(var_id, value.clone()); stack.add_var(var_id, value.clone());
} }
!eval_block( !eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
&engine_state, .map_or(false, |pipeline_data| {
&mut stack, pipeline_data.into_value(span).is_true()
&block, })
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})
}) })
.into_pipeline_data_with_metadata(metadata, ctrlc)) .into_pipeline_data_with_metadata(metadata, ctrlc))
} }

18
crates/nu-command/src/filters/skip/skip_while.rs
View File

@ -97,9 +97,6 @@ impl Command for SkipWhile {
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state); let eval_block = get_eval_block(&engine_state);
Ok(input Ok(input
@ -109,17 +106,10 @@ impl Command for SkipWhile {
stack.add_var(var_id, value.clone()); stack.add_var(var_id, value.clone());
} }
eval_block( eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
&engine_state, .map_or(false, |pipeline_data| {
&mut stack, pipeline_data.into_value(span).is_true()
&block, })
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})
}) })
.into_pipeline_data_with_metadata(metadata, ctrlc)) .into_pipeline_data_with_metadata(metadata, ctrlc))
} }

18
crates/nu-command/src/filters/take/take_until.rs
View File

@ -89,9 +89,6 @@ impl Command for TakeUntil {
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state); let eval_block = get_eval_block(&engine_state);
Ok(input Ok(input
@ -101,17 +98,10 @@ impl Command for TakeUntil {
stack.add_var(var_id, value.clone()); stack.add_var(var_id, value.clone());
} }
!eval_block( !eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
&engine_state, .map_or(false, |pipeline_data| {
&mut stack, pipeline_data.into_value(span).is_true()
&block, })
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})
}) })
.into_pipeline_data_with_metadata(metadata, ctrlc)) .into_pipeline_data_with_metadata(metadata, ctrlc))
} }

18
crates/nu-command/src/filters/take/take_while.rs
View File

@ -88,9 +88,6 @@ impl Command for TakeWhile {
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state); let eval_block = get_eval_block(&engine_state);
Ok(input Ok(input
@ -100,17 +97,10 @@ impl Command for TakeWhile {
stack.add_var(var_id, value.clone()); stack.add_var(var_id, value.clone());
} }
eval_block( eval_block(&engine_state, &mut stack, &block, PipelineData::empty())
&engine_state, .map_or(false, |pipeline_data| {
&mut stack, pipeline_data.into_value(span).is_true()
&block, })
PipelineData::empty(),
redirect_stdout,
redirect_stderr,
)
.map_or(false, |pipeline_data| {
pipeline_data.into_value(span).is_true()
})
}) })
.into_pipeline_data_with_metadata(metadata, ctrlc)) .into_pipeline_data_with_metadata(metadata, ctrlc))
} }

70
crates/nu-command/src/filters/tee.rs
View File

@ -4,8 +4,8 @@ use nu_engine::{get_eval_block_with_early_return, CallExt};
use nu_protocol::{ use nu_protocol::{
ast::Call, ast::Call,
engine::{Closure, Command, EngineState, Stack}, engine::{Closure, Command, EngineState, Stack},
Category, Example, IntoInterruptiblePipelineData, IntoSpanned, PipelineData, RawStream, Category, Example, IntoInterruptiblePipelineData, IntoSpanned, IoStream, PipelineData,
ShellError, Signature, Spanned, SyntaxShape, Type, Value, RawStream, ShellError, Signature, Spanned, SyntaxShape, Type, Value,
}; };
#[derive(Clone)] #[derive(Clone)]
@ -49,8 +49,8 @@ use it in your pipeline."#
result: None, result: None,
}, },
Example { Example {
example: "do { nu --commands 'print -e error; print ok' } | \ example:
tee --stderr { save error.log } | complete", "nu -c 'print -e error; print ok' | tee --stderr { save error.log } | complete",
description: "Save error messages from an external command to a file without \ description: "Save error messages from an external command to a file without \
redirecting them", redirecting them",
result: None, result: None,
@ -78,7 +78,9 @@ use it in your pipeline."#
} = call.req(engine_state, stack, 0)?; } = call.req(engine_state, stack, 0)?;
let closure_engine_state = engine_state.clone(); let closure_engine_state = engine_state.clone();
let mut closure_stack = stack.captures_to_stack(captures); let mut closure_stack = stack
.captures_to_stack_preserve_stdio(captures)
.reset_pipes();
let metadata = input.metadata(); let metadata = input.metadata();
let metadata_clone = metadata.clone(); let metadata_clone = metadata.clone();
@ -121,46 +123,32 @@ use it in your pipeline."#
&mut closure_stack, &mut closure_stack,
closure_engine_state.get_block(block_id), closure_engine_state.get_block(block_id),
input_from_channel, input_from_channel,
false,
false,
); );
// Make sure to drain any iterator produced to avoid unexpected behavior // Make sure to drain any iterator produced to avoid unexpected behavior
result.and_then(|data| data.drain()) result.and_then(|data| data.drain())
}; };
if use_stderr { if use_stderr {
if let Some(stderr) = stderr { let stderr = stderr
let iter = tee(stderr.stream, with_stream) .map(|stderr| {
.map_err(|e| e.into_spanned(call.head))?; let iter = tee(stderr.stream, with_stream)
let raw_stream = RawStream::new( .map_err(|e| e.into_spanned(call.head))?;
Box::new(iter.map(flatten_result)), Ok::<_, ShellError>(RawStream::new(
stderr.ctrlc, Box::new(iter.map(flatten_result)),
stderr.span, stderr.ctrlc,
stderr.known_size, stderr.span,
); stderr.known_size,
Ok(PipelineData::ExternalStream { ))
stdout,
stderr: Some(raw_stream),
exit_code,
span,
metadata,
trim_end_newline,
}) })
} else { .transpose()?;
// Throw an error if the stream doesn't have stderr. This is probably the Ok(PipelineData::ExternalStream {
// user's mistake (e.g., forgetting to use `do`) stdout,
Err(ShellError::GenericError { stderr,
error: "Stream passed to `tee --stderr` does not have stderr".into(), exit_code,
msg: "this stream does not contain stderr".into(), span,
span: Some(span), metadata,
help: Some( trim_end_newline,
"if this is an external command, you probably need to wrap \ })
it in `do { ... }`"
.into(),
),
inner: vec![],
})
}
} else { } else {
let stdout = stdout let stdout = stdout
.map(|stdout| { .map(|stdout| {
@ -203,8 +191,6 @@ use it in your pipeline."#
&mut closure_stack, &mut closure_stack,
closure_engine_state.get_block(block_id), closure_engine_state.get_block(block_id),
input_from_channel, input_from_channel,
false,
false,
); );
// Make sure to drain any iterator produced to avoid unexpected behavior // Make sure to drain any iterator produced to avoid unexpected behavior
result.and_then(|data| data.drain()) result.and_then(|data| data.drain())
@ -217,6 +203,10 @@ use it in your pipeline."#
} }
} }
} }
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(Some(IoStream::Capture), Some(IoStream::Capture))
}
} }
fn panic_error() -> ShellError { fn panic_error() -> ShellError {

19
crates/nu-command/src/filters/update.rs
View File

@ -112,9 +112,6 @@ fn update(
let cell_path: CellPath = call.req(engine_state, stack, 0)?; let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let replacement: Value = call.req(engine_state, stack, 1)?; let replacement: Value = call.req(engine_state, stack, 1)?;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
@ -135,8 +132,6 @@ fn update(
span, span,
engine_state, engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
block, block,
&cell_path.members, &cell_path.members,
false, false,
@ -150,8 +145,6 @@ fn update(
replacement, replacement,
engine_state, engine_state,
stack, stack,
redirect_stdout,
redirect_stderr,
&cell_path.members, &cell_path.members,
matches!(first, Some(PathMember::Int { .. })), matches!(first, Some(PathMember::Int { .. })),
eval_block, eval_block,
@ -197,8 +190,6 @@ fn update(
replacement, replacement,
engine_state, engine_state,
stack, stack,
redirect_stdout,
redirect_stderr,
path, path,
true, true,
eval_block, eval_block,
@ -229,8 +220,6 @@ fn update(
replacement_span, replacement_span,
&engine_state, &engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
&block, &block,
&cell_path.members, &cell_path.members,
false, false,
@ -275,8 +264,6 @@ fn update_value_by_closure(
span: Span, span: Span,
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
block: &Block, block: &Block,
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
@ -302,8 +289,6 @@ fn update_value_by_closure(
stack, stack,
block, block,
input_at_path.into_pipeline_data(), input_at_path.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)?; )?;
value.update_data_at_cell_path(cell_path, output.into_value(span)) value.update_data_at_cell_path(cell_path, output.into_value(span))
@ -315,8 +300,6 @@ fn update_single_value_by_closure(
replacement: Value, replacement: Value,
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
eval_block_fn: EvalBlockFn, eval_block_fn: EvalBlockFn,
@ -331,8 +314,6 @@ fn update_single_value_by_closure(
span, span,
engine_state, engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
block, block,
cell_path, cell_path,
first_path_member_int, first_path_member_int,

28
crates/nu-command/src/filters/upsert.rs
View File

@ -156,9 +156,6 @@ fn upsert(
let cell_path: CellPath = call.req(engine_state, stack, 0)?; let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let replacement: Value = call.req(engine_state, stack, 1)?; let replacement: Value = call.req(engine_state, stack, 1)?;
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(engine_state); let eval_block = get_eval_block(engine_state);
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
@ -179,8 +176,6 @@ fn upsert(
span, span,
engine_state, engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
block, block,
&cell_path.members, &cell_path.members,
false, false,
@ -194,8 +189,6 @@ fn upsert(
replacement, replacement,
engine_state, engine_state,
stack, stack,
redirect_stdout,
redirect_stderr,
&cell_path.members, &cell_path.members,
matches!(first, Some(PathMember::Int { .. })), matches!(first, Some(PathMember::Int { .. })),
eval_block, eval_block,
@ -249,8 +242,6 @@ fn upsert(
&mut stack, &mut stack,
block, block,
value.clone().into_pipeline_data(), value.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
)?; )?;
pre_elems.push(output.into_value(span)); pre_elems.push(output.into_value(span));
@ -264,8 +255,6 @@ fn upsert(
replacement, replacement,
engine_state, engine_state,
stack, stack,
redirect_stdout,
redirect_stderr,
path, path,
true, true,
eval_block, eval_block,
@ -303,8 +292,6 @@ fn upsert(
replacement_span, replacement_span,
&engine_state, &engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
&block, &block,
&cell_path.members, &cell_path.members,
false, false,
@ -349,8 +336,6 @@ fn upsert_value_by_closure(
span: Span, span: Span,
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
block: &Block, block: &Block,
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
@ -375,14 +360,7 @@ fn upsert_value_by_closure(
.map(IntoPipelineData::into_pipeline_data) .map(IntoPipelineData::into_pipeline_data)
.unwrap_or(PipelineData::Empty); .unwrap_or(PipelineData::Empty);
let output = eval_block_fn( let output = eval_block_fn(engine_state, stack, block, input_at_path)?;
engine_state,
stack,
block,
input_at_path,
redirect_stdout,
redirect_stderr,
)?;
value.upsert_data_at_cell_path(cell_path, output.into_value(span)) value.upsert_data_at_cell_path(cell_path, output.into_value(span))
} }
@ -393,8 +371,6 @@ fn upsert_single_value_by_closure(
replacement: Value, replacement: Value,
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
redirect_stdout: bool,
redirect_stderr: bool,
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
eval_block_fn: EvalBlockFn, eval_block_fn: EvalBlockFn,
@ -409,8 +385,6 @@ fn upsert_single_value_by_closure(
span, span,
engine_state, engine_state,
&mut stack, &mut stack,
redirect_stdout,
redirect_stderr,
block, block,
cell_path, cell_path,
first_path_member_int, first_path_member_int,

9
crates/nu-command/src/filters/utils.rs
View File

@ -53,14 +53,7 @@ pub fn boolean_fold(
stack.add_var(var_id, value.clone()); stack.add_var(var_id, value.clone());
} }
let eval = eval_block( let eval = eval_block(engine_state, &mut stack, block, value.into_pipeline_data());
engine_state,
&mut stack,
block,
value.into_pipeline_data(),
call.redirect_stdout,
call.redirect_stderr,
);
match eval { match eval {
Err(e) => { Err(e) => {
return Err(e); return Err(e);

5
crates/nu-command/src/filters/where_.rs
View File

@ -69,9 +69,6 @@ not supported."#
let ctrlc = engine_state.ctrlc.clone(); let ctrlc = engine_state.ctrlc.clone();
let engine_state = engine_state.clone(); let engine_state = engine_state.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block = get_eval_block(&engine_state); let eval_block = get_eval_block(&engine_state);
Ok(input Ok(input
@ -91,8 +88,6 @@ not supported."#
&block, &block,
// clone() is used here because x is given to Ok() below. // clone() is used here because x is given to Ok() below.
value.clone().into_pipeline_data(), value.clone().into_pipeline_data(),
redirect_stdout,
redirect_stderr,
); );
match result { match result {

9
crates/nu-command/src/filters/zip.rs
View File

@ -109,14 +109,7 @@ impl Command for Zip {
Value::Closure { val, .. } => { Value::Closure { val, .. } => {
let block = engine_state.get_block(val.block_id); let block = engine_state.get_block(val.block_id);
let mut stack = stack.captures_to_stack(val.captures); let mut stack = stack.captures_to_stack(val.captures);
eval_block_with_early_return( eval_block_with_early_return(engine_state, &mut stack, block, PipelineData::Empty)?
engine_state,
&mut stack,
block,
PipelineData::Empty,
true,
false,
)?
} }
// If any other value, use it as-is. // If any other value, use it as-is.
val => val.into_pipeline_data(), val => val.into_pipeline_data(),

23
crates/nu-command/src/formats/from/nuon.rs
View File

@ -1,4 +1,4 @@
use nu_protocol::ast::{Call, Expr, Expression, PipelineElement, RecordItem}; use nu_protocol::ast::{Call, Expr, Expression, RecordItem};
use nu_protocol::engine::{Command, EngineState, Stack, StateWorkingSet}; use nu_protocol::engine::{Command, EngineState, Stack, StateWorkingSet};
use nu_protocol::{ use nu_protocol::{
record, Category, Example, IntoPipelineData, PipelineData, Range, Record, ShellError, record, Category, Example, IntoPipelineData, PipelineData, Range, Record, ShellError,
@ -69,7 +69,7 @@ impl Command for FromNuon {
src: string_input, src: string_input,
error: "error when loading".into(), error: "error when loading".into(),
msg: "excess values when loading".into(), msg: "excess values when loading".into(),
span: element.span(), span: element.expr.span,
}], }],
}); });
} else { } else {
@ -109,7 +109,7 @@ impl Command for FromNuon {
src: string_input, src: string_input,
error: "error when loading".into(), error: "error when loading".into(),
msg: "detected a pipeline in nuon file".into(), msg: "detected a pipeline in nuon file".into(),
span: expr.span(), span: expr.expr.span,
}], }],
}); });
} }
@ -122,22 +122,7 @@ impl Command for FromNuon {
ty: Type::Nothing, ty: Type::Nothing,
} }
} else { } else {
match pipeline.elements.remove(0) { pipeline.elements.remove(0).expr
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
}
| PipelineElement::SeparateRedirection {
out: (_, expression, _),
..
} => expression,
}
} }
}; };

4
crates/nu-command/src/generators/generate.rs
View File

@ -106,8 +106,6 @@ used as the next argument to the closure, otherwise generation stops.
let mut stack = stack.captures_to_stack(capture_block.item.captures); let mut stack = stack.captures_to_stack(capture_block.item.captures);
let orig_env_vars = stack.env_vars.clone(); let orig_env_vars = stack.env_vars.clone();
let orig_env_hidden = stack.env_hidden.clone(); let orig_env_hidden = stack.env_hidden.clone();
let redirect_stdout = call.redirect_stdout;
let redirect_stderr = call.redirect_stderr;
let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(&engine_state);
// A type of Option<S> is used to represent state. Invocation // A type of Option<S> is used to represent state. Invocation
@ -135,8 +133,6 @@ used as the next argument to the closure, otherwise generation stops.
&mut stack, &mut stack,
&block, &block,
arg.into_pipeline_data(), arg.into_pipeline_data(),
redirect_stdout,
redirect_stderr,
) { ) {
// no data -> output nothing and stop. // no data -> output nothing and stop.
Ok(PipelineData::Empty) => (None, None), Ok(PipelineData::Empty) => (None, None),

9
crates/nu-command/src/misc/source.rs
View File

@ -51,14 +51,7 @@ impl Command for Source {
let eval_block_with_early_return = get_eval_block_with_early_return(engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(engine_state);
eval_block_with_early_return( eval_block_with_early_return(engine_state, stack, &block, input)
engine_state,
stack,
&block,
input,
call.redirect_stdout,
call.redirect_stderr,
)
} }
fn examples(&self) -> Vec<Example> { fn examples(&self) -> Vec<Example> {

28
crates/nu-command/src/system/complete.rs
View File

@ -1,8 +1,8 @@
use nu_protocol::{ use nu_protocol::{
ast::Call, ast::Call,
engine::{Command, EngineState, Stack}, engine::{Command, EngineState, Stack},
Category, Example, IntoPipelineData, IntoSpanned, PipelineData, Record, ShellError, Signature, Category, Example, IntoPipelineData, IntoSpanned, IoStream, PipelineData, Record, ShellError,
Type, Value, Signature, Type, Value,
}; };
use std::thread; use std::thread;
@ -115,19 +115,15 @@ impl Command for Complete {
} }
fn examples(&self) -> Vec<Example> { fn examples(&self) -> Vec<Example> {
vec![ vec![Example {
Example { description:
description: "Run the external command to completion, capturing stdout, stderr, and exit_code",
"Run the external command to completion, capturing stdout and exit_code", example: "^external arg1 | complete",
example: "^external arg1 | complete", result: None,
result: None, }]
}, }
Example {
description: fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
"Run external command to completion, capturing, stdout, stderr and exit_code", (Some(IoStream::Capture), Some(IoStream::Capture))
example: "do { ^external arg1 } | complete",
result: None,
},
]
} }
} }

7
crates/nu-command/src/system/exec.rs
View File

@ -3,7 +3,7 @@ use nu_engine::current_dir;
use nu_protocol::{ use nu_protocol::{
ast::Call, ast::Call,
engine::{Command, EngineState, Stack}, engine::{Command, EngineState, Stack},
Category, Example, PipelineData, ShellError, Signature, Span, SyntaxShape, Type, Category, Example, IoStream, PipelineData, ShellError, Signature, Span, SyntaxShape, Type,
}; };
#[derive(Clone)] #[derive(Clone)]
@ -62,8 +62,9 @@ fn exec(
stack: &mut Stack, stack: &mut Stack,
call: &Call, call: &Call,
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
let external_command = let mut external_command = create_external_command(engine_state, stack, call)?;
create_external_command(engine_state, stack, call, false, false, false, false)?; external_command.out = IoStream::Inherit;
external_command.err = IoStream::Inherit;
let cwd = current_dir(engine_state, stack)?; let cwd = current_dir(engine_state, stack)?;
let mut command = external_command.spawn_simple_command(&cwd.to_string_lossy())?; let mut command = external_command.spawn_simple_command(&cwd.to_string_lossy())?;

322
crates/nu-command/src/system/run_external.rs
View File

@ -2,14 +2,12 @@ use nu_cmd_base::hook::eval_hook;
use nu_engine::env_to_strings; use nu_engine::env_to_strings;
use nu_engine::get_eval_expression; use nu_engine::get_eval_expression;
use nu_engine::CallExt; use nu_engine::CallExt;
use nu_protocol::IntoSpanned;
use nu_protocol::NuGlob;
use nu_protocol::{ use nu_protocol::{
ast::{Call, Expr}, ast::{Call, Expr},
did_you_mean, did_you_mean,
engine::{Command, EngineState, Stack}, engine::{Command, EngineState, Stack},
Category, Example, ListStream, PipelineData, RawStream, ShellError, Signature, Span, Spanned, Category, Example, IntoSpanned, IoStream, ListStream, NuGlob, PipelineData, RawStream,
SyntaxShape, Type, Value, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value,
}; };
use nu_system::ForegroundChild; use nu_system::ForegroundChild;
use nu_utils::IgnoreCaseExt; use nu_utils::IgnoreCaseExt;
@ -19,14 +17,9 @@ use std::collections::HashMap;
use std::io::{BufRead, BufReader, Read, Write}; use std::io::{BufRead, BufReader, Read, Write};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::process::{Command as CommandSys, Stdio}; use std::process::{Command as CommandSys, Stdio};
use std::sync::atomic::AtomicBool; use std::sync::mpsc;
use std::sync::mpsc::{self, SyncSender};
use std::sync::Arc;
use std::thread; use std::thread;
const OUTPUT_BUFFER_SIZE: usize = 1024;
const OUTPUT_BUFFERS_IN_FLIGHT: usize = 3;
#[derive(Clone)] #[derive(Clone)]
pub struct External; pub struct External;
@ -76,15 +69,61 @@ impl Command for External {
}); });
} }
let command = create_external_command( if trim_end_newline {
engine_state, nu_protocol::report_error_new(
stack, engine_state,
call, &ShellError::GenericError {
redirect_stdout, error: "Deprecated flag".into(),
redirect_stderr, msg: "`--trim-end-newline` is deprecated".into(),
redirect_combine, span: Some(call.arguments_span()),
trim_end_newline, help: Some(
)?; "trailing new lines are now removed by default when collecting into a value"
.into(),
),
inner: vec![],
},
);
}
if redirect_combine {
nu_protocol::report_error_new(
engine_state,
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--redirect-combine` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some("use the `o+e>|` pipe redirection instead".into()),
inner: vec![],
},
);
} else if redirect_stdout {
nu_protocol::report_error_new(
engine_state,
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--redirect-stdout` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some(
"`run-external` will now always redirect stdout if there is a pipe `|` afterwards"
.into(),
),
inner: vec![],
},
);
} else if redirect_stderr {
nu_protocol::report_error_new(
engine_state,
&ShellError::GenericError {
error: "Deprecated flag".into(),
msg: "`--redirect-stderr` is deprecated".into(),
span: Some(call.arguments_span()),
help: Some("use the `e>|` stderr pipe redirection instead".into()),
inner: vec![],
},
);
}
let command = create_external_command(engine_state, stack, call)?;
command.run_with_input(engine_state, stack, input, false) command.run_with_input(engine_state, stack, input, false)
} }
@ -98,7 +137,12 @@ impl Command for External {
}, },
Example { Example {
description: "Redirect stdout from an external command into the pipeline", description: "Redirect stdout from an external command into the pipeline",
example: r#"run-external --redirect-stdout "echo" "-n" "hello" | split chars"#, example: r#"run-external "echo" "-n" "hello" | split chars"#,
result: None,
},
Example {
description: "Redirect stderr from an external command into the pipeline",
example: r#"run-external "nu" "-c" "print -e hello" e>| split chars"#,
result: None, result: None,
}, },
] ]
@ -110,10 +154,6 @@ pub fn create_external_command(
engine_state: &EngineState, engine_state: &EngineState,
stack: &mut Stack, stack: &mut Stack,
call: &Call, call: &Call,
redirect_stdout: bool,
redirect_stderr: bool,
redirect_combine: bool,
trim_end_newline: bool,
) -> Result<ExternalCommand, ShellError> { ) -> Result<ExternalCommand, ShellError> {
let name: Spanned<String> = call.req(engine_state, stack, 0)?; let name: Spanned<String> = call.req(engine_state, stack, 0)?;
@ -180,11 +220,9 @@ pub fn create_external_command(
name, name,
args: spanned_args, args: spanned_args,
arg_keep_raw, arg_keep_raw,
redirect_stdout, out: stack.stdout().clone(),
redirect_stderr, err: stack.stderr().clone(),
redirect_combine,
env_vars: env_vars_str, env_vars: env_vars_str,
trim_end_newline,
}) })
} }
@ -193,11 +231,9 @@ pub struct ExternalCommand {
pub name: Spanned<String>, pub name: Spanned<String>,
pub args: Vec<Spanned<String>>, pub args: Vec<Spanned<String>>,
pub arg_keep_raw: Vec<bool>, pub arg_keep_raw: Vec<bool>,
pub redirect_stdout: bool, pub out: IoStream,
pub redirect_stderr: bool, pub err: IoStream,
pub redirect_combine: bool,
pub env_vars: HashMap<String, String>, pub env_vars: HashMap<String, String>,
pub trim_end_newline: bool,
} }
impl ExternalCommand { impl ExternalCommand {
@ -364,6 +400,7 @@ impl ExternalCommand {
let mut engine_state = engine_state.clone(); let mut engine_state = engine_state.clone();
if let Some(hook) = engine_state.config.hooks.command_not_found.clone() if let Some(hook) = engine_state.config.hooks.command_not_found.clone()
{ {
let stack = &mut stack.start_capture();
if let Ok(PipelineData::Value(Value::String { val, .. }, ..)) = if let Ok(PipelineData::Value(Value::String { val, .. }, ..)) =
eval_hook( eval_hook(
&mut engine_state, &mut engine_state,
@ -412,6 +449,7 @@ impl ExternalCommand {
thread::Builder::new() thread::Builder::new()
.name("external stdin worker".to_string()) .name("external stdin worker".to_string())
.spawn(move || { .spawn(move || {
let stack = &mut stack.start_capture();
// Attempt to render the input as a table before piping it to the external. // Attempt to render the input as a table before piping it to the external.
// This is important for pagers like `less`; // This is important for pagers like `less`;
// they need to get Nu data rendered for display to users. // they need to get Nu data rendered for display to users.
@ -421,7 +459,7 @@ impl ExternalCommand {
let input = crate::Table::run( let input = crate::Table::run(
&crate::Table, &crate::Table,
&engine_state, &engine_state,
&mut stack, stack,
&Call::new(head), &Call::new(head),
input, input,
); );
@ -447,63 +485,66 @@ impl ExternalCommand {
#[cfg(unix)] #[cfg(unix)]
let commandname = self.name.item.clone(); let commandname = self.name.item.clone();
let redirect_stdout = self.redirect_stdout;
let redirect_stderr = self.redirect_stderr;
let redirect_combine = self.redirect_combine;
let span = self.name.span; let span = self.name.span;
let output_ctrlc = ctrlc.clone();
let stderr_ctrlc = ctrlc.clone();
let (stdout_tx, stdout_rx) = mpsc::sync_channel(OUTPUT_BUFFERS_IN_FLIGHT);
let (exit_code_tx, exit_code_rx) = mpsc::channel(); let (exit_code_tx, exit_code_rx) = mpsc::channel();
let stdout = child.as_mut().stdout.take(); let (stdout, stderr) = if let Some(combined) = reader {
let stderr = child.as_mut().stderr.take(); (
Some(RawStream::new(
Box::new(ByteLines::new(combined)),
ctrlc.clone(),
head,
None,
)),
None,
)
} else {
let stdout = child.as_mut().stdout.take().map(|out| {
RawStream::new(Box::new(ByteLines::new(out)), ctrlc.clone(), head, None)
});
// If this external is not the last expression, then its output is piped to a channel let stderr = child.as_mut().stderr.take().map(|err| {
// and we create a ListStream that can be consumed RawStream::new(Box::new(ByteLines::new(err)), ctrlc.clone(), head, None)
});
// First create a thread to redirect the external's stdout and wait for an exit code. if matches!(self.err, IoStream::Pipe) {
(stderr, stdout)
} else {
(stdout, stderr)
}
};
// Create a thread to wait for an exit code.
thread::Builder::new() thread::Builder::new()
.name("stdout redirector + exit code waiter".to_string()) .name("exit code waiter".into())
.spawn(move || { .spawn(move || {
if redirect_stdout { match child.as_mut().wait() {
let stdout = stdout.ok_or_else(|| { Err(err) => Err(ShellError::ExternalCommand {
ShellError::ExternalCommand { label: "Error taking stdout from external".to_string(), help: "Redirects need access to stdout of an external command" label: "External command exited with error".into(),
.to_string(), span } help: err.to_string(),
})?; span
}),
Ok(x) => {
#[cfg(unix)]
{
use nu_ansi_term::{Color, Style};
use std::ffi::CStr;
use std::os::unix::process::ExitStatusExt;
read_and_redirect_message(stdout, stdout_tx, ctrlc) if x.core_dumped() {
} else if redirect_combine { let cause = x.signal().and_then(|sig| unsafe {
let stdout = reader.ok_or_else(|| { // SAFETY: We should be the first to call `char * strsignal(int sig)`
ShellError::ExternalCommand { label: "Error taking combined stdout and stderr from external".to_string(), help: "Combined redirects need access to reader pipe of an external command" let sigstr_ptr = libc::strsignal(sig);
.to_string(), span } if sigstr_ptr.is_null() {
})?; return None;
read_and_redirect_message(stdout, stdout_tx, ctrlc) }
}
match child.as_mut().wait() { // SAFETY: The pointer points to a valid non-null string
Err(err) => Err(ShellError::ExternalCommand { label: "External command exited with error".into(), help: err.to_string(), span }), let sigstr = CStr::from_ptr(sigstr_ptr);
Ok(x) => { sigstr.to_str().map(String::from).ok()
#[cfg(unix)] });
{
use nu_ansi_term::{Color, Style};
use std::ffi::CStr;
use std::os::unix::process::ExitStatusExt;
if x.core_dumped() { let cause = cause.as_deref().unwrap_or("Something went wrong");
let cause = x.signal().and_then(|sig| unsafe {
// SAFETY: We should be the first to call `char * strsignal(int sig)`
let sigstr_ptr = libc::strsignal(sig);
if sigstr_ptr.is_null() {
return None;
}
// SAFETY: The pointer points to a valid non-null string
let sigstr = CStr::from_ptr(sigstr_ptr);
sigstr.to_str().map(String::from).ok()
});
let cause = cause.as_deref().unwrap_or("Something went wrong");
let style = Style::new().bold().on(Color::Red); let style = Style::new().bold().on(Color::Red);
eprintln!( eprintln!(
@ -531,56 +572,18 @@ impl ExternalCommand {
} }
}).map_err(|e| e.into_spanned(head))?; }).map_err(|e| e.into_spanned(head))?;
let (stderr_tx, stderr_rx) = mpsc::sync_channel(OUTPUT_BUFFERS_IN_FLIGHT);
if redirect_stderr {
thread::Builder::new()
.name("stderr redirector".to_string())
.spawn(move || {
let stderr = stderr.ok_or_else(|| ShellError::ExternalCommand {
label: "Error taking stderr from external".to_string(),
help: "Redirects need access to stderr of an external command"
.to_string(),
span,
})?;
read_and_redirect_message(stderr, stderr_tx, stderr_ctrlc);
Ok::<(), ShellError>(())
})
.map_err(|e| e.into_spanned(head))?;
}
let stdout_receiver = ChannelReceiver::new(stdout_rx);
let stderr_receiver = ChannelReceiver::new(stderr_rx);
let exit_code_receiver = ValueReceiver::new(exit_code_rx); let exit_code_receiver = ValueReceiver::new(exit_code_rx);
Ok(PipelineData::ExternalStream { Ok(PipelineData::ExternalStream {
stdout: if redirect_stdout || redirect_combine { stdout,
Some(RawStream::new( stderr,
Box::new(stdout_receiver),
output_ctrlc.clone(),
head,
None,
))
} else {
None
},
stderr: if redirect_stderr {
Some(RawStream::new(
Box::new(stderr_receiver),
output_ctrlc.clone(),
head,
None,
))
} else {
None
},
exit_code: Some(ListStream::from_stream( exit_code: Some(ListStream::from_stream(
Box::new(exit_code_receiver), Box::new(exit_code_receiver),
output_ctrlc, ctrlc.clone(),
)), )),
span: head, span: head,
metadata: None, metadata: None,
trim_end_newline: self.trim_end_newline, trim_end_newline: true,
}) })
} }
} }
@ -621,20 +624,15 @@ impl ExternalCommand {
// If the external is not the last command, its output will get piped // If the external is not the last command, its output will get piped
// either as a string or binary // either as a string or binary
let reader = if self.redirect_combine { let reader = if matches!(self.out, IoStream::Pipe) && matches!(self.err, IoStream::Pipe) {
let (reader, writer) = os_pipe::pipe()?; let (reader, writer) = os_pipe::pipe()?;
let writer_clone = writer.try_clone()?; let writer_clone = writer.try_clone()?;
process.stdout(writer); process.stdout(writer);
process.stderr(writer_clone); process.stderr(writer_clone);
Some(reader) Some(reader)
} else { } else {
if self.redirect_stdout { process.stdout(Stdio::try_from(&self.out)?);
process.stdout(Stdio::piped()); process.stderr(Stdio::try_from(&self.err)?);
}
if self.redirect_stderr {
process.stderr(Stdio::piped());
}
None None
}; };
@ -824,63 +822,27 @@ fn remove_quotes(input: String) -> String {
} }
} }
// read message from given `reader`, and send out through `sender`. struct ByteLines<R: Read>(BufReader<R>);
//
// `ctrlc` is used to control the process, if ctrl-c is pressed, the read and redirect
// process will be breaked.
fn read_and_redirect_message<R>(
reader: R,
sender: SyncSender<Vec<u8>>,
ctrlc: Option<Arc<AtomicBool>>,
) where
R: Read,
{
// read using the BufferReader. It will do so until there is an
// error or there are no more bytes to read
let mut buf_read = BufReader::with_capacity(OUTPUT_BUFFER_SIZE, reader);
while let Ok(bytes) = buf_read.fill_buf() {
if bytes.is_empty() {
break;
}
// The Cow generated from the function represents the conversion impl<R: Read> ByteLines<R> {
// from bytes to String. If no replacements are required, then the fn new(read: R) -> Self {
// borrowed value is a proper UTF-8 string. The Owned option represents Self(BufReader::new(read))
// a string where the values had to be replaced, thus marking it as bytes
let bytes = bytes.to_vec();
let length = bytes.len();
buf_read.consume(length);
if nu_utils::ctrl_c::was_pressed(&ctrlc) {
break;
}
match sender.send(bytes) {
Ok(_) => continue,
Err(_) => break,
}
} }
} }
// Receiver used for the RawStream impl<R: Read> Iterator for ByteLines<R> {
// It implements iterator so it can be used as a RawStream
struct ChannelReceiver {
rx: mpsc::Receiver<Vec<u8>>,
}
impl ChannelReceiver {
pub fn new(rx: mpsc::Receiver<Vec<u8>>) -> Self {
Self { rx }
}
}
impl Iterator for ChannelReceiver {
type Item = Result<Vec<u8>, ShellError>; type Item = Result<Vec<u8>, ShellError>;
fn next(&mut self) -> Option<Self::Item> { fn next(&mut self) -> Option<Self::Item> {
match self.rx.recv() { let mut buf = Vec::new();
Ok(v) => Some(Ok(v)), // `read_until` will never stop reading unless `\n` or EOF is encountered,
Err(_) => None, // so let's limit the number of bytes using `take` as the Rust docs suggest.
let capacity = self.0.capacity() as u64;
let mut reader = (&mut self.0).take(capacity);
match reader.read_until(b'\n', &mut buf) {
Ok(0) => None,
Ok(_) => Some(Ok(buf)),
Err(e) => Some(Err(e.into())),
} }
} }
} }

11
crates/nu-command/src/viewers/table.rs
View File

@ -9,10 +9,10 @@ use nu_engine::{env::get_config, env_to_string, CallExt};
use nu_protocol::{ use nu_protocol::{
ast::Call, ast::Call,
engine::{Command, EngineState, Stack}, engine::{Command, EngineState, Stack},
Category, Config, DataSource, Example, IntoPipelineData, ListStream, PipelineData, record, Category, Config, DataSource, Example, IntoPipelineData, IoStream, ListStream,
PipelineMetadata, RawStream, Record, ShellError, Signature, Span, SyntaxShape, Type, Value, PipelineData, PipelineMetadata, RawStream, Record, ShellError, Signature, Span, SyntaxShape,
TableMode, Type, Value,
}; };
use nu_protocol::{record, TableMode};
use nu_table::common::create_nu_table_config; use nu_table::common::create_nu_table_config;
use nu_table::{ use nu_table::{
CollapsedTable, ExpandedTable, JustTable, NuTable, NuTableCell, StringResult, TableOpts, CollapsedTable, ExpandedTable, JustTable, NuTable, NuTableCell, StringResult, TableOpts,
@ -370,7 +370,10 @@ fn handle_table_command(
match input.data { match input.data {
PipelineData::ExternalStream { .. } => Ok(input.data), PipelineData::ExternalStream { .. } => Ok(input.data),
PipelineData::Value(Value::Binary { val, .. }, ..) => { PipelineData::Value(Value::Binary { val, .. }, ..) => {
let stream_list = if input.call.redirect_stdout { let stream_list = if matches!(
input.stack.stdout(),
IoStream::Pipe | IoStream::Capture | IoStream::Null
) {
vec![Ok(val)] vec![Ok(val)]
} else { } else {
let hex = format!("{}\n", nu_pretty_hex::pretty_hex(&val)) let hex = format!("{}\n", nu_pretty_hex::pretty_hex(&val))

69
crates/nu-command/tests/commands/complete.rs
View File

@ -23,7 +23,72 @@ fn basic_exit_code() {
#[test] #[test]
fn error() { fn error() {
let actual = nu!("do { not-found } | complete"); let actual = nu!("not-found | complete");
assert!(actual.err.contains("executable was not found")); assert!(actual.err.contains("executable was not found"));
} }
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stderr_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup("external with many stderr message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual =
nu!(cwd: dirs.test(), "sh -c 'cat a_large_file.txt 1>&2' | complete | get stderr");
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stdout_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup("external with many stdout message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual = nu!(cwd: dirs.test(), "sh -c 'cat a_large_file.txt' | complete | get stdout");
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_both_stdout_stderr_messages_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup(
"external with many stdout and stderr messages",
|dirs, sandbox| {
let script_body = r#"
x=$(printf '=%.0s' $(seq 40960))
echo $x
echo $x 1>&2
"#;
let expect_body = "=".repeat(40960);
sandbox.with_files(vec![FileWithContent("test.sh", script_body)]);
// check for stdout
let actual = nu!(cwd: dirs.test(), "sh test.sh | complete | get stdout | str trim");
assert_eq!(actual.out, expect_body);
// check for stderr
let actual = nu!(cwd: dirs.test(), "sh test.sh | complete | get stderr | str trim");
assert_eq!(actual.out, expect_body);
},
)
}

85
crates/nu-command/tests/commands/do_.rs
View File

@ -1,6 +1,4 @@
use nu_test_support::nu; use nu_test_support::nu;
#[cfg(not(windows))]
use nu_test_support::pipeline;
#[test] #[test]
fn capture_errors_works() { fn capture_errors_works() {
@ -63,89 +61,6 @@ fn ignore_error_should_work_for_external_command() {
assert_eq!(actual.out, "post"); assert_eq!(actual.out, "post");
} }
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stderr_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::pipeline;
use nu_test_support::playground::Playground;
Playground::setup("external with many stderr message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual = nu!(
cwd: dirs.test(), pipeline(
r#"
do -c {sh -c "cat a_large_file.txt 1>&2"} | complete | get stderr
"#,
));
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_too_much_stdout_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::pipeline;
use nu_test_support::playground::Playground;
Playground::setup("external with many stdout message", |dirs, sandbox| {
let bytes: usize = 81920;
let mut large_file_body = String::with_capacity(bytes);
for _ in 0..bytes {
large_file_body.push('a');
}
sandbox.with_files(vec![FileWithContent("a_large_file.txt", &large_file_body)]);
let actual = nu!(
cwd: dirs.test(), pipeline(
r#"
do -c {sh -c "cat a_large_file.txt"} | complete | get stdout
"#,
));
assert_eq!(actual.out, large_file_body);
})
}
#[test]
#[cfg(not(windows))]
fn capture_error_with_both_stdout_stderr_messages_not_hang_nushell() {
use nu_test_support::fs::Stub::FileWithContent;
use nu_test_support::playground::Playground;
Playground::setup(
"external with many stdout and stderr messages",
|dirs, sandbox| {
let script_body = r#"
x=$(printf '=%.0s' $(seq 40960))
echo $x
echo $x 1>&2
"#;
let expect_body = "=".repeat(40960);
sandbox.with_files(vec![FileWithContent("test.sh", script_body)]);
// check for stdout
let actual = nu!(
cwd: dirs.test(), pipeline(
"do -c {sh test.sh} | complete | get stdout | str trim",
));
assert_eq!(actual.out, expect_body);
// check for stderr
let actual = nu!(
cwd: dirs.test(), pipeline(
"do -c {sh test.sh} | complete | get stderr | str trim",
));
assert_eq!(actual.out, expect_body);
},
)
}
#[test] #[test]
fn ignore_error_works_with_list_stream() { fn ignore_error_works_with_list_stream() {
let actual = nu!(r#"do -i { ["a", null, "b"] | ansi strip }"#); let actual = nu!(r#"do -i { ["a", null, "b"] | ansi strip }"#);

8
crates/nu-command/tests/commands/let_.rs
View File

@ -65,9 +65,7 @@ fn let_err_pipeline_redirects_externals() {
let actual = nu!( let actual = nu!(
r#"let x = with-env [FOO "foo"] {nu --testbin echo_env_stderr FOO e>| str length}; $x"# r#"let x = with-env [FOO "foo"] {nu --testbin echo_env_stderr FOO e>| str length}; $x"#
); );
assert_eq!(actual.out, "3");
// have an extra \n, so length is 4.
assert_eq!(actual.out, "4");
} }
#[test] #[test]
@ -75,9 +73,7 @@ fn let_outerr_pipeline_redirects_externals() {
let actual = nu!( let actual = nu!(
r#"let x = with-env [FOO "foo"] {nu --testbin echo_env_stderr FOO o+e>| str length}; $x"# r#"let x = with-env [FOO "foo"] {nu --testbin echo_env_stderr FOO o+e>| str length}; $x"#
); );
assert_eq!(actual.out, "3");
// have an extra \n, so length is 4.
assert_eq!(actual.out, "4");
} }
#[ignore] #[ignore]

14
crates/nu-command/tests/commands/redirection.rs
View File

@ -164,7 +164,7 @@ fn redirection_keep_exit_codes() {
Playground::setup("redirection preserves exit code", |dirs, _| { Playground::setup("redirection preserves exit code", |dirs, _| {
let out = nu!( let out = nu!(
cwd: dirs.test(), cwd: dirs.test(),
"do -i { nu --testbin fail e> a.txt } | complete | get exit_code" "nu --testbin fail e> a.txt | complete | get exit_code"
); );
// needs to use contains "1", because it complete will output `Some(RawStream)`. // needs to use contains "1", because it complete will output `Some(RawStream)`.
assert!(out.out.contains('1')); assert!(out.out.contains('1'));
@ -358,7 +358,7 @@ fn redirection_with_out_pipe() {
r#"$env.BAZ = "message"; nu --testbin echo_env_mixed out-err BAZ BAZ err> tmp_file | str length"#, r#"$env.BAZ = "message"; nu --testbin echo_env_mixed out-err BAZ BAZ err> tmp_file | str length"#,
); );
assert_eq!(actual.out, "8"); assert_eq!(actual.out, "7");
// check for stderr redirection file. // check for stderr redirection file.
let expected_out_file = dirs.test().join("tmp_file"); let expected_out_file = dirs.test().join("tmp_file");
let actual_len = file_contents(expected_out_file).len(); let actual_len = file_contents(expected_out_file).len();
@ -376,7 +376,7 @@ fn redirection_with_err_pipe() {
r#"$env.BAZ = "message"; nu --testbin echo_env_mixed out-err BAZ BAZ out> tmp_file e>| str length"#, r#"$env.BAZ = "message"; nu --testbin echo_env_mixed out-err BAZ BAZ out> tmp_file e>| str length"#,
); );
assert_eq!(actual.out, "8"); assert_eq!(actual.out, "7");
// check for stdout redirection file. // check for stdout redirection file.
let expected_out_file = dirs.test().join("tmp_file"); let expected_out_file = dirs.test().join("tmp_file");
let actual_len = file_contents(expected_out_file).len(); let actual_len = file_contents(expected_out_file).len();
@ -392,7 +392,7 @@ fn no_redirection_with_outerr_pipe() {
cwd: dirs.test(), cwd: dirs.test(),
&format!("echo 3 {redirect_type} a.txt o+e>| str length") &format!("echo 3 {redirect_type} a.txt o+e>| str length")
); );
assert!(actual.err.contains("not allowed to use with redirection")); assert!(actual.err.contains("Multiple redirections provided"));
assert!( assert!(
!dirs.test().join("a.txt").exists(), !dirs.test().join("a.txt").exists(),
"No file should be created on error" "No file should be created on error"
@ -404,7 +404,7 @@ fn no_redirection_with_outerr_pipe() {
cwd: dirs.test(), cwd: dirs.test(),
"echo 3 o> a.txt e> b.txt o+e>| str length" "echo 3 o> a.txt e> b.txt o+e>| str length"
); );
assert!(actual.err.contains("not allowed to use with redirection")); assert!(actual.err.contains("Multiple redirections provided"));
assert!( assert!(
!dirs.test().join("a.txt").exists(), !dirs.test().join("a.txt").exists(),
"No file should be created on error" "No file should be created on error"
@ -423,7 +423,7 @@ fn no_duplicate_redirection() {
cwd: dirs.test(), cwd: dirs.test(),
"echo 3 o> a.txt o> a.txt" "echo 3 o> a.txt o> a.txt"
); );
assert!(actual.err.contains("Redirection can be set only once")); assert!(actual.err.contains("Multiple redirections provided"));
assert!( assert!(
!dirs.test().join("a.txt").exists(), !dirs.test().join("a.txt").exists(),
"No file should be created on error" "No file should be created on error"
@ -432,7 +432,7 @@ fn no_duplicate_redirection() {
cwd: dirs.test(), cwd: dirs.test(),
"echo 3 e> a.txt e> a.txt" "echo 3 e> a.txt e> a.txt"
); );
assert!(actual.err.contains("Redirection can be set only once")); assert!(actual.err.contains("Multiple redirections provided"));
assert!( assert!(
!dirs.test().join("a.txt").exists(), !dirs.test().join("a.txt").exists(),
"No file should be created on error" "No file should be created on error"

2
crates/nu-command/tests/commands/run_external.rs
View File

@ -325,7 +325,7 @@ fn redirect_combine() {
let actual = nu!( let actual = nu!(
cwd: dirs.test(), pipeline( cwd: dirs.test(), pipeline(
r#" r#"
run-external --redirect-combine sh ...[-c 'echo Foo; echo >&2 Bar'] run-external sh ...[-c 'echo Foo; echo >&2 Bar'] o+e>| print
"# "#
)); ));

6
crates/nu-engine/src/call_ext.rs
View File

@ -70,6 +70,7 @@ impl CallExt for Call {
if flag_name == name.0.item { if flag_name == name.0.item {
return if let Some(expr) = &name.2 { return if let Some(expr) = &name.2 {
// Check --flag=false // Check --flag=false
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?; let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
match result { match result {
Value::Bool { val, .. } => Ok(val), Value::Bool { val, .. } => Ok(val),
@ -96,6 +97,7 @@ impl CallExt for Call {
name: &str, name: &str,
) -> Result<Option<T>, ShellError> { ) -> Result<Option<T>, ShellError> {
if let Some(expr) = self.get_flag_expr(name) { if let Some(expr) = self.get_flag_expr(name) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?; let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result).map(Some) FromValue::from_value(result).map(Some)
} else { } else {
@ -109,6 +111,7 @@ impl CallExt for Call {
stack: &mut Stack, stack: &mut Stack,
starting_pos: usize, starting_pos: usize,
) -> Result<Vec<T>, ShellError> { ) -> Result<Vec<T>, ShellError> {
let stack = &mut stack.use_call_arg_stdio();
let mut output = vec![]; let mut output = vec![];
for result in self.rest_iter_flattened(starting_pos, |expr| { for result in self.rest_iter_flattened(starting_pos, |expr| {
@ -127,6 +130,7 @@ impl CallExt for Call {
pos: usize, pos: usize,
) -> Result<Option<T>, ShellError> { ) -> Result<Option<T>, ShellError> {
if let Some(expr) = self.positional_nth(pos) { if let Some(expr) = self.positional_nth(pos) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?; let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result).map(Some) FromValue::from_value(result).map(Some)
} else { } else {
@ -154,6 +158,7 @@ impl CallExt for Call {
pos: usize, pos: usize,
) -> Result<T, ShellError> { ) -> Result<T, ShellError> {
if let Some(expr) = self.positional_nth(pos) { if let Some(expr) = self.positional_nth(pos) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?; let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result) FromValue::from_value(result)
} else if self.positional_len() == 0 { } else if self.positional_len() == 0 {
@ -173,6 +178,7 @@ impl CallExt for Call {
name: &str, name: &str,
) -> Result<T, ShellError> { ) -> Result<T, ShellError> {
if let Some(expr) = self.get_parser_info(name) { if let Some(expr) = self.get_parser_info(name) {
let stack = &mut stack.use_call_arg_stdio();
let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?; let result = eval_expression::<WithoutDebug>(engine_state, stack, expr)?;
FromValue::from_value(result) FromValue::from_value(result)
} else if self.parser_info.is_empty() { } else if self.parser_info.is_empty() {

15
crates/nu-engine/src/documentation.rs
View File

@ -23,6 +23,9 @@ pub fn get_full_help(
no_color: !config.use_ansi_coloring, no_color: !config.use_ansi_coloring,
brief: false, brief: false,
}; };
let stack = &mut stack.start_capture();
get_documentation( get_documentation(
sig, sig,
examples, examples,
@ -235,16 +238,14 @@ fn get_documentation(
)); ));
} }
let mut caller_stack = Stack::new(); let caller_stack = &mut Stack::new().capture();
if let Ok(result) = eval_call::<WithoutDebug>( if let Ok(result) = eval_call::<WithoutDebug>(
engine_state, engine_state,
&mut caller_stack, caller_stack,
&Call { &Call {
decl_id, decl_id,
head: span, head: span,
arguments: vec![], arguments: vec![],
redirect_stdout: true,
redirect_stderr: true,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}, },
PipelineData::Value(Value::list(vals, span), None), PipelineData::Value(Value::list(vals, span), None),
@ -340,7 +341,7 @@ fn get_ansi_color_for_component_or_default(
default: &str, default: &str,
) -> String { ) -> String {
if let Some(color) = &engine_state.get_config().color_config.get(theme_component) { if let Some(color) = &engine_state.get_config().color_config.get(theme_component) {
let mut caller_stack = Stack::new(); let caller_stack = &mut Stack::new().capture();
let span = Span::unknown(); let span = Span::unknown();
let argument_opt = get_argument_for_color_value(engine_state, color, span); let argument_opt = get_argument_for_color_value(engine_state, color, span);
@ -350,13 +351,11 @@ fn get_ansi_color_for_component_or_default(
if let Some(decl_id) = engine_state.find_decl(b"ansi", &[]) { if let Some(decl_id) = engine_state.find_decl(b"ansi", &[]) {
if let Ok(result) = eval_call::<WithoutDebug>( if let Ok(result) = eval_call::<WithoutDebug>(
engine_state, engine_state,
&mut caller_stack, caller_stack,
&Call { &Call {
decl_id, decl_id,
head: span, head: span,
arguments: vec![argument], arguments: vec![argument],
redirect_stdout: true,
redirect_stderr: true,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}, },
PipelineData::Empty, PipelineData::Empty,

4
crates/nu-engine/src/env.rs
View File

@ -379,7 +379,7 @@ fn get_converted_value(
let block = engine_state.get_block(val.block_id); let block = engine_state.get_block(val.block_id);
if let Some(var) = block.signature.get_positional(0) { if let Some(var) = block.signature.get_positional(0) {
let mut stack = stack.gather_captures(engine_state, &block.captures); let mut stack = stack.captures_to_stack(val.captures.clone());
if let Some(var_id) = &var.var_id { if let Some(var_id) = &var.var_id {
stack.add_var(*var_id, orig_val.clone()); stack.add_var(*var_id, orig_val.clone());
} }
@ -391,8 +391,6 @@ fn get_converted_value(
&mut stack, &mut stack,
block, block,
PipelineData::new_with_metadata(None, val_span), PipelineData::new_with_metadata(None, val_span),
true,
true,
); );
match result { match result {

1003
crates/nu-engine/src/eval.rs
View File

File diff suppressed because it is too large Load Diff

22
crates/nu-engine/src/eval_helpers.rs
View File

@ -8,24 +8,12 @@ use nu_protocol::engine::{EngineState, Stack};
use nu_protocol::{PipelineData, ShellError, Value}; use nu_protocol::{PipelineData, ShellError, Value};
/// Type of eval_block() function /// Type of eval_block() function
pub type EvalBlockFn = fn( pub type EvalBlockFn =
&EngineState, fn(&EngineState, &mut Stack, &Block, PipelineData) -> Result<PipelineData, ShellError>;
&mut Stack,
&Block,
PipelineData,
bool,
bool,
) -> Result<PipelineData, ShellError>;
/// Type of eval_block_with_early_return() function /// Type of eval_block_with_early_return() function
pub type EvalBlockWithEarlyReturnFn = fn( pub type EvalBlockWithEarlyReturnFn =
&EngineState, fn(&EngineState, &mut Stack, &Block, PipelineData) -> Result<PipelineData, ShellError>;
&mut Stack,
&Block,
PipelineData,
bool,
bool,
) -> Result<PipelineData, ShellError>;
/// Type of eval_expression() function /// Type of eval_expression() function
pub type EvalExpressionFn = fn(&EngineState, &mut Stack, &Expression) -> Result<Value, ShellError>; pub type EvalExpressionFn = fn(&EngineState, &mut Stack, &Expression) -> Result<Value, ShellError>;
@ -36,8 +24,6 @@ pub type EvalExpressionWithInputFn = fn(
&mut Stack, &mut Stack,
&Expression, &Expression,
PipelineData, PipelineData,
bool,
bool,
) -> Result<(PipelineData, bool), ShellError>; ) -> Result<(PipelineData, bool), ShellError>;
/// Type of eval_subexpression() function /// Type of eval_subexpression() function

10
crates/nu-explore/src/nu_common/command.rs
View File

@ -2,8 +2,8 @@ use nu_engine::eval_block;
use nu_parser::parse; use nu_parser::parse;
use nu_protocol::debugger::WithoutDebug; use nu_protocol::debugger::WithoutDebug;
use nu_protocol::{ use nu_protocol::{
engine::{EngineState, Stack, StateWorkingSet}, engine::{EngineState, Redirection, Stack, StateWorkingSet},
PipelineData, ShellError, Value, IoStream, PipelineData, ShellError, Value,
}; };
pub fn run_command_with_value( pub fn run_command_with_value(
@ -91,5 +91,9 @@ fn eval_source2(
block.pipelines.drain(..block.pipelines.len() - 1); block.pipelines.drain(..block.pipelines.len() - 1);
} }
eval_block::<WithoutDebug>(engine_state, stack, &block, input, true, true) let stack = &mut stack.push_redirection(
Some(Redirection::Pipe(IoStream::Capture)),
Some(Redirection::Pipe(IoStream::Capture)),
);
eval_block::<WithoutDebug>(engine_state, stack, &block, input)
} }

87
crates/nu-parser/src/flatten.rs
View File

@ -1,6 +1,6 @@
use nu_protocol::ast::{ use nu_protocol::ast::{
Argument, Block, Expr, Expression, ExternalArgument, ImportPatternMember, MatchPattern, Argument, Block, Expr, Expression, ExternalArgument, ImportPatternMember, MatchPattern,
PathMember, Pattern, Pipeline, PipelineElement, RecordItem, PathMember, Pattern, Pipeline, PipelineElement, PipelineRedirection, RecordItem,
}; };
use nu_protocol::{engine::StateWorkingSet, Span}; use nu_protocol::{engine::StateWorkingSet, Span};
use nu_protocol::{DeclId, VarId}; use nu_protocol::{DeclId, VarId};
@ -223,7 +223,7 @@ pub fn flatten_expression(
output.extend(args); output.extend(args);
output output
} }
Expr::ExternalCall(head, args, _) => { Expr::ExternalCall(head, args) => {
let mut output = vec![]; let mut output = vec![];
match **head { match **head {
@ -559,59 +559,42 @@ pub fn flatten_pipeline_element(
working_set: &StateWorkingSet, working_set: &StateWorkingSet,
pipeline_element: &PipelineElement, pipeline_element: &PipelineElement,
) -> Vec<(Span, FlatShape)> { ) -> Vec<(Span, FlatShape)> {
match pipeline_element { let mut output = if let Some(span) = pipeline_element.pipe {
PipelineElement::Expression(span, expr) let mut output = vec![(span, FlatShape::Pipe)];
| PipelineElement::ErrPipedExpression(span, expr) output.extend(flatten_expression(working_set, &pipeline_element.expr));
| PipelineElement::OutErrPipedExpression(span, expr) => { output
if let Some(span) = span { } else {
let mut output = vec![(*span, FlatShape::Pipe)]; flatten_expression(working_set, &pipeline_element.expr)
output.append(&mut flatten_expression(working_set, expr)); };
output
} else { if let Some(redirection) = pipeline_element.redirection.as_ref() {
flatten_expression(working_set, expr) match redirection {
PipelineRedirection::Single { target, .. } => {
output.push((target.span(), FlatShape::Redirection));
if let Some(expr) = target.expr() {
output.extend(flatten_expression(working_set, expr));
}
}
PipelineRedirection::Separate { out, err } => {
let (out, err) = if out.span() <= err.span() {
(out, err)
} else {
(err, out)
};
output.push((out.span(), FlatShape::Redirection));
if let Some(expr) = out.expr() {
output.extend(flatten_expression(working_set, expr));
}
output.push((err.span(), FlatShape::Redirection));
if let Some(expr) = err.expr() {
output.extend(flatten_expression(working_set, expr));
}
} }
} }
PipelineElement::Redirection(span, _, expr, _) => {
let mut output = vec![(*span, FlatShape::Redirection)];
output.append(&mut flatten_expression(working_set, expr));
output
}
PipelineElement::SeparateRedirection {
out: (out_span, out_expr, _),
err: (err_span, err_expr, _),
} => {
let mut output = vec![(*out_span, FlatShape::Redirection)];
output.append(&mut flatten_expression(working_set, out_expr));
output.push((*err_span, FlatShape::Redirection));
output.append(&mut flatten_expression(working_set, err_expr));
output
}
PipelineElement::SameTargetRedirection {
cmd: (cmd_span, cmd_expr),
redirection: (redirect_span, redirect_expr, _),
} => {
let mut output = if let Some(span) = cmd_span {
let mut output = vec![(*span, FlatShape::Pipe)];
output.append(&mut flatten_expression(working_set, cmd_expr));
output
} else {
flatten_expression(working_set, cmd_expr)
};
output.push((*redirect_span, FlatShape::Redirection));
output.append(&mut flatten_expression(working_set, redirect_expr));
output
}
PipelineElement::And(span, expr) => {
let mut output = vec![(*span, FlatShape::And)];
output.append(&mut flatten_expression(working_set, expr));
output
}
PipelineElement::Or(span, expr) => {
let mut output = vec![(*span, FlatShape::Or)];
output.append(&mut flatten_expression(working_set, expr));
output
}
} }
output
} }
pub fn flatten_pipeline( pub fn flatten_pipeline(

25
crates/nu-parser/src/known_external.rs
View File

@ -4,7 +4,7 @@ use nu_protocol::{
engine::Command, engine::Command,
ShellError, Signature, ShellError, Signature,
}; };
use nu_protocol::{PipelineData, Spanned, Type}; use nu_protocol::{PipelineData, Type};
#[derive(Clone)] #[derive(Clone)]
pub struct KnownExternal { pub struct KnownExternal {
@ -42,7 +42,6 @@ impl Command for KnownExternal {
call: &Call, call: &Call,
input: PipelineData, input: PipelineData,
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
let call_span = call.span();
let head_span = call.head; let head_span = call.head;
let decl_id = engine_state let decl_id = engine_state
.find_decl("run-external".as_bytes(), &[]) .find_decl("run-external".as_bytes(), &[])
@ -110,28 +109,6 @@ impl Command for KnownExternal {
} }
} }
if call.redirect_stdout {
extern_call.add_named((
Spanned {
item: "redirect-stdout".into(),
span: call_span,
},
None,
None,
))
}
if call.redirect_stderr {
extern_call.add_named((
Spanned {
item: "redirect-stderr".into(),
span: call_span,
},
None,
None,
))
}
command.run(engine_state, stack, &extern_call, input) command.run(engine_state, stack, &extern_call, input)
} }
} }

2
crates/nu-parser/src/lib.rs
View File

@ -17,7 +17,7 @@ pub use flatten::{
}; };
pub use known_external::KnownExternal; pub use known_external::KnownExternal;
pub use lex::{lex, lex_signature, Token, TokenContents}; pub use lex::{lex, lex_signature, Token, TokenContents};
pub use lite_parser::{lite_parse, LiteBlock, LiteElement}; pub use lite_parser::{lite_parse, LiteBlock, LiteCommand};
pub use parse_keywords::*; pub use parse_keywords::*;
pub use parser_path::*; pub use parser_path::*;

696
crates/nu-parser/src/lite_parser.rs
View File

@ -1,212 +1,135 @@
/// Lite parsing converts a flat stream of tokens from the lexer to a syntax element structure that //! Lite parsing converts a flat stream of tokens from the lexer to a syntax element structure that
/// can be parsed. //! can be parsed.
use std::mem;
use crate::{Token, TokenContents}; use crate::{Token, TokenContents};
use nu_protocol::{ast::Redirection, ParseError, Span}; use nu_protocol::{ast::RedirectionSource, ParseError, Span};
#[derive(Debug)] #[derive(Debug, Clone, Copy)]
pub struct LiteCommand { pub enum LiteRedirectionTarget {
pub comments: Vec<Span>, File {
pub parts: Vec<Span>, connector: Span,
file: Span,
append: bool,
},
Pipe {
connector: Span,
},
} }
impl Default for LiteCommand { impl LiteRedirectionTarget {
fn default() -> Self { pub fn connector(&self) -> Span {
Self::new() match self {
LiteRedirectionTarget::File { connector, .. }
| LiteRedirectionTarget::Pipe { connector } => *connector,
}
} }
} }
#[derive(Debug, Clone)]
pub enum LiteRedirection {
Single {
source: RedirectionSource,
target: LiteRedirectionTarget,
},
Separate {
out: LiteRedirectionTarget,
err: LiteRedirectionTarget,
},
}
#[derive(Debug, Clone, Default)]
pub struct LiteCommand {
pub pipe: Option<Span>,
pub comments: Vec<Span>,
pub parts: Vec<Span>,
pub redirection: Option<LiteRedirection>,
}
impl LiteCommand { impl LiteCommand {
pub fn new() -> Self { fn push(&mut self, span: Span) {
Self {
comments: vec![],
parts: vec![],
}
}
pub fn push(&mut self, span: Span) {
self.parts.push(span); self.parts.push(span);
} }
pub fn is_empty(&self) -> bool { fn try_add_redirection(
self.parts.is_empty() &mut self,
source: RedirectionSource,
target: LiteRedirectionTarget,
) -> Result<(), ParseError> {
let redirection = match (self.redirection.take(), source) {
(None, source) => Ok(LiteRedirection::Single { source, target }),
(
Some(LiteRedirection::Single {
source: RedirectionSource::Stdout,
target: out,
}),
RedirectionSource::Stderr,
) => Ok(LiteRedirection::Separate { out, err: target }),
(
Some(LiteRedirection::Single {
source: RedirectionSource::Stderr,
target: err,
}),
RedirectionSource::Stdout,
) => Ok(LiteRedirection::Separate { out: target, err }),
(
Some(LiteRedirection::Single {
source,
target: first,
}),
_,
) => Err(ParseError::MultipleRedirections(
source,
first.connector(),
target.connector(),
)),
(
Some(LiteRedirection::Separate { out, .. }),
RedirectionSource::Stdout | RedirectionSource::StdoutAndStderr,
) => Err(ParseError::MultipleRedirections(
RedirectionSource::Stdout,
out.connector(),
target.connector(),
)),
(Some(LiteRedirection::Separate { err, .. }), RedirectionSource::Stderr) => {
Err(ParseError::MultipleRedirections(
RedirectionSource::Stderr,
err.connector(),
target.connector(),
))
}
}?;
self.redirection = Some(redirection);
Ok(())
} }
} }
// Note: the Span is the span of the connector not the whole element #[derive(Debug, Clone, Default)]
#[derive(Debug)]
pub enum LiteElement {
Command(Option<Span>, LiteCommand),
// Similar to LiteElement::Command, except the previous command's output is stderr piped.
// e.g: `e>| cmd`
ErrPipedCommand(Option<Span>, LiteCommand),
// Similar to LiteElement::Command, except the previous command's output is stderr + stdout piped.
// e.g: `o+e>| cmd`
OutErrPipedCommand(Option<Span>, LiteCommand),
// final field indicates if it's in append mode
Redirection(Span, Redirection, LiteCommand, bool),
// SeparateRedirection variant can only be generated by two different Redirection variant
// final bool field indicates if it's in append mode
SeparateRedirection {
out: (Span, LiteCommand, bool),
err: (Span, LiteCommand, bool),
},
// SameTargetRedirection variant can only be generated by Command with Redirection::OutAndErr
// redirection's final bool field indicates if it's in append mode
SameTargetRedirection {
cmd: (Option<Span>, LiteCommand),
redirection: (Span, LiteCommand, bool),
},
}
#[derive(Debug, Default)]
pub struct LitePipeline { pub struct LitePipeline {
pub commands: Vec<LiteElement>, pub commands: Vec<LiteCommand>,
} }
impl LitePipeline { impl LitePipeline {
pub fn new() -> Self { fn push(&mut self, element: &mut LiteCommand) {
Self { commands: vec![] } if !element.parts.is_empty() || element.redirection.is_some() {
} self.commands.push(mem::take(element));
pub fn push(&mut self, element: LiteElement) {
self.commands.push(element);
}
pub fn insert(&mut self, index: usize, element: LiteElement) {
self.commands.insert(index, element);
}
pub fn is_empty(&self) -> bool {
self.commands.is_empty()
}
pub fn exists(&self, new_target: &Redirection) -> bool {
for cmd in &self.commands {
if let LiteElement::Redirection(_, exists_target, _, _) = cmd {
if exists_target == new_target {
return true;
}
}
} }
false
} }
} }
#[derive(Debug)] #[derive(Debug, Clone, Default)]
pub struct LiteBlock { pub struct LiteBlock {
pub block: Vec<LitePipeline>, pub block: Vec<LitePipeline>,
} }
impl Default for LiteBlock {
fn default() -> Self {
Self::new()
}
}
impl LiteBlock { impl LiteBlock {
pub fn new() -> Self { fn push(&mut self, pipeline: &mut LitePipeline) {
Self { block: vec![] } if !pipeline.commands.is_empty() {
} self.block.push(mem::take(pipeline));
pub fn push(&mut self, mut pipeline: LitePipeline) {
// once we push `pipeline` to our block
// the block takes ownership of `pipeline`, which means that
// our `pipeline` is complete on collecting commands.
self.merge_redirections(&mut pipeline);
self.merge_cmd_with_outerr_redirection(&mut pipeline);
self.block.push(pipeline);
}
pub fn is_empty(&self) -> bool {
self.block.is_empty()
}
fn merge_cmd_with_outerr_redirection(&self, pipeline: &mut LitePipeline) {
let mut cmd_index = None;
let mut outerr_index = None;
for (index, cmd) in pipeline.commands.iter().enumerate() {
if let LiteElement::Command(..) = cmd {
cmd_index = Some(index);
}
if let LiteElement::Redirection(
_span,
Redirection::StdoutAndStderr,
_target_cmd,
_is_append_mode,
) = cmd
{
outerr_index = Some(index);
break;
}
}
if let (Some(cmd_index), Some(outerr_index)) = (cmd_index, outerr_index) {
// we can make sure that cmd_index is less than outerr_index.
let outerr_redirect = pipeline.commands.remove(outerr_index);
let cmd = pipeline.commands.remove(cmd_index);
// `outerr_redirect` and `cmd` should always be `LiteElement::Command` and `LiteElement::Redirection`
if let (
LiteElement::Command(cmd_span, lite_cmd),
LiteElement::Redirection(span, _, outerr_cmd, is_append_mode),
) = (cmd, outerr_redirect)
{
pipeline.insert(
cmd_index,
LiteElement::SameTargetRedirection {
cmd: (cmd_span, lite_cmd),
redirection: (span, outerr_cmd, is_append_mode),
},
)
}
}
}
fn merge_redirections(&self, pipeline: &mut LitePipeline) {
// In case our command may contains both stdout and stderr redirection.
// We pick them out and Combine them into one LiteElement::SeparateRedirection variant.
let mut stdout_index = None;
let mut stderr_index = None;
for (index, cmd) in pipeline.commands.iter().enumerate() {
if let LiteElement::Redirection(_span, redirection, _target_cmd, _is_append_mode) = cmd
{
match *redirection {
Redirection::Stderr => stderr_index = Some(index),
Redirection::Stdout => stdout_index = Some(index),
Redirection::StdoutAndStderr => {}
}
}
}
if let (Some(out_indx), Some(err_indx)) = (stdout_index, stderr_index) {
let (out_redirect, err_redirect, new_indx) = {
// to avoid panic, we need to remove commands which have larger index first.
if out_indx > err_indx {
let out_redirect = pipeline.commands.remove(out_indx);
let err_redirect = pipeline.commands.remove(err_indx);
(out_redirect, err_redirect, err_indx)
} else {
let err_redirect = pipeline.commands.remove(err_indx);
let out_redirect = pipeline.commands.remove(out_indx);
(out_redirect, err_redirect, out_indx)
}
};
// `out_redirect` and `err_redirect` should always be `LiteElement::Redirection`
if let (
LiteElement::Redirection(out_span, _, out_command, out_append_mode),
LiteElement::Redirection(err_span, _, err_command, err_append_mode),
) = (out_redirect, err_redirect)
{
// using insert with specific index to keep original
// pipeline commands order.
pipeline.insert(
new_indx,
LiteElement::SeparateRedirection {
out: (out_span, out_command, out_append_mode),
err: (err_span, err_command, err_append_mode),
},
)
}
} }
} }
} }
@ -226,162 +149,230 @@ fn last_non_comment_token(tokens: &[Token], cur_idx: usize) -> Option<TokenConte
} }
pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) { pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
let mut block = LiteBlock::new();
let mut curr_pipeline = LitePipeline::new();
let mut curr_command = LiteCommand::new();
let mut last_token = TokenContents::Eol;
let mut last_connector = TokenContents::Pipe;
let mut last_connector_span: Option<Span> = None;
if tokens.is_empty() { if tokens.is_empty() {
return (LiteBlock::new(), None); return (LiteBlock::default(), None);
} }
let mut curr_comment: Option<Vec<Span>> = None; let mut block = LiteBlock::default();
let mut pipeline = LitePipeline::default();
let mut command = LiteCommand::default();
let mut last_token = TokenContents::Eol;
let mut file_redirection = None;
let mut curr_comment: Option<Vec<Span>> = None;
let mut error = None; let mut error = None;
for (idx, token) in tokens.iter().enumerate() { for (idx, token) in tokens.iter().enumerate() {
match &token.contents { if let Some((source, append, span)) = file_redirection.take() {
TokenContents::PipePipe => { if command.parts.is_empty() {
error = error.or(Some(ParseError::ShellOrOr(token.span))); error = error.or(Some(ParseError::LabeledError(
curr_command.push(token.span); "Redirection without command or expression".into(),
last_token = TokenContents::Item; "there is nothing to redirect".into(),
} span,
TokenContents::Item => { )));
// If we have a comment, go ahead and attach it
if let Some(curr_comment) = curr_comment.take() {
curr_command.comments = curr_comment;
}
curr_command.push(token.span);
last_token = TokenContents::Item;
}
TokenContents::OutGreaterThan
| TokenContents::OutGreaterGreaterThan
| TokenContents::ErrGreaterThan
| TokenContents::ErrGreaterGreaterThan
| TokenContents::OutErrGreaterThan
| TokenContents::OutErrGreaterGreaterThan => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
curr_command = LiteCommand::new(); command.push(span);
last_token = token.contents;
last_connector = token.contents;
last_connector_span = Some(token.span);
}
pipe_token @ (TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe) => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
curr_command = LiteCommand::new(); match token.contents {
last_token = *pipe_token; TokenContents::Comment => {
last_connector = *pipe_token; command.comments.push(token.span);
last_connector_span = Some(token.span); curr_comment = None;
}
TokenContents::Eol => {
// Handle `[Command] [Pipe] ([Comment] | [Eol])+ [Command]`
//
// `[Eol]` branch checks if previous token is `[Pipe]` to construct pipeline
// and so `[Comment] | [Eol]` should be ignore to make it work
let actual_token = last_non_comment_token(tokens, idx);
if actual_token != Some(TokenContents::Pipe)
&& actual_token != Some(TokenContents::OutGreaterThan)
{
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
} }
TokenContents::Pipe
curr_command = LiteCommand::new(); | TokenContents::ErrGreaterPipe
if !curr_pipeline.is_empty() { | TokenContents::OutErrGreaterPipe => {
block.push(curr_pipeline); pipeline.push(&mut command);
command.pipe = Some(token.span);
curr_pipeline = LitePipeline::new(); }
last_connector = TokenContents::Pipe; TokenContents::Semicolon => {
last_connector_span = None; pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Eol => {
pipeline.push(&mut command);
}
_ => command.push(token.span),
}
} else {
match &token.contents {
TokenContents::PipePipe => {
error = error.or(Some(ParseError::ShellOrOr(token.span)));
command.push(span);
command.push(token.span);
}
TokenContents::Item => {
let target = LiteRedirectionTarget::File {
connector: span,
file: token.span,
append,
};
if let Err(err) = command.try_add_redirection(source, target) {
error = error.or(Some(err));
command.push(span);
command.push(token.span)
}
}
TokenContents::OutGreaterThan
| TokenContents::OutGreaterGreaterThan
| TokenContents::ErrGreaterThan
| TokenContents::ErrGreaterGreaterThan
| TokenContents::OutErrGreaterThan
| TokenContents::OutErrGreaterGreaterThan => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
command.push(token.span);
}
TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Eol => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
}
TokenContents::Semicolon => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Comment => {
error = error.or(Some(ParseError::Expected("redirection target", span)));
command.push(span);
command.comments.push(token.span);
curr_comment = None;
} }
} }
if last_token == TokenContents::Eol {
// Clear out the comment as we're entering a new comment
curr_comment = None;
}
last_token = TokenContents::Eol;
} }
TokenContents::Semicolon => { } else {
if let Some(err) = push_command_to( match &token.contents {
&mut curr_pipeline, TokenContents::PipePipe => {
curr_command, error = error.or(Some(ParseError::ShellOrOr(token.span)));
last_connector, command.push(token.span);
last_connector_span,
) {
error = Some(err);
} }
TokenContents::Item => {
// This is commented out to preserve old parser behavior,
// but we should probably error here.
//
// if element.redirection.is_some() {
// error = error.or(Some(ParseError::LabeledError(
// "Unexpected positional".into(),
// "cannot add positional arguments after output redirection".into(),
// token.span,
// )));
// }
//
// For example, this is currently allowed: ^echo thing o> out.txt extra_arg
curr_command = LiteCommand::new(); // If we have a comment, go ahead and attach it
if !curr_pipeline.is_empty() { if let Some(curr_comment) = curr_comment.take() {
block.push(curr_pipeline); command.comments = curr_comment;
}
curr_pipeline = LitePipeline::new(); command.push(token.span);
last_connector = TokenContents::Pipe;
last_connector_span = None;
} }
TokenContents::OutGreaterThan => {
file_redirection = Some((RedirectionSource::Stdout, false, token.span));
}
TokenContents::OutGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::Stdout, true, token.span));
}
TokenContents::ErrGreaterThan => {
file_redirection = Some((RedirectionSource::Stderr, false, token.span));
}
TokenContents::ErrGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::Stderr, true, token.span));
}
TokenContents::OutErrGreaterThan => {
file_redirection =
Some((RedirectionSource::StdoutAndStderr, false, token.span));
}
TokenContents::OutErrGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::StdoutAndStderr, true, token.span));
}
TokenContents::ErrGreaterPipe => {
let target = LiteRedirectionTarget::Pipe {
connector: token.span,
};
if let Err(err) = command.try_add_redirection(RedirectionSource::Stderr, target)
{
error = error.or(Some(err));
}
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::OutErrGreaterPipe => {
let target = LiteRedirectionTarget::Pipe {
connector: token.span,
};
if let Err(err) =
command.try_add_redirection(RedirectionSource::StdoutAndStderr, target)
{
error = error.or(Some(err));
}
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Pipe => {
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Eol => {
// Handle `[Command] [Pipe] ([Comment] | [Eol])+ [Command]`
//
// `[Eol]` branch checks if previous token is `[Pipe]` to construct pipeline
// and so `[Comment] | [Eol]` should be ignore to make it work
let actual_token = last_non_comment_token(tokens, idx);
if actual_token != Some(TokenContents::Pipe) {
pipeline.push(&mut command);
block.push(&mut pipeline);
}
last_token = TokenContents::Semicolon; if last_token == TokenContents::Eol {
} // Clear out the comment as we're entering a new comment
TokenContents::Comment => { curr_comment = None;
// Comment is beside something }
if last_token != TokenContents::Eol { }
curr_command.comments.push(token.span); TokenContents::Semicolon => {
curr_comment = None; pipeline.push(&mut command);
} else { block.push(&mut pipeline);
// Comment precedes something }
if let Some(curr_comment) = &mut curr_comment { TokenContents::Comment => {
curr_comment.push(token.span); // Comment is beside something
if last_token != TokenContents::Eol {
command.comments.push(token.span);
curr_comment = None;
} else { } else {
curr_comment = Some(vec![token.span]); // Comment precedes something
if let Some(curr_comment) = &mut curr_comment {
curr_comment.push(token.span);
} else {
curr_comment = Some(vec![token.span]);
}
} }
} }
last_token = TokenContents::Comment;
} }
} }
last_token = token.contents;
} }
if let Some(err) = push_command_to( if let Some((_, _, span)) = file_redirection {
&mut curr_pipeline, command.push(span);
curr_command, error = error.or(Some(ParseError::Expected("redirection target", span)));
last_connector,
last_connector_span,
) {
error = Some(err);
}
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
} }
pipeline.push(&mut command);
block.push(&mut pipeline);
if last_non_comment_token(tokens, tokens.len()) == Some(TokenContents::Pipe) { if last_non_comment_token(tokens, tokens.len()) == Some(TokenContents::Pipe) {
( (
block, block,
@ -394,86 +385,3 @@ pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
(block, error) (block, error)
} }
} }
fn get_redirection(connector: TokenContents) -> Option<(Redirection, bool)> {
match connector {
TokenContents::OutGreaterThan => Some((Redirection::Stdout, false)),
TokenContents::OutGreaterGreaterThan => Some((Redirection::Stdout, true)),
TokenContents::ErrGreaterThan => Some((Redirection::Stderr, false)),
TokenContents::ErrGreaterGreaterThan => Some((Redirection::Stderr, true)),
TokenContents::OutErrGreaterThan => Some((Redirection::StdoutAndStderr, false)),
TokenContents::OutErrGreaterGreaterThan => Some((Redirection::StdoutAndStderr, true)),
_ => None,
}
}
/// push a `command` to `pipeline`
///
/// It will return Some(err) if `command` is empty and we want to push a
/// redirection command, or we have meet the same redirection in `pipeline`.
fn push_command_to(
pipeline: &mut LitePipeline,
command: LiteCommand,
last_connector: TokenContents,
last_connector_span: Option<Span>,
) -> Option<ParseError> {
if !command.is_empty() {
match get_redirection(last_connector) {
Some((redirect, is_append_mode)) => {
let span = last_connector_span
.expect("internal error: redirection missing span information");
if pipeline.exists(&redirect) {
return Some(ParseError::LabeledError(
"Redirection can be set only once".into(),
"try to remove one".into(),
span,
));
}
pipeline.push(LiteElement::Redirection(
last_connector_span
.expect("internal error: redirection missing span information"),
redirect,
command,
is_append_mode,
))
}
None => {
if last_connector == TokenContents::ErrGreaterPipe {
pipeline.push(LiteElement::ErrPipedCommand(last_connector_span, command))
} else if last_connector == TokenContents::OutErrGreaterPipe {
// Don't allow o+e>| along with redirection.
for cmd in &pipeline.commands {
if matches!(
cmd,
LiteElement::Redirection { .. }
| LiteElement::SameTargetRedirection { .. }
| LiteElement::SeparateRedirection { .. }
) {
return Some(ParseError::LabeledError(
"`o+e>|` pipe is not allowed to use with redirection".into(),
"try to use different type of pipe, or remove redirection".into(),
last_connector_span
.expect("internal error: outerr pipe missing span information"),
));
}
}
pipeline.push(LiteElement::OutErrPipedCommand(
last_connector_span,
command,
))
} else {
pipeline.push(LiteElement::Command(last_connector_span, command))
}
}
}
None
} else if get_redirection(last_connector).is_some() {
Some(ParseError::Expected(
"redirection target",
last_connector_span.expect("internal error: redirection missing span information"),
))
} else {
None
}
}

522
crates/nu-parser/src/parse_keywords.rs
View File

@ -1,6 +1,7 @@
use crate::{ use crate::{
exportable::Exportable, exportable::Exportable,
parse_block, parse_block,
parser::{parse_redirection, redirecting_builtin_error},
parser_path::ParserPath, parser_path::ParserPath,
type_check::{check_block_input_output, type_compatible}, type_check::{check_block_input_output, type_compatible},
}; };
@ -28,7 +29,7 @@ use crate::{
is_math_expression_like, is_math_expression_like,
known_external::KnownExternal, known_external::KnownExternal,
lex, lex,
lite_parser::{lite_parse, LiteCommand, LiteElement}, lite_parser::{lite_parse, LiteCommand},
parser::{ parser::{
check_call, check_name, garbage, garbage_pipeline, parse, parse_call, parse_expression, check_call, check_name, garbage, garbage_pipeline, parse, parse_call, parse_expression,
parse_full_signature, parse_import_pattern, parse_internal_call, parse_multispan_value, parse_full_signature, parse_import_pattern, parse_internal_call, parse_multispan_value,
@ -88,17 +89,8 @@ pub fn is_unaliasable_parser_keyword(working_set: &StateWorkingSet, spans: &[Spa
/// This is a new more compact method of calling parse_xxx() functions without repeating the /// This is a new more compact method of calling parse_xxx() functions without repeating the
/// parse_call() in each function. Remaining keywords can be moved here. /// parse_call() in each function. Remaining keywords can be moved here.
pub fn parse_keyword( pub fn parse_keyword(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
working_set: &mut StateWorkingSet, let call_expr = parse_call(working_set, &lite_command.parts, lite_command.parts[0]);
lite_command: &LiteCommand,
is_subexpression: bool,
) -> Pipeline {
let call_expr = parse_call(
working_set,
&lite_command.parts,
lite_command.parts[0],
is_subexpression,
);
// if err.is_some() { // if err.is_some() {
// return (Pipeline::from_vec(vec![call_expr]), err); // return (Pipeline::from_vec(vec![call_expr]), err);
@ -246,7 +238,8 @@ pub fn parse_def_predecl(working_set: &mut StateWorkingSet, spans: &[Span]) {
} }
} }
pub fn parse_for(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression { pub fn parse_for(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Expression {
let spans = &lite_command.parts;
// Checking that the function is used with the correct name // Checking that the function is used with the correct name
// Maybe this is not necessary but it is a sanity check // Maybe this is not necessary but it is a sanity check
if working_set.get_span_contents(spans[0]) != b"for" { if working_set.get_span_contents(spans[0]) != b"for" {
@ -256,6 +249,10 @@ pub fn parse_for(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expressio
)); ));
return garbage(spans[0]); return garbage(spans[0]);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("for", redirection));
return garbage(spans[0]);
}
// Parsing the spans and checking that they match the register signature // Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call // Using a parsed call makes more sense than checking for how many spans are in the call
@ -393,6 +390,10 @@ pub fn parse_def(
)); ));
return (garbage_pipeline(spans), None); return (garbage_pipeline(spans), None);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("def", redirection));
return (garbage_pipeline(spans), None);
}
// Parsing the spans and checking that they match the register signature // Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call // Using a parsed call makes more sense than checking for how many spans are in the call
@ -667,6 +668,10 @@ pub fn parse_extern(
)); ));
return garbage_pipeline(spans); return garbage_pipeline(spans);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("extern", redirection));
return garbage_pipeline(spans);
}
// Parsing the spans and checking that they match the register signature // Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call // Using a parsed call makes more sense than checking for how many spans are in the call
@ -818,6 +823,10 @@ pub fn parse_alias(
)); ));
return garbage_pipeline(spans); return garbage_pipeline(spans);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("alias", redirection));
return garbage_pipeline(spans);
}
if let Some(span) = check_name(working_set, spans) { if let Some(span) = check_name(working_set, spans) {
return Pipeline::from_vec(vec![garbage(*span)]); return Pipeline::from_vec(vec![garbage(*span)]);
@ -907,7 +916,7 @@ pub fn parse_alias(
{ {
// TODO: Maybe we need to implement a Display trait for Expression? // TODO: Maybe we need to implement a Display trait for Expression?
let starting_error_count = working_set.parse_errors.len(); let starting_error_count = working_set.parse_errors.len();
let expr = parse_expression(working_set, replacement_spans, false); let expr = parse_expression(working_set, replacement_spans);
working_set.parse_errors.truncate(starting_error_count); working_set.parse_errors.truncate(starting_error_count);
let msg = format!("{:?}", expr.expr); let msg = format!("{:?}", expr.expr);
@ -923,12 +932,7 @@ pub fn parse_alias(
let starting_error_count = working_set.parse_errors.len(); let starting_error_count = working_set.parse_errors.len();
working_set.search_predecls = false; working_set.search_predecls = false;
let expr = parse_call( let expr = parse_call(working_set, replacement_spans, replacement_spans[0]);
working_set,
replacement_spans,
replacement_spans[0],
false, // TODO: Should this be set properly???
);
working_set.search_predecls = true; working_set.search_predecls = true;
@ -1063,24 +1067,32 @@ pub fn parse_export_in_block(
let full_name = if lite_command.parts.len() > 1 { let full_name = if lite_command.parts.len() > 1 {
let sub = working_set.get_span_contents(lite_command.parts[1]); let sub = working_set.get_span_contents(lite_command.parts[1]);
match sub { match sub {
b"alias" | b"def" | b"extern" | b"use" | b"module" | b"const" => { b"alias" => "export alias",
[b"export ", sub].concat() b"def" => "export def",
} b"extern" => "export extern",
_ => b"export".to_vec(), b"use" => "export use",
b"module" => "export module",
b"const" => "export const",
_ => "export",
} }
} else { } else {
b"export".to_vec() "export"
}; };
if let Some(decl_id) = working_set.find_decl(&full_name) { if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error(full_name, redirection));
return garbage_pipeline(&lite_command.parts);
}
if let Some(decl_id) = working_set.find_decl(full_name.as_bytes()) {
let ParsedInternalCall { call, output, .. } = parse_internal_call( let ParsedInternalCall { call, output, .. } = parse_internal_call(
working_set, working_set,
if full_name == b"export" { if full_name == "export" {
lite_command.parts[0] lite_command.parts[0]
} else { } else {
span(&lite_command.parts[0..2]) span(&lite_command.parts[0..2])
}, },
if full_name == b"export" { if full_name == "export" {
&lite_command.parts[1..] &lite_command.parts[1..]
} else { } else {
&lite_command.parts[2..] &lite_command.parts[2..]
@ -1107,16 +1119,13 @@ pub fn parse_export_in_block(
} }
} else { } else {
working_set.error(ParseError::UnknownState( working_set.error(ParseError::UnknownState(
format!( format!("internal error: '{full_name}' declaration not found",),
"internal error: '{}' declaration not found",
String::from_utf8_lossy(&full_name)
),
span(&lite_command.parts), span(&lite_command.parts),
)); ));
return garbage_pipeline(&lite_command.parts); return garbage_pipeline(&lite_command.parts);
}; };
if &full_name == b"export" { if full_name == "export" {
// export by itself is meaningless // export by itself is meaningless
working_set.error(ParseError::UnexpectedKeyword( working_set.error(ParseError::UnexpectedKeyword(
"export".into(), "export".into(),
@ -1125,19 +1134,16 @@ pub fn parse_export_in_block(
return garbage_pipeline(&lite_command.parts); return garbage_pipeline(&lite_command.parts);
} }
match full_name.as_slice() { match full_name {
b"export alias" => parse_alias(working_set, lite_command, None), "export alias" => parse_alias(working_set, lite_command, None),
b"export def" => parse_def(working_set, lite_command, None).0, "export def" => parse_def(working_set, lite_command, None).0,
b"export const" => parse_const(working_set, &lite_command.parts[1..]), "export const" => parse_const(working_set, &lite_command.parts[1..]),
b"export use" => { "export use" => parse_use(working_set, lite_command).0,
let (pipeline, _) = parse_use(working_set, &lite_command.parts); "export module" => parse_module(working_set, lite_command, None).0,
pipeline "export extern" => parse_extern(working_set, lite_command, None),
}
b"export module" => parse_module(working_set, lite_command, None).0,
b"export extern" => parse_extern(working_set, lite_command, None),
_ => { _ => {
working_set.error(ParseError::UnexpectedKeyword( working_set.error(ParseError::UnexpectedKeyword(
String::from_utf8_lossy(&full_name).to_string(), full_name.into(),
lite_command.parts[0], lite_command.parts[0],
)); ));
@ -1186,8 +1192,6 @@ pub fn parse_export_in_module(
head: spans[0], head: spans[0],
decl_id: export_decl_id, decl_id: export_decl_id,
arguments: vec![], arguments: vec![],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}); });
@ -1198,6 +1202,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand { let lite_command = LiteCommand {
comments: lite_command.comments.clone(), comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(), parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
}; };
let (pipeline, cmd_result) = let (pipeline, cmd_result) =
parse_def(working_set, &lite_command, Some(module_name)); parse_def(working_set, &lite_command, Some(module_name));
@ -1222,16 +1228,9 @@ pub fn parse_export_in_module(
}; };
// Trying to warp the 'def' call into the 'export def' in a very clumsy way // Trying to warp the 'def' call into the 'export def' in a very clumsy way
if let Some(PipelineElement::Expression( if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
{ {
call = def_call.clone(); call = def_call.clone();
call.head = span(&spans[0..=1]); call.head = span(&spans[0..=1]);
call.decl_id = export_def_decl_id; call.decl_id = export_def_decl_id;
} else { } else {
@ -1247,6 +1246,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand { let lite_command = LiteCommand {
comments: lite_command.comments.clone(), comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(), parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
}; };
let extern_name = [b"export ", kw_name].concat(); let extern_name = [b"export ", kw_name].concat();
@ -1263,16 +1264,9 @@ pub fn parse_export_in_module(
}; };
// Trying to warp the 'def' call into the 'export def' in a very clumsy way // Trying to warp the 'def' call into the 'export def' in a very clumsy way
if let Some(PipelineElement::Expression( if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
{ {
call = def_call.clone(); call = def_call.clone();
call.head = span(&spans[0..=1]); call.head = span(&spans[0..=1]);
call.decl_id = export_def_decl_id; call.decl_id = export_def_decl_id;
} else { } else {
@ -1308,6 +1302,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand { let lite_command = LiteCommand {
comments: lite_command.comments.clone(), comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(), parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
}; };
let pipeline = parse_alias(working_set, &lite_command, Some(module_name)); let pipeline = parse_alias(working_set, &lite_command, Some(module_name));
@ -1323,13 +1319,8 @@ pub fn parse_export_in_module(
}; };
// Trying to warp the 'alias' call into the 'export alias' in a very clumsy way // Trying to warp the 'alias' call into the 'export alias' in a very clumsy way
if let Some(PipelineElement::Expression( if let Some(Expr::Call(alias_call)) =
_, pipeline.elements.first().map(|e| &e.expr.expr)
Expression {
expr: Expr::Call(ref alias_call),
..
},
)) = pipeline.elements.first()
{ {
call = alias_call.clone(); call = alias_call.clone();
@ -1368,8 +1359,10 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand { let lite_command = LiteCommand {
comments: lite_command.comments.clone(), comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(), parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
}; };
let (pipeline, exportables) = parse_use(working_set, &lite_command.parts); let (pipeline, exportables) = parse_use(working_set, &lite_command);
let export_use_decl_id = if let Some(id) = working_set.find_decl(b"export use") { let export_use_decl_id = if let Some(id) = working_set.find_decl(b"export use") {
id id
@ -1382,13 +1375,7 @@ pub fn parse_export_in_module(
}; };
// Trying to warp the 'use' call into the 'export use' in a very clumsy way // Trying to warp the 'use' call into the 'export use' in a very clumsy way
if let Some(PipelineElement::Expression( if let Some(Expr::Call(use_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
_,
Expression {
expr: Expr::Call(ref use_call),
..
},
)) = pipeline.elements.first()
{ {
call = use_call.clone(); call = use_call.clone();
@ -1419,13 +1406,8 @@ pub fn parse_export_in_module(
}; };
// Trying to warp the 'module' call into the 'export module' in a very clumsy way // Trying to warp the 'module' call into the 'export module' in a very clumsy way
if let Some(PipelineElement::Expression( if let Some(Expr::Call(module_call)) =
_, pipeline.elements.first().map(|e| &e.expr.expr)
Expression {
expr: Expr::Call(ref module_call),
..
},
)) = pipeline.elements.first()
{ {
call = module_call.clone(); call = module_call.clone();
@ -1476,13 +1458,7 @@ pub fn parse_export_in_module(
}; };
// Trying to warp the 'const' call into the 'export const' in a very clumsy way // Trying to warp the 'const' call into the 'export const' in a very clumsy way
if let Some(PipelineElement::Expression( if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
{ {
call = def_call.clone(); call = def_call.clone();
@ -1690,9 +1666,7 @@ pub fn parse_module_block(
for pipeline in &output.block { for pipeline in &output.block {
if pipeline.commands.len() == 1 { if pipeline.commands.len() == 1 {
if let LiteElement::Command(_, command) = &pipeline.commands[0] { parse_def_predecl(working_set, &pipeline.commands[0].parts);
parse_def_predecl(working_set, &command.parts);
}
} }
} }
@ -1702,186 +1676,146 @@ pub fn parse_module_block(
for pipeline in output.block.iter() { for pipeline in output.block.iter() {
if pipeline.commands.len() == 1 { if pipeline.commands.len() == 1 {
match &pipeline.commands[0] { let command = &pipeline.commands[0];
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command) => {
let name = working_set.get_span_contents(command.parts[0]);
match name { let name = working_set.get_span_contents(command.parts[0]);
b"def" => {
block.pipelines.push(
parse_def(
working_set,
command,
None, // using commands named as the module locally is OK
)
.0,
)
}
b"const" => block
.pipelines
.push(parse_const(working_set, &command.parts)),
b"extern" => block
.pipelines
.push(parse_extern(working_set, command, None)),
b"alias" => {
block.pipelines.push(parse_alias(
working_set,
command,
None, // using aliases named as the module locally is OK
))
}
b"use" => {
let (pipeline, _) = parse_use(working_set, &command.parts);
block.pipelines.push(pipeline) match name {
} b"def" => {
b"module" => { block.pipelines.push(
let (pipeline, _) = parse_module( parse_def(
working_set, working_set,
command, command,
None, // using modules named as the module locally is OK None, // using commands named as the module locally is OK
); )
.0,
)
}
b"const" => block
.pipelines
.push(parse_const(working_set, &command.parts)),
b"extern" => block
.pipelines
.push(parse_extern(working_set, command, None)),
b"alias" => {
block.pipelines.push(parse_alias(
working_set,
command,
None, // using aliases named as the module locally is OK
))
}
b"use" => {
let (pipeline, _) = parse_use(working_set, command);
block.pipelines.push(pipeline) block.pipelines.push(pipeline)
} }
b"export" => { b"module" => {
let (pipe, exportables) = let (pipeline, _) = parse_module(
parse_export_in_module(working_set, command, module_name); working_set,
command,
None, // using modules named as the module locally is OK
);
for exportable in exportables { block.pipelines.push(pipeline)
match exportable { }
Exportable::Decl { name, id } => { b"export" => {
if &name == b"main" { let (pipe, exportables) =
if module.main.is_some() { parse_export_in_module(working_set, command, module_name);
let err_span = if !pipe.elements.is_empty() {
if let PipelineElement::Expression( for exportable in exportables {
_, match exportable {
Expression { Exportable::Decl { name, id } => {
expr: Expr::Call(call), if &name == b"main" {
.. if module.main.is_some() {
}, let err_span = if !pipe.elements.is_empty() {
) = &pipe.elements[0] if let Expr::Call(call) = &pipe.elements[0].expr.expr {
{ call.head
call.head
} else {
pipe.elements[0].span()
}
} else {
span
};
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name)
.to_string(),
err_span,
));
} else { } else {
module.main = Some(id); pipe.elements[0].expr.span
} }
} else { } else {
module.add_decl(name, id); span
} };
} working_set.error(ParseError::ModuleDoubleMain(
Exportable::Module { name, id } => { String::from_utf8_lossy(module_name).to_string(),
if &name == b"mod" { err_span,
let ( ));
submodule_main, } else {
submodule_decls, module.main = Some(id);
submodule_submodules,
) = {
let submodule = working_set.get_module(id);
(
submodule.main,
submodule.decls(),
submodule.submodules(),
)
};
// Add submodule's decls to the parent module
for (decl_name, decl_id) in submodule_decls {
module.add_decl(decl_name, decl_id);
}
// Add submodule's main command to the parent module
if let Some(main_decl_id) = submodule_main {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &pipe.elements[0]
{
call.head
} else {
pipe.elements[0].span()
}
} else {
span
};
working_set.error(
ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name)
.to_string(),
err_span,
),
);
} else {
module.main = Some(main_decl_id);
}
}
// Add submodule's submodules to the parent module
for (submodule_name, submodule_id) in
submodule_submodules
{
module.add_submodule(submodule_name, submodule_id);
}
} else {
module.add_submodule(name, id);
}
}
Exportable::VarDecl { name, id } => {
module.add_variable(name, id);
} }
} else {
module.add_decl(name, id);
} }
} }
Exportable::Module { name, id } => {
if &name == b"mod" {
let (submodule_main, submodule_decls, submodule_submodules) = {
let submodule = working_set.get_module(id);
(submodule.main, submodule.decls(), submodule.submodules())
};
block.pipelines.push(pipe) // Add submodule's decls to the parent module
} for (decl_name, decl_id) in submodule_decls {
b"export-env" => { module.add_decl(decl_name, decl_id);
let (pipe, maybe_env_block) = }
parse_export_env(working_set, &command.parts);
if let Some(block_id) = maybe_env_block { // Add submodule's main command to the parent module
module.add_env_block(block_id); if let Some(main_decl_id) = submodule_main {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let Expr::Call(call) =
&pipe.elements[0].expr.expr
{
call.head
} else {
pipe.elements[0].expr.span
}
} else {
span
};
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name).to_string(),
err_span,
));
} else {
module.main = Some(main_decl_id);
}
}
// Add submodule's submodules to the parent module
for (submodule_name, submodule_id) in submodule_submodules {
module.add_submodule(submodule_name, submodule_id);
}
} else {
module.add_submodule(name, id);
}
}
Exportable::VarDecl { name, id } => {
module.add_variable(name, id);
} }
block.pipelines.push(pipe)
}
_ => {
working_set.error(ParseError::ExpectedKeyword(
"def, const, extern, alias, use, module, export or export-env keyword".into(),
command.parts[0],
));
block.pipelines.push(garbage_pipeline(&command.parts))
} }
} }
block.pipelines.push(pipe)
} }
LiteElement::Redirection(_, _, command, _) => { b"export-env" => {
let (pipe, maybe_env_block) = parse_export_env(working_set, &command.parts);
if let Some(block_id) = maybe_env_block {
module.add_env_block(block_id);
}
block.pipelines.push(pipe)
}
_ => {
working_set.error(ParseError::ExpectedKeyword(
"def, const, extern, alias, use, module, export or export-env keyword"
.into(),
command.parts[0],
));
block.pipelines.push(garbage_pipeline(&command.parts)) block.pipelines.push(garbage_pipeline(&command.parts))
} }
LiteElement::SeparateRedirection {
out: (_, command, _),
..
} => block.pipelines.push(garbage_pipeline(&command.parts)),
LiteElement::SameTargetRedirection {
cmd: (_, command), ..
} => block.pipelines.push(garbage_pipeline(&command.parts)),
} }
} else { } else {
working_set.error(ParseError::Expected("not a pipeline", span)); working_set.error(ParseError::Expected("not a pipeline", span));
@ -2069,6 +2003,12 @@ pub fn parse_module(
// visible and usable in this module's scope). We want to disable that for files. // visible and usable in this module's scope). We want to disable that for files.
let spans = &lite_command.parts; let spans = &lite_command.parts;
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("module", redirection));
return (garbage_pipeline(spans), None);
}
let mut module_comments = lite_command.comments.clone(); let mut module_comments = lite_command.comments.clone();
let split_id = if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" { let split_id = if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" {
@ -2236,8 +2176,6 @@ pub fn parse_module(
Argument::Positional(module_name_or_path_expr), Argument::Positional(module_name_or_path_expr),
Argument::Positional(block_expr), Argument::Positional(block_expr),
], ],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}); });
@ -2252,7 +2190,12 @@ pub fn parse_module(
) )
} }
pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline, Vec<Exportable>) { pub fn parse_use(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
) -> (Pipeline, Vec<Exportable>) {
let spans = &lite_command.parts;
let (name_span, split_id) = let (name_span, split_id) =
if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" { if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" {
(spans[1], 2) (spans[1], 2)
@ -2277,6 +2220,11 @@ pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline
return (garbage_pipeline(spans), vec![]); return (garbage_pipeline(spans), vec![]);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("use", redirection));
return (garbage_pipeline(spans), vec![]);
}
let (call, call_span, args_spans) = match working_set.find_decl(b"use") { let (call, call_span, args_spans) = match working_set.find_decl(b"use") {
Some(decl_id) => { Some(decl_id) => {
let (command_spans, rest_spans) = spans.split_at(split_id); let (command_spans, rest_spans) = spans.split_at(split_id);
@ -2460,7 +2408,9 @@ pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline
) )
} }
pub fn parse_hide(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline { pub fn parse_hide(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let spans = &lite_command.parts;
if working_set.get_span_contents(spans[0]) != b"hide" { if working_set.get_span_contents(spans[0]) != b"hide" {
working_set.error(ParseError::UnknownState( working_set.error(ParseError::UnknownState(
"internal error: Wrong call name for 'hide' command".into(), "internal error: Wrong call name for 'hide' command".into(),
@ -2468,6 +2418,10 @@ pub fn parse_hide(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
)); ));
return garbage_pipeline(spans); return garbage_pipeline(spans);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("hide", redirection));
return garbage_pipeline(spans);
}
let (call, args_spans) = match working_set.find_decl(b"hide") { let (call, args_spans) = match working_set.find_decl(b"hide") {
Some(decl_id) => { Some(decl_id) => {
@ -3054,8 +3008,6 @@ pub fn parse_let(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
decl_id, decl_id,
head: spans[0], head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)], arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}); });
@ -3198,8 +3150,6 @@ pub fn parse_const(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipelin
decl_id, decl_id,
head: spans[0], head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)], arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}); });
@ -3315,8 +3265,6 @@ pub fn parse_mut(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
decl_id, decl_id,
head: spans[0], head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)], arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
}); });
@ -3353,10 +3301,21 @@ pub fn parse_mut(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
garbage_pipeline(spans) garbage_pipeline(spans)
} }
pub fn parse_source(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline { pub fn parse_source(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let spans = &lite_command.parts;
let name = working_set.get_span_contents(spans[0]); let name = working_set.get_span_contents(spans[0]);
if name == b"source" || name == b"source-env" { if name == b"source" || name == b"source-env" {
if let Some(redirection) = lite_command.redirection.as_ref() {
let name = if name == b"source" {
"source"
} else {
"source-env"
};
working_set.error(redirecting_builtin_error(name, redirection));
return garbage_pipeline(spans);
}
let scoped = name == b"source-env"; let scoped = name == b"source-env";
if let Some(decl_id) = working_set.find_decl(name) { if let Some(decl_id) = working_set.find_decl(name) {
@ -3537,13 +3496,26 @@ pub fn parse_where_expr(working_set: &mut StateWorkingSet, spans: &[Span]) -> Ex
} }
} }
pub fn parse_where(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline { pub fn parse_where(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let expression = parse_where_expr(working_set, spans); let expr = parse_where_expr(working_set, &lite_command.parts);
Pipeline::from_vec(vec![expression]) let redirection = lite_command
.redirection
.as_ref()
.map(|r| parse_redirection(working_set, r));
let element = PipelineElement {
pipe: None,
expr,
redirection,
};
Pipeline {
elements: vec![element],
}
} }
#[cfg(feature = "plugin")] #[cfg(feature = "plugin")]
pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline { pub fn parse_register(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
use std::sync::Arc; use std::sync::Arc;
use nu_plugin::{get_signature, PersistentPlugin, PluginDeclaration}; use nu_plugin::{get_signature, PersistentPlugin, PluginDeclaration};
@ -3551,6 +3523,8 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
engine::Stack, IntoSpanned, PluginIdentity, PluginSignature, RegisteredPlugin, engine::Stack, IntoSpanned, PluginIdentity, PluginSignature, RegisteredPlugin,
}; };
let spans = &lite_command.parts;
let cwd = working_set.get_cwd(); let cwd = working_set.get_cwd();
// Checking that the function is used with the correct name // Checking that the function is used with the correct name
@ -3562,6 +3536,10 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
)); ));
return garbage_pipeline(spans); return garbage_pipeline(spans);
} }
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("register", redirection));
return garbage_pipeline(spans);
}
// Parsing the spans and checking that they match the register signature // Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call // Using a parsed call makes more sense than checking for how many spans are in the call
@ -3677,7 +3655,7 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
// We need the current environment variables for `python` based plugins // We need the current environment variables for `python` based plugins
// Or we'll likely have a problem when a plugin is implemented in a virtual Python environment. // Or we'll likely have a problem when a plugin is implemented in a virtual Python environment.
let get_envs = || { let get_envs = || {
let stack = Stack::new(); let stack = Stack::new().capture();
nu_engine::env::env_to_strings(working_set.permanent_state, &stack) nu_engine::env::env_to_strings(working_set.permanent_state, &stack)
}; };

63
crates/nu-parser/src/parse_patterns.rs
View File

@ -7,7 +7,6 @@ use nu_protocol::{
use crate::{ use crate::{
lex, lite_parse, lex, lite_parse,
parser::{is_variable, parse_value}, parser::{is_variable, parse_value},
LiteElement,
}; };
pub fn garbage(span: Span) -> MatchPattern { pub fn garbage(span: Span) -> MatchPattern {
@ -108,48 +107,46 @@ pub fn parse_list_pattern(working_set: &mut StateWorkingSet, span: Span) -> Matc
let mut args = vec![]; let mut args = vec![];
if !output.block.is_empty() { if !output.block.is_empty() {
for arg in &output.block[0].commands { for command in &output.block[0].commands {
let mut spans_idx = 0; let mut spans_idx = 0;
if let LiteElement::Command(_, command) = arg { while spans_idx < command.parts.len() {
while spans_idx < command.parts.len() { let contents = working_set.get_span_contents(command.parts[spans_idx]);
let contents = working_set.get_span_contents(command.parts[spans_idx]); if contents == b".." {
if contents == b".." { args.push(MatchPattern {
pattern: Pattern::IgnoreRest,
guard: None,
span: command.parts[spans_idx],
});
break;
} else if contents.starts_with(b"..$") {
if let Some(var_id) = parse_variable_pattern_helper(
working_set,
Span::new(
command.parts[spans_idx].start + 2,
command.parts[spans_idx].end,
),
) {
args.push(MatchPattern { args.push(MatchPattern {
pattern: Pattern::IgnoreRest, pattern: Pattern::Rest(var_id),
guard: None, guard: None,
span: command.parts[spans_idx], span: command.parts[spans_idx],
}); });
break; break;
} else if contents.starts_with(b"..$") {
if let Some(var_id) = parse_variable_pattern_helper(
working_set,
Span::new(
command.parts[spans_idx].start + 2,
command.parts[spans_idx].end,
),
) {
args.push(MatchPattern {
pattern: Pattern::Rest(var_id),
guard: None,
span: command.parts[spans_idx],
});
break;
} else {
args.push(garbage(command.parts[spans_idx]));
working_set.error(ParseError::Expected(
"valid variable name",
command.parts[spans_idx],
));
}
} else { } else {
let arg = parse_pattern(working_set, command.parts[spans_idx]); args.push(garbage(command.parts[spans_idx]));
working_set.error(ParseError::Expected(
"valid variable name",
command.parts[spans_idx],
));
}
} else {
let arg = parse_pattern(working_set, command.parts[spans_idx]);
args.push(arg); args.push(arg);
}; };
spans_idx += 1; spans_idx += 1;
}
} }
} }
} }

742
crates/nu-parser/src/parser.rs
View File

@ -1,6 +1,6 @@
use crate::{ use crate::{
lex::{lex, lex_signature}, lex::{lex, lex_signature},
lite_parser::{lite_parse, LiteCommand, LiteElement, LitePipeline}, lite_parser::{lite_parse, LiteCommand, LitePipeline, LiteRedirection, LiteRedirectionTarget},
parse_mut, parse_mut,
parse_patterns::parse_pattern, parse_patterns::parse_pattern,
parse_shape_specs::{parse_shape_name, parse_type, ShapeDescriptorUse}, parse_shape_specs::{parse_shape_name, parse_type, ShapeDescriptorUse},
@ -14,7 +14,7 @@ use nu_protocol::{
Argument, Assignment, Bits, Block, Boolean, Call, CellPath, Comparison, Expr, Expression, Argument, Assignment, Bits, Block, Boolean, Call, CellPath, Comparison, Expr, Expression,
ExternalArgument, FullCellPath, ImportPattern, ImportPatternHead, ImportPatternMember, ExternalArgument, FullCellPath, ImportPattern, ImportPatternHead, ImportPatternMember,
MatchPattern, Math, Operator, PathMember, Pattern, Pipeline, PipelineElement, MatchPattern, Math, Operator, PathMember, Pattern, Pipeline, PipelineElement,
RangeInclusion, RangeOperator, RecordItem, PipelineRedirection, RangeInclusion, RangeOperator, RecordItem, RedirectionTarget,
}, },
engine::StateWorkingSet, engine::StateWorkingSet,
eval_const::eval_constant, eval_const::eval_constant,
@ -303,11 +303,7 @@ fn parse_external_arg(working_set: &mut StateWorkingSet, span: Span) -> External
} }
} }
pub fn parse_external_call( pub fn parse_external_call(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
working_set: &mut StateWorkingSet,
spans: &[Span],
is_subexpression: bool,
) -> Expression {
trace!("parse external"); trace!("parse external");
let mut args = vec![]; let mut args = vec![];
@ -324,7 +320,7 @@ pub fn parse_external_call(
let head = if head_contents.starts_with(b"$") || head_contents.starts_with(b"(") { let head = if head_contents.starts_with(b"$") || head_contents.starts_with(b"(") {
// the expression is inside external_call, so it's a subexpression // the expression is inside external_call, so it's a subexpression
let arg = parse_expression(working_set, &[head_span], true); let arg = parse_expression(working_set, &[head_span]);
Box::new(arg) Box::new(arg)
} else { } else {
let (contents, err) = unescape_unquote_string(&head_contents, head_span); let (contents, err) = unescape_unquote_string(&head_contents, head_span);
@ -346,7 +342,7 @@ pub fn parse_external_call(
} }
Expression { Expression {
expr: Expr::ExternalCall(head, args, is_subexpression), expr: Expr::ExternalCall(head, args),
span: span(spans), span: span(spans),
ty: Type::Any, ty: Type::Any,
custom_completion: None, custom_completion: None,
@ -708,7 +704,7 @@ pub fn parse_multispan_value(
// is it subexpression? // is it subexpression?
// Not sure, but let's make it not, so the behavior is the same as previous version of nushell. // Not sure, but let's make it not, so the behavior is the same as previous version of nushell.
let arg = parse_expression(working_set, &spans[*spans_idx..], false); let arg = parse_expression(working_set, &spans[*spans_idx..]);
*spans_idx = spans.len() - 1; *spans_idx = spans.len() - 1;
arg arg
@ -1095,12 +1091,7 @@ pub fn parse_internal_call(
} }
} }
pub fn parse_call( pub fn parse_call(working_set: &mut StateWorkingSet, spans: &[Span], head: Span) -> Expression {
working_set: &mut StateWorkingSet,
spans: &[Span],
head: Span,
is_subexpression: bool,
) -> Expression {
trace!("parsing: call"); trace!("parsing: call");
if spans.is_empty() { if spans.is_empty() {
@ -1179,7 +1170,7 @@ pub fn parse_call(
let parsed_call = if let Some(alias) = decl.as_alias() { let parsed_call = if let Some(alias) = decl.as_alias() {
if let Expression { if let Expression {
expr: Expr::ExternalCall(head, args, is_subexpression), expr: Expr::ExternalCall(head, args),
span: _, span: _,
ty, ty,
custom_completion, custom_completion,
@ -1198,7 +1189,7 @@ pub fn parse_call(
head.span = spans[0]; // replacing the spans preserves syntax highlighting head.span = spans[0]; // replacing the spans preserves syntax highlighting
return Expression { return Expression {
expr: Expr::ExternalCall(head, final_args, *is_subexpression), expr: Expr::ExternalCall(head, final_args),
span: span(spans), span: span(spans),
ty: ty.clone(), ty: ty.clone(),
custom_completion: *custom_completion, custom_completion: *custom_completion,
@ -1246,7 +1237,7 @@ pub fn parse_call(
trace!("parsing: external call"); trace!("parsing: external call");
// Otherwise, try external command // Otherwise, try external command
parse_external_call(working_set, spans, is_subexpression) parse_external_call(working_set, spans)
} }
} }
@ -3139,9 +3130,11 @@ pub fn parse_row_condition(working_set: &mut StateWorkingSet, spans: &[Span]) ->
// We have an expression, so let's convert this into a block. // We have an expression, so let's convert this into a block.
let mut block = Block::new(); let mut block = Block::new();
let mut pipeline = Pipeline::new(); let mut pipeline = Pipeline::new();
pipeline pipeline.elements.push(PipelineElement {
.elements pipe: None,
.push(PipelineElement::Expression(None, expression)); expr: expression,
redirection: None,
});
block.pipelines.push(pipeline); block.pipelines.push(pipeline);
@ -3846,61 +3839,59 @@ pub fn parse_list_expression(
let mut contained_type: Option<Type> = None; let mut contained_type: Option<Type> = None;
if !output.block.is_empty() { if !output.block.is_empty() {
for arg in output.block.remove(0).commands { for mut command in output.block.remove(0).commands {
let mut spans_idx = 0; let mut spans_idx = 0;
if let LiteElement::Command(_, mut command) = arg { while spans_idx < command.parts.len() {
while spans_idx < command.parts.len() { let curr_span = command.parts[spans_idx];
let curr_span = command.parts[spans_idx]; let curr_tok = working_set.get_span_contents(curr_span);
let curr_tok = working_set.get_span_contents(curr_span); let (arg, ty) = if curr_tok.starts_with(b"...")
let (arg, ty) = if curr_tok.starts_with(b"...") && curr_tok.len() > 3
&& curr_tok.len() > 3 && (curr_tok[3] == b'$' || curr_tok[3] == b'[' || curr_tok[3] == b'(')
&& (curr_tok[3] == b'$' || curr_tok[3] == b'[' || curr_tok[3] == b'(') {
{ // Parse the spread operator
// Parse the spread operator // Remove "..." before parsing argument to spread operator
// Remove "..." before parsing argument to spread operator command.parts[spans_idx] = Span::new(curr_span.start + 3, curr_span.end);
command.parts[spans_idx] = Span::new(curr_span.start + 3, curr_span.end); let spread_arg = parse_multispan_value(
let spread_arg = parse_multispan_value( working_set,
working_set, &command.parts,
&command.parts, &mut spans_idx,
&mut spans_idx, &SyntaxShape::List(Box::new(element_shape.clone())),
&SyntaxShape::List(Box::new(element_shape.clone())), );
); let elem_ty = match &spread_arg.ty {
let elem_ty = match &spread_arg.ty { Type::List(elem_ty) => *elem_ty.clone(),
Type::List(elem_ty) => *elem_ty.clone(), _ => Type::Any,
_ => Type::Any,
};
let span = Span::new(curr_span.start, spread_arg.span.end);
let spread_expr = Expression {
expr: Expr::Spread(Box::new(spread_arg)),
span,
ty: elem_ty.clone(),
custom_completion: None,
};
(spread_expr, elem_ty)
} else {
let arg = parse_multispan_value(
working_set,
&command.parts,
&mut spans_idx,
element_shape,
);
let ty = arg.ty.clone();
(arg, ty)
}; };
let span = Span::new(curr_span.start, spread_arg.span.end);
let spread_expr = Expression {
expr: Expr::Spread(Box::new(spread_arg)),
span,
ty: elem_ty.clone(),
custom_completion: None,
};
(spread_expr, elem_ty)
} else {
let arg = parse_multispan_value(
working_set,
&command.parts,
&mut spans_idx,
element_shape,
);
let ty = arg.ty.clone();
(arg, ty)
};
if let Some(ref ctype) = contained_type { if let Some(ref ctype) = contained_type {
if *ctype != ty { if *ctype != ty {
contained_type = Some(Type::Any); contained_type = Some(Type::Any);
}
} else {
contained_type = Some(ty);
} }
} else {
args.push(arg); contained_type = Some(ty);
spans_idx += 1;
} }
args.push(arg);
spans_idx += 1;
} }
} }
} }
@ -4861,7 +4852,7 @@ pub fn parse_math_expression(
if first_span == b"if" || first_span == b"match" { if first_span == b"if" || first_span == b"match" {
// If expression // If expression
if spans.len() > 1 { if spans.len() > 1 {
return parse_call(working_set, spans, spans[0], false); return parse_call(working_set, spans, spans[0]);
} else { } else {
working_set.error(ParseError::Expected( working_set.error(ParseError::Expected(
"expression", "expression",
@ -4936,7 +4927,7 @@ pub fn parse_math_expression(
// allow `if` to be a special value for assignment. // allow `if` to be a special value for assignment.
if content == b"if" || content == b"match" { if content == b"if" || content == b"match" {
let rhs = parse_call(working_set, &spans[idx..], spans[0], false); let rhs = parse_call(working_set, &spans[idx..], spans[0]);
expr_stack.push(op); expr_stack.push(op);
expr_stack.push(rhs); expr_stack.push(rhs);
break; break;
@ -5055,11 +5046,7 @@ pub fn parse_math_expression(
.expect("internal error: expression stack empty") .expect("internal error: expression stack empty")
} }
pub fn parse_expression( pub fn parse_expression(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
working_set: &mut StateWorkingSet,
spans: &[Span],
is_subexpression: bool,
) -> Expression {
trace!("parsing: expression"); trace!("parsing: expression");
let mut pos = 0; let mut pos = 0;
@ -5134,7 +5121,7 @@ pub fn parse_expression(
spans[0], spans[0],
)); ));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression) parse_call(working_set, &spans[pos..], spans[0])
} }
b"let" | b"const" | b"mut" => { b"let" | b"const" | b"mut" => {
working_set.error(ParseError::AssignInPipeline( working_set.error(ParseError::AssignInPipeline(
@ -5152,19 +5139,19 @@ pub fn parse_expression(
.to_string(), .to_string(),
spans[0], spans[0],
)); ));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression) parse_call(working_set, &spans[pos..], spans[0])
} }
b"overlay" => { b"overlay" => {
if spans.len() > 1 && working_set.get_span_contents(spans[1]) == b"list" { if spans.len() > 1 && working_set.get_span_contents(spans[1]) == b"list" {
// whitelist 'overlay list' // whitelist 'overlay list'
parse_call(working_set, &spans[pos..], spans[0], is_subexpression) parse_call(working_set, &spans[pos..], spans[0])
} else { } else {
working_set.error(ParseError::BuiltinCommandInPipeline( working_set.error(ParseError::BuiltinCommandInPipeline(
"overlay".into(), "overlay".into(),
spans[0], spans[0],
)); ));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression) parse_call(working_set, &spans[pos..], spans[0])
} }
} }
b"where" => parse_where_expr(working_set, &spans[pos..]), b"where" => parse_where_expr(working_set, &spans[pos..]),
@ -5175,10 +5162,10 @@ pub fn parse_expression(
spans[0], spans[0],
)); ));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression) parse_call(working_set, &spans[pos..], spans[0])
} }
_ => parse_call(working_set, &spans[pos..], spans[0], is_subexpression), _ => parse_call(working_set, &spans[pos..], spans[0]),
} }
}; };
@ -5217,8 +5204,6 @@ pub fn parse_expression(
head: Span::unknown(), head: Span::unknown(),
decl_id, decl_id,
arguments, arguments,
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
})); }));
@ -5251,7 +5236,6 @@ pub fn parse_variable(working_set: &mut StateWorkingSet, span: Span) -> Option<V
pub fn parse_builtin_commands( pub fn parse_builtin_commands(
working_set: &mut StateWorkingSet, working_set: &mut StateWorkingSet,
lite_command: &LiteCommand, lite_command: &LiteCommand,
is_subexpression: bool,
) -> Pipeline { ) -> Pipeline {
trace!("parsing: builtin commands"); trace!("parsing: builtin commands");
if !is_math_expression_like(working_set, lite_command.parts[0]) if !is_math_expression_like(working_set, lite_command.parts[0])
@ -5264,12 +5248,7 @@ pub fn parse_builtin_commands(
if cmd.is_alias() { if cmd.is_alias() {
// Parse keywords that can be aliased. Note that we check for "unaliasable" keywords // Parse keywords that can be aliased. Note that we check for "unaliasable" keywords
// because alias can have any name, therefore, we can't check for "aliasable" keywords. // because alias can have any name, therefore, we can't check for "aliasable" keywords.
let call_expr = parse_call( let call_expr = parse_call(working_set, &lite_command.parts, lite_command.parts[0]);
working_set,
&lite_command.parts,
lite_command.parts[0],
is_subexpression,
);
if let Expression { if let Expression {
expr: Expr::Call(call), expr: Expr::Call(call),
@ -5299,26 +5278,31 @@ pub fn parse_builtin_commands(
b"const" => parse_const(working_set, &lite_command.parts), b"const" => parse_const(working_set, &lite_command.parts),
b"mut" => parse_mut(working_set, &lite_command.parts), b"mut" => parse_mut(working_set, &lite_command.parts),
b"for" => { b"for" => {
let expr = parse_for(working_set, &lite_command.parts); let expr = parse_for(working_set, lite_command);
Pipeline::from_vec(vec![expr]) Pipeline::from_vec(vec![expr])
} }
b"alias" => parse_alias(working_set, lite_command, None), b"alias" => parse_alias(working_set, lite_command, None),
b"module" => parse_module(working_set, lite_command, None).0, b"module" => parse_module(working_set, lite_command, None).0,
b"use" => { b"use" => parse_use(working_set, lite_command).0,
let (pipeline, _) = parse_use(working_set, &lite_command.parts); b"overlay" => {
pipeline if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("overlay", redirection));
return garbage_pipeline(&lite_command.parts);
}
parse_keyword(working_set, lite_command)
} }
b"overlay" => parse_keyword(working_set, lite_command, is_subexpression), b"source" | b"source-env" => parse_source(working_set, lite_command),
b"source" | b"source-env" => parse_source(working_set, &lite_command.parts),
b"export" => parse_export_in_block(working_set, lite_command), b"export" => parse_export_in_block(working_set, lite_command),
b"hide" => parse_hide(working_set, &lite_command.parts), b"hide" => parse_hide(working_set, lite_command),
b"where" => parse_where(working_set, &lite_command.parts), b"where" => parse_where(working_set, lite_command),
#[cfg(feature = "plugin")] #[cfg(feature = "plugin")]
b"register" => parse_register(working_set, &lite_command.parts), b"register" => parse_register(working_set, lite_command),
_ => { _ => {
let expr = parse_expression(working_set, &lite_command.parts, is_subexpression); let element = parse_pipeline_element(working_set, lite_command);
Pipeline::from_vec(vec![expr]) Pipeline {
elements: vec![element],
}
} }
} }
} }
@ -5459,6 +5443,76 @@ pub fn parse_record(working_set: &mut StateWorkingSet, span: Span) -> Expression
} }
} }
fn parse_redirection_target(
working_set: &mut StateWorkingSet,
target: &LiteRedirectionTarget,
) -> RedirectionTarget {
match target {
LiteRedirectionTarget::File {
connector,
file,
append,
} => RedirectionTarget::File {
expr: parse_value(working_set, *file, &SyntaxShape::Any),
append: *append,
span: *connector,
},
LiteRedirectionTarget::Pipe { connector } => RedirectionTarget::Pipe { span: *connector },
}
}
pub(crate) fn parse_redirection(
working_set: &mut StateWorkingSet,
target: &LiteRedirection,
) -> PipelineRedirection {
match target {
LiteRedirection::Single { source, target } => PipelineRedirection::Single {
source: *source,
target: parse_redirection_target(working_set, target),
},
LiteRedirection::Separate { out, err } => PipelineRedirection::Separate {
out: parse_redirection_target(working_set, out),
err: parse_redirection_target(working_set, err),
},
}
}
fn parse_pipeline_element(
working_set: &mut StateWorkingSet,
command: &LiteCommand,
) -> PipelineElement {
trace!("parsing: pipeline element");
let expr = parse_expression(working_set, &command.parts);
let redirection = command
.redirection
.as_ref()
.map(|r| parse_redirection(working_set, r));
PipelineElement {
pipe: command.pipe,
expr,
redirection,
}
}
pub(crate) fn redirecting_builtin_error(
name: &'static str,
redirection: &LiteRedirection,
) -> ParseError {
match redirection {
LiteRedirection::Single { target, .. } => {
ParseError::RedirectingBuiltinCommand(name, target.connector(), None)
}
LiteRedirection::Separate { out, err } => ParseError::RedirectingBuiltinCommand(
name,
out.connector().min(err.connector()),
Some(out.connector().max(err.connector())),
),
}
}
pub fn parse_pipeline( pub fn parse_pipeline(
working_set: &mut StateWorkingSet, working_set: &mut StateWorkingSet,
pipeline: &LitePipeline, pipeline: &LitePipeline,
@ -5467,271 +5521,161 @@ pub fn parse_pipeline(
) -> Pipeline { ) -> Pipeline {
if pipeline.commands.len() > 1 { if pipeline.commands.len() > 1 {
// Special case: allow `let` and `mut` to consume the whole pipeline, eg) `let abc = "foo" | str length` // Special case: allow `let` and `mut` to consume the whole pipeline, eg) `let abc = "foo" | str length`
match &pipeline.commands[0] { if let Some(&first) = pipeline.commands[0].parts.first() {
LiteElement::Command(_, command) if !command.parts.is_empty() => { let first = working_set.get_span_contents(first);
if working_set.get_span_contents(command.parts[0]) == b"let" if first == b"let" || first == b"mut" {
|| working_set.get_span_contents(command.parts[0]) == b"mut" let name = if first == b"let" { "let" } else { "mut" };
{ let mut new_command = LiteCommand {
let mut new_command = LiteCommand { comments: vec![],
comments: vec![], parts: pipeline.commands[0].parts.clone(),
parts: command.parts.clone(), pipe: None,
}; redirection: None,
};
for command in &pipeline.commands[1..] { if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
match command { working_set.error(redirecting_builtin_error(name, redirection));
LiteElement::Command(Some(pipe_span), command) }
| LiteElement::ErrPipedCommand(Some(pipe_span), command)
| LiteElement::OutErrPipedCommand(Some(pipe_span), command) => {
new_command.parts.push(*pipe_span);
new_command.comments.extend_from_slice(&command.comments); for element in &pipeline.commands[1..] {
new_command.parts.extend_from_slice(&command.parts); if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
} working_set.error(redirecting_builtin_error(name, redirection));
LiteElement::Redirection(span, ..) => { } else {
working_set.error(ParseError::RedirectionInLetMut(*span, None)) new_command.parts.push(element.pipe.expect("pipe span"));
} new_command.comments.extend_from_slice(&element.comments);
LiteElement::SeparateRedirection { out, err } => { new_command.parts.extend_from_slice(&element.parts);
working_set.error(ParseError::RedirectionInLetMut(
out.0.min(err.0),
Some(out.0.max(err.0)),
))
}
LiteElement::SameTargetRedirection { redirection, .. } => working_set
.error(ParseError::RedirectionInLetMut(redirection.0, None)),
_ => panic!("unsupported"),
}
} }
}
// if the 'let' is complete enough, use it, if not, fall through for now // if the 'let' is complete enough, use it, if not, fall through for now
if new_command.parts.len() > 3 { if new_command.parts.len() > 3 {
let rhs_span = nu_protocol::span(&new_command.parts[3..]); let rhs_span = nu_protocol::span(&new_command.parts[3..]);
new_command.parts.truncate(3); new_command.parts.truncate(3);
new_command.parts.push(rhs_span); new_command.parts.push(rhs_span);
let mut pipeline = let mut pipeline = parse_builtin_commands(working_set, &new_command);
parse_builtin_commands(working_set, &new_command, is_subexpression);
if pipeline_index == 0 { if pipeline_index == 0 {
let let_decl_id = working_set.find_decl(b"let"); let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut"); let mut_decl_id = working_set.find_decl(b"mut");
for element in pipeline.elements.iter_mut() { for element in pipeline.elements.iter_mut() {
if let PipelineElement::Expression( if let Expr::Call(call) = &element.expr.expr {
_, if Some(call.decl_id) == let_decl_id
Expression { || Some(call.decl_id) == mut_decl_id
expr: Expr::Call(call),
..
},
) = element
{ {
if Some(call.decl_id) == let_decl_id // Do an expansion
|| Some(call.decl_id) == mut_decl_id if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter().nth(1)
{ {
// Do an expansion let block = working_set.get_block(*block_id);
if let Some(Expression {
expr: Expr::Block(block_id), if let Some(element) = block
.. .pipelines
}) = call.positional_iter_mut().nth(1) .first()
.and_then(|p| p.elements.first())
.cloned()
{ {
let block = working_set.get_block(*block_id); if element.has_in_variable(working_set) {
let element = wrap_element_with_collect(
if let Some(PipelineElement::Expression( working_set,
prepend, &element,
expr, );
)) = block let block = working_set.get_block_mut(*block_id);
.pipelines block.pipelines[0].elements[0] = element;
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if expr.has_in_variable(working_set) {
let new_expr = PipelineElement::Expression(
prepend,
wrap_expr_with_collect(working_set, &expr),
);
let block =
working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = new_expr;
}
} }
} }
continue;
} else if element.has_in_variable(working_set)
&& !is_subexpression
{
*element = wrap_element_with_collect(working_set, element);
} }
continue;
} else if element.has_in_variable(working_set) && !is_subexpression } else if element.has_in_variable(working_set) && !is_subexpression
{ {
*element = wrap_element_with_collect(working_set, element); *element = wrap_element_with_collect(working_set, element);
} }
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
} }
} }
return pipeline;
} }
return pipeline;
} }
} }
_ => {} }
};
let mut output = pipeline let mut elements = pipeline
.commands .commands
.iter() .iter()
.map(|command| match command { .map(|element| parse_pipeline_element(working_set, element))
LiteElement::Command(span, command) => { .collect::<Vec<_>>();
trace!("parsing: pipeline element: command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::Expression(*span, expr)
}
LiteElement::ErrPipedCommand(span, command) => {
trace!("parsing: pipeline element: err piped command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::ErrPipedExpression(*span, expr)
}
LiteElement::OutErrPipedCommand(span, command) => {
trace!("parsing: pipeline element: err piped command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::OutErrPipedExpression(*span, expr)
}
LiteElement::Redirection(span, redirection, command, is_append_mode) => {
let expr = parse_value(working_set, command.parts[0], &SyntaxShape::Any);
PipelineElement::Redirection(*span, redirection.clone(), expr, *is_append_mode)
}
LiteElement::SeparateRedirection {
out: (out_span, out_command, out_append_mode),
err: (err_span, err_command, err_append_mode),
} => {
trace!("parsing: pipeline element: separate redirection");
let out_expr =
parse_value(working_set, out_command.parts[0], &SyntaxShape::Any);
let err_expr =
parse_value(working_set, err_command.parts[0], &SyntaxShape::Any);
PipelineElement::SeparateRedirection {
out: (*out_span, out_expr, *out_append_mode),
err: (*err_span, err_expr, *err_append_mode),
}
}
LiteElement::SameTargetRedirection {
cmd: (cmd_span, command),
redirection: (redirect_span, redirect_command, is_append_mode),
} => {
trace!("parsing: pipeline element: same target redirection");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
let redirect_expr =
parse_value(working_set, redirect_command.parts[0], &SyntaxShape::Any);
PipelineElement::SameTargetRedirection {
cmd: (*cmd_span, expr),
redirection: (*redirect_span, redirect_expr, *is_append_mode),
}
}
})
.collect::<Vec<PipelineElement>>();
if is_subexpression { if is_subexpression {
for element in output.iter_mut().skip(1) { for element in elements.iter_mut().skip(1) {
if element.has_in_variable(working_set) { if element.has_in_variable(working_set) {
*element = wrap_element_with_collect(working_set, element); *element = wrap_element_with_collect(working_set, element);
} }
} }
} else { } else {
for element in output.iter_mut() { for element in elements.iter_mut() {
if element.has_in_variable(working_set) { if element.has_in_variable(working_set) {
*element = wrap_element_with_collect(working_set, element); *element = wrap_element_with_collect(working_set, element);
} }
} }
} }
Pipeline { elements: output } Pipeline { elements }
} else { } else {
match &pipeline.commands[0] { if let Some(&first) = pipeline.commands[0].parts.first() {
LiteElement::Command(_, command) let first = working_set.get_span_contents(first);
| LiteElement::ErrPipedCommand(_, command) if first == b"let" || first == b"mut" {
| LiteElement::OutErrPipedCommand(_, command) if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
| LiteElement::Redirection(_, _, command, _) let name = if first == b"let" { "let" } else { "mut" };
| LiteElement::SeparateRedirection { working_set.error(redirecting_builtin_error(name, redirection));
out: (_, command, _),
..
} => {
let mut pipeline = parse_builtin_commands(working_set, command, is_subexpression);
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
if pipeline_index == 0 {
for element in pipeline.elements.iter_mut() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = element
{
if Some(call.decl_id) == let_decl_id
|| Some(call.decl_id) == mut_decl_id
{
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter_mut().nth(1)
{
let block = working_set.get_block(*block_id);
if let Some(PipelineElement::Expression(prepend, expr)) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if expr.has_in_variable(working_set) {
let new_expr = PipelineElement::Expression(
prepend,
wrap_expr_with_collect(working_set, &expr),
);
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = new_expr;
}
}
}
continue;
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
}
}
pipeline
}
LiteElement::SameTargetRedirection {
cmd: (span, command),
redirection: (redirect_span, redirect_cmd, is_append_mode),
} => {
trace!("parsing: pipeline element: same target redirection");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
let redirect_expr =
parse_value(working_set, redirect_cmd.parts[0], &SyntaxShape::Any);
Pipeline {
elements: vec![PipelineElement::SameTargetRedirection {
cmd: (*span, expr),
redirection: (*redirect_span, redirect_expr, *is_append_mode),
}],
} }
} }
} }
let mut pipeline = parse_builtin_commands(working_set, &pipeline.commands[0]);
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
if pipeline_index == 0 {
for element in pipeline.elements.iter_mut() {
if let Expr::Call(call) = &element.expr.expr {
if Some(call.decl_id) == let_decl_id || Some(call.decl_id) == mut_decl_id {
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter().nth(1)
{
let block = working_set.get_block(*block_id);
if let Some(element) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if element.has_in_variable(working_set) {
let element = wrap_element_with_collect(working_set, &element);
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = element;
}
}
}
continue;
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
}
}
pipeline
} }
} }
@ -5757,19 +5701,7 @@ pub fn parse_block(
// that share the same block can see each other // that share the same block can see each other
for pipeline in &lite_block.block { for pipeline in &lite_block.block {
if pipeline.commands.len() == 1 { if pipeline.commands.len() == 1 {
match &pipeline.commands[0] { parse_def_predecl(working_set, &pipeline.commands[0].parts)
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command)
| LiteElement::Redirection(_, _, command, _)
| LiteElement::SeparateRedirection {
out: (_, command, _),
..
}
| LiteElement::SameTargetRedirection {
cmd: (_, command), ..
} => parse_def_predecl(working_set, &command.parts),
}
} }
} }
@ -5852,32 +5784,27 @@ pub fn discover_captures_in_pipeline_element(
seen_blocks: &mut HashMap<BlockId, Vec<(VarId, Span)>>, seen_blocks: &mut HashMap<BlockId, Vec<(VarId, Span)>>,
output: &mut Vec<(VarId, Span)>, output: &mut Vec<(VarId, Span)>,
) -> Result<(), ParseError> { ) -> Result<(), ParseError> {
match element { discover_captures_in_expr(working_set, &element.expr, seen, seen_blocks, output)?;
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression) if let Some(redirection) = element.redirection.as_ref() {
| PipelineElement::OutErrPipedExpression(_, expression) match redirection {
| PipelineElement::Redirection(_, _, expression, _) PipelineRedirection::Single { target, .. } => {
| PipelineElement::And(_, expression) if let Some(expr) = target.expr() {
| PipelineElement::Or(_, expression) => { discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
discover_captures_in_expr(working_set, expression, seen, seen_blocks, output) }
} }
PipelineElement::SeparateRedirection { PipelineRedirection::Separate { out, err } => {
out: (_, out_expr, _), if let Some(expr) = out.expr() {
err: (_, err_expr, _), discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
} => { }
discover_captures_in_expr(working_set, out_expr, seen, seen_blocks, output)?; if let Some(expr) = err.expr() {
discover_captures_in_expr(working_set, err_expr, seen, seen_blocks, output)?; discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
Ok(()) }
} }
PipelineElement::SameTargetRedirection {
cmd: (_, cmd_expr),
redirection: (_, redirect_expr, _),
} => {
discover_captures_in_expr(working_set, cmd_expr, seen, seen_blocks, output)?;
discover_captures_in_expr(working_set, redirect_expr, seen, seen_blocks, output)?;
Ok(())
} }
} }
Ok(())
} }
pub fn discover_captures_in_pattern(pattern: &MatchPattern, seen: &mut Vec<VarId>) { pub fn discover_captures_in_pattern(pattern: &MatchPattern, seen: &mut Vec<VarId>) {
@ -6043,7 +5970,7 @@ pub fn discover_captures_in_expr(
} }
Expr::CellPath(_) => {} Expr::CellPath(_) => {}
Expr::DateTime(_) => {} Expr::DateTime(_) => {}
Expr::ExternalCall(head, args, _) => { Expr::ExternalCall(head, args) => {
discover_captures_in_expr(working_set, head, seen, seen_blocks, output)?; discover_captures_in_expr(working_set, head, seen, seen_blocks, output)?;
for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args { for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args {
@ -6193,66 +6120,37 @@ pub fn discover_captures_in_expr(
Ok(()) Ok(())
} }
fn wrap_redirection_with_collect(
working_set: &mut StateWorkingSet,
target: &RedirectionTarget,
) -> RedirectionTarget {
match target {
RedirectionTarget::File { expr, append, span } => RedirectionTarget::File {
expr: wrap_expr_with_collect(working_set, expr),
span: *span,
append: *append,
},
RedirectionTarget::Pipe { span } => RedirectionTarget::Pipe { span: *span },
}
}
fn wrap_element_with_collect( fn wrap_element_with_collect(
working_set: &mut StateWorkingSet, working_set: &mut StateWorkingSet,
element: &PipelineElement, element: &PipelineElement,
) -> PipelineElement { ) -> PipelineElement {
match element { PipelineElement {
PipelineElement::Expression(span, expression) => { pipe: element.pipe,
PipelineElement::Expression(*span, wrap_expr_with_collect(working_set, expression)) expr: wrap_expr_with_collect(working_set, &element.expr),
} redirection: element.redirection.as_ref().map(|r| match r {
PipelineElement::ErrPipedExpression(span, expression) => { PipelineRedirection::Single { source, target } => PipelineRedirection::Single {
PipelineElement::ErrPipedExpression( source: *source,
*span, target: wrap_redirection_with_collect(working_set, target),
wrap_expr_with_collect(working_set, expression), },
) PipelineRedirection::Separate { out, err } => PipelineRedirection::Separate {
} out: wrap_redirection_with_collect(working_set, out),
PipelineElement::OutErrPipedExpression(span, expression) => { err: wrap_redirection_with_collect(working_set, err),
PipelineElement::OutErrPipedExpression( },
*span, }),
wrap_expr_with_collect(working_set, expression),
)
}
PipelineElement::Redirection(span, redirection, expression, is_append_mode) => {
PipelineElement::Redirection(
*span,
redirection.clone(),
wrap_expr_with_collect(working_set, expression),
*is_append_mode,
)
}
PipelineElement::SeparateRedirection {
out: (out_span, out_exp, out_append_mode),
err: (err_span, err_exp, err_append_mode),
} => PipelineElement::SeparateRedirection {
out: (
*out_span,
wrap_expr_with_collect(working_set, out_exp),
*out_append_mode,
),
err: (
*err_span,
wrap_expr_with_collect(working_set, err_exp),
*err_append_mode,
),
},
PipelineElement::SameTargetRedirection {
cmd: (cmd_span, cmd_exp),
redirection: (redirect_span, redirect_exp, is_append_mode),
} => PipelineElement::SameTargetRedirection {
cmd: (*cmd_span, wrap_expr_with_collect(working_set, cmd_exp)),
redirection: (
*redirect_span,
wrap_expr_with_collect(working_set, redirect_exp),
*is_append_mode,
),
},
PipelineElement::And(span, expression) => {
PipelineElement::And(*span, wrap_expr_with_collect(working_set, expression))
}
PipelineElement::Or(span, expression) => {
PipelineElement::Or(*span, wrap_expr_with_collect(working_set, expression))
}
} }
} }
@ -6304,8 +6202,6 @@ fn wrap_expr_with_collect(working_set: &mut StateWorkingSet, expr: &Expression)
head: Span::new(0, 0), head: Span::new(0, 0),
arguments: output, arguments: output,
decl_id, decl_id,
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
})), })),
span, span,

85
crates/nu-parser/src/type_check.rs
View File

@ -1,7 +1,6 @@
use nu_protocol::{ use nu_protocol::{
ast::{ ast::{
Assignment, Bits, Block, Boolean, Comparison, Expr, Expression, Math, Operator, Pipeline, Assignment, Bits, Block, Boolean, Comparison, Expr, Expression, Math, Operator, Pipeline,
PipelineElement,
}, },
engine::StateWorkingSet, engine::StateWorkingSet,
ParseError, Type, ParseError, Type,
@ -917,62 +916,51 @@ pub fn check_pipeline_type(
let mut output_errors: Option<Vec<ParseError>> = None; let mut output_errors: Option<Vec<ParseError>> = None;
'elem: for elem in &pipeline.elements { 'elem: for elem in &pipeline.elements {
match elem { if elem.redirection.is_some() {
PipelineElement::Expression( current_type = Type::Any;
_, } else if let Expr::Call(call) = &elem.expr.expr {
Expression { let decl = working_set.get_decl(call.decl_id);
expr: Expr::Call(call),
..
},
) => {
let decl = working_set.get_decl(call.decl_id);
if current_type == Type::Any { if current_type == Type::Any {
let mut new_current_type = None; let mut new_current_type = None;
for (_, call_output) in decl.signature().input_output_types { for (_, call_output) in decl.signature().input_output_types {
if let Some(inner_current_type) = &new_current_type { if let Some(inner_current_type) = &new_current_type {
if inner_current_type == &Type::Any { if inner_current_type == &Type::Any {
break; break;
} else if inner_current_type != &call_output { } else if inner_current_type != &call_output {
// Union unequal types to Any for now // Union unequal types to Any for now
new_current_type = Some(Type::Any) new_current_type = Some(Type::Any)
}
} else {
new_current_type = Some(call_output.clone())
} }
}
if let Some(new_current_type) = new_current_type {
current_type = new_current_type
} else { } else {
current_type = Type::Any; new_current_type = Some(call_output.clone())
} }
continue 'elem; }
if let Some(new_current_type) = new_current_type {
current_type = new_current_type
} else { } else {
for (call_input, call_output) in decl.signature().input_output_types { current_type = Type::Any;
if type_compatible(&call_input, &current_type) { }
current_type = call_output.clone(); continue 'elem;
continue 'elem; } else {
} for (call_input, call_output) in decl.signature().input_output_types {
if type_compatible(&call_input, &current_type) {
current_type = call_output.clone();
continue 'elem;
} }
} }
}
if !decl.signature().input_output_types.is_empty() { if !decl.signature().input_output_types.is_empty() {
if let Some(output_errors) = &mut output_errors { if let Some(output_errors) = &mut output_errors {
output_errors.push(ParseError::InputMismatch(current_type, call.head)) output_errors.push(ParseError::InputMismatch(current_type, call.head))
} else { } else {
output_errors = output_errors = Some(vec![ParseError::InputMismatch(current_type, call.head)]);
Some(vec![ParseError::InputMismatch(current_type, call.head)]);
}
} }
current_type = Type::Any;
}
PipelineElement::Expression(_, Expression { ty, .. }) => {
current_type = ty.clone();
}
_ => {
current_type = Type::Any;
} }
current_type = Type::Any;
} else {
current_type = elem.expr.ty.clone();
} }
} }
@ -1015,7 +1003,8 @@ pub fn check_block_input_output(working_set: &StateWorkingSet, block: &Block) ->
.elements .elements
.last() .last()
.expect("internal error: we should have elements") .expect("internal error: we should have elements")
.span() .expr
.span
}; };
output_errors.push(ParseError::OutputMismatch(output_type.clone(), span)) output_errors.push(ParseError::OutputMismatch(output_type.clone(), span))

690
crates/nu-parser/tests/test_parser.rs
View File

@ -1,10 +1,8 @@
use nu_parser::*; use nu_parser::*;
use nu_protocol::ast::{Argument, Call, PathMember};
use nu_protocol::Span;
use nu_protocol::{ use nu_protocol::{
ast::{Expr, Expression, PipelineElement}, ast::{Argument, Call, Expr, PathMember},
engine::{Command, EngineState, Stack, StateWorkingSet}, engine::{Command, EngineState, Stack, StateWorkingSet},
ParseError, PipelineData, ShellError, Signature, SyntaxShape, ParseError, PipelineData, ShellError, Signature, Span, SyntaxShape,
}; };
use rstest::rstest; use rstest::rstest;
@ -73,21 +71,15 @@ fn test_int(
} else { } else {
assert!(err.is_none(), "{test_tag}: unexpected error {err:#?}"); assert!(err.is_none(), "{test_tag}: unexpected error {err:#?}");
assert_eq!(block.len(), 1, "{test_tag}: result block length > 1"); assert_eq!(block.len(), 1, "{test_tag}: result block length > 1");
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!( assert_eq!(
expressions.len(), pipeline.len(),
1, 1,
"{test_tag}: got multiple result expressions, expected 1" "{test_tag}: got multiple result expressions, expected 1"
); );
if let PipelineElement::Expression( let element = &pipeline.elements[0];
_, assert!(element.redirection.is_none());
Expression { compare_rhs_binary_op(test_tag, &expected_val, &element.expr.expr);
expr: observed_val, ..
},
) = &expressions.elements[0]
{
compare_rhs_binary_op(test_tag, &expected_val, observed_val);
}
} }
} }
@ -112,7 +104,7 @@ fn compare_rhs_binary_op(
"{test_tag}: Expected: {expected:#?}, observed: {observed:#?}" "{test_tag}: Expected: {expected:#?}, observed: {observed:#?}"
) )
} }
Expr::ExternalCall(e, _, _) => { Expr::ExternalCall(e, _) => {
let observed_expr = &e.expr; let observed_expr = &e.expr;
assert_eq!( assert_eq!(
expected, observed_expr, expected, observed_expr,
@ -259,6 +251,7 @@ pub fn multi_test_parse_number() {
test_int(test.0, test.1, test.2, test.3); test_int(test.0, test.1, test.2, test.3);
} }
} }
#[ignore] #[ignore]
#[test] #[test]
fn test_parse_any() { fn test_parse_any() {
@ -277,6 +270,7 @@ fn test_parse_any() {
} }
} }
} }
#[test] #[test]
pub fn parse_int() { pub fn parse_int() {
let engine_state = EngineState::new(); let engine_state = EngineState::new();
@ -286,18 +280,11 @@ pub fn parse_int() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
assert!(matches!( let element = &pipeline.elements[0];
expressions.elements[0], assert!(element.redirection.is_none());
PipelineElement::Expression( assert_eq!(element.expr.expr, Expr::Int(3));
_,
Expression {
expr: Expr::Int(3),
..
}
)
))
} }
#[test] #[test]
@ -309,18 +296,11 @@ pub fn parse_int_with_underscores() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
assert!(matches!( let element = &pipeline.elements[0];
expressions.elements[0], assert!(element.redirection.is_none());
PipelineElement::Expression( assert_eq!(element.expr.expr, Expr::Int(420692023));
_,
Expression {
expr: Expr::Int(420692023),
..
}
)
))
} }
#[test] #[test]
@ -339,41 +319,32 @@ pub fn parse_cell_path() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
// hoo boy this pattern matching is a pain if let Expr::FullCellPath(b) = &element.expr.expr {
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { assert!(matches!(b.head.expr, Expr::Var(_)));
if let Expr::FullCellPath(b) = &expr.expr { if let [a, b] = &b.tail[..] {
assert!(matches!( if let PathMember::String { val, optional, .. } = a {
b.head, assert_eq!(val, "bar");
Expression { assert_eq!(optional, &false);
expr: Expr::Var(_),
..
}
));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
} else { } else {
panic!("cell path tail is unexpected") panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
} }
} else { } else {
panic!("Not a cell path"); panic!("cell path tail is unexpected")
} }
} else { } else {
panic!("Not an expression") panic!("Not a cell path");
} }
} }
@ -394,41 +365,32 @@ pub fn parse_cell_path_optional() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
// hoo boy this pattern matching is a pain if let Expr::FullCellPath(b) = &element.expr.expr {
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { assert!(matches!(b.head.expr, Expr::Var(_)));
if let Expr::FullCellPath(b) = &expr.expr { if let [a, b] = &b.tail[..] {
assert!(matches!( if let PathMember::String { val, optional, .. } = a {
b.head, assert_eq!(val, "bar");
Expression { assert_eq!(optional, &true);
expr: Expr::Var(_),
..
}
));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
assert_eq!(optional, &true);
} else {
panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
} else { } else {
panic!("cell path tail is unexpected") panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
} }
} else { } else {
panic!("Not a cell path"); panic!("cell path tail is unexpected")
} }
} else { } else {
panic!("Not an expression") panic!("Not a cell path");
} }
} }
@ -441,13 +403,11 @@ pub fn parse_binary_with_hex_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::Binary(vec![0x13])) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::Binary(vec![0x13]));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -459,13 +419,11 @@ pub fn parse_binary_with_incomplete_hex_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::Binary(vec![0x03])) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::Binary(vec![0x03]));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -477,13 +435,11 @@ pub fn parse_binary_with_binary_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::Binary(vec![0b10101000])) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::Binary(vec![0b10101000]));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -495,13 +451,11 @@ pub fn parse_binary_with_incomplete_binary_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::Binary(vec![0b00000010])) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::Binary(vec![0b00000010]));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -513,13 +467,11 @@ pub fn parse_binary_with_octal_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::Binary(vec![0o250])) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::Binary(vec![0o250]));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -531,13 +483,11 @@ pub fn parse_binary_with_incomplete_octal_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::Binary(vec![0o2])) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::Binary(vec![0o2]));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -549,13 +499,11 @@ pub fn parse_binary_with_invalid_octal_format() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert!(!matches!(&expr.expr, Expr::Binary(_))) assert!(element.redirection.is_none());
} else { assert!(!matches!(element.expr.expr, Expr::Binary(_)));
panic!("Not an expression")
}
} }
#[test] #[test]
@ -569,13 +517,11 @@ pub fn parse_binary_with_multi_byte_char() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert!(!matches!(&expr.expr, Expr::Binary(_))) assert!(element.redirection.is_none());
} else { assert!(!matches!(element.expr.expr, Expr::Binary(_)))
panic!("Not an expression")
}
} }
#[test] #[test]
@ -591,17 +537,12 @@ pub fn parse_call() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression( if let Expr::Call(call) = &element.expr.expr {
_,
Expression {
expr: Expr::Call(call),
..
},
) = &expressions.elements[0]
{
assert_eq!(call.decl_id, 0); assert_eq!(call.decl_id, 0);
} }
} }
@ -650,17 +591,12 @@ pub fn parse_call_short_flag_batch_arg_allowed() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression( if let Expr::Call(call) = &element.expr.expr {
_,
Expression {
expr: Expr::Call(call),
..
},
) = &expressions.elements[0]
{
assert_eq!(call.decl_id, 0); assert_eq!(call.decl_id, 0);
assert_eq!(call.arguments.len(), 2); assert_eq!(call.arguments.len(), 2);
matches!(call.arguments[0], Argument::Named((_, None, None))); matches!(call.arguments[0], Argument::Named((_, None, None)));
@ -767,42 +703,28 @@ fn test_nothing_comparison_eq() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
assert!(matches!( let element = &pipeline.elements[0];
&expressions.elements[0], assert!(element.redirection.is_none());
PipelineElement::Expression( assert!(matches!(&element.expr.expr, Expr::BinaryOp(..)));
_,
Expression {
expr: Expr::BinaryOp(..),
..
}
)
))
} }
#[rstest] #[rstest]
#[case(b"let a = 1 err> /dev/null", "RedirectionInLetMut")] #[case(b"let a = 1 err> /dev/null")]
#[case(b"let a = 1 out> /dev/null", "RedirectionInLetMut")] #[case(b"let a = 1 out> /dev/null")]
#[case(b"mut a = 1 err> /dev/null", "RedirectionInLetMut")] #[case(b"mut a = 1 err> /dev/null")]
#[case(b"mut a = 1 out> /dev/null", "RedirectionInLetMut")] #[case(b"mut a = 1 out> /dev/null")]
// This two cases cause AssignInPipeline instead of RedirectionInLetMut #[case(b"let a = 1 out+err> /dev/null")]
#[case(b"let a = 1 out+err> /dev/null", "AssignInPipeline")] #[case(b"mut a = 1 out+err> /dev/null")]
#[case(b"mut a = 1 out+err> /dev/null", "AssignInPipeline")] fn test_redirection_with_letmut(#[case] phase: &[u8]) {
fn test_redirection_with_letmut(#[case] phase: &[u8], #[case] expected: &str) {
let engine_state = EngineState::new(); let engine_state = EngineState::new();
let mut working_set = StateWorkingSet::new(&engine_state); let mut working_set = StateWorkingSet::new(&engine_state);
let _block = parse(&mut working_set, None, phase, true); let _block = parse(&mut working_set, None, phase, true);
match expected { assert!(matches!(
"RedirectionInLetMut" => assert!(matches!( working_set.parse_errors.first(),
working_set.parse_errors.first(), Some(ParseError::RedirectingBuiltinCommand(_, _, _))
Some(ParseError::RedirectionInLetMut(_, _)) ));
)),
"AssignInPipeline" => assert!(matches!(
working_set.parse_errors.first(),
Some(ParseError::AssignInPipeline(_, _, _, _))
)),
_ => panic!("unexpected pattern"),
}
} }
#[test] #[test]
@ -814,18 +736,11 @@ fn test_nothing_comparison_neq() {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
assert!(matches!( let element = &pipeline.elements[0];
&expressions.elements[0], assert!(element.redirection.is_none());
PipelineElement::Expression( assert!(matches!(&element.expr.expr, Expr::BinaryOp(..)));
_,
Expression {
expr: Expr::BinaryOp(..),
..
}
)
))
} }
mod string { mod string {
@ -840,13 +755,11 @@ mod string {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::String("hello nushell".to_string())) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::String("hello nushell".to_string()))
panic!("Not an expression")
}
} }
mod interpolation { mod interpolation {
@ -864,26 +777,23 @@ mod string {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let subexprs: Vec<&Expr> = match &element.expr.expr {
let subexprs: Vec<&Expr> = match expr { Expr::StringInterpolation(expressions) => {
Expression { expressions.iter().map(|e| &e.expr).collect()
expr: Expr::StringInterpolation(expressions), }
.. _ => panic!("Expected an `Expr::StringInterpolation`"),
} => expressions.iter().map(|e| &e.expr).collect(), };
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 2); assert_eq!(subexprs.len(), 2);
assert_eq!(subexprs[0], &Expr::String("hello ".to_string())); assert_eq!(subexprs[0], &Expr::String("hello ".to_string()));
assert!(matches!(subexprs[1], &Expr::FullCellPath(..))); assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
} else {
panic!("Not an expression")
}
} }
#[test] #[test]
@ -896,25 +806,21 @@ mod string {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1); let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { assert_eq!(subexprs.len(), 1);
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 1); assert_eq!(subexprs[0], &Expr::String("hello (39 + 3)".to_string()));
assert_eq!(subexprs[0], &Expr::String("hello (39 + 3)".to_string()));
} else {
panic!("Not an expression")
}
} }
#[test] #[test]
@ -927,27 +833,23 @@ mod string {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1); let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { assert_eq!(subexprs.len(), 2);
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 2); assert_eq!(subexprs[0], &Expr::String("hello \\".to_string()));
assert_eq!(subexprs[0], &Expr::String("hello \\".to_string())); assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
} else {
panic!("Not an expression")
}
} }
#[test] #[test]
@ -960,24 +862,20 @@ mod string {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1); let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { assert_eq!(subexprs.len(), 1);
let subexprs: Vec<&Expr> = match expr { assert_eq!(subexprs[0], &Expr::String("(1 + 3)(7 - 5)".to_string()));
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs[0], &Expr::String("(1 + 3)(7 - 5)".to_string()));
} else {
panic!("Not an expression")
}
} }
#[test] #[test]
@ -1084,24 +982,19 @@ mod range {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block length"); assert_eq!(block.len(), 1, "{tag}: block length");
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length"); assert_eq!(pipeline.len(), 1, "{tag}: expression length");
if let PipelineElement::Expression( let element = &pipeline.elements[0];
_, assert!(element.redirection.is_none());
Expression { if let Expr::Range(
expr: Some(_),
Expr::Range( None,
Some(_), Some(_),
None, RangeOperator {
Some(_), inclusion: the_inclusion,
RangeOperator {
inclusion: the_inclusion,
..
},
),
.. ..
}, },
) = expressions.elements[0] ) = element.expr.expr
{ {
assert_eq!( assert_eq!(
the_inclusion, inclusion, the_inclusion, inclusion,
@ -1143,24 +1036,19 @@ mod range {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 2, "{tag} block len 2"); assert_eq!(block.len(), 2, "{tag} block len 2");
let expressions = &block.pipelines[1]; let pipeline = &block.pipelines[1];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1"); assert_eq!(pipeline.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression( let element = &pipeline.elements[0];
_, assert!(element.redirection.is_none());
Expression { if let Expr::Range(
expr: Some(_),
Expr::Range( None,
Some(_), Some(_),
None, RangeOperator {
Some(_), inclusion: the_inclusion,
RangeOperator {
inclusion: the_inclusion,
..
},
),
.. ..
}, },
) = expressions.elements[0] ) = element.expr.expr
{ {
assert_eq!( assert_eq!(
the_inclusion, inclusion, the_inclusion, inclusion,
@ -1189,24 +1077,19 @@ mod range {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block len 1"); assert_eq!(block.len(), 1, "{tag}: block len 1");
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1"); assert_eq!(pipeline.len(), 1, "{tag}: expression length");
if let PipelineElement::Expression( let element = &pipeline.elements[0];
_, assert!(element.redirection.is_none());
Expression { if let Expr::Range(
expr: Some(_),
Expr::Range( None,
Some(_), None,
None, RangeOperator {
None, inclusion: the_inclusion,
RangeOperator {
inclusion: the_inclusion,
..
},
),
.. ..
}, },
) = expressions.elements[0] ) = element.expr.expr
{ {
assert_eq!( assert_eq!(
the_inclusion, inclusion, the_inclusion, inclusion,
@ -1235,24 +1118,19 @@ mod range {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block len 1"); assert_eq!(block.len(), 1, "{tag}: block len 1");
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1"); assert_eq!(pipeline.len(), 1, "{tag}: expression length");
if let PipelineElement::Expression( let element = &pipeline.elements[0];
_, assert!(element.redirection.is_none());
Expression { if let Expr::Range(
expr: None,
Expr::Range( None,
None, Some(_),
None, RangeOperator {
Some(_), inclusion: the_inclusion,
RangeOperator {
inclusion: the_inclusion,
..
},
),
.. ..
}, },
) = expressions.elements[0] ) = element.expr.expr
{ {
assert_eq!( assert_eq!(
the_inclusion, inclusion, the_inclusion, inclusion,
@ -1281,24 +1159,19 @@ mod range {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block length 1"); assert_eq!(block.len(), 1, "{tag}: block length 1");
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1"); assert_eq!(pipeline.len(), 1, "{tag}: expression length");
if let PipelineElement::Expression( let element = &pipeline.elements[0];
_, assert!(element.redirection.is_none());
Expression { if let Expr::Range(
expr: Some(_),
Expr::Range( Some(_),
Some(_), Some(_),
Some(_), RangeOperator {
Some(_), inclusion: the_inclusion,
RangeOperator {
inclusion: the_inclusion,
..
},
),
.. ..
}, },
) = expressions.elements[0] ) = element.expr.expr
{ {
assert_eq!( assert_eq!(
the_inclusion, inclusion, the_inclusion, inclusion,
@ -1671,31 +1544,21 @@ mod input_types {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 2); assert_eq!(pipeline.len(), 2);
assert!(pipeline.elements[0].redirection.is_none());
assert!(pipeline.elements[1].redirection.is_none());
match &expressions.elements[0] { match &pipeline.elements[0].expr.expr {
PipelineElement::Expression( Expr::Call(call) => {
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let expected_id = working_set.find_decl(b"ls").unwrap(); let expected_id = working_set.find_decl(b"ls").unwrap();
assert_eq!(call.decl_id, expected_id) assert_eq!(call.decl_id, expected_id)
} }
_ => panic!("Expected expression Call not found"), _ => panic!("Expected expression Call not found"),
} }
match &expressions.elements[1] { match &pipeline.elements[1].expr.expr {
PipelineElement::Expression( Expr::Call(call) => {
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let expected_id = working_set.find_decl(b"group-by").unwrap(); let expected_id = working_set.find_decl(b"group-by").unwrap();
assert_eq!(call.decl_id, expected_id) assert_eq!(call.decl_id, expected_id)
} }
@ -1718,15 +1581,10 @@ mod input_types {
engine_state.merge_delta(delta).unwrap(); engine_state.merge_delta(delta).unwrap();
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
match &expressions.elements[3] { assert!(pipeline.elements[3].redirection.is_none());
PipelineElement::Expression( match &pipeline.elements[3].expr.expr {
_, Expr::Call(call) => {
Expression {
expr: Expr::Call(call),
..
},
) => {
let arg = &call.arguments[0]; let arg = &call.arguments[0];
match arg { match arg {
Argument::Positional(a) => match &a.expr { Argument::Positional(a) => match &a.expr {
@ -1734,17 +1592,12 @@ mod input_types {
Expr::Subexpression(id) => { Expr::Subexpression(id) => {
let block = engine_state.get_block(*id); let block = engine_state.get_block(*id);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 2); assert_eq!(pipeline.len(), 2);
assert!(pipeline.elements[1].redirection.is_none());
match &expressions.elements[1] { match &pipeline.elements[1].expr.expr {
PipelineElement::Expression( Expr::Call(call) => {
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let working_set = StateWorkingSet::new(&engine_state); let working_set = StateWorkingSet::new(&engine_state);
let expected_id = working_set.find_decl(b"min").unwrap(); let expected_id = working_set.find_decl(b"min").unwrap();
assert_eq!(call.decl_id, expected_id) assert_eq!(call.decl_id, expected_id)
@ -1776,29 +1629,20 @@ mod input_types {
assert!(working_set.parse_errors.is_empty()); assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
match &expressions.elements[2] { assert!(pipeline.elements[2].redirection.is_none());
PipelineElement::Expression( assert!(pipeline.elements[3].redirection.is_none());
_,
Expression { match &pipeline.elements[2].expr.expr {
expr: Expr::Call(call), Expr::Call(call) => {
..
},
) => {
let expected_id = working_set.find_decl(b"with-column").unwrap(); let expected_id = working_set.find_decl(b"with-column").unwrap();
assert_eq!(call.decl_id, expected_id) assert_eq!(call.decl_id, expected_id)
} }
_ => panic!("Expected expression Call not found"), _ => panic!("Expected expression Call not found"),
} }
match &expressions.elements[3] { match &pipeline.elements[3].expr.expr {
PipelineElement::Expression( Expr::Call(call) => {
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let expected_id = working_set.find_decl(b"collect").unwrap(); let expected_id = working_set.find_decl(b"collect").unwrap();
assert_eq!(call.decl_id, expected_id) assert_eq!(call.decl_id, expected_id)
} }

18
crates/nu-parser/tests/test_parser_unicode_escapes.rs
View File

@ -1,13 +1,9 @@
#![cfg(test)] #![cfg(test)]
//use nu_parser::ParseError;
use nu_parser::*; use nu_parser::*;
use nu_protocol::{ use nu_protocol::{
//ast::{Expr, Expression, PipelineElement}, ast::Expr,
ast::{Expr, PipelineElement},
//engine::{Command, EngineState, Stack, StateWorkingSet},
engine::{EngineState, StateWorkingSet}, engine::{EngineState, StateWorkingSet},
//Signature, SyntaxShape,
}; };
pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) { pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) {
@ -19,13 +15,11 @@ pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) {
match working_set.parse_errors.first() { match working_set.parse_errors.first() {
None => { None => {
assert_eq!(block.len(), 1); assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0]; let pipeline = &block.pipelines[0];
assert_eq!(expressions.len(), 1); assert_eq!(pipeline.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] { let element = &pipeline.elements[0];
assert_eq!(expr.expr, Expr::String(expected.to_string())) assert!(element.redirection.is_none());
} else { assert_eq!(element.expr.expr, Expr::String(expected.to_string()));
panic!("Not an expression")
}
} }
Some(pev) => match error_contains { Some(pev) => match error_contains {
None => { None => {

34
crates/nu-plugin/src/plugin/context.rs
View File

@ -6,8 +6,8 @@ use std::{
use nu_engine::get_eval_block_with_early_return; use nu_engine::get_eval_block_with_early_return;
use nu_protocol::{ use nu_protocol::{
ast::Call, ast::Call,
engine::{Closure, EngineState, Stack}, engine::{Closure, EngineState, Redirection, Stack},
Config, IntoSpanned, PipelineData, PluginIdentity, ShellError, Span, Spanned, Value, Config, IntoSpanned, IoStream, PipelineData, PluginIdentity, ShellError, Span, Spanned, Value,
}; };
/// Object safe trait for abstracting operations required of the plugin context. /// Object safe trait for abstracting operations required of the plugin context.
@ -107,8 +107,6 @@ impl PluginExecutionContext for PluginExecutionCommandContext {
&mut stack, &mut stack,
&block, &block,
input, input,
false,
false,
) { ) {
Ok(v) => v.into_value(span), Ok(v) => v.into_value(span),
Err(e) => Value::error(e, self.call.head), Err(e) => Value::error(e, self.call.head),
@ -155,7 +153,24 @@ impl PluginExecutionContext for PluginExecutionCommandContext {
inner: vec![], inner: vec![],
})?; })?;
let mut stack = self.stack.captures_to_stack(closure.item.captures); let mut stack = self
.stack
.captures_to_stack(closure.item.captures)
.reset_pipes();
let stdout = if redirect_stdout {
Some(Redirection::Pipe(IoStream::Capture))
} else {
None
};
let stderr = if redirect_stderr {
Some(Redirection::Pipe(IoStream::Capture))
} else {
None
};
let stack = &mut stack.push_redirection(stdout, stderr);
// Set up the positional arguments // Set up the positional arguments
for (idx, value) in positional.into_iter().enumerate() { for (idx, value) in positional.into_iter().enumerate() {
@ -174,14 +189,7 @@ impl PluginExecutionContext for PluginExecutionCommandContext {
let eval_block_with_early_return = get_eval_block_with_early_return(&self.engine_state); let eval_block_with_early_return = get_eval_block_with_early_return(&self.engine_state);
eval_block_with_early_return( eval_block_with_early_return(&self.engine_state, stack, block, input)
&self.engine_state,
&mut stack,
block,
input,
redirect_stdout,
redirect_stderr,
)
} }
} }

1
crates/nu-plugin/src/plugin/declaration.rs
View File

@ -92,6 +92,7 @@ impl Command for PluginDeclaration {
// We need the current environment variables for `python` based plugins. Or // We need the current environment variables for `python` based plugins. Or
// we'll likely have a problem when a plugin is implemented in a virtual Python // we'll likely have a problem when a plugin is implemented in a virtual Python
// environment. // environment.
let stack = &mut stack.start_capture();
nu_engine::env::env_to_strings(engine_state, stack) nu_engine::env::env_to_strings(engine_state, stack)
}) })
}) })

6
crates/nu-protocol/src/alias.rs
View File

@ -1,9 +1,7 @@
use crate::engine::{EngineState, Stack};
use crate::PipelineData;
use crate::{ use crate::{
ast::{Call, Expression}, ast::{Call, Expression},
engine::Command, engine::{Command, EngineState, Stack},
ShellError, Signature, PipelineData, ShellError, Signature,
}; };
#[derive(Clone)] #[derive(Clone)]

26
crates/nu-protocol/src/ast/block.rs
View File

@ -1,5 +1,5 @@
use super::Pipeline; use super::Pipeline;
use crate::{ast::PipelineElement, Signature, Span, Type, VarId}; use crate::{engine::EngineState, IoStream, Signature, Span, Type, VarId};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
@ -19,6 +19,17 @@ impl Block {
pub fn is_empty(&self) -> bool { pub fn is_empty(&self) -> bool {
self.pipelines.is_empty() self.pipelines.is_empty()
} }
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
if let Some(first) = self.pipelines.first() {
first.stdio_redirect(engine_state)
} else {
(None, None)
}
}
} }
impl Default for Block { impl Default for Block {
@ -51,15 +62,10 @@ impl Block {
pub fn output_type(&self) -> Type { pub fn output_type(&self) -> Type {
if let Some(last) = self.pipelines.last() { if let Some(last) = self.pipelines.last() {
if let Some(last) = last.elements.last() { if let Some(last) = last.elements.last() {
match last { if last.redirection.is_some() {
PipelineElement::Expression(_, expr) => expr.ty.clone(), Type::Any
PipelineElement::ErrPipedExpression(_, expr) => expr.ty.clone(), } else {
PipelineElement::OutErrPipedExpression(_, expr) => expr.ty.clone(), last.expr.ty.clone()
PipelineElement::Redirection(_, _, _, _) => Type::Any,
PipelineElement::SeparateRedirection { .. } => Type::Any,
PipelineElement::SameTargetRedirection { .. } => Type::Any,
PipelineElement::And(_, expr) => expr.ty.clone(),
PipelineElement::Or(_, expr) => expr.ty.clone(),
} }
} else { } else {
Type::Nothing Type::Nothing

9
crates/nu-protocol/src/ast/call.rs
View File

@ -2,10 +2,9 @@ use std::collections::HashMap;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use super::Expression;
use crate::{ use crate::{
engine::StateWorkingSet, eval_const::eval_constant, DeclId, FromValue, ShellError, Span, ast::Expression, engine::StateWorkingSet, eval_const::eval_constant, DeclId, FromValue,
Spanned, Value, ShellError, Span, Spanned, Value,
}; };
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
@ -51,8 +50,6 @@ pub struct Call {
pub decl_id: DeclId, pub decl_id: DeclId,
pub head: Span, pub head: Span,
pub arguments: Vec<Argument>, pub arguments: Vec<Argument>,
pub redirect_stdout: bool,
pub redirect_stderr: bool,
/// this field is used by the parser to pass additional command-specific information /// this field is used by the parser to pass additional command-specific information
pub parser_info: HashMap<String, Expression>, pub parser_info: HashMap<String, Expression>,
} }
@ -63,8 +60,6 @@ impl Call {
decl_id: 0, decl_id: 0,
head, head,
arguments: vec![], arguments: vec![],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(), parser_info: HashMap::new(),
} }
} }

74
crates/nu-protocol/src/ast/expr.rs
View File

@ -5,7 +5,10 @@ use super::{
Call, CellPath, Expression, ExternalArgument, FullCellPath, MatchPattern, Operator, Call, CellPath, Expression, ExternalArgument, FullCellPath, MatchPattern, Operator,
RangeOperator, RangeOperator,
}; };
use crate::{ast::ImportPattern, ast::Unit, BlockId, Signature, Span, Spanned, VarId}; use crate::{
ast::ImportPattern, ast::Unit, engine::EngineState, BlockId, IoStream, Signature, Span,
Spanned, VarId,
};
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum Expr { pub enum Expr {
@ -22,7 +25,7 @@ pub enum Expr {
Var(VarId), Var(VarId),
VarDecl(VarId), VarDecl(VarId),
Call(Box<Call>), Call(Box<Call>),
ExternalCall(Box<Expression>, Vec<ExternalArgument>, bool), // head, args, is_subexpression ExternalCall(Box<Expression>, Vec<ExternalArgument>), // head, args
Operator(Operator), Operator(Operator),
RowCondition(BlockId), RowCondition(BlockId),
UnaryNot(Box<Expression>), UnaryNot(Box<Expression>),
@ -52,6 +55,73 @@ pub enum Expr {
Garbage, Garbage,
} }
impl Expr {
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
// Usages of `$in` will be wrapped by a `collect` call by the parser,
// so we do not have to worry about that when considering
// which of the expressions below may consume pipeline output.
match self {
Expr::Call(call) => engine_state.get_decl(call.decl_id).stdio_redirect(),
Expr::Subexpression(block_id) | Expr::Block(block_id) => engine_state
.get_block(*block_id)
.stdio_redirect(engine_state),
Expr::FullCellPath(cell_path) => cell_path.head.expr.stdio_redirect(engine_state),
Expr::Bool(_)
| Expr::Int(_)
| Expr::Float(_)
| Expr::Binary(_)
| Expr::Range(_, _, _, _)
| Expr::Var(_)
| Expr::UnaryNot(_)
| Expr::BinaryOp(_, _, _)
| Expr::Closure(_) // piping into a closure value, not into a closure call
| Expr::List(_)
| Expr::Table(_, _)
| Expr::Record(_)
| Expr::ValueWithUnit(_, _)
| Expr::DateTime(_)
| Expr::String(_)
| Expr::CellPath(_)
| Expr::StringInterpolation(_)
| Expr::Nothing => {
// These expressions do not use the output of the pipeline in any meaningful way,
// so we can discard the previous output by redirecting it to `Null`.
(Some(IoStream::Null), None)
}
Expr::VarDecl(_)
| Expr::Operator(_)
| Expr::Filepath(_, _)
| Expr::Directory(_, _)
| Expr::GlobPattern(_, _)
| Expr::ImportPattern(_)
| Expr::Overlay(_)
| Expr::Signature(_)
| Expr::Spread(_)
| Expr::Garbage => {
// These should be impossible to pipe to,
// but even it is, the pipeline output is not used in any way.
(Some(IoStream::Null), None)
}
Expr::RowCondition(_) | Expr::MatchBlock(_) => {
// These should be impossible to pipe to,
// but if they are, then the pipeline output could be used.
(None, None)
}
Expr::ExternalCall(_, _) => {
// No override necessary, pipes will always be created in eval
(None, None)
}
Expr::Keyword(_, _, _) => {
// Not sure about this; let's return no redirection override for now.
(None, None)
}
}
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] #[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum RecordItem { pub enum RecordItem {
/// A key: val mapping /// A key: val mapping

4
crates/nu-protocol/src/ast/expression.rs
View File

@ -184,7 +184,7 @@ impl Expression {
} }
Expr::CellPath(_) => false, Expr::CellPath(_) => false,
Expr::DateTime(_) => false, Expr::DateTime(_) => false,
Expr::ExternalCall(head, args, _) => { Expr::ExternalCall(head, args) => {
if head.has_in_variable(working_set) { if head.has_in_variable(working_set) {
return true; return true;
} }
@ -369,7 +369,7 @@ impl Expression {
} }
Expr::CellPath(_) => {} Expr::CellPath(_) => {}
Expr::DateTime(_) => {} Expr::DateTime(_) => {}
Expr::ExternalCall(head, args, _) => { Expr::ExternalCall(head, args) => {
head.replace_span(working_set, replaced, new_span); head.replace_span(working_set, replaced, new_span);
for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args { for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args {
expr.replace_span(working_set, replaced, new_span); expr.replace_span(working_set, replaced, new_span);

207
crates/nu-protocol/src/ast/pipeline.rs
View File

@ -1,100 +1,56 @@
use crate::{ast::Expression, engine::StateWorkingSet, Span}; use crate::{
ast::Expression,
engine::{EngineState, StateWorkingSet},
IoStream, Span,
};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::fmt::Display;
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq)] #[derive(Debug, Clone, Copy, Serialize, Deserialize, Eq, PartialEq)]
pub enum Redirection { pub enum RedirectionSource {
Stdout, Stdout,
Stderr, Stderr,
StdoutAndStderr, StdoutAndStderr,
} }
// Note: Span in the below is for the span of the connector not the whole element impl Display for RedirectionSource {
#[derive(Debug, Clone, Serialize, Deserialize)] fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
pub enum PipelineElement { f.write_str(match self {
Expression(Option<Span>, Expression), RedirectionSource::Stdout => "stdout",
ErrPipedExpression(Option<Span>, Expression), RedirectionSource::Stderr => "stderr",
OutErrPipedExpression(Option<Span>, Expression), RedirectionSource::StdoutAndStderr => "stdout and stderr",
// final field indicates if it's in append mode })
Redirection(Span, Redirection, Expression, bool), }
// final bool field indicates if it's in append mode
SeparateRedirection {
out: (Span, Expression, bool),
err: (Span, Expression, bool),
},
// redirection's final bool field indicates if it's in append mode
SameTargetRedirection {
cmd: (Option<Span>, Expression),
redirection: (Span, Expression, bool),
},
And(Span, Expression),
Or(Span, Expression),
} }
impl PipelineElement { #[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub fn expression(&self) -> &Expression { pub enum RedirectionTarget {
File {
expr: Expression,
append: bool,
span: Span,
},
Pipe {
span: Span,
},
}
impl RedirectionTarget {
pub fn span(&self) -> Span {
match self { match self {
PipelineElement::Expression(_, expression) RedirectionTarget::File { span, .. } | RedirectionTarget::Pipe { span } => *span,
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression) => expression,
PipelineElement::Redirection(_, _, expression, _) => expression,
PipelineElement::SeparateRedirection {
out: (_, expression, _),
..
} => expression,
PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression,
PipelineElement::And(_, expression) => expression,
PipelineElement::Or(_, expression) => expression,
} }
} }
pub fn span(&self) -> Span { pub fn expr(&self) -> Option<&Expression> {
match self { match self {
PipelineElement::Expression(None, expression) RedirectionTarget::File { expr, .. } => Some(expr),
| PipelineElement::ErrPipedExpression(None, expression) RedirectionTarget::Pipe { .. } => None,
| PipelineElement::OutErrPipedExpression(None, expression)
| PipelineElement::SameTargetRedirection {
cmd: (None, expression),
..
} => expression.span,
PipelineElement::Expression(Some(span), expression)
| PipelineElement::ErrPipedExpression(Some(span), expression)
| PipelineElement::OutErrPipedExpression(Some(span), expression)
| PipelineElement::Redirection(span, _, expression, _)
| PipelineElement::SeparateRedirection {
out: (span, expression, _),
..
}
| PipelineElement::And(span, expression)
| PipelineElement::Or(span, expression)
| PipelineElement::SameTargetRedirection {
cmd: (Some(span), expression),
..
} => Span {
start: span.start,
end: expression.span.end,
},
} }
} }
pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool { pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool {
match self { self.expr().is_some_and(|e| e.has_in_variable(working_set))
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} => expression.has_in_variable(working_set),
PipelineElement::SeparateRedirection {
out: (_, out_expr, _),
err: (_, err_expr, _),
} => out_expr.has_in_variable(working_set) || err_expr.has_in_variable(working_set),
}
} }
pub fn replace_span( pub fn replace_span(
@ -104,24 +60,72 @@ impl PipelineElement {
new_span: Span, new_span: Span,
) { ) {
match self { match self {
PipelineElement::Expression(_, expression) RedirectionTarget::File { expr, .. } => {
| PipelineElement::ErrPipedExpression(_, expression) expr.replace_span(working_set, replaced, new_span)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression)
| PipelineElement::SameTargetRedirection {
cmd: (_, expression),
..
} }
| PipelineElement::SeparateRedirection { RedirectionTarget::Pipe { .. } => {}
out: (_, expression, _),
..
} => expression.replace_span(working_set, replaced, new_span),
} }
} }
} }
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub enum PipelineRedirection {
Single {
source: RedirectionSource,
target: RedirectionTarget,
},
Separate {
out: RedirectionTarget,
err: RedirectionTarget,
},
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct PipelineElement {
pub pipe: Option<Span>,
pub expr: Expression,
pub redirection: Option<PipelineRedirection>,
}
impl PipelineElement {
pub fn has_in_variable(&self, working_set: &StateWorkingSet) -> bool {
self.expr.has_in_variable(working_set)
|| self.redirection.as_ref().is_some_and(|r| match r {
PipelineRedirection::Single { target, .. } => target.has_in_variable(working_set),
PipelineRedirection::Separate { out, err } => {
out.has_in_variable(working_set) || err.has_in_variable(working_set)
}
})
}
pub fn replace_span(
&mut self,
working_set: &mut StateWorkingSet,
replaced: Span,
new_span: Span,
) {
self.expr.replace_span(working_set, replaced, new_span);
if let Some(expr) = self.redirection.as_mut() {
match expr {
PipelineRedirection::Single { target, .. } => {
target.replace_span(working_set, replaced, new_span)
}
PipelineRedirection::Separate { out, err } => {
out.replace_span(working_set, replaced, new_span);
err.replace_span(working_set, replaced, new_span);
}
}
}
}
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
self.expr.expr.stdio_redirect(engine_state)
}
}
#[derive(Debug, Clone, Serialize, Deserialize)] #[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Pipeline { pub struct Pipeline {
pub elements: Vec<PipelineElement>, pub elements: Vec<PipelineElement>,
@ -143,8 +147,10 @@ impl Pipeline {
elements: expressions elements: expressions
.into_iter() .into_iter()
.enumerate() .enumerate()
.map(|(idx, x)| { .map(|(idx, expr)| PipelineElement {
PipelineElement::Expression(if idx == 0 { None } else { Some(x.span) }, x) pipe: if idx == 0 { None } else { Some(expr.span) },
expr,
redirection: None,
}) })
.collect(), .collect(),
} }
@ -157,4 +163,15 @@ impl Pipeline {
pub fn is_empty(&self) -> bool { pub fn is_empty(&self) -> bool {
self.elements.is_empty() self.elements.is_empty()
} }
pub fn stdio_redirect(
&self,
engine_state: &EngineState,
) -> (Option<IoStream>, Option<IoStream>) {
if let Some(first) = self.elements.first() {
first.stdio_redirect(engine_state)
} else {
(None, None)
}
}
} }

13
crates/nu-protocol/src/debugger/profiler.rs
View File

@ -142,19 +142,14 @@ impl Debugger for Profiler {
}; };
let expr_opt = if self.collect_exprs { let expr_opt = if self.collect_exprs {
Some(match element { Some(expr_to_string(engine_state, &element.expr.expr))
PipelineElement::Expression(_, expression) => {
expr_to_string(engine_state, &expression.expr)
}
_ => "other".to_string(),
})
} else { } else {
None None
}; };
let new_id = ElementId(self.elements.len()); let new_id = ElementId(self.elements.len());
let mut new_element = ElementInfo::new(self.depth, element.span()); let mut new_element = ElementInfo::new(self.depth, element.expr.span);
new_element.expr = expr_opt; new_element.expr = expr_opt;
self.elements.push(new_element); self.elements.push(new_element);
@ -178,7 +173,7 @@ impl Debugger for Profiler {
return; return;
} }
let element_span = element.span(); let element_span = element.expr.span;
let out_opt = if self.collect_values { let out_opt = if self.collect_values {
Some(match result { Some(match result {
@ -250,7 +245,7 @@ fn expr_to_string(engine_state: &EngineState, expr: &Expr) -> String {
Expr::Closure(_) => "closure".to_string(), Expr::Closure(_) => "closure".to_string(),
Expr::DateTime(_) => "datetime".to_string(), Expr::DateTime(_) => "datetime".to_string(),
Expr::Directory(_, _) => "directory".to_string(), Expr::Directory(_, _) => "directory".to_string(),
Expr::ExternalCall(_, _, _) => "external call".to_string(), Expr::ExternalCall(_, _) => "external call".to_string(),
Expr::Filepath(_, _) => "filepath".to_string(), Expr::Filepath(_, _) => "filepath".to_string(),
Expr::Float(_) => "float".to_string(), Expr::Float(_) => "float".to_string(),
Expr::FullCellPath(full_cell_path) => { Expr::FullCellPath(full_cell_path) => {

6
crates/nu-protocol/src/engine/command.rs
View File

@ -1,4 +1,4 @@
use crate::{ast::Call, Alias, BlockId, Example, PipelineData, ShellError, Signature}; use crate::{ast::Call, Alias, BlockId, Example, IoStream, PipelineData, ShellError, Signature};
use super::{EngineState, Stack, StateWorkingSet}; use super::{EngineState, Stack, StateWorkingSet};
@ -133,6 +133,10 @@ pub trait Command: Send + Sync + CommandClone {
_ => CommandType::Other, _ => CommandType::Other,
} }
} }
fn stdio_redirect(&self) -> (Option<IoStream>, Option<IoStream>) {
(None, None)
}
} }
pub trait CommandClone { pub trait CommandClone {

2
crates/nu-protocol/src/engine/mod.rs
View File

@ -7,6 +7,7 @@ mod pattern_match;
mod stack; mod stack;
mod state_delta; mod state_delta;
mod state_working_set; mod state_working_set;
mod stdio;
mod usage; mod usage;
mod variable; mod variable;
@ -19,4 +20,5 @@ pub use pattern_match::*;
pub use stack::*; pub use stack::*;
pub use state_delta::*; pub use state_delta::*;
pub use state_working_set::*; pub use state_working_set::*;
pub use stdio::*;
pub use variable::*; pub use variable::*;

127
crates/nu-protocol/src/engine/stack.rs
View File

@ -1,10 +1,12 @@
use std::collections::{HashMap, HashSet}; use std::collections::{HashMap, HashSet};
use std::sync::Arc; use std::sync::Arc;
use crate::engine::EngineState; use crate::{
use crate::engine::DEFAULT_OVERLAY_NAME; engine::{EngineState, DEFAULT_OVERLAY_NAME},
use crate::{ShellError, Span, Value, VarId}; IoStream, ShellError, Span, Value, VarId, ENV_VARIABLE_ID, NU_VARIABLE_ID,
use crate::{ENV_VARIABLE_ID, NU_VARIABLE_ID}; };
use super::{Redirection, StackCallArgGuard, StackCaptureGuard, StackIoGuard, StackStdio};
/// Environment variables per overlay /// Environment variables per overlay
pub type EnvVars = HashMap<String, HashMap<String, Value>>; pub type EnvVars = HashMap<String, HashMap<String, Value>>;
@ -37,22 +39,36 @@ pub struct Stack {
/// List of active overlays /// List of active overlays
pub active_overlays: Vec<String>, pub active_overlays: Vec<String>,
pub recursion_count: u64, pub recursion_count: u64,
pub parent_stack: Option<Arc<Stack>>, pub parent_stack: Option<Arc<Stack>>,
/// Variables that have been deleted (this is used to hide values from parent stack lookups) /// Variables that have been deleted (this is used to hide values from parent stack lookups)
pub parent_deletions: Vec<VarId>, pub parent_deletions: Vec<VarId>,
pub(crate) stdio: StackStdio,
}
impl Default for Stack {
fn default() -> Self {
Self::new()
}
} }
impl Stack { impl Stack {
pub fn new() -> Stack { /// Create a new stack.
Stack { ///
vars: vec![], /// Stdio will be set to [`IoStream::Inherit`]. So, if the last command is an external command,
env_vars: vec![], /// then its output will be forwarded to the terminal/stdio streams.
///
/// Use [`Stack::capture`] afterwards if you need to evaluate an expression to a [`Value`](crate::Value)
/// (as opposed to a [`PipelineData`](crate::PipelineData)).
pub fn new() -> Self {
Self {
vars: Vec::new(),
env_vars: Vec::new(),
env_hidden: HashMap::new(), env_hidden: HashMap::new(),
active_overlays: vec![DEFAULT_OVERLAY_NAME.to_string()], active_overlays: vec![DEFAULT_OVERLAY_NAME.to_string()],
recursion_count: 0, recursion_count: 0,
parent_stack: None, parent_stack: None,
parent_deletions: vec![], parent_deletions: vec![],
stdio: StackStdio::new(),
} }
} }
@ -82,9 +98,10 @@ impl Stack {
env_hidden: parent.env_hidden.clone(), env_hidden: parent.env_hidden.clone(),
active_overlays: parent.active_overlays.clone(), active_overlays: parent.active_overlays.clone(),
recursion_count: parent.recursion_count, recursion_count: parent.recursion_count,
parent_stack: Some(parent),
vars: vec![], vars: vec![],
parent_deletions: vec![], parent_deletions: vec![],
stdio: parent.stdio.clone(),
parent_stack: Some(parent),
} }
} }
@ -235,6 +252,10 @@ impl Stack {
} }
pub fn captures_to_stack(&self, captures: Vec<(VarId, Value)>) -> Stack { pub fn captures_to_stack(&self, captures: Vec<(VarId, Value)>) -> Stack {
self.captures_to_stack_preserve_stdio(captures).capture()
}
pub fn captures_to_stack_preserve_stdio(&self, captures: Vec<(VarId, Value)>) -> Stack {
// FIXME: this is probably slow // FIXME: this is probably slow
let mut env_vars = self.env_vars.clone(); let mut env_vars = self.env_vars.clone();
env_vars.push(HashMap::new()); env_vars.push(HashMap::new());
@ -247,6 +268,7 @@ impl Stack {
recursion_count: self.recursion_count, recursion_count: self.recursion_count,
parent_stack: None, parent_stack: None,
parent_deletions: vec![], parent_deletions: vec![],
stdio: self.stdio.clone(),
} }
} }
@ -276,6 +298,7 @@ impl Stack {
recursion_count: self.recursion_count, recursion_count: self.recursion_count,
parent_stack: None, parent_stack: None,
parent_deletions: vec![], parent_deletions: vec![],
stdio: self.stdio.clone(),
} }
} }
@ -481,11 +504,87 @@ impl Stack {
pub fn remove_overlay(&mut self, name: &str) { pub fn remove_overlay(&mut self, name: &str) {
self.active_overlays.retain(|o| o != name); self.active_overlays.retain(|o| o != name);
} }
}
impl Default for Stack { /// Returns the [`IoStream`] to use for the current command's stdout.
fn default() -> Self { ///
Self::new() /// This will be the pipe redirection if one is set,
/// otherwise it will be the current file redirection,
/// otherwise it will be the process's stdout indicated by [`IoStream::Inherit`].
pub fn stdout(&self) -> &IoStream {
self.stdio.stdout()
}
/// Returns the [`IoStream`] to use for the current command's stderr.
///
/// This will be the pipe redirection if one is set,
/// otherwise it will be the current file redirection,
/// otherwise it will be the process's stderr indicated by [`IoStream::Inherit`].
pub fn stderr(&self) -> &IoStream {
self.stdio.stderr()
}
/// Returns the [`IoStream`] to use for the last command's stdout.
pub fn pipe_stdout(&self) -> Option<&IoStream> {
self.stdio.pipe_stdout.as_ref()
}
/// Returns the [`IoStream`] to use for the last command's stderr.
pub fn pipe_stderr(&self) -> Option<&IoStream> {
self.stdio.pipe_stderr.as_ref()
}
/// Temporarily set the pipe stdout redirection to [`IoStream::Capture`].
///
/// This is used before evaluating an expression into a `Value`.
pub fn start_capture(&mut self) -> StackCaptureGuard {
StackCaptureGuard::new(self)
}
/// Temporarily use the stdio redirections in the parent scope.
///
/// This is used before evaluating an argument to a call.
pub fn use_call_arg_stdio(&mut self) -> StackCallArgGuard {
StackCallArgGuard::new(self)
}
/// Temporarily apply redirections to stdout and/or stderr.
pub fn push_redirection(
&mut self,
stdout: Option<Redirection>,
stderr: Option<Redirection>,
) -> StackIoGuard {
StackIoGuard::new(self, stdout, stderr)
}
/// Mark stdout for the last command as [`IoStream::Capture`].
///
/// This will irreversibly alter the stdio redirections, and so it only makes sense to use this on an owned `Stack`
/// (which is why this function does not take `&mut self`).
///
/// See [`Stack::start_capture`] which can temporarily set stdout as [`IoStream::Capture`] for a mutable `Stack` reference.
pub fn capture(mut self) -> Self {
self.stdio.pipe_stdout = Some(IoStream::Capture);
self.stdio.pipe_stderr = None;
self
}
/// Clears any pipe and file redirections and resets stdout and stderr to [`IoStream::Inherit`].
///
/// This will irreversibly reset the stdio redirections, and so it only makes sense to use this on an owned `Stack`
/// (which is why this function does not take `&mut self`).
pub fn reset_stdio(mut self) -> Self {
self.stdio = StackStdio::new();
self
}
/// Clears any pipe redirections, keeping the current stdout and stderr.
///
/// This will irreversibly reset some of the stdio redirections, and so it only makes sense to use this on an owned `Stack`
/// (which is why this function does not take `&mut self`).
pub fn reset_pipes(mut self) -> Self {
self.stdio.pipe_stdout = None;
self.stdio.pipe_stderr = None;
self
} }
} }

Some files were not shown because too many files have changed in this diff Show More