extern/nushell
1
0
Fork 1
mirror of https://github.com/nushell/nushell.git synced 2025-07-01 07:00:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

IO and redirection overhaul (#11934)

Browse Source 
# Description
The PR overhauls how IO redirection is handled, allowing more explicit
and fine-grain control over `stdout` and `stderr` output as well as more
efficient IO and piping.

To summarize the changes in this PR:
- Added a new `IoStream` type to indicate the intended destination for a
pipeline element's `stdout` and `stderr`.
- The `stdout` and `stderr` `IoStream`s are stored in the `Stack` and to
avoid adding 6 additional arguments to every eval function and
`Command::run`. The `stdout` and `stderr` streams can be temporarily
overwritten through functions on `Stack` and these functions will return
a guard that restores the original `stdout` and `stderr` when dropped.
- In the AST, redirections are now directly part of a `PipelineElement`
as a `Option<Redirection>` field instead of having multiple different
`PipelineElement` enum variants for each kind of redirection. This
required changes to the parser, mainly in `lite_parser.rs`.
- `Command`s can also set a `IoStream` override/redirection which will
apply to the previous command in the pipeline. This is used, for
example, in `ignore` to allow the previous external command to have its
stdout redirected to `Stdio::null()` at spawn time. In contrast, the
current implementation has to create an os pipe and manually consume the
output on nushell's side. File and pipe redirections (`o>`, `e>`, `e>|`,
etc.) have precedence over overrides from commands.

This PR improves piping and IO speed, partially addressing #10763. Using
the `throughput` command from that issue, this PR gives the following
speedup on my setup for the commands below:
| Command | Before (MB/s) | After (MB/s) | Bash (MB/s) |
| --------------------------- | -------------:| ------------:|
-----------:|
| `throughput o> /dev/null` | 1169 | 52938 | 54305 |
| `throughput \| ignore` | 840 | 55438 | N/A |
| `throughput \| null` | Error | 53617 | N/A |
| `throughput \| rg 'x'` | 1165 | 3049 | 3736 |
| `(throughput) \| rg 'x'` | 810 | 3085 | 3815 |

(Numbers above are the median samples for throughput)

This PR also paves the way to refactor our `ExternalStream` handling in
the various commands. For example, this PR already fixes the following
code:
```nushell
^sh -c 'echo -n "hello "; sleep 0; echo "world"' | find "hello world"
```
This returns an empty list on 0.90.1 and returns a highlighted "hello
world" on this PR.

Since the `stdout` and `stderr` `IoStream`s are available to commands
when they are run, then this unlocks the potential for more convenient
behavior. E.g., the `find` command can disable its ansi highlighting if
it detects that the output `IoStream` is not the terminal. Knowing the
output streams will also allow background job output to be redirected
more easily and efficiently.

# User-Facing Changes
- External commands returned from closures will be collected (in most
cases):
  ```nushell
  1..2 | each {|_| nu -c "print a" }
  ```
This gives `["a", "a"]` on this PR, whereas this used to print "a\na\n"
and then return an empty list.

  ```nushell
  1..2 | each {|_| nu -c "print -e a" }
  ```
This gives `["", ""]` and prints "a\na\n" to stderr, whereas this used
to return an empty list and print "a\na\n" to stderr.

- Trailing new lines are always trimmed for external commands when
piping into internal commands or collecting it as a value. (Failure to
decode the output as utf-8 will keep the trailing newline for the last
binary value.) In the current nushell version, the following three code
snippets differ only in parenthesis placement, but they all also have
different outputs:

  1. `1..2 | each { ^echo a }`
     ```
     a
     a
     ╭────────────╮
     │ empty list │
     ╰────────────╯
     ```
  2. `1..2 | each { (^echo a) }`
     ```
     ╭───┬───╮
     │ 0 │ a │
     │ 1 │ a │
     ╰───┴───╯
     ```
  3. `1..2 | (each { ^echo a })`
     ```
     ╭───┬───╮
     │ 0 │ a │
     │   │   │
     │ 1 │ a │
     │   │   │
     ╰───┴───╯
     ```

  But in this PR, the above snippets will all have the same output:
  ```
  ╭───┬───╮
  │ 0 │ a │
  │ 1 │ a │
  ╰───┴───╯
  ```

- All existing flags on `run-external` are now deprecated.

- File redirections now apply to all commands inside a code block:
  ```nushell
  (nu -c "print -e a"; nu -c "print -e b") e> test.out
  ```
This gives "a\nb\n" in `test.out` and prints nothing. The same result
would happen when printing to stdout and using a `o>` file redirection.

- External command output will (almost) never be ignored, and ignoring
output must be explicit now:
  ```nushell
  (^echo a; ^echo b)
  ```
This prints "a\nb\n", whereas this used to print only "b\n". This only
applies to external commands; values and internal commands not in return
position will not print anything (e.g., `(echo a; echo b)` still only
prints "b").

- `complete` now always captures stderr (`do` is not necessary).

# After Submitting
The language guide and other documentation will need to be updated.
This commit is contained in:
Ian Manske
2024-03-14 20:51:55 +00:00
committed by GitHub
parent e2907e7e3a
commit b6c7656194
113 changed files with 3272 additions and 4022 deletions
Download Patch File Download Diff File Expand all files Collapse all files

87
crates/nu-parser/src/flatten.rs
View File

@ -1,6 +1,6 @@
use nu_protocol::ast::{
Argument, Block, Expr, Expression, ExternalArgument, ImportPatternMember, MatchPattern,
PathMember, Pattern, Pipeline, PipelineElement, RecordItem,
PathMember, Pattern, Pipeline, PipelineElement, PipelineRedirection, RecordItem,
};
use nu_protocol::{engine::StateWorkingSet, Span};
use nu_protocol::{DeclId, VarId};
@ -223,7 +223,7 @@ pub fn flatten_expression(
output.extend(args);
output
}
Expr::ExternalCall(head, args, _) => {
Expr::ExternalCall(head, args) => {
let mut output = vec![];
match **head {
@ -559,59 +559,42 @@ pub fn flatten_pipeline_element(
working_set: &StateWorkingSet,
pipeline_element: &PipelineElement,
) -> Vec<(Span, FlatShape)> {
match pipeline_element {
PipelineElement::Expression(span, expr)
| PipelineElement::ErrPipedExpression(span, expr)
| PipelineElement::OutErrPipedExpression(span, expr) => {
if let Some(span) = span {
let mut output = vec![(*span, FlatShape::Pipe)];
output.append(&mut flatten_expression(working_set, expr));
output
} else {
flatten_expression(working_set, expr)
let mut output = if let Some(span) = pipeline_element.pipe {
let mut output = vec![(span, FlatShape::Pipe)];
output.extend(flatten_expression(working_set, &pipeline_element.expr));
output
} else {
flatten_expression(working_set, &pipeline_element.expr)
};
if let Some(redirection) = pipeline_element.redirection.as_ref() {
match redirection {
PipelineRedirection::Single { target, .. } => {
output.push((target.span(), FlatShape::Redirection));
if let Some(expr) = target.expr() {
output.extend(flatten_expression(working_set, expr));
}
}
PipelineRedirection::Separate { out, err } => {
let (out, err) = if out.span() <= err.span() {
(out, err)
} else {
(err, out)
};
output.push((out.span(), FlatShape::Redirection));
if let Some(expr) = out.expr() {
output.extend(flatten_expression(working_set, expr));
}
output.push((err.span(), FlatShape::Redirection));
if let Some(expr) = err.expr() {
output.extend(flatten_expression(working_set, expr));
}
}
}
PipelineElement::Redirection(span, _, expr, _) => {
let mut output = vec![(*span, FlatShape::Redirection)];
output.append(&mut flatten_expression(working_set, expr));
output
}
PipelineElement::SeparateRedirection {
out: (out_span, out_expr, _),
err: (err_span, err_expr, _),
} => {
let mut output = vec![(*out_span, FlatShape::Redirection)];
output.append(&mut flatten_expression(working_set, out_expr));
output.push((*err_span, FlatShape::Redirection));
output.append(&mut flatten_expression(working_set, err_expr));
output
}
PipelineElement::SameTargetRedirection {
cmd: (cmd_span, cmd_expr),
redirection: (redirect_span, redirect_expr, _),
} => {
let mut output = if let Some(span) = cmd_span {
let mut output = vec![(*span, FlatShape::Pipe)];
output.append(&mut flatten_expression(working_set, cmd_expr));
output
} else {
flatten_expression(working_set, cmd_expr)
};
output.push((*redirect_span, FlatShape::Redirection));
output.append(&mut flatten_expression(working_set, redirect_expr));
output
}
PipelineElement::And(span, expr) => {
let mut output = vec![(*span, FlatShape::And)];
output.append(&mut flatten_expression(working_set, expr));
output
}
PipelineElement::Or(span, expr) => {
let mut output = vec![(*span, FlatShape::Or)];
output.append(&mut flatten_expression(working_set, expr));
output
}
}
output
}
pub fn flatten_pipeline(

25
crates/nu-parser/src/known_external.rs
View File

@ -4,7 +4,7 @@ use nu_protocol::{
engine::Command,
ShellError, Signature,
};
use nu_protocol::{PipelineData, Spanned, Type};
use nu_protocol::{PipelineData, Type};
#[derive(Clone)]
pub struct KnownExternal {
@ -42,7 +42,6 @@ impl Command for KnownExternal {
call: &Call,
input: PipelineData,
) -> Result<PipelineData, ShellError> {
let call_span = call.span();
let head_span = call.head;
let decl_id = engine_state
.find_decl("run-external".as_bytes(), &[])
@ -110,28 +109,6 @@ impl Command for KnownExternal {
}
}
if call.redirect_stdout {
extern_call.add_named((
Spanned {
item: "redirect-stdout".into(),
span: call_span,
},
None,
None,
))
}
if call.redirect_stderr {
extern_call.add_named((
Spanned {
item: "redirect-stderr".into(),
span: call_span,
},
None,
None,
))
}
command.run(engine_state, stack, &extern_call, input)
}
}

2
crates/nu-parser/src/lib.rs
View File

@ -17,7 +17,7 @@ pub use flatten::{
};
pub use known_external::KnownExternal;
pub use lex::{lex, lex_signature, Token, TokenContents};
pub use lite_parser::{lite_parse, LiteBlock, LiteElement};
pub use lite_parser::{lite_parse, LiteBlock, LiteCommand};
pub use parse_keywords::*;
pub use parser_path::*;

696
crates/nu-parser/src/lite_parser.rs
View File

@ -1,212 +1,135 @@
/// Lite parsing converts a flat stream of tokens from the lexer to a syntax element structure that
/// can be parsed.
//! Lite parsing converts a flat stream of tokens from the lexer to a syntax element structure that
//! can be parsed.
use std::mem;
use crate::{Token, TokenContents};
use nu_protocol::{ast::Redirection, ParseError, Span};
use nu_protocol::{ast::RedirectionSource, ParseError, Span};
#[derive(Debug)]
pub struct LiteCommand {
pub comments: Vec<Span>,
pub parts: Vec<Span>,
#[derive(Debug, Clone, Copy)]
pub enum LiteRedirectionTarget {
File {
connector: Span,
file: Span,
append: bool,
},
Pipe {
connector: Span,
},
}
impl Default for LiteCommand {
fn default() -> Self {
Self::new()
impl LiteRedirectionTarget {
pub fn connector(&self) -> Span {
match self {
LiteRedirectionTarget::File { connector, .. }
| LiteRedirectionTarget::Pipe { connector } => *connector,
}
}
}
#[derive(Debug, Clone)]
pub enum LiteRedirection {
Single {
source: RedirectionSource,
target: LiteRedirectionTarget,
},
Separate {
out: LiteRedirectionTarget,
err: LiteRedirectionTarget,
},
}
#[derive(Debug, Clone, Default)]
pub struct LiteCommand {
pub pipe: Option<Span>,
pub comments: Vec<Span>,
pub parts: Vec<Span>,
pub redirection: Option<LiteRedirection>,
}
impl LiteCommand {
pub fn new() -> Self {
Self {
comments: vec![],
parts: vec![],
}
}
pub fn push(&mut self, span: Span) {
fn push(&mut self, span: Span) {
self.parts.push(span);
}
pub fn is_empty(&self) -> bool {
self.parts.is_empty()
fn try_add_redirection(
&mut self,
source: RedirectionSource,
target: LiteRedirectionTarget,
) -> Result<(), ParseError> {
let redirection = match (self.redirection.take(), source) {
(None, source) => Ok(LiteRedirection::Single { source, target }),
(
Some(LiteRedirection::Single {
source: RedirectionSource::Stdout,
target: out,
}),
RedirectionSource::Stderr,
) => Ok(LiteRedirection::Separate { out, err: target }),
(
Some(LiteRedirection::Single {
source: RedirectionSource::Stderr,
target: err,
}),
RedirectionSource::Stdout,
) => Ok(LiteRedirection::Separate { out: target, err }),
(
Some(LiteRedirection::Single {
source,
target: first,
}),
_,
) => Err(ParseError::MultipleRedirections(
source,
first.connector(),
target.connector(),
)),
(
Some(LiteRedirection::Separate { out, .. }),
RedirectionSource::Stdout | RedirectionSource::StdoutAndStderr,
) => Err(ParseError::MultipleRedirections(
RedirectionSource::Stdout,
out.connector(),
target.connector(),
)),
(Some(LiteRedirection::Separate { err, .. }), RedirectionSource::Stderr) => {
Err(ParseError::MultipleRedirections(
RedirectionSource::Stderr,
err.connector(),
target.connector(),
))
}
}?;
self.redirection = Some(redirection);
Ok(())
}
}
// Note: the Span is the span of the connector not the whole element
#[derive(Debug)]
pub enum LiteElement {
Command(Option<Span>, LiteCommand),
// Similar to LiteElement::Command, except the previous command's output is stderr piped.
// e.g: `e>| cmd`
ErrPipedCommand(Option<Span>, LiteCommand),
// Similar to LiteElement::Command, except the previous command's output is stderr + stdout piped.
// e.g: `o+e>| cmd`
OutErrPipedCommand(Option<Span>, LiteCommand),
// final field indicates if it's in append mode
Redirection(Span, Redirection, LiteCommand, bool),
// SeparateRedirection variant can only be generated by two different Redirection variant
// final bool field indicates if it's in append mode
SeparateRedirection {
out: (Span, LiteCommand, bool),
err: (Span, LiteCommand, bool),
},
// SameTargetRedirection variant can only be generated by Command with Redirection::OutAndErr
// redirection's final bool field indicates if it's in append mode
SameTargetRedirection {
cmd: (Option<Span>, LiteCommand),
redirection: (Span, LiteCommand, bool),
},
}
#[derive(Debug, Default)]
#[derive(Debug, Clone, Default)]
pub struct LitePipeline {
pub commands: Vec<LiteElement>,
pub commands: Vec<LiteCommand>,
}
impl LitePipeline {
pub fn new() -> Self {
Self { commands: vec![] }
}
pub fn push(&mut self, element: LiteElement) {
self.commands.push(element);
}
pub fn insert(&mut self, index: usize, element: LiteElement) {
self.commands.insert(index, element);
}
pub fn is_empty(&self) -> bool {
self.commands.is_empty()
}
pub fn exists(&self, new_target: &Redirection) -> bool {
for cmd in &self.commands {
if let LiteElement::Redirection(_, exists_target, _, _) = cmd {
if exists_target == new_target {
return true;
}
}
fn push(&mut self, element: &mut LiteCommand) {
if !element.parts.is_empty() || element.redirection.is_some() {
self.commands.push(mem::take(element));
}
false
}
}
#[derive(Debug)]
#[derive(Debug, Clone, Default)]
pub struct LiteBlock {
pub block: Vec<LitePipeline>,
}
impl Default for LiteBlock {
fn default() -> Self {
Self::new()
}
}
impl LiteBlock {
pub fn new() -> Self {
Self { block: vec![] }
}
pub fn push(&mut self, mut pipeline: LitePipeline) {
// once we push `pipeline` to our block
// the block takes ownership of `pipeline`, which means that
// our `pipeline` is complete on collecting commands.
self.merge_redirections(&mut pipeline);
self.merge_cmd_with_outerr_redirection(&mut pipeline);
self.block.push(pipeline);
}
pub fn is_empty(&self) -> bool {
self.block.is_empty()
}
fn merge_cmd_with_outerr_redirection(&self, pipeline: &mut LitePipeline) {
let mut cmd_index = None;
let mut outerr_index = None;
for (index, cmd) in pipeline.commands.iter().enumerate() {
if let LiteElement::Command(..) = cmd {
cmd_index = Some(index);
}
if let LiteElement::Redirection(
_span,
Redirection::StdoutAndStderr,
_target_cmd,
_is_append_mode,
) = cmd
{
outerr_index = Some(index);
break;
}
}
if let (Some(cmd_index), Some(outerr_index)) = (cmd_index, outerr_index) {
// we can make sure that cmd_index is less than outerr_index.
let outerr_redirect = pipeline.commands.remove(outerr_index);
let cmd = pipeline.commands.remove(cmd_index);
// `outerr_redirect` and `cmd` should always be `LiteElement::Command` and `LiteElement::Redirection`
if let (
LiteElement::Command(cmd_span, lite_cmd),
LiteElement::Redirection(span, _, outerr_cmd, is_append_mode),
) = (cmd, outerr_redirect)
{
pipeline.insert(
cmd_index,
LiteElement::SameTargetRedirection {
cmd: (cmd_span, lite_cmd),
redirection: (span, outerr_cmd, is_append_mode),
},
)
}
}
}
fn merge_redirections(&self, pipeline: &mut LitePipeline) {
// In case our command may contains both stdout and stderr redirection.
// We pick them out and Combine them into one LiteElement::SeparateRedirection variant.
let mut stdout_index = None;
let mut stderr_index = None;
for (index, cmd) in pipeline.commands.iter().enumerate() {
if let LiteElement::Redirection(_span, redirection, _target_cmd, _is_append_mode) = cmd
{
match *redirection {
Redirection::Stderr => stderr_index = Some(index),
Redirection::Stdout => stdout_index = Some(index),
Redirection::StdoutAndStderr => {}
}
}
}
if let (Some(out_indx), Some(err_indx)) = (stdout_index, stderr_index) {
let (out_redirect, err_redirect, new_indx) = {
// to avoid panic, we need to remove commands which have larger index first.
if out_indx > err_indx {
let out_redirect = pipeline.commands.remove(out_indx);
let err_redirect = pipeline.commands.remove(err_indx);
(out_redirect, err_redirect, err_indx)
} else {
let err_redirect = pipeline.commands.remove(err_indx);
let out_redirect = pipeline.commands.remove(out_indx);
(out_redirect, err_redirect, out_indx)
}
};
// `out_redirect` and `err_redirect` should always be `LiteElement::Redirection`
if let (
LiteElement::Redirection(out_span, _, out_command, out_append_mode),
LiteElement::Redirection(err_span, _, err_command, err_append_mode),
) = (out_redirect, err_redirect)
{
// using insert with specific index to keep original
// pipeline commands order.
pipeline.insert(
new_indx,
LiteElement::SeparateRedirection {
out: (out_span, out_command, out_append_mode),
err: (err_span, err_command, err_append_mode),
},
)
}
fn push(&mut self, pipeline: &mut LitePipeline) {
if !pipeline.commands.is_empty() {
self.block.push(mem::take(pipeline));
}
}
}
@ -226,162 +149,230 @@ fn last_non_comment_token(tokens: &[Token], cur_idx: usize) -> Option<TokenConte
}
pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
let mut block = LiteBlock::new();
let mut curr_pipeline = LitePipeline::new();
let mut curr_command = LiteCommand::new();
let mut last_token = TokenContents::Eol;
let mut last_connector = TokenContents::Pipe;
let mut last_connector_span: Option<Span> = None;
if tokens.is_empty() {
return (LiteBlock::new(), None);
return (LiteBlock::default(), None);
}
let mut curr_comment: Option<Vec<Span>> = None;
let mut block = LiteBlock::default();
let mut pipeline = LitePipeline::default();
let mut command = LiteCommand::default();
let mut last_token = TokenContents::Eol;
let mut file_redirection = None;
let mut curr_comment: Option<Vec<Span>> = None;
let mut error = None;
for (idx, token) in tokens.iter().enumerate() {
match &token.contents {
TokenContents::PipePipe => {
error = error.or(Some(ParseError::ShellOrOr(token.span)));
curr_command.push(token.span);
last_token = TokenContents::Item;
}
TokenContents::Item => {
// If we have a comment, go ahead and attach it
if let Some(curr_comment) = curr_comment.take() {
curr_command.comments = curr_comment;
}
curr_command.push(token.span);
last_token = TokenContents::Item;
}
TokenContents::OutGreaterThan
| TokenContents::OutGreaterGreaterThan
| TokenContents::ErrGreaterThan
| TokenContents::ErrGreaterGreaterThan
| TokenContents::OutErrGreaterThan
| TokenContents::OutErrGreaterGreaterThan => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
if let Some((source, append, span)) = file_redirection.take() {
if command.parts.is_empty() {
error = error.or(Some(ParseError::LabeledError(
"Redirection without command or expression".into(),
"there is nothing to redirect".into(),
span,
)));
curr_command = LiteCommand::new();
last_token = token.contents;
last_connector = token.contents;
last_connector_span = Some(token.span);
}
pipe_token @ (TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe) => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
command.push(span);
curr_command = LiteCommand::new();
last_token = *pipe_token;
last_connector = *pipe_token;
last_connector_span = Some(token.span);
}
TokenContents::Eol => {
// Handle `[Command] [Pipe] ([Comment] | [Eol])+ [Command]`
//
// `[Eol]` branch checks if previous token is `[Pipe]` to construct pipeline
// and so `[Comment] | [Eol]` should be ignore to make it work
let actual_token = last_non_comment_token(tokens, idx);
if actual_token != Some(TokenContents::Pipe)
&& actual_token != Some(TokenContents::OutGreaterThan)
{
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
match token.contents {
TokenContents::Comment => {
command.comments.push(token.span);
curr_comment = None;
}
curr_command = LiteCommand::new();
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
curr_pipeline = LitePipeline::new();
last_connector = TokenContents::Pipe;
last_connector_span = None;
TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe => {
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Semicolon => {
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Eol => {
pipeline.push(&mut command);
}
_ => command.push(token.span),
}
} else {
match &token.contents {
TokenContents::PipePipe => {
error = error.or(Some(ParseError::ShellOrOr(token.span)));
command.push(span);
command.push(token.span);
}
TokenContents::Item => {
let target = LiteRedirectionTarget::File {
connector: span,
file: token.span,
append,
};
if let Err(err) = command.try_add_redirection(source, target) {
error = error.or(Some(err));
command.push(span);
command.push(token.span)
}
}
TokenContents::OutGreaterThan
| TokenContents::OutGreaterGreaterThan
| TokenContents::ErrGreaterThan
| TokenContents::ErrGreaterGreaterThan
| TokenContents::OutErrGreaterThan
| TokenContents::OutErrGreaterGreaterThan => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
command.push(token.span);
}
TokenContents::Pipe
| TokenContents::ErrGreaterPipe
| TokenContents::OutErrGreaterPipe => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Eol => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
}
TokenContents::Semicolon => {
error =
error.or(Some(ParseError::Expected("redirection target", token.span)));
command.push(span);
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Comment => {
error = error.or(Some(ParseError::Expected("redirection target", span)));
command.push(span);
command.comments.push(token.span);
curr_comment = None;
}
}
if last_token == TokenContents::Eol {
// Clear out the comment as we're entering a new comment
curr_comment = None;
}
last_token = TokenContents::Eol;
}
TokenContents::Semicolon => {
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
} else {
match &token.contents {
TokenContents::PipePipe => {
error = error.or(Some(ParseError::ShellOrOr(token.span)));
command.push(token.span);
}
TokenContents::Item => {
// This is commented out to preserve old parser behavior,
// but we should probably error here.
//
// if element.redirection.is_some() {
// error = error.or(Some(ParseError::LabeledError(
// "Unexpected positional".into(),
// "cannot add positional arguments after output redirection".into(),
// token.span,
// )));
// }
//
// For example, this is currently allowed: ^echo thing o> out.txt extra_arg
curr_command = LiteCommand::new();
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
curr_pipeline = LitePipeline::new();
last_connector = TokenContents::Pipe;
last_connector_span = None;
// If we have a comment, go ahead and attach it
if let Some(curr_comment) = curr_comment.take() {
command.comments = curr_comment;
}
command.push(token.span);
}
TokenContents::OutGreaterThan => {
file_redirection = Some((RedirectionSource::Stdout, false, token.span));
}
TokenContents::OutGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::Stdout, true, token.span));
}
TokenContents::ErrGreaterThan => {
file_redirection = Some((RedirectionSource::Stderr, false, token.span));
}
TokenContents::ErrGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::Stderr, true, token.span));
}
TokenContents::OutErrGreaterThan => {
file_redirection =
Some((RedirectionSource::StdoutAndStderr, false, token.span));
}
TokenContents::OutErrGreaterGreaterThan => {
file_redirection = Some((RedirectionSource::StdoutAndStderr, true, token.span));
}
TokenContents::ErrGreaterPipe => {
let target = LiteRedirectionTarget::Pipe {
connector: token.span,
};
if let Err(err) = command.try_add_redirection(RedirectionSource::Stderr, target)
{
error = error.or(Some(err));
}
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::OutErrGreaterPipe => {
let target = LiteRedirectionTarget::Pipe {
connector: token.span,
};
if let Err(err) =
command.try_add_redirection(RedirectionSource::StdoutAndStderr, target)
{
error = error.or(Some(err));
}
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Pipe => {
pipeline.push(&mut command);
command.pipe = Some(token.span);
}
TokenContents::Eol => {
// Handle `[Command] [Pipe] ([Comment] | [Eol])+ [Command]`
//
// `[Eol]` branch checks if previous token is `[Pipe]` to construct pipeline
// and so `[Comment] | [Eol]` should be ignore to make it work
let actual_token = last_non_comment_token(tokens, idx);
if actual_token != Some(TokenContents::Pipe) {
pipeline.push(&mut command);
block.push(&mut pipeline);
}
last_token = TokenContents::Semicolon;
}
TokenContents::Comment => {
// Comment is beside something
if last_token != TokenContents::Eol {
curr_command.comments.push(token.span);
curr_comment = None;
} else {
// Comment precedes something
if let Some(curr_comment) = &mut curr_comment {
curr_comment.push(token.span);
if last_token == TokenContents::Eol {
// Clear out the comment as we're entering a new comment
curr_comment = None;
}
}
TokenContents::Semicolon => {
pipeline.push(&mut command);
block.push(&mut pipeline);
}
TokenContents::Comment => {
// Comment is beside something
if last_token != TokenContents::Eol {
command.comments.push(token.span);
curr_comment = None;
} else {
curr_comment = Some(vec![token.span]);
// Comment precedes something
if let Some(curr_comment) = &mut curr_comment {
curr_comment.push(token.span);
} else {
curr_comment = Some(vec![token.span]);
}
}
}
last_token = TokenContents::Comment;
}
}
last_token = token.contents;
}
if let Some(err) = push_command_to(
&mut curr_pipeline,
curr_command,
last_connector,
last_connector_span,
) {
error = Some(err);
}
if !curr_pipeline.is_empty() {
block.push(curr_pipeline);
if let Some((_, _, span)) = file_redirection {
command.push(span);
error = error.or(Some(ParseError::Expected("redirection target", span)));
}
pipeline.push(&mut command);
block.push(&mut pipeline);
if last_non_comment_token(tokens, tokens.len()) == Some(TokenContents::Pipe) {
(
block,
@ -394,86 +385,3 @@ pub fn lite_parse(tokens: &[Token]) -> (LiteBlock, Option<ParseError>) {
(block, error)
}
}
fn get_redirection(connector: TokenContents) -> Option<(Redirection, bool)> {
match connector {
TokenContents::OutGreaterThan => Some((Redirection::Stdout, false)),
TokenContents::OutGreaterGreaterThan => Some((Redirection::Stdout, true)),
TokenContents::ErrGreaterThan => Some((Redirection::Stderr, false)),
TokenContents::ErrGreaterGreaterThan => Some((Redirection::Stderr, true)),
TokenContents::OutErrGreaterThan => Some((Redirection::StdoutAndStderr, false)),
TokenContents::OutErrGreaterGreaterThan => Some((Redirection::StdoutAndStderr, true)),
_ => None,
}
}
/// push a `command` to `pipeline`
///
/// It will return Some(err) if `command` is empty and we want to push a
/// redirection command, or we have meet the same redirection in `pipeline`.
fn push_command_to(
pipeline: &mut LitePipeline,
command: LiteCommand,
last_connector: TokenContents,
last_connector_span: Option<Span>,
) -> Option<ParseError> {
if !command.is_empty() {
match get_redirection(last_connector) {
Some((redirect, is_append_mode)) => {
let span = last_connector_span
.expect("internal error: redirection missing span information");
if pipeline.exists(&redirect) {
return Some(ParseError::LabeledError(
"Redirection can be set only once".into(),
"try to remove one".into(),
span,
));
}
pipeline.push(LiteElement::Redirection(
last_connector_span
.expect("internal error: redirection missing span information"),
redirect,
command,
is_append_mode,
))
}
None => {
if last_connector == TokenContents::ErrGreaterPipe {
pipeline.push(LiteElement::ErrPipedCommand(last_connector_span, command))
} else if last_connector == TokenContents::OutErrGreaterPipe {
// Don't allow o+e>| along with redirection.
for cmd in &pipeline.commands {
if matches!(
cmd,
LiteElement::Redirection { .. }
| LiteElement::SameTargetRedirection { .. }
| LiteElement::SeparateRedirection { .. }
) {
return Some(ParseError::LabeledError(
"`o+e>|` pipe is not allowed to use with redirection".into(),
"try to use different type of pipe, or remove redirection".into(),
last_connector_span
.expect("internal error: outerr pipe missing span information"),
));
}
}
pipeline.push(LiteElement::OutErrPipedCommand(
last_connector_span,
command,
))
} else {
pipeline.push(LiteElement::Command(last_connector_span, command))
}
}
}
None
} else if get_redirection(last_connector).is_some() {
Some(ParseError::Expected(
"redirection target",
last_connector_span.expect("internal error: redirection missing span information"),
))
} else {
None
}
}

522
crates/nu-parser/src/parse_keywords.rs
View File

@ -1,6 +1,7 @@
use crate::{
exportable::Exportable,
parse_block,
parser::{parse_redirection, redirecting_builtin_error},
parser_path::ParserPath,
type_check::{check_block_input_output, type_compatible},
};
@ -28,7 +29,7 @@ use crate::{
is_math_expression_like,
known_external::KnownExternal,
lex,
lite_parser::{lite_parse, LiteCommand, LiteElement},
lite_parser::{lite_parse, LiteCommand},
parser::{
check_call, check_name, garbage, garbage_pipeline, parse, parse_call, parse_expression,
parse_full_signature, parse_import_pattern, parse_internal_call, parse_multispan_value,
@ -88,17 +89,8 @@ pub fn is_unaliasable_parser_keyword(working_set: &StateWorkingSet, spans: &[Spa
/// This is a new more compact method of calling parse_xxx() functions without repeating the
/// parse_call() in each function. Remaining keywords can be moved here.
pub fn parse_keyword(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
is_subexpression: bool,
) -> Pipeline {
let call_expr = parse_call(
working_set,
&lite_command.parts,
lite_command.parts[0],
is_subexpression,
);
pub fn parse_keyword(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let call_expr = parse_call(working_set, &lite_command.parts, lite_command.parts[0]);
// if err.is_some() {
// return (Pipeline::from_vec(vec![call_expr]), err);
@ -246,7 +238,8 @@ pub fn parse_def_predecl(working_set: &mut StateWorkingSet, spans: &[Span]) {
}
}
pub fn parse_for(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
pub fn parse_for(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Expression {
let spans = &lite_command.parts;
// Checking that the function is used with the correct name
// Maybe this is not necessary but it is a sanity check
if working_set.get_span_contents(spans[0]) != b"for" {
@ -256,6 +249,10 @@ pub fn parse_for(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expressio
));
return garbage(spans[0]);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("for", redirection));
return garbage(spans[0]);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -393,6 +390,10 @@ pub fn parse_def(
));
return (garbage_pipeline(spans), None);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("def", redirection));
return (garbage_pipeline(spans), None);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -667,6 +668,10 @@ pub fn parse_extern(
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("extern", redirection));
return garbage_pipeline(spans);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -818,6 +823,10 @@ pub fn parse_alias(
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("alias", redirection));
return garbage_pipeline(spans);
}
if let Some(span) = check_name(working_set, spans) {
return Pipeline::from_vec(vec![garbage(*span)]);
@ -907,7 +916,7 @@ pub fn parse_alias(
{
// TODO: Maybe we need to implement a Display trait for Expression?
let starting_error_count = working_set.parse_errors.len();
let expr = parse_expression(working_set, replacement_spans, false);
let expr = parse_expression(working_set, replacement_spans);
working_set.parse_errors.truncate(starting_error_count);
let msg = format!("{:?}", expr.expr);
@ -923,12 +932,7 @@ pub fn parse_alias(
let starting_error_count = working_set.parse_errors.len();
working_set.search_predecls = false;
let expr = parse_call(
working_set,
replacement_spans,
replacement_spans[0],
false, // TODO: Should this be set properly???
);
let expr = parse_call(working_set, replacement_spans, replacement_spans[0]);
working_set.search_predecls = true;
@ -1063,24 +1067,32 @@ pub fn parse_export_in_block(
let full_name = if lite_command.parts.len() > 1 {
let sub = working_set.get_span_contents(lite_command.parts[1]);
match sub {
b"alias" | b"def" | b"extern" | b"use" | b"module" | b"const" => {
[b"export ", sub].concat()
}
_ => b"export".to_vec(),
b"alias" => "export alias",
b"def" => "export def",
b"extern" => "export extern",
b"use" => "export use",
b"module" => "export module",
b"const" => "export const",
_ => "export",
}
} else {
b"export".to_vec()
"export"
};
if let Some(decl_id) = working_set.find_decl(&full_name) {
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error(full_name, redirection));
return garbage_pipeline(&lite_command.parts);
}
if let Some(decl_id) = working_set.find_decl(full_name.as_bytes()) {
let ParsedInternalCall { call, output, .. } = parse_internal_call(
working_set,
if full_name == b"export" {
if full_name == "export" {
lite_command.parts[0]
} else {
span(&lite_command.parts[0..2])
},
if full_name == b"export" {
if full_name == "export" {
&lite_command.parts[1..]
} else {
&lite_command.parts[2..]
@ -1107,16 +1119,13 @@ pub fn parse_export_in_block(
}
} else {
working_set.error(ParseError::UnknownState(
format!(
"internal error: '{}' declaration not found",
String::from_utf8_lossy(&full_name)
),
format!("internal error: '{full_name}' declaration not found",),
span(&lite_command.parts),
));
return garbage_pipeline(&lite_command.parts);
};
if &full_name == b"export" {
if full_name == "export" {
// export by itself is meaningless
working_set.error(ParseError::UnexpectedKeyword(
"export".into(),
@ -1125,19 +1134,16 @@ pub fn parse_export_in_block(
return garbage_pipeline(&lite_command.parts);
}
match full_name.as_slice() {
b"export alias" => parse_alias(working_set, lite_command, None),
b"export def" => parse_def(working_set, lite_command, None).0,
b"export const" => parse_const(working_set, &lite_command.parts[1..]),
b"export use" => {
let (pipeline, _) = parse_use(working_set, &lite_command.parts);
pipeline
}
b"export module" => parse_module(working_set, lite_command, None).0,
b"export extern" => parse_extern(working_set, lite_command, None),
match full_name {
"export alias" => parse_alias(working_set, lite_command, None),
"export def" => parse_def(working_set, lite_command, None).0,
"export const" => parse_const(working_set, &lite_command.parts[1..]),
"export use" => parse_use(working_set, lite_command).0,
"export module" => parse_module(working_set, lite_command, None).0,
"export extern" => parse_extern(working_set, lite_command, None),
_ => {
working_set.error(ParseError::UnexpectedKeyword(
String::from_utf8_lossy(&full_name).to_string(),
full_name.into(),
lite_command.parts[0],
));
@ -1186,8 +1192,6 @@ pub fn parse_export_in_module(
head: spans[0],
decl_id: export_decl_id,
arguments: vec![],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -1198,6 +1202,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let (pipeline, cmd_result) =
parse_def(working_set, &lite_command, Some(module_name));
@ -1222,16 +1228,9 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'def' call into the 'export def' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = def_call.clone();
call.head = span(&spans[0..=1]);
call.decl_id = export_def_decl_id;
} else {
@ -1247,6 +1246,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let extern_name = [b"export ", kw_name].concat();
@ -1263,16 +1264,9 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'def' call into the 'export def' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = def_call.clone();
call.head = span(&spans[0..=1]);
call.decl_id = export_def_decl_id;
} else {
@ -1308,6 +1302,8 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let pipeline = parse_alias(working_set, &lite_command, Some(module_name));
@ -1323,13 +1319,8 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'alias' call into the 'export alias' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref alias_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(alias_call)) =
pipeline.elements.first().map(|e| &e.expr.expr)
{
call = alias_call.clone();
@ -1368,8 +1359,10 @@ pub fn parse_export_in_module(
let lite_command = LiteCommand {
comments: lite_command.comments.clone(),
parts: spans[1..].to_vec(),
pipe: lite_command.pipe,
redirection: lite_command.redirection.clone(),
};
let (pipeline, exportables) = parse_use(working_set, &lite_command.parts);
let (pipeline, exportables) = parse_use(working_set, &lite_command);
let export_use_decl_id = if let Some(id) = working_set.find_decl(b"export use") {
id
@ -1382,13 +1375,7 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'use' call into the 'export use' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref use_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(use_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = use_call.clone();
@ -1419,13 +1406,8 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'module' call into the 'export module' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref module_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(module_call)) =
pipeline.elements.first().map(|e| &e.expr.expr)
{
call = module_call.clone();
@ -1476,13 +1458,7 @@ pub fn parse_export_in_module(
};
// Trying to warp the 'const' call into the 'export const' in a very clumsy way
if let Some(PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(ref def_call),
..
},
)) = pipeline.elements.first()
if let Some(Expr::Call(def_call)) = pipeline.elements.first().map(|e| &e.expr.expr)
{
call = def_call.clone();
@ -1690,9 +1666,7 @@ pub fn parse_module_block(
for pipeline in &output.block {
if pipeline.commands.len() == 1 {
if let LiteElement::Command(_, command) = &pipeline.commands[0] {
parse_def_predecl(working_set, &command.parts);
}
parse_def_predecl(working_set, &pipeline.commands[0].parts);
}
}
@ -1702,186 +1676,146 @@ pub fn parse_module_block(
for pipeline in output.block.iter() {
if pipeline.commands.len() == 1 {
match &pipeline.commands[0] {
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command) => {
let name = working_set.get_span_contents(command.parts[0]);
let command = &pipeline.commands[0];
match name {
b"def" => {
block.pipelines.push(
parse_def(
working_set,
command,
None, // using commands named as the module locally is OK
)
.0,
)
}
b"const" => block
.pipelines
.push(parse_const(working_set, &command.parts)),
b"extern" => block
.pipelines
.push(parse_extern(working_set, command, None)),
b"alias" => {
block.pipelines.push(parse_alias(
working_set,
command,
None, // using aliases named as the module locally is OK
))
}
b"use" => {
let (pipeline, _) = parse_use(working_set, &command.parts);
let name = working_set.get_span_contents(command.parts[0]);
block.pipelines.push(pipeline)
}
b"module" => {
let (pipeline, _) = parse_module(
working_set,
command,
None, // using modules named as the module locally is OK
);
match name {
b"def" => {
block.pipelines.push(
parse_def(
working_set,
command,
None, // using commands named as the module locally is OK
)
.0,
)
}
b"const" => block
.pipelines
.push(parse_const(working_set, &command.parts)),
b"extern" => block
.pipelines
.push(parse_extern(working_set, command, None)),
b"alias" => {
block.pipelines.push(parse_alias(
working_set,
command,
None, // using aliases named as the module locally is OK
))
}
b"use" => {
let (pipeline, _) = parse_use(working_set, command);
block.pipelines.push(pipeline)
}
b"export" => {
let (pipe, exportables) =
parse_export_in_module(working_set, command, module_name);
block.pipelines.push(pipeline)
}
b"module" => {
let (pipeline, _) = parse_module(
working_set,
command,
None, // using modules named as the module locally is OK
);
for exportable in exportables {
match exportable {
Exportable::Decl { name, id } => {
if &name == b"main" {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &pipe.elements[0]
{
call.head
} else {
pipe.elements[0].span()
}
} else {
span
};
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name)
.to_string(),
err_span,
));
block.pipelines.push(pipeline)
}
b"export" => {
let (pipe, exportables) =
parse_export_in_module(working_set, command, module_name);
for exportable in exportables {
match exportable {
Exportable::Decl { name, id } => {
if &name == b"main" {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let Expr::Call(call) = &pipe.elements[0].expr.expr {
call.head
} else {
module.main = Some(id);
pipe.elements[0].expr.span
}
} else {
module.add_decl(name, id);
}
}
Exportable::Module { name, id } => {
if &name == b"mod" {
let (
submodule_main,
submodule_decls,
submodule_submodules,
) = {
let submodule = working_set.get_module(id);
(
submodule.main,
submodule.decls(),
submodule.submodules(),
)
};
// Add submodule's decls to the parent module
for (decl_name, decl_id) in submodule_decls {
module.add_decl(decl_name, decl_id);
}
// Add submodule's main command to the parent module
if let Some(main_decl_id) = submodule_main {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &pipe.elements[0]
{
call.head
} else {
pipe.elements[0].span()
}
} else {
span
};
working_set.error(
ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name)
.to_string(),
err_span,
),
);
} else {
module.main = Some(main_decl_id);
}
}
// Add submodule's submodules to the parent module
for (submodule_name, submodule_id) in
submodule_submodules
{
module.add_submodule(submodule_name, submodule_id);
}
} else {
module.add_submodule(name, id);
}
}
Exportable::VarDecl { name, id } => {
module.add_variable(name, id);
span
};
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name).to_string(),
err_span,
));
} else {
module.main = Some(id);
}
} else {
module.add_decl(name, id);
}
}
Exportable::Module { name, id } => {
if &name == b"mod" {
let (submodule_main, submodule_decls, submodule_submodules) = {
let submodule = working_set.get_module(id);
(submodule.main, submodule.decls(), submodule.submodules())
};
block.pipelines.push(pipe)
}
b"export-env" => {
let (pipe, maybe_env_block) =
parse_export_env(working_set, &command.parts);
// Add submodule's decls to the parent module
for (decl_name, decl_id) in submodule_decls {
module.add_decl(decl_name, decl_id);
}
if let Some(block_id) = maybe_env_block {
module.add_env_block(block_id);
// Add submodule's main command to the parent module
if let Some(main_decl_id) = submodule_main {
if module.main.is_some() {
let err_span = if !pipe.elements.is_empty() {
if let Expr::Call(call) =
&pipe.elements[0].expr.expr
{
call.head
} else {
pipe.elements[0].expr.span
}
} else {
span
};
working_set.error(ParseError::ModuleDoubleMain(
String::from_utf8_lossy(module_name).to_string(),
err_span,
));
} else {
module.main = Some(main_decl_id);
}
}
// Add submodule's submodules to the parent module
for (submodule_name, submodule_id) in submodule_submodules {
module.add_submodule(submodule_name, submodule_id);
}
} else {
module.add_submodule(name, id);
}
}
Exportable::VarDecl { name, id } => {
module.add_variable(name, id);
}
block.pipelines.push(pipe)
}
_ => {
working_set.error(ParseError::ExpectedKeyword(
"def, const, extern, alias, use, module, export or export-env keyword".into(),
command.parts[0],
));
block.pipelines.push(garbage_pipeline(&command.parts))
}
}
block.pipelines.push(pipe)
}
LiteElement::Redirection(_, _, command, _) => {
b"export-env" => {
let (pipe, maybe_env_block) = parse_export_env(working_set, &command.parts);
if let Some(block_id) = maybe_env_block {
module.add_env_block(block_id);
}
block.pipelines.push(pipe)
}
_ => {
working_set.error(ParseError::ExpectedKeyword(
"def, const, extern, alias, use, module, export or export-env keyword"
.into(),
command.parts[0],
));
block.pipelines.push(garbage_pipeline(&command.parts))
}
LiteElement::SeparateRedirection {
out: (_, command, _),
..
} => block.pipelines.push(garbage_pipeline(&command.parts)),
LiteElement::SameTargetRedirection {
cmd: (_, command), ..
} => block.pipelines.push(garbage_pipeline(&command.parts)),
}
} else {
working_set.error(ParseError::Expected("not a pipeline", span));
@ -2069,6 +2003,12 @@ pub fn parse_module(
// visible and usable in this module's scope). We want to disable that for files.
let spans = &lite_command.parts;
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("module", redirection));
return (garbage_pipeline(spans), None);
}
let mut module_comments = lite_command.comments.clone();
let split_id = if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" {
@ -2236,8 +2176,6 @@ pub fn parse_module(
Argument::Positional(module_name_or_path_expr),
Argument::Positional(block_expr),
],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -2252,7 +2190,12 @@ pub fn parse_module(
)
}
pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline, Vec<Exportable>) {
pub fn parse_use(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
) -> (Pipeline, Vec<Exportable>) {
let spans = &lite_command.parts;
let (name_span, split_id) =
if spans.len() > 1 && working_set.get_span_contents(spans[0]) == b"export" {
(spans[1], 2)
@ -2277,6 +2220,11 @@ pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline
return (garbage_pipeline(spans), vec![]);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("use", redirection));
return (garbage_pipeline(spans), vec![]);
}
let (call, call_span, args_spans) = match working_set.find_decl(b"use") {
Some(decl_id) => {
let (command_spans, rest_spans) = spans.split_at(split_id);
@ -2460,7 +2408,9 @@ pub fn parse_use(working_set: &mut StateWorkingSet, spans: &[Span]) -> (Pipeline
)
}
pub fn parse_hide(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
pub fn parse_hide(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let spans = &lite_command.parts;
if working_set.get_span_contents(spans[0]) != b"hide" {
working_set.error(ParseError::UnknownState(
"internal error: Wrong call name for 'hide' command".into(),
@ -2468,6 +2418,10 @@ pub fn parse_hide(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("hide", redirection));
return garbage_pipeline(spans);
}
let (call, args_spans) = match working_set.find_decl(b"hide") {
Some(decl_id) => {
@ -3054,8 +3008,6 @@ pub fn parse_let(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
decl_id,
head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -3198,8 +3150,6 @@ pub fn parse_const(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipelin
decl_id,
head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -3315,8 +3265,6 @@ pub fn parse_mut(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
decl_id,
head: spans[0],
arguments: vec![Argument::Positional(lvalue), Argument::Positional(rvalue)],
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
});
@ -3353,10 +3301,21 @@ pub fn parse_mut(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline
garbage_pipeline(spans)
}
pub fn parse_source(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
pub fn parse_source(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let spans = &lite_command.parts;
let name = working_set.get_span_contents(spans[0]);
if name == b"source" || name == b"source-env" {
if let Some(redirection) = lite_command.redirection.as_ref() {
let name = if name == b"source" {
"source"
} else {
"source-env"
};
working_set.error(redirecting_builtin_error(name, redirection));
return garbage_pipeline(spans);
}
let scoped = name == b"source-env";
if let Some(decl_id) = working_set.find_decl(name) {
@ -3537,13 +3496,26 @@ pub fn parse_where_expr(working_set: &mut StateWorkingSet, spans: &[Span]) -> Ex
}
}
pub fn parse_where(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
let expression = parse_where_expr(working_set, spans);
Pipeline::from_vec(vec![expression])
pub fn parse_where(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
let expr = parse_where_expr(working_set, &lite_command.parts);
let redirection = lite_command
.redirection
.as_ref()
.map(|r| parse_redirection(working_set, r));
let element = PipelineElement {
pipe: None,
expr,
redirection,
};
Pipeline {
elements: vec![element],
}
}
#[cfg(feature = "plugin")]
pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipeline {
pub fn parse_register(working_set: &mut StateWorkingSet, lite_command: &LiteCommand) -> Pipeline {
use std::sync::Arc;
use nu_plugin::{get_signature, PersistentPlugin, PluginDeclaration};
@ -3551,6 +3523,8 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
engine::Stack, IntoSpanned, PluginIdentity, PluginSignature, RegisteredPlugin,
};
let spans = &lite_command.parts;
let cwd = working_set.get_cwd();
// Checking that the function is used with the correct name
@ -3562,6 +3536,10 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
));
return garbage_pipeline(spans);
}
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("register", redirection));
return garbage_pipeline(spans);
}
// Parsing the spans and checking that they match the register signature
// Using a parsed call makes more sense than checking for how many spans are in the call
@ -3677,7 +3655,7 @@ pub fn parse_register(working_set: &mut StateWorkingSet, spans: &[Span]) -> Pipe
// We need the current environment variables for `python` based plugins
// Or we'll likely have a problem when a plugin is implemented in a virtual Python environment.
let get_envs = || {
let stack = Stack::new();
let stack = Stack::new().capture();
nu_engine::env::env_to_strings(working_set.permanent_state, &stack)
};

63
crates/nu-parser/src/parse_patterns.rs
View File

@ -7,7 +7,6 @@ use nu_protocol::{
use crate::{
lex, lite_parse,
parser::{is_variable, parse_value},
LiteElement,
};
pub fn garbage(span: Span) -> MatchPattern {
@ -108,48 +107,46 @@ pub fn parse_list_pattern(working_set: &mut StateWorkingSet, span: Span) -> Matc
let mut args = vec![];
if !output.block.is_empty() {
for arg in &output.block[0].commands {
for command in &output.block[0].commands {
let mut spans_idx = 0;
if let LiteElement::Command(_, command) = arg {
while spans_idx < command.parts.len() {
let contents = working_set.get_span_contents(command.parts[spans_idx]);
if contents == b".." {
while spans_idx < command.parts.len() {
let contents = working_set.get_span_contents(command.parts[spans_idx]);
if contents == b".." {
args.push(MatchPattern {
pattern: Pattern::IgnoreRest,
guard: None,
span: command.parts[spans_idx],
});
break;
} else if contents.starts_with(b"..$") {
if let Some(var_id) = parse_variable_pattern_helper(
working_set,
Span::new(
command.parts[spans_idx].start + 2,
command.parts[spans_idx].end,
),
) {
args.push(MatchPattern {
pattern: Pattern::IgnoreRest,
pattern: Pattern::Rest(var_id),
guard: None,
span: command.parts[spans_idx],
});
break;
} else if contents.starts_with(b"..$") {
if let Some(var_id) = parse_variable_pattern_helper(
working_set,
Span::new(
command.parts[spans_idx].start + 2,
command.parts[spans_idx].end,
),
) {
args.push(MatchPattern {
pattern: Pattern::Rest(var_id),
guard: None,
span: command.parts[spans_idx],
});
break;
} else {
args.push(garbage(command.parts[spans_idx]));
working_set.error(ParseError::Expected(
"valid variable name",
command.parts[spans_idx],
));
}
} else {
let arg = parse_pattern(working_set, command.parts[spans_idx]);
args.push(garbage(command.parts[spans_idx]));
working_set.error(ParseError::Expected(
"valid variable name",
command.parts[spans_idx],
));
}
} else {
let arg = parse_pattern(working_set, command.parts[spans_idx]);
args.push(arg);
};
args.push(arg);
};
spans_idx += 1;
}
spans_idx += 1;
}
}
}

742
crates/nu-parser/src/parser.rs
View File

@ -1,6 +1,6 @@
use crate::{
lex::{lex, lex_signature},
lite_parser::{lite_parse, LiteCommand, LiteElement, LitePipeline},
lite_parser::{lite_parse, LiteCommand, LitePipeline, LiteRedirection, LiteRedirectionTarget},
parse_mut,
parse_patterns::parse_pattern,
parse_shape_specs::{parse_shape_name, parse_type, ShapeDescriptorUse},
@ -14,7 +14,7 @@ use nu_protocol::{
Argument, Assignment, Bits, Block, Boolean, Call, CellPath, Comparison, Expr, Expression,
ExternalArgument, FullCellPath, ImportPattern, ImportPatternHead, ImportPatternMember,
MatchPattern, Math, Operator, PathMember, Pattern, Pipeline, PipelineElement,
RangeInclusion, RangeOperator, RecordItem,
PipelineRedirection, RangeInclusion, RangeOperator, RecordItem, RedirectionTarget,
},
engine::StateWorkingSet,
eval_const::eval_constant,
@ -303,11 +303,7 @@ fn parse_external_arg(working_set: &mut StateWorkingSet, span: Span) -> External
}
}
pub fn parse_external_call(
working_set: &mut StateWorkingSet,
spans: &[Span],
is_subexpression: bool,
) -> Expression {
pub fn parse_external_call(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
trace!("parse external");
let mut args = vec![];
@ -324,7 +320,7 @@ pub fn parse_external_call(
let head = if head_contents.starts_with(b"$") || head_contents.starts_with(b"(") {
// the expression is inside external_call, so it's a subexpression
let arg = parse_expression(working_set, &[head_span], true);
let arg = parse_expression(working_set, &[head_span]);
Box::new(arg)
} else {
let (contents, err) = unescape_unquote_string(&head_contents, head_span);
@ -346,7 +342,7 @@ pub fn parse_external_call(
}
Expression {
expr: Expr::ExternalCall(head, args, is_subexpression),
expr: Expr::ExternalCall(head, args),
span: span(spans),
ty: Type::Any,
custom_completion: None,
@ -708,7 +704,7 @@ pub fn parse_multispan_value(
// is it subexpression?
// Not sure, but let's make it not, so the behavior is the same as previous version of nushell.
let arg = parse_expression(working_set, &spans[*spans_idx..], false);
let arg = parse_expression(working_set, &spans[*spans_idx..]);
*spans_idx = spans.len() - 1;
arg
@ -1095,12 +1091,7 @@ pub fn parse_internal_call(
}
}
pub fn parse_call(
working_set: &mut StateWorkingSet,
spans: &[Span],
head: Span,
is_subexpression: bool,
) -> Expression {
pub fn parse_call(working_set: &mut StateWorkingSet, spans: &[Span], head: Span) -> Expression {
trace!("parsing: call");
if spans.is_empty() {
@ -1179,7 +1170,7 @@ pub fn parse_call(
let parsed_call = if let Some(alias) = decl.as_alias() {
if let Expression {
expr: Expr::ExternalCall(head, args, is_subexpression),
expr: Expr::ExternalCall(head, args),
span: _,
ty,
custom_completion,
@ -1198,7 +1189,7 @@ pub fn parse_call(
head.span = spans[0]; // replacing the spans preserves syntax highlighting
return Expression {
expr: Expr::ExternalCall(head, final_args, *is_subexpression),
expr: Expr::ExternalCall(head, final_args),
span: span(spans),
ty: ty.clone(),
custom_completion: *custom_completion,
@ -1246,7 +1237,7 @@ pub fn parse_call(
trace!("parsing: external call");
// Otherwise, try external command
parse_external_call(working_set, spans, is_subexpression)
parse_external_call(working_set, spans)
}
}
@ -3139,9 +3130,11 @@ pub fn parse_row_condition(working_set: &mut StateWorkingSet, spans: &[Span]) ->
// We have an expression, so let's convert this into a block.
let mut block = Block::new();
let mut pipeline = Pipeline::new();
pipeline
.elements
.push(PipelineElement::Expression(None, expression));
pipeline.elements.push(PipelineElement {
pipe: None,
expr: expression,
redirection: None,
});
block.pipelines.push(pipeline);
@ -3846,61 +3839,59 @@ pub fn parse_list_expression(
let mut contained_type: Option<Type> = None;
if !output.block.is_empty() {
for arg in output.block.remove(0).commands {
for mut command in output.block.remove(0).commands {
let mut spans_idx = 0;
if let LiteElement::Command(_, mut command) = arg {
while spans_idx < command.parts.len() {
let curr_span = command.parts[spans_idx];
let curr_tok = working_set.get_span_contents(curr_span);
let (arg, ty) = if curr_tok.starts_with(b"...")
&& curr_tok.len() > 3
&& (curr_tok[3] == b'$' || curr_tok[3] == b'[' || curr_tok[3] == b'(')
{
// Parse the spread operator
// Remove "..." before parsing argument to spread operator
command.parts[spans_idx] = Span::new(curr_span.start + 3, curr_span.end);
let spread_arg = parse_multispan_value(
working_set,
&command.parts,
&mut spans_idx,
&SyntaxShape::List(Box::new(element_shape.clone())),
);
let elem_ty = match &spread_arg.ty {
Type::List(elem_ty) => *elem_ty.clone(),
_ => Type::Any,
};
let span = Span::new(curr_span.start, spread_arg.span.end);
let spread_expr = Expression {
expr: Expr::Spread(Box::new(spread_arg)),
span,
ty: elem_ty.clone(),
custom_completion: None,
};
(spread_expr, elem_ty)
} else {
let arg = parse_multispan_value(
working_set,
&command.parts,
&mut spans_idx,
element_shape,
);
let ty = arg.ty.clone();
(arg, ty)
while spans_idx < command.parts.len() {
let curr_span = command.parts[spans_idx];
let curr_tok = working_set.get_span_contents(curr_span);
let (arg, ty) = if curr_tok.starts_with(b"...")
&& curr_tok.len() > 3
&& (curr_tok[3] == b'$' || curr_tok[3] == b'[' || curr_tok[3] == b'(')
{
// Parse the spread operator
// Remove "..." before parsing argument to spread operator
command.parts[spans_idx] = Span::new(curr_span.start + 3, curr_span.end);
let spread_arg = parse_multispan_value(
working_set,
&command.parts,
&mut spans_idx,
&SyntaxShape::List(Box::new(element_shape.clone())),
);
let elem_ty = match &spread_arg.ty {
Type::List(elem_ty) => *elem_ty.clone(),
_ => Type::Any,
};
let span = Span::new(curr_span.start, spread_arg.span.end);
let spread_expr = Expression {
expr: Expr::Spread(Box::new(spread_arg)),
span,
ty: elem_ty.clone(),
custom_completion: None,
};
(spread_expr, elem_ty)
} else {
let arg = parse_multispan_value(
working_set,
&command.parts,
&mut spans_idx,
element_shape,
);
let ty = arg.ty.clone();
(arg, ty)
};
if let Some(ref ctype) = contained_type {
if *ctype != ty {
contained_type = Some(Type::Any);
}
} else {
contained_type = Some(ty);
if let Some(ref ctype) = contained_type {
if *ctype != ty {
contained_type = Some(Type::Any);
}
args.push(arg);
spans_idx += 1;
} else {
contained_type = Some(ty);
}
args.push(arg);
spans_idx += 1;
}
}
}
@ -4861,7 +4852,7 @@ pub fn parse_math_expression(
if first_span == b"if" || first_span == b"match" {
// If expression
if spans.len() > 1 {
return parse_call(working_set, spans, spans[0], false);
return parse_call(working_set, spans, spans[0]);
} else {
working_set.error(ParseError::Expected(
"expression",
@ -4936,7 +4927,7 @@ pub fn parse_math_expression(
// allow `if` to be a special value for assignment.
if content == b"if" || content == b"match" {
let rhs = parse_call(working_set, &spans[idx..], spans[0], false);
let rhs = parse_call(working_set, &spans[idx..], spans[0]);
expr_stack.push(op);
expr_stack.push(rhs);
break;
@ -5055,11 +5046,7 @@ pub fn parse_math_expression(
.expect("internal error: expression stack empty")
}
pub fn parse_expression(
working_set: &mut StateWorkingSet,
spans: &[Span],
is_subexpression: bool,
) -> Expression {
pub fn parse_expression(working_set: &mut StateWorkingSet, spans: &[Span]) -> Expression {
trace!("parsing: expression");
let mut pos = 0;
@ -5134,7 +5121,7 @@ pub fn parse_expression(
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
b"let" | b"const" | b"mut" => {
working_set.error(ParseError::AssignInPipeline(
@ -5152,19 +5139,19 @@ pub fn parse_expression(
.to_string(),
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
b"overlay" => {
if spans.len() > 1 && working_set.get_span_contents(spans[1]) == b"list" {
// whitelist 'overlay list'
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
} else {
working_set.error(ParseError::BuiltinCommandInPipeline(
"overlay".into(),
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
}
b"where" => parse_where_expr(working_set, &spans[pos..]),
@ -5175,10 +5162,10 @@ pub fn parse_expression(
spans[0],
));
parse_call(working_set, &spans[pos..], spans[0], is_subexpression)
parse_call(working_set, &spans[pos..], spans[0])
}
_ => parse_call(working_set, &spans[pos..], spans[0], is_subexpression),
_ => parse_call(working_set, &spans[pos..], spans[0]),
}
};
@ -5217,8 +5204,6 @@ pub fn parse_expression(
head: Span::unknown(),
decl_id,
arguments,
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
}));
@ -5251,7 +5236,6 @@ pub fn parse_variable(working_set: &mut StateWorkingSet, span: Span) -> Option<V
pub fn parse_builtin_commands(
working_set: &mut StateWorkingSet,
lite_command: &LiteCommand,
is_subexpression: bool,
) -> Pipeline {
trace!("parsing: builtin commands");
if !is_math_expression_like(working_set, lite_command.parts[0])
@ -5264,12 +5248,7 @@ pub fn parse_builtin_commands(
if cmd.is_alias() {
// Parse keywords that can be aliased. Note that we check for "unaliasable" keywords
// because alias can have any name, therefore, we can't check for "aliasable" keywords.
let call_expr = parse_call(
working_set,
&lite_command.parts,
lite_command.parts[0],
is_subexpression,
);
let call_expr = parse_call(working_set, &lite_command.parts, lite_command.parts[0]);
if let Expression {
expr: Expr::Call(call),
@ -5299,26 +5278,31 @@ pub fn parse_builtin_commands(
b"const" => parse_const(working_set, &lite_command.parts),
b"mut" => parse_mut(working_set, &lite_command.parts),
b"for" => {
let expr = parse_for(working_set, &lite_command.parts);
let expr = parse_for(working_set, lite_command);
Pipeline::from_vec(vec![expr])
}
b"alias" => parse_alias(working_set, lite_command, None),
b"module" => parse_module(working_set, lite_command, None).0,
b"use" => {
let (pipeline, _) = parse_use(working_set, &lite_command.parts);
pipeline
b"use" => parse_use(working_set, lite_command).0,
b"overlay" => {
if let Some(redirection) = lite_command.redirection.as_ref() {
working_set.error(redirecting_builtin_error("overlay", redirection));
return garbage_pipeline(&lite_command.parts);
}
parse_keyword(working_set, lite_command)
}
b"overlay" => parse_keyword(working_set, lite_command, is_subexpression),
b"source" | b"source-env" => parse_source(working_set, &lite_command.parts),
b"source" | b"source-env" => parse_source(working_set, lite_command),
b"export" => parse_export_in_block(working_set, lite_command),
b"hide" => parse_hide(working_set, &lite_command.parts),
b"where" => parse_where(working_set, &lite_command.parts),
b"hide" => parse_hide(working_set, lite_command),
b"where" => parse_where(working_set, lite_command),
#[cfg(feature = "plugin")]
b"register" => parse_register(working_set, &lite_command.parts),
b"register" => parse_register(working_set, lite_command),
_ => {
let expr = parse_expression(working_set, &lite_command.parts, is_subexpression);
let element = parse_pipeline_element(working_set, lite_command);
Pipeline::from_vec(vec![expr])
Pipeline {
elements: vec![element],
}
}
}
}
@ -5459,6 +5443,76 @@ pub fn parse_record(working_set: &mut StateWorkingSet, span: Span) -> Expression
}
}
fn parse_redirection_target(
working_set: &mut StateWorkingSet,
target: &LiteRedirectionTarget,
) -> RedirectionTarget {
match target {
LiteRedirectionTarget::File {
connector,
file,
append,
} => RedirectionTarget::File {
expr: parse_value(working_set, *file, &SyntaxShape::Any),
append: *append,
span: *connector,
},
LiteRedirectionTarget::Pipe { connector } => RedirectionTarget::Pipe { span: *connector },
}
}
pub(crate) fn parse_redirection(
working_set: &mut StateWorkingSet,
target: &LiteRedirection,
) -> PipelineRedirection {
match target {
LiteRedirection::Single { source, target } => PipelineRedirection::Single {
source: *source,
target: parse_redirection_target(working_set, target),
},
LiteRedirection::Separate { out, err } => PipelineRedirection::Separate {
out: parse_redirection_target(working_set, out),
err: parse_redirection_target(working_set, err),
},
}
}
fn parse_pipeline_element(
working_set: &mut StateWorkingSet,
command: &LiteCommand,
) -> PipelineElement {
trace!("parsing: pipeline element");
let expr = parse_expression(working_set, &command.parts);
let redirection = command
.redirection
.as_ref()
.map(|r| parse_redirection(working_set, r));
PipelineElement {
pipe: command.pipe,
expr,
redirection,
}
}
pub(crate) fn redirecting_builtin_error(
name: &'static str,
redirection: &LiteRedirection,
) -> ParseError {
match redirection {
LiteRedirection::Single { target, .. } => {
ParseError::RedirectingBuiltinCommand(name, target.connector(), None)
}
LiteRedirection::Separate { out, err } => ParseError::RedirectingBuiltinCommand(
name,
out.connector().min(err.connector()),
Some(out.connector().max(err.connector())),
),
}
}
pub fn parse_pipeline(
working_set: &mut StateWorkingSet,
pipeline: &LitePipeline,
@ -5467,271 +5521,161 @@ pub fn parse_pipeline(
) -> Pipeline {
if pipeline.commands.len() > 1 {
// Special case: allow `let` and `mut` to consume the whole pipeline, eg) `let abc = "foo" | str length`
match &pipeline.commands[0] {
LiteElement::Command(_, command) if !command.parts.is_empty() => {
if working_set.get_span_contents(command.parts[0]) == b"let"
|| working_set.get_span_contents(command.parts[0]) == b"mut"
{
let mut new_command = LiteCommand {
comments: vec![],
parts: command.parts.clone(),
};
if let Some(&first) = pipeline.commands[0].parts.first() {
let first = working_set.get_span_contents(first);
if first == b"let" || first == b"mut" {
let name = if first == b"let" { "let" } else { "mut" };
let mut new_command = LiteCommand {
comments: vec![],
parts: pipeline.commands[0].parts.clone(),
pipe: None,
redirection: None,
};
for command in &pipeline.commands[1..] {
match command {
LiteElement::Command(Some(pipe_span), command)
| LiteElement::ErrPipedCommand(Some(pipe_span), command)
| LiteElement::OutErrPipedCommand(Some(pipe_span), command) => {
new_command.parts.push(*pipe_span);
if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
working_set.error(redirecting_builtin_error(name, redirection));
}
new_command.comments.extend_from_slice(&command.comments);
new_command.parts.extend_from_slice(&command.parts);
}
LiteElement::Redirection(span, ..) => {
working_set.error(ParseError::RedirectionInLetMut(*span, None))
}
LiteElement::SeparateRedirection { out, err } => {
working_set.error(ParseError::RedirectionInLetMut(
out.0.min(err.0),
Some(out.0.max(err.0)),
))
}
LiteElement::SameTargetRedirection { redirection, .. } => working_set
.error(ParseError::RedirectionInLetMut(redirection.0, None)),
_ => panic!("unsupported"),
}
for element in &pipeline.commands[1..] {
if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
working_set.error(redirecting_builtin_error(name, redirection));
} else {
new_command.parts.push(element.pipe.expect("pipe span"));
new_command.comments.extend_from_slice(&element.comments);
new_command.parts.extend_from_slice(&element.parts);
}
}
// if the 'let' is complete enough, use it, if not, fall through for now
if new_command.parts.len() > 3 {
let rhs_span = nu_protocol::span(&new_command.parts[3..]);
// if the 'let' is complete enough, use it, if not, fall through for now
if new_command.parts.len() > 3 {
let rhs_span = nu_protocol::span(&new_command.parts[3..]);
new_command.parts.truncate(3);
new_command.parts.push(rhs_span);
new_command.parts.truncate(3);
new_command.parts.push(rhs_span);
let mut pipeline =
parse_builtin_commands(working_set, &new_command, is_subexpression);
let mut pipeline = parse_builtin_commands(working_set, &new_command);
if pipeline_index == 0 {
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
for element in pipeline.elements.iter_mut() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = element
if pipeline_index == 0 {
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
for element in pipeline.elements.iter_mut() {
if let Expr::Call(call) = &element.expr.expr {
if Some(call.decl_id) == let_decl_id
|| Some(call.decl_id) == mut_decl_id
{
if Some(call.decl_id) == let_decl_id
|| Some(call.decl_id) == mut_decl_id
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter().nth(1)
{
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter_mut().nth(1)
let block = working_set.get_block(*block_id);
if let Some(element) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
let block = working_set.get_block(*block_id);
if let Some(PipelineElement::Expression(
prepend,
expr,
)) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if expr.has_in_variable(working_set) {
let new_expr = PipelineElement::Expression(
prepend,
wrap_expr_with_collect(working_set, &expr),
);
let block =
working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = new_expr;
}
if element.has_in_variable(working_set) {
let element = wrap_element_with_collect(
working_set,
&element,
);
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = element;
}
}
continue;
} else if element.has_in_variable(working_set)
&& !is_subexpression
{
*element = wrap_element_with_collect(working_set, element);
}
continue;
} else if element.has_in_variable(working_set) && !is_subexpression
{
*element = wrap_element_with_collect(working_set, element);
}
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
}
return pipeline;
}
return pipeline;
}
}
_ => {}
};
}
let mut output = pipeline
let mut elements = pipeline
.commands
.iter()
.map(|command| match command {
LiteElement::Command(span, command) => {
trace!("parsing: pipeline element: command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::Expression(*span, expr)
}
LiteElement::ErrPipedCommand(span, command) => {
trace!("parsing: pipeline element: err piped command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::ErrPipedExpression(*span, expr)
}
LiteElement::OutErrPipedCommand(span, command) => {
trace!("parsing: pipeline element: err piped command");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
PipelineElement::OutErrPipedExpression(*span, expr)
}
LiteElement::Redirection(span, redirection, command, is_append_mode) => {
let expr = parse_value(working_set, command.parts[0], &SyntaxShape::Any);
PipelineElement::Redirection(*span, redirection.clone(), expr, *is_append_mode)
}
LiteElement::SeparateRedirection {
out: (out_span, out_command, out_append_mode),
err: (err_span, err_command, err_append_mode),
} => {
trace!("parsing: pipeline element: separate redirection");
let out_expr =
parse_value(working_set, out_command.parts[0], &SyntaxShape::Any);
let err_expr =
parse_value(working_set, err_command.parts[0], &SyntaxShape::Any);
PipelineElement::SeparateRedirection {
out: (*out_span, out_expr, *out_append_mode),
err: (*err_span, err_expr, *err_append_mode),
}
}
LiteElement::SameTargetRedirection {
cmd: (cmd_span, command),
redirection: (redirect_span, redirect_command, is_append_mode),
} => {
trace!("parsing: pipeline element: same target redirection");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
let redirect_expr =
parse_value(working_set, redirect_command.parts[0], &SyntaxShape::Any);
PipelineElement::SameTargetRedirection {
cmd: (*cmd_span, expr),
redirection: (*redirect_span, redirect_expr, *is_append_mode),
}
}
})
.collect::<Vec<PipelineElement>>();
.map(|element| parse_pipeline_element(working_set, element))
.collect::<Vec<_>>();
if is_subexpression {
for element in output.iter_mut().skip(1) {
for element in elements.iter_mut().skip(1) {
if element.has_in_variable(working_set) {
*element = wrap_element_with_collect(working_set, element);
}
}
} else {
for element in output.iter_mut() {
for element in elements.iter_mut() {
if element.has_in_variable(working_set) {
*element = wrap_element_with_collect(working_set, element);
}
}
}
Pipeline { elements: output }
Pipeline { elements }
} else {
match &pipeline.commands[0] {
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command)
| LiteElement::Redirection(_, _, command, _)
| LiteElement::SeparateRedirection {
out: (_, command, _),
..
} => {
let mut pipeline = parse_builtin_commands(working_set, command, is_subexpression);
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
if pipeline_index == 0 {
for element in pipeline.elements.iter_mut() {
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = element
{
if Some(call.decl_id) == let_decl_id
|| Some(call.decl_id) == mut_decl_id
{
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter_mut().nth(1)
{
let block = working_set.get_block(*block_id);
if let Some(PipelineElement::Expression(prepend, expr)) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if expr.has_in_variable(working_set) {
let new_expr = PipelineElement::Expression(
prepend,
wrap_expr_with_collect(working_set, &expr),
);
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = new_expr;
}
}
}
continue;
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
}
}
pipeline
}
LiteElement::SameTargetRedirection {
cmd: (span, command),
redirection: (redirect_span, redirect_cmd, is_append_mode),
} => {
trace!("parsing: pipeline element: same target redirection");
let expr = parse_expression(working_set, &command.parts, is_subexpression);
let redirect_expr =
parse_value(working_set, redirect_cmd.parts[0], &SyntaxShape::Any);
Pipeline {
elements: vec![PipelineElement::SameTargetRedirection {
cmd: (*span, expr),
redirection: (*redirect_span, redirect_expr, *is_append_mode),
}],
if let Some(&first) = pipeline.commands[0].parts.first() {
let first = working_set.get_span_contents(first);
if first == b"let" || first == b"mut" {
if let Some(redirection) = pipeline.commands[0].redirection.as_ref() {
let name = if first == b"let" { "let" } else { "mut" };
working_set.error(redirecting_builtin_error(name, redirection));
}
}
}
let mut pipeline = parse_builtin_commands(working_set, &pipeline.commands[0]);
let let_decl_id = working_set.find_decl(b"let");
let mut_decl_id = working_set.find_decl(b"mut");
if pipeline_index == 0 {
for element in pipeline.elements.iter_mut() {
if let Expr::Call(call) = &element.expr.expr {
if Some(call.decl_id) == let_decl_id || Some(call.decl_id) == mut_decl_id {
// Do an expansion
if let Some(Expression {
expr: Expr::Block(block_id),
..
}) = call.positional_iter().nth(1)
{
let block = working_set.get_block(*block_id);
if let Some(element) = block
.pipelines
.first()
.and_then(|p| p.elements.first())
.cloned()
{
if element.has_in_variable(working_set) {
let element = wrap_element_with_collect(working_set, &element);
let block = working_set.get_block_mut(*block_id);
block.pipelines[0].elements[0] = element;
}
}
}
continue;
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
} else if element.has_in_variable(working_set) && !is_subexpression {
*element = wrap_element_with_collect(working_set, element);
}
}
}
pipeline
}
}
@ -5757,19 +5701,7 @@ pub fn parse_block(
// that share the same block can see each other
for pipeline in &lite_block.block {
if pipeline.commands.len() == 1 {
match &pipeline.commands[0] {
LiteElement::Command(_, command)
| LiteElement::ErrPipedCommand(_, command)
| LiteElement::OutErrPipedCommand(_, command)
| LiteElement::Redirection(_, _, command, _)
| LiteElement::SeparateRedirection {
out: (_, command, _),
..
}
| LiteElement::SameTargetRedirection {
cmd: (_, command), ..
} => parse_def_predecl(working_set, &command.parts),
}
parse_def_predecl(working_set, &pipeline.commands[0].parts)
}
}
@ -5852,32 +5784,27 @@ pub fn discover_captures_in_pipeline_element(
seen_blocks: &mut HashMap<BlockId, Vec<(VarId, Span)>>,
output: &mut Vec<(VarId, Span)>,
) -> Result<(), ParseError> {
match element {
PipelineElement::Expression(_, expression)
| PipelineElement::ErrPipedExpression(_, expression)
| PipelineElement::OutErrPipedExpression(_, expression)
| PipelineElement::Redirection(_, _, expression, _)
| PipelineElement::And(_, expression)
| PipelineElement::Or(_, expression) => {
discover_captures_in_expr(working_set, expression, seen, seen_blocks, output)
}
PipelineElement::SeparateRedirection {
out: (_, out_expr, _),
err: (_, err_expr, _),
} => {
discover_captures_in_expr(working_set, out_expr, seen, seen_blocks, output)?;
discover_captures_in_expr(working_set, err_expr, seen, seen_blocks, output)?;
Ok(())
}
PipelineElement::SameTargetRedirection {
cmd: (_, cmd_expr),
redirection: (_, redirect_expr, _),
} => {
discover_captures_in_expr(working_set, cmd_expr, seen, seen_blocks, output)?;
discover_captures_in_expr(working_set, redirect_expr, seen, seen_blocks, output)?;
Ok(())
discover_captures_in_expr(working_set, &element.expr, seen, seen_blocks, output)?;
if let Some(redirection) = element.redirection.as_ref() {
match redirection {
PipelineRedirection::Single { target, .. } => {
if let Some(expr) = target.expr() {
discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
}
}
PipelineRedirection::Separate { out, err } => {
if let Some(expr) = out.expr() {
discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
}
if let Some(expr) = err.expr() {
discover_captures_in_expr(working_set, expr, seen, seen_blocks, output)?;
}
}
}
}
Ok(())
}
pub fn discover_captures_in_pattern(pattern: &MatchPattern, seen: &mut Vec<VarId>) {
@ -6043,7 +5970,7 @@ pub fn discover_captures_in_expr(
}
Expr::CellPath(_) => {}
Expr::DateTime(_) => {}
Expr::ExternalCall(head, args, _) => {
Expr::ExternalCall(head, args) => {
discover_captures_in_expr(working_set, head, seen, seen_blocks, output)?;
for ExternalArgument::Regular(expr) | ExternalArgument::Spread(expr) in args {
@ -6193,66 +6120,37 @@ pub fn discover_captures_in_expr(
Ok(())
}
fn wrap_redirection_with_collect(
working_set: &mut StateWorkingSet,
target: &RedirectionTarget,
) -> RedirectionTarget {
match target {
RedirectionTarget::File { expr, append, span } => RedirectionTarget::File {
expr: wrap_expr_with_collect(working_set, expr),
span: *span,
append: *append,
},
RedirectionTarget::Pipe { span } => RedirectionTarget::Pipe { span: *span },
}
}
fn wrap_element_with_collect(
working_set: &mut StateWorkingSet,
element: &PipelineElement,
) -> PipelineElement {
match element {
PipelineElement::Expression(span, expression) => {
PipelineElement::Expression(*span, wrap_expr_with_collect(working_set, expression))
}
PipelineElement::ErrPipedExpression(span, expression) => {
PipelineElement::ErrPipedExpression(
*span,
wrap_expr_with_collect(working_set, expression),
)
}
PipelineElement::OutErrPipedExpression(span, expression) => {
PipelineElement::OutErrPipedExpression(
*span,
wrap_expr_with_collect(working_set, expression),
)
}
PipelineElement::Redirection(span, redirection, expression, is_append_mode) => {
PipelineElement::Redirection(
*span,
redirection.clone(),
wrap_expr_with_collect(working_set, expression),
*is_append_mode,
)
}
PipelineElement::SeparateRedirection {
out: (out_span, out_exp, out_append_mode),
err: (err_span, err_exp, err_append_mode),
} => PipelineElement::SeparateRedirection {
out: (
*out_span,
wrap_expr_with_collect(working_set, out_exp),
*out_append_mode,
),
err: (
*err_span,
wrap_expr_with_collect(working_set, err_exp),
*err_append_mode,
),
},
PipelineElement::SameTargetRedirection {
cmd: (cmd_span, cmd_exp),
redirection: (redirect_span, redirect_exp, is_append_mode),
} => PipelineElement::SameTargetRedirection {
cmd: (*cmd_span, wrap_expr_with_collect(working_set, cmd_exp)),
redirection: (
*redirect_span,
wrap_expr_with_collect(working_set, redirect_exp),
*is_append_mode,
),
},
PipelineElement::And(span, expression) => {
PipelineElement::And(*span, wrap_expr_with_collect(working_set, expression))
}
PipelineElement::Or(span, expression) => {
PipelineElement::Or(*span, wrap_expr_with_collect(working_set, expression))
}
PipelineElement {
pipe: element.pipe,
expr: wrap_expr_with_collect(working_set, &element.expr),
redirection: element.redirection.as_ref().map(|r| match r {
PipelineRedirection::Single { source, target } => PipelineRedirection::Single {
source: *source,
target: wrap_redirection_with_collect(working_set, target),
},
PipelineRedirection::Separate { out, err } => PipelineRedirection::Separate {
out: wrap_redirection_with_collect(working_set, out),
err: wrap_redirection_with_collect(working_set, err),
},
}),
}
}
@ -6304,8 +6202,6 @@ fn wrap_expr_with_collect(working_set: &mut StateWorkingSet, expr: &Expression)
head: Span::new(0, 0),
arguments: output,
decl_id,
redirect_stdout: true,
redirect_stderr: false,
parser_info: HashMap::new(),
})),
span,

85
crates/nu-parser/src/type_check.rs
View File

@ -1,7 +1,6 @@
use nu_protocol::{
ast::{
Assignment, Bits, Block, Boolean, Comparison, Expr, Expression, Math, Operator, Pipeline,
PipelineElement,
},
engine::StateWorkingSet,
ParseError, Type,
@ -917,62 +916,51 @@ pub fn check_pipeline_type(
let mut output_errors: Option<Vec<ParseError>> = None;
'elem: for elem in &pipeline.elements {
match elem {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let decl = working_set.get_decl(call.decl_id);
if elem.redirection.is_some() {
current_type = Type::Any;
} else if let Expr::Call(call) = &elem.expr.expr {
let decl = working_set.get_decl(call.decl_id);
if current_type == Type::Any {
let mut new_current_type = None;
for (_, call_output) in decl.signature().input_output_types {
if let Some(inner_current_type) = &new_current_type {
if inner_current_type == &Type::Any {
break;
} else if inner_current_type != &call_output {
// Union unequal types to Any for now
new_current_type = Some(Type::Any)
}
} else {
new_current_type = Some(call_output.clone())
if current_type == Type::Any {
let mut new_current_type = None;
for (_, call_output) in decl.signature().input_output_types {
if let Some(inner_current_type) = &new_current_type {
if inner_current_type == &Type::Any {
break;
} else if inner_current_type != &call_output {
// Union unequal types to Any for now
new_current_type = Some(Type::Any)
}
}
if let Some(new_current_type) = new_current_type {
current_type = new_current_type
} else {
current_type = Type::Any;
new_current_type = Some(call_output.clone())
}
continue 'elem;
}
if let Some(new_current_type) = new_current_type {
current_type = new_current_type
} else {
for (call_input, call_output) in decl.signature().input_output_types {
if type_compatible(&call_input, &current_type) {
current_type = call_output.clone();
continue 'elem;
}
current_type = Type::Any;
}
continue 'elem;
} else {
for (call_input, call_output) in decl.signature().input_output_types {
if type_compatible(&call_input, &current_type) {
current_type = call_output.clone();
continue 'elem;
}
}
}
if !decl.signature().input_output_types.is_empty() {
if let Some(output_errors) = &mut output_errors {
output_errors.push(ParseError::InputMismatch(current_type, call.head))
} else {
output_errors =
Some(vec![ParseError::InputMismatch(current_type, call.head)]);
}
if !decl.signature().input_output_types.is_empty() {
if let Some(output_errors) = &mut output_errors {
output_errors.push(ParseError::InputMismatch(current_type, call.head))
} else {
output_errors = Some(vec![ParseError::InputMismatch(current_type, call.head)]);
}
current_type = Type::Any;
}
PipelineElement::Expression(_, Expression { ty, .. }) => {
current_type = ty.clone();
}
_ => {
current_type = Type::Any;
}
current_type = Type::Any;
} else {
current_type = elem.expr.ty.clone();
}
}
@ -1015,7 +1003,8 @@ pub fn check_block_input_output(working_set: &StateWorkingSet, block: &Block) ->
.elements
.last()
.expect("internal error: we should have elements")
.span()
.expr
.span
};
output_errors.push(ParseError::OutputMismatch(output_type.clone(), span))

690
crates/nu-parser/tests/test_parser.rs
View File

@ -1,10 +1,8 @@
use nu_parser::*;
use nu_protocol::ast::{Argument, Call, PathMember};
use nu_protocol::Span;
use nu_protocol::{
ast::{Expr, Expression, PipelineElement},
ast::{Argument, Call, Expr, PathMember},
engine::{Command, EngineState, Stack, StateWorkingSet},
ParseError, PipelineData, ShellError, Signature, SyntaxShape,
ParseError, PipelineData, ShellError, Signature, Span, SyntaxShape,
};
use rstest::rstest;
@ -73,21 +71,15 @@ fn test_int(
} else {
assert!(err.is_none(), "{test_tag}: unexpected error {err:#?}");
assert_eq!(block.len(), 1, "{test_tag}: result block length > 1");
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(
expressions.len(),
pipeline.len(),
1,
"{test_tag}: got multiple result expressions, expected 1"
);
if let PipelineElement::Expression(
_,
Expression {
expr: observed_val, ..
},
) = &expressions.elements[0]
{
compare_rhs_binary_op(test_tag, &expected_val, observed_val);
}
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
compare_rhs_binary_op(test_tag, &expected_val, &element.expr.expr);
}
}
@ -112,7 +104,7 @@ fn compare_rhs_binary_op(
"{test_tag}: Expected: {expected:#?}, observed: {observed:#?}"
)
}
Expr::ExternalCall(e, _, _) => {
Expr::ExternalCall(e, _) => {
let observed_expr = &e.expr;
assert_eq!(
expected, observed_expr,
@ -259,6 +251,7 @@ pub fn multi_test_parse_number() {
test_int(test.0, test.1, test.2, test.3);
}
}
#[ignore]
#[test]
fn test_parse_any() {
@ -277,6 +270,7 @@ fn test_parse_any() {
}
}
}
#[test]
pub fn parse_int() {
let engine_state = EngineState::new();
@ -286,18 +280,11 @@ pub fn parse_int() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::Int(3),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Int(3));
}
#[test]
@ -309,18 +296,11 @@ pub fn parse_int_with_underscores() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::Int(420692023),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Int(420692023));
}
#[test]
@ -339,41 +319,32 @@ pub fn parse_cell_path() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
// hoo boy this pattern matching is a pain
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
if let Expr::FullCellPath(b) = &expr.expr {
assert!(matches!(
b.head,
Expression {
expr: Expr::Var(_),
..
}
));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
if let Expr::FullCellPath(b) = &element.expr.expr {
assert!(matches!(b.head.expr, Expr::Var(_)));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
assert_eq!(optional, &false);
} else {
panic!("cell path tail is unexpected")
panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
} else {
panic!("Not a cell path");
panic!("cell path tail is unexpected")
}
} else {
panic!("Not an expression")
panic!("Not a cell path");
}
}
@ -394,41 +365,32 @@ pub fn parse_cell_path_optional() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
// hoo boy this pattern matching is a pain
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
if let Expr::FullCellPath(b) = &expr.expr {
assert!(matches!(
b.head,
Expression {
expr: Expr::Var(_),
..
}
));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
assert_eq!(optional, &true);
} else {
panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
if let Expr::FullCellPath(b) = &element.expr.expr {
assert!(matches!(b.head.expr, Expr::Var(_)));
if let [a, b] = &b.tail[..] {
if let PathMember::String { val, optional, .. } = a {
assert_eq!(val, "bar");
assert_eq!(optional, &true);
} else {
panic!("cell path tail is unexpected")
panic!("wrong type")
}
if let PathMember::String { val, optional, .. } = b {
assert_eq!(val, "baz");
assert_eq!(optional, &false);
} else {
panic!("wrong type")
}
} else {
panic!("Not a cell path");
panic!("cell path tail is unexpected")
}
} else {
panic!("Not an expression")
panic!("Not a cell path");
}
}
@ -441,13 +403,11 @@ pub fn parse_binary_with_hex_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0x13]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0x13]));
}
#[test]
@ -459,13 +419,11 @@ pub fn parse_binary_with_incomplete_hex_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0x03]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0x03]));
}
#[test]
@ -477,13 +435,11 @@ pub fn parse_binary_with_binary_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0b10101000]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0b10101000]));
}
#[test]
@ -495,13 +451,11 @@ pub fn parse_binary_with_incomplete_binary_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0b00000010]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0b00000010]));
}
#[test]
@ -513,13 +467,11 @@ pub fn parse_binary_with_octal_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0o250]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0o250]));
}
#[test]
@ -531,13 +483,11 @@ pub fn parse_binary_with_incomplete_octal_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::Binary(vec![0o2]))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::Binary(vec![0o2]));
}
#[test]
@ -549,13 +499,11 @@ pub fn parse_binary_with_invalid_octal_format() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert!(!matches!(&expr.expr, Expr::Binary(_)))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(!matches!(element.expr.expr, Expr::Binary(_)));
}
#[test]
@ -569,13 +517,11 @@ pub fn parse_binary_with_multi_byte_char() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert!(!matches!(&expr.expr, Expr::Binary(_)))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(!matches!(element.expr.expr, Expr::Binary(_)))
}
#[test]
@ -591,17 +537,12 @@ pub fn parse_call() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &expressions.elements[0]
{
if let Expr::Call(call) = &element.expr.expr {
assert_eq!(call.decl_id, 0);
}
}
@ -650,17 +591,12 @@ pub fn parse_call_short_flag_batch_arg_allowed() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) = &expressions.elements[0]
{
if let Expr::Call(call) = &element.expr.expr {
assert_eq!(call.decl_id, 0);
assert_eq!(call.arguments.len(), 2);
matches!(call.arguments[0], Argument::Named((_, None, None)));
@ -767,42 +703,28 @@ fn test_nothing_comparison_eq() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
&expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::BinaryOp(..),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(matches!(&element.expr.expr, Expr::BinaryOp(..)));
}
#[rstest]
#[case(b"let a = 1 err> /dev/null", "RedirectionInLetMut")]
#[case(b"let a = 1 out> /dev/null", "RedirectionInLetMut")]
#[case(b"mut a = 1 err> /dev/null", "RedirectionInLetMut")]
#[case(b"mut a = 1 out> /dev/null", "RedirectionInLetMut")]
// This two cases cause AssignInPipeline instead of RedirectionInLetMut
#[case(b"let a = 1 out+err> /dev/null", "AssignInPipeline")]
#[case(b"mut a = 1 out+err> /dev/null", "AssignInPipeline")]
fn test_redirection_with_letmut(#[case] phase: &[u8], #[case] expected: &str) {
#[case(b"let a = 1 err> /dev/null")]
#[case(b"let a = 1 out> /dev/null")]
#[case(b"mut a = 1 err> /dev/null")]
#[case(b"mut a = 1 out> /dev/null")]
#[case(b"let a = 1 out+err> /dev/null")]
#[case(b"mut a = 1 out+err> /dev/null")]
fn test_redirection_with_letmut(#[case] phase: &[u8]) {
let engine_state = EngineState::new();
let mut working_set = StateWorkingSet::new(&engine_state);
let _block = parse(&mut working_set, None, phase, true);
match expected {
"RedirectionInLetMut" => assert!(matches!(
working_set.parse_errors.first(),
Some(ParseError::RedirectionInLetMut(_, _))
)),
"AssignInPipeline" => assert!(matches!(
working_set.parse_errors.first(),
Some(ParseError::AssignInPipeline(_, _, _, _))
)),
_ => panic!("unexpected pattern"),
}
assert!(matches!(
working_set.parse_errors.first(),
Some(ParseError::RedirectingBuiltinCommand(_, _, _))
));
}
#[test]
@ -814,18 +736,11 @@ fn test_nothing_comparison_neq() {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
assert!(matches!(
&expressions.elements[0],
PipelineElement::Expression(
_,
Expression {
expr: Expr::BinaryOp(..),
..
}
)
))
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert!(matches!(&element.expr.expr, Expr::BinaryOp(..)));
}
mod string {
@ -840,13 +755,11 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::String("hello nushell".to_string()))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::String("hello nushell".to_string()))
}
mod interpolation {
@ -864,26 +777,23 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 2);
assert_eq!(subexprs.len(), 2);
assert_eq!(subexprs[0], &Expr::String("hello ".to_string()));
assert_eq!(subexprs[0], &Expr::String("hello ".to_string()));
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
} else {
panic!("Not an expression")
}
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
}
#[test]
@ -896,25 +806,21 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1);
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs[0], &Expr::String("hello (39 + 3)".to_string()));
} else {
panic!("Not an expression")
}
assert_eq!(subexprs[0], &Expr::String("hello (39 + 3)".to_string()));
}
#[test]
@ -927,27 +833,23 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1);
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 2);
assert_eq!(subexprs.len(), 2);
assert_eq!(subexprs[0], &Expr::String("hello \\".to_string()));
assert_eq!(subexprs[0], &Expr::String("hello \\".to_string()));
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
} else {
panic!("Not an expression")
}
assert!(matches!(subexprs[1], &Expr::FullCellPath(..)));
}
#[test]
@ -960,24 +862,20 @@ mod string {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(expressions.len(), 1);
let subexprs: Vec<&Expr> = match &element.expr.expr {
Expr::StringInterpolation(expressions) => {
expressions.iter().map(|e| &e.expr).collect()
}
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
let subexprs: Vec<&Expr> = match expr {
Expression {
expr: Expr::StringInterpolation(expressions),
..
} => expressions.iter().map(|e| &e.expr).collect(),
_ => panic!("Expected an `Expr::StringInterpolation`"),
};
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs[0], &Expr::String("(1 + 3)(7 - 5)".to_string()));
} else {
panic!("Not an expression")
}
assert_eq!(subexprs.len(), 1);
assert_eq!(subexprs[0], &Expr::String("(1 + 3)(7 - 5)".to_string()));
}
#[test]
@ -1084,24 +982,19 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block length");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
Some(_),
None,
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
),
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
None,
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1143,24 +1036,19 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 2, "{tag} block len 2");
let expressions = &block.pipelines[1];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
Some(_),
None,
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
),
let pipeline = &block.pipelines[1];
assert_eq!(pipeline.len(), 1, "{tag}: expression length 1");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
None,
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1189,24 +1077,19 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block len 1");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
Some(_),
None,
None,
RangeOperator {
inclusion: the_inclusion,
..
},
),
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
None,
None,
RangeOperator {
inclusion: the_inclusion,
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1235,24 +1118,19 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block len 1");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
None,
None,
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
),
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
None,
None,
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1281,24 +1159,19 @@ mod range {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1, "{tag}: block length 1");
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1, "{tag}: expression length 1");
if let PipelineElement::Expression(
_,
Expression {
expr:
Expr::Range(
Some(_),
Some(_),
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
),
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1, "{tag}: expression length");
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
if let Expr::Range(
Some(_),
Some(_),
Some(_),
RangeOperator {
inclusion: the_inclusion,
..
},
) = expressions.elements[0]
) = element.expr.expr
{
assert_eq!(
the_inclusion, inclusion,
@ -1671,31 +1544,21 @@ mod input_types {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 2);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 2);
assert!(pipeline.elements[0].redirection.is_none());
assert!(pipeline.elements[1].redirection.is_none());
match &expressions.elements[0] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[0].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"ls").unwrap();
assert_eq!(call.decl_id, expected_id)
}
_ => panic!("Expected expression Call not found"),
}
match &expressions.elements[1] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[1].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"group-by").unwrap();
assert_eq!(call.decl_id, expected_id)
}
@ -1718,15 +1581,10 @@ mod input_types {
engine_state.merge_delta(delta).unwrap();
let expressions = &block.pipelines[0];
match &expressions.elements[3] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let pipeline = &block.pipelines[0];
assert!(pipeline.elements[3].redirection.is_none());
match &pipeline.elements[3].expr.expr {
Expr::Call(call) => {
let arg = &call.arguments[0];
match arg {
Argument::Positional(a) => match &a.expr {
@ -1734,17 +1592,12 @@ mod input_types {
Expr::Subexpression(id) => {
let block = engine_state.get_block(*id);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 2);
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 2);
assert!(pipeline.elements[1].redirection.is_none());
match &expressions.elements[1] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[1].expr.expr {
Expr::Call(call) => {
let working_set = StateWorkingSet::new(&engine_state);
let expected_id = working_set.find_decl(b"min").unwrap();
assert_eq!(call.decl_id, expected_id)
@ -1776,29 +1629,20 @@ mod input_types {
assert!(working_set.parse_errors.is_empty());
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
match &expressions.elements[2] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
let pipeline = &block.pipelines[0];
assert!(pipeline.elements[2].redirection.is_none());
assert!(pipeline.elements[3].redirection.is_none());
match &pipeline.elements[2].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"with-column").unwrap();
assert_eq!(call.decl_id, expected_id)
}
_ => panic!("Expected expression Call not found"),
}
match &expressions.elements[3] {
PipelineElement::Expression(
_,
Expression {
expr: Expr::Call(call),
..
},
) => {
match &pipeline.elements[3].expr.expr {
Expr::Call(call) => {
let expected_id = working_set.find_decl(b"collect").unwrap();
assert_eq!(call.decl_id, expected_id)
}

18
crates/nu-parser/tests/test_parser_unicode_escapes.rs
View File

@ -1,13 +1,9 @@
#![cfg(test)]
//use nu_parser::ParseError;
use nu_parser::*;
use nu_protocol::{
//ast::{Expr, Expression, PipelineElement},
ast::{Expr, PipelineElement},
//engine::{Command, EngineState, Stack, StateWorkingSet},
ast::Expr,
engine::{EngineState, StateWorkingSet},
//Signature, SyntaxShape,
};
pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) {
@ -19,13 +15,11 @@ pub fn do_test(test: &[u8], expected: &str, error_contains: Option<&str>) {
match working_set.parse_errors.first() {
None => {
assert_eq!(block.len(), 1);
let expressions = &block.pipelines[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions.elements[0] {
assert_eq!(expr.expr, Expr::String(expected.to_string()))
} else {
panic!("Not an expression")
}
let pipeline = &block.pipelines[0];
assert_eq!(pipeline.len(), 1);
let element = &pipeline.elements[0];
assert!(element.redirection.is_none());
assert_eq!(element.expr.expr, Expr::String(expected.to_string()));
}
Some(pev) => match error_contains {
None => {