nushell/crates/nu-parser/src
Horasal b943cbedff
skip comments and eols while parsing pipeline (#10149)
This pr 
- fixes https://github.com/nushell/nushell/issues/10143
- fixes https://github.com/nushell/nushell/issues/5559

# Description

Current `lite_parse` does not handle multiple line comments and eols in
pipeline.
When parsing the following tokens:


| `"abcdefg"` | ` \|` | `# foobar` | ` \n` | `split chars` |
| ------------- | ------------- |------------- |-------------
|------------- |
| [Command] | [Pipe] | [Comment] | [Eol] | [Command] |
| | | Last Token |Current Token | |

`TokenContent::Eol` handler only checks if `last_token` is `Pipe` but it
will be broken if there exist any other thing, e.g. extra `[Comment]` in
this example.

This pr make the following change:

- While parsing `[Eol]`, try to find the last non-comment token as
`last_token`
- Comment is supposed as `[Comment]+` or `([Comment] [Eol])+`
- `[Eol]+` is still parsed just like current nu (i.e. generates
`nothing`).

Notice that this pr is just a quick patch if more comment/eol related
issue occures, `lite_parser` may need a rewrite.

# User-Facing Changes

Now the following pipeline works: 

```bash
1 | # comment
each { |it| $it + 2 } | # comment
math sum
```

Comment will not end the pipeline in interactive mode:

```bash
❯ 1 | # comment   (now enter multiple line mode instead of end)
▶▶ # foo
▶▶ 2
```

# Tests + Formatting

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

# After Submitting

None

---------

Co-authored-by: Horasal <horsal@horsal.dev>
2023-08-30 13:24:13 -05:00
..
deparse.rs allow empty string arguments (#9420) 2023-06-13 07:30:30 -05:00
flatten.rs Avoid blocking when o+e> redirects too much stderr message (#8784) 2023-05-17 17:47:03 -05:00
known_external.rs Sort entries in scope commands; Fix usage of externs (#10039) 2023-08-17 16:37:01 +02:00
lex.rs allow comment in multiple line pipeline (#9436) 2023-06-15 13:11:42 +02:00
lib.rs Allow parse-time evaluation of calls, pipelines and subexpressions (#9499) 2023-08-26 16:41:29 +03:00
lite_parser.rs skip comments and eols while parsing pipeline (#10149) 2023-08-30 13:24:13 -05:00
parse_keywords.rs Allow parse-time evaluation of calls, pipelines and subexpressions (#9499) 2023-08-26 16:41:29 +03:00
parse_patterns.rs add match guards (#9621) 2023-07-16 12:25:12 +12:00
parser_path.rs Add virtual path abstraction layer (#9245) 2023-05-23 23:48:50 +03:00
parser.rs handle empty pipeline while parsing let (fix Issue10083) (#10116) 2023-08-28 13:38:11 +02:00
type_check.rs Custom command input/output types (#9690) 2023-07-15 09:51:28 +12:00