nushell/src/tests/test_parser.rs

742 lines
15 KiB
Rust
Raw Normal View History

use crate::tests::{fail_test, run_test, run_test_with_env, TestResult};
use std::collections::HashMap;
use super::run_test_contains;
#[test]
fn env_shorthand() -> TestResult {
run_test("FOO=BAR if false { 3 } else { 4 }", "4")
}
#[test]
fn subcommand() -> TestResult {
run_test("def foo [] {}; def \"foo bar\" [] {3}; foo bar", "3")
}
#[test]
fn alias_1() -> TestResult {
run_test("def foo [$x] { $x + 10 }; alias f = foo; f 100", "110")
}
#[test]
fn ints_with_underscores() -> TestResult {
run_test("1_0000_0000_0000 + 10", "1000000000010")
}
#[test]
fn floats_with_underscores() -> TestResult {
run_test("3.1415_9265_3589_793 * 2", "6.283185307179586")
}
#[test]
fn bin_ints_with_underscores() -> TestResult {
run_test("0b_10100_11101_10010", "21426")
}
#[test]
fn oct_ints_with_underscores() -> TestResult {
run_test("0o2443_6442_7652_0044", "90422533333028")
}
#[test]
fn hex_ints_with_underscores() -> TestResult {
run_test("0x68__9d__6a", "6856042")
}
#[test]
fn alias_2() -> TestResult {
run_test(
"def foo [$x $y] { $x + $y + 10 }; alias f = foo 33; f 100",
"143",
)
}
#[test]
fn alias_2_multi_word() -> TestResult {
run_test(
r#"def "foo bar" [$x $y] { $x + $y + 10 }; alias f = foo bar 33; f 100"#,
"143",
)
}
Re-implement aliases (#8123) # Description This PR adds an alternative alias implementation. Old aliases still work but you need to use `old-alias` instead of `alias`. Instead of replacing spans in the original code and re-parsing, which proved to be extremely error-prone and a constant source of panics, the new implementation creates a new command that references the old command. Consider the new alias defined as `alias ll = ls -l`. The parser creates a new command called `ll` and remembers that it is actually a `ls` command called with the `-l` flag. Then, when the parser sees the `ll` command, it will translate it to `ls -l` and passes to it any parameters that were passed to the call to `ll`. It works quite similar to how known externals defined with `extern` are implemented. The new alias implementation should work the same way as the old aliases, including exporting from modules, referencing both known and unknown externals. It seems to preserve custom completions and pipeline metadata. It is quite robust in most cases but there are some rough edges (see later). Fixes https://github.com/nushell/nushell/issues/7648, https://github.com/nushell/nushell/issues/8026, https://github.com/nushell/nushell/issues/7512, https://github.com/nushell/nushell/issues/5780, https://github.com/nushell/nushell/issues/7754 No effect: https://github.com/nushell/nushell/issues/8122 (we might revisit the completions code after this PR) Should use custom command instead: https://github.com/nushell/nushell/issues/6048 # User-Facing Changes Since aliases are now basically commands, it has some new implications: 1. `alias spam = "spam"` (requires command call) * **workaround**: use `alias spam = echo "spam"` 2. `def foo [] { 'foo' }; alias foo = ls -l` (foo defined more than once) * **workaround**: use different name (commands also have this limitation) 4. `alias ls = (ls | sort-by type name -i)` * **workaround**: Use custom command. _The common issue with this is that it is currently not easy to pass flags through custom commands and command referencing itself will lead to stack overflow. Both of these issues are meant to be addressed._ 5. TODO: Help messages, `which` command, `$nu.scope.aliases`, etc. * Should we treat the aliases as commands or should they be separated from regular commands? 6. Needs better error message and syntax highlight for recursed alias (`alias f = f`) 7. Can't create alias with the same name as existing command (`alias ls = ls -a`) * Might be possible to add support for it (not 100% sure) 8. Standalone `alias` doesn't list aliases anymore 9. Can't alias parser keywords (e.g., stuff like `alias ou = overlay use` won't work) * TODO: Needs a better error message when attempting to do so # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-27 08:44:05 +01:00
#[ignore = "TODO: Allow alias to alias existing command with the same name"]
#[test]
fn alias_recursion() -> TestResult {
Re-implement aliases (#8123) # Description This PR adds an alternative alias implementation. Old aliases still work but you need to use `old-alias` instead of `alias`. Instead of replacing spans in the original code and re-parsing, which proved to be extremely error-prone and a constant source of panics, the new implementation creates a new command that references the old command. Consider the new alias defined as `alias ll = ls -l`. The parser creates a new command called `ll` and remembers that it is actually a `ls` command called with the `-l` flag. Then, when the parser sees the `ll` command, it will translate it to `ls -l` and passes to it any parameters that were passed to the call to `ll`. It works quite similar to how known externals defined with `extern` are implemented. The new alias implementation should work the same way as the old aliases, including exporting from modules, referencing both known and unknown externals. It seems to preserve custom completions and pipeline metadata. It is quite robust in most cases but there are some rough edges (see later). Fixes https://github.com/nushell/nushell/issues/7648, https://github.com/nushell/nushell/issues/8026, https://github.com/nushell/nushell/issues/7512, https://github.com/nushell/nushell/issues/5780, https://github.com/nushell/nushell/issues/7754 No effect: https://github.com/nushell/nushell/issues/8122 (we might revisit the completions code after this PR) Should use custom command instead: https://github.com/nushell/nushell/issues/6048 # User-Facing Changes Since aliases are now basically commands, it has some new implications: 1. `alias spam = "spam"` (requires command call) * **workaround**: use `alias spam = echo "spam"` 2. `def foo [] { 'foo' }; alias foo = ls -l` (foo defined more than once) * **workaround**: use different name (commands also have this limitation) 4. `alias ls = (ls | sort-by type name -i)` * **workaround**: Use custom command. _The common issue with this is that it is currently not easy to pass flags through custom commands and command referencing itself will lead to stack overflow. Both of these issues are meant to be addressed._ 5. TODO: Help messages, `which` command, `$nu.scope.aliases`, etc. * Should we treat the aliases as commands or should they be separated from regular commands? 6. Needs better error message and syntax highlight for recursed alias (`alias f = f`) 7. Can't create alias with the same name as existing command (`alias ls = ls -a`) * Might be possible to add support for it (not 100% sure) 8. Standalone `alias` doesn't list aliases anymore 9. Can't alias parser keywords (e.g., stuff like `alias ou = overlay use` won't work) * TODO: Needs a better error message when attempting to do so # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-27 08:44:05 +01:00
run_test_contains(r#"alias ls = ls -a; ls"#, " ")
}
#[test]
fn block_param1() -> TestResult {
run_test("[3] | each { |it| $it + 10 } | get 0", "13")
}
#[test]
fn block_param2() -> TestResult {
run_test("[3] | each { |y| $y + 10 } | get 0", "13")
}
#[test]
fn block_param3_list_iteration() -> TestResult {
run_test("[1,2,3] | each { |it| $it + 10 } | get 1", "12")
}
#[test]
fn block_param4_list_iteration() -> TestResult {
run_test("[1,2,3] | each { |y| $y + 10 } | get 2", "13")
}
#[test]
fn range_iteration1() -> TestResult {
run_test("1..4 | each { |y| $y + 10 } | get 0", "11")
}
#[test]
fn range_iteration2() -> TestResult {
run_test("4..1 | each { |y| $y + 100 } | get 3", "101")
}
#[test]
fn simple_value_iteration() -> TestResult {
run_test("4 | each { |it| $it + 10 }", "14")
}
#[test]
fn comment_multiline() -> TestResult {
run_test(
r#"def foo [] {
let x = 1 + 2 # comment
let y = 3 + 4 # another comment
$x + $y
}; foo"#,
"10",
)
}
#[test]
fn comment_skipping_1() -> TestResult {
run_test(
r#"let x = {
y: 20
# foo
}; $x.y"#,
"20",
)
}
#[test]
fn comment_skipping_2() -> TestResult {
run_test(
r#"let x = {
y: 20
# foo
z: 40
}; $x.z"#,
"40",
)
}
skip comments and eols while parsing pipeline (#10149) This pr - fixes https://github.com/nushell/nushell/issues/10143 - fixes https://github.com/nushell/nushell/issues/5559 # Description Current `lite_parse` does not handle multiple line comments and eols in pipeline. When parsing the following tokens: | `"abcdefg"` | ` \|` | `# foobar` | ` \n` | `split chars` | | ------------- | ------------- |------------- |------------- |------------- | | [Command] | [Pipe] | [Comment] | [Eol] | [Command] | | | | Last Token |Current Token | | `TokenContent::Eol` handler only checks if `last_token` is `Pipe` but it will be broken if there exist any other thing, e.g. extra `[Comment]` in this example. This pr make the following change: - While parsing `[Eol]`, try to find the last non-comment token as `last_token` - Comment is supposed as `[Comment]+` or `([Comment] [Eol])+` - `[Eol]+` is still parsed just like current nu (i.e. generates `nothing`). Notice that this pr is just a quick patch if more comment/eol related issue occures, `lite_parser` may need a rewrite. # User-Facing Changes Now the following pipeline works: ```bash 1 | # comment each { |it| $it + 2 } | # comment math sum ``` Comment will not end the pipeline in interactive mode: ```bash ❯ 1 | # comment (now enter multiple line mode instead of end) ▶▶ # foo ▶▶ 2 ``` # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` # After Submitting None --------- Co-authored-by: Horasal <horsal@horsal.dev>
2023-08-30 20:24:13 +02:00
#[test]
fn comment_skipping_in_pipeline_1() -> TestResult {
run_test(
r#"[1,2,3] | #comment
each { |$it| $it + 2 } | # foo
math sum #bar"#,
"12",
)
}
#[test]
fn comment_skipping_in_pipeline_2() -> TestResult {
run_test(
r#"[1,2,3] #comment
| #comment2
each { |$it| $it + 2 } #foo
| # bar
math sum #baz"#,
"12",
)
}
#[test]
fn comment_skipping_in_pipeline_3() -> TestResult {
run_test(
r#"[1,2,3] | #comment
#comment2
each { |$it| $it + 2 } #foo
| # bar
#baz
math sum #foobar"#,
"12",
)
}
#[test]
fn bad_var_name() -> TestResult {
fail_test(r#"let $"foo bar" = 4"#, "can't contain")
}
#[test]
fn bad_var_name2() -> TestResult {
fail_test(r#"let $foo-bar = 4"#, "valid variable")
}
#[test]
fn assignment_with_no_var() -> TestResult {
let cases = [
"let = if $",
"mut = if $",
"const = if $",
"let = 'foo' | $in; $x | describe",
"mut = 'foo' | $in; $x | describe",
];
let expected = "valid variable";
for case in cases {
fail_test(case, expected)?;
}
Ok(())
}
#[test]
fn long_flag() -> TestResult {
run_test(
r#"([a, b, c] | enumerate | each --keep-empty { |e| if $e.index != 1 { 100 }}).1 | to nuon"#,
"null",
)
}
2021-12-27 04:04:22 +01:00
#[test]
fn for_in_missing_var_name() -> TestResult {
fail_test("for in", "missing")
}
#[test]
fn multiline_pipe_in_block() -> TestResult {
run_test(
r#"do {
echo hello |
str length
}"#,
"5",
)
}
2022-01-06 22:06:54 +01:00
#[test]
fn bad_short_flag() -> TestResult {
fail_test(r#"def foo3 [-l?:int] { $l }"#, "short flag")
}
2022-01-10 03:52:01 +01:00
2022-01-19 15:58:12 +01:00
#[test]
fn quotes_with_equals() -> TestResult {
run_test(
r#"let query_prefix = "https://api.github.com/search/issues?q=repo:nushell/"; $query_prefix"#,
"https://api.github.com/search/issues?q=repo:nushell/",
)
}
#[test]
fn string_interp_with_equals() -> TestResult {
run_test(
r#"let query_prefix = $"https://api.github.com/search/issues?q=repo:nushell/"; $query_prefix"#,
"https://api.github.com/search/issues?q=repo:nushell/",
)
}
#[test]
fn recursive_parse() -> TestResult {
run_test(r#"def c [] { c }; echo done"#, "done")
}
#[test]
fn commands_have_usage() -> TestResult {
run_test_contains(
r#"
# This is a test
#
# To see if I have cool usage
def foo [] {}
help foo"#,
"cool usage",
)
}
#[test]
fn equals_separates_long_flag() -> TestResult {
run_test(
A `fill` command to replace `str lpad` and `str rpad` (#7846) # Description The point of this command is to allow you to be able to format ints, floats, filesizes, and strings with an alignment, padding, and a fill character, as strings. It's meant to take the place of `str lpad` and `str rpad`. ``` > help fill Fill and Align Search terms: display, render, format, pad, align Usage: > fill {flags} Flags: -h, --help - Display the help message for this command -w, --width <Int> - The width of the output. Defaults to 1 -a, --alignment <String> - The alignment of the output. Defaults to Left (Left(l), Right(r), Center(c/m), MiddleRight(cr/mr)) -c, --character <String> - The character to fill with. Defaults to ' ' (space) Signatures: <number> | fill -> <string> <string> | fill -> <string> Examples: Fill a string on the left side to a width of 15 with the character '─' > 'nushell' | fill -a l -c '─' -w 15 Fill a string on the right side to a width of 15 with the character '─' > 'nushell' | fill -a r -c '─' -w 15 Fill a string on both sides to a width of 15 with the character '─' > 'nushell' | fill -a m -c '─' -w 15 Fill a number on the left side to a width of 5 with the character '0' > 1 | fill --alignment right --character 0 --width 5 Fill a filesize on the left side to a width of 5 with the character '0' > 1kib | fill --alignment middle --character 0 --width 10 ``` ![image](https://user-images.githubusercontent.com/343840/214133752-6fc93fa7-4003-4eb4-96ed-cd967312e244.png) # User-Facing Changes Deprecated `str lpad` and `str rpad`. # Tests + Formatting Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-02-09 21:56:52 +01:00
r#"'nushell' | fill --alignment right --width=10 --character='-'"#,
"---nushell",
)
}
#[test]
remove let-env, focus on mutating $env (#9574) # Description For years, Nushell has used `let-env` to set a single environment variable. As our work on scoping continued, we refined what it meant for a variable to be in scope using `let` but never updated how `let-env` would work. Instead, `let-env` confusingly created mutations to the command's copy of `$env`. So, to help fix the mental model and point people to the right way of thinking about what changing the environment means, this PR removes `let-env` to encourage people to think of it as updating the command's environment variable via mutation. Before: ``` let-env FOO = "BAR" ``` Now: ``` $env.FOO = "BAR" ``` It's also a good reminder that the environment owned by the command is in the `$env` variable rather than global like it is in other shells. # User-Facing Changes BREAKING CHANGE BREAKING CHANGE This completely removes `let-env FOO = "BAR"` so that we can focus on `$env.FOO = "BAR"`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After / Before Submitting integration scripts to update: - :heavy_check_mark: [starship](https://github.com/starship/starship/blob/master/src/init/starship.nu) - :heavy_check_mark: [virtualenv](https://github.com/pypa/virtualenv/blob/main/src/virtualenv/activation/nushell/activate.nu) - :heavy_check_mark: [atuin](https://github.com/ellie/atuin/blob/main/atuin/src/shell/atuin.nu) (PR: https://github.com/ellie/atuin/pull/1080) - :x: [zoxide](https://github.com/ajeetdsouza/zoxide/blob/main/templates/nushell.txt) (PR: https://github.com/ajeetdsouza/zoxide/pull/587) - :heavy_check_mark: [oh-my-posh](https://github.com/JanDeDobbeleer/oh-my-posh/blob/main/src/shell/scripts/omp.nu) (pr: https://github.com/JanDeDobbeleer/oh-my-posh/pull/4011)
2023-06-30 21:57:51 +02:00
fn assign_expressions() -> TestResult {
let env = HashMap::from([("VENV_OLD_PATH", "Foobar"), ("Path", "Quux")]);
run_test_with_env(
remove let-env, focus on mutating $env (#9574) # Description For years, Nushell has used `let-env` to set a single environment variable. As our work on scoping continued, we refined what it meant for a variable to be in scope using `let` but never updated how `let-env` would work. Instead, `let-env` confusingly created mutations to the command's copy of `$env`. So, to help fix the mental model and point people to the right way of thinking about what changing the environment means, this PR removes `let-env` to encourage people to think of it as updating the command's environment variable via mutation. Before: ``` let-env FOO = "BAR" ``` Now: ``` $env.FOO = "BAR" ``` It's also a good reminder that the environment owned by the command is in the `$env` variable rather than global like it is in other shells. # User-Facing Changes BREAKING CHANGE BREAKING CHANGE This completely removes `let-env FOO = "BAR"` so that we can focus on `$env.FOO = "BAR"`. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After / Before Submitting integration scripts to update: - :heavy_check_mark: [starship](https://github.com/starship/starship/blob/master/src/init/starship.nu) - :heavy_check_mark: [virtualenv](https://github.com/pypa/virtualenv/blob/main/src/virtualenv/activation/nushell/activate.nu) - :heavy_check_mark: [atuin](https://github.com/ellie/atuin/blob/main/atuin/src/shell/atuin.nu) (PR: https://github.com/ellie/atuin/pull/1080) - :x: [zoxide](https://github.com/ajeetdsouza/zoxide/blob/main/templates/nushell.txt) (PR: https://github.com/ajeetdsouza/zoxide/pull/587) - :heavy_check_mark: [oh-my-posh](https://github.com/JanDeDobbeleer/oh-my-posh/blob/main/src/shell/scripts/omp.nu) (pr: https://github.com/JanDeDobbeleer/oh-my-posh/pull/4011)
2023-06-30 21:57:51 +02:00
r#"$env.Path = (if ($env | columns | "VENV_OLD_PATH" in $in) { $env.VENV_OLD_PATH } else { $env.Path }); echo $env.Path"#,
"Foobar",
&env,
)
}
#[test]
fn string_interpolation_paren_test() -> TestResult {
run_test(r#"$"('(')(')')""#, "()")
}
#[test]
fn string_interpolation_paren_test2() -> TestResult {
run_test(r#"$"('(')test(')')""#, "(test)")
}
#[test]
fn string_interpolation_paren_test3() -> TestResult {
run_test(r#"$"('(')("test")test(')')""#, "(testtest)")
}
#[test]
fn string_interpolation_escaping() -> TestResult {
run_test(r#"$"hello\nworld" | lines | length"#, "2")
}
#[test]
fn capture_multiple_commands() -> TestResult {
run_test(
r#"
let CONST_A = 'Hello'
def 'say-hi' [] {
echo (call-me)
}
def 'call-me' [] {
echo $CONST_A
}
[(say-hi) (call-me)] | str join
"#,
"HelloHello",
)
}
#[test]
fn capture_multiple_commands2() -> TestResult {
run_test(
r#"
let CONST_A = 'Hello'
def 'call-me' [] {
echo $CONST_A
}
def 'say-hi' [] {
echo (call-me)
}
[(say-hi) (call-me)] | str join
"#,
"HelloHello",
)
}
#[test]
fn capture_multiple_commands3() -> TestResult {
run_test(
r#"
let CONST_A = 'Hello'
def 'say-hi' [] {
echo (call-me)
}
def 'call-me' [] {
echo $CONST_A
}
[(call-me) (say-hi)] | str join
"#,
"HelloHello",
)
}
#[test]
fn capture_multiple_commands4() -> TestResult {
run_test(
r#"
let CONST_A = 'Hello'
def 'call-me' [] {
echo $CONST_A
}
def 'say-hi' [] {
echo (call-me)
}
[(call-me) (say-hi)] | str join
"#,
"HelloHello",
)
}
#[test]
fn capture_row_condition() -> TestResult {
run_test(
r#"let name = "foo"; [foo] | where $'($name)' =~ $it | str join"#,
"foo",
)
}
#[test]
fn starts_with_operator_succeeds() -> TestResult {
run_test(
r#"[Moe Larry Curly] | where $it starts-with L | str join"#,
"Larry",
)
}
#[test]
fn ends_with_operator_succeeds() -> TestResult {
run_test(
r#"[Moe Larry Curly] | where $it ends-with ly | str join"#,
"Curly",
)
}
#[test]
fn proper_missing_param() -> TestResult {
fail_test(r#"def foo [x y z w] { }; foo a b c"#, "missing w")
}
#[test]
fn block_arity_check1() -> TestResult {
fail_test(
r#"ls | each { |x, y, z| 1}"#,
"expected 2 closure parameters",
)
}
// deprecating former support for escapes like `/uNNNN`, dropping test.
#[test]
fn string_escape_unicode_extended() -> TestResult {
run_test(r#""\u{015B}\u{1f10b}""#, "ś🄋")
}
#[test]
fn string_escape_interpolation() -> TestResult {
run_test(r#"$"\u{015B}(char hamburger)abc""#, "ś≡abc")
}
#[test]
fn string_escape_interpolation2() -> TestResult {
run_test(r#"$"2 + 2 is \(2 + 2)""#, "2 + 2 is (2 + 2)")
}
#[test]
fn proper_rest_types() -> TestResult {
run_test(
r#"def foo [--verbose(-v), # my test flag
...rest: int # my rest comment
] { if $verbose { print "verbose!" } else { print "not verbose!" } }; foo"#,
"not verbose!",
)
}
#[test]
fn single_value_row_condition() -> TestResult {
run_test(
r#"[[a, b]; [true, false], [true, true]] | where a | length"#,
"2",
)
}
2022-04-06 21:10:25 +02:00
Fix exponential parser time on sequence of [[[[ (#10439) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> Before this change, parsing `[[[[[[[[[[[[[[[[[[[[[[` would cause nushell to consume several gigabytes of memory, now it should be linear in time. The old code first tried parsing the head of the table as a list and then after that it checked if it got more arguments. If it didn't, it throws away the previous result and tries to parse the whole thing as a list, which means we call `parse_list_expression` twice for each call to `parse_table_expression`, resulting in the exponential growth The fix is to simply check that we have all the arguments we need before parsing the head of the table, so we know that we will either call parse_list_expression only on sub-expressions or on the whole thing, never both. Fixes #10438 # User-Facing Changes Should give a noticable speedup when typing a sequence of `[[[[[[` open brackets <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting I would like to add tests, but I'm not sure how to do that without crashing CI with OOM on regression - [x] Don't forget to add tests that cover your changes. - [x] `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - [x] `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - [x] `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library <!-- > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-09-20 17:53:48 +02:00
#[test]
fn performance_nested_lists() -> TestResult {
// Parser used to be exponential on deeply nested lists
// TODO: Add a timeout
fail_test(r#"[[[[[[[[[[[[[[[[[[[[[[[[[[[["#, "Unexpected end of code")
}
2022-04-06 21:10:25 +02:00
#[test]
fn unary_not_1() -> TestResult {
run_test(r#"not false"#, "true")
}
#[test]
fn unary_not_2() -> TestResult {
run_test(r#"not (false)"#, "true")
}
#[test]
fn unary_not_3() -> TestResult {
run_test(r#"(not false)"#, "true")
}
#[test]
fn unary_not_4() -> TestResult {
run_test(r#"if not false { "hello" } else { "world" }"#, "hello")
}
#[test]
fn unary_not_5() -> TestResult {
run_test(
r#"if not not not not false { "hello" } else { "world" }"#,
"world",
)
}
#[test]
fn unary_not_6() -> TestResult {
run_test(
r#"[[name, present]; [abc, true], [def, false]] | where not present | get name.0"#,
"def",
)
}
allow comment in multiple line pipeline (#9436) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [*linking keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description - fixes: #5517 - fixes: #9250 For the following commands: ``` ls # | le | length ``` I found that it generates a bad lite parsing result: ``` LiteBlock { block: [ LitePipeline { commands: [ Command(None, LiteCommand { comments: [], parts: [Span { start: 138600, end: 138602 }] }) ] }, LitePipeline { commands: [ Command(Some(Span { start: 138610, end: 138611 }), LiteCommand { comments: [Span { start: 138603, end: 138609 }], parts: [Span { start: 138612, end: 138618 }] }) ] } ] } ``` Which should contains only one `LitePipeline`, and the second `LitePipeline` is generated because of `Eol` lex token: ``` [ Token { contents: Item, span: Span { start: 138600, end: 138602 } }, Token { contents: Eol, span: Span { start: 138602, end: 138603 } }, // it generates the second LitePipeline Token { contents: Comment, span: Span { start: 138603, end: 138609 } }, Token { contents: Pipe, span: Span { start: 138610, end: 138611 } }, Token { contents: Item, span: Span { start: 138612, end: 138618 } } ] ``` To fix the issue, I remove the `Eol` token when we meet `Comment` right after `Eol`, then it will generate a good LiteBlock, and everything will work fine. ### After the fix: Token: ``` [ Token { contents: Item, span: Span { start: 138618, end: 138620 } }, Token { contents: Comment, span: Span { start: 138622, end: 138628 } }, Token { contents: Pipe, span: Span { start: 138629, end: 138630 } }, Token { contents: Item, span: Span { start: 138631, end: 138637 } } ] ``` LiteBlock: ``` LiteBlock { block: [ LitePipeline { commands: [ Command( None, LiteCommand { comments: [Span { start: 138622, end: 138628 }], parts: [Span { start: 138618, end: 138620 }] } ), Command( Some(Span { start: 138629, end: 138630 }), LiteCommand { comments: [], parts: [Span { start: 138631, end: 138637 }] })] }] } ``` <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience. --> # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-06-15 13:11:42 +02:00
#[test]
fn comment_in_multiple_pipelines() -> TestResult {
run_test(
r#"[[name, present]; [abc, true], [def, false]]
# | where not present
| get name.0"#,
"abc",
)
}
#[test]
fn date_literal() -> TestResult {
run_test(r#"2022-09-10 | date to-record | get day"#, "10")
}
2022-04-22 21:14:31 +02:00
#[test]
fn and_and_or() -> TestResult {
run_test(r#"true and false or true"#, "true")
}
#[test]
fn and_and_xor() -> TestResult {
// Assumes the precedence NOT > AND > XOR > OR
run_test(r#"true and true xor true and false"#, "true")
}
#[test]
fn or_and_xor() -> TestResult {
// Assumes the precedence NOT > AND > XOR > OR
run_test(r#"true or false xor true or false"#, "true")
}
#[test]
fn unbalanced_delimiter() -> TestResult {
fail_test(r#"{a:{b:5}}}"#, "unbalanced { and }")
}
#[test]
fn unbalanced_delimiter2() -> TestResult {
fail_test(r#"{}#.}"#, "unbalanced { and }")
}
#[test]
fn unbalanced_delimiter3() -> TestResult {
fail_test(r#"{"#, "Unexpected end of code")
}
#[test]
fn unbalanced_delimiter4() -> TestResult {
fail_test(r#"}"#, "unbalanced { and }")
}
#[test]
fn register_with_string_literal() -> TestResult {
fail_test(r#"register 'nu-plugin-math'"#, "File not found")
}
#[test]
fn register_with_string_constant() -> TestResult {
let input = "\
const file = 'nu-plugin-math'
register $file
";
// should not fail with `not a constant`
fail_test(input, "File not found")
}
#[test]
fn register_with_string_variable() -> TestResult {
let input = "\
let file = 'nu-plugin-math'
register $file
";
fail_test(input, "Value is not a parse-time constant")
}
#[test]
fn register_with_non_string_constant() -> TestResult {
let input = "\
const file = 6
register $file
";
fail_test(input, "expected string, found int")
}
#[test]
fn extern_errors_with_no_space_between_params_and_name_1() -> TestResult {
fail_test("extern cmd[]", "expected space")
}
#[test]
fn extern_errors_with_no_space_between_params_and_name_2() -> TestResult {
fail_test("extern cmd(--flag)", "expected space")
}
#[test]
fn duration_with_underscores_1() -> TestResult {
run_test("420_min", "7hr")
}
#[test]
fn duration_with_underscores_2() -> TestResult {
run_test("1_000_000sec", "1wk 4day 13hr 46min 40sec")
}
#[test]
fn duration_with_underscores_3() -> TestResult {
fail_test("1_000_d_ay", "executable was not found")
}
#[test]
fn duration_with_faulty_number() -> TestResult {
fail_test("sleep 4-ms", "duration value must be a number")
}
#[test]
fn filesize_with_underscores_1() -> TestResult {
run_test("420_mb", "400.5 MiB")
}
#[test]
fn filesize_with_underscores_2() -> TestResult {
run_test("1_000_000B", "976.6 KiB")
}
#[test]
fn filesize_with_underscores_3() -> TestResult {
fail_test("42m_b", "executable was not found")
}
#[test]
fn filesize_is_not_hex() -> TestResult {
run_test("0x42b", "1067")
}
Improve type hovers (#9515) # Description This PR does a few things to help improve type hovers and, in the process, fixes a few outstanding issues in the type system. Here's a list of the changes: * `for` now will try to infer the type of the iteration variable based on the expression it's given. This fixes things like `for x in [1, 2, 3] { }` where `x` now properly gets the int type. * Removed old input/output type fields from the signature, focuses on the vec of signatures. Updated a bunch of dataframe commands that hadn't moved over. This helps tie things together a bit better * Fixed inference of types from subexpressions to use the last expression in the block * Fixed handling of explicit types in `let` and `mut` calls, so we now respect that as the authoritative type I also tried to add `def` input/output type inference, but unfortunately we only know the predecl types universally, which means we won't have enough information to properly know what the types of the custom commands are. # User-Facing Changes Script typechecking will get tighter in some cases Hovers should be more accurate in some cases that previously resorted to any. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- crates/nu-std/tests/run.nu` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> --------- Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2023-06-28 19:19:48 +02:00
#[test]
fn let_variable_type_mismatch() -> TestResult {
fail_test(r#"let x: int = "foo""#, "expected int, found string")
}
Custom command input/output types (#9690) # Description This adds input/output types to custom commands. These are input/output pairs that related an input type to an output type. For example (a single int-to-int input/output pair): ``` def foo []: int -> int { ... } ``` You can also have multiple input/output pairs: ``` def bar []: [int -> string, string -> list<string>] { ... } ``` These types are checked during definition time in the parser. If the block does not match the type, the user will get a parser error. This `:` to begin the input/output signatures should immediately follow the argument signature as shown above. The PR also improves type parsing by re-using the shape parser. The shape parser is now the canonical way to parse types/shapes in user code. This PR also splits `extern` into `extern`/`extern-wrapped` because of the parser limitation that a multi-span argument (which Signature now is) can't precede an optional argument. `extern-wrapped` now takes the required block that was previously optional. # User-Facing Changes The change to `extern` to split into `extern` and `extern-wrapped` is a breaking change. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-07-14 23:51:28 +02:00
#[test]
fn def_with_input_output_1() -> TestResult {
run_test(r#"def foo []: nothing -> int { 3 }; foo"#, "3")
}
#[test]
fn def_with_input_output_2() -> TestResult {
run_test(
r#"def foo []: [int -> int, string -> int] { 3 }; 10 | foo"#,
"3",
)
}
#[test]
fn def_with_input_output_3() -> TestResult {
run_test(
r#"def foo []: [int -> int, string -> int] { 3 }; "bob" | foo"#,
"3",
)
}
#[test]
fn def_with_input_output_mismatch_1() -> TestResult {
fail_test(
r#"def foo []: [int -> int, string -> int] { 3 }; foo"#,
"command doesn't support",
)
}
#[test]
fn def_with_input_output_mismatch_2() -> TestResult {
fail_test(
r#"def foo []: [int -> int, string -> int] { 3 }; {x: 2} | foo"#,
"command doesn't support",
)
}
#[test]
fn def_with_input_output_broken_1() -> TestResult {
fail_test(r#"def foo []: int { 3 }"#, "expected arrow")
}
#[test]
fn def_with_input_output_broken_2() -> TestResult {
fail_test(r#"def foo []: int -> { 3 }"#, "expected type")
}
#[test]
fn def_with_in_var_let_1() -> TestResult {
run_test(
r#"def foo []: [int -> int, string -> int] { let x = $in; if ($x | describe) == "int" { 3 } else { 4 } }; "100" | foo"#,
"4",
)
}
#[test]
fn def_with_in_var_let_2() -> TestResult {
run_test(
r#"def foo []: [int -> int, string -> int] { let x = $in; if ($x | describe) == "int" { 3 } else { 4 } }; 100 | foo"#,
"3",
)
}
#[test]
fn def_with_in_var_mut_1() -> TestResult {
run_test(
r#"def foo []: [int -> int, string -> int] { mut x = $in; if ($x | describe) == "int" { 3 } else { 4 } }; "100" | foo"#,
"4",
)
}
#[test]
fn def_with_in_var_mut_2() -> TestResult {
run_test(
r#"def foo []: [int -> int, string -> int] { mut x = $in; if ($x | describe) == "int" { 3 } else { 4 } }; 100 | foo"#,
"3",
)
}
Fix capture logic for inner closures (#9754) # Description This fixes the variable capture logic for closures in two cases: * Closures inside of closures did not properly register the closures (or lack thereof) in the outer closure * Closures which called their inner closures before definition did not properly calculate the closures of the outer closure Example of the first case: ``` do { let b = 3; def c [] { $b }; c } ``` Example of the second case (notice `c` is called before it is defined): ``` do { let b = 3; c; def c [] { $b }; c } ``` # User-Facing Changes This should strictly allow closures to work more correctly. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-07-20 21:10:54 +02:00
#[test]
fn properly_nest_captures() -> TestResult {
run_test(r#"do { let b = 3; def c [] { $b }; c }"#, "3")
}
#[test]
fn properly_nest_captures_call_first() -> TestResult {
run_test(r#"do { let b = 3; c; def c [] { $b }; c }"#, "3")
}
#[test]
fn properly_typecheck_rest_param() -> TestResult {
run_test(
r#"def foo [...rest: string] { $rest | length }; foo "a" "b" "c""#,
"3",
)
}
Fix the implied collect type to 'any' (#9827) # Description Previously, we had a bug slip in about implied collection caused by `$in`, that this output type would be of type `string`. The type system fixes in 0.83 now make this more visible and cause issues. This PR changes the output of the implied collection to `any`. At some point in the future, we may want to carry the type through where we can, but `any` should unblock using `$in`. fixes #9825 # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect -A clippy::result_large_err` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > **Note** > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. -->
2023-07-27 20:26:28 +02:00
#[test]
fn implied_collect_has_compatible_type() -> TestResult {
run_test(r#"let idx = 3 | $in; $idx < 1"#, "false")
}