nushell/crates/nu-utils/standard_library/tests.nu

130 lines
3.7 KiB
Plaintext
Raw Normal View History

use std.nu *
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
# show a test record in a pretty way
#
# `$in` must be a `record<file: string, module: string, name: string, pass: bool>`.
#
# the output would be like
# - "<indentation> x <module> <test>" all in red if failed
# - "<indentation> <module> <test>" all in green if passed
def show-pretty-test [indent: int = 4] {
let test = $in
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
[
(" " * $indent)
(if $test.pass { ansi green } else { ansi red})
(if $test.pass { " " } else { char failed})
" "
$"($test.module) ($test.name)"
(ansi reset)
] | str join
}
stdlib: optimize test search and add better errors (#8626) the first part of this PR comes from a request from @presidento in #8525. the second one is an improvement of the error support. # Description this PR - computes `module_search_pattern` to only `ls` the selected modules => the goal is to save search time in the future with more tests - gives better errors when - the `--path` is invalid - the `--module` does not exist - the search is too strict ### examples ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × directory_not_found ╭─[<commandline>:1:1] 1 │ main --path does-not-exist · ───────┬────── · ╰── no such directory ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × module_not_found ╭─[<commandline>:1:1] 1 │ main --module does-not-exist · ───────┬────── · ╰── no such module in /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/ ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does_not_exist Error: × no test to run ``` instead of the previous ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × No matches found for /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/does-not-exist/test_*.nu ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:32:1] 32 │ let tests = ( 33 │ ls ($path | default $env.FILE_PWD | path join "test_*.nu") · ───────────────────────────┬─────────────────────────── · ╰── Pattern, file or folder not found 34 │ | each {|row| {file: $row.name name: ($row.name | path parse | get stem)}} ╰──── help: no matches found ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` # User-Facing Changes ``` $nothing ``` # Tests + Formatting ``` $nothing ``` # After Submitting ``` $nothing ```
2023-03-26 15:09:26 +02:00
def throw-error [error: record] {
error make {
msg: $"(ansi red)($error.msg)(ansi reset)"
label: {
text: ($error.label)
start: $error.span.start
end: $error.span.end
}
}
}
# Test executor
#
# It executes exported "test_*" commands in "test_*" modules
def main [
--path: path, # Path to look for tests. Default: directory of this file.
--module: string, # Module to run tests. Default: all test modules found.
--command: string, # Test command to run. Default: all test command found in the files.
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
--list, # list the selected tests without running them.
] {
stdlib: optimize test search and add better errors (#8626) the first part of this PR comes from a request from @presidento in #8525. the second one is an improvement of the error support. # Description this PR - computes `module_search_pattern` to only `ls` the selected modules => the goal is to save search time in the future with more tests - gives better errors when - the `--path` is invalid - the `--module` does not exist - the search is too strict ### examples ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × directory_not_found ╭─[<commandline>:1:1] 1 │ main --path does-not-exist · ───────┬────── · ╰── no such directory ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × module_not_found ╭─[<commandline>:1:1] 1 │ main --module does-not-exist · ───────┬────── · ╰── no such module in /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/ ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does_not_exist Error: × no test to run ``` instead of the previous ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × No matches found for /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/does-not-exist/test_*.nu ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:32:1] 32 │ let tests = ( 33 │ ls ($path | default $env.FILE_PWD | path join "test_*.nu") · ───────────────────────────┬─────────────────────────── · ╰── Pattern, file or folder not found 34 │ | each {|row| {file: $row.name name: ($row.name | path parse | get stem)}} ╰──── help: no matches found ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` # User-Facing Changes ``` $nothing ``` # Tests + Formatting ``` $nothing ``` # After Submitting ``` $nothing ```
2023-03-26 15:09:26 +02:00
let module_search_pattern = ({
stem: ($module | default "test_*")
extension: nu
} | path join)
if not ($path | is-empty) {
if not ($path | path exists) {
throw-error {
msg: "directory_not_found"
label: "no such directory"
span: (metadata $path | get span)
}
}
}
let path = ($path | default $env.FILE_PWD)
if not ($module | is-empty) {
if not ($path | path join $module_search_pattern | path exists) {
throw-error {
msg: "module_not_found"
label: $"no such module in ($path)"
span: (metadata $module | get span)
}
}
}
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
let tests = (
stdlib: optimize test search and add better errors (#8626) the first part of this PR comes from a request from @presidento in #8525. the second one is an improvement of the error support. # Description this PR - computes `module_search_pattern` to only `ls` the selected modules => the goal is to save search time in the future with more tests - gives better errors when - the `--path` is invalid - the `--module` does not exist - the search is too strict ### examples ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × directory_not_found ╭─[<commandline>:1:1] 1 │ main --path does-not-exist · ───────┬────── · ╰── no such directory ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × module_not_found ╭─[<commandline>:1:1] 1 │ main --module does-not-exist · ───────┬────── · ╰── no such module in /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/ ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does_not_exist Error: × no test to run ``` instead of the previous ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × No matches found for /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/does-not-exist/test_*.nu ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:32:1] 32 │ let tests = ( 33 │ ls ($path | default $env.FILE_PWD | path join "test_*.nu") · ───────────────────────────┬─────────────────────────── · ╰── Pattern, file or folder not found 34 │ | each {|row| {file: $row.name name: ($row.name | path parse | get stem)}} ╰──── help: no matches found ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` # User-Facing Changes ``` $nothing ``` # Tests + Formatting ``` $nothing ``` # After Submitting ``` $nothing ```
2023-03-26 15:09:26 +02:00
ls ($path | path join $module_search_pattern)
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
| each {|row| {file: $row.name name: ($row.name | path parse | get stem)}}
| upsert test {|module|
nu -c $'use ($module.file) *; $nu.scope.commands | select name module_name | to nuon'
| from nuon
| where module_name == $module.name
| where ($it.name | str starts-with "test_")
| get name
}
| flatten
| rename file module name
)
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
let tests_to_run = (if not ($command | is-empty) {
$tests | where name == $command
} else if not ($module | is-empty) {
$tests | where module == $module
} else {
$tests
})
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
if $list {
return ($tests_to_run | select module name file)
}
stdlib: optimize test search and add better errors (#8626) the first part of this PR comes from a request from @presidento in #8525. the second one is an improvement of the error support. # Description this PR - computes `module_search_pattern` to only `ls` the selected modules => the goal is to save search time in the future with more tests - gives better errors when - the `--path` is invalid - the `--module` does not exist - the search is too strict ### examples ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × directory_not_found ╭─[<commandline>:1:1] 1 │ main --path does-not-exist · ───────┬────── · ╰── no such directory ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × module_not_found ╭─[<commandline>:1:1] 1 │ main --module does-not-exist · ───────┬────── · ╰── no such module in /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/ ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does_not_exist Error: × no test to run ``` instead of the previous ```bash >_ nu crates/nu-utils/standard_library/tests.nu --path does-not-exist Error: × No matches found for /home/amtoine/.local/share/git/store/github.com/amtoine/nushell/does-not-exist/test_*.nu ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:32:1] 32 │ let tests = ( 33 │ ls ($path | default $env.FILE_PWD | path join "test_*.nu") · ───────────────────────────┬─────────────────────────── · ╰── Pattern, file or folder not found 34 │ | each {|row| {file: $row.name name: ($row.name | path parse | get stem)}} ╰──── help: no matches found ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --module does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` ```bash >_ nu crates/nu-utils/standard_library/tests.nu --command does-not-exist Error: × expected table from pipeline ╭─[/home/amtoine/.local/share/git/store/github.com/amtoine/nushell/crates/nu-utils/standard_library/tests.nu:59:1] 59 │ $tests_to_run 60 │ | group-by module · ────┬─── · ╰── requires a table input 61 │ | transpose name tests ╰──── ``` # User-Facing Changes ``` $nothing ``` # Tests + Formatting ``` $nothing ``` # After Submitting ``` $nothing ```
2023-03-26 15:09:26 +02:00
if ($tests_to_run | is-empty) {
error make --unspanned {msg: "no test to run"}
}
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
let tests = (
$tests_to_run
| group-by module
| transpose name tests
| each {|module|
log info $"Running tests in ($module.name)"
$module.tests | each {|test|
log debug $"Running test ($test.name)"
let did_pass = (try {
nu -c $'use ($test.file) ($test.name); ($test.name)'
true
} catch { false })
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
$test | merge ({pass: $did_pass})
}
add `dirs` command to std lib (#8368) # Description Prototype replacement for `enter`, `n`, `p`, `exit` built-ins implemented as scripts in standard library. MVP-level capabilities (rough hack), for feedback please. Not intended to merge and ship as is. _(Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience.)_ # User-Facing Changes New command in standard library ```nushell 〉use ~/src/rust/nushell/crates/nu-utils/standard_library/dirs.nu ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉help dirs module dirs.nu -- maintain list of remembered directories + navigate them todo: * expand relative to absolute paths (or relative to some prefix?) * what if user does `cd` by hand? Module: dirs Exported commands: add (dirs add), drop, next (dirs next), prev (dirs prev), show (dirs show) This module exports environment. ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉dirs add ~/src/rust/nushell /etc ~/.cargo -------------------------------------- /home/bobhy/src/rust/nushell -------------------------------------- 〉dirs next 2 ------------------------------------------- /home/bobhy/.cargo ------------------------------------------- 〉dirs show ╭───┬─────────┬────────────────────╮ │ # │ current │ path │ ├───┼─────────┼────────────────────┤ │ 0 │ │ /home/bobhy │ │ 1 │ │ ~/src/rust/nushell │ │ 2 │ │ /etc │ │ 3 │ ==> │ ~/.cargo │ ╰───┴─────────┴────────────────────╯ ------------------------------------------- /home/bobhy/.cargo ------------------------------------------- 〉dirs drop ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉dirs show ╭───┬─────────┬────────────────────╮ │ # │ current │ path │ ├───┼─────────┼────────────────────┤ │ 0 │ ==> │ /home/bobhy │ │ 1 │ │ ~/src/rust/nushell │ │ 2 │ │ /etc │ ╰───┴─────────┴────────────────────╯ ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉 ``` # Tests + Formatting Haven't even looked at stdlib `tests.nu` yet. Other todos: * address module todos. * integrate into std lib, rather than as standalone module. Somehow arrange for `use .../standard_library/std.nu` to load this module without having to put all the source in `std.nu`? * Maybe command should be `std dirs ...`? * what else do `enter` and `exit` do that this should do? Then deprecate those commands. Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-11 23:31:09 +01:00
}
standard library: bring the tests into the main CI (#8525) Should close one of the tasks in #8450. # Description > **Note** > in order of appearance in the global diff - 1b7497c41966306aa3103a95a9b5ef5df7111ee4 adds the `std-tests` job to the CI which 1. installs `nushell` in the runner 2. run the `tests.nu` module > see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml` - [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) is where all the magic happens => see below - :test_tube: 799c7eb7fd5f140289b36b9dbc00329c50e2fbda introduces some bugs and failing test to see how the CI behaves => see how the [tests failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256) as expected :x: - :test_tube: and c3de1fafb5c5313e30c08c9ca57e09df33b61b74 reverts the failing tests, i.e. the previous commit, leaving a standard library whose tests all pass :tada: => see the [tests passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1) now :heavy_check_mark: ## the changes to the runner > see [`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1) the issue with the previous runner was the following: the clever trick of using `nu -c "use ...; test"` did print the errors when occuring but they did not capture the true "failure", i.e. in all cases the `$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test failed :thinking: i first tried to `try` / `catch` the error in ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629 which kinda worked but only throw a single error, the first one i thought it was not the best and started thinking about a solution to have a complete report of all failing tests, at once, to avoid running the CI multiple times! the easiest solution i found was the one i implemented in 9c12211564ca8ee90ed65ae45776dccb8f8e4ef1 > **Warning** > this changes the structure of the runner quite a bit, but the `for` loops where annoying to manipulate structured data and allow the runner to draw a complete report... now the runner does the following - compute the list of all available tests in a table with the `file`, `module` and `name` columns (first part of the pipe until `flatten` and `rename`) - run the tests one by one computing the new `pass` column - with a `log info` - captures the failing ones => puts `true` in `pass` if the test passes, `false` otherwise - if at least one test has failed, throw a single error with the list of failing tests ### hope you'll like it :relieved: # User-Facing Changes ``` $nothing ``` # Tests + Formatting the standard tests now return a true error that will stop the CI # After Submitting ``` $nothing ```
2023-03-25 19:29:08 +01:00
| flatten
)
if not ($tests | where not pass | is-empty) {
let text = ([
$"(ansi purple)some tests did not pass (char lparen)see complete errors above(char rparen):(ansi reset)"
""
($tests | each {|test| ($test | show-pretty-test 4)} | str join "\n")
""
] | str join "\n")
error make --unspanned { msg: $text }
add `dirs` command to std lib (#8368) # Description Prototype replacement for `enter`, `n`, `p`, `exit` built-ins implemented as scripts in standard library. MVP-level capabilities (rough hack), for feedback please. Not intended to merge and ship as is. _(Description of your pull request goes here. **Provide examples and/or screenshots** if your changes affect the user experience.)_ # User-Facing Changes New command in standard library ```nushell 〉use ~/src/rust/nushell/crates/nu-utils/standard_library/dirs.nu ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉help dirs module dirs.nu -- maintain list of remembered directories + navigate them todo: * expand relative to absolute paths (or relative to some prefix?) * what if user does `cd` by hand? Module: dirs Exported commands: add (dirs add), drop, next (dirs next), prev (dirs prev), show (dirs show) This module exports environment. ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉dirs add ~/src/rust/nushell /etc ~/.cargo -------------------------------------- /home/bobhy/src/rust/nushell -------------------------------------- 〉dirs next 2 ------------------------------------------- /home/bobhy/.cargo ------------------------------------------- 〉dirs show ╭───┬─────────┬────────────────────╮ │ # │ current │ path │ ├───┼─────────┼────────────────────┤ │ 0 │ │ /home/bobhy │ │ 1 │ │ ~/src/rust/nushell │ │ 2 │ │ /etc │ │ 3 │ ==> │ ~/.cargo │ ╰───┴─────────┴────────────────────╯ ------------------------------------------- /home/bobhy/.cargo ------------------------------------------- 〉dirs drop ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉dirs show ╭───┬─────────┬────────────────────╮ │ # │ current │ path │ ├───┼─────────┼────────────────────┤ │ 0 │ ==> │ /home/bobhy │ │ 1 │ │ ~/src/rust/nushell │ │ 2 │ │ /etc │ ╰───┴─────────┴────────────────────╯ ---------------------------------------------- /home/bobhy ---------------------------------------------- 〉 ``` # Tests + Formatting Haven't even looked at stdlib `tests.nu` yet. Other todos: * address module todos. * integrate into std lib, rather than as standalone module. Somehow arrange for `use .../standard_library/std.nu` to load this module without having to put all the source in `std.nu`? * Maybe command should be `std dirs ...`? * what else do `enter` and `exit` do that this should do? Then deprecate those commands. Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A clippy::needless_collect` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass # After Submitting If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date.
2023-03-11 23:31:09 +01:00
}
}