Move test runner to standard library.
Originated from #8819
# After Submitting
I'll update the documentation about testing:
http://www.nushell.sh/book/testing.html
---------
Co-authored-by: Mate Farkas <Mate.Farkas@oneidentity.com>
# Description
Don't wait for the tests to pass to try running the coverage job. In
theory that would save some wasted runs but waiting for it can also slow
down the review if you want to inspect that a certain branch is covered
by tests.
# Tests + Formatting
Coverage runs immediately and can also fail due to failing tests
# Description
Let's see if we don't run out of disk space as quickly again
This reverts commit 0e496f900d.
(#8677)
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
# Tests + Formatting
Same coverage setup as before
# Description
as we now have a prelude thanks to #8627, i'd like to work on the
structure of the library 😋
and i think the first step is to make it a true standalone crate 😏
this PR
- moves all the library from `crates/nu-utils/standard_library/` to
`crates/nu-std/`
- moves the `rust` loading code from `src/run.rs` to
`crates/nu-std/src/lib.rs`
# Description
The coverage testing on CI has been broken for at least a week. It looks
like we might have to do some work, but for now, disabling it so we
don't give bad stability info to contributors.
# User-Facing Changes
_(List of all changes that impact the user experience here. This helps
us keep track of breaking changes.)_
# Tests + Formatting
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass
- `cargo run -- crates/nu-utils/standard_library/tests.nu` to run the
tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
# After Submitting
If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
Should close one of the tasks in #8450.
# Description
> **Note**
> in order of appearance in the global diff
- 1b7497c419 adds the `std-tests` job to
the CI which
1. installs `nushell` in the runner
2. run the `tests.nu` module
> see `open .github/workflows/ci.yml | get jobs.std-tests | to yaml`
-
[`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1)
is where all the magic happens => see below
- 🧪799c7eb7fd introduces some
bugs and failing test to see how the CI behaves => see how the [tests
failed](https://github.com/nushell/nushell/actions/runs/4460098237/jobs/7833018256)
as expected ❌
- 🧪 and c3de1fafb5 reverts the
failing tests, i.e. the previous commit, leaving a standard library
whose tests all pass 🎉 => see the [tests
passing](https://github.com/nushell/nushell/actions/runs/4460153434/jobs/7833110719?pr=8525#step:5:1)
now ✔️
## the changes to the runner
> see
[`ec85b6fd`..`9c122115`](ec85b6fd3fc004cd94e3fada5c8e5fe2714fd629..9c12211564ca8ee90ed65ae45776dccb8f8e4ef1)
the issue with the previous runner was the following: the clever trick
of using `nu -c "use ...; test"` did print the errors when occuring but
they did not capture the true "failure", i.e. in all cases the
`$env.LAST_EXIT_CODE` was set to `0`, never stopping the CI when a test
failed 🤔
i first tried to `try` / `catch` the error in
ec85b6fd3f which kinda worked but only
throw a single error, the first one
i thought it was not the best and started thinking about a solution to
have a complete report of all failing tests, at once, to avoid running
the CI multiple times!
the easiest solution i found was the one i implemented in
9c12211564
> **Warning**
> this changes the structure of the runner quite a bit, but the `for`
loops where annoying to manipulate structured data and allow the runner
to draw a complete report...
now the runner does the following
- compute the list of all available tests in a table with the `file`,
`module` and `name` columns (first part of the pipe until `flatten` and
`rename`)
- run the tests one by one computing the new `pass` column
- with a `log info`
- captures the failing ones => puts `true` in `pass` if the test passes,
`false` otherwise
- if at least one test has failed, throw a single error with the list of
failing tests
### hope you'll like it 😌
# User-Facing Changes
```
$nothing
```
# Tests + Formatting
the standard tests now return a true error that will stop the CI
# After Submitting
```
$nothing
```
Bumps
[actions-rust-lang/setup-rust-toolchain](https://github.com/actions-rust-lang/setup-rust-toolchain)
from 1.4.2 to 1.4.3.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions-rust-lang/setup-rust-toolchain/blob/main/CHANGELOG.md">actions-rust-lang/setup-rust-toolchain's
changelog</a>.</em></p>
<blockquote>
<h2>[1.4.3] - 2023-02-21</h2>
<h3>Fixed</h3>
<ul>
<li>Executing the action twice for different toolchains now no longer
fails around unstable features <a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/12">#12</a>.
If multiple toolchains are installed, the
"CARGO_REGISTRIES_CRATES_IO_PROTOCOL" can be downgraded to
"git" if any of the installed toolchains require it.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="64fef3b541"><code>64fef3b</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/13">#13</a>
from actions-rust-lang/double-run</li>
<li><a
href="cea2ca57ed"><code>cea2ca5</code></a>
Add changelog entry</li>
<li><a
href="f010a58728"><code>f010a58</code></a>
Always downgrade the registry protocol to supported versions</li>
<li>See full diff in <a
href="https://github.com/actions-rust-lang/setup-rust-toolchain/compare/v1.4.2...v1.4.3">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Description
- Try to run `cargo test` with out narrowing.
- Remove restrictions around plugins -> include plugin tests
# User-Facing Changes
None
# Tests + Formatting
Qualifying test suite unchanged
# Description
Avoids running out of disk through additional files after cratification
on Windows.
Revert as soon as we reduced the binary or test footprint.
Will mess with the coverage values without change in test suite as we
only track the linux conditional compilation
Should resolve the CI failure for #8181
# User-Facing Changes
None
# Tests + Formatting
Altered coverage value thresholds might be the result
Bumps
[actions-rust-lang/setup-rust-toolchain](https://github.com/actions-rust-lang/setup-rust-toolchain)
from 1.3.7 to 1.4.2.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions-rust-lang/setup-rust-toolchain/blob/main/CHANGELOG.md">actions-rust-lang/setup-rust-toolchain's
changelog</a>.</em></p>
<blockquote>
<h2>[1.4.2] - 2023-02-15</h2>
<h3>Fixed</h3>
<ul>
<li>Tweak sparse registry version regex to better work with 1.68 nightly
versions.</li>
<li>Fix command not found issue</li>
</ul>
<h2>[1.4.1] - 2023-02-13</h2>
<h3>Fixed</h3>
<ul>
<li>
<p>Fixed running on macOS <a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/9">#9</a>
<a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/10">#10</a>
The macOS images have an ancient version of bash, but the action relies
on "newer" features than 2014.
We install bash via brew (already pre-installed) to have a new enough
version.</p>
<p>The CI is extended to also run on Windows and macOS to catch such
issues earlier in the future.</p>
<p>Thanks to <a
href="https://github.com/GeorgeHahn"><code>@GeorgeHahn</code></a> for
reporting the issue.</p>
</li>
</ul>
<h2>[1.4.0] - 2023-02-13</h2>
<h3>Changed</h3>
<ul>
<li>Only set environment variables, if they are not set before.
This allows setting environment variables before using this action and
keeping their values.</li>
<li>Enable stable sparse registry, except on versions 1.66 and 1.67
where this leads to errors.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9fa7c33ef0"><code>9fa7c33</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/11">#11</a>
from actions-rust-lang/tweak-regex</li>
<li><a
href="5f4f30a995"><code>5f4f30a</code></a>
Maybe fix rustup warning</li>
<li><a
href="2d7b97c05c"><code>2d7b97c</code></a>
Tweak sparse registry version regex and command not found</li>
<li><a
href="c7c759a5c9"><code>c7c759a</code></a>
New version with macOS fix</li>
<li><a
href="ba68a52e42"><code>ba68a52</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/10">#10</a>
from actions-rust-lang/issue-9</li>
<li><a
href="40e33d4912"><code>40e33d4</code></a>
Install newer bash on macOS</li>
<li><a
href="8ba1b441e5"><code>8ba1b44</code></a>
Run CI on more platforms</li>
<li><a
href="51b4f8316a"><code>51b4f83</code></a>
Bump version to 1.4.0</li>
<li><a
href="4605df10a0"><code>4605df1</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/actions-rust-lang/setup-rust-toolchain/issues/8">#8</a>
from actions-rust-lang/sparse-registry</li>
<li><a
href="eac5ebb2ae"><code>eac5ebb</code></a>
Fix: Use acceptable crate name (lowercase)</li>
<li>Additional commits viewable in <a
href="https://github.com/actions-rust-lang/setup-rust-toolchain/compare/v1.3.7...v1.4.2">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps
[actions-rust-lang/setup-rust-toolchain](https://github.com/actions-rust-lang/setup-rust-toolchain)
from 1.3.4 to 1.3.5.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions-rust-lang/setup-rust-toolchain/blob/main/CHANGELOG.md">actions-rust-lang/setup-rust-toolchain's
changelog</a>.</em></p>
<blockquote>
<h2>[1.3.5] - 2023-01-21</h2>
<h3>Changed</h3>
<ul>
<li>Use the newly stabilized setting to enable sparse registry access.
This speeds up access to the crate registry and is in addition to the
unstable nightly env var.
<a
href="https://github-redirect.dependabot.com/rust-lang/cargo/pull/11224">rust-lang/cargo#11224</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="bc88fd0b3e"><code>bc88fd0</code></a>
Enable sparse registry access after stabilization</li>
<li>See full diff in <a
href="https://github.com/actions-rust-lang/setup-rust-toolchain/compare/v1.3.4...v1.3.5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
The release job was pinned to Ubuntu 20.04 to avoid glibc breakage in
https://github.com/nushell/nushell/pull/7290. This PR updates the CI
jobs to keep things consistent (it would sure be unpleasant if things
worked in CI but not at release time).
Following up on #7180 with some feature cleanup:
- Move the `database` feature from `plugin` to `default`
- Rename the `database` feature to `sqlite`
- Remove `--features=extra` from a lot of scripts etc.
- No need to specify this, the `extra` feature is now the same as the
default feature set
- Remove the now-redundant 2nd Ubuntu test run
Currently we see CI failures due to a `chrono` upgrade with
deprecations.
Also on every new reedline release we also suffer from regular compile
problems.
Let's see, if we can use `cargo-cache` again for the tests after #6389
reduced the number of test binaries to build that are quite large due as
they statically link copies of the same engine.
This might be one of the reasons why the tests on windows exceeded the
allotted disk space.
We're no longer using `cargo nextest` for our main test job. The separate action for doctests was only necessary because `cargo nextest` does not support doctests, it can be removed.
Hoping this will result in less data cached but we'll see.
* Try removing debuginfo for ci builds
* oops, wrong inherits
* extra flag
* nextest doesn't support --profile in the same way
* try to allow for a ci-specific target
* Oops, run more tests
* Add cache+docs to plugin CI job
* CI perf: don't statically link OpenSSL
* Run Clippy in plugin job
* comment
* bust cache
* trigger build
* remove nextest, split plugins better
* trigger CI
* try disabling embed-resource
* try disabling libgit2 in shadow-rs
* use lld linker on Windows
* Skip embedding Windows resource (slow) during tests
* disable shadow-rs git integration during tests
* go back to simpler shadow-rs and embed-resources setup
* some renaming
* forgot nextest
* trigger ci
* Remove Clippy and unnecessary build
* trigger CI
* disable lld
* reenable lld
* cleanup
* revert embed_resource change
The faster `cargo nextest` currently doesn't support running the doctests.
Thus, add an additional step for them with cargo's default test runner.
- Fix doctests for the `nu-pretty-hex` crate
* Add different features combinations
* Specify styles manually
* Fix args
* Fix typo
* Let other CI jobs finish if one fails
* Fix unused symbols without plugin feature
* Put "which" tests behind "which" feature
* Add Python virtualenv job
* Oops forgot git command
* Install Nushell in virtualenv tests
* Add names to steps; Test v.env in separate step
* cd into virtualenv
* Do not run on Python 2.7
* Build Nushell after formatting and clippy checks
