forked from extern/nushell
fa2e6e5d53
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx
you can also mention related issues, PRs or discussions!
-->
# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.
Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
> [!NOTE]
> This PR description originally used examples where the `generator`
closure returned a list. It has since been updated to use records
instead.
The `unfold` command allows users to dynamically generate streams of
data. The stream is generated by repeatedly invoking a `generator`
closure. The `generator` closure accepts a single argument and returns a
record containing two optional keys: 'out' and 'next'. Each invocation,
the 'out' value, if present, is added to the stream. If a 'next' key is
present, it is used as the next argument to the closure, otherwise
generation stops.
The name "unfold" is borrowed from other functional-programming
languages. Whereas `fold` (or `reduce`) takes a stream of values and
outputs a single value, `unfold` takes a single value and outputs a
stream of values.
### Examples
A common example of using `unfold` is to generate a fibbonacci sequence.
See
[here](6ffdac103c/src/sources.rs (L65))
for an example of this in rust's `itertools`.
```nushell
> unfold [0, 1] {|fib| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} } | first 10
───┬────
0 │ 0
1 │ 1
2 │ 1
3 │ 2
4 │ 3
5 │ 5
6 │ 8
7 │ 13
8 │ 21
9 │ 34
───┴────
```
This command is particularly useful when consuming paginated APIs, like
Github's. Previously, nushell users might use a loop and buffer
responses into a list, before returning all responses at once. However,
this behavior is not desirable if the result result is very large. Using
`unfold` avoids buffering and allows subsequent pipeline stages to use
the data concurrently, as it's being fetched.
#### Before
```nushell
mut pages = []
for page in 1.. {
let resp = http get (
{
scheme: https,
host: "api.github.com",
path: "/repos/nushell/nushell/issues",
params: {
page: $page,
per_page: $PAGE_SIZE
}
} | url join)
$pages = ($pages | append $resp)
if ($resp | length) < $PAGE_SIZE {
break
}
}
$pages
```
#### After
```nu
unfold 1 {|page|
let resp = http get (
{
scheme: https,
host: "api.github.com",
path: "/repos/nushell/nushell/issues",
params: {
page: $page,
per_page: $PAGE_SIZE
}
} | url join)
if ($resp | length) < $PAGE_SIZE {
{out: $resp}
} else {
{out: $resp, next: ($page + 1)}
}
}
```
# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
- An `unfold` generator is added to the default context.
# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.
Make sure you've run and fixed any issues with these commands:
- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use std testing; testing run-tests --path
crates/nu-std"` to run the tests for the standard library
> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
Given the complexity of the `generator` closure's return value, it would
be good to document the semantics of `unfold` and provide some in-depth
examples showcasing what it can accomplish.
103 lines
2.1 KiB
Rust
103 lines
2.1 KiB
Rust
use nu_test_support::{nu, pipeline};
|
|
|
|
#[test]
|
|
fn unfold_no_next_break() {
|
|
let actual =
|
|
nu!("unfold 1 {|x| if $x == 3 { {out: $x}} else { {out: $x, next: ($x + 1)} }} | to nuon");
|
|
|
|
assert_eq!(actual.out, "[1, 2, 3]");
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_null_break() {
|
|
let actual = nu!("unfold 1 {|x| if $x <= 3 { {out: $x, next: ($x + 1)} }} | to nuon");
|
|
|
|
assert_eq!(actual.out, "[1, 2, 3]");
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_allows_empty_output() {
|
|
let actual = nu!(pipeline(
|
|
r#"
|
|
unfold 0 {|x|
|
|
if $x == 1 {
|
|
{next: ($x + 1)}
|
|
} else if $x < 3 {
|
|
{out: $x, next: ($x + 1)}
|
|
}
|
|
} | to nuon
|
|
"#
|
|
));
|
|
|
|
assert_eq!(actual.out, "[0, 2]");
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_allows_no_output() {
|
|
let actual = nu!(pipeline(
|
|
r#"
|
|
unfold 0 {|x|
|
|
if $x < 3 {
|
|
{next: ($x + 1)}
|
|
}
|
|
} | to nuon
|
|
"#
|
|
));
|
|
|
|
assert_eq!(actual.out, "[]");
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_allows_null_state() {
|
|
let actual = nu!(pipeline(
|
|
r#"
|
|
unfold 0 {|x|
|
|
if $x == null {
|
|
{out: "done"}
|
|
} else if $x < 1 {
|
|
{out: "going", next: ($x + 1)}
|
|
} else {
|
|
{out: "stopping", next: null}
|
|
}
|
|
} | to nuon
|
|
"#
|
|
));
|
|
|
|
assert_eq!(actual.out, "[going, stopping, done]");
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_allows_null_output() {
|
|
let actual = nu!(pipeline(
|
|
r#"
|
|
unfold 0 {|x|
|
|
if $x == 3 {
|
|
{out: "done"}
|
|
} else {
|
|
{out: null, next: ($x + 1)}
|
|
}
|
|
} | to nuon
|
|
"#
|
|
));
|
|
|
|
assert_eq!(actual.out, "[null, null, null, done]");
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_disallows_extra_keys() {
|
|
let actual = nu!("unfold 0 {|x| {foo: bar, out: $x}}");
|
|
assert!(actual.err.contains("Invalid block return"));
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_disallows_list() {
|
|
let actual = nu!("unfold 0 {|x| [$x, ($x + 1)]}");
|
|
assert!(actual.err.contains("Invalid block return"));
|
|
}
|
|
|
|
#[test]
|
|
fn unfold_disallows_primitive() {
|
|
let actual = nu!("unfold 0 {|x| 1}");
|
|
assert!(actual.err.contains("Invalid block return"));
|
|
}
|