History

Hudson Clark fa2e6e5d53 feat: Add unfold command (#10489 ) <!-- if this PR closes one or more issues, you can automatically link the PR with them by using one of the [linking keywords](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword), e.g. - this PR should close #xxxx - fixes #xxxx you can also mention related issues, PRs or discussions! --> # Description <!-- Thank you for improving Nushell. Please, check our [contributing guide](../CONTRIBUTING.md) and talk to the core team before making major changes. Description of your pull request goes here. Provide examples and/or screenshots if your changes affect the user experience. --> > [!NOTE] > This PR description originally used examples where the `generator` closure returned a list. It has since been updated to use records instead. The `unfold` command allows users to dynamically generate streams of data. The stream is generated by repeatedly invoking a `generator` closure. The `generator` closure accepts a single argument and returns a record containing two optional keys: 'out' and 'next'. Each invocation, the 'out' value, if present, is added to the stream. If a 'next' key is present, it is used as the next argument to the closure, otherwise generation stops. The name "unfold" is borrowed from other functional-programming languages. Whereas `fold` (or `reduce`) takes a stream of values and outputs a single value, `unfold` takes a single value and outputs a stream of values. ### Examples A common example of using `unfold` is to generate a fibbonacci sequence. See [here](`6ffdac103c/src/sources.rs (L65)`) for an example of this in rust's `itertools`. ```nushell > unfold [0, 1] {\|fib\| {out: $fib.0, next: [$fib.1, ($fib.0 + $fib.1)]} } \| first 10 ───┬──── 0 │ 0 1 │ 1 2 │ 1 3 │ 2 4 │ 3 5 │ 5 6 │ 8 7 │ 13 8 │ 21 9 │ 34 ───┴──── ``` This command is particularly useful when consuming paginated APIs, like Github's. Previously, nushell users might use a loop and buffer responses into a list, before returning all responses at once. However, this behavior is not desirable if the result result is very large. Using `unfold` avoids buffering and allows subsequent pipeline stages to use the data concurrently, as it's being fetched. #### Before ```nushell mut pages = [] for page in 1.. { let resp = http get ( { scheme: https, host: "api.github.com", path: "/repos/nushell/nushell/issues", params: { page: $page, per_page: $PAGE_SIZE } } \| url join) $pages = ($pages \| append $resp) if ($resp \| length) < $PAGE_SIZE { break } } $pages ``` #### After ```nu unfold 1 {\|page\| let resp = http get ( { scheme: https, host: "api.github.com", path: "/repos/nushell/nushell/issues", params: { page: $page, per_page: $PAGE_SIZE } } \| url join) if ($resp \| length) < $PAGE_SIZE { {out: $resp} } else { {out: $resp, next: ($page + 1)} } } ``` # User-Facing Changes <!-- List of all changes that impact the user experience here. This helps us keep track of breaking changes. --> - An `unfold` generator is added to the default context. # Tests + Formatting <!-- Don't forget to add tests that cover your changes. Make sure you've run and fixed any issues with these commands: - `cargo fmt --all -- --check` to check standard code formatting (`cargo fmt --all` applies these changes) - `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to check that you're using the standard code style - `cargo test --workspace` to check that all tests pass (on Windows make sure to [enable developer mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging)) - `cargo run -- -c "use std testing; testing run-tests --path crates/nu-std"` to run the tests for the standard library > Note > from `nushell` you can also use the `toolkit` as follows > ```bash > use toolkit.nu # or use an `env_change` hook to activate it automatically > toolkit check pr > ``` --> # After Submitting <!-- If your PR had any user-facing changes, update [the documentation](https://github.com/nushell/nushell.github.io) after the PR is merged, if necessary. This will help us keep the docs up to date. --> Given the complexity of the `generator` closure's return value, it would be good to document the semantics of `unfold` and provide some in-depth examples showcasing what it can accomplish.		2023-09-30 09:08:06 -05:00
..
nu_plugin_custom_values	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu_plugin_example	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu_plugin_formats	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu_plugin_gstat	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu_plugin_inc	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu_plugin_python	remove `vectorize_over_list` from python plugin (#9905 )	2023-08-03 16:46:48 +02:00
nu_plugin_query	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-cli	Fix editor config for reedline and `config nu/env` (#10535 )	2023-09-29 16:36:03 +02:00
nu-cmd-base	Fix editor config for reedline and `config nu/env` (#10535 )	2023-09-29 16:36:03 +02:00
nu-cmd-dataframe	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-cmd-extra	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-cmd-lang	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-color-config	fix magenta_reverse and friends (#10491 )	2023-09-24 14:43:17 -05:00
nu-command	feat: Add unfold command (#10489 )	2023-09-30 09:08:06 -05:00
nu-engine	chore: Small refactor of `eval.rs` (#10554 )	2023-09-29 21:57:15 +02:00
nu-explore	`explore`: highlight selected cell using background colour instead of cursor (#10533 )	2023-09-28 20:17:56 -05:00
nu-glob	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-json	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-parser	Rename `SyntaxShape::Custom` to `CompleterWrapper` (#10548 )	2023-09-29 19:22:58 +02:00
nu-path	Fix tilde-expansion for multi-byte unicode chars (#10434 )	2023-09-21 04:04:28 +12:00
nu-plugin	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-pretty-hex	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-protocol	Rename `SyntaxShape::Custom` to `CompleterWrapper` (#10548 )	2023-09-29 19:22:58 +02:00
nu-std	std dt datetime-diff: fix uninitialized field ref when borrowing (#10466 )	2023-09-24 10:53:56 +02:00
nu-system	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-table	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-term-grid	Bump to `0.85.1` development version (#10431 )	2023-09-20 18:38:42 +12:00
nu-test-support	Simplify `nu!` test macros. (#10403 )	2023-09-21 20:11:56 +02:00
nu-utils	fix windows default prompt slash direction (#10555 )	2023-09-30 08:18:14 +13:00
README.md	Remove old nushell/merge engine-q	2022-02-07 14:54:06 -05:00

README.md

Nushell core libraries and plugins

These sub-crates form both the foundation for Nu and a set of plugins which extend Nu with additional functionality.

Foundational libraries are split into two kinds of crates:

Core crates - those crates that work together to build the Nushell language engine
Support crates - a set of crates that support the engine with additional features like JSON support, ANSI support, and more.

Plugins are likewise also split into two types:

Core plugins - plugins that provide part of the default experience of Nu, including access to the system properties, processes, and web-connectivity features.
Extra plugins - these plugins run a wide range of different capabilities like working with different file types, charting, viewing binary data, and more.