nushell/tests/plugin_persistence/mod.rs

391 lines
11 KiB
Rust
Raw Normal View History

Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
//! The tests in this file check the soundness of plugin persistence. When a plugin is needed by Nu,
//! it is spawned only if it was not already running. Plugins that are spawned are kept running and
//! are referenced in the engine state. Plugins can be stopped by the user if desired, but not
//! removed.
use nu_test_support::{nu, nu_with_plugins};
#[test]
fn plugin_list_shows_installed_plugins() {
let out = nu_with_plugins!(
cwd: ".",
plugins: [("nu_plugin_inc"), ("nu_plugin_custom_values")],
r#"(plugin list).name | str join ','"#
);
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
assert_eq!("custom_values,inc", out.out);
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
assert!(out.status.success());
Allow plugins to report their own version and store it in the registry (#12883) # Description This allows plugins to report their version (and potentially other metadata in the future). The version is shown in `plugin list` and in `version`. The metadata is stored in the registry file, and reflects whatever was retrieved on `plugin add`, not necessarily the running binary. This can help you to diagnose if there's some kind of mismatch with what you expect. We could potentially use this functionality to show a warning or error if a plugin being run does not have the same version as what was in the cache file, suggesting `plugin add` be run again, but I haven't done that at this point. It is optional, and it requires the plugin author to make some code changes if they want to provide it, since I can't automatically determine the version of the calling crate or anything tricky like that to do it. Example: ``` > plugin list | select name version is_running pid ╭───┬────────────────┬─────────┬────────────┬─────╮ │ # │ name │ version │ is_running │ pid │ ├───┼────────────────┼─────────┼────────────┼─────┤ │ 0 │ example │ 0.93.1 │ false │ │ │ 1 │ gstat │ 0.93.1 │ false │ │ │ 2 │ inc │ 0.93.1 │ false │ │ │ 3 │ python_example │ 0.1.0 │ false │ │ ╰───┴────────────────┴─────────┴────────────┴─────╯ ``` cc @maxim-uvarov (he asked for it) # User-Facing Changes - `plugin list` gets a `version` column - `version` shows plugin versions when available - plugin authors *should* add `fn metadata()` to their `impl Plugin`, but don't have to # Tests + Formatting Tested the low level stuff and also the `plugin list` column. # After Submitting - [ ] update plugin guide docs - [ ] update plugin protocol docs (`Metadata` call & response) - [ ] update plugin template (`fn metadata()` should be easy) - [ ] release notes
2024-06-21 13:27:09 +02:00
}
#[test]
fn plugin_list_shows_installed_plugin_version() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"(plugin list).version.0"#
);
assert_eq!(env!("CARGO_PKG_VERSION"), out.out);
assert!(out.status.success());
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
}
#[test]
fn plugin_keeps_running_after_calling_it() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
plugin stop inc
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list).0.status == running | print
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
print ";"
"2.0.0" | inc -m | ignore
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list).0.status == running | print
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert_eq!(
"false;true", out.out,
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
"plugin list didn't show status = running"
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
);
assert!(out.status.success());
}
#[test]
fn plugin_process_exits_after_stop() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
"2.0.0" | inc -m | ignore
sleep 500ms
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
let pid = (plugin list).0.pid
if (ps | where pid == $pid | is-empty) {
error make {
msg: "plugin process not running initially"
}
}
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
plugin stop inc
let start = (date now)
mut cond = true
while $cond {
sleep 100ms
$cond = (
(ps | where pid == $pid | is-not-empty) and
((date now) - $start) < 5sec
)
}
((date now) - $start) | into int
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
assert!(out.status.success());
let nanos = out.out.parse::<i64>().expect("not a number");
assert!(
nanos < 5_000_000_000,
"not stopped after more than 5 seconds: {nanos} ns"
);
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
}
#[test]
fn plugin_stop_can_find_by_filename() {
let result = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"plugin stop (plugin list | where name == inc).0.filename"#
);
assert!(result.status.success());
assert!(result.err.is_empty());
}
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
#[test]
fn plugin_process_exits_when_nushell_exits() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
"2.0.0" | inc -m | ignore
(plugin list).0.pid | print
"#
);
assert!(!out.out.is_empty());
assert!(out.status.success());
let pid = out.out.parse::<u32>().expect("failed to parse pid");
// use nu to check if process exists
assert_eq!(
"0",
nu!(format!("sleep 500ms; ps | where pid == {pid} | length")).out,
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"plugin process {pid} is still running"
);
}
#[test]
fn plugin_commands_run_without_error() {
let out = nu_with_plugins!(
cwd: ".",
plugins: [
("nu_plugin_inc"),
("nu_plugin_example"),
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
("nu_plugin_custom_values"),
],
r#"
"2.0.0" | inc -m | ignore
example seq 1 10 | ignore
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
custom-value generate | ignore
"#
);
assert!(out.err.is_empty());
assert!(out.status.success());
}
#[test]
fn plugin_commands_run_multiple_times_without_error() {
let out = nu_with_plugins!(
cwd: ".",
plugins: [
("nu_plugin_inc"),
("nu_plugin_example"),
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
("nu_plugin_custom_values"),
],
r#"
["2.0.0" "2.1.0" "2.2.0"] | each { inc -m } | print
example seq 1 10 | ignore
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
custom-value generate | ignore
example seq 1 20 | ignore
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
custom-value generate2 | ignore
"#
);
assert!(out.err.is_empty());
assert!(out.status.success());
}
#[test]
fn multiple_plugin_commands_run_with_the_same_plugin_pid() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_custom_values"),
r#"
custom-value generate | ignore
(plugin list).0.pid | print
print ";"
custom-value generate2 | ignore
(plugin list).0.pid | print
"#
);
assert!(out.status.success());
let pids: Vec<&str> = out.out.split(';').collect();
assert_eq!(2, pids.len());
assert_eq!(pids[0], pids[1]);
}
#[test]
fn plugin_pid_changes_after_stop_then_run_again() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_custom_values"),
r#"
custom-value generate | ignore
(plugin list).0.pid | print
print ";"
plugin stop custom_values
custom-value generate2 | ignore
(plugin list).0.pid | print
"#
);
assert!(out.status.success());
let pids: Vec<&str> = out.out.split(';').collect();
assert_eq!(2, pids.len());
assert_ne!(pids[0], pids[1]);
}
#[test]
fn custom_values_can_still_be_passed_to_plugin_after_stop() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_custom_values"),
r#"
let cv = custom-value generate
plugin stop custom_values
$cv | custom-value update
"#
);
assert!(!out.out.is_empty());
assert!(out.err.is_empty());
assert!(out.status.success());
}
#[test]
fn custom_values_can_still_be_collapsed_after_stop() {
// print causes a collapse (ToBaseValue) call.
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_custom_values"),
r#"
let cv = custom-value generate
plugin stop custom_values
$cv | print
"#
);
assert!(!out.out.is_empty());
assert!(out.err.is_empty());
assert!(out.status.success());
}
#[test]
fn plugin_gc_can_be_configured_to_stop_plugins_immediately() {
// I know the test is to stop "immediately", but if we actually check immediately it could
// lead to a race condition. Using 100ms sleep just because with contention we don't really
// know for sure how long this could take
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
$env.config.plugin_gc = { default: { stop_after: 0sec } }
"2.3.0" | inc -M
sleep 100ms
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == inc).0.status == running
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
assert_eq!("false", out.out, "with config as default");
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
$env.config.plugin_gc = {
plugins: {
inc: { stop_after: 0sec }
}
}
"2.3.0" | inc -M
sleep 100ms
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == inc).0.status == running
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
assert_eq!("false", out.out, "with inc-specific config");
}
#[test]
fn plugin_gc_can_be_configured_to_stop_plugins_after_delay() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
$env.config.plugin_gc = { default: { stop_after: 50ms } }
"2.3.0" | inc -M
let start = (date now)
mut cond = true
while $cond {
sleep 100ms
$cond = (
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == inc).0.status == running and
((date now) - $start) < 5sec
)
}
((date now) - $start) | into int
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
let nanos = out.out.parse::<i64>().expect("not a number");
assert!(
nanos < 5_000_000_000,
"with config as default: more than 5 seconds: {nanos} ns"
);
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
$env.config.plugin_gc = {
plugins: {
inc: { stop_after: 50ms }
}
}
"2.3.0" | inc -M
let start = (date now)
mut cond = true
while $cond {
sleep 100ms
$cond = (
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == inc).0.status == running and
((date now) - $start) < 5sec
)
}
((date now) - $start) | into int
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
let nanos = out.out.parse::<i64>().expect("not a number");
assert!(
nanos < 5_000_000_000,
"with inc-specific config: more than 5 seconds: {nanos} ns"
);
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
}
#[test]
fn plugin_gc_can_be_configured_as_disabled() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
$env.config.plugin_gc = { default: { enabled: false, stop_after: 0sec } }
"2.3.0" | inc -M
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == inc).0.status == running
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
assert_eq!("true", out.out, "with config as default");
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_inc"),
r#"
$env.config.plugin_gc = {
default: { enabled: true, stop_after: 0sec }
plugins: {
inc: { enabled: false, stop_after: 0sec }
}
}
"2.3.0" | inc -M
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == inc).0.status == running
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
assert_eq!("true", out.out, "with inc-specific config");
}
#[test]
fn plugin_gc_can_be_disabled_by_plugin() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_example"),
r#"
example disable-gc
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
$env.config.plugin_gc = { default: { stop_after: 0sec } }
example one 1 foo | ignore # ensure we've run the plugin with the new config
sleep 100ms
Make `plugin list` read state from plugin registry file as well (#14085) # Description [Context on Discord](https://discord.com/channels/601130461678272522/855947301380947968/1292279795035668583) **This is a breaking change, due to the removal of `is_running`.** Some users find the `plugin list` command confusing, because it doesn't show anything different after running `plugin add` or `plugin rm`. This modifies the `plugin list` command to also look at the plugin registry file to give some idea of how the plugins in engine state differ from those in the plugin registry file. The following values of `status` are now produced instead of `is_running`: - `added`: The plugin is present in the plugin registry file, but not in the engine. - `loaded`: The plugin is present both in the plugin registry file and in the engine, but is not running. - `running`: The plugin is currently running, and the `pid` column should contain its process ID. - `modified`: The plugin state present in the plugin registry file is different from the state in the engine. - `removed`: The plugin is still loaded in the engine, but is not present in the plugin registry file. - `invalid`: The data in the plugin registry file couldn't be deserialized, and the plugin most likely needs to be added again. Example (`commands` omitted): ``` ╭──────┬─────────────────────┬────────────┬───────────┬──────────┬─────────────────────────────────────────────────────┬─────────╮ │ # │ name │ version │ status │ pid │ filename │ shell │ ├──────┼─────────────────────┼────────────┼───────────┼──────────┼─────────────────────────────────────────────────────┼─────────┤ │ 0 │ custom_values │ 0.1.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_custom_values │ │ │ 1 │ dbus │ 0.11.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_dbus │ │ │ 2 │ example │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_example │ │ │ 3 │ explore_ir │ 0.3.0 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_explore_ir │ │ │ 4 │ formats │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_formats │ │ │ 5 │ gstat │ 0.98.1 │ running │ 236662 │ /home/devyn/.cargo/bin/nu_plugin_gstat │ │ │ 6 │ inc │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_inc │ │ │ 7 │ polars │ 0.98.1 │ added │ │ /home/devyn/.cargo/bin/nu_plugin_polars │ │ │ 8 │ query │ 0.98.1 │ removed │ │ /home/devyn/.cargo/bin/nu_plugin_query │ │ │ 9 │ stress_internals │ 0.98.1 │ loaded │ │ /home/devyn/.cargo/bin/nu_plugin_stress_internals │ │ ╰──────┴─────────────────────┴────────────┴───────────┴──────────┴─────────────────────────────────────────────────────┴─────────╯ ``` # User-Facing Changes To `plugin list`: * **Breaking:** The `is_running` column is removed and replaced with `status`. Use `status == running` to filter equivalently. * The `--plugin-config` from other plugin management commands is now supported. * Added an `--engine` flag which behaves more or less like before, and doesn't load the plugin registry file at all. * Added a `--registry` flag which only checks the plugin registry file. All plugins appear as `added` since there is no state to compare with. Because the default is to check both, the `plugin list` command might be a little bit slower. If you don't need to check the plugin registry file, the `--engine` flag does not load the plugin registry file at all, so it should be just as fast as before. # Tests + Formatting Added tests for `added` and `removed` statuses. `modified` and `invalid` are a bit more tricky so I didn't try. # After Submitting - [ ] update documentation that references the `plugin list` command - [ ] release notes
2024-10-17 04:24:45 +02:00
(plugin list | where name == example).0.status == running
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
assert_eq!("true", out.out);
}
#[test]
fn plugin_gc_does_not_stop_plugin_while_stream_output_is_active() {
let out = nu_with_plugins!(
cwd: ".",
plugin: ("nu_plugin_example"),
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
r#"
$env.config.plugin_gc = { default: { stop_after: 10ms } }
# This would exceed the configured time
example seq 1 500 | each { |n| sleep 1ms; $n } | length | print
Keep plugins persistently running in the background (#12064) # Description This PR uses the new plugin protocol to intelligently keep plugin processes running in the background for further plugin calls. Running plugins can be seen by running the new `plugin list` command, and stopped by running the new `plugin stop` command. This is an enhancement for the performance of plugins, as starting new plugin processes has overhead, especially for plugins in languages that take a significant amount of time on startup. It also enables plugins that have persistent state between commands, making the migration of features like dataframes and `stor` to plugins possible. Plugins are automatically stopped by the new plugin garbage collector, configurable with `$env.config.plugin_gc`: ```nushell $env.config.plugin_gc = { # Configuration for plugin garbage collection default: { enabled: true # true to enable stopping of inactive plugins stop_after: 10sec # how long to wait after a plugin is inactive to stop it } plugins: { # alternate configuration for specific plugins, by name, for example: # # gstat: { # enabled: false # } } } ``` If garbage collection is enabled, plugins will be stopped after `stop_after` passes after they were last active. Plugins are counted as inactive if they have no running plugin calls. Reading the stream from the response of a plugin call is still considered to be activity, but if a plugin holds on to a stream but the call ends without an active streaming response, it is not counted as active even if it is reading it. Plugins can explicitly disable the GC as appropriate with `engine.set_gc_disabled(true)`. The `version` command now lists plugin names rather than plugin commands. The list of plugin commands is accessible via `plugin list`. Recommend doing this together with #12029, because it will likely force plugin developers to do the right thing with mutability and lead to less unexpected behavior when running plugins nested / in parallel. # User-Facing Changes - new command: `plugin list` - new command: `plugin stop` - changed command: `version` (now lists plugin names, rather than commands) - new config: `$env.config.plugin_gc` - Plugins will keep running and be reused, at least for the configured GC period - Plugins that used mutable state in weird ways like `inc` did might misbehave until fixed - Plugins can disable GC if they need to - Had to change plugin signature to accept `&EngineInterface` so that the GC disable feature works. #12029 does this anyway, and I'm expecting (resolvable) conflicts with that # Tests + Formatting - :green_circle: `toolkit fmt` - :green_circle: `toolkit clippy` - :green_circle: `toolkit test` - :green_circle: `toolkit test stdlib` Because there is some specific OS behavior required for plugins to not respond to Ctrl-C directly, I've developed against and tested on both Linux and Windows to ensure that works properly. # After Submitting I think this probably needs to be in the book somewhere
2024-03-10 00:10:22 +01:00
"#
);
assert!(out.status.success());
assert_eq!("500", out.out);
}