extern/nushell
1
0
Fork 1
mirror of https://github.com/nushell/nushell.git synced 2025-08-09 19:37:45 +02:00
Code Issues Packages Projects Releases Wiki Activity

Split the plugin crate (#12563)

Browse Source 
# Description

This breaks `nu-plugin` up into four crates:

- `nu-plugin-protocol`: just the type definitions for the protocol, no
I/O. If someone wanted to wire up something more bare metal, maybe for
async I/O, they could use this.
- `nu-plugin-core`: the shared stuff between engine/plugin. Less stable
interface.
- `nu-plugin-engine`: everything required for the engine to talk to
plugins. Less stable interface.
- `nu-plugin`: everything required for the plugin to talk to the engine,
what plugin developers use. Should be the most stable interface.

No changes are made to the interface exposed by `nu-plugin` - it should
all still be there. Re-exports from `nu-plugin-protocol` or
`nu-plugin-core` are used as required. Plugins shouldn't ever have to
use those crates directly.

This should be somewhat faster to compile as `nu-plugin-engine` and
`nu-plugin` can compile in parallel, and the engine doesn't need
`nu-plugin` and plugins don't need `nu-plugin-engine` (except for test
support), so that should reduce what needs to be compiled too.

The only significant change here other than splitting stuff up was to
break the `source` out of `PluginCustomValue` and create a new
`PluginCustomValueWithSource` type that contains that instead. One bonus
of that is we get rid of the option and it's now more type-safe, but it
also means that the logic for that stuff (actually running the plugin
for custom value ops) can live entirely within the `nu-plugin-engine`
crate.

# User-Facing Changes
- New crates.
- Added `local-socket` feature for `nu` to try to make it possible to
compile without that support if needed.

# Tests + Formatting
- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`
This commit is contained in:
Devyn Cairns
2024-04-27 10:08:12 -07:00
committed by GitHub
parent 884d5312bb
commit 0c4d5330ee
74 changed files with 3514 additions and 3110 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
crates/nu-plugin-engine/Cargo.toml Normal file
View File

@ -0,0 +1,34 @@
[package]
authors = ["The Nushell Project Developers"]
description = "Functionality for running Nushell plugins from a Nushell engine"
repository = "https://github.com/nushell/nushell/tree/main/crates/nu-plugin-engine"
edition = "2021"
license = "MIT"
name = "nu-plugin-engine"
version = "0.92.3"
[lib]
bench = false
[dependencies]
nu-engine = { path = "../nu-engine", version = "0.92.3" }
nu-protocol = { path = "../nu-protocol", version = "0.92.3" }
nu-system = { path = "../nu-system", version = "0.92.3" }
nu-plugin-protocol = { path = "../nu-plugin-protocol", version = "0.92.3" }
nu-plugin-core = { path = "../nu-plugin-core", version = "0.92.3", default-features = false }
serde = { workspace = true }
log = { workspace = true }
[dev-dependencies]
typetag = "0.2"
[features]
default = ["local-socket"]
local-socket = ["nu-plugin-core/local-socket"]
[target.'cfg(target_os = "windows")'.dependencies]
windows = { workspace = true, features = [
# For setting process creation flags
"Win32_System_Threading",
] }

21
crates/nu-plugin-engine/LICENSE Normal file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 - 2023 The Nushell Project Developers
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

3
crates/nu-plugin-engine/README.md Normal file
View File

@ -0,0 +1,3 @@
# nu-plugin-engine
This crate provides functionality for the [Nushell](https://nushell.sh/) engine to spawn and interact with plugins.

310
crates/nu-plugin-engine/src/context.rs Normal file
View File

@ -0,0 +1,310 @@
use crate::util::MutableCow;
use nu_engine::{get_eval_block_with_early_return, get_full_help, ClosureEvalOnce};
use nu_protocol::{
ast::Call,
engine::{Closure, EngineState, Redirection, Stack},
Config, IntoSpanned, OutDest, PipelineData, PluginIdentity, ShellError, Span, Spanned, Value,
};
use std::{
borrow::Cow,
collections::HashMap,
sync::{
atomic::{AtomicBool, AtomicU32},
Arc,
},
};
/// Object safe trait for abstracting operations required of the plugin context.
pub trait PluginExecutionContext: Send + Sync {
/// A span pointing to the command being executed
fn span(&self) -> Span;
/// The interrupt signal, if present
fn ctrlc(&self) -> Option<&Arc<AtomicBool>>;
/// The pipeline externals state, for tracking the foreground process group, if present
fn pipeline_externals_state(&self) -> Option<&Arc<(AtomicU32, AtomicU32)>>;
/// Get engine configuration
fn get_config(&self) -> Result<Config, ShellError>;
/// Get plugin configuration
fn get_plugin_config(&self) -> Result<Option<Value>, ShellError>;
/// Get an environment variable from `$env`
fn get_env_var(&self, name: &str) -> Result<Option<Value>, ShellError>;
/// Get all environment variables
fn get_env_vars(&self) -> Result<HashMap<String, Value>, ShellError>;
/// Get current working directory
fn get_current_dir(&self) -> Result<Spanned<String>, ShellError>;
/// Set an environment variable
fn add_env_var(&mut self, name: String, value: Value) -> Result<(), ShellError>;
/// Get help for the current command
fn get_help(&self) -> Result<Spanned<String>, ShellError>;
/// Get the contents of a [`Span`]
fn get_span_contents(&self, span: Span) -> Result<Spanned<Vec<u8>>, ShellError>;
/// Evaluate a closure passed to the plugin
fn eval_closure(
&self,
closure: Spanned<Closure>,
positional: Vec<Value>,
input: PipelineData,
redirect_stdout: bool,
redirect_stderr: bool,
) -> Result<PipelineData, ShellError>;
/// Create an owned version of the context with `'static` lifetime
fn boxed(&self) -> Box<dyn PluginExecutionContext>;
}
/// The execution context of a plugin command. Can be borrowed.
pub struct PluginExecutionCommandContext<'a> {
identity: Arc<PluginIdentity>,
engine_state: Cow<'a, EngineState>,
stack: MutableCow<'a, Stack>,
call: Cow<'a, Call>,
}
impl<'a> PluginExecutionCommandContext<'a> {
pub fn new(
identity: Arc<PluginIdentity>,
engine_state: &'a EngineState,
stack: &'a mut Stack,
call: &'a Call,
) -> PluginExecutionCommandContext<'a> {
PluginExecutionCommandContext {
identity,
engine_state: Cow::Borrowed(engine_state),
stack: MutableCow::Borrowed(stack),
call: Cow::Borrowed(call),
}
}
}
impl<'a> PluginExecutionContext for PluginExecutionCommandContext<'a> {
fn span(&self) -> Span {
self.call.head
}
fn ctrlc(&self) -> Option<&Arc<AtomicBool>> {
self.engine_state.ctrlc.as_ref()
}
fn pipeline_externals_state(&self) -> Option<&Arc<(AtomicU32, AtomicU32)>> {
Some(&self.engine_state.pipeline_externals_state)
}
fn get_config(&self) -> Result<Config, ShellError> {
Ok(nu_engine::get_config(&self.engine_state, &self.stack))
}
fn get_plugin_config(&self) -> Result<Option<Value>, ShellError> {
// Fetch the configuration for a plugin
//
// The `plugin` must match the registered name of a plugin. For
// `register nu_plugin_example` the plugin config lookup uses `"example"`
Ok(self
.get_config()?
.plugins
.get(self.identity.name())
.cloned()
.map(|value| {
let span = value.span();
match value {
Value::Closure { val, .. } => {
ClosureEvalOnce::new(&self.engine_state, &self.stack, val)
.run_with_input(PipelineData::Empty)
.map(|data| data.into_value(span))
.unwrap_or_else(|err| Value::error(err, self.call.head))
}
_ => value.clone(),
}
}))
}
fn get_env_var(&self, name: &str) -> Result<Option<Value>, ShellError> {
Ok(self.stack.get_env_var(&self.engine_state, name))
}
fn get_env_vars(&self) -> Result<HashMap<String, Value>, ShellError> {
Ok(self.stack.get_env_vars(&self.engine_state))
}
fn get_current_dir(&self) -> Result<Spanned<String>, ShellError> {
let cwd = nu_engine::env::current_dir_str(&self.engine_state, &self.stack)?;
// The span is not really used, so just give it call.head
Ok(cwd.into_spanned(self.call.head))
}
fn add_env_var(&mut self, name: String, value: Value) -> Result<(), ShellError> {
self.stack.add_env_var(name, value);
Ok(())
}
fn get_help(&self) -> Result<Spanned<String>, ShellError> {
let decl = self.engine_state.get_decl(self.call.decl_id);
Ok(get_full_help(
&decl.signature(),
&decl.examples(),
&self.engine_state,
&mut self.stack.clone(),
false,
)
.into_spanned(self.call.head))
}
fn get_span_contents(&self, span: Span) -> Result<Spanned<Vec<u8>>, ShellError> {
Ok(self
.engine_state
.get_span_contents(span)
.to_vec()
.into_spanned(self.call.head))
}
fn eval_closure(
&self,
closure: Spanned<Closure>,
positional: Vec<Value>,
input: PipelineData,
redirect_stdout: bool,
redirect_stderr: bool,
) -> Result<PipelineData, ShellError> {
let block = self
.engine_state
.try_get_block(closure.item.block_id)
.ok_or_else(|| ShellError::GenericError {
error: "Plugin misbehaving".into(),
msg: format!(
"Tried to evaluate unknown block id: {}",
closure.item.block_id
),
span: Some(closure.span),
help: None,
inner: vec![],
})?;
let mut stack = self
.stack
.captures_to_stack(closure.item.captures)
.reset_pipes();
let stdout = if redirect_stdout {
Some(Redirection::Pipe(OutDest::Capture))
} else {
None
};
let stderr = if redirect_stderr {
Some(Redirection::Pipe(OutDest::Capture))
} else {
None
};
let stack = &mut stack.push_redirection(stdout, stderr);
// Set up the positional arguments
for (idx, value) in positional.into_iter().enumerate() {
if let Some(arg) = block.signature.get_positional(idx) {
if let Some(var_id) = arg.var_id {
stack.add_var(var_id, value);
} else {
return Err(ShellError::NushellFailedSpanned {
msg: "Error while evaluating closure from plugin".into(),
label: "closure argument missing var_id".into(),
span: closure.span,
});
}
}
}
let eval_block_with_early_return = get_eval_block_with_early_return(&self.engine_state);
eval_block_with_early_return(&self.engine_state, stack, block, input)
}
fn boxed(&self) -> Box<dyn PluginExecutionContext + 'static> {
Box::new(PluginExecutionCommandContext {
identity: self.identity.clone(),
engine_state: Cow::Owned(self.engine_state.clone().into_owned()),
stack: self.stack.owned(),
call: Cow::Owned(self.call.clone().into_owned()),
})
}
}
/// A bogus execution context for testing that doesn't really implement anything properly
#[cfg(test)]
pub(crate) struct PluginExecutionBogusContext;
#[cfg(test)]
impl PluginExecutionContext for PluginExecutionBogusContext {
fn span(&self) -> Span {
Span::test_data()
}
fn ctrlc(&self) -> Option<&Arc<AtomicBool>> {
None
}
fn pipeline_externals_state(&self) -> Option<&Arc<(AtomicU32, AtomicU32)>> {
None
}
fn get_config(&self) -> Result<Config, ShellError> {
Err(ShellError::NushellFailed {
msg: "get_config not implemented on bogus".into(),
})
}
fn get_plugin_config(&self) -> Result<Option<Value>, ShellError> {
Ok(None)
}
fn get_env_var(&self, _name: &str) -> Result<Option<Value>, ShellError> {
Err(ShellError::NushellFailed {
msg: "get_env_var not implemented on bogus".into(),
})
}
fn get_env_vars(&self) -> Result<HashMap<String, Value>, ShellError> {
Err(ShellError::NushellFailed {
msg: "get_env_vars not implemented on bogus".into(),
})
}
fn get_current_dir(&self) -> Result<Spanned<String>, ShellError> {
Err(ShellError::NushellFailed {
msg: "get_current_dir not implemented on bogus".into(),
})
}
fn add_env_var(&mut self, _name: String, _value: Value) -> Result<(), ShellError> {
Err(ShellError::NushellFailed {
msg: "add_env_var not implemented on bogus".into(),
})
}
fn get_help(&self) -> Result<Spanned<String>, ShellError> {
Err(ShellError::NushellFailed {
msg: "get_help not implemented on bogus".into(),
})
}
fn get_span_contents(&self, _span: Span) -> Result<Spanned<Vec<u8>>, ShellError> {
Err(ShellError::NushellFailed {
msg: "get_span_contents not implemented on bogus".into(),
})
}
fn eval_closure(
&self,
_closure: Spanned<Closure>,
_positional: Vec<Value>,
_input: PipelineData,
_redirect_stdout: bool,
_redirect_stderr: bool,
) -> Result<PipelineData, ShellError> {
Err(ShellError::NushellFailed {
msg: "eval_closure not implemented on bogus".into(),
})
}
fn boxed(&self) -> Box<dyn PluginExecutionContext + 'static> {
Box::new(PluginExecutionBogusContext)
}
}

126
crates/nu-plugin-engine/src/declaration.rs Normal file
View File

@ -0,0 +1,126 @@
use nu_engine::{command_prelude::*, get_eval_expression};
use nu_plugin_protocol::{CallInfo, EvaluatedCall};
use nu_protocol::{PluginIdentity, PluginSignature};
use std::sync::Arc;
use crate::{GetPlugin, PluginExecutionCommandContext, PluginSource};
/// The command declaration proxy used within the engine for all plugin commands.
#[derive(Clone)]
pub struct PluginDeclaration {
name: String,
signature: PluginSignature,
source: PluginSource,
}
impl PluginDeclaration {
pub fn new(plugin: Arc<dyn GetPlugin>, signature: PluginSignature) -> Self {
Self {
name: signature.sig.name.clone(),
signature,
source: PluginSource::new(plugin),
}
}
}
impl Command for PluginDeclaration {
fn name(&self) -> &str {
&self.name
}
fn signature(&self) -> Signature {
self.signature.sig.clone()
}
fn usage(&self) -> &str {
self.signature.sig.usage.as_str()
}
fn extra_usage(&self) -> &str {
self.signature.sig.extra_usage.as_str()
}
fn search_terms(&self) -> Vec<&str> {
self.signature
.sig
.search_terms
.iter()
.map(|term| term.as_str())
.collect()
}
fn examples(&self) -> Vec<Example> {
let mut res = vec![];
for e in self.signature.examples.iter() {
res.push(Example {
example: &e.example,
description: &e.description,
result: e.result.clone(),
})
}
res
}
fn run(
&self,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
input: PipelineData,
) -> Result<PipelineData, ShellError> {
let eval_expression = get_eval_expression(engine_state);
// Create the EvaluatedCall to send to the plugin first - it's best for this to fail early,
// before we actually try to run the plugin command
let evaluated_call =
EvaluatedCall::try_from_call(call, engine_state, stack, eval_expression)?;
// Get the engine config
let engine_config = nu_engine::get_config(engine_state, stack);
// Get, or start, the plugin.
let plugin = self
.source
.persistent(None)
.and_then(|p| {
// Set the garbage collector config from the local config before running
p.set_gc_config(engine_config.plugin_gc.get(p.identity().name()));
p.get_plugin(Some((engine_state, stack)))
})
.map_err(|err| {
let decl = engine_state.get_decl(call.decl_id);
ShellError::GenericError {
error: format!("Unable to spawn plugin for `{}`", decl.name()),
msg: err.to_string(),
span: Some(call.head),
help: None,
inner: vec![],
}
})?;
// Create the context to execute in - this supports engine calls and custom values
let mut context = PluginExecutionCommandContext::new(
self.source.identity.clone(),
engine_state,
stack,
call,
);
plugin.run(
CallInfo {
name: self.name.clone(),
call: evaluated_call,
input,
},
&mut context,
)
}
fn is_plugin(&self) -> bool {
true
}
fn plugin_identity(&self) -> Option<&PluginIdentity> {
Some(&self.source.identity)
}
}

303
crates/nu-plugin-engine/src/gc.rs Normal file
View File

@ -0,0 +1,303 @@
use crate::PersistentPlugin;
use nu_protocol::{PluginGcConfig, RegisteredPlugin};
use std::{
sync::{mpsc, Arc, Weak},
thread,
time::{Duration, Instant},
};
/// Plugin garbage collector
///
/// Many users don't want all of their plugins to stay running indefinitely after using them, so
/// this runs a thread that monitors the plugin's usage and stops it automatically if it meets
/// certain conditions of inactivity.
#[derive(Debug, Clone)]
pub struct PluginGc {
sender: mpsc::Sender<PluginGcMsg>,
}
impl PluginGc {
/// Start a new plugin garbage collector. Returns an error if the thread failed to spawn.
pub fn new(
config: PluginGcConfig,
plugin: &Arc<PersistentPlugin>,
) -> std::io::Result<PluginGc> {
let (sender, receiver) = mpsc::channel();
let mut state = PluginGcState {
config,
last_update: None,
locks: 0,
disabled: false,
plugin: Arc::downgrade(plugin),
name: plugin.identity().name().to_owned(),
};
thread::Builder::new()
.name(format!("plugin gc ({})", plugin.identity().name()))
.spawn(move || state.run(receiver))?;
Ok(PluginGc { sender })
}
/// Update the garbage collector config
pub fn set_config(&self, config: PluginGcConfig) {
let _ = self.sender.send(PluginGcMsg::SetConfig(config));
}
/// Ensure all GC messages have been processed
pub fn flush(&self) {
let (tx, rx) = mpsc::channel();
let _ = self.sender.send(PluginGcMsg::Flush(tx));
// This will block until the channel is dropped, which could be because the send failed, or
// because the GC got the message
let _ = rx.recv();
}
/// Increment the number of locks held by the plugin
pub fn increment_locks(&self, amount: i64) {
let _ = self.sender.send(PluginGcMsg::AddLocks(amount));
}
/// Decrement the number of locks held by the plugin
pub fn decrement_locks(&self, amount: i64) {
let _ = self.sender.send(PluginGcMsg::AddLocks(-amount));
}
/// Set whether the GC is disabled by explicit request from the plugin. This is separate from
/// the `enabled` option in the config, and overrides that option.
pub fn set_disabled(&self, disabled: bool) {
let _ = self.sender.send(PluginGcMsg::SetDisabled(disabled));
}
/// Tell the GC to stop tracking the plugin. The plugin will not be stopped. The GC cannot be
/// reactivated after this request - a new one must be created instead.
pub fn stop_tracking(&self) {
let _ = self.sender.send(PluginGcMsg::StopTracking);
}
/// Tell the GC that the plugin exited so that it can remove it from the persistent plugin.
///
/// The reason the plugin tells the GC rather than just stopping itself via `source` is that
/// it can't guarantee that the plugin currently pointed to by `source` is itself, but if the
/// GC is still running, it hasn't received [`.stop_tracking()`] yet, which means it should be
/// the right plugin.
pub fn exited(&self) {
let _ = self.sender.send(PluginGcMsg::Exited);
}
}
#[derive(Debug)]
enum PluginGcMsg {
SetConfig(PluginGcConfig),
Flush(mpsc::Sender<()>),
AddLocks(i64),
SetDisabled(bool),
StopTracking,
Exited,
}
#[derive(Debug)]
struct PluginGcState {
config: PluginGcConfig,
last_update: Option<Instant>,
locks: i64,
disabled: bool,
plugin: Weak<PersistentPlugin>,
name: String,
}
impl PluginGcState {
fn next_timeout(&self, now: Instant) -> Option<Duration> {
if self.locks <= 0 && !self.disabled {
self.last_update
.zip(self.config.enabled.then_some(self.config.stop_after))
.map(|(last_update, stop_after)| {
// If configured to stop, and used at some point, calculate the difference
let stop_after_duration = Duration::from_nanos(stop_after.max(0) as u64);
let duration_since_last_update = now.duration_since(last_update);
stop_after_duration.saturating_sub(duration_since_last_update)
})
} else {
// Don't timeout if there are locks set, or disabled
None
}
}
// returns `Some()` if the GC should not continue to operate, with `true` if it should stop the
// plugin, or `false` if it should not
fn handle_message(&mut self, msg: PluginGcMsg) -> Option<bool> {
match msg {
PluginGcMsg::SetConfig(config) => {
self.config = config;
}
PluginGcMsg::Flush(sender) => {
// Rather than sending a message, we just drop the channel, which causes the other
// side to disconnect equally well
drop(sender);
}
PluginGcMsg::AddLocks(amount) => {
self.locks += amount;
if self.locks < 0 {
log::warn!(
"Plugin GC ({name}) problem: locks count below zero after adding \
{amount}: locks={locks}",
name = self.name,
locks = self.locks,
);
}
// Any time locks are modified, that counts as activity
self.last_update = Some(Instant::now());
}
PluginGcMsg::SetDisabled(disabled) => {
self.disabled = disabled;
}
PluginGcMsg::StopTracking => {
// Immediately exit without stopping the plugin
return Some(false);
}
PluginGcMsg::Exited => {
// Exit and stop the plugin
return Some(true);
}
}
None
}
fn run(&mut self, receiver: mpsc::Receiver<PluginGcMsg>) {
let mut always_stop = false;
loop {
let Some(msg) = (match self.next_timeout(Instant::now()) {
Some(duration) => receiver.recv_timeout(duration).ok(),
None => receiver.recv().ok(),
}) else {
// If the timeout was reached, or the channel is disconnected, break the loop
break;
};
log::trace!("Plugin GC ({name}) message: {msg:?}", name = self.name);
if let Some(should_stop) = self.handle_message(msg) {
// Exit the GC
if should_stop {
// If should_stop = true, attempt to stop the plugin
always_stop = true;
break;
} else {
// Don't stop the plugin
return;
}
}
}
// Upon exiting the loop, if the timeout reached zero, or we are exiting due to an Exited
// message, stop the plugin
if always_stop
|| self
.next_timeout(Instant::now())
.is_some_and(|t| t.is_zero())
{
// We only hold a weak reference, and it's not an error if we fail to upgrade it -
// that just means the plugin is definitely stopped anyway.
if let Some(plugin) = self.plugin.upgrade() {
let name = &self.name;
if let Err(err) = plugin.stop() {
log::warn!("Plugin `{name}` failed to be stopped by GC: {err}");
} else {
log::debug!("Plugin `{name}` successfully stopped by GC");
}
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
fn test_state() -> PluginGcState {
PluginGcState {
config: PluginGcConfig::default(),
last_update: None,
locks: 0,
disabled: false,
plugin: Weak::new(),
name: "test".into(),
}
}
#[test]
fn timeout_configured_as_zero() {
let now = Instant::now();
let mut state = test_state();
state.config.enabled = true;
state.config.stop_after = 0;
state.last_update = Some(now);
assert_eq!(Some(Duration::ZERO), state.next_timeout(now));
}
#[test]
fn timeout_past_deadline() {
let now = Instant::now();
let mut state = test_state();
state.config.enabled = true;
state.config.stop_after = Duration::from_secs(1).as_nanos() as i64;
state.last_update = Some(now - Duration::from_secs(2));
assert_eq!(Some(Duration::ZERO), state.next_timeout(now));
}
#[test]
fn timeout_with_deadline_in_future() {
let now = Instant::now();
let mut state = test_state();
state.config.enabled = true;
state.config.stop_after = Duration::from_secs(1).as_nanos() as i64;
state.last_update = Some(now);
assert_eq!(Some(Duration::from_secs(1)), state.next_timeout(now));
}
#[test]
fn no_timeout_if_disabled_by_config() {
let now = Instant::now();
let mut state = test_state();
state.config.enabled = false;
state.last_update = Some(now);
assert_eq!(None, state.next_timeout(now));
}
#[test]
fn no_timeout_if_disabled_by_plugin() {
let now = Instant::now();
let mut state = test_state();
state.config.enabled = true;
state.disabled = true;
state.last_update = Some(now);
assert_eq!(None, state.next_timeout(now));
}
#[test]
fn no_timeout_if_locks_count_over_zero() {
let now = Instant::now();
let mut state = test_state();
state.config.enabled = true;
state.locks = 1;
state.last_update = Some(now);
assert_eq!(None, state.next_timeout(now));
}
#[test]
fn adding_locks_changes_last_update() {
let mut state = test_state();
let original_last_update = Some(Instant::now() - Duration::from_secs(1));
state.last_update = original_last_update;
state.handle_message(PluginGcMsg::AddLocks(1));
assert_ne!(original_last_update, state.last_update, "not updated");
}
}

306
crates/nu-plugin-engine/src/init.rs Normal file
View File

@ -0,0 +1,306 @@
use std::{
io::{BufReader, BufWriter},
path::Path,
process::Child,
sync::{Arc, Mutex},
};
#[cfg(unix)]
use std::os::unix::process::CommandExt;
#[cfg(windows)]
use std::os::windows::process::CommandExt;
use nu_plugin_core::{
CommunicationMode, EncodingType, InterfaceManager, PreparedServerCommunication,
ServerCommunicationIo,
};
use nu_protocol::{
engine::StateWorkingSet, report_error_new, PluginIdentity, PluginRegistryFile,
PluginRegistryItem, PluginRegistryItemData, RegisteredPlugin, ShellError, Span,
};
use crate::{
PersistentPlugin, PluginDeclaration, PluginGc, PluginInterface, PluginInterfaceManager,
PluginSource,
};
pub(crate) const OUTPUT_BUFFER_SIZE: usize = 8192;
/// Spawn the command for a plugin, in the given `mode`. After spawning, it can be passed to
/// [`make_plugin_interface()`] to get a [`PluginInterface`].
pub fn create_command(
path: &Path,
mut shell: Option<&Path>,
mode: &CommunicationMode,
) -> std::process::Command {
log::trace!("Starting plugin: {path:?}, shell = {shell:?}, mode = {mode:?}");
let mut shell_args = vec![];
if shell.is_none() {
// We only have to do this for things that are not executable by Rust's Command API on
// Windows. They do handle bat/cmd files for us, helpfully.
//
// Also include anything that wouldn't be executable with a shebang, like JAR files.
shell = match path.extension().and_then(|e| e.to_str()) {
Some("sh") => {
if cfg!(unix) {
// We don't want to override what might be in the shebang if this is Unix, since
// some scripts will have a shebang specifying bash even if they're .sh
None
} else {
Some(Path::new("sh"))
}
}
Some("nu") => {
shell_args.push("--stdin");
Some(Path::new("nu"))
}
Some("py") => Some(Path::new("python")),
Some("rb") => Some(Path::new("ruby")),
Some("jar") => {
shell_args.push("-jar");
Some(Path::new("java"))
}
_ => None,
};
}
let mut process = if let Some(shell) = shell {
let mut process = std::process::Command::new(shell);
process.args(shell_args);
process.arg(path);
process
} else {
std::process::Command::new(path)
};
process.args(mode.args());
// Setup I/O according to the communication mode
mode.setup_command_io(&mut process);
// The plugin should be run in a new process group to prevent Ctrl-C from stopping it
#[cfg(unix)]
process.process_group(0);
#[cfg(windows)]
process.creation_flags(windows::Win32::System::Threading::CREATE_NEW_PROCESS_GROUP.0);
// In order to make bugs with improper use of filesystem without getting the engine current
// directory more obvious, the plugin always starts in the directory of its executable
if let Some(dirname) = path.parent() {
process.current_dir(dirname);
}
process
}
/// Create a plugin interface from a spawned child process.
///
/// `comm` determines the communication type the process was spawned with, and whether stdio will
/// be taken from the child.
pub fn make_plugin_interface(
mut child: Child,
comm: PreparedServerCommunication,
source: Arc<PluginSource>,
pid: Option<u32>,
gc: Option<PluginGc>,
) -> Result<PluginInterface, ShellError> {
match comm.connect(&mut child)? {
ServerCommunicationIo::Stdio(stdin, stdout) => make_plugin_interface_with_streams(
stdout,
stdin,
move || {
let _ = child.wait();
},
source,
pid,
gc,
),
#[cfg(feature = "local-socket")]
ServerCommunicationIo::LocalSocket { read_out, write_in } => {
make_plugin_interface_with_streams(
read_out,
write_in,
move || {
let _ = child.wait();
},
source,
pid,
gc,
)
}
}
}
/// Create a plugin interface from low-level components.
///
/// - `after_close` is called to clean up after the `reader` ends.
/// - `source` is required so that custom values produced by the plugin can spawn it.
/// - `pid` may be provided for process management (e.g. `EnterForeground`).
/// - `gc` may be provided for communication with the plugin's GC (e.g. `SetGcDisabled`).
pub fn make_plugin_interface_with_streams(
mut reader: impl std::io::Read + Send + 'static,
writer: impl std::io::Write + Send + 'static,
after_close: impl FnOnce() + Send + 'static,
source: Arc<PluginSource>,
pid: Option<u32>,
gc: Option<PluginGc>,
) -> Result<PluginInterface, ShellError> {
let encoder = get_plugin_encoding(&mut reader)?;
let reader = BufReader::with_capacity(OUTPUT_BUFFER_SIZE, reader);
let writer = BufWriter::with_capacity(OUTPUT_BUFFER_SIZE, writer);
let mut manager =
PluginInterfaceManager::new(source.clone(), pid, (Mutex::new(writer), encoder));
manager.set_garbage_collector(gc);
let interface = manager.get_interface();
interface.hello()?;
// Spawn the reader on a new thread. We need to be able to read messages at the same time that
// we write, because we are expected to be able to handle multiple messages coming in from the
// plugin at any time, including stream messages like `Drop`.
std::thread::Builder::new()
.name(format!(
"plugin interface reader ({})",
source.identity.name()
))
.spawn(move || {
if let Err(err) = manager.consume_all((reader, encoder)) {
log::warn!("Error in PluginInterfaceManager: {err}");
}
// If the loop has ended, drop the manager so everyone disconnects and then run
// after_close
drop(manager);
after_close();
})
.map_err(|err| ShellError::PluginFailedToLoad {
msg: format!("Failed to spawn thread for plugin: {err}"),
})?;
Ok(interface)
}
/// Determine the plugin's encoding from a freshly opened stream.
///
/// The plugin is expected to send a 1-byte length and either `json` or `msgpack`, so this reads
/// that and determines the right length.
pub fn get_plugin_encoding(
child_stdout: &mut impl std::io::Read,
) -> Result<EncodingType, ShellError> {
let mut length_buf = [0u8; 1];
child_stdout
.read_exact(&mut length_buf)
.map_err(|e| ShellError::PluginFailedToLoad {
msg: format!("unable to get encoding from plugin: {e}"),
})?;
let mut buf = vec![0u8; length_buf[0] as usize];
child_stdout
.read_exact(&mut buf)
.map_err(|e| ShellError::PluginFailedToLoad {
msg: format!("unable to get encoding from plugin: {e}"),
})?;
EncodingType::try_from_bytes(&buf).ok_or_else(|| {
let encoding_for_debug = String::from_utf8_lossy(&buf);
ShellError::PluginFailedToLoad {
msg: format!("get unsupported plugin encoding: {encoding_for_debug}"),
}
})
}
/// Load the definitions from the plugin file into the engine state
pub fn load_plugin_file(
working_set: &mut StateWorkingSet,
plugin_registry_file: &PluginRegistryFile,
span: Option<Span>,
) {
for plugin in &plugin_registry_file.plugins {
// Any errors encountered should just be logged.
if let Err(err) = load_plugin_registry_item(working_set, plugin, span) {
report_error_new(working_set.permanent_state, &err)
}
}
}
/// Load a definition from the plugin file into the engine state
pub fn load_plugin_registry_item(
working_set: &mut StateWorkingSet,
plugin: &PluginRegistryItem,
span: Option<Span>,
) -> Result<Arc<PersistentPlugin>, ShellError> {
let identity =
PluginIdentity::new(plugin.filename.clone(), plugin.shell.clone()).map_err(|_| {
ShellError::GenericError {
error: "Invalid plugin filename in plugin registry file".into(),
msg: "loaded from here".into(),
span,
help: Some(format!(
"the filename for `{}` is not a valid nushell plugin: {}",
plugin.name,
plugin.filename.display()
)),
inner: vec![],
}
})?;
match &plugin.data {
PluginRegistryItemData::Valid { commands } => {
let plugin = add_plugin_to_working_set(working_set, &identity)?;
// Ensure that the plugin is reset. We're going to load new signatures, so we want to
// make sure the running plugin reflects those new signatures, and it's possible that it
// doesn't.
plugin.reset()?;
// Create the declarations from the commands
for signature in commands {
let decl = PluginDeclaration::new(plugin.clone(), signature.clone());
working_set.add_decl(Box::new(decl));
}
Ok(plugin)
}
PluginRegistryItemData::Invalid => Err(ShellError::PluginRegistryDataInvalid {
plugin_name: identity.name().to_owned(),
span,
add_command: identity.add_command(),
}),
}
}
/// Find [`PersistentPlugin`] with the given `identity` in the `working_set`, or construct it
/// if it doesn't exist.
///
/// The garbage collection config is always found and set in either case.
pub fn add_plugin_to_working_set(
working_set: &mut StateWorkingSet,
identity: &PluginIdentity,
) -> Result<Arc<PersistentPlugin>, ShellError> {
// Find garbage collection config for the plugin
let gc_config = working_set
.get_config()
.plugin_gc
.get(identity.name())
.clone();
// Add it to / get it from the working set
let plugin = working_set.find_or_create_plugin(identity, || {
Arc::new(PersistentPlugin::new(identity.clone(), gc_config.clone()))
});
plugin.set_gc_config(&gc_config);
// Downcast the plugin to `PersistentPlugin` - we generally expect this to succeed.
// The trait object only exists so that nu-protocol can contain plugins without knowing
// anything about their implementation, but we only use `PersistentPlugin` in practice.
plugin
.as_any()
.downcast()
.map_err(|_| ShellError::NushellFailed {
msg: "encountered unexpected RegisteredPlugin type".into(),
})
}

1338
crates/nu-plugin-engine/src/interface/mod.rs Normal file
View File

File diff suppressed because it is too large Load Diff

1495
crates/nu-plugin-engine/src/interface/tests.rs Normal file
View File

File diff suppressed because it is too large Load Diff

24
crates/nu-plugin-engine/src/lib.rs Normal file
View File

@ -0,0 +1,24 @@
//! Provides functionality for running Nushell plugins from a Nushell engine.
mod context;
mod declaration;
mod gc;
mod init;
mod interface;
mod persistent;
mod plugin_custom_value_with_source;
mod process;
mod source;
mod util;
#[cfg(test)]
mod test_util;
pub use context::{PluginExecutionCommandContext, PluginExecutionContext};
pub use declaration::PluginDeclaration;
pub use gc::PluginGc;
pub use init::*;
pub use interface::{PluginInterface, PluginInterfaceManager};
pub use persistent::{GetPlugin, PersistentPlugin};
pub use plugin_custom_value_with_source::{PluginCustomValueWithSource, WithSource};
pub use source::PluginSource;

322
crates/nu-plugin-engine/src/persistent.rs Normal file
View File

@ -0,0 +1,322 @@
use crate::{
init::{create_command, make_plugin_interface},
PluginGc,
};
use super::{PluginInterface, PluginSource};
use nu_plugin_core::CommunicationMode;
use nu_protocol::{
engine::{EngineState, Stack},
PluginGcConfig, PluginIdentity, RegisteredPlugin, ShellError,
};
use std::{
collections::HashMap,
sync::{Arc, Mutex},
};
/// A box that can keep a plugin that was spawned persistent for further uses. The plugin may or
/// may not be currently running. [`.get()`] gets the currently running plugin, or spawns it if it's
/// not running.
#[derive(Debug)]
pub struct PersistentPlugin {
/// Identity (filename, shell, name) of the plugin
identity: PluginIdentity,
/// Mutable state
mutable: Mutex<MutableState>,
}
/// The mutable state for the persistent plugin. This should all be behind one lock to prevent lock
/// order problems.
#[derive(Debug)]
struct MutableState {
/// Reference to the plugin if running
running: Option<RunningPlugin>,
/// Plugin's preferred communication mode (if known)
preferred_mode: Option<PreferredCommunicationMode>,
/// Garbage collector config
gc_config: PluginGcConfig,
}
#[derive(Debug, Clone, Copy)]
enum PreferredCommunicationMode {
Stdio,
#[cfg(feature = "local-socket")]
LocalSocket,
}
#[derive(Debug)]
struct RunningPlugin {
/// Interface (which can be cloned) to the running plugin
interface: PluginInterface,
/// Garbage collector for the plugin
gc: PluginGc,
}
impl PersistentPlugin {
/// Create a new persistent plugin. The plugin will not be spawned immediately.
pub fn new(identity: PluginIdentity, gc_config: PluginGcConfig) -> PersistentPlugin {
PersistentPlugin {
identity,
mutable: Mutex::new(MutableState {
running: None,
preferred_mode: None,
gc_config,
}),
}
}
/// Get the plugin interface of the running plugin, or spawn it if it's not currently running.
///
/// Will call `envs` to get environment variables to spawn the plugin if the plugin needs to be
/// spawned.
pub fn get(
self: Arc<Self>,
envs: impl FnOnce() -> Result<HashMap<String, String>, ShellError>,
) -> Result<PluginInterface, ShellError> {
let mut mutable = self.mutable.lock().map_err(|_| ShellError::NushellFailed {
msg: format!(
"plugin `{}` mutex poisoned, probably panic during spawn",
self.identity.name()
),
})?;
if let Some(ref running) = mutable.running {
// It exists, so just clone the interface
Ok(running.interface.clone())
} else {
// Try to spawn. On success, `mutable.running` should have been set to the new running
// plugin by `spawn()` so we just then need to clone the interface from there.
//
// We hold the lock the whole time to prevent others from trying to spawn and ending
// up with duplicate plugins
//
// TODO: We should probably store the envs somewhere, in case we have to launch without
// envs (e.g. from a custom value)
let envs = envs()?;
let result = self.clone().spawn(&envs, &mut mutable);
// Check if we were using an alternate communication mode and may need to fall back to
// stdio.
if result.is_err()
&& !matches!(
mutable.preferred_mode,
Some(PreferredCommunicationMode::Stdio)
)
{
log::warn!("{}: Trying again with stdio communication because mode {:?} failed with {result:?}",
self.identity.name(),
mutable.preferred_mode);
// Reset to stdio and try again, but this time don't catch any error
mutable.preferred_mode = Some(PreferredCommunicationMode::Stdio);
self.clone().spawn(&envs, &mut mutable)?;
}
Ok(mutable
.running
.as_ref()
.ok_or_else(|| ShellError::NushellFailed {
msg: "spawn() succeeded but didn't set interface".into(),
})?
.interface
.clone())
}
}
/// Run the plugin command, then set up and set `mutable.running` to the new running plugin.
fn spawn(
self: Arc<Self>,
envs: &HashMap<String, String>,
mutable: &mut MutableState,
) -> Result<(), ShellError> {
// Make sure `running` is set to None to begin
if let Some(running) = mutable.running.take() {
// Stop the GC if there was a running plugin
running.gc.stop_tracking();
}
let source_file = self.identity.filename();
// Determine the mode to use based on the preferred mode
let mode = match mutable.preferred_mode {
// If not set, we try stdio first and then might retry if another mode is supported
Some(PreferredCommunicationMode::Stdio) | None => CommunicationMode::Stdio,
// Local socket only if enabled
#[cfg(feature = "local-socket")]
Some(PreferredCommunicationMode::LocalSocket) => {
CommunicationMode::local_socket(source_file)
}
};
let mut plugin_cmd = create_command(source_file, self.identity.shell(), &mode);
// We need the current environment variables for `python` based plugins
// Or we'll likely have a problem when a plugin is implemented in a virtual Python environment.
plugin_cmd.envs(envs);
let program_name = plugin_cmd.get_program().to_os_string().into_string();
// Before running the command, prepare communication
let comm = mode.serve()?;
// Run the plugin command
let child = plugin_cmd.spawn().map_err(|err| {
let error_msg = match err.kind() {
std::io::ErrorKind::NotFound => match program_name {
Ok(prog_name) => {
format!("Can't find {prog_name}, please make sure that {prog_name} is in PATH.")
}
_ => {
format!("Error spawning child process: {err}")
}
},
_ => {
format!("Error spawning child process: {err}")
}
};
ShellError::PluginFailedToLoad { msg: error_msg }
})?;
// Start the plugin garbage collector
let gc = PluginGc::new(mutable.gc_config.clone(), &self)?;
let pid = child.id();
let interface = make_plugin_interface(
child,
comm,
Arc::new(PluginSource::new(self.clone())),
Some(pid),
Some(gc.clone()),
)?;
// If our current preferred mode is None, check to see if the plugin might support another
// mode. If so, retry spawn() with that mode
#[cfg(feature = "local-socket")]
if mutable.preferred_mode.is_none()
&& interface
.protocol_info()?
.supports_feature(&nu_plugin_protocol::Feature::LocalSocket)
{
log::trace!(
"{}: Attempting to upgrade to local socket mode",
self.identity.name()
);
// Stop the GC we just created from tracking so that we don't accidentally try to
// stop the new plugin
gc.stop_tracking();
// Set the mode and try again
mutable.preferred_mode = Some(PreferredCommunicationMode::LocalSocket);
return self.spawn(envs, mutable);
}
mutable.running = Some(RunningPlugin { interface, gc });
Ok(())
}
fn stop_internal(&self, reset: bool) -> Result<(), ShellError> {
let mut mutable = self.mutable.lock().map_err(|_| ShellError::NushellFailed {
msg: format!(
"plugin `{}` mutable mutex poisoned, probably panic during spawn",
self.identity.name()
),
})?;
// If the plugin is running, stop its GC, so that the GC doesn't accidentally try to stop
// a future plugin
if let Some(ref running) = mutable.running {
running.gc.stop_tracking();
}
// We don't try to kill the process or anything, we just drop the RunningPlugin. It should
// exit soon after
mutable.running = None;
// If this is a reset, we should also reset other learned attributes like preferred_mode
if reset {
mutable.preferred_mode = None;
}
Ok(())
}
}
impl RegisteredPlugin for PersistentPlugin {
fn identity(&self) -> &PluginIdentity {
&self.identity
}
fn is_running(&self) -> bool {
// If the lock is poisoned, we return false here. That may not be correct, but this is a
// failure state anyway that would be noticed at some point
self.mutable
.lock()
.map(|m| m.running.is_some())
.unwrap_or(false)
}
fn pid(&self) -> Option<u32> {
// Again, we return None for a poisoned lock.
self.mutable
.lock()
.ok()
.and_then(|r| r.running.as_ref().and_then(|r| r.interface.pid()))
}
fn stop(&self) -> Result<(), ShellError> {
self.stop_internal(false)
}
fn reset(&self) -> Result<(), ShellError> {
self.stop_internal(true)
}
fn set_gc_config(&self, gc_config: &PluginGcConfig) {
if let Ok(mut mutable) = self.mutable.lock() {
// Save the new config for future calls
mutable.gc_config = gc_config.clone();
// If the plugin is already running, propagate the config change to the running GC
if let Some(gc) = mutable.running.as_ref().map(|running| running.gc.clone()) {
// We don't want to get caught holding the lock
drop(mutable);
gc.set_config(gc_config.clone());
gc.flush();
}
}
}
fn as_any(self: Arc<Self>) -> Arc<dyn std::any::Any + Send + Sync> {
self
}
}
/// Anything that can produce a plugin interface.
pub trait GetPlugin: RegisteredPlugin {
/// Retrieve or spawn a [`PluginInterface`]. The `context` may be used for determining
/// environment variables to launch the plugin with.
fn get_plugin(
self: Arc<Self>,
context: Option<(&EngineState, &mut Stack)>,
) -> Result<PluginInterface, ShellError>;
}
impl GetPlugin for PersistentPlugin {
fn get_plugin(
self: Arc<Self>,
mut context: Option<(&EngineState, &mut Stack)>,
) -> Result<PluginInterface, ShellError> {
self.get(|| {
// Get envs from the context if provided.
let envs = context
.as_mut()
.map(|(engine_state, stack)| {
// We need the current environment variables for `python` based plugins. Or
// we'll likely have a problem when a plugin is implemented in a virtual Python
// environment.
let stack = &mut stack.start_capture();
nu_engine::env::env_to_strings(engine_state, stack)
})
.transpose()?;
Ok(envs.unwrap_or_default())
})
}
}

274
crates/nu-plugin-engine/src/plugin_custom_value_with_source/mod.rs Normal file
View File

@ -0,0 +1,274 @@
use std::{cmp::Ordering, sync::Arc};
use nu_plugin_core::util::with_custom_values_in;
use nu_plugin_protocol::PluginCustomValue;
use nu_protocol::{ast::Operator, CustomValue, IntoSpanned, ShellError, Span, Spanned, Value};
use serde::Serialize;
use crate::{PluginInterface, PluginSource};
#[cfg(test)]
mod tests;
/// Wraps a [`PluginCustomValue`] together with its [`PluginSource`], so that the [`CustomValue`]
/// methods can be implemented by calling the plugin, and to ensure that any custom values sent to a
/// plugin came from it originally.
#[derive(Debug, Clone)]
pub struct PluginCustomValueWithSource {
inner: PluginCustomValue,
/// Which plugin the custom value came from. This is not sent over the serialization boundary.
source: Arc<PluginSource>,
}
impl PluginCustomValueWithSource {
/// Wrap a [`PluginCustomValue`] together with its source.
pub fn new(inner: PluginCustomValue, source: Arc<PluginSource>) -> PluginCustomValueWithSource {
PluginCustomValueWithSource { inner, source }
}
/// Create a [`Value`] containing this custom value.
pub fn into_value(self, span: Span) -> Value {
Value::custom(Box::new(self), span)
}
/// Which plugin the custom value came from. This provides a direct reference to be able to get
/// a plugin interface in order to make a call, when needed.
pub fn source(&self) -> &Arc<PluginSource> {
&self.source
}
/// Unwrap the [`PluginCustomValueWithSource`], discarding the source.
pub fn without_source(self) -> PluginCustomValue {
// Because of the `Drop` implementation, we can't destructure this.
self.inner.clone()
}
/// Helper to get the plugin to implement an op
fn get_plugin(&self, span: Option<Span>, for_op: &str) -> Result<PluginInterface, ShellError> {
let wrap_err = |err: ShellError| ShellError::GenericError {
error: format!(
"Unable to spawn plugin `{}` to {for_op}",
self.source.name()
),
msg: err.to_string(),
span,
help: None,
inner: vec![err],
};
self.source
.clone()
.persistent(span)
.and_then(|p| p.get_plugin(None))
.map_err(wrap_err)
}
/// Add a [`PluginSource`] to the given [`CustomValue`] if it is a [`PluginCustomValue`].
pub fn add_source(value: &mut Box<dyn CustomValue>, source: &Arc<PluginSource>) {
if let Some(custom_value) = value.as_any().downcast_ref::<PluginCustomValue>() {
*value = Box::new(custom_value.clone().with_source(source.clone()));
}
}
/// Add a [`PluginSource`] to all [`PluginCustomValue`]s within the value, recursively.
pub fn add_source_in(value: &mut Value, source: &Arc<PluginSource>) -> Result<(), ShellError> {
with_custom_values_in(value, |custom_value| {
Self::add_source(custom_value.item, source);
Ok::<_, ShellError>(())
})
}
/// Remove a [`PluginSource`] from the given [`CustomValue`] if it is a
/// [`PluginCustomValueWithSource`]. This will turn it back into a [`PluginCustomValue`].
pub fn remove_source(value: &mut Box<dyn CustomValue>) {
if let Some(custom_value) = value.as_any().downcast_ref::<PluginCustomValueWithSource>() {
*value = Box::new(custom_value.clone().without_source());
}
}
/// Remove the [`PluginSource`] from all [`PluginCustomValue`]s within the value, recursively.
pub fn remove_source_in(value: &mut Value) -> Result<(), ShellError> {
with_custom_values_in(value, |custom_value| {
Self::remove_source(custom_value.item);
Ok::<_, ShellError>(())
})
}
/// Check that `self` came from the given `source`, and return an `error` if not.
pub fn verify_source(&self, span: Span, source: &PluginSource) -> Result<(), ShellError> {
if self.source.is_compatible(source) {
Ok(())
} else {
Err(ShellError::CustomValueIncorrectForPlugin {
name: self.name().to_owned(),
span,
dest_plugin: source.name().to_owned(),
src_plugin: Some(self.source.name().to_owned()),
})
}
}
/// Check that a [`CustomValue`] is a [`PluginCustomValueWithSource`] that came from the given
/// `source`, and return an error if not.
pub fn verify_source_of_custom_value(
value: Spanned<&dyn CustomValue>,
source: &PluginSource,
) -> Result<(), ShellError> {
if let Some(custom_value) = value
.item
.as_any()
.downcast_ref::<PluginCustomValueWithSource>()
{
custom_value.verify_source(value.span, source)
} else {
// Only PluginCustomValueWithSource can be sent
Err(ShellError::CustomValueIncorrectForPlugin {
name: value.item.type_name(),
span: value.span,
dest_plugin: source.name().to_owned(),
src_plugin: None,
})
}
}
}
impl std::ops::Deref for PluginCustomValueWithSource {
type Target = PluginCustomValue;
fn deref(&self) -> &PluginCustomValue {
&self.inner
}
}
/// This `Serialize` implementation always produces an error. Strip the source before sending.
impl Serialize for PluginCustomValueWithSource {
fn serialize<S>(&self, _serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
use serde::ser::Error;
Err(Error::custom(
"can't serialize PluginCustomValueWithSource, remove the source first",
))
}
}
impl CustomValue for PluginCustomValueWithSource {
fn clone_value(&self, span: Span) -> Value {
self.clone().into_value(span)
}
fn type_name(&self) -> String {
self.name().to_owned()
}
fn to_base_value(&self, span: Span) -> Result<Value, ShellError> {
self.get_plugin(Some(span), "get base value")?
.custom_value_to_base_value(self.clone().into_spanned(span))
}
fn follow_path_int(
&self,
self_span: Span,
index: usize,
path_span: Span,
) -> Result<Value, ShellError> {
self.get_plugin(Some(self_span), "follow cell path")?
.custom_value_follow_path_int(
self.clone().into_spanned(self_span),
index.into_spanned(path_span),
)
}
fn follow_path_string(
&self,
self_span: Span,
column_name: String,
path_span: Span,
) -> Result<Value, ShellError> {
self.get_plugin(Some(self_span), "follow cell path")?
.custom_value_follow_path_string(
self.clone().into_spanned(self_span),
column_name.into_spanned(path_span),
)
}
fn partial_cmp(&self, other: &Value) -> Option<Ordering> {
self.get_plugin(Some(other.span()), "perform comparison")
.and_then(|plugin| {
// We're passing Span::unknown() here because we don't have one, and it probably
// shouldn't matter here and is just a consequence of the API
plugin.custom_value_partial_cmp(self.clone(), other.clone())
})
.unwrap_or_else(|err| {
// We can't do anything with the error other than log it.
log::warn!(
"Error in partial_cmp on plugin custom value (source={source:?}): {err}",
source = self.source
);
None
})
.map(|ordering| ordering.into())
}
fn operation(
&self,
lhs_span: Span,
operator: Operator,
op_span: Span,
right: &Value,
) -> Result<Value, ShellError> {
self.get_plugin(Some(lhs_span), "invoke operator")?
.custom_value_operation(
self.clone().into_spanned(lhs_span),
operator.into_spanned(op_span),
right.clone(),
)
}
fn as_any(&self) -> &dyn std::any::Any {
self
}
fn as_mut_any(&mut self) -> &mut dyn std::any::Any {
self
}
#[doc(hidden)]
fn typetag_name(&self) -> &'static str {
"PluginCustomValueWithSource"
}
#[doc(hidden)]
fn typetag_deserialize(&self) {}
}
impl Drop for PluginCustomValueWithSource {
fn drop(&mut self) {
// If the custom value specifies notify_on_drop and this is the last copy, we need to let
// the plugin know about it if we can.
if self.notify_on_drop() && self.inner.ref_count() == 1 {
self.get_plugin(None, "drop")
// While notifying drop, we don't need a copy of the source
.and_then(|plugin| plugin.custom_value_dropped(self.inner.clone()))
.unwrap_or_else(|err| {
// We shouldn't do anything with the error except log it
let name = self.name();
log::warn!("Failed to notify drop of custom value ({name}): {err}")
});
}
}
}
/// Helper trait for adding a source to a [`PluginCustomValue`]
pub trait WithSource {
/// Add a source to a plugin custom value
fn with_source(self, source: Arc<PluginSource>) -> PluginCustomValueWithSource;
}
impl WithSource for PluginCustomValue {
fn with_source(self, source: Arc<PluginSource>) -> PluginCustomValueWithSource {
PluginCustomValueWithSource::new(self, source)
}
}

198
crates/nu-plugin-engine/src/plugin_custom_value_with_source/tests.rs Normal file
View File

@ -0,0 +1,198 @@
use std::sync::Arc;
use nu_plugin_protocol::test_util::{test_plugin_custom_value, TestCustomValue};
use nu_protocol::{engine::Closure, record, CustomValue, IntoSpanned, ShellError, Span, Value};
use crate::{
test_util::test_plugin_custom_value_with_source, PluginCustomValueWithSource, PluginSource,
};
use super::WithSource;
#[test]
fn add_source_in_at_root() -> Result<(), ShellError> {
let mut val = Value::test_custom_value(Box::new(test_plugin_custom_value()));
let source = Arc::new(PluginSource::new_fake("foo"));
PluginCustomValueWithSource::add_source_in(&mut val, &source)?;
let custom_value = val.as_custom_value()?;
let plugin_custom_value: &PluginCustomValueWithSource = custom_value
.as_any()
.downcast_ref()
.expect("not PluginCustomValueWithSource");
assert_eq!(
Arc::as_ptr(&source),
Arc::as_ptr(&plugin_custom_value.source)
);
Ok(())
}
fn check_record_custom_values(
val: &Value,
keys: &[&str],
mut f: impl FnMut(&str, &dyn CustomValue) -> Result<(), ShellError>,
) -> Result<(), ShellError> {
let record = val.as_record()?;
for key in keys {
let val = record
.get(key)
.unwrap_or_else(|| panic!("record does not contain '{key}'"));
let custom_value = val
.as_custom_value()
.unwrap_or_else(|_| panic!("'{key}' not custom value"));
f(key, custom_value)?;
}
Ok(())
}
#[test]
fn add_source_in_nested_record() -> Result<(), ShellError> {
let orig_custom_val = Value::test_custom_value(Box::new(test_plugin_custom_value()));
let mut val = Value::test_record(record! {
"foo" => orig_custom_val.clone(),
"bar" => orig_custom_val.clone(),
});
let source = Arc::new(PluginSource::new_fake("foo"));
PluginCustomValueWithSource::add_source_in(&mut val, &source)?;
check_record_custom_values(&val, &["foo", "bar"], |key, custom_value| {
let plugin_custom_value: &PluginCustomValueWithSource = custom_value
.as_any()
.downcast_ref()
.unwrap_or_else(|| panic!("'{key}' not PluginCustomValueWithSource"));
assert_eq!(
Arc::as_ptr(&source),
Arc::as_ptr(&plugin_custom_value.source),
"'{key}' source not set correctly"
);
Ok(())
})
}
fn check_list_custom_values(
val: &Value,
indices: impl IntoIterator<Item = usize>,
mut f: impl FnMut(usize, &dyn CustomValue) -> Result<(), ShellError>,
) -> Result<(), ShellError> {
let list = val.as_list()?;
for index in indices {
let val = list
.get(index)
.unwrap_or_else(|| panic!("[{index}] not present in list"));
let custom_value = val
.as_custom_value()
.unwrap_or_else(|_| panic!("[{index}] not custom value"));
f(index, custom_value)?;
}
Ok(())
}
#[test]
fn add_source_in_nested_list() -> Result<(), ShellError> {
let orig_custom_val = Value::test_custom_value(Box::new(test_plugin_custom_value()));
let mut val = Value::test_list(vec![orig_custom_val.clone(), orig_custom_val.clone()]);
let source = Arc::new(PluginSource::new_fake("foo"));
PluginCustomValueWithSource::add_source_in(&mut val, &source)?;
check_list_custom_values(&val, 0..=1, |index, custom_value| {
let plugin_custom_value: &PluginCustomValueWithSource = custom_value
.as_any()
.downcast_ref()
.unwrap_or_else(|| panic!("[{index}] not PluginCustomValueWithSource"));
assert_eq!(
Arc::as_ptr(&source),
Arc::as_ptr(&plugin_custom_value.source),
"[{index}] source not set correctly"
);
Ok(())
})
}
fn check_closure_custom_values(
val: &Value,
indices: impl IntoIterator<Item = usize>,
mut f: impl FnMut(usize, &dyn CustomValue) -> Result<(), ShellError>,
) -> Result<(), ShellError> {
let closure = val.as_closure()?;
for index in indices {
let val = closure
.captures
.get(index)
.unwrap_or_else(|| panic!("[{index}] not present in closure"));
let custom_value = val
.1
.as_custom_value()
.unwrap_or_else(|_| panic!("[{index}] not custom value"));
f(index, custom_value)?;
}
Ok(())
}
#[test]
fn add_source_in_nested_closure() -> Result<(), ShellError> {
let orig_custom_val = Value::test_custom_value(Box::new(test_plugin_custom_value()));
let mut val = Value::test_closure(Closure {
block_id: 0,
captures: vec![(0, orig_custom_val.clone()), (1, orig_custom_val.clone())],
});
let source = Arc::new(PluginSource::new_fake("foo"));
PluginCustomValueWithSource::add_source_in(&mut val, &source)?;
check_closure_custom_values(&val, 0..=1, |index, custom_value| {
let plugin_custom_value: &PluginCustomValueWithSource = custom_value
.as_any()
.downcast_ref()
.unwrap_or_else(|| panic!("[{index}] not PluginCustomValueWithSource"));
assert_eq!(
Arc::as_ptr(&source),
Arc::as_ptr(&plugin_custom_value.source),
"[{index}] source not set correctly"
);
Ok(())
})
}
#[test]
fn verify_source_error_message() -> Result<(), ShellError> {
let span = Span::new(5, 7);
let ok_val = test_plugin_custom_value_with_source();
let native_val = TestCustomValue(32);
let foreign_val =
test_plugin_custom_value().with_source(Arc::new(PluginSource::new_fake("other")));
let source = PluginSource::new_fake("test");
PluginCustomValueWithSource::verify_source_of_custom_value(
(&ok_val as &dyn CustomValue).into_spanned(span),
&source,
)
.expect("ok_val should be verified ok");
for (val, src_plugin) in [
(&native_val as &dyn CustomValue, None),
(&foreign_val as &dyn CustomValue, Some("other")),
] {
let error = PluginCustomValueWithSource::verify_source_of_custom_value(
val.into_spanned(span),
&source,
)
.expect_err(&format!(
"a custom value from {src_plugin:?} should result in an error"
));
if let ShellError::CustomValueIncorrectForPlugin {
name,
span: err_span,
dest_plugin,
src_plugin: err_src_plugin,
} = error
{
assert_eq!("TestCustomValue", name, "error.name from {src_plugin:?}");
assert_eq!(span, err_span, "error.span from {src_plugin:?}");
assert_eq!("test", dest_plugin, "error.dest_plugin from {src_plugin:?}");
assert_eq!(src_plugin, err_src_plugin.as_deref(), "error.src_plugin");
} else {
panic!("the error returned should be CustomValueIncorrectForPlugin");
}
}
Ok(())
}

90
crates/nu-plugin-engine/src/process.rs Normal file
View File

@ -0,0 +1,90 @@
use std::sync::{atomic::AtomicU32, Arc, Mutex, MutexGuard};
use nu_protocol::{ShellError, Span};
use nu_system::ForegroundGuard;
/// Provides a utility interface for a plugin interface to manage the process the plugin is running
/// in.
#[derive(Debug)]
pub(crate) struct PluginProcess {
pid: u32,
mutable: Mutex<MutablePart>,
}
#[derive(Debug)]
struct MutablePart {
foreground_guard: Option<ForegroundGuard>,
}
impl PluginProcess {
/// Manage a plugin process.
pub(crate) fn new(pid: u32) -> PluginProcess {
PluginProcess {
pid,
mutable: Mutex::new(MutablePart {
foreground_guard: None,
}),
}
}
/// The process ID of the plugin.
pub(crate) fn pid(&self) -> u32 {
self.pid
}
fn lock_mutable(&self) -> Result<MutexGuard<MutablePart>, ShellError> {
self.mutable.lock().map_err(|_| ShellError::NushellFailed {
msg: "the PluginProcess mutable lock has been poisoned".into(),
})
}
/// Move the plugin process to the foreground. See [`ForegroundGuard::new`].
///
/// This produces an error if the plugin process was already in the foreground.
///
/// Returns `Some()` on Unix with the process group ID if the plugin process will need to join
/// another process group to be part of the foreground.
pub(crate) fn enter_foreground(
&self,
span: Span,
pipeline_state: &Arc<(AtomicU32, AtomicU32)>,
) -> Result<Option<u32>, ShellError> {
let pid = self.pid;
let mut mutable = self.lock_mutable()?;
if mutable.foreground_guard.is_none() {
let guard = ForegroundGuard::new(pid, pipeline_state).map_err(|err| {
ShellError::GenericError {
error: "Failed to enter foreground".into(),
msg: err.to_string(),
span: Some(span),
help: None,
inner: vec![],
}
})?;
let pgrp = guard.pgrp();
mutable.foreground_guard = Some(guard);
Ok(pgrp)
} else {
Err(ShellError::GenericError {
error: "Can't enter foreground".into(),
msg: "this plugin is already running in the foreground".into(),
span: Some(span),
help: Some(
"you may be trying to run the command in parallel, or this may be a bug in \
the plugin"
.into(),
),
inner: vec![],
})
}
}
/// Move the plugin process out of the foreground. See [`ForegroundGuard::reset`].
///
/// This is a no-op if the plugin process was already in the background.
pub(crate) fn exit_foreground(&self) -> Result<(), ShellError> {
let mut mutable = self.lock_mutable()?;
drop(mutable.foreground_guard.take());
Ok(())
}
}

63
crates/nu-plugin-engine/src/source.rs Normal file
View File

@ -0,0 +1,63 @@
use super::GetPlugin;
use nu_protocol::{PluginIdentity, ShellError, Span};
use std::sync::{Arc, Weak};
/// The source of a custom value or plugin command. Includes a weak reference to the persistent
/// plugin so it can be retrieved.
#[derive(Debug, Clone)]
pub struct PluginSource {
/// The identity of the plugin
pub(crate) identity: Arc<PluginIdentity>,
/// A weak reference to the persistent plugin that might hold an interface to the plugin.
///
/// This is weak to avoid cyclic references, but it does mean we might fail to upgrade if
/// the engine state lost the [`PersistentPlugin`] at some point.
pub(crate) persistent: Weak<dyn GetPlugin>,
}
impl PluginSource {
/// Create from an implementation of `GetPlugin`
pub fn new(plugin: Arc<dyn GetPlugin>) -> PluginSource {
PluginSource {
identity: plugin.identity().clone().into(),
persistent: Arc::downgrade(&plugin),
}
}
/// Create a new fake source with a fake identity, for testing
///
/// Warning: [`.persistent()`] will always return an error.
pub fn new_fake(name: &str) -> PluginSource {
PluginSource {
identity: PluginIdentity::new_fake(name).into(),
persistent: Weak::<crate::PersistentPlugin>::new(),
}
}
/// Try to upgrade the persistent reference, and return an error referencing `span` as the
/// object that referenced it otherwise
pub fn persistent(&self, span: Option<Span>) -> Result<Arc<dyn GetPlugin>, ShellError> {
self.persistent
.upgrade()
.ok_or_else(|| ShellError::GenericError {
error: format!("The `{}` plugin is no longer present", self.identity.name()),
msg: "removed since this object was created".into(),
span,
help: Some("try recreating the object that came from the plugin".into()),
inner: vec![],
})
}
/// Sources are compatible if their identities are equal
pub(crate) fn is_compatible(&self, other: &PluginSource) -> bool {
self.identity == other.identity
}
}
impl std::ops::Deref for PluginSource {
type Target = PluginIdentity;
fn deref(&self) -> &PluginIdentity {
&self.identity
}
}

24
crates/nu-plugin-engine/src/test_util.rs Normal file
View File

@ -0,0 +1,24 @@
use std::sync::Arc;
use nu_plugin_core::interface_test_util::TestCase;
use nu_plugin_protocol::{test_util::test_plugin_custom_value, PluginInput, PluginOutput};
use crate::{PluginCustomValueWithSource, PluginInterfaceManager, PluginSource};
pub trait TestCaseExt {
/// Create a new [`PluginInterfaceManager`] that writes to this test case.
fn plugin(&self, name: &str) -> PluginInterfaceManager;
}
impl TestCaseExt for TestCase<PluginOutput, PluginInput> {
fn plugin(&self, name: &str) -> PluginInterfaceManager {
PluginInterfaceManager::new(PluginSource::new_fake(name).into(), None, self.clone())
}
}
pub fn test_plugin_custom_value_with_source() -> PluginCustomValueWithSource {
PluginCustomValueWithSource::new(
test_plugin_custom_value(),
Arc::new(PluginSource::new_fake("test")),
)
}

3
crates/nu-plugin-engine/src/util/mod.rs Normal file
View File

@ -0,0 +1,3 @@
mod mutable_cow;
pub use mutable_cow::MutableCow;

35
crates/nu-plugin-engine/src/util/mutable_cow.rs Normal file
View File

@ -0,0 +1,35 @@
/// Like [`Cow`] but with a mutable reference instead. So not exactly clone-on-write, but can be
/// made owned.
pub enum MutableCow<'a, T> {
Borrowed(&'a mut T),
Owned(T),
}
impl<'a, T: Clone> MutableCow<'a, T> {
pub fn owned(&self) -> MutableCow<'static, T> {
match self {
MutableCow::Borrowed(r) => MutableCow::Owned((*r).clone()),
MutableCow::Owned(o) => MutableCow::Owned(o.clone()),
}
}
}
impl<'a, T> std::ops::Deref for MutableCow<'a, T> {
type Target = T;
fn deref(&self) -> &T {
match self {
MutableCow::Borrowed(r) => r,
MutableCow::Owned(o) => o,
}
}
}
impl<'a, T> std::ops::DerefMut for MutableCow<'a, T> {
fn deref_mut(&mut self) -> &mut Self::Target {
match self {
MutableCow::Borrowed(r) => r,
MutableCow::Owned(o) => o,
}
}
}