holm/nushell
1
1
Fork 0
forked from extern/nushell
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Start to Add WASM Support Again (#14418)

Browse Source 
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
The [nushell/demo](https://github.com/nushell/demo) project successfully
demonstrated running Nushell in the browser using WASM. However, the
current version of Nushell cannot be easily built for the
`wasm32-unknown-unknown` target, the default for `wasm-bindgen`.

This PR introduces initial support for the `wasm32-unknown-unknown`
target by disabling OS-dependent features such as filesystem access, IO,
and platform/system-specific functionality. This separation is achieved
using a new `os` feature in the following crates:

 - `nu-cmd-lang`
 - `nu-command`
 - `nu-engine`
 - `nu-protocol`

The `os` feature includes all functionality that interacts with an
operating system. It is enabled by default, but can be disabled using
`--no-default-features`. All crates that depend on these core crates now
use `--no-default-features` to allow compilation for WASM.

To demonstrate compatibility, the following script builds all crates
expected to work with WASM. Direct user interaction, running external
commands, working with plugins, and features requiring `openssl` are out
of scope for now due to their complexity or reliance on C libraries,
which are difficult to compile and link in a WASM environment.

```nushell
[ # compatible crates
	"nu-cmd-base",
	"nu-cmd-extra",
	"nu-cmd-lang",
	"nu-color-config",
	"nu-command",
	"nu-derive-value",
	"nu-engine",
	"nu-glob",
	"nu-json",
	"nu-parser",
	"nu-path",
	"nu-pretty-hex",
	"nu-protocol",
	"nu-std",
	"nu-system",
	"nu-table",
	"nu-term-grid",
	"nu-utils",
	"nuon"
] | each {cargo build -p $in --target wasm32-unknown-unknown --no-default-features}
```

## Caveats
This PR has a few caveats:
1. **`miette` and `terminal-size` Dependency Issue**
`miette` depends on `terminal-size`, which uses `rustix` when the target
is not Windows. However, `rustix` requires `std::os::unix`, which is
unavailable in WASM. To address this, I opened a
[PR](https://github.com/eminence/terminal-size/pull/68) for
`terminal-size` to conditionally compile `rustix` only when the target
is Unix. For now, the `Cargo.toml` includes patches to:
    - Use my forked version of `terminal-size`.
- ~~Use an unreleased version of `miette` that depends on
`terminal-size@0.4`.~~

These patches are temporary and can be removed once the upstream changes
are merged and released.

2. **Test Output Adjustments**
Due to the slight bump in the `miette` version, one test required
adjustments to accommodate minor formatting changes in the error output,
such as shifted newlines.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
This shouldn't break anything but allows using some crates for targeting
`wasm32-unknown-unknown` to revive the demo page eventually.

# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->

- 🟢 `toolkit fmt`
- 🟢 `toolkit clippy`
- 🟢 `toolkit test`
- 🟢 `toolkit test stdlib`

I did not add any extra tests, I just checked that compiling works, also
when using the host target but unselecting the `os` feature.

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
~~Breaking the wasm support can be easily done by adding some `use`s or
by adding a new dependency, we should definitely add some CI that also
at least builds against wasm to make sure that building for it keep
working.~~
I added a job to build wasm.

---------

Co-authored-by: Ian Manske <ian.manske@pm.me>
This commit is contained in:
Piepmatz
2024-11-30 14:57:11 +01:00
committed by GitHub
parent 07a37f9b47
commit 3d5f853b03
48 changed files with 490 additions and 234 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
crates/nu-command/src/debug/info.rs
View File

@ -177,4 +177,9 @@ fn get_thread_id() -> u64 {
{
nix::sys::pthread::pthread_self() as u64
}
#[cfg(target_arch = "wasm32")]
{
// wasm doesn't have any threads accessible, so we return 0 as a fallback
0
}
}

4
crates/nu-command/src/debug/inspect.rs
View File

@ -1,6 +1,6 @@
use super::inspect_table;
use crossterm::terminal::size;
use nu_engine::command_prelude::*;
use nu_utils::terminal_size;
#[derive(Clone)]
pub struct Inspect;
@ -38,7 +38,7 @@ impl Command for Inspect {
let original_input = input_val.clone();
let description = input_val.get_type().to_string();
let (cols, _rows) = size().unwrap_or((0, 0));
let (cols, _rows) = terminal_size().unwrap_or((0, 0));
let table = inspect_table::build_table(input_val, description, cols as usize);

37
crates/nu-command/src/default_context.rs
View File

@ -27,6 +27,10 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
}
// Filters
#[cfg(feature = "rand")]
bind_command! {
Shuffle
}
bind_command! {
All,
Any,
@ -72,7 +76,6 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
Rename,
Reverse,
Select,
Shuffle,
Skip,
SkipUntil,
SkipWhile,
@ -114,6 +117,7 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
};
// System
#[cfg(feature = "os")]
bind_command! {
Complete,
External,
@ -161,17 +165,20 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
ViewSpan,
};
#[cfg(windows)]
#[cfg(all(feature = "os", windows))]
bind_command! { RegistryQuery }
#[cfg(any(
target_os = "android",
target_os = "linux",
target_os = "freebsd",
target_os = "netbsd",
target_os = "openbsd",
target_os = "macos",
target_os = "windows"
#[cfg(all(
feature = "os",
any(
target_os = "android",
target_os = "linux",
target_os = "freebsd",
target_os = "netbsd",
target_os = "openbsd",
target_os = "macos",
target_os = "windows"
)
))]
bind_command! { Ps };
@ -219,6 +226,7 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
};
// FileSystem
#[cfg(feature = "os")]
bind_command! {
Cd,
Ls,
@ -237,6 +245,7 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
};
// Platform
#[cfg(feature = "os")]
bind_command! {
Ansi,
AnsiLink,
@ -255,7 +264,7 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
Whoami,
};
#[cfg(unix)]
#[cfg(all(unix, feature = "os"))]
bind_command! { ULimit };
// Date
@ -380,6 +389,7 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
}
// Network
#[cfg(feature = "network")]
bind_command! {
Http,
HttpDelete,
@ -389,6 +399,9 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
HttpPost,
HttpPut,
HttpOptions,
Port,
}
bind_command! {
Url,
UrlBuildQuery,
UrlSplitQuery,
@ -396,10 +409,10 @@ pub fn add_shell_command_context(mut engine_state: EngineState) -> EngineState {
UrlEncode,
UrlJoin,
UrlParse,
Port,
}
// Random
#[cfg(feature = "rand")]
bind_command! {
Random,
RandomBool,

80
crates/nu-command/src/env/config/config_.rs vendored
View File

@ -1,4 +1,6 @@
use nu_engine::{command_prelude::*, get_full_help};
use nu_cmd_base::util::get_editor;
use nu_engine::{command_prelude::*, env_to_strings, get_full_help};
use nu_system::ForegroundChild;
#[derive(Clone)]
pub struct ConfigMeta;
@ -36,3 +38,79 @@ impl Command for ConfigMeta {
vec!["options", "setup"]
}
}
#[cfg(not(feature = "os"))]
pub(super) fn start_editor(
_: &'static str,
_: &EngineState,
_: &mut Stack,
call: &Call,
) -> Result<PipelineData, ShellError> {
Err(ShellError::DisabledOsSupport {
msg: "Running external commands is not available without OS support.".to_string(),
span: Some(call.head),
})
}
#[cfg(feature = "os")]
pub(super) fn start_editor(
config_path: &'static str,
engine_state: &EngineState,
stack: &mut Stack,
call: &Call,
) -> Result<PipelineData, ShellError> {
// Find the editor executable.
let (editor_name, editor_args) = get_editor(engine_state, stack, call.head)?;
let paths = nu_engine::env::path_str(engine_state, stack, call.head)?;
let cwd = engine_state.cwd(Some(stack))?;
let editor_executable =
crate::which(&editor_name, &paths, cwd.as_ref()).ok_or(ShellError::ExternalCommand {
label: format!("`{editor_name}` not found"),
help: "Failed to find the editor executable".into(),
span: call.head,
})?;
let Some(config_path) = engine_state.get_config_path(config_path) else {
return Err(ShellError::GenericError {
error: format!("Could not find $nu.{config_path}"),
msg: format!("Could not find $nu.{config_path}"),
span: None,
help: None,
inner: vec![],
});
};
let config_path = config_path.to_string_lossy().to_string();
// Create the command.
let mut command = std::process::Command::new(editor_executable);
// Configure PWD.
command.current_dir(cwd);
// Configure environment variables.
let envs = env_to_strings(engine_state, stack)?;
command.env_clear();
command.envs(envs);
// Configure args.
command.arg(config_path);
command.args(editor_args);
// Spawn the child process. On Unix, also put the child process to
// foreground if we're in an interactive session.
#[cfg(windows)]
let child = ForegroundChild::spawn(command)?;
#[cfg(unix)]
let child = ForegroundChild::spawn(
command,
engine_state.is_interactive,
&engine_state.pipeline_externals_state,
)?;
// Wrap the output into a `PipelineData::ByteStream`.
let child = nu_protocol::process::ChildProcess::new(child, None, false, call.head)?;
Ok(PipelineData::ByteStream(
ByteStream::child(child, call.head),
None,
))
}

61
crates/nu-command/src/env/config/config_env.rs vendored
View File

@ -1,7 +1,4 @@
use nu_cmd_base::util::get_editor;
use nu_engine::{command_prelude::*, env_to_strings};
use nu_protocol::{process::ChildProcess, ByteStream};
use nu_system::ForegroundChild;
use nu_engine::command_prelude::*;
#[derive(Clone)]
pub struct ConfigEnv;
@ -81,60 +78,6 @@ impl Command for ConfigEnv {
return Ok(Value::string(nu_utils::get_sample_env(), head).into_pipeline_data());
}
// Find the editor executable.
let (editor_name, editor_args) = get_editor(engine_state, stack, call.head)?;
let paths = nu_engine::env::path_str(engine_state, stack, call.head)?;
let cwd = engine_state.cwd(Some(stack))?;
let editor_executable = crate::which(&editor_name, &paths, cwd.as_ref()).ok_or(
ShellError::ExternalCommand {
label: format!("`{editor_name}` not found"),
help: "Failed to find the editor executable".into(),
span: call.head,
},
)?;
let Some(env_path) = engine_state.get_config_path("env-path") else {
return Err(ShellError::GenericError {
error: "Could not find $nu.env-path".into(),
msg: "Could not find $nu.env-path".into(),
span: None,
help: None,
inner: vec![],
});
};
let env_path = env_path.to_string_lossy().to_string();
// Create the command.
let mut command = std::process::Command::new(editor_executable);
// Configure PWD.
command.current_dir(cwd);
// Configure environment variables.
let envs = env_to_strings(engine_state, stack)?;
command.env_clear();
command.envs(envs);
// Configure args.
command.arg(env_path);
command.args(editor_args);
// Spawn the child process. On Unix, also put the child process to
// foreground if we're in an interactive session.
#[cfg(windows)]
let child = ForegroundChild::spawn(command)?;
#[cfg(unix)]
let child = ForegroundChild::spawn(
command,
engine_state.is_interactive,
&engine_state.pipeline_externals_state,
)?;
// Wrap the output into a `PipelineData::ByteStream`.
let child = ChildProcess::new(child, None, false, call.head)?;
Ok(PipelineData::ByteStream(
ByteStream::child(child, call.head),
None,
))
super::config_::start_editor("env-path", engine_state, stack, call)
}
}

61
crates/nu-command/src/env/config/config_nu.rs vendored
View File

@ -1,7 +1,4 @@
use nu_cmd_base::util::get_editor;
use nu_engine::{command_prelude::*, env_to_strings};
use nu_protocol::{process::ChildProcess, ByteStream};
use nu_system::ForegroundChild;
use nu_engine::command_prelude::*;
#[derive(Clone)]
pub struct ConfigNu;
@ -83,60 +80,6 @@ impl Command for ConfigNu {
return Ok(Value::string(nu_utils::get_sample_config(), head).into_pipeline_data());
}
// Find the editor executable.
let (editor_name, editor_args) = get_editor(engine_state, stack, call.head)?;
let paths = nu_engine::env::path_str(engine_state, stack, call.head)?;
let cwd = engine_state.cwd(Some(stack))?;
let editor_executable = crate::which(&editor_name, &paths, cwd.as_ref()).ok_or(
ShellError::ExternalCommand {
label: format!("`{editor_name}` not found"),
help: "Failed to find the editor executable".into(),
span: call.head,
},
)?;
let Some(config_path) = engine_state.get_config_path("config-path") else {
return Err(ShellError::GenericError {
error: "Could not find $nu.config-path".into(),
msg: "Could not find $nu.config-path".into(),
span: None,
help: None,
inner: vec![],
});
};
let config_path = config_path.to_string_lossy().to_string();
// Create the command.
let mut command = std::process::Command::new(editor_executable);
// Configure PWD.
command.current_dir(cwd);
// Configure environment variables.
let envs = env_to_strings(engine_state, stack)?;
command.env_clear();
command.envs(envs);
// Configure args.
command.arg(config_path);
command.args(editor_args);
// Spawn the child process. On Unix, also put the child process to
// foreground if we're in an interactive session.
#[cfg(windows)]
let child = ForegroundChild::spawn(command)?;
#[cfg(unix)]
let child = ForegroundChild::spawn(
command,
engine_state.is_interactive,
&engine_state.pipeline_externals_state,
)?;
// Wrap the output into a `PipelineData::ByteStream`.
let child = ChildProcess::new(child, None, false, call.head)?;
Ok(PipelineData::ByteStream(
ByteStream::child(child, call.head),
None,
))
super::config_::start_editor("config-path", engine_state, stack, call)
}
}

6
crates/nu-command/src/experimental/is_admin.rs
View File

@ -103,3 +103,9 @@ fn is_root_impl() -> bool {
elevated
}
#[cfg(target_arch = "wasm32")]
fn is_root_impl() -> bool {
// in wasm we don't have a user system, so technically we are never root
false
}

1
crates/nu-command/src/filesystem/save.rs
View File

@ -102,6 +102,7 @@ impl Command for Save {
ByteStreamSource::File(source) => {
stream_to_file(source, size, signals, file, span, progress)?;
}
#[cfg(feature = "os")]
ByteStreamSource::Child(mut child) => {
fn write_or_consume_stderr(
stderr: ChildPipe,

2
crates/nu-command/src/filters/mod.rs
View File

@ -37,6 +37,7 @@ mod reject;
mod rename;
mod reverse;
mod select;
#[cfg(feature = "rand")]
mod shuffle;
mod skip;
mod sort;
@ -95,6 +96,7 @@ pub use reject::Reject;
pub use rename::Rename;
pub use reverse::Reverse;
pub use select::Select;
#[cfg(feature = "rand")]
pub use shuffle::Shuffle;
pub use skip::*;
pub use sort::Sort;

9
crates/nu-command/src/filters/tee.rs
View File

@ -1,7 +1,9 @@
use nu_engine::{command_prelude::*, get_eval_block_with_early_return};
#[cfg(feature = "os")]
use nu_protocol::process::ChildPipe;
use nu_protocol::{
byte_stream::copy_with_signals, engine::Closure, process::ChildPipe, report_shell_error,
ByteStream, ByteStreamSource, OutDest, PipelineMetadata, Signals,
byte_stream::copy_with_signals, engine::Closure, report_shell_error, ByteStream,
ByteStreamSource, OutDest, PipelineMetadata, Signals,
};
use std::{
io::{self, Read, Write},
@ -152,6 +154,7 @@ use it in your pipeline."#
metadata,
))
}
#[cfg(feature = "os")]
ByteStreamSource::Child(mut child) => {
let stderr_thread = if use_stderr {
let stderr_thread = if let Some(stderr) = child.stderr.take() {
@ -454,6 +457,7 @@ fn copy(src: impl Read, dest: impl Write, info: &StreamInfo) -> Result<(), Shell
Ok(())
}
#[cfg(feature = "os")]
fn copy_pipe(pipe: ChildPipe, dest: impl Write, info: &StreamInfo) -> Result<(), ShellError> {
match pipe {
ChildPipe::Pipe(pipe) => copy(pipe, dest, info),
@ -477,6 +481,7 @@ fn copy_on_thread(
.map_err(|e| e.into_spanned(span).into())
}
#[cfg(feature = "os")]
fn copy_pipe_on_thread(
pipe: ChildPipe,
dest: impl Write + Send + 'static,

9
crates/nu-command/src/lib.rs
View File

@ -1,3 +1,4 @@
#![cfg_attr(not(feature = "os"), allow(unused))]
#![doc = include_str!("../README.md")]
mod bytes;
mod charting;
@ -8,6 +9,7 @@ mod default_context;
mod env;
mod example_test;
mod experimental;
#[cfg(feature = "os")]
mod filesystem;
mod filters;
mod formats;
@ -18,8 +20,10 @@ mod math;
mod misc;
mod network;
mod path;
#[cfg(feature = "os")]
mod platform;
mod progress_bar;
#[cfg(feature = "rand")]
mod random;
mod removed;
mod shells;
@ -27,6 +31,7 @@ mod sort_utils;
#[cfg(feature = "sqlite")]
mod stor;
mod strings;
#[cfg(feature = "os")]
mod system;
mod viewers;
@ -40,6 +45,7 @@ pub use env::*;
#[cfg(test)]
pub use example_test::{test_examples, test_examples_with_commands};
pub use experimental::*;
#[cfg(feature = "os")]
pub use filesystem::*;
pub use filters::*;
pub use formats::*;
@ -50,7 +56,9 @@ pub use math::*;
pub use misc::*;
pub use network::*;
pub use path::*;
#[cfg(feature = "os")]
pub use platform::*;
#[cfg(feature = "rand")]
pub use random::*;
pub use removed::*;
pub use shells::*;
@ -58,6 +66,7 @@ pub use sort_utils::*;
#[cfg(feature = "sqlite")]
pub use stor::*;
pub use strings::*;
#[cfg(feature = "os")]
pub use system::*;
pub use viewers::*;

4
crates/nu-command/src/network/mod.rs
View File

@ -1,8 +1,12 @@
#[cfg(feature = "network")]
mod http;
#[cfg(feature = "network")]
mod port;
mod url;
#[cfg(feature = "network")]
pub use self::http::*;
pub use self::url::*;
#[cfg(feature = "network")]
pub use port::SubCommand as Port;

5
crates/nu-command/src/viewers/griddle.rs
View File

@ -1,11 +1,10 @@
// use super::icons::{icon_for_file, iconify_style_ansi_to_nu};
use super::icons::icon_for_file;
use crossterm::terminal::size;
use lscolors::Style;
use nu_engine::{command_prelude::*, env_to_string};
use nu_protocol::Config;
use nu_term_grid::grid::{Alignment, Cell, Direction, Filling, Grid, GridOptions};
use nu_utils::get_ls_colors;
use nu_utils::{get_ls_colors, terminal_size};
use std::path::Path;
#[derive(Clone)]
@ -192,7 +191,7 @@ fn create_grid_output(
let cols = if let Some(col) = width_param {
col as u16
} else if let Ok((w, _h)) = size() {
} else if let Ok((w, _h)) = terminal_size() {
w
} else {
80u16

24
crates/nu-command/src/viewers/table.rs
View File

@ -2,7 +2,6 @@
// overall reduce the redundant calls to StyleComputer etc.
// the goal is to configure it once...
use crossterm::terminal::size;
use lscolors::{LsColors, Style};
use nu_color_config::{color_from_hex, StyleComputer, TextStyle};
use nu_engine::{command_prelude::*, env_to_string};
@ -15,7 +14,7 @@ use nu_table::{
common::create_nu_table_config, CollapsedTable, ExpandedTable, JustTable, NuTable, NuTableCell,
StringResult, TableOpts, TableOutput,
};
use nu_utils::get_ls_colors;
use nu_utils::{get_ls_colors, terminal_size};
use std::{
collections::VecDeque,
io::{IsTerminal, Read},
@ -30,7 +29,7 @@ const STREAM_PAGE_SIZE: usize = 1000;
fn get_width_param(width_param: Option<i64>) -> usize {
if let Some(col) = width_param {
col as usize
} else if let Ok((w, _h)) = size() {
} else if let Ok((w, _h)) = terminal_size() {
w as usize
} else {
80
@ -712,6 +711,13 @@ fn make_clickable_link(
) -> String {
// uri's based on this https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
#[cfg(any(
unix,
windows,
target_os = "redox",
target_os = "wasi",
target_os = "hermit"
))]
if show_clickable_links {
format!(
"\x1b]8;;{}\x1b\\{}\x1b]8;;\x1b\\",
@ -727,6 +733,18 @@ fn make_clickable_link(
None => full_path,
}
}
#[cfg(not(any(
unix,
windows,
target_os = "redox",
target_os = "wasi",
target_os = "hermit"
)))]
match link_name {
Some(link_name) => link_name.to_string(),
None => full_path,
}
}
struct PagingTableCreator {