nushell/crates
alex-tdrn 40e629beb1
Fix multibyte codepoint handling in detect columns --guess (#13272)
<!--
if this PR closes one or more issues, you can automatically link the PR
with
them by using one of the [*linking
keywords*](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword),
e.g.
- this PR should close #xxxx
- fixes #xxxx

you can also mention related issues, PRs or discussions!
-->

# Description
<!--
Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.

Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.
-->
This PR fixes #13269. The splitting code in `guess_width.rs` was
creating slices from char indices, instead of byte indices. This works
perfectly fine for 1-byte code points, but panics or returns wrong
results as soon as multibyte codepoints appear in the input. I
originally discovered this by piping `winget list` into `detect columns
--guess`, since winget sometimes uses the unicode ellipsis symbol (`…`)
which is 3 bytes long when encoded in utf-8.

# User-Facing Changes
<!-- List of all changes that impact the user experience here. This
helps us keep track of breaking changes. -->
`detect columns --guess` should not crash due to multibyte unicode input
anymore

before:

![image](https://github.com/nushell/nushell/assets/20356389/833cd732-be3b-4158-97f7-0ca2616ce23f)

after:

![image](https://github.com/nushell/nushell/assets/20356389/15358b40-4083-4a33-9f2c-87e63f39d985)


# Tests + Formatting
<!--
Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used` to
check that you're using the standard code style
- `cargo test --workspace` to check that all tests pass (on Windows make
sure to [enable developer
mode](https://learn.microsoft.com/en-us/windows/apps/get-started/developer-mode-features-and-debugging))
- `cargo run -- -c "use toolkit.nu; toolkit test stdlib"` to run the
tests for the standard library

> **Note**
> from `nushell` you can also use the `toolkit` as follows
> ```bash
> use toolkit.nu # or use an `env_change` hook to activate it
automatically
> toolkit check pr
> ```
-->
- Added tests to `guess_width.rs` for testing handling of multibyte as
well as combining diacritical marks

# After Submitting
<!-- If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
-->
2024-06-29 16:12:17 -05:00
..
nu_plugin_custom_values Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu_plugin_example Use IntoValue and FromValue derive macros in nu_plugin_example for example usage (#13220) 2024-06-26 17:50:14 -05:00
nu_plugin_formats Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu_plugin_gstat Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu_plugin_inc Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu_plugin_nu_example Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu_plugin_polars Polars 0.41 Upgrade (#13238) 2024-06-28 06:37:45 -05:00
nu_plugin_python Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu_plugin_query Added support for multiple attributes to query web -a (#13256) 2024-06-28 12:50:20 -05:00
nu_plugin_stress_internals Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-cli Converted perf function to be a macro. Utilized the perf macro within the polars plugin. (#13224) 2024-06-27 18:56:56 -05:00
nu-cmd-base Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-cmd-extra Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-cmd-lang Bump shadow-rs from 0.28.0 to 0.29.0 (#13226) 2024-06-26 22:48:45 +00:00
nu-cmd-plugin Add context to the I/O error messages in nu_cmd_plugin::util::modify_plugin_file() (#13259) 2024-06-27 23:49:06 -07:00
nu-color-config Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-command Fix multibyte codepoint handling in detect columns --guess (#13272) 2024-06-29 16:12:17 -05:00
nu-derive-value Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-engine Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-explore Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-glob Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-json Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-lsp Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-parser Restrict strings beginning with quote should also ending with quote (#13131) 2024-06-28 09:47:12 +08:00
nu-path Add typed path forms (#13115) 2024-06-25 18:33:57 -07:00
nu-plugin Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-plugin-core Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-plugin-engine Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-plugin-protocol Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-plugin-test-support Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-pretty-hex Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-protocol Restrict strings beginning with quote should also ending with quote (#13131) 2024-06-28 09:47:12 +08:00
nu-std Surprising symlink resolution for std path add (#13258) 2024-06-28 18:11:48 -05:00
nu-system Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-table Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-term-grid Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-test-support Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
nu-utils Converted perf function to be a macro. Utilized the perf macro within the polars plugin. (#13224) 2024-06-27 18:56:56 -05:00
nuon Bumping version to 0.95.1 (#13231) 2024-06-25 18:26:07 -07:00
README.md Remove old nushell/merge engine-q 2022-02-07 14:54:06 -05:00

Nushell core libraries and plugins

These sub-crates form both the foundation for Nu and a set of plugins which extend Nu with additional functionality.

Foundational libraries are split into two kinds of crates:

  • Core crates - those crates that work together to build the Nushell language engine
  • Support crates - a set of crates that support the engine with additional features like JSON support, ANSI support, and more.

Plugins are likewise also split into two types:

  • Core plugins - plugins that provide part of the default experience of Nu, including access to the system properties, processes, and web-connectivity features.
  • Extra plugins - these plugins run a wide range of different capabilities like working with different file types, charting, viewing binary data, and more.