Support extended unicode escapes in strings: "\u{10fff}" (#7883)

# Description

Support extended unicode escapes in strings with same syntax as Rust:
`"\u{6e}"`.

# User-Facing Changes

New syntax in string literals, `\u{NNNNNN}`, to go along with the
existing `\uNNNN`.
New syntax accepts 1-6 hex digits and rejects values greater than
0x10FFFF (max Unicode char)..

_(List of all changes that impact the user experience here. This helps
us keep track of breaking changes.)_

Won't break existing scripts, since this is new syntax.  

We might consider deprecating `char -u`, since users can now embed
unicode chars > 0xFFFF with the new escape.

# Tests + Formatting

Several unit tests and one integration test added.

- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
Done
- [x] `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
Done
- [x] `cargo test --workspace` to check that all tests pass  
Done

# After Submitting

- [ ] If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
This commit is contained in:
Bob Hyman
2023-01-28 15:25:53 -05:00
committed by GitHub
parent 2a39332d51
commit e616b2e247
4 changed files with 146 additions and 64 deletions

View File

@@ -412,30 +412,6 @@ mod string {
}
}
#[test]
pub fn parse_escaped_string() {
let engine_state = EngineState::new();
let mut working_set = StateWorkingSet::new(&engine_state);
let (block, err) = parse(
&mut working_set,
None,
b"\"hello \\u006e\\u0075\\u0073hell\"",
true,
&[],
);
assert!(err.is_none());
assert_eq!(block.len(), 1);
let expressions = &block[0];
assert_eq!(expressions.len(), 1);
if let PipelineElement::Expression(_, expr) = &expressions[0] {
assert_eq!(expr.expr, Expr::String("hello nushell".to_string()))
} else {
panic!("Not an expression")
}
}
mod interpolation {
use nu_protocol::Span;