mirror of
https://github.com/nushell/nushell.git
synced 2024-12-12 10:10:51 +01:00
ae0e13733d
This PR is an attempt to fix #8257 and fix #10985 (which is duplicate-ish) # Description The parser currently doesn't know how to deal with colons appearing while lexing whitespace-terminated tokens specifying a record value. Most notably, this means you can't use datetime literals in record value position (and as a consequence, `| to nuon | from nuon` roundtrips can fail), but it also means that bare words containing colons cause a non-useful error message.  `parser::parse_record` calls `lex::lex` with the `:` colon character in the `special_tokens` argument. This allows colons to terminate record keys, but as a side effect, it also causes colons to terminate record *values*. I added a new function `lex::lex_n_tokens`, which allows the caller to drive the lexing process more explicitly, and used it in `parser::parse_record` to let colons terminate record keys while not giving them special treatment when appearing in record values. This PR description previously said: *Another approach suggested in one of the issues was to support an additional datetime literal format that doesn't require colons. I like that that wouldn't require new `lex::lex_internal` behaviour, but an advantage of my approach is that it also newly allows for string record values given as bare words containing colons. I think this eliminates another possible source of confusion.* It was determined that this is undesirable, and in the current state of this PR, bare word record values with colons are rejected explicitly. The better error message is still a win. # User-Facing Changes In addition to the above, this PR also disables the use of "special" (non-item) tokens in record key and value position, and the use of a single bare `:` as a record key. Examples of behaviour *before* this PR: ```nu { a: b } # Valid, same as { 'a': 'b' } { a: b:c } # Error: expected ':' { a: 2024-08-13T22:11:09 } # Error: expected ':' { :: 1 } # Valid, same as { ':': 1 } { ;: 1 } # Valid, same as { ';': 1 } { a: || } # Valid, same as { 'a': '||' } ``` Examples of behaviour *after* this PR: ```nu { a: b } # (Unchanged) Valid, same as { 'a': 'b' } { a: b:c } # Error: colon in bare word specifying record value { a: 2024-08-13T22:11:09 } # Valid, same as { a: (2024-08-13T22:11:09) } { :: 1 } # Error: colon in bare word specifying record key { ;: 1 } # Error: expected item in record key position { a: || } # Error: expected item in record value position ``` # Tests + Formatting I added tests, but I'm not sure if they're sufficient and in the right place. # After Submitting I don't think documentation changes are needed for this, but please let me know if you disagree.
28 lines
850 B
Rust
28 lines
850 B
Rust
#![doc = include_str!("../README.md")]
|
|
mod deparse;
|
|
mod exportable;
|
|
mod flatten;
|
|
mod known_external;
|
|
mod lex;
|
|
mod lite_parser;
|
|
mod parse_keywords;
|
|
mod parse_patterns;
|
|
mod parse_shape_specs;
|
|
mod parser;
|
|
mod type_check;
|
|
|
|
pub use deparse::{escape_for_script_arg, escape_quote_string};
|
|
pub use flatten::{
|
|
flatten_block, flatten_expression, flatten_pipeline, flatten_pipeline_element, FlatShape,
|
|
};
|
|
pub use known_external::KnownExternal;
|
|
pub use lex::{lex, lex_n_tokens, lex_signature, LexState, Token, TokenContents};
|
|
pub use lite_parser::{lite_parse, LiteBlock, LiteCommand};
|
|
pub use nu_protocol::parser_path::*;
|
|
pub use parse_keywords::*;
|
|
|
|
pub use parser::{
|
|
is_math_expression_like, parse, parse_block, parse_expression, parse_external_call,
|
|
parse_unit_value, trim_quotes, trim_quotes_str, unescape_unquote_string, DURATION_UNIT_GROUPS,
|
|
};
|