feat!: Explicit cell-path case sensitivity syntax (#15692)

Related:
- #15683
- #14551
- #849
- #12701
- #11527

# Description
Currently various commands have differing behavior regarding cell-paths

```nushell
{a: 1, A: 2} | get a A
# => ╭───┬───╮
# => │ 0 │ 2 │
# => │ 1 │ 2 │
# => ╰───┴───╯
{a: 1, A: 2} | select a A
# => ╭───┬───╮
# => │ a │ 1 │
# => │ A │ 2 │
# => ╰───┴───╯
{A: 1} | update a 2
# => Error: nu:🐚:column_not_found
# => 
# =>   × Cannot find column 'a'
# =>    ╭─[entry #62:1:1]
# =>  1 │ {A: 1} | update a 2
# =>    · ───┬──          ┬
# =>    ·    │            ╰── cannot find column 'a'
# =>    ·    ╰── value originates here
# =>    ╰────
```

Proposal: making cell-path access case-sensitive by default and adding
new syntax for case-insensitive parts, similar to optional (?) parts.

```nushell
{FOO: BAR}.foo
# => Error: nu:🐚:name_not_found
# => 
# =>   × Name not found
# =>    ╭─[entry #60:1:21]
# =>  1 │ {FOO: BAR}.foo
# =>    ·            ─┬─
# =>    ·             ╰── did you mean 'FOO'?
# =>    ╰────
{FOO: BAR}.foo!
# => BAR
```

This would solve the problem of case sensitivity for all commands
without causing an explosion of flags _and_ make it more granular

Assigning to a field using a case-insensitive path is case-preserving.
```nushell
mut val = {FOO: "I'm FOO"}; $val
# => ╭─────┬─────────╮
# => │ FOO │ I'm FOO │
# => ╰─────┴─────────╯
$val.foo! = "I'm still FOO"; $val
# => ╭─────┬───────────────╮
# => │ FOO │ I'm still FOO │
# => ╰─────┴───────────────╯
```

For `update`, case-insensitive is case-preserving.
```nushell
{FOO: 1} | update foo! { $in + 1 }
# => ╭─────┬───╮
# => │ FOO │ 2 │
# => ╰─────┴───╯
```

`insert` can insert values into nested values so accessing into existing
columns is case-insensitive, but creating new columns uses the cell-path
as it is.
So `insert foo! ...` and `insert FOO! ...` would work exactly as they do
without `!`
```nushell
{FOO: {quox: 0}}
# => ╭─────┬──────────────╮
# => │     │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │     │ ╰──────┴───╯ │
# => ╰─────┴──────────────╯
{FOO: {quox: 0}} | insert foo.bar 1
# => ╭─────┬──────────────╮
# => │     │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │     │ ╰──────┴───╯ │
# => │     │ ╭─────┬───╮  │
# => │ foo │ │ bar │ 1 │  │
# => │     │ ╰─────┴───╯  │
# => ╰─────┴──────────────╯
{FOO: {quox: 0}} | insert foo!.bar 1
# => ╭─────┬──────────────╮
# => │     │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │     │ │ bar  │ 1 │ │
# => │     │ ╰──────┴───╯ │
# => ╰─────┴──────────────╯
```

`upsert` is tricky, depending on the input, the data might end up with
different column names in rows. We can either forbid case-insensitive
cell-paths for `upsert` or trust the user to keep their data in a
sensible shape.

This would be a breaking change as it would make existing cell-path
accesses case-sensitive, however the case-sensitivity is already
inconsistent and any attempt at making it consistent would be a breaking
change.

> What about `$env`?

1. Initially special case it so it keeps its current behavior.
2. Accessing environment variables with non-matching paths gives a
deprecation warning urging users to either use exact casing or use the
new explicit case-sensitivity syntax
3. Eventuall remove `$env`'s special case, making `$env` accesses
case-sensitive by default as well.

> `$env.ENV_CONVERSIONS`?

In addition to `from_string` and `to_string` add an optional field to
opt into case insensitive/preserving behavior.

# User-Facing Changes

- `get`, `where` and other previously case-insensitive commands are now
case-sensitive by default.
- `get`'s `--sensitive` flag removed, similar to `--ignore-errors` there
is now an `--ignore-case` flag that treats all parts of the cell-path as
case-insensitive.
- Users can explicitly choose the case case-sensitivity of cell-path
accesses or commands.

# Tests + Formatting

Existing tests required minimal modification. ***However, new tests are
not yet added***.

- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib

# After Submitting

- Update the website to include the new syntax
- Update [tree-sitter-nu](https://github.com/nushell/tree-sitter-nu)

---------

Co-authored-by: Bahex <17417311+Bahex@users.noreply.github.com>
This commit is contained in:
Bahex 2025-05-18 12:19:09 +03:00 committed by GitHub
parent 1e8876b076
commit c4dcfdb77b
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
34 changed files with 466 additions and 207 deletions

View File

@ -104,7 +104,7 @@ pub(crate) fn eval_cell_path(
eval_constant(working_set, head) eval_constant(working_set, head)
}?; }?;
head_value head_value
.follow_cell_path(path_members, false) .follow_cell_path(path_members)
.map(Cow::into_owned) .map(Cow::into_owned)
} }

View File

@ -1,5 +1,5 @@
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::{Config, ListStream, ast::PathMember, engine::StateWorkingSet}; use nu_protocol::{Config, ListStream, ast::PathMember, casing::Casing, engine::StateWorkingSet};
#[derive(Clone)] #[derive(Clone)]
pub struct FormatPattern; pub struct FormatPattern;
@ -251,11 +251,12 @@ fn format_record(
val: path.to_string(), val: path.to_string(),
span: *span, span: *span,
optional: false, optional: false,
casing: Casing::Sensitive,
}) })
.collect(); .collect();
let expanded_string = data_as_value let expanded_string = data_as_value
.follow_cell_path(&path_members, false)? .follow_cell_path(&path_members)?
.to_expanded_string(", ", config); .to_expanded_string(", ", config);
output.push_str(expanded_string.as_str()) output.push_str(expanded_string.as_str())
} }

View File

@ -1,5 +1,5 @@
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::ast::PathMember; use nu_protocol::{ast::PathMember, casing::Casing};
#[derive(Clone)] #[derive(Clone)]
pub struct IntoCellPath; pub struct IntoCellPath;
@ -17,7 +17,12 @@ impl Command for IntoCellPath {
(Type::List(Box::new(Type::Any)), Type::CellPath), (Type::List(Box::new(Type::Any)), Type::CellPath),
( (
Type::List(Box::new(Type::Record( Type::List(Box::new(Type::Record(
[("value".into(), Type::Any), ("optional".into(), Type::Bool)].into(), [
("value".into(), Type::Any),
("optional".into(), Type::Bool),
("insensitive".into(), Type::Bool),
]
.into(),
))), ))),
Type::CellPath, Type::CellPath,
), ),
@ -69,8 +74,8 @@ impl Command for IntoCellPath {
example: "'some.path' | split row '.' | into cell-path", example: "'some.path' | split row '.' | into cell-path",
result: Some(Value::test_cell_path(CellPath { result: Some(Value::test_cell_path(CellPath {
members: vec![ members: vec![
PathMember::test_string("some".into(), false), PathMember::test_string("some".into(), false, Casing::Sensitive),
PathMember::test_string("path".into(), false), PathMember::test_string("path".into(), false, Casing::Sensitive),
], ],
})), })),
}, },
@ -80,19 +85,20 @@ impl Command for IntoCellPath {
result: Some(Value::test_cell_path(CellPath { result: Some(Value::test_cell_path(CellPath {
members: vec![ members: vec![
PathMember::test_int(5, false), PathMember::test_int(5, false),
PathMember::test_string("c".into(), false), PathMember::test_string("c".into(), false, Casing::Sensitive),
PathMember::test_int(7, false), PathMember::test_int(7, false),
PathMember::test_string("h".into(), false), PathMember::test_string("h".into(), false, Casing::Sensitive),
], ],
})), })),
}, },
Example { Example {
description: "Convert table into cell path", description: "Convert table into cell path",
example: "[[value, optional]; [5 true] [c false]] | into cell-path", example: "[[value, optional, insensitive]; [5 true false] [c false false] [d false true]] | into cell-path",
result: Some(Value::test_cell_path(CellPath { result: Some(Value::test_cell_path(CellPath {
members: vec![ members: vec![
PathMember::test_int(5, true), PathMember::test_int(5, true),
PathMember::test_string("c".into(), false), PathMember::test_string("c".into(), false, Casing::Sensitive),
PathMember::test_string("d".into(), false, Casing::Insensitive),
], ],
})), })),
}, },
@ -175,6 +181,12 @@ fn record_to_path_member(
} }
}; };
if let Some(insensitive) = record.get("insensitive") {
if insensitive.as_bool()? {
member.make_insensitive();
}
};
Ok(member) Ok(member)
} }
@ -196,7 +208,9 @@ fn value_to_path_member(val: &Value, span: Span) -> Result<PathMember, ShellErro
let val_span = val.span(); let val_span = val.span();
let member = match val { let member = match val {
Value::Int { val, .. } => int_to_path_member(*val, val_span)?, Value::Int { val, .. } => int_to_path_member(*val, val_span)?,
Value::String { val, .. } => PathMember::string(val.into(), false, val_span), Value::String { val, .. } => {
PathMember::string(val.into(), false, Casing::Sensitive, val_span)
}
Value::Record { val, .. } => record_to_path_member(val, val_span, span)?, Value::Record { val, .. } => record_to_path_member(val, val_span, span)?,
other => { other => {
return Err(ShellError::CantConvert { return Err(ShellError::CantConvert {

View File

@ -1,5 +1,5 @@
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::{IntoValue, ast::PathMember}; use nu_protocol::{IntoValue, ast::PathMember, casing::Casing};
#[derive(Clone)] #[derive(Clone)]
pub struct SplitCellPath; pub struct SplitCellPath;
@ -16,7 +16,12 @@ impl Command for SplitCellPath {
( (
Type::CellPath, Type::CellPath,
Type::List(Box::new(Type::Record( Type::List(Box::new(Type::Record(
[("value".into(), Type::Any), ("optional".into(), Type::Bool)].into(), [
("value".into(), Type::Any),
("optional".into(), Type::Bool),
("insensitive".into(), Type::Bool),
]
.into(),
))), ))),
), ),
]) ])
@ -72,36 +77,43 @@ impl Command for SplitCellPath {
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_int(5), "value" => Value::test_int(5),
"optional" => Value::test_bool(true), "optional" => Value::test_bool(true),
"insensitive" => Value::test_bool(false),
}), }),
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_string("c"), "value" => Value::test_string("c"),
"optional" => Value::test_bool(false), "optional" => Value::test_bool(false),
"insensitive" => Value::test_bool(false),
}), }),
])), ])),
}, },
Example { Example {
description: "Split a complex cell-path", description: "Split a complex cell-path",
example: r#"$.a.b?.1."2"."c.d" | split cell-path"#, example: r#"$.a!.b?.1."2"."c.d" | split cell-path"#,
result: Some(Value::test_list(vec![ result: Some(Value::test_list(vec![
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_string("a"), "value" => Value::test_string("a"),
"optional" => Value::test_bool(false), "optional" => Value::test_bool(false),
"insensitive" => Value::test_bool(true),
}), }),
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_string("b"), "value" => Value::test_string("b"),
"optional" => Value::test_bool(true), "optional" => Value::test_bool(true),
"insensitive" => Value::test_bool(false),
}), }),
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_int(1), "value" => Value::test_int(1),
"optional" => Value::test_bool(false), "optional" => Value::test_bool(false),
"insensitive" => Value::test_bool(false),
}), }),
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_string("2"), "value" => Value::test_string("2"),
"optional" => Value::test_bool(false), "optional" => Value::test_bool(false),
"insensitive" => Value::test_bool(false),
}), }),
Value::test_record(record! { Value::test_record(record! {
"value" => Value::test_string("c.d"), "value" => Value::test_string("c.d"),
"optional" => Value::test_bool(false), "optional" => Value::test_bool(false),
"insensitive" => Value::test_bool(false),
}), }),
])), ])),
}, },
@ -114,19 +126,29 @@ fn split_cell_path(val: CellPath, span: Span) -> Result<Value, ShellError> {
struct PathMemberRecord { struct PathMemberRecord {
value: Value, value: Value,
optional: bool, optional: bool,
insensitive: bool,
} }
impl PathMemberRecord { impl PathMemberRecord {
fn from_path_member(pm: PathMember) -> Self { fn from_path_member(pm: PathMember) -> Self {
let (optional, internal_span) = match pm { let (optional, insensitive, internal_span) = match pm {
PathMember::String { optional, span, .. } PathMember::String {
| PathMember::Int { optional, span, .. } => (optional, span), optional,
casing,
span,
..
} => (optional, casing == Casing::Insensitive, span),
PathMember::Int { optional, span, .. } => (optional, false, span),
}; };
let value = match pm { let value = match pm {
PathMember::String { val, .. } => Value::string(val, internal_span), PathMember::String { val, .. } => Value::string(val, internal_span),
PathMember::Int { val, .. } => Value::int(val as i64, internal_span), PathMember::Int { val, .. } => Value::int(val as i64, internal_span),
}; };
Self { value, optional } Self {
value,
optional,
insensitive,
}
} }
} }

View File

@ -15,7 +15,7 @@ pub fn empty(
if !columns.is_empty() { if !columns.is_empty() {
for val in input { for val in input {
for column in &columns { for column in &columns {
if !val.follow_cell_path(&column.members, false)?.is_nothing() { if !val.follow_cell_path(&column.members)?.is_nothing() {
return Ok(Value::bool(negate, head).into_pipeline_data()); return Ok(Value::bool(negate, head).into_pipeline_data());
} }
} }

View File

@ -1,7 +1,7 @@
use std::borrow::Cow; use std::borrow::Cow;
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::{Signals, ast::PathMember}; use nu_protocol::{Signals, ast::PathMember, report_shell_warning};
#[derive(Clone)] #[derive(Clone)]
pub struct Get; pub struct Get;
@ -47,7 +47,7 @@ If multiple cell paths are given, this will produce a list of values."#
) )
.switch( .switch(
"sensitive", "sensitive",
"get path in a case sensitive manner", "get path in a case sensitive manner (deprecated)",
Some('s'), Some('s'),
) )
.allow_variants_without_examples(true) .allow_variants_without_examples(true)
@ -86,12 +86,12 @@ If multiple cell paths are given, this will produce a list of values."#
}, },
Example { Example {
description: "Getting Path/PATH in a case insensitive way", description: "Getting Path/PATH in a case insensitive way",
example: "$env | get paTH", example: "$env | get paTH!",
result: None, result: None,
}, },
Example { Example {
description: "Getting Path in a case sensitive way, won't work for 'PATH'", description: "Getting Path in a case sensitive way, won't work for 'PATH'",
example: "$env | get --sensitive Path", example: "$env | get Path",
result: None, result: None,
}, },
] ]
@ -110,14 +110,12 @@ If multiple cell paths are given, this will produce a list of values."#
let cell_path: CellPath = call.req_const(working_set, 0)?; let cell_path: CellPath = call.req_const(working_set, 0)?;
let rest: Vec<CellPath> = call.rest_const(working_set, 1)?; let rest: Vec<CellPath> = call.rest_const(working_set, 1)?;
let ignore_errors = call.has_flag_const(working_set, "ignore-errors")?; let ignore_errors = call.has_flag_const(working_set, "ignore-errors")?;
let sensitive = call.has_flag_const(working_set, "sensitive")?;
let metadata = input.metadata(); let metadata = input.metadata();
action( action(
input, input,
cell_path, cell_path,
rest, rest,
ignore_errors, ignore_errors,
sensitive,
working_set.permanent().signals().clone(), working_set.permanent().signals().clone(),
call.head, call.head,
) )
@ -134,14 +132,24 @@ If multiple cell paths are given, this will produce a list of values."#
let cell_path: CellPath = call.req(engine_state, stack, 0)?; let cell_path: CellPath = call.req(engine_state, stack, 0)?;
let rest: Vec<CellPath> = call.rest(engine_state, stack, 1)?; let rest: Vec<CellPath> = call.rest(engine_state, stack, 1)?;
let ignore_errors = call.has_flag(engine_state, stack, "ignore-errors")?; let ignore_errors = call.has_flag(engine_state, stack, "ignore-errors")?;
let sensitive = call.has_flag(engine_state, stack, "sensitive")?; let sensitive_span = call.get_flag_span(stack, "sensitive");
let metadata = input.metadata(); let metadata = input.metadata();
if let Some(span) = sensitive_span {
report_shell_warning(
engine_state,
&ShellError::Deprecated {
deprecated: "sensitive flag",
suggestion: "",
span,
help: Some("cell-paths are case-sensitive by default"),
},
);
}
action( action(
input, input,
cell_path, cell_path,
rest, rest,
ignore_errors, ignore_errors,
sensitive,
engine_state.signals().clone(), engine_state.signals().clone(),
call.head, call.head,
) )
@ -154,7 +162,6 @@ fn action(
mut cell_path: CellPath, mut cell_path: CellPath,
mut rest: Vec<CellPath>, mut rest: Vec<CellPath>,
ignore_errors: bool, ignore_errors: bool,
sensitive: bool,
signals: Signals, signals: Signals,
span: Span, span: Span,
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
@ -180,7 +187,7 @@ fn action(
} }
if rest.is_empty() { if rest.is_empty() {
follow_cell_path_into_stream(input, signals, cell_path.members, span, !sensitive) follow_cell_path_into_stream(input, signals, cell_path.members, span)
} else { } else {
let mut output = vec![]; let mut output = vec![];
@ -189,11 +196,7 @@ fn action(
let input = input.into_value(span)?; let input = input.into_value(span)?;
for path in paths { for path in paths {
output.push( output.push(input.follow_cell_path(&path.members)?.into_owned());
input
.follow_cell_path(&path.members, !sensitive)?
.into_owned(),
);
} }
Ok(output.into_iter().into_pipeline_data(span, signals)) Ok(output.into_iter().into_pipeline_data(span, signals))
@ -212,7 +215,6 @@ pub fn follow_cell_path_into_stream(
signals: Signals, signals: Signals,
cell_path: Vec<PathMember>, cell_path: Vec<PathMember>,
head: Span, head: Span,
insensitive: bool,
) -> Result<PipelineData, ShellError> { ) -> Result<PipelineData, ShellError> {
// when given an integer/indexing, we fallback to // when given an integer/indexing, we fallback to
// the default nushell indexing behaviour // the default nushell indexing behaviour
@ -227,7 +229,7 @@ pub fn follow_cell_path_into_stream(
let span = value.span(); let span = value.span();
value value
.follow_cell_path(&cell_path, insensitive) .follow_cell_path(&cell_path)
.map(Cow::into_owned) .map(Cow::into_owned)
.unwrap_or_else(|error| Value::error(error, span)) .unwrap_or_else(|error| Value::error(error, span))
}) })
@ -237,7 +239,7 @@ pub fn follow_cell_path_into_stream(
} }
_ => data _ => data
.follow_cell_path(&cell_path, head, insensitive) .follow_cell_path(&cell_path, head)
.map(|x| x.into_pipeline_data()), .map(|x| x.into_pipeline_data()),
} }
} }

View File

@ -322,7 +322,7 @@ fn group_cell_path(
let mut groups = IndexMap::<_, Vec<_>>::new(); let mut groups = IndexMap::<_, Vec<_>>::new();
for value in values.into_iter() { for value in values.into_iter() {
let key = value.follow_cell_path(&column_name.members, false)?; let key = value.follow_cell_path(&column_name.members)?;
if key.is_nothing() { if key.is_nothing() {
continue; // likely the result of a failed optional access, ignore this value continue; // likely the result of a failed optional access, ignore this value

View File

@ -301,7 +301,7 @@ fn insert_value_by_closure(
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let value_at_path = if first_path_member_int { let value_at_path = if first_path_member_int {
value value
.follow_cell_path(cell_path, false) .follow_cell_path(cell_path)
.map(Cow::into_owned) .map(Cow::into_owned)
.unwrap_or(Value::nothing(span)) .unwrap_or(Value::nothing(span))
} else { } else {
@ -321,7 +321,7 @@ fn insert_single_value_by_closure(
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let value_at_path = if first_path_member_int { let value_at_path = if first_path_member_int {
value value
.follow_cell_path(cell_path, false) .follow_cell_path(cell_path)
.map(Cow::into_owned) .map(Cow::into_owned)
.unwrap_or(Value::nothing(span)) .unwrap_or(Value::nothing(span))
} else { } else {

View File

@ -1,5 +1,5 @@
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::ast::PathMember; use nu_protocol::{ast::PathMember, casing::Casing};
use std::{cmp::Reverse, collections::HashSet}; use std::{cmp::Reverse, collections::HashSet};
#[derive(Clone)] #[derive(Clone)]
@ -63,6 +63,7 @@ impl Command for Reject {
val: val.clone(), val: val.clone(),
span: *col_span, span: *col_span,
optional: false, optional: false,
casing: Casing::Sensitive,
}], }],
}; };
new_columns.push(cv.clone()); new_columns.push(cv.clone());

View File

@ -1,5 +1,5 @@
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::{PipelineIterator, ast::PathMember}; use nu_protocol::{PipelineIterator, ast::PathMember, casing::Casing};
use std::collections::BTreeSet; use std::collections::BTreeSet;
#[derive(Clone)] #[derive(Clone)]
@ -67,6 +67,7 @@ produce a table, a list will produce a list, and a record will produce a record.
val, val,
span: col_span, span: col_span,
optional: false, optional: false,
casing: Casing::Sensitive,
}], }],
}; };
new_columns.push(cv); new_columns.push(cv);
@ -233,7 +234,7 @@ fn select(
if !columns.is_empty() { if !columns.is_empty() {
let mut record = Record::new(); let mut record = Record::new();
for path in &columns { for path in &columns {
match input_val.follow_cell_path(&path.members, false) { match input_val.follow_cell_path(&path.members) {
Ok(fetcher) => { Ok(fetcher) => {
record.push(path.to_column_name(), fetcher.into_owned()); record.push(path.to_column_name(), fetcher.into_owned());
} }
@ -256,7 +257,7 @@ fn select(
let mut record = Record::new(); let mut record = Record::new();
for cell_path in columns { for cell_path in columns {
let result = v.follow_cell_path(&cell_path.members, false)?; let result = v.follow_cell_path(&cell_path.members)?;
record.push(cell_path.to_column_name(), result.into_owned()); record.push(cell_path.to_column_name(), result.into_owned());
} }
@ -273,7 +274,7 @@ fn select(
if !columns.is_empty() { if !columns.is_empty() {
let mut record = Record::new(); let mut record = Record::new();
for path in &columns { for path in &columns {
match x.follow_cell_path(&path.members, false) { match x.follow_cell_path(&path.members) {
Ok(value) => { Ok(value) => {
record.push(path.to_column_name(), value.into_owned()); record.push(path.to_column_name(), value.into_owned());
} }

View File

@ -1,5 +1,5 @@
use nu_engine::command_prelude::*; use nu_engine::command_prelude::*;
use nu_protocol::ast::PathMember; use nu_protocol::{ast::PathMember, casing::Casing};
use crate::Comparator; use crate::Comparator;
@ -164,7 +164,14 @@ impl Command for Sort {
if let Type::Table(cols) = r#type { if let Type::Table(cols) = r#type {
let columns: Vec<Comparator> = cols let columns: Vec<Comparator> = cols
.iter() .iter()
.map(|col| vec![PathMember::string(col.0.clone(), false, Span::unknown())]) .map(|col| {
vec![PathMember::string(
col.0.clone(),
false,
Casing::Sensitive,
Span::unknown(),
)]
})
.map(|members| CellPath { members }) .map(|members| CellPath { members })
.map(Comparator::CellPath) .map(Comparator::CellPath)
.collect(); .collect();

View File

@ -243,7 +243,7 @@ fn update_value_by_closure(
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let value_at_path = value.follow_cell_path(cell_path, false)?; let value_at_path = value.follow_cell_path(cell_path)?;
let arg = if first_path_member_int { let arg = if first_path_member_int {
value_at_path.as_ref() value_at_path.as_ref()
@ -266,7 +266,7 @@ fn update_single_value_by_closure(
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let value_at_path = value.follow_cell_path(cell_path, false)?; let value_at_path = value.follow_cell_path(cell_path)?;
let arg = if first_path_member_int { let arg = if first_path_member_int {
value_at_path.as_ref() value_at_path.as_ref()

View File

@ -321,7 +321,7 @@ fn upsert_value_by_closure(
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let value_at_path = value.follow_cell_path(cell_path, false); let value_at_path = value.follow_cell_path(cell_path);
let arg = if first_path_member_int { let arg = if first_path_member_int {
value_at_path value_at_path
@ -352,7 +352,7 @@ fn upsert_single_value_by_closure(
cell_path: &[PathMember], cell_path: &[PathMember],
first_path_member_int: bool, first_path_member_int: bool,
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let value_at_path = value.follow_cell_path(cell_path, false); let value_at_path = value.follow_cell_path(cell_path);
let arg = if first_path_member_int { let arg = if first_path_member_int {
value_at_path value_at_path

View File

@ -89,7 +89,7 @@ impl Command for InputList {
.into_iter() .into_iter()
.map(move |val| { .map(move |val| {
let display_value = if let Some(ref cellpath) = display_path { let display_value = if let Some(ref cellpath) = display_path {
val.follow_cell_path(&cellpath.members, false)? val.follow_cell_path(&cellpath.members)?
.to_expanded_string(", ", &config) .to_expanded_string(", ", &config)
} else { } else {
val.to_expanded_string(", ", &config) val.to_expanded_string(", ", &config)

View File

@ -239,8 +239,8 @@ pub fn compare_cell_path(
insensitive: bool, insensitive: bool,
natural: bool, natural: bool,
) -> Result<Ordering, ShellError> { ) -> Result<Ordering, ShellError> {
let left = left.follow_cell_path(&cell_path.members, false)?; let left = left.follow_cell_path(&cell_path.members)?;
let right = right.follow_cell_path(&cell_path.members, false)?; let right = right.follow_cell_path(&cell_path.members)?;
compare_values(&left, &right, insensitive, natural) compare_values(&left, &right, insensitive, natural)
} }

View File

@ -1,6 +1,6 @@
use chrono::{DateTime, FixedOffset}; use chrono::{DateTime, FixedOffset};
use nu_path::AbsolutePathBuf; use nu_path::AbsolutePathBuf;
use nu_protocol::{Span, Value, ast::PathMember, engine::EngineState, record}; use nu_protocol::{Span, Value, ast::PathMember, casing::Casing, engine::EngineState, record};
use nu_test_support::{ use nu_test_support::{
fs::{Stub, line_ending}, fs::{Stub, line_ending},
nu, pipeline, nu, pipeline,
@ -327,6 +327,7 @@ fn into_sqlite_big_insert() {
val: "somedate".into(), val: "somedate".into(),
span: Span::unknown(), span: Span::unknown(),
optional: false, optional: false,
casing: Casing::Sensitive,
}], }],
Box::new(|dateval| { Box::new(|dateval| {
Value::string(dateval.coerce_string().unwrap(), dateval.span()) Value::string(dateval.coerce_string().unwrap(), dateval.span())

View File

@ -2,6 +2,7 @@ use nu_command::{Comparator, sort, sort_by, sort_record};
use nu_protocol::{ use nu_protocol::{
Record, Span, Value, Record, Span, Value,
ast::{CellPath, PathMember}, ast::{CellPath, PathMember},
casing::Casing,
record, record,
}; };
@ -527,6 +528,7 @@ fn test_sort_equivalent() {
val: "value".to_string(), val: "value".to_string(),
span: Span::test_data(), span: Span::test_data(),
optional: false, optional: false,
casing: Casing::Sensitive,
}], }],
}); });

View File

@ -268,7 +268,7 @@ pub fn eval_expression_with_input<D: DebugContext>(
// FIXME: protect this collect with ctrl-c // FIXME: protect this collect with ctrl-c
input = eval_subexpression::<D>(engine_state, stack, block, input)? input = eval_subexpression::<D>(engine_state, stack, block, input)?
.into_value(*span)? .into_value(*span)?
.follow_cell_path(&full_cell_path.tail, false)? .follow_cell_path(&full_cell_path.tail)?
.into_owned() .into_owned()
.into_pipeline_data() .into_pipeline_data()
} else { } else {
@ -592,8 +592,11 @@ impl Eval for EvalRuntime {
// Retrieve the updated environment value. // Retrieve the updated environment value.
lhs.upsert_data_at_cell_path(&cell_path.tail, rhs)?; lhs.upsert_data_at_cell_path(&cell_path.tail, rhs)?;
let value = let value = lhs.follow_cell_path(&[{
lhs.follow_cell_path(&[cell_path.tail[0].clone()], true)?; let mut pm = cell_path.tail[0].clone();
pm.make_insensitive();
pm
}])?;
// Reject attempts to set automatic environment variables. // Reject attempts to set automatic environment variables.
if is_automatic_env_var(&original_key) { if is_automatic_env_var(&original_key) {

View File

@ -678,7 +678,7 @@ fn eval_instruction<D: DebugContext>(
let data = ctx.take_reg(*src_dst); let data = ctx.take_reg(*src_dst);
let path = ctx.take_reg(*path); let path = ctx.take_reg(*path);
if let PipelineData::Value(Value::CellPath { val: path, .. }, _) = path { if let PipelineData::Value(Value::CellPath { val: path, .. }, _) = path {
let value = data.follow_cell_path(&path.members, *span, true)?; let value = data.follow_cell_path(&path.members, *span)?;
ctx.put_reg(*src_dst, value.into_pipeline_data()); ctx.put_reg(*src_dst, value.into_pipeline_data());
Ok(Continue) Ok(Continue)
} else if let PipelineData::Value(Value::Error { error, .. }, _) = path { } else if let PipelineData::Value(Value::Error { error, .. }, _) = path {
@ -694,7 +694,7 @@ fn eval_instruction<D: DebugContext>(
let value = ctx.clone_reg_value(*src, *span)?; let value = ctx.clone_reg_value(*src, *span)?;
let path = ctx.take_reg(*path); let path = ctx.take_reg(*path);
if let PipelineData::Value(Value::CellPath { val: path, .. }, _) = path { if let PipelineData::Value(Value::CellPath { val: path, .. }, _) = path {
let value = value.follow_cell_path(&path.members, true)?; let value = value.follow_cell_path(&path.members)?;
ctx.put_reg(*dst, value.into_owned().into_pipeline_data()); ctx.put_reg(*dst, value.into_owned().into_pipeline_data());
Ok(Continue) Ok(Continue)
} else if let PipelineData::Value(Value::Error { error, .. }, _) = path { } else if let PipelineData::Value(Value::Error { error, .. }, _) = path {

View File

@ -64,7 +64,7 @@ impl LanguageServer {
Some( Some(
var.const_val var.const_val
.as_ref() .as_ref()
.and_then(|val| val.follow_cell_path(cell_path, false).ok()) .and_then(|val| val.follow_cell_path(cell_path).ok())
.map(|val| val.span()) .map(|val| val.span())
.unwrap_or(var.declaration_span), .unwrap_or(var.declaration_span),
) )

View File

@ -161,7 +161,7 @@ impl LanguageServer {
markdown_hover( markdown_hover(
var.const_val var.const_val
.as_ref() .as_ref()
.and_then(|val| val.follow_cell_path(&cell_path, false).ok()) .and_then(|val| val.follow_cell_path(&cell_path).ok())
.map(|val| { .map(|val| {
let ty = val.get_type(); let ty = val.get_type();
if let Ok(s) = val.coerce_str() { if let Ok(s) = val.coerce_str() {

View File

@ -15,7 +15,7 @@ use nu_engine::DIR_VAR_PARSER_INFO;
use nu_protocol::{ use nu_protocol::{
BlockId, DeclId, DidYouMean, ENV_VARIABLE_ID, FilesizeUnit, Flag, IN_VARIABLE_ID, ParseError, BlockId, DeclId, DidYouMean, ENV_VARIABLE_ID, FilesizeUnit, Flag, IN_VARIABLE_ID, ParseError,
PositionalArg, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value, VarId, ast::*, PositionalArg, ShellError, Signature, Span, Spanned, SyntaxShape, Type, Value, VarId, ast::*,
engine::StateWorkingSet, eval_const::eval_constant, casing::Casing, engine::StateWorkingSet, eval_const::eval_constant,
}; };
use std::{ use std::{
collections::{HashMap, HashSet}, collections::{HashMap, HashSet},
@ -1799,7 +1799,7 @@ pub fn parse_range(working_set: &mut StateWorkingSet, span: Span) -> Option<Expr
&contents[..dotdot_pos[0]], &contents[..dotdot_pos[0]],
span.start, span.start,
&[], &[],
&[b'.', b'?'], &[b'.', b'?', b'!'],
true, true,
); );
if let Some(_err) = err { if let Some(_err) = err {
@ -2317,9 +2317,55 @@ pub fn parse_cell_path(
expect_dot: bool, expect_dot: bool,
) -> Vec<PathMember> { ) -> Vec<PathMember> {
enum TokenType { enum TokenType {
Dot, // . Dot, // .
QuestionOrDot, // ? or . DotOrSign, // . or ? or !
PathMember, // an int or string, like `1` or `foo` DotOrExclamation, // . or !
DotOrQuestion, // . or ?
PathMember, // an int or string, like `1` or `foo`
}
enum ModifyMember {
No,
Optional,
Insensitive,
}
impl TokenType {
fn expect(&mut self, byte: u8) -> Result<ModifyMember, &'static str> {
match (&*self, byte) {
(Self::PathMember, _) => {
*self = Self::DotOrSign;
Ok(ModifyMember::No)
}
(
Self::Dot | Self::DotOrSign | Self::DotOrExclamation | Self::DotOrQuestion,
b'.',
) => {
*self = Self::PathMember;
Ok(ModifyMember::No)
}
(Self::DotOrSign, b'!') => {
*self = Self::DotOrQuestion;
Ok(ModifyMember::Insensitive)
}
(Self::DotOrSign, b'?') => {
*self = Self::DotOrExclamation;
Ok(ModifyMember::Optional)
}
(Self::DotOrSign, _) => Err(". or ! or ?"),
(Self::DotOrExclamation, b'!') => {
*self = Self::Dot;
Ok(ModifyMember::Insensitive)
}
(Self::DotOrExclamation, _) => Err(". or !"),
(Self::DotOrQuestion, b'?') => {
*self = Self::Dot;
Ok(ModifyMember::Optional)
}
(Self::DotOrQuestion, _) => Err(". or ?"),
(Self::Dot, _) => Err("."),
}
}
} }
// Parsing a cell path is essentially a state machine, and this is the state // Parsing a cell path is essentially a state machine, and this is the state
@ -2334,69 +2380,68 @@ pub fn parse_cell_path(
for path_element in tokens { for path_element in tokens {
let bytes = working_set.get_span_contents(path_element.span); let bytes = working_set.get_span_contents(path_element.span);
match expected_token { // both parse_int and parse_string require their source to be non-empty
TokenType::Dot => { // all cases where `bytes` is empty is an error
if bytes.len() != 1 || bytes[0] != b'.' { let Some((&first, rest)) = bytes.split_first() else {
working_set.error(ParseError::Expected(".", path_element.span)); working_set.error(ParseError::Expected("string", path_element.span));
return tail; return tail;
} };
expected_token = TokenType::PathMember; let single_char = rest.is_empty();
}
TokenType::QuestionOrDot => {
if bytes.len() == 1 && bytes[0] == b'.' {
expected_token = TokenType::PathMember;
} else if bytes.len() == 1 && bytes[0] == b'?' {
if let Some(last) = tail.last_mut() {
match last {
PathMember::String { optional, .. } => *optional = true,
PathMember::Int { optional, .. } => *optional = true,
}
}
expected_token = TokenType::Dot;
} else {
working_set.error(ParseError::Expected(". or ?", path_element.span));
return tail;
}
}
TokenType::PathMember => {
let starting_error_count = working_set.parse_errors.len();
let expr = parse_int(working_set, path_element.span); if let TokenType::PathMember = expected_token {
working_set.parse_errors.truncate(starting_error_count); let starting_error_count = working_set.parse_errors.len();
match expr { let expr = parse_int(working_set, path_element.span);
Expression { working_set.parse_errors.truncate(starting_error_count);
expr: Expr::Int(val),
span, match expr {
.. Expression {
} => tail.push(PathMember::Int { expr: Expr::Int(val),
val: val as usize, span,
span, ..
optional: false, } => tail.push(PathMember::Int {
}), val: val as usize,
_ => { span,
let result = parse_string(working_set, path_element.span); optional: false,
match result { }),
Expression { _ => {
expr: Expr::String(string), let result = parse_string(working_set, path_element.span);
match result {
Expression {
expr: Expr::String(string),
span,
..
} => {
tail.push(PathMember::String {
val: string,
span, span,
.. optional: false,
} => { casing: Casing::Sensitive,
tail.push(PathMember::String { });
val: string, }
span, _ => {
optional: false, working_set.error(ParseError::Expected("string", path_element.span));
}); return tail;
}
_ => {
working_set
.error(ParseError::Expected("string", path_element.span));
return tail;
}
} }
} }
} }
expected_token = TokenType::QuestionOrDot; }
expected_token = TokenType::DotOrSign;
} else {
match expected_token.expect(if single_char { first } else { b' ' }) {
Ok(modify) => {
if let Some(last) = tail.last_mut() {
match modify {
ModifyMember::No => {}
ModifyMember::Optional => last.make_optional(),
ModifyMember::Insensitive => last.make_insensitive(),
}
};
}
Err(expected) => {
working_set.error(ParseError::Expected(expected, path_element.span));
return tail;
}
} }
} }
} }
@ -2407,7 +2452,13 @@ pub fn parse_cell_path(
pub fn parse_simple_cell_path(working_set: &mut StateWorkingSet, span: Span) -> Expression { pub fn parse_simple_cell_path(working_set: &mut StateWorkingSet, span: Span) -> Expression {
let source = working_set.get_span_contents(span); let source = working_set.get_span_contents(span);
let (tokens, err) = lex(source, span.start, &[b'\n', b'\r'], &[b'.', b'?'], true); let (tokens, err) = lex(
source,
span.start,
&[b'\n', b'\r'],
&[b'.', b'?', b'!'],
true,
);
if let Some(err) = err { if let Some(err) = err {
working_set.error(err) working_set.error(err)
} }
@ -2433,7 +2484,13 @@ pub fn parse_full_cell_path(
let full_cell_span = span; let full_cell_span = span;
let source = working_set.get_span_contents(span); let source = working_set.get_span_contents(span);
let (tokens, err) = lex(source, span.start, &[b'\n', b'\r'], &[b'.', b'?'], true); let (tokens, err) = lex(
source,
span.start,
&[b'\n', b'\r'],
&[b'.', b'?', b'!'],
true,
);
if let Some(err) = err { if let Some(err) = err {
working_set.error(err) working_set.error(err)
} }

View File

@ -1,5 +1,5 @@
use super::Expression; use super::Expression;
use crate::Span; use crate::{Span, casing::Casing};
use nu_utils::{escape_quote_string, needs_quoting}; use nu_utils::{escape_quote_string, needs_quoting};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::{cmp::Ordering, fmt::Display}; use std::{cmp::Ordering, fmt::Display};
@ -14,6 +14,8 @@ pub enum PathMember {
/// If marked as optional don't throw an error if not found but perform default handling /// If marked as optional don't throw an error if not found but perform default handling
/// (e.g. return `Value::Nothing`) /// (e.g. return `Value::Nothing`)
optional: bool, optional: bool,
/// Affects column lookup
casing: Casing,
}, },
/// Accessing a member by index (i.e. row of a table or item in a list) /// Accessing a member by index (i.e. row of a table or item in a list)
Int { Int {
@ -34,11 +36,12 @@ impl PathMember {
} }
} }
pub fn string(val: String, optional: bool, span: Span) -> Self { pub fn string(val: String, optional: bool, casing: Casing, span: Span) -> Self {
PathMember::String { PathMember::String {
val, val,
span, span,
optional, optional,
casing,
} }
} }
@ -50,10 +53,11 @@ impl PathMember {
} }
} }
pub fn test_string(val: String, optional: bool) -> Self { pub fn test_string(val: String, optional: bool, casing: Casing) -> Self {
PathMember::String { PathMember::String {
val, val,
optional, optional,
casing,
span: Span::test_data(), span: Span::test_data(),
} }
} }
@ -65,6 +69,13 @@ impl PathMember {
} }
} }
pub fn make_insensitive(&mut self) {
match self {
PathMember::String { casing, .. } => *casing = Casing::Insensitive,
PathMember::Int { .. } => {}
}
}
pub fn span(&self) -> Span { pub fn span(&self) -> Span {
match self { match self {
PathMember::String { span, .. } => *span, PathMember::String { span, .. } => *span,
@ -178,6 +189,12 @@ impl CellPath {
} }
} }
pub fn make_insensitive(&mut self) {
for member in &mut self.members {
member.make_insensitive();
}
}
// Formats the cell-path as a column name, i.e. without quoting and optional markers ('?'). // Formats the cell-path as a column name, i.e. without quoting and optional markers ('?').
pub fn to_column_name(&self) -> String { pub fn to_column_name(&self) -> String {
let mut s = String::new(); let mut s = String::new();
@ -209,17 +226,31 @@ impl Display for CellPath {
let question_mark = if *optional { "?" } else { "" }; let question_mark = if *optional { "?" } else { "" };
write!(f, ".{val}{question_mark}")? write!(f, ".{val}{question_mark}")?
} }
PathMember::String { val, optional, .. } => { PathMember::String {
val,
optional,
casing,
..
} => {
let question_mark = if *optional { "?" } else { "" }; let question_mark = if *optional { "?" } else { "" };
let exclamation_mark = if *casing == Casing::Insensitive {
"!"
} else {
""
};
let val = if needs_quoting(val) { let val = if needs_quoting(val) {
&escape_quote_string(val) &escape_quote_string(val)
} else { } else {
val val
}; };
write!(f, ".{val}{question_mark}")? write!(f, ".{val}{exclamation_mark}{question_mark}")?
} }
} }
} }
// Empty cell-paths are `$.` not `$`
if self.members.is_empty() {
write!(f, ".")?;
}
Ok(()) Ok(())
} }
} }
@ -239,7 +270,11 @@ mod test {
fn path_member_partial_ord() { fn path_member_partial_ord() {
assert_eq!( assert_eq!(
Some(Greater), Some(Greater),
PathMember::test_int(5, true).partial_cmp(&PathMember::test_string("e".into(), true)) PathMember::test_int(5, true).partial_cmp(&PathMember::test_string(
"e".into(),
true,
Casing::Sensitive
))
); );
assert_eq!( assert_eq!(
@ -254,14 +289,16 @@ mod test {
assert_eq!( assert_eq!(
Some(Greater), Some(Greater),
PathMember::test_string("e".into(), true) PathMember::test_string("e".into(), true, Casing::Sensitive).partial_cmp(
.partial_cmp(&PathMember::test_string("e".into(), false)) &PathMember::test_string("e".into(), false, Casing::Sensitive)
)
); );
assert_eq!( assert_eq!(
Some(Greater), Some(Greater),
PathMember::test_string("f".into(), true) PathMember::test_string("f".into(), true, Casing::Sensitive).partial_cmp(
.partial_cmp(&PathMember::test_string("e".into(), true)) &PathMember::test_string("e".into(), true, Casing::Sensitive)
)
); );
} }
} }

View File

@ -0,0 +1,8 @@
use serde::{Deserialize, Serialize};
#[derive(Clone, Copy, PartialEq, Eq, Debug, Default, Serialize, Deserialize)]
pub enum Casing {
#[default]
Sensitive,
Insensitive,
}

View File

@ -43,8 +43,16 @@ pub trait Eval {
// Cell paths are usually case-sensitive, but we give $env // Cell paths are usually case-sensitive, but we give $env
// special treatment. // special treatment.
let insensitive = cell_path.head.expr == Expr::Var(ENV_VARIABLE_ID); let tail = if cell_path.head.expr == Expr::Var(ENV_VARIABLE_ID) {
value.follow_cell_path(&cell_path.tail, insensitive).map(Cow::into_owned) let mut tail = cell_path.tail.clone();
if let Some(pm) = tail.first_mut() {
pm.make_insensitive();
}
Cow::Owned(tail)
} else {
Cow::Borrowed(&cell_path.tail)
};
value.follow_cell_path(&tail).map(Cow::into_owned)
} }
Expr::DateTime(dt) => Ok(Value::date(*dt, expr_span)), Expr::DateTime(dt) => Ok(Value::date(*dt, expr_span)),
Expr::List(list) => { Expr::List(list) => {

View File

@ -2,6 +2,7 @@
#![doc = include_str!("../README.md")] #![doc = include_str!("../README.md")]
mod alias; mod alias;
pub mod ast; pub mod ast;
pub mod casing;
pub mod config; pub mod config;
pub mod debugger; pub mod debugger;
mod did_you_mean; mod did_you_mean;

View File

@ -411,16 +411,13 @@ impl PipelineData {
self, self,
cell_path: &[PathMember], cell_path: &[PathMember],
head: Span, head: Span,
insensitive: bool,
) -> Result<Value, ShellError> { ) -> Result<Value, ShellError> {
match self { match self {
// FIXME: there are probably better ways of doing this // FIXME: there are probably better ways of doing this
PipelineData::ListStream(stream, ..) => Value::list(stream.into_iter().collect(), head) PipelineData::ListStream(stream, ..) => Value::list(stream.into_iter().collect(), head)
.follow_cell_path(cell_path, insensitive) .follow_cell_path(cell_path)
.map(Cow::into_owned),
PipelineData::Value(v, ..) => v
.follow_cell_path(cell_path, insensitive)
.map(Cow::into_owned), .map(Cow::into_owned),
PipelineData::Value(v, ..) => v.follow_cell_path(cell_path).map(Cow::into_owned),
PipelineData::Empty => Err(ShellError::IncompatiblePathAccess { PipelineData::Empty => Err(ShellError::IncompatiblePathAccess {
type_name: "empty pipeline".to_string(), type_name: "empty pipeline".to_string(),
span: head, span: head,

View File

@ -1,6 +1,7 @@
use crate::{ use crate::{
NuGlob, Range, Record, ShellError, Span, Spanned, Type, Value, NuGlob, Range, Record, ShellError, Span, Spanned, Type, Value,
ast::{CellPath, PathMember}, ast::{CellPath, PathMember},
casing::Casing,
engine::Closure, engine::Closure,
}; };
use chrono::{DateTime, FixedOffset}; use chrono::{DateTime, FixedOffset};
@ -585,6 +586,7 @@ impl FromValue for CellPath {
val, val,
span, span,
optional: false, optional: false,
casing: Casing::Sensitive,
}], }],
}), }),
Value::Int { val, .. } => { Value::Int { val, .. } => {

View File

@ -29,7 +29,7 @@ use chrono::{DateTime, Datelike, Duration, FixedOffset, Local, Locale, TimeZone}
use chrono_humanize::HumanTime; use chrono_humanize::HumanTime;
use fancy_regex::Regex; use fancy_regex::Regex;
use nu_utils::{ use nu_utils::{
IgnoreCaseExt, SharedCow, contains_emoji, SharedCow, contains_emoji,
locale::{LOCALE_OVERRIDE_ENV_VAR, get_system_locale_string}, locale::{LOCALE_OVERRIDE_ENV_VAR, get_system_locale_string},
}; };
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@ -1082,7 +1082,6 @@ impl Value {
pub fn follow_cell_path<'out>( pub fn follow_cell_path<'out>(
&'out self, &'out self,
cell_path: &[PathMember], cell_path: &[PathMember],
insensitive: bool,
) -> Result<Cow<'out, Value>, ShellError> { ) -> Result<Cow<'out, Value>, ShellError> {
enum MultiLife<'out, 'local, T> enum MultiLife<'out, 'local, T>
where where
@ -1115,7 +1114,7 @@ impl Value {
for member in cell_path { for member in cell_path {
current = match current { current = match current {
MultiLife::Out(current) => match get_value_member(current, member, insensitive)? { MultiLife::Out(current) => match get_value_member(current, member)? {
ControlFlow::Break(span) => return Ok(Cow::Owned(Value::nothing(span))), ControlFlow::Break(span) => return Ok(Cow::Owned(Value::nothing(span))),
ControlFlow::Continue(x) => match x { ControlFlow::Continue(x) => match x {
Cow::Borrowed(x) => MultiLife::Out(x), Cow::Borrowed(x) => MultiLife::Out(x),
@ -1125,18 +1124,16 @@ impl Value {
} }
}, },
}, },
MultiLife::Local(current) => { MultiLife::Local(current) => match get_value_member(current, member)? {
match get_value_member(current, member, insensitive)? { ControlFlow::Break(span) => return Ok(Cow::Owned(Value::nothing(span))),
ControlFlow::Break(span) => return Ok(Cow::Owned(Value::nothing(span))), ControlFlow::Continue(x) => match x {
ControlFlow::Continue(x) => match x { Cow::Borrowed(x) => MultiLife::Local(x),
Cow::Borrowed(x) => MultiLife::Local(x), Cow::Owned(x) => {
Cow::Owned(x) => { store = x;
store = x; MultiLife::Local(&store)
MultiLife::Local(&store) }
} },
}, },
}
}
}; };
} }
@ -1165,7 +1162,7 @@ impl Value {
cell_path: &[PathMember], cell_path: &[PathMember],
callback: Box<dyn FnOnce(&Value) -> Value>, callback: Box<dyn FnOnce(&Value) -> Value>,
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let new_val = callback(self.follow_cell_path(cell_path, false)?.as_ref()); let new_val = callback(self.follow_cell_path(cell_path)?.as_ref());
match new_val { match new_val {
Value::Error { error, .. } => Err(*error), Value::Error { error, .. } => Err(*error),
@ -1184,6 +1181,7 @@ impl Value {
PathMember::String { PathMember::String {
val: col_name, val: col_name,
span, span,
casing,
.. ..
} => match self { } => match self {
Value::List { vals, .. } => { Value::List { vals, .. } => {
@ -1191,7 +1189,7 @@ impl Value {
match val { match val {
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
let record = record.to_mut(); let record = record.to_mut();
if let Some(val) = record.get_mut(col_name) { if let Some(val) = record.cased_mut(*casing).get_mut(col_name) {
val.upsert_data_at_cell_path(path, new_val.clone())?; val.upsert_data_at_cell_path(path, new_val.clone())?;
} else { } else {
let new_col = let new_col =
@ -1212,7 +1210,7 @@ impl Value {
} }
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
let record = record.to_mut(); let record = record.to_mut();
if let Some(val) = record.get_mut(col_name) { if let Some(val) = record.cased_mut(*casing).get_mut(col_name) {
val.upsert_data_at_cell_path(path, new_val)?; val.upsert_data_at_cell_path(path, new_val)?;
} else { } else {
let new_col = Value::with_data_at_cell_path(path, new_val.clone())?; let new_col = Value::with_data_at_cell_path(path, new_val.clone())?;
@ -1265,7 +1263,7 @@ impl Value {
cell_path: &[PathMember], cell_path: &[PathMember],
callback: Box<dyn FnOnce(&Value) -> Value + 'a>, callback: Box<dyn FnOnce(&Value) -> Value + 'a>,
) -> Result<(), ShellError> { ) -> Result<(), ShellError> {
let new_val = callback(self.follow_cell_path(cell_path, false)?.as_ref()); let new_val = callback(self.follow_cell_path(cell_path)?.as_ref());
match new_val { match new_val {
Value::Error { error, .. } => Err(*error), Value::Error { error, .. } => Err(*error),
@ -1284,6 +1282,7 @@ impl Value {
PathMember::String { PathMember::String {
val: col_name, val: col_name,
span, span,
casing,
.. ..
} => match self { } => match self {
Value::List { vals, .. } => { Value::List { vals, .. } => {
@ -1291,7 +1290,9 @@ impl Value {
let v_span = val.span(); let v_span = val.span();
match val { match val {
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
if let Some(val) = record.to_mut().get_mut(col_name) { if let Some(val) =
record.to_mut().cased_mut(*casing).get_mut(col_name)
{
val.update_data_at_cell_path(path, new_val.clone())?; val.update_data_at_cell_path(path, new_val.clone())?;
} else { } else {
return Err(ShellError::CantFindColumn { return Err(ShellError::CantFindColumn {
@ -1313,7 +1314,7 @@ impl Value {
} }
} }
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
if let Some(val) = record.to_mut().get_mut(col_name) { if let Some(val) = record.to_mut().cased_mut(*casing).get_mut(col_name) {
val.update_data_at_cell_path(path, new_val)?; val.update_data_at_cell_path(path, new_val)?;
} else { } else {
return Err(ShellError::CantFindColumn { return Err(ShellError::CantFindColumn {
@ -1372,13 +1373,16 @@ impl Value {
val: col_name, val: col_name,
span, span,
optional, optional,
casing,
} => match self { } => match self {
Value::List { vals, .. } => { Value::List { vals, .. } => {
for val in vals.iter_mut() { for val in vals.iter_mut() {
let v_span = val.span(); let v_span = val.span();
match val { match val {
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
if record.to_mut().remove(col_name).is_none() && !optional { let value =
record.to_mut().cased_mut(*casing).remove(col_name);
if value.is_none() && !optional {
return Err(ShellError::CantFindColumn { return Err(ShellError::CantFindColumn {
col_name: col_name.clone(), col_name: col_name.clone(),
span: Some(*span), span: Some(*span),
@ -1398,7 +1402,13 @@ impl Value {
Ok(()) Ok(())
} }
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
if record.to_mut().remove(col_name).is_none() && !optional { if record
.to_mut()
.cased_mut(*casing)
.remove(col_name)
.is_none()
&& !optional
{
return Err(ShellError::CantFindColumn { return Err(ShellError::CantFindColumn {
col_name: col_name.clone(), col_name: col_name.clone(),
span: Some(*span), span: Some(*span),
@ -1447,13 +1457,16 @@ impl Value {
val: col_name, val: col_name,
span, span,
optional, optional,
casing,
} => match self { } => match self {
Value::List { vals, .. } => { Value::List { vals, .. } => {
for val in vals.iter_mut() { for val in vals.iter_mut() {
let v_span = val.span(); let v_span = val.span();
match val { match val {
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
if let Some(val) = record.to_mut().get_mut(col_name) { let val =
record.to_mut().cased_mut(*casing).get_mut(col_name);
if let Some(val) = val {
val.remove_data_at_cell_path(path)?; val.remove_data_at_cell_path(path)?;
} else if !optional { } else if !optional {
return Err(ShellError::CantFindColumn { return Err(ShellError::CantFindColumn {
@ -1475,7 +1488,8 @@ impl Value {
Ok(()) Ok(())
} }
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
if let Some(val) = record.to_mut().get_mut(col_name) { if let Some(val) = record.to_mut().cased_mut(*casing).get_mut(col_name)
{
val.remove_data_at_cell_path(path)?; val.remove_data_at_cell_path(path)?;
} else if !optional { } else if !optional {
return Err(ShellError::CantFindColumn { return Err(ShellError::CantFindColumn {
@ -1532,6 +1546,7 @@ impl Value {
PathMember::String { PathMember::String {
val: col_name, val: col_name,
span, span,
casing,
.. ..
} => match self { } => match self {
Value::List { vals, .. } => { Value::List { vals, .. } => {
@ -1540,7 +1555,7 @@ impl Value {
match val { match val {
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
let record = record.to_mut(); let record = record.to_mut();
if let Some(val) = record.get_mut(col_name) { if let Some(val) = record.cased_mut(*casing).get_mut(col_name) {
if path.is_empty() { if path.is_empty() {
return Err(ShellError::ColumnAlreadyExists { return Err(ShellError::ColumnAlreadyExists {
col_name: col_name.clone(), col_name: col_name.clone(),
@ -1574,7 +1589,7 @@ impl Value {
} }
Value::Record { val: record, .. } => { Value::Record { val: record, .. } => {
let record = record.to_mut(); let record = record.to_mut();
if let Some(val) = record.get_mut(col_name) { if let Some(val) = record.cased_mut(*casing).get_mut(col_name) {
if path.is_empty() { if path.is_empty() {
return Err(ShellError::ColumnAlreadyExists { return Err(ShellError::ColumnAlreadyExists {
col_name: col_name.clone(), col_name: col_name.clone(),
@ -2004,7 +2019,6 @@ impl Value {
fn get_value_member<'a>( fn get_value_member<'a>(
current: &'a Value, current: &'a Value,
member: &PathMember, member: &PathMember,
insensitive: bool,
) -> Result<ControlFlow<Span, Cow<'a, Value>>, ShellError> { ) -> Result<ControlFlow<Span, Cow<'a, Value>>, ShellError> {
match member { match member {
PathMember::Int { PathMember::Int {
@ -2091,18 +2105,14 @@ fn get_value_member<'a>(
val: column_name, val: column_name,
span: origin_span, span: origin_span,
optional, optional,
casing,
} => { } => {
let span = current.span(); let span = current.span();
match current { match current {
Value::Record { val, .. } => { Value::Record { val, .. } => {
if let Some(found) = val.iter().rev().find(|x| { let found = val.cased(*casing).get(column_name);
if insensitive { if let Some(found) = found {
x.0.eq_ignore_case(column_name) Ok(ControlFlow::Continue(Cow::Borrowed(found)))
} else {
x.0 == column_name
}
}) {
Ok(ControlFlow::Continue(Cow::Borrowed(found.1)))
} else if *optional { } else if *optional {
Ok(ControlFlow::Break(*origin_span)) Ok(ControlFlow::Break(*origin_span))
// short-circuit // short-circuit
@ -2129,14 +2139,9 @@ fn get_value_member<'a>(
let val_span = val.span(); let val_span = val.span();
match val { match val {
Value::Record { val, .. } => { Value::Record { val, .. } => {
if let Some(found) = val.iter().rev().find(|x| { let found = val.cased(*casing).get(column_name);
if insensitive { if let Some(found) = found {
x.0.eq_ignore_case(column_name) Ok(found.clone())
} else {
x.0 == column_name
}
}) {
Ok(found.1.clone())
} else if *optional { } else if *optional {
Ok(Value::nothing(*origin_span)) Ok(Value::nothing(*origin_span))
} else if let Some(suggestion) = } else if let Some(suggestion) =
@ -4089,6 +4094,8 @@ mod tests {
use crate::record; use crate::record;
mod at_cell_path { mod at_cell_path {
use crate::casing::Casing;
use crate::{IntoValue, Span}; use crate::{IntoValue, Span};
use super::super::PathMember; use super::super::PathMember;
@ -4101,10 +4108,10 @@ mod tests {
assert_eq!( assert_eq!(
Value::with_data_at_cell_path( Value::with_data_at_cell_path(
&[ &[
PathMember::test_string("a".to_string(), false), PathMember::test_string("a".to_string(), false, Casing::Sensitive),
PathMember::test_string("b".to_string(), false), PathMember::test_string("b".to_string(), false, Casing::Sensitive),
PathMember::test_string("c".to_string(), false), PathMember::test_string("c".to_string(), false, Casing::Sensitive),
PathMember::test_string("d".to_string(), false), PathMember::test_string("d".to_string(), false, Casing::Sensitive),
], ],
value_to_insert, value_to_insert,
), ),
@ -4148,13 +4155,13 @@ mod tests {
assert_eq!( assert_eq!(
Value::with_data_at_cell_path( Value::with_data_at_cell_path(
&[ &[
PathMember::test_string("a".to_string(), false), PathMember::test_string("a".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
PathMember::test_string("b".to_string(), false), PathMember::test_string("b".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
PathMember::test_string("c".to_string(), false), PathMember::test_string("c".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
PathMember::test_string("d".to_string(), false), PathMember::test_string("d".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
], ],
value_to_insert.clone(), value_to_insert.clone(),
@ -4184,9 +4191,9 @@ mod tests {
let value_to_insert = Value::test_string("value"); let value_to_insert = Value::test_string("value");
let res = base_value.upsert_data_at_cell_path( let res = base_value.upsert_data_at_cell_path(
&[ &[
PathMember::test_string("a".to_string(), false), PathMember::test_string("a".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
PathMember::test_string("b".to_string(), false), PathMember::test_string("b".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
], ],
value_to_insert.clone(), value_to_insert.clone(),
@ -4218,9 +4225,9 @@ mod tests {
let value_to_insert = Value::test_string("value"); let value_to_insert = Value::test_string("value");
let res = base_value.insert_data_at_cell_path( let res = base_value.insert_data_at_cell_path(
&[ &[
PathMember::test_string("a".to_string(), false), PathMember::test_string("a".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
PathMember::test_string("b".to_string(), false), PathMember::test_string("b".to_string(), false, Casing::Sensitive),
PathMember::test_int(0, false), PathMember::test_int(0, false),
], ],
value_to_insert.clone(), value_to_insert.clone(),

View File

@ -1,8 +1,9 @@
//! Our insertion ordered map-type [`Record`] //! Our insertion ordered map-type [`Record`]
use std::{iter::FusedIterator, ops::RangeBounds}; use std::{iter::FusedIterator, ops::RangeBounds};
use crate::{ShellError, Span, Value}; use crate::{ShellError, Span, Value, casing::Casing};
use nu_utils::IgnoreCaseExt;
use serde::{Deserialize, Serialize, de::Visitor, ser::SerializeMap}; use serde::{Deserialize, Serialize, de::Visitor, ser::SerializeMap};
#[derive(Debug, Clone, Default)] #[derive(Debug, Clone, Default)]
@ -10,6 +11,62 @@ pub struct Record {
inner: Vec<(String, Value)>, inner: Vec<(String, Value)>,
} }
/// A wrapper around [`Record`] that affects whether key comparisons are case sensitive or not.
///
/// Implements commonly used methods of [`Record`].
pub struct CasedRecord<R> {
record: R,
casing: Casing,
}
impl<R> CasedRecord<R> {
fn cmp(&self, lhs: &str, rhs: &str) -> bool {
match self.casing {
Casing::Sensitive => lhs == rhs,
Casing::Insensitive => lhs.eq_ignore_case(rhs),
}
}
}
impl<'a> CasedRecord<&'a Record> {
pub fn contains(&self, col: impl AsRef<str>) -> bool {
self.record.columns().any(|k| self.cmp(k, col.as_ref()))
}
pub fn index_of(&self, col: impl AsRef<str>) -> Option<usize> {
self.record
.columns()
.rposition(|k| self.cmp(k, col.as_ref()))
}
pub fn get(self, col: impl AsRef<str>) -> Option<&'a Value> {
let idx = self.index_of(col)?;
let (_, value) = self.record.get_index(idx)?;
Some(value)
}
}
impl<'a> CasedRecord<&'a mut Record> {
fn shared(&'a self) -> CasedRecord<&'a Record> {
CasedRecord {
record: &*self.record,
casing: self.casing,
}
}
pub fn get_mut(self, col: impl AsRef<str>) -> Option<&'a mut Value> {
let idx = self.shared().index_of(col)?;
let (_, value) = self.record.get_index_mut(idx)?;
Some(value)
}
pub fn remove(&mut self, col: impl AsRef<str>) -> Option<Value> {
let idx = self.shared().index_of(col)?;
let (_, val) = self.record.inner.remove(idx);
Some(val)
}
}
impl Record { impl Record {
pub fn new() -> Self { pub fn new() -> Self {
Self::default() Self::default()
@ -21,6 +78,20 @@ impl Record {
} }
} }
pub fn cased(&self, casing: Casing) -> CasedRecord<&Record> {
CasedRecord {
record: self,
casing,
}
}
pub fn cased_mut(&mut self, casing: Casing) -> CasedRecord<&mut Record> {
CasedRecord {
record: self,
casing,
}
}
/// Create a [`Record`] from a `Vec` of columns and a `Vec` of [`Value`]s /// Create a [`Record`] from a `Vec` of columns and a `Vec` of [`Value`]s
/// ///
/// Returns an error if `cols` and `vals` have different lengths. /// Returns an error if `cols` and `vals` have different lengths.
@ -108,6 +179,12 @@ impl Record {
self.inner.get(idx).map(|(col, val): &(_, _)| (col, val)) self.inner.get(idx).map(|(col, val): &(_, _)| (col, val))
} }
pub fn get_index_mut(&mut self, idx: usize) -> Option<(&mut String, &mut Value)> {
self.inner
.get_mut(idx)
.map(|(col, val): &mut (_, _)| (col, val))
}
/// Remove single value by key /// Remove single value by key
/// ///
/// Returns `None` if key not found /// Returns `None` if key not found

View File

@ -2,12 +2,12 @@ use fancy_regex::Regex;
use std::sync::LazyLock; use std::sync::LazyLock;
// This hits, in order: // This hits, in order:
// • Any character of []:`{}#'";()|$,. // • Any character of []:`{}#'";()|$,.!?
// • Any digit (\d) // • Any digit (\d)
// • Any whitespace (\s) // • Any whitespace (\s)
// • Case-insensitive sign-insensitive float "keywords" inf, infinity and nan. // • Case-insensitive sign-insensitive float "keywords" inf, infinity and nan.
static NEEDS_QUOTING_REGEX: LazyLock<Regex> = LazyLock::new(|| { static NEEDS_QUOTING_REGEX: LazyLock<Regex> = LazyLock::new(|| {
Regex::new(r#"[\[\]:`\{\}#'";\(\)\|\$,\.\d\s]|(?i)^[+\-]?(inf(inity)?|nan)$"#) Regex::new(r#"[\[\]:`\{\}#'";\(\)\|\$,\.\d\s!?]|(?i)^[+\-]?(inf(inity)?|nan)$"#)
.expect("internal error: NEEDS_QUOTING_REGEX didn't compile") .expect("internal error: NEEDS_QUOTING_REGEX didn't compile")
}); });

View File

@ -91,7 +91,7 @@ impl Inc {
pub fn inc(&self, head: Span, value: &Value) -> Result<Value, LabeledError> { pub fn inc(&self, head: Span, value: &Value) -> Result<Value, LabeledError> {
if let Some(cell_path) = &self.cell_path { if let Some(cell_path) = &self.cell_path {
let cell_value = value.follow_cell_path(&cell_path.members, false)?; let cell_value = value.follow_cell_path(&cell_path.members)?;
let cell_value = self.inc_value(head, &cell_value)?; let cell_value = self.inc_value(head, &cell_value)?;

View File

@ -12,6 +12,7 @@ mod tests {
use nu_protocol::{ use nu_protocol::{
BlockId, IntRange, Range, Span, Value, BlockId, IntRange, Range, Span, Value,
ast::{CellPath, PathMember, RangeInclusion}, ast::{CellPath, PathMember, RangeInclusion},
casing::Casing,
engine::{Closure, EngineState}, engine::{Closure, EngineState},
record, record,
}; };
@ -401,8 +402,18 @@ mod tests {
r#"$.foo.bar.0"#, r#"$.foo.bar.0"#,
Some(Value::test_cell_path(CellPath { Some(Value::test_cell_path(CellPath {
members: vec![ members: vec![
PathMember::string("foo".to_string(), false, Span::new(2, 5)), PathMember::string(
PathMember::string("bar".to_string(), false, Span::new(6, 9)), "foo".to_string(),
false,
Casing::Sensitive,
Span::new(2, 5),
),
PathMember::string(
"bar".to_string(),
false,
Casing::Sensitive,
Span::new(6, 9),
),
PathMember::int(0, false, Span::new(10, 11)), PathMember::int(0, false, Span::new(10, 11)),
], ],
})), })),

View File

@ -307,7 +307,7 @@ fn nullify_holes() -> TestResult {
#[test] #[test]
fn get_insensitive() -> TestResult { fn get_insensitive() -> TestResult {
run_test( run_test(
r#"[[name, age]; [a, 1] [b, 2]] | get NAmE | select 0 | get 0"#, r#"[[name, age]; [a, 1] [b, 2]] | get NAmE! | select 0 | get 0"#,
"a", "a",
) )
} }