nushell/tests/repl/test_table_operations.rs
Bahex c4dcfdb77b
feat!: Explicit cell-path case sensitivity syntax (#15692)
Related:
- #15683
- #14551
- #849
- #12701
- #11527

# Description
Currently various commands have differing behavior regarding cell-paths

```nushell
{a: 1, A: 2} | get a A
# => ╭───┬───╮
# => │ 0 │ 2 │
# => │ 1 │ 2 │
# => ╰───┴───╯
{a: 1, A: 2} | select a A
# => ╭───┬───╮
# => │ a │ 1 │
# => │ A │ 2 │
# => ╰───┴───╯
{A: 1} | update a 2
# => Error: nu:🐚:column_not_found
# => 
# =>   × Cannot find column 'a'
# =>    ╭─[entry #62:1:1]
# =>  1 │ {A: 1} | update a 2
# =>    · ───┬──          ┬
# =>    ·    │            ╰── cannot find column 'a'
# =>    ·    ╰── value originates here
# =>    ╰────
```

Proposal: making cell-path access case-sensitive by default and adding
new syntax for case-insensitive parts, similar to optional (?) parts.

```nushell
{FOO: BAR}.foo
# => Error: nu:🐚:name_not_found
# => 
# =>   × Name not found
# =>    ╭─[entry #60:1:21]
# =>  1 │ {FOO: BAR}.foo
# =>    ·            ─┬─
# =>    ·             ╰── did you mean 'FOO'?
# =>    ╰────
{FOO: BAR}.foo!
# => BAR
```

This would solve the problem of case sensitivity for all commands
without causing an explosion of flags _and_ make it more granular

Assigning to a field using a case-insensitive path is case-preserving.
```nushell
mut val = {FOO: "I'm FOO"}; $val
# => ╭─────┬─────────╮
# => │ FOO │ I'm FOO │
# => ╰─────┴─────────╯
$val.foo! = "I'm still FOO"; $val
# => ╭─────┬───────────────╮
# => │ FOO │ I'm still FOO │
# => ╰─────┴───────────────╯
```

For `update`, case-insensitive is case-preserving.
```nushell
{FOO: 1} | update foo! { $in + 1 }
# => ╭─────┬───╮
# => │ FOO │ 2 │
# => ╰─────┴───╯
```

`insert` can insert values into nested values so accessing into existing
columns is case-insensitive, but creating new columns uses the cell-path
as it is.
So `insert foo! ...` and `insert FOO! ...` would work exactly as they do
without `!`
```nushell
{FOO: {quox: 0}}
# => ╭─────┬──────────────╮
# => │     │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │     │ ╰──────┴───╯ │
# => ╰─────┴──────────────╯
{FOO: {quox: 0}} | insert foo.bar 1
# => ╭─────┬──────────────╮
# => │     │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │     │ ╰──────┴───╯ │
# => │     │ ╭─────┬───╮  │
# => │ foo │ │ bar │ 1 │  │
# => │     │ ╰─────┴───╯  │
# => ╰─────┴──────────────╯
{FOO: {quox: 0}} | insert foo!.bar 1
# => ╭─────┬──────────────╮
# => │     │ ╭──────┬───╮ │
# => │ FOO │ │ quox │ 0 │ │
# => │     │ │ bar  │ 1 │ │
# => │     │ ╰──────┴───╯ │
# => ╰─────┴──────────────╯
```

`upsert` is tricky, depending on the input, the data might end up with
different column names in rows. We can either forbid case-insensitive
cell-paths for `upsert` or trust the user to keep their data in a
sensible shape.

This would be a breaking change as it would make existing cell-path
accesses case-sensitive, however the case-sensitivity is already
inconsistent and any attempt at making it consistent would be a breaking
change.

> What about `$env`?

1. Initially special case it so it keeps its current behavior.
2. Accessing environment variables with non-matching paths gives a
deprecation warning urging users to either use exact casing or use the
new explicit case-sensitivity syntax
3. Eventuall remove `$env`'s special case, making `$env` accesses
case-sensitive by default as well.

> `$env.ENV_CONVERSIONS`?

In addition to `from_string` and `to_string` add an optional field to
opt into case insensitive/preserving behavior.

# User-Facing Changes

- `get`, `where` and other previously case-insensitive commands are now
case-sensitive by default.
- `get`'s `--sensitive` flag removed, similar to `--ignore-errors` there
is now an `--ignore-case` flag that treats all parts of the cell-path as
case-insensitive.
- Users can explicitly choose the case case-sensitivity of cell-path
accesses or commands.

# Tests + Formatting

Existing tests required minimal modification. ***However, new tests are
not yet added***.

- 🟢 toolkit fmt
- 🟢 toolkit clippy
- 🟢 toolkit test
- 🟢 toolkit test stdlib

# After Submitting

- Update the website to include the new syntax
- Update [tree-sitter-nu](https://github.com/nushell/tree-sitter-nu)

---------

Co-authored-by: Bahex <17417311+Bahex@users.noreply.github.com>
2025-05-18 12:19:09 +03:00

314 lines
6.5 KiB
Rust

use crate::repl::tests::{TestResult, fail_test, run_test};
#[test]
fn illegal_column_duplication() -> TestResult {
fail_test("[[lang, lang]; [nu, 100]]", "column_defined_twice")
}
#[test]
fn cell_path_subexpr1() -> TestResult {
run_test("([[lang, gems]; [nu, 100]]).lang | get 0", "nu")
}
#[test]
fn cell_path_subexpr2() -> TestResult {
run_test("([[lang, gems]; [nu, 100]]).lang.0", "nu")
}
#[test]
fn cell_path_var1() -> TestResult {
run_test("let x = [[lang, gems]; [nu, 100]]; $x.lang | get 0", "nu")
}
#[test]
fn cell_path_var2() -> TestResult {
run_test("let x = [[lang, gems]; [nu, 100]]; $x.lang.0", "nu")
}
#[test]
fn flatten_simple_list() -> TestResult {
run_test(
"[[N, u, s, h, e, l, l]] | flatten | str join (char nl)",
"N\nu\ns\nh\ne\nl\nl",
)
}
#[test]
fn flatten_get_simple_list() -> TestResult {
run_test("[[N, u, s, h, e, l, l]] | flatten | get 0", "N")
}
#[test]
fn flatten_table_get() -> TestResult {
run_test(
"[[origin, people]; [Ecuador, ([[name, meal]; ['Andres', 'arepa']])]] | flatten --all | get meal.0",
"arepa",
)
}
#[test]
fn flatten_table_column_get_last() -> TestResult {
run_test(
"[[origin, crate, versions]; [World, ([[name]; ['nu-cli']]), ['0.21', '0.22']]] | flatten versions --all | last | get versions",
"0.22",
)
}
#[test]
fn flatten_should_just_flatten_one_level() -> TestResult {
run_test(
"[[origin, crate, versions]; [World, ([[name]; ['nu-cli']]), ['0.21', '0.22']]] | flatten crate | get crate.name.0",
"nu-cli",
)
}
#[test]
fn flatten_nest_table_when_all_provided() -> TestResult {
run_test(
"[[origin, crate, versions]; [World, ([[name]; ['nu-cli']]), ['0.21', '0.22']]] | flatten crate --all | get name.0",
"nu-cli",
)
}
#[test]
fn get_table_columns_1() -> TestResult {
run_test(
"[[name, age, grade]; [paul,21,a]] | columns | first",
"name",
)
}
#[test]
fn get_table_columns_2() -> TestResult {
run_test("[[name, age, grade]; [paul,21,a]] | columns | get 1", "age")
}
#[test]
fn flatten_should_flatten_inner_table() -> TestResult {
run_test(
"[[[name, value]; [abc, 123]]] | flatten --all | get value.0",
"123",
)
}
#[test]
fn command_filter_reject_1() -> TestResult {
run_test(
"[[lang, gems]; [nu, 100]] | reject gems | to json",
r#"[
{
"lang": "nu"
}
]"#,
)
}
#[test]
fn command_filter_reject_2() -> TestResult {
run_test(
"[[lang, gems, grade]; [nu, 100, a]] | reject gems grade | to json",
r#"[
{
"lang": "nu"
}
]"#,
)
}
#[test]
fn command_filter_reject_3() -> TestResult {
run_test(
"[[lang, gems, grade]; [nu, 100, a]] | reject grade gems | to json",
r#"[
{
"lang": "nu"
}
]"#,
)
}
#[test]
#[rustfmt::skip]
fn command_filter_reject_4() -> TestResult {
run_test(
"[[lang, gems, grade]; [nu, 100, a]] | reject gems | to json -r",
r#"[{"lang":"nu","grade":"a"}]"#,
)
}
#[test]
fn command_drop_column_1() -> TestResult {
run_test(
"[[lang, gems, grade]; [nu, 100, a]] | drop column 2 | to json",
r#"[
{
"lang": "nu"
}
]"#,
)
}
#[test]
fn record_1() -> TestResult {
run_test(r#"{'a': 'b'} | get a"#, "b")
}
#[test]
fn record_2() -> TestResult {
run_test(r#"{'b': 'c'}.b"#, "c")
}
#[test]
fn where_on_ranges() -> TestResult {
run_test(r#"1..10 | where $it > 8 | math sum"#, "19")
}
#[test]
fn index_on_list() -> TestResult {
run_test(r#"[1, 2, 3].1"#, "2")
}
#[test]
fn update_cell_path_1() -> TestResult {
run_test(
r#"[[name, size]; [a, 1.1]] | into int size | get size.0"#,
"1",
)
}
#[test]
fn missing_column_errors() -> TestResult {
fail_test(
r#"[ { name: ABC, size: 20 }, { name: HIJ } ].size.1 == null"#,
"cannot find column",
)
}
#[test]
fn missing_optional_column_fills_in_nothing() -> TestResult {
// The empty value will be replaced with null because of the ?
run_test(
r#"[ { name: ABC, size: 20 }, { name: HIJ } ].size?.1 == null"#,
"true",
)
}
#[test]
fn missing_required_row_fails() -> TestResult {
// .3 will fail if there is no 3rd row
fail_test(
r#"[ { name: ABC, size: 20 }, { name: HIJ } ].3"#,
"", // we just care if it errors
)
}
#[test]
fn missing_optional_row_fills_in_nothing() -> TestResult {
// ?.3 will return null if there is no 3rd row
run_test(
r#"[ { name: ABC, size: 20 }, { name: HIJ } ].3? == null"#,
"true",
)
}
#[test]
fn string_cell_path() -> TestResult {
run_test(
r#"let x = "name"; [["name", "score"]; [a, b], [c, d]] | get $x | get 1"#,
"c",
)
}
#[test]
fn split_row() -> TestResult {
run_test(r#""hello world" | split row " " | get 1"#, "world")
}
#[test]
fn split_column() -> TestResult {
run_test(
r#""hello world" | split column " " | get "column1".0"#,
"hello",
)
}
#[test]
fn wrap() -> TestResult {
run_test(r#"([1, 2, 3] | wrap foo).foo.1"#, "2")
}
#[test]
fn get() -> TestResult {
run_test(
r#"[[name, grade]; [Alice, A], [Betty, B]] | get grade.1"#,
"B",
)
}
#[test]
fn select_1() -> TestResult {
run_test(
r#"([[name, age]; [a, 1], [b, 2]]) | select name | get 1 | get name"#,
"b",
)
}
#[test]
fn select_2() -> TestResult {
run_test(
r#"[[name, age]; [a, 1] [b, 2]] | get 1 | select age | get age"#,
"2",
)
}
#[test]
fn update_will_insert() -> TestResult {
run_test(r#"{} | upsert a b | get a"#, "b")
}
#[test]
fn length_for_columns() -> TestResult {
run_test(
r#"[[name,age,grade]; [bill,20,a] [a b c]] | columns | length"#,
"3",
)
}
#[test]
fn length_for_rows() -> TestResult {
run_test(r#"[[name,age,grade]; [bill,20,a] [a b c]] | length"#, "2")
}
#[test]
fn length_defaulted_columns() -> TestResult {
run_test(
r#"[[name, age]; [test, 10]] | default 11 age | get 0 | columns | length"#,
"2",
)
}
#[test]
fn nullify_errors() -> TestResult {
run_test("([{a:1} {a:2} {a:3}] | get foo? | length) == 3", "true")?;
run_test(
"([{a:1} {a:2} {a:3}] | get foo? | to nuon) == '[null, null, null]'",
"true",
)
}
#[test]
fn nullify_holes() -> TestResult {
run_test(
"([{a:1} {b:2} {a:3}] | get a? | to nuon) == '[1, null, 3]'",
"true",
)
}
#[test]
fn get_insensitive() -> TestResult {
run_test(
r#"[[name, age]; [a, 1] [b, 2]] | get NAmE! | select 0 | get 0"#,
"a",
)
}