Feat: construct datetime from record (#15455)

Issue #12289, can be closed when this is merged

# Description
Currently, the ``into datetime`` command's signature indicates that it
supports input as record, but it was actually not supported.

This PR implements this feature.

# User-Facing Changes

``into datetime``'s signature changed (see comments)

**Happy paths**

Note: I'm in +02:00 timezone.

```nushell
> date now | into record | into datetime
Fri, 4 Apr 2025 18:32:34 +0200 (now)

> {year: 2025, month: 12, day: 6, second: 59} | into datetime | into record
╭─────────────┬────────╮
│ year        │ 2025   │
│ month       │ 12     │
│ day         │ 6      │
│ hour        │ 0      │
│ minute      │ 0      │
│ second      │ 59     │
│ millisecond │ 0      │
│ microsecond │ 0      │
│ nanosecond  │ 0      │
│ timezone    │ +02:00 │
╰─────────────┴────────╯

> {day: 6, second: 59, timezone: '-06:00'} | into datetime | into record
╭─────────────┬────────╮
│ year        │ 2025   │
│ month       │ 4      │
│ day         │ 6      │
│ hour        │ 0      │
│ minute      │ 0      │
│ second      │ 59     │
│ millisecond │ 0      │
│ microsecond │ 0      │
│ nanosecond  │ 0      │
│ timezone    │ -06:00 │
╰─────────────┴────────╯
```

**Edge cases**

```nushell
{} | into datetime
Fri, 4 Apr 2025 18:35:19 +0200 (now)
```

**Error paths**

- A key has a wrong type
  ```nushell
  > {month: 12, year: '2023'} | into datetime
  Error: nu:🐚:only_supports_this_input_type

    × Input type not supported.
    ╭─[entry #8:1:19]
  1 │ {month: 12, year: '2023'} | into datetime
    ·                   ───┬──    ──────┬──────
· │ ╰── only int input data is supported
    ·                      ╰── input type: string
    ╰────
  ```
  ```nushell
  > {month: 12, year: 2023, timezone: 100} | into datetime
  Error: nu:🐚:only_supports_this_input_type

    × Input type not supported.
    ╭─[entry #10:1:35]
  1 │ {month: 12, year: 2023, timezone: 100} | into datetime
    ·                                   ─┬─    ──────┬──────
· │ ╰── only string input data is supported
    ·                                    ╰── input type: int
    ╰────
  ```
- Key has the right type but value invalid (e.g. month=13, or day=0)
  ```nushell
  > {month: 13, year: 2023} | into datetime
  Error: nu:🐚:incorrect_value

    × Incorrect value.
    ╭─[entry #9:1:1]
  1 │ {month: 13, year: 2023} | into datetime
    · ───────────┬───────────   ──────┬──────
· │ ╰── one of more values are incorrect and do not represent valid date
    ·            ╰── encountered here
    ╰────
  ```
  ```nushell
  > {hour: 1, minute: 1, second: 70} | into datetime
  Error: nu:🐚:incorrect_value
  
    × Incorrect value.
     ╭─[entry #3:1:1]
   1 │ {hour: 1, minute: 1, second: 70} | into datetime
     · ────────────────┬───────────────   ──────┬──────
· │ ╰── one of more values are incorrect and do not represent valid time
     ·                 ╰── encountered here
     ╰────
  ```
- Timezone has right type but is invalid
  ```nushell
  > {month: 12, year: 2023, timezone: "+100:00"} | into datetime
  Error: nu:🐚:incorrect_value

    × Incorrect value.
    ╭─[entry #11:1:35]
  1 │ {month: 12, year: 2023, timezone: "+100:00"} | into datetime
    ·                                   ────┬────    ──────┬──────
· │ ╰── encountered here
    ·                                       ╰── invalid timezone
    ╰────
  ```
- Record contains an invalid key
  ```nushell
  > {month: 12, year: 2023, unknown: 1} | into datetime
  Error: nu:🐚:unsupported_input

    × Unsupported input
    ╭─[entry #12:1:1]
  1 │ {month: 12, year: 2023, unknown: 1} | into datetime
    · ─────────────────┬─────────────────   ──────┬──────
· │ ╰── Column 'unknown' is not valid for a structured datetime. Allowed
columns are: year, month, day, hour, minute, second, millisecond,
microsecond, nanosecond, timezone
    ·                  ╰── value originates from here
    ╰────
  ```
- If several issues are present, the user can get the error msg for only
one, though
  ```nushell
  > {month: 20, year: '2023'} | into datetime
  Error: nu:🐚:only_supports_this_input_type

    × Input type not supported.
    ╭─[entry #7:1:19]
  1 │ {month: 20, year: '2023'} | into datetime
    ·                   ───┬──    ──────┬──────
· │ ╰── only int input data is supported
    ·                      ╰── input type: string
    ╰
  ```


# Tests + Formatting
Tests added
Fmt + clippy OK

# After Submitting
Maybe indicate that in the release notes
I added an example in the command, so the documentation will be
automatically updated.
This commit is contained in:
Loïc Riegel 2025-04-10 15:33:06 +02:00 committed by GitHub
parent 29eb109b1e
commit dfca117551
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 485 additions and 15 deletions

View File

@ -1,14 +1,29 @@
use crate::{generate_strftime_list, parse_date_from_string};
use chrono::{DateTime, FixedOffset, Local, NaiveDateTime, TimeZone, Utc};
use chrono::{
DateTime, Datelike, FixedOffset, Local, NaiveDate, NaiveDateTime, NaiveTime, TimeZone,
Timelike, Utc,
};
use nu_cmd_base::input_handler::{operate, CmdArgument};
use nu_engine::command_prelude::*;
const HOUR: i32 = 60 * 60;
const ALLOWED_COLUMNS: [&str; 10] = [
"year",
"month",
"day",
"hour",
"minute",
"second",
"millisecond",
"microsecond",
"nanosecond",
"timezone",
];
#[derive(Clone, Debug)]
struct Arguments {
zone_options: Option<Spanned<Zone>>,
format_options: Option<DatetimeFormat>,
format_options: Option<Spanned<DatetimeFormat>>,
cell_paths: Option<Vec<CellPath>>,
}
@ -66,8 +81,12 @@ impl Command for IntoDatetime {
(Type::String, Type::Date),
(Type::List(Box::new(Type::String)), Type::List(Box::new(Type::Date))),
(Type::table(), Type::table()),
(Type::record(), Type::record()),
(Type::Nothing, Type::table()),
// FIXME: https://github.com/nushell/nushell/issues/15485
// 'record -> any' was added as a temporary workaround to avoid type inference issues. The Any arm needs to be appear first.
(Type::record(), Type::Any),
(Type::record(), Type::record()),
(Type::record(), Type::Date),
// FIXME Type::Any input added to disable pipeline input type checking, as run-time checks can raise undesirable type errors
// which aren't caught by the parser. see https://github.com/nushell/nushell/pull/14922 for more details
// only applicable for --list flag
@ -133,13 +152,16 @@ impl Command for IntoDatetime {
};
let format_options = call
.get_flag::<String>(engine_state, stack, "format")?
.get_flag::<Spanned<String>>(engine_state, stack, "format")?
.as_ref()
.map(|fmt| DatetimeFormat(fmt.to_string()));
.map(|fmt| Spanned {
item: DatetimeFormat(fmt.item.to_string()),
span: fmt.span,
});
let args = Arguments {
format_options,
zone_options,
format_options,
cell_paths,
};
operate(action, args, input, call.head, engine_state.signals())
@ -215,6 +237,12 @@ impl Command for IntoDatetime {
#[allow(clippy::inconsistent_digit_grouping)]
result: example_result_1(1614434140_000000000),
},
Example {
description: "Using a record as input",
example: "{year: 2025, month: 3, day: 30, hour: 12, minute: 15, second: 59, timezone: '+02:00'} | into datetime",
#[allow(clippy::inconsistent_digit_grouping)]
result: example_result_1(1743329759_000000000),
},
Example {
description: "Convert list of timestamps to datetimes",
example: r#"["2023-03-30 10:10:07 -05:00", "2023-05-05 13:43:49 -05:00", "2023-06-05 01:37:42 -05:00"] | into datetime"#,
@ -264,6 +292,34 @@ fn action(input: &Value, args: &Arguments, head: Span) -> Value {
return input.clone();
}
if let Value::Record { val: record, .. } = input {
if let Some(tz) = timezone {
return Value::error(
ShellError::IncompatibleParameters {
left_message: "got a record as input".into(),
left_span: head,
right_message: "the timezone should be included in the record".into(),
right_span: tz.span,
},
head,
);
}
if let Some(dt) = dateformat {
return Value::error(
ShellError::IncompatibleParameters {
left_message: "got a record as input".into(),
left_span: head,
right_message: "cannot be used with records".into(),
right_span: dt.span,
},
head,
);
}
return merge_record(record, head, input.span());
}
// Let's try dtparse first
if matches!(input, Value::String { .. }) && dateformat.is_none() {
let span = input.span();
@ -353,7 +409,7 @@ fn action(input: &Value, args: &Arguments, head: Span) -> Value {
let parse_as_string = |val: &str| {
match dateformat {
Some(dt_format) => match DateTime::parse_from_str(val, &dt_format.0) {
Some(dt_format) => match DateTime::parse_from_str(val, &dt_format.item.0) {
Ok(dt) => {
match timezone {
None => {
@ -402,7 +458,7 @@ fn action(input: &Value, args: &Arguments, head: Span) -> Value {
}
},
Err(reason) => {
match NaiveDateTime::parse_from_str(val, &dt_format.0) {
match NaiveDateTime::parse_from_str(val, &dt_format.item.0) {
Ok(d) => {
let dt_fixed =
Local.from_local_datetime(&d).single().unwrap_or_default();
@ -411,7 +467,7 @@ fn action(input: &Value, args: &Arguments, head: Span) -> Value {
}
Err(_) => {
Value::error (
ShellError::CantConvert { to_type: format!("could not parse as datetime using format '{}'", dt_format.0), from_type: reason.to_string(), span: head, help: Some("you can use `into datetime` without a format string to enable flexible parsing".to_string()) },
ShellError::CantConvert { to_type: format!("could not parse as datetime using format '{}'", dt_format.item.0), from_type: reason.to_string(), span: head, help: Some("you can use `into datetime` without a format string to enable flexible parsing".to_string()) },
head,
)
}
@ -450,6 +506,308 @@ fn action(input: &Value, args: &Arguments, head: Span) -> Value {
}
}
fn merge_record(record: &Record, head: Span, span: Span) -> Value {
if let Some(invalid_col) = record
.columns()
.find(|key| !ALLOWED_COLUMNS.contains(&key.as_str()))
{
let allowed_cols = ALLOWED_COLUMNS.join(", ");
return Value::error(ShellError::UnsupportedInput {
msg: format!(
"Column '{invalid_col}' is not valid for a structured datetime. Allowed columns are: {allowed_cols}"
),
input: "value originates from here".into(),
msg_span: head,
input_span: span
},
span,
);
};
// Empty fields are filled in a specific way: the time units bigger than the biggest provided fields are assumed to be current and smaller ones are zeroed.
// And local timezone is used if not provided.
#[derive(Debug)]
enum RecordColumnDefault {
Now,
Zero,
}
let mut record_column_default = RecordColumnDefault::Now;
let now = Local::now();
let mut now_nanosecond = now.nanosecond();
let now_millisecond = now_nanosecond / 1_000_000;
now_nanosecond %= 1_000_000;
let now_microsecond = now_nanosecond / 1_000;
now_nanosecond %= 1_000;
let year: i32 = match record.get("year") {
Some(val) => {
record_column_default = RecordColumnDefault::Zero;
match val {
Value::Int { val, .. } => *val as i32,
other => {
return Value::error(
ShellError::OnlySupportsThisInputType {
exp_input_type: "int".to_string(),
wrong_type: other.get_type().to_string(),
dst_span: head,
src_span: other.span(),
},
span,
);
}
}
}
None => now.year(),
};
let month = match record.get("month") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("month", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now.month(),
RecordColumnDefault::Zero => 1,
},
};
let day = match record.get("day") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("day", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now.day(),
RecordColumnDefault::Zero => 1,
},
};
let hour = match record.get("hour") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("hour", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now.hour(),
RecordColumnDefault::Zero => 0,
},
};
let minute = match record.get("minute") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("minute", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now.minute(),
RecordColumnDefault::Zero => 0,
},
};
let second = match record.get("second") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("second", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now.second(),
RecordColumnDefault::Zero => 0,
},
};
let millisecond = match record.get("millisecond") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("millisecond", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now_millisecond,
RecordColumnDefault::Zero => 0,
},
};
let microsecond = match record.get("microsecond") {
Some(col_val) => {
record_column_default = RecordColumnDefault::Zero;
match parse_value_from_record_as_u32("microsecond", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now_microsecond,
RecordColumnDefault::Zero => 0,
},
};
let nanosecond = match record.get("nanosecond") {
Some(col_val) => {
match parse_value_from_record_as_u32("nanosecond", col_val, &head, &span) {
Ok(value) => value,
Err(err) => {
return err;
}
}
}
None => match record_column_default {
RecordColumnDefault::Now => now_nanosecond,
RecordColumnDefault::Zero => 0,
},
};
let offset: FixedOffset = match record.get("timezone") {
Some(timezone) => match parse_timezone_from_record(timezone, &head, &timezone.span()) {
Ok(value) => value,
Err(err) => {
return err;
}
},
None => now.offset().to_owned(),
};
let total_nanoseconds = nanosecond + microsecond * 1_000 + millisecond * 1_000_000;
let date = match NaiveDate::from_ymd_opt(year, month, day) {
Some(d) => d,
None => {
return Value::error(
ShellError::IncorrectValue {
msg: "one of more values are incorrect and do not represent valid date"
.to_string(),
val_span: head,
call_span: span,
},
span,
)
}
};
let time = match NaiveTime::from_hms_nano_opt(hour, minute, second, total_nanoseconds) {
Some(t) => t,
None => {
return Value::error(
ShellError::IncorrectValue {
msg: "one of more values are incorrect and do not represent valid time"
.to_string(),
val_span: head,
call_span: span,
},
span,
)
}
};
let date_time = NaiveDateTime::new(date, time);
let date_time_fixed = match offset.from_local_datetime(&date_time).single() {
Some(d) => d,
None => {
return Value::error(
ShellError::IncorrectValue {
msg: "Ambiguous or invalid timezone conversion".to_string(),
val_span: head,
call_span: span,
},
span,
)
}
};
Value::date(date_time_fixed, span)
}
fn parse_value_from_record_as_u32(
col: &str,
col_val: &Value,
head: &Span,
span: &Span,
) -> Result<u32, Value> {
let value: u32 = match col_val {
Value::Int { val, .. } => {
if *val < 0 || *val > u32::MAX as i64 {
return Err(Value::error(
ShellError::IncorrectValue {
msg: format!("incorrect value for {}", col),
val_span: *head,
call_span: *span,
},
*span,
));
}
*val as u32
}
other => {
return Err(Value::error(
ShellError::OnlySupportsThisInputType {
exp_input_type: "int".to_string(),
wrong_type: other.get_type().to_string(),
dst_span: *head,
src_span: other.span(),
},
*span,
));
}
};
Ok(value)
}
fn parse_timezone_from_record(
timezone: &Value,
head: &Span,
span: &Span,
) -> Result<FixedOffset, Value> {
match timezone {
Value::String { val, .. } => {
let offset: FixedOffset = match val.parse() {
Ok(offset) => offset,
Err(_) => {
return Err(Value::error(
ShellError::IncorrectValue {
msg: "invalid timezone".to_string(),
val_span: *span,
call_span: *head,
},
*span,
))
}
};
Ok(offset)
}
other => Err(Value::error(
ShellError::OnlySupportsThisInputType {
exp_input_type: "string".to_string(),
wrong_type: other.get_type().to_string(),
dst_span: *head,
src_span: other.span(),
},
*span,
)),
}
}
#[cfg(test)]
mod tests {
use super::*;
@ -466,7 +824,10 @@ mod tests {
#[test]
fn takes_a_date_format_with_timezone() {
let date_str = Value::test_string("16.11.1984 8:00 am +0000");
let fmt_options = Some(DatetimeFormat("%d.%m.%Y %H:%M %P %z".to_string()));
let fmt_options = Some(Spanned {
item: DatetimeFormat("%d.%m.%Y %H:%M %P %z".to_string()),
span: Span::test_data(),
});
let args = Arguments {
zone_options: None,
format_options: fmt_options,
@ -483,7 +844,10 @@ mod tests {
#[test]
fn takes_a_date_format_without_timezone() {
let date_str = Value::test_string("16.11.1984 8:00 am");
let fmt_options = Some(DatetimeFormat("%d.%m.%Y %H:%M %P".to_string()));
let fmt_options = Some(Spanned {
item: DatetimeFormat("%d.%m.%Y %H:%M %P".to_string()),
span: Span::test_data(),
});
let args = Arguments {
zone_options: None,
format_options: fmt_options,
@ -565,7 +929,10 @@ mod tests {
#[test]
fn takes_int_with_formatstring() {
let date_int = Value::test_int(1_614_434_140);
let fmt_options = Some(DatetimeFormat("%s".to_string()));
let fmt_options = Some(Spanned {
item: DatetimeFormat("%s".to_string()),
span: Span::test_data(),
});
let args = Arguments {
zone_options: None,
format_options: fmt_options,
@ -587,7 +954,10 @@ mod tests {
item: Zone::East(8),
span: Span::test_data(),
});
let fmt_options = Some(DatetimeFormat("%s".to_string()));
let fmt_options = Some(Spanned {
item: DatetimeFormat("%s".to_string()),
span: Span::test_data(),
});
let args = Arguments {
zone_options: timezone_option,
format_options: fmt_options,
@ -609,7 +979,10 @@ mod tests {
item: Zone::Local,
span: Span::test_data(),
});
let fmt_options = Some(DatetimeFormat("%s".to_string()));
let fmt_options = Some(Spanned {
item: DatetimeFormat("%s".to_string()),
span: Span::test_data(),
});
let args = Arguments {
zone_options: timezone_option,
format_options: fmt_options,
@ -685,7 +1058,10 @@ mod tests {
#[test]
fn communicates_parsing_error_given_an_invalid_datetimelike_string() {
let date_str = Value::test_string("16.11.1984 8:00 am Oops0000");
let fmt_options = Some(DatetimeFormat("%d.%m.%Y %H:%M %P %z".to_string()));
let fmt_options = Some(Spanned {
item: DatetimeFormat("%d.%m.%Y %H:%M %P %z".to_string()),
span: Span::test_data(),
});
let args = Arguments {
zone_options: None,
format_options: fmt_options,

View File

@ -1,8 +1,102 @@
use nu_test_support::nu;
// Tests happy paths
#[test]
fn into_datetime_from_record() {
let actual = nu!(
r#"{year: 2023, month: 1, day: 2, hour: 3, minute: 4, second: 5, millisecond: 6, microsecond: 7, nanosecond: 8, timezone: '+01:00'} | into datetime | into record"#
);
let expected = nu!(
r#"{year: 2023, month: 1, day: 2, hour: 3, minute: 4, second: 5, millisecond: 6, microsecond: 7, nanosecond: 8, timezone: '+01:00'}"#
);
assert_eq!(expected.out, actual.out);
}
#[test]
fn into_datetime_from_record_defaults() {
let actual = nu!(r#"{year: 2025, timezone: '+02:00'} | into datetime | into record"#);
let expected = nu!(
r#"{year: 2025, month: 1, day: 1, hour: 0, minute: 0, second: 0, millisecond: 0, microsecond: 0, nanosecond: 0, timezone: '+02:00'}"#
);
assert_eq!(expected.out, actual.out);
}
#[test]
fn into_datetime_from_record_round_trip() {
let actual = nu!(
r#"(1743348798 | into datetime | into record | into datetime | into int) == 1743348798"#
);
assert!(actual.out.contains("true"));
}
#[test]
fn into_datetime_table_column() {
let actual = nu!(r#"[[date]; ["2022-01-01"] ["2023-01-01"]] | into datetime date"#);
assert!(actual.out.contains(" ago"));
}
// Tests error paths
#[test]
fn into_datetime_from_record_fails_with_wrong_type() {
let actual = nu!(r#"{year: '2023'} | into datetime"#);
assert!(actual
.err
.contains("nu::shell::only_supports_this_input_type"));
}
#[test]
fn into_datetime_from_record_fails_with_invalid_date_time_values() {
let actual = nu!(r#"{year: 2023, month: 13} | into datetime"#);
assert!(actual.err.contains("nu::shell::incorrect_value"));
}
#[test]
fn into_datetime_from_record_fails_with_invalid_timezone() {
let actual = nu!(r#"{year: 2023, timezone: '+100:00'} | into datetime"#);
assert!(actual.err.contains("nu::shell::incorrect_value"));
}
// Tests invalid usage
#[test]
fn into_datetime_from_record_fails_with_unknown_key() {
let actual = nu!(r#"{year: 2023, unknown: 1} | into datetime"#);
assert!(actual.err.contains("nu::shell::unsupported_input"));
}
#[test]
fn into_datetime_from_record_incompatible_with_format_flag() {
let actual = nu!(
r#"{year: 2023, month: 1, day: 2, hour: 3, minute: 4, second: 5} | into datetime --format ''"#
);
assert!(actual.err.contains("nu::shell::incompatible_parameters"));
}
#[test]
fn into_datetime_from_record_incompatible_with_timezone_flag() {
let actual = nu!(
r#"{year: 2023, month: 1, day: 2, hour: 3, minute: 4, second: 5} | into datetime --timezone UTC"#
);
assert!(actual.err.contains("nu::shell::incompatible_parameters"));
}
#[test]
fn into_datetime_from_record_incompatible_with_offset_flag() {
let actual = nu!(
r#"{year: 2023, month: 1, day: 2, hour: 3, minute: 4, second: 5} | into datetime --offset 1"#
);
assert!(actual.err.contains("nu::shell::incompatible_parameters"));
}