mirror of
https://github.com/nushell/nushell.git
synced 2025-07-09 19:07:16 +02:00
22d1fdcdf6
- improves rounding error reported in #15851 - my ~~discussion~~ monologue on how filesizes are parsed currently: #15944 # Description The issue linked above reported rounding errors when converting MiB to GiB, which is mainly caused by parsing of the literal. Nushell tries to convert all filesize values to bytes, but currently does so in 2 steps: - first converting it to the next smaller unit in `nu-parser` (so `MiB` to `KiB`, in this case), and truncating to an `i64` here - then converting that to bytes in `nu-engine`, again truncating to `i64` In the specific example above (`95307.27MiB`), this causes 419 bytes of rounding error. By instead directly converting to bytes while parsing, the value is accurate (truncating those 0.52 bytes, or 4.12 bits). Rounding error in the conversion to GiB is also multiple magnitudes lower. (Note that I haven't thoroughly tested this, so I can't say with confidence that all values would be parsed accurate to the byte.) # User-Facing Changes More accurate filesize values, and lower accumulated rounding error in calculations. # Tests + Formatting new test: `parse_filesize` in `nu-parser` - verifies that `95307.27MiB` is parsed correctly as `99_936_915_947B` # After Submitting