holm/nushell
1
1
Fork 0
forked from extern/nushell
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Stop nu panicks in math.round on a large decimal value(Most of the time) (#3224)

Browse Source 
* Stop crashing when dealing with large numbers in math round

* Fix formatting

* add tests

* just to trigger wasm build

* trigger wasm build
This commit is contained in:
Luccas Mateus
2021-03-31 03:01:39 -03:00
committed by GitHub
parent c42b588782
commit 387098fc87
3 changed files with 27 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
crates/nu-command/src/commands/math/round.rs
View File

@ -76,9 +76,12 @@ fn round_big_int(val: BigInt) -> Value {
fn round_big_decimal(val: BigDecimal, precision: i64) -> Value {
if precision > 0 {
UntaggedValue::decimal(val.round(precision)).into()
UntaggedValue::decimal(val.with_scale(precision + 1).round(precision)).into()
} else {
let (rounded, _) = val.round(precision).as_bigint_and_exponent();
let (rounded, _) = val
.with_scale(precision + 1)
.round(precision)
.as_bigint_and_exponent();
UntaggedValue::int(rounded).into()
}
}