liukun4515 commented on PR #2360:
URL: https://github.com/apache/arrow-rs/pull/2360#issuecomment-1210093333
> > > refactor the validation for decimal256 @viirya
> > > almost 50x faster than before. You can get the benchmark code in the
`decimal_validate.rs` file.
> >
> >
> > Yea, I think for 256-bit decimal, current value validation is somehow
inefficient. Comparing directly on bytes should be more efficient on 256-bit
decimal. For 128-bit, I think that it won't be too much different than i128
comparison, maybe some marginal number.
>
> agree with your thought about 128-bit, but you miss a point. When we
validate the decimal128array with 128-bit/i128, we should convert the [u8;16]
to i128 first with the struct`decimal128`. This will cause some unnecessary
overhead.
>
> @HaoYang670 @viirya I think this can explains why the validation of
decimal128array got the performance improvement
I add some benchmarks for above explanation.
Got the result:
```
validate_decimal128_bytes 2000
time: [46.438 us 46.965 us 47.578 us]
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
validate_decimal128_i128 2000
time: [69.251 us 69.591 us 69.959 us]
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
4 (4.00%) high mild
1 (1.00%) high severe
```
The performance using the bytes is better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
