neilconway opened a new pull request, #11199:
URL: https://github.com/apache/arrow-rs/pull/11199
# Which issue does this PR close?
- N/A
# Rationale for this change
Casting from timestamp to another datetime type currently goes through
Chrono. When casting timestamp values without a timezone to `Date32`, `Time32`,
or `Time64`, we can avoid going through Chrono and do the cast directly from
the stored integer timestamp value. This is similar to the approach taken in
#11187; it improves performance by about 10-20x.
Benchmarks: (Arm64)
- timestamp_s/date32/no_nulls: 87.078 → 4.213 µs, −95.2%
- timestamp_s/date32/mixed_nulls: 72.433 → 5.617 µs, −92.2%
- timestamp_s/date32/timezone_control: 145.015 → 145.380 µs, +0.3%
- timestamp_s/time32_s/no_nulls: 66.189 → 2.308 µs, −96.5%
- timestamp_s/time32_s/mixed_nulls: 55.035 → 2.316 µs, −95.8%
- timestamp_s/time32_s/timezone_control: 76.010 → 73.405 µs, −3.4%
- timestamp_s/time32_ms/no_nulls: 68.525 → 2.371 µs, −96.5%
- timestamp_s/time32_ms/mixed_nulls: 56.806 → 2.374 µs, −95.8%
- timestamp_s/time32_ms/timezone_control: 79.450 → 77.596 µs, −2.3%
- timestamp_s/time64_us/no_nulls: 69.051 → 4.585 µs, −93.4%
- timestamp_s/time64_us/mixed_nulls: 58.651 → 4.548 µs, −92.2%
- timestamp_s/time64_us/timezone_control: 79.975 → 79.428 µs, −0.7%
- timestamp_s/time64_ns/no_nulls: 67.930 → 4.456 µs, −93.4%
- timestamp_s/time64_ns/mixed_nulls: 57.641 → 4.522 µs, −92.2%
- timestamp_s/time64_ns/timezone_control: 78.689 → 77.041 µs, −2.1%
- timestamp_ms/date32/no_nulls: 101.283 → 4.152 µs, −95.9%
- timestamp_ms/date32/mixed_nulls: 83.949 → 5.880 µs, −93.0%
- timestamp_ms/date32/timezone_control: 165.457 → 164.299 µs, −0.7%
- timestamp_ms/time32_s/no_nulls: 78.389 → 2.652 µs, −96.6%
- timestamp_ms/time32_s/mixed_nulls: 64.086 → 2.641 µs, −95.9%
- timestamp_ms/time32_s/timezone_control: 89.451 → 87.082 µs, −2.6%
- timestamp_ms/time32_ms/no_nulls: 80.910 → 2.325 µs, −97.1%
- timestamp_ms/time32_ms/mixed_nulls: 66.337 → 2.323 µs, −96.5%
- timestamp_ms/time32_ms/timezone_control: 92.393 → 91.689 µs, −0.8%
- timestamp_ms/time64_us/no_nulls: 81.425 → 4.647 µs, −94.3%
- timestamp_ms/time64_us/mixed_nulls: 67.533 → 4.561 µs, −93.2%
- timestamp_ms/time64_us/timezone_control: 94.523 → 93.327 µs, −1.3%
- timestamp_ms/time64_ns/no_nulls: 80.386 → 4.668 µs, −94.2%
- timestamp_ms/time64_ns/mixed_nulls: 66.684 → 4.623 µs, −93.1%
- timestamp_ms/time64_ns/timezone_control: 92.500 → 90.079 µs, −2.6%
- timestamp_us/date32/no_nulls: 101.470 → 2.433 µs, −97.6%
- timestamp_us/date32/mixed_nulls: 83.837 → 2.435 µs, −97.1%
- timestamp_us/date32/timezone_control: 162.027 → 161.063 µs, −0.6%
- timestamp_us/time32_s/no_nulls: 78.709 → 4.159 µs, −94.7%
- timestamp_us/time32_s/mixed_nulls: 64.217 → 4.287 µs, −93.3%
- timestamp_us/time32_s/timezone_control: 86.706 → 86.476 µs, −0.3%
- timestamp_us/time32_ms/no_nulls: 79.663 → 5.017 µs, −93.7%
- timestamp_us/time32_ms/mixed_nulls: 65.707 → 4.888 µs, −92.6%
- timestamp_us/time32_ms/timezone_control: 88.219 → 90.175 µs, +2.2%
- timestamp_us/time64_us/no_nulls: 81.027 → 2.397 µs, −97.0%
- timestamp_us/time64_us/mixed_nulls: 67.051 → 2.377 µs, −96.5%
- timestamp_us/time64_us/timezone_control: 89.425 → 91.079 µs, +1.8%
- timestamp_us/time64_ns/no_nulls: 79.922 → 4.487 µs, −94.4%
- timestamp_us/time64_ns/mixed_nulls: 66.862 → 4.489 µs, −93.3%
- timestamp_us/time64_ns/timezone_control: 88.451 → 89.621 µs, +1.3%
- timestamp_ns/date32/no_nulls: 100.969 → 2.419 µs, −97.6%
- timestamp_ns/date32/mixed_nulls: 83.965 → 2.417 µs, −97.1%
- timestamp_ns/date32/timezone_control: 161.721 → 161.486 µs, −0.1%
- timestamp_ns/time32_s/no_nulls: 78.099 → 4.314 µs, −94.5%
- timestamp_ns/time32_s/mixed_nulls: 63.725 → 4.315 µs, −93.2%
- timestamp_ns/time32_s/timezone_control: 85.657 → 86.544 µs, +1.0%
- timestamp_ns/time32_ms/no_nulls: 80.054 → 4.606 µs, −94.2%
- timestamp_ns/time32_ms/mixed_nulls: 65.776 → 4.399 µs, −93.3%
- timestamp_ns/time32_ms/timezone_control: 87.615 → 90.203 µs, +3.0%
- timestamp_ns/time64_us/no_nulls: 80.365 → 5.315 µs, −93.4%
- timestamp_ns/time64_us/mixed_nulls: 66.501 → 5.205 µs, −92.2%
- timestamp_ns/time64_us/timezone_control: 89.166 → 91.184 µs, +2.3%
- timestamp_ns/time64_ns/no_nulls: 79.216 → 2.371 µs, −97.0%
- timestamp_ns/time64_ns/mixed_nulls: 65.746 → 2.377 µs, −96.4%
- timestamp_ns/time64_ns/timezone_control: 88.115 → 89.139 µs, +1.2%
# What changes are included in this PR?
* Optimize casting timestamp without timezone to time-of-day /
days-since-epoch
* Add tests ensuring that optimized code path preserves the same behavior as
the Chrono code path
* Add benchmarks
# Are these changes tested?
Yes; existing tests pass, new tests added.
# Are there any user-facing changes?
Previously, attempting to cast a timestamp value that was outside Chrono's
supported calendar range produced an error. Such casts will now succeed,
provided that we take the optimized code path (the timestamp does not have a
timezone and the target of the cast is `Date32`, `Time32`, or `Time64`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
