mbutrovich opened a new issue, #11601:
URL: https://github.com/apache/arrow-rs/issues/11601
### Describe the bug
`cast` from `Timestamp` to `Date64` converts the value to milliseconds and
stops there. The result keeps the time of day, so it is not a whole number of
days, and it uses the UTC date where the cast to `Date32` uses the local date
in the timestamp's time zone.
### To Reproduce
```rust
use arrow_array::cast::AsArray;
use arrow_array::types::{Date32Type, Date64Type};
use arrow_array::{Array, TimestampSecondArray};
use arrow_cast::cast;
use arrow_schema::DataType;
fn main() {
// 1970-01-01T00:00:01Z and 1970-01-01T20:00:00Z, which are 1969-12-31
and 1970-01-01 in -05:00
let ts = TimestampSecondArray::from(vec![1,
72_000]).with_timezone("-05:00");
let d64 = cast(&ts, &DataType::Date64).unwrap();
let d32 = cast(&ts, &DataType::Date32).unwrap();
println!("Date64: {:?}", d64.as_primitive::<Date64Type>().values());
// [1000, 72000000]: not whole days, and the time zone is not applied
println!("Date32: {:?}", d32.as_primitive::<Date32Type>().values());
// [-1, 0]: the local dates
}
```
### Expected behavior
`[-86400000, 0]`, the same dates as the `Date32` cast, in milliseconds.
### Additional context
The
[format](https://github.com/apache/arrow/blob/fcf8f3014227cafe24deeb044c2d51f5fc9902e5/format/Schema.fbs?plain=1#L246-L251)
defines `Date64` as milliseconds "evenly divisible by 86400000". The cast docs
say precision is lost going from `Timestamp` to `Date{32|64}`
([mod.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-cast/src/cast/mod.rs#L652)).
The `Date64` arms convert units only
([mod.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-cast/src/cast/mod.rs#L1945-L1968)),
while the `Date32` arms go through `timestamp_to_date32`, which applies the
time zone
([mod.rs](https://github.com/apache/arrow-rs/blob/72887ed91c80002bb7ec755ce1e964bab07bebea/arrow-cast/src/cast/mod.rs#L1930-L1944)).
#5288 settled that arrow-rs does not always validate that `Date64` values
are whole days. Here the cast itself produces the partial days.
Found with property-based tests that build arrays in every physical layout
the Arrow format allows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
