Gabriel39 commented on code in PR #67784:
URL: https://github.com/apache/doris/pull/67784#discussion_r3987517221
##########
be/src/format/transformer/vparquet_transformer.cpp:
##########
@@ -249,8 +253,10 @@ Status VParquetTransformer::_parse_schema() {
} else {
for (size_t i = 0; i < _output_vexpr_ctxs.size(); i++) {
std::shared_ptr<arrow::DataType> type;
+ // DATETIMEV2 is a wall-clock value. Giving its Arrow schema the
session timezone would
+ // turn it into an instant and shift it when a Parquet file is
read in another zone.
RETURN_IF_ERROR(convert_to_arrow_type(_output_vexpr_ctxs[i]->root()->data_type(),
&type,
- _state->timezone()));
+ _state->timezone(),
/*datetime_naive=*/true));
Review Comment:
Fixed by matching Trino’s Hive INT96 semantics. When
`enable_int96_timestamps` is enabled, DATETIMEV2 now retains the writer
timezone so the local civil value is normalized to the correct UTC instant
before INT96 encoding. The default INT64 path remains timezone-naive
(`isAdjustedToUTC=false`). I also added an encoded-value read-back unit test
and made the Hive round-trip regression explicitly use FileScanner V2; V1
behavior is outside this PR’s updated scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
