github-actions[bot] commented on code in PR #67784:
URL: https://github.com/apache/doris/pull/67784#discussion_r3986879829
##########
be/src/core/data_type_serde/data_type_serde.cpp:
##########
@@ -496,8 +528,33 @@ Status decode_timestamp_orc_values(IColumn& nested_column,
const OrcDecodedColum
}
auto& value =
reinterpret_cast<DateV2Value<DateTimeV2ValueType>&>(data[old_data_size + row]);
- value.from_unixtime(orc_batch->data[source_row], timezone);
-
value.set_microsecond(cast_set<uint64_t>(orc_batch->nanoseconds[source_row] /
1000));
+ RoundedOrcTimestamp timestamp;
+ auto status = round_orc_timestamp_to_microseconds(
Review Comment:
[P1] Apply this rounding contract to the legacy ORC scanner too. This helper
is only used by the V2 reader, but `enable_file_scanner_v2=false` still routes
ordinary ORC queries through `vorc_reader.h`, and all load scans are forced
there. That decoder keeps the original second and assigns `nanoseconds / 1000`,
so `05.999999500` is `06.000000` here but `05.999999` in V1 (the TIMESTAMPTZ
path has the same truncation). Please share the checked half-up/carry
conversion with V1 or force the affected scans onto one implementation, and
cover both scanner settings plus the load path.
##########
be/src/format/transformer/vparquet_transformer.cpp:
##########
@@ -249,8 +253,10 @@ Status VParquetTransformer::_parse_schema() {
} else {
for (size_t i = 0; i < _output_vexpr_ctxs.size(); i++) {
std::shared_ptr<arrow::DataType> type;
+ // DATETIMEV2 is a wall-clock value. Giving its Arrow schema the
session timezone would
+ // turn it into an instant and shift it when a Parquet file is
read in another zone.
RETURN_IF_ERROR(convert_to_arrow_type(_output_vexpr_ctxs[i]->root()->data_type(),
&type,
- _state->timezone()));
+ _state->timezone(),
/*datetime_naive=*/true));
Review Comment:
[P1] Keep DATETIMEV2 timezone-aware when this writer is producing legacy
INT96. The unconditional naive schema makes `write_column_to_arrow` interpret
civil values in UTC, but forced INT96 files are still read by applying
`hive.parquet.time-zone`. For example, an Asia/Shanghai value `2023-04-20
00:00` is now stored as the `00:00` INT96 carrier and read back as `08:00`,
whereas the new cross-zone round-trip test expects the original value. Please
use the naive schema only for the default INT64 path (for example, gate it on
`!enable_int96_timestamps`) and cover the encoded/read-back INT96 result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
