github-actions[bot] commented on code in PR #65446:
URL: https://github.com/apache/doris/pull/65446#discussion_r3710860233
##########
fe/fe-connector/fe-connector-spi/src/main/java/org/apache/doris/connector/spi/scan/ConnectorScanPlanProvider.java:
##########
@@ -466,6 +466,15 @@ default void populateScanLevelParams(TFileScanRangeParams
params,
// Default: no scan-level params needed
}
+ /**
+ * Whether unannotated Parquet INT96 values use the Hive writer timezone
configured on the catalog.
+ * The default is false because table formats such as Iceberg and Paimon
define their own timestamp
+ * semantics even when they are discovered through an HMS-backed catalog.
+ */
+ default boolean usesHiveParquetInt96TimeZone() {
Review Comment:
[P1] Bump the connector plugin API major for this SPI addition
This new default method changes the public connector SPI, but the connector
POM and `ConnectorPluginSurfaceTest` still advertise major 3. Because the SPI
is loaded parent-first, a separately built major-3 Hive plugin is accepted by a
new FE and inherits this default `false`; `PluginDrivenScanNode` then silently
omits `hive.parquet.time-zone`, decoding configured INT96 data with raw-wall
semantics instead of rejecting the incompatible plugin. Please bump the API to
4.0 and update its pin, add `ConnectorScanPlanProvider` to the recorded plugin
surface (the metadata baseline itself stays unchanged), regenerate/check the
baselines, and cover rejection of an old-major plugin.
##########
be/src/format_v2/jni/jni_table_reader.cpp:
##########
@@ -480,6 +478,14 @@ Status JniTableReader::_open_jni_scanner() {
return Status::OK();
}
+void JniTableReader::_apply_common_scanner_params() {
+ if (_runtime_state != nullptr) {
+ // A connector-specific zone must win over the query default. Hudi
uses this to keep JNI
+ // and native file slices on the same catalog-level INT96
interpretation.
+ _scanner_params.try_emplace("time_zone", _runtime_state->timezone());
Review Comment:
[P1] Keep the common JNI timezone session-owned
`try_emplace` preserves any connector map entry named `time_zone`, but
MaxCompute copies every catalog property into this map and its provider accepts
unknown keys. A catalog created with `time_zone=UTC` and queried under an
`Asia/Shanghai` session now makes `MaxComputeJniScanner` materialize Arrow
timestamp instants in UTC, while `MaxComputePredicateConverter` intentionally
converts pushed DATETIME/TIMESTAMP literals from the SQL session zone; the Java
values and FE/BE predicates are then in different domains, so the post-JNI
filter can shift values or discard matching rows. No connector currently
installs a deliberate override after Hudi was restored to session semantics.
Please restore unconditional session assignment here (or introduce a narrowly
scoped, non-colliding connector hook/key) and cover a MaxCompute
catalog/session timezone collision.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
