[
https://issues.apache.org/jira/browse/HIVE-28667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17980020#comment-17980020
]
Stamatis Zampetakis commented on HIVE-28667:
--------------------------------------------
RelFieldTrimmer class is a reflection based visitor that tries to find the
appropriate method based on the type (Class) of the argument. The errors occurs
while trying to register a method dispatcher for {{JdbcSort.class}} since there
are multiple candidate methods that qualify:
* trimFields(JdbcRel rel, ImmutableBitSet fieldsUsed, Set<RelDataTypeField>
extraFields)
* trimFields(Sort sort, ImmutableBitSet fieldsUsed, Set<RelDataTypeField>
extraFields)
Given that there is no inheritance relationship between `Sort` and `JdbcRel` it
is unclear which method should be used to handle JdbcSort leading to an
exception that informs the user/dev about the ambiguous semantics.
It seems that the intention of adding trimFields(JdbcRel...) method was to have
a fallback for Jdbc operators that are not handled explicitly by the trimmer
but this can never work and just creates ambiguity. The reason that we get this
error only during startup of HS2 and not during compilation is because in the
current code it is impossible to call the RelFieldTrimmer code with Jdbc
operators other than HiveJdbcConverter and JdbcHiveTableScan.
> Starting HS2 will occur warn log: rules.HiveRelFieldTrimmer: Error
> initializing field trimmer instance
> ------------------------------------------------------------------------------------------------------
>
> Key: HIVE-28667
> URL: https://issues.apache.org/jira/browse/HIVE-28667
> Project: Hive
> Issue Type: Improvement
> Reporter: Butao Zhang
> Assignee: Stamatis Zampetakis
> Priority: Major
>
> Tested with laster master branch: commit id:
> *4898d7bfaa1437d39d25070e0ab63c5e0e0de9d7*
> Starting HS2 will see the following warn log. But HS2 would start
> successfully at the end.
>
> {code:java}
> 2024-12-13T16:49:13,207 WARN [main] rules.HiveRelFieldTrimmer: Error
> initializing field trimmer instance
> java.lang.IllegalArgumentException: dispatch ambiguity between public
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Sort,org.apache.calcite.util.ImmutableBitSet,java.util.Set)
> and public
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer.trimFields(org.apache.calcite.adapter.jdbc.JdbcRel,org.apache.calcite.util.ImmutableBitSet,java.util.Set)
> at
> org.apache.calcite.util.ReflectUtil.lookupVisitMethod(ReflectUtil.java:391)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.calcite.util.ReflectUtil.lookupVisitMethod(ReflectUtil.java:325)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReflectUtil$VisitDispatcher.lookupVisitFunc(HiveReflectUtil.java:165)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveReflectUtil$HiveMethodDispatcher.register(HiveReflectUtil.java:255)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.register(HiveRelFieldTrimmer.java:871)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.initializeFieldTrimmerClass(HiveRelFieldTrimmer.java:880)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.warmup(CalcitePlanner.java:5605)
> ~[hive-exec-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:311)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:1190)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:1551)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at
> org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:1358)
> ~[hive-service-4.1.0-SNAPSHOT.jar:4.1.0-SNAPSHOT]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
> at org.apache.hadoop.util.RunJar.run(RunJar.java:328)
> ~[hadoop-common-3.3.6.jar:?]
> at org.apache.hadoop.util.RunJar.main(RunJar.java:241)
> ~[hadoop-common-3.3.6.jar:?]
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
