yinghua_zh created HIVE-22672:
---------------------------------
Summary: Error in tez optimizing SQL statement
Key: HIVE-22672
URL: https://issues.apache.org/jira/browse/HIVE-22672
Project: Hive
Issue Type: Bug
Components: hpl/sql
Affects Versions: 2.2.0
Reporter: yinghua_zh
(): FAILED: ConcurrentModificationException null
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
at java.util.HashMap$KeyIterator.next(HashMap.java:1461)
at
org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationFromSMBJoins(TezCompiler.java:640)
at
org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:117)
at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:138)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11169)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:257)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:485)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1252)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1239)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:191)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:276)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:312)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:500)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:487)
at sun.reflect.GeneratedMethodAccessor101.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
When map traverses, its elements are deleted，The code as follow:
for (TableScanOperator ts : tsOps) {
for (ReduceSinkOperator rs : pctx.getRsOpToTsOpMap().keySet()) {
if (ts == pctx.getRsOpToTsOpMap().get(rs))
{ // match! GenTezUtils.removeBranch(rs);
GenTezUtils.removeSemiJoinOperator(pctx, rs, ts); }
}
}
The function:GenTezUtils.removeSemiJoinOperator(pctx, rs, ts) will delete the
eletement from pctx.getRsOpToTsOpMap().
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
