Daniel Dai created HIVE-15936:
---------------------------------
Summary: ConcurrentModificationException in ATSHook
Key: HIVE-15936
URL: https://issues.apache.org/jira/browse/HIVE-15936
Project: Hive
Issue Type: Bug
Reporter: Daniel Dai
Assignee: Daniel Dai
Attachments: HIVE-15936.1.patch
See ATSHook error:
{noformat}
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437)
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1471)
~[?:1.8.0_112]
at java.util.HashMap$EntryIterator.next(HashMap.java:1469)
~[?:1.8.0_112]
at java.util.AbstractCollection.toArray(AbstractCollection.java:196)
~[?:1.8.0_112]
at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290)
~[guava-14.0.1.jar:?]
at
org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219)
~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at
org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347)
~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206)
[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[?:1.8.0_112]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[?:1.8.0_112]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[?:1.8.0_112]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]
{noformat}
According to [~jdere], ATSHook is currently accessing the PerfLogger on a
separate thread, which means the main query thread can potentially write to the
PerfLogger at the same time.
The ATSHook should access the PerfLogger on the main query thread, before it
sends the execution to the ATS Logger thread.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
