Daniel Dai created HIVE-15936: --------------------------------- Summary: ConcurrentModificationException in ATSHook Key: HIVE-15936 URL: https://issues.apache.org/jira/browse/HIVE-15936 Project: Hive Issue Type: Bug Reporter: Daniel Dai Assignee: Daniel Dai Attachments: HIVE-15936.1.patch
See ATSHook error: {noformat} java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_112] at java.util.HashMap$EntryIterator.next(HashMap.java:1471) ~[?:1.8.0_112] at java.util.HashMap$EntryIterator.next(HashMap.java:1469) ~[?:1.8.0_112] at java.util.AbstractCollection.toArray(AbstractCollection.java:196) ~[?:1.8.0_112] at com.google.common.collect.ImmutableMap.copyOf(ImmutableMap.java:290) ~[guava-14.0.1.jar:?] at org.apache.hadoop.hive.ql.log.PerfLogger.getEndTimes(PerfLogger.java:219) ~[hive-common-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457] at org.apache.hadoop.hive.ql.hooks.ATSHook.createPostHookEvent(ATSHook.java:347) ~[hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457] at org.apache.hadoop.hive.ql.hooks.ATSHook$2.run(ATSHook.java:206) [hive-exec-2.1.0.2.6.0.0-457.jar:2.1.0.2.6.0.0-457] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_112] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_112] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_112] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112] {noformat} According to [~jdere], ATSHook is currently accessing the PerfLogger on a separate thread, which means the main query thread can potentially write to the PerfLogger at the same time. The ATSHook should access the PerfLogger on the main query thread, before it sends the execution to the ATS Logger thread. -- This message was sent by Atlassian JIRA (v6.3.15#6346)