[ https://issues.apache.org/jira/browse/HIVE-20665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16634803#comment-16634803 ]
Hive QA commented on HIVE-20665: -------------------------------- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 44s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 53s{color} | {color:blue} ql in master has 2321 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 24m 14s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14166/dev-support/hive-personality.sh | | git revision | master / c05733e | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14166/yetus.txt | | Powered by | Apache Yetus http://yetus.apache.org | This message was automatically generated. > Hive Parallel Tasks - Hive Configuration ConcurrentModificationException > ------------------------------------------------------------------------ > > Key: HIVE-20665 > URL: https://issues.apache.org/jira/browse/HIVE-20665 > Project: Hive > Issue Type: Bug > Components: HiveServer2 > Affects Versions: 2.3.2, 3.1.0, 4.0.0 > Reporter: BELUGA BEHR > Assignee: BELUGA BEHR > Priority: Major > Attachments: HIVE-20665.1.patch > > > When parallel tasks are enabled in Hive, all of the resulting queries share > the same Hive configuration. This is problematic as each query will modify > the same {{HiveConf}} object with things like query ID and query text. This > will overwrite each other and cause {{ConcurrentModificationException}} > issues. > {code:java|title=SQLOperation.java} > public Object run() throws HiveSQLException { > Hive.set(parentHive, false); > // TODO: can this result in cross-thread reuse of session state? > SessionState.setCurrentSessionState(parentSessionState); > PerfLogger.setPerfLogger(SessionState.getPerfLogger()); > LogUtils.registerLoggingContext(queryState.getConf()); > try { > if (asyncPrepare) { > prepare(queryState); > } > runQuery(); > } catch (HiveSQLException e) { > // ... > {code} > [Code > Here|https://github.com/apache/hive/blob/6e27a5315a44c55ef3b178e7212c9068de322d01/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java#L308-L319] > From this code it can be seen that for every thread launched, it is all > calling {{setCurrentSessionState}}. > {code:java|title=SessionStates.java} > /** > * Sets the given session state in the thread local var for sessions. > */ > public static void setCurrentSessionState(SessionState startSs) { > tss.get().attach(startSs); > } > // SessionState is not available in runtime and Hive.get().getConf() is not > safe to call > private static class SessionStates { > private SessionState state; > private HiveConf conf; > private void attach(SessionState state) { > this.state = state; > attach(state.getConf()); > } > private void attach(HiveConf conf) { > // -- SHALLOW COPY HERE, ALL THREADS SHARING SAME REFERENCE -- // > this.conf = conf; > ClassLoader classLoader = conf.getClassLoader(); > if (classLoader != null) { > Thread.currentThread().setContextClassLoader(classLoader); > } > } > } > {code} > [Code > Here|https://github.com/apache/hive/blob/7795c0a7dc59941671f8845d78b16d9e5ddc9ea3/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L540-L556] > Ensure that all threads get their own copy of the {{HiveConf}} object to use > and modify. -- This message was sent by Atlassian JIRA (v7.6.3#76005)