[ https://issues.apache.org/jira/browse/HIVE-7372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xuefu Zhang updated HIVE-7372: ------------------------------ Description: In SparkClient.java, if the following property is set, unpredictable, incorrect result may be observed. {code} sparkConf.set("spark.default.parallelism", "1"); {code} It's suspected that there are some concurrency issues, as Spark may process multiple datasets in a single JVM when parallelism is greater than 1 in order to use multiple cores. NO PRECOMMIT TESTS. This is for spark branch only. was: In SparkClient.java, if the following property is set, unpredictable, incorrect result may be observed. {code} sparkConf.set("spark.default.parallelism", "1"); {code} It's suspected that there are some concurrency issues, as Spark may process multiple datasets in a single JVM when parallelism is greater than 1 in order to use multiple cores. > Select query gives unpredictable incorrect result when parallelism is greater > than 1 [Spark Branch] > --------------------------------------------------------------------------------------------------- > > Key: HIVE-7372 > URL: https://issues.apache.org/jira/browse/HIVE-7372 > Project: Hive > Issue Type: Bug > Components: Spark > Reporter: Xuefu Zhang > Assignee: Chengxiang Li > Attachments: HIVE-7372.patch > > > In SparkClient.java, if the following property is set, unpredictable, > incorrect result may be observed. > {code} > sparkConf.set("spark.default.parallelism", "1"); > {code} > It's suspected that there are some concurrency issues, as Spark may process > multiple datasets in a single JVM when parallelism is greater than 1 in order > to use multiple cores. > NO PRECOMMIT TESTS. This is for spark branch only. -- This message was sent by Atlassian JIRA (v6.2#6252)