[jira] [Updated] (HIVE-13530) Hive on Spark throws Kryo exception in some cases
[ https://issues.apache.org/jira/browse/HIVE-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-13530: Resolution: Fixed Fix Version/s: 2.1.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks Szehon for the work. > Hive on Spark throws Kryo exception in some cases > - > > Key: HIVE-13530 > URL: https://issues.apache.org/jira/browse/HIVE-13530 > Project: Hive > Issue Type: Bug > Components: Spark >Reporter: Szehon Ho >Assignee: Szehon Ho > Fix For: 2.1.0 > > Attachments: HIVE-13530.2.patch, HIVE-13530.patch > > > After recent changes, Hive on Spark throws KryoException: > {noformat} > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: ERROR : Failed to execute spark > task, with exception 'java.lang.Exception(Failed to submit Spark work, please > retry later)' > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: java.lang.Exception: Failed to > submit Spark work, please retry later > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:174) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:103) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1769) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1526) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1305) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1114) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1107) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:178) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:72) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:232) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.security.AccessController.doPrivileged(Native Method) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > javax.security.auth.Subject.doAs(Subject.java:415) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:245) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.FutureTask.run(FutureTask.java:262) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.lang.Thread.run(Thread.java:745) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Caused by: > org.apache.hive.com.esotericsoftware.kryo.KryoException: > java.util.ConcurrentModificationException > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Serialization trace: > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classes > (sun.misc.Launcher$AppClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classloader > (java.security.ProtectionDomain) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: context > (java.security.AccessControlContext) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: acc (java.net.URLClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classLoader > (org.apache.hadoop.hive.conf.HiveConf) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: conf > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: metrics > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: openScopes > (org.apache.hadoop.hive.ql.log.PerfLogger) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: perfLogger > (org.apache.hadoop.hive.ql.exec
[jira] [Updated] (HIVE-13530) Hive on Spark throws Kryo exception in some cases
[ https://issues.apache.org/jira/browse/HIVE-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-13530: - Attachment: HIVE-13530.2.patch New patch marks the PerfLogger as transient. > Hive on Spark throws Kryo exception in some cases > - > > Key: HIVE-13530 > URL: https://issues.apache.org/jira/browse/HIVE-13530 > Project: Hive > Issue Type: Bug > Components: Spark >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-13530.2.patch, HIVE-13530.patch > > > After recent changes, Hive on Spark throws KryoException: > {noformat} > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: ERROR : Failed to execute spark > task, with exception 'java.lang.Exception(Failed to submit Spark work, please > retry later)' > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: java.lang.Exception: Failed to > submit Spark work, please retry later > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:174) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:103) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1769) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1526) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1305) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1114) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1107) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:178) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:72) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:232) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.security.AccessController.doPrivileged(Native Method) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > javax.security.auth.Subject.doAs(Subject.java:415) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:245) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.FutureTask.run(FutureTask.java:262) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.lang.Thread.run(Thread.java:745) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Caused by: > org.apache.hive.com.esotericsoftware.kryo.KryoException: > java.util.ConcurrentModificationException > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Serialization trace: > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classes > (sun.misc.Launcher$AppClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classloader > (java.security.ProtectionDomain) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: context > (java.security.AccessControlContext) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: acc (java.net.URLClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classLoader > (org.apache.hadoop.hive.conf.HiveConf) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: conf > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: metrics > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: openScopes > (org.apache.hadoop.hive.ql.log.PerfLogger) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: perfLogger > (org.apache.hadoop.hive.ql.exec.SparkHashTableSinkOperator) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: childOperators > (org
[jira] [Updated] (HIVE-13530) Hive on Spark throws Kryo exception in some cases
[ https://issues.apache.org/jira/browse/HIVE-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-13530: - Status: Patch Available (was: Open) > Hive on Spark throws Kryo exception in some cases > - > > Key: HIVE-13530 > URL: https://issues.apache.org/jira/browse/HIVE-13530 > Project: Hive > Issue Type: Bug > Components: Spark >Reporter: Szehon Ho >Assignee: Szehon Ho > Attachments: HIVE-13530.patch > > > After recent changes, Hive on Spark throws KryoException: > {noformat} > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: ERROR : Failed to execute spark > task, with exception 'java.lang.Exception(Failed to submit Spark work, please > retry later)' > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: java.lang.Exception: Failed to > submit Spark work, please retry later > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:174) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:103) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1769) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1526) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1305) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1114) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1107) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:178) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:72) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:232) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.security.AccessController.doPrivileged(Native Method) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > javax.security.auth.Subject.doAs(Subject.java:415) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:245) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.FutureTask.run(FutureTask.java:262) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.lang.Thread.run(Thread.java:745) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Caused by: > org.apache.hive.com.esotericsoftware.kryo.KryoException: > java.util.ConcurrentModificationException > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Serialization trace: > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classes > (sun.misc.Launcher$AppClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classloader > (java.security.ProtectionDomain) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: context > (java.security.AccessControlContext) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: acc (java.net.URLClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classLoader > (org.apache.hadoop.hive.conf.HiveConf) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: conf > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: metrics > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: openScopes > (org.apache.hadoop.hive.ql.log.PerfLogger) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: perfLogger > (org.apache.hadoop.hive.ql.exec.SparkHashTableSinkOperator) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: childOperators > (org.apache.hadoop.hive.ql.exec.TableScanOperator) > 16/04/14 21
[jira] [Updated] (HIVE-13530) Hive on Spark throws Kryo exception in some cases
[ https://issues.apache.org/jira/browse/HIVE-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-13530: - Attachment: HIVE-13530.patch I think this is my fault from HIVE-13410. The new field is bringing in all of codahale. [~jxiang] do you think it makes sense? Thanks. > Hive on Spark throws Kryo exception in some cases > - > > Key: HIVE-13530 > URL: https://issues.apache.org/jira/browse/HIVE-13530 > Project: Hive > Issue Type: Bug > Components: Spark >Reporter: Szehon Ho > Attachments: HIVE-13530.patch > > > After recent changes, Hive on Spark throws KryoException: > {noformat} > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: ERROR : Failed to execute spark > task, with exception 'java.lang.Exception(Failed to submit Spark work, please > retry later)' > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: java.lang.Exception: Failed to > submit Spark work, please retry later > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:174) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:103) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1769) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1526) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1305) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1114) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1107) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:178) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:72) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:232) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.security.AccessController.doPrivileged(Native Method) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > javax.security.auth.Subject.doAs(Subject.java:415) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:245) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.FutureTask.run(FutureTask.java:262) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.lang.Thread.run(Thread.java:745) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Caused by: > org.apache.hive.com.esotericsoftware.kryo.KryoException: > java.util.ConcurrentModificationException > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Serialization trace: > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classes > (sun.misc.Launcher$AppClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classloader > (java.security.ProtectionDomain) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: context > (java.security.AccessControlContext) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: acc (java.net.URLClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classLoader > (org.apache.hadoop.hive.conf.HiveConf) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: conf > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: metrics > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: openScopes > (org.apache.hadoop.hive.ql.log.PerfLogger) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: perfLogger > (org.apache.hadoop.hive.ql.exec.SparkHashTableSinkOperator) > 16/04/14 21:53:24 INFO hive
[jira] [Updated] (HIVE-13530) Hive on Spark throws Kryo exception in some cases
[ https://issues.apache.org/jira/browse/HIVE-13530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szehon Ho updated HIVE-13530: - Component/s: Spark > Hive on Spark throws Kryo exception in some cases > - > > Key: HIVE-13530 > URL: https://issues.apache.org/jira/browse/HIVE-13530 > Project: Hive > Issue Type: Bug > Components: Spark >Reporter: Szehon Ho > > After recent changes, Hive on Spark throws KryoException: > {noformat} > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: ERROR : Failed to execute spark > task, with exception 'java.lang.Exception(Failed to submit Spark work, please > retry later)' > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: java.lang.Exception: Failed to > submit Spark work, please retry later > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.execute(RemoteHiveSparkClient.java:174) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.submit(SparkSessionImpl.java:71) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:103) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1769) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1526) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1305) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1114) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.hive.ql.Driver.run(Driver.java:1107) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:178) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:72) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:232) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.security.AccessController.doPrivileged(Native Method) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > javax.security.auth.Subject.doAs(Subject.java:415) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:245) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.FutureTask.run(FutureTask.java:262) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: at > java.lang.Thread.run(Thread.java:745) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Caused by: > org.apache.hive.com.esotericsoftware.kryo.KryoException: > java.util.ConcurrentModificationException > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: Serialization trace: > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classes > (sun.misc.Launcher$AppClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classloader > (java.security.ProtectionDomain) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: context > (java.security.AccessControlContext) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: acc (java.net.URLClassLoader) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: classLoader > (org.apache.hadoop.hive.conf.HiveConf) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: conf > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: metrics > (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: openScopes > (org.apache.hadoop.hive.ql.log.PerfLogger) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: perfLogger > (org.apache.hadoop.hive.ql.exec.SparkHashTableSinkOperator) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: childOperators > (org.apache.hadoop.hive.ql.exec.TableScanOperator) > 16/04/14 21:53:24 INFO hiveserver2.DDLTest: aliasToWork > (org.apache.hadoop.hive.ql.plan.MapWork) > 16