[jira] [Updated] (HIVE-14293) PerfLogger.openScopes should be transient
[
https://issues.apache.org/jira/browse/HIVE-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-14293:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.1.1
2.2.0
Status: Resolved (was: Patch Available)
The last precommit run is clear. All test failures are seen in other runs.
Patch pushed to both branch-2.1 and master.
> PerfLogger.openScopes should be transient
> -
>
> Key: HIVE-14293
> URL: https://issues.apache.org/jira/browse/HIVE-14293
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Fix For: 2.2.0, 2.1.1
>
> Attachments: HIVE-14293.1.patch, HIVE-14293.2.patch,
> HIVE-14293.3.patch, HIVE-14293.4.patch
>
>
> See the following exception when running Hive e2e tests:
> {code}
> 0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
> v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name
> = v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name)
> INNER JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age =
> s.age ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
> INFO : Compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
> type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
> FieldSchema(name:s.gpa, type:double, comment:null),
> FieldSchema(name:v.registration, type:string, comment:null),
> FieldSchema(name:v2.contributions, type:float, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8);
> Time taken: 1.165 seconds
> INFO : Executing
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Session is already open
> INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
> ERROR : Failed to execute tez graph.
> java.lang.RuntimeException: Error caching map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException
> Serialization trace:
> classes (sun.misc.Launcher$AppClassLoader)
> classloader (java.security.ProtectionDomain)
> context (java.security.AccessControlContext)
> acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
> classLoader (org.apache.hadoop.hive.conf.HiveConf)
> conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
> metrics
> (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
> openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
> perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2
[jira] [Updated] (HIVE-14293) PerfLogger.openScopes should be transient
[
https://issues.apache.org/jira/browse/HIVE-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-14293:
--
Attachment: HIVE-14293.4.patch
For retest.
> PerfLogger.openScopes should be transient
> -
>
> Key: HIVE-14293
> URL: https://issues.apache.org/jira/browse/HIVE-14293
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-14293.1.patch, HIVE-14293.2.patch,
> HIVE-14293.3.patch, HIVE-14293.4.patch
>
>
> See the following exception when running Hive e2e tests:
> {code}
> 0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
> v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name
> = v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name)
> INNER JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age =
> s.age ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
> INFO : Compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
> type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
> FieldSchema(name:s.gpa, type:double, comment:null),
> FieldSchema(name:v.registration, type:string, comment:null),
> FieldSchema(name:v2.contributions, type:float, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8);
> Time taken: 1.165 seconds
> INFO : Executing
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Session is already open
> INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
> ERROR : Failed to execute tez graph.
> java.lang.RuntimeException: Error caching map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException
> Serialization trace:
> classes (sun.misc.Launcher$AppClassLoader)
> classloader (java.security.ProtectionDomain)
> context (java.security.AccessControlContext)
> acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
> classLoader (org.apache.hadoop.hive.conf.HiveConf)
> conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
> metrics
> (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
> openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
> perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0
[jira] [Updated] (HIVE-14293) PerfLogger.openScopes should be transient
[
https://issues.apache.org/jira/browse/HIVE-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-14293:
--
Attachment: HIVE-14293.3.patch
Don't think test failures are related. For safety, reattach the same patch to
retest.
> PerfLogger.openScopes should be transient
> -
>
> Key: HIVE-14293
> URL: https://issues.apache.org/jira/browse/HIVE-14293
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-14293.1.patch, HIVE-14293.2.patch,
> HIVE-14293.3.patch
>
>
> See the following exception when running Hive e2e tests:
> {code}
> 0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
> v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name
> = v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name)
> INNER JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age =
> s.age ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
> INFO : Compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
> type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
> FieldSchema(name:s.gpa, type:double, comment:null),
> FieldSchema(name:v.registration, type:string, comment:null),
> FieldSchema(name:v2.contributions, type:float, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8);
> Time taken: 1.165 seconds
> INFO : Executing
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Session is already open
> INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
> ERROR : Failed to execute tez graph.
> java.lang.RuntimeException: Error caching map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException
> Serialization trace:
> classes (sun.misc.Launcher$AppClassLoader)
> classloader (java.security.ProtectionDomain)
> context (java.security.AccessControlContext)
> acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
> classLoader (org.apache.hadoop.hive.conf.HiveConf)
> conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
> metrics
> (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
> openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
> perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Ta
[jira] [Updated] (HIVE-14293) PerfLogger.openScopes should be transient
[
https://issues.apache.org/jira/browse/HIVE-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-14293:
--
Attachment: HIVE-14293.2.patch
Get it. Piggyback HIVE-14232 in HIVE-14293.2.patch.
> PerfLogger.openScopes should be transient
> -
>
> Key: HIVE-14293
> URL: https://issues.apache.org/jira/browse/HIVE-14293
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-14293.1.patch, HIVE-14293.2.patch
>
>
> See the following exception when running Hive e2e tests:
> {code}
> 0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
> v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name
> = v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name)
> INNER JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age =
> s.age ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
> INFO : Compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
> type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
> FieldSchema(name:s.gpa, type:double, comment:null),
> FieldSchema(name:v.registration, type:string, comment:null),
> FieldSchema(name:v2.contributions, type:float, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8);
> Time taken: 1.165 seconds
> INFO : Executing
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Session is already open
> INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
> ERROR : Failed to execute tez graph.
> java.lang.RuntimeException: Error caching map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException
> Serialization trace:
> classes (sun.misc.Launcher$AppClassLoader)
> classloader (java.security.ProtectionDomain)
> context (java.security.AccessControlContext)
> acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
> classLoader (org.apache.hadoop.hive.conf.HiveConf)
> conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
> metrics
> (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
> openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
> perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.
[jira] [Updated] (HIVE-14293) PerfLogger.openScopes should be transient
[
https://issues.apache.org/jira/browse/HIVE-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-14293:
--
Status: Patch Available (was: Open)
> PerfLogger.openScopes should be transient
> -
>
> Key: HIVE-14293
> URL: https://issues.apache.org/jira/browse/HIVE-14293
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-14293.1.patch
>
>
> See the following exception when running Hive e2e tests:
> {code}
> 0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
> v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name
> = v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name)
> INNER JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age =
> s.age ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
> INFO : Compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
> type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
> FieldSchema(name:s.gpa, type:double, comment:null),
> FieldSchema(name:v.registration, type:string, comment:null),
> FieldSchema(name:v2.contributions, type:float, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8);
> Time taken: 1.165 seconds
> INFO : Executing
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Session is already open
> INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
> ERROR : Failed to execute tez graph.
> java.lang.RuntimeException: Error caching map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException
> Serialization trace:
> classes (sun.misc.Launcher$AppClassLoader)
> classloader (java.security.ProtectionDomain)
> context (java.security.AccessControlContext)
> acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
> classLoader (org.apache.hadoop.hive.conf.HiveConf)
> conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
> metrics
> (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
> openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
> perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner
[jira] [Updated] (HIVE-14293) PerfLogger.openScopes should be transient
[
https://issues.apache.org/jira/browse/HIVE-14293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-14293:
--
Attachment: HIVE-14293.1.patch
> PerfLogger.openScopes should be transient
> -
>
> Key: HIVE-14293
> URL: https://issues.apache.org/jira/browse/HIVE-14293
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-14293.1.patch
>
>
> See the following exception when running Hive e2e tests:
> {code}
> 0: jdbc:hive2://nat-r6-ojss-hsihs2-1.openstac> SELECT s.name, s2.age, s.gpa,
> v.registration, v2.contributions FROM student s INNER JOIN voter v ON (s.name
> = v.name) INNER JOIN student s2 ON (s2.age = v.age and s.name = s2.name)
> INNER JOIN voter v2 ON (v2.name = s2.name and v2.age = s2.age) WHERE v2.age =
> s.age ORDER BY s.name, s2.age, s.gpa, v.registration, v2.contributions;
> INFO : Compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Semantic Analysis Completed
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:s.name,
> type:string, comment:null), FieldSchema(name:s2.age, type:int, comment:null),
> FieldSchema(name:s.gpa, type:double, comment:null),
> FieldSchema(name:v.registration, type:string, comment:null),
> FieldSchema(name:v2.contributions, type:float, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8);
> Time taken: 1.165 seconds
> INFO : Executing
> command(queryId=hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8):
> SELECT s.name, s2.age, s.gpa, v.registration, v2.contributions FROM student s
> INNER JOIN voter v ON (s.name = v.name) INNER JOIN student s2 ON (s2.age =
> v.age and s.name = s2.name) INNER JOIN voter v2 ON (v2.name = s2.name and
> v2.age = s2.age) WHERE v2.age = s.age ORDER BY s.name, s2.age, s.gpa,
> v.registration, v2.contributions
> INFO : Query ID = hive_20160717224915_3a52719f-539f-4f82-a9cd-0c0af4e09ef8
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Session is already open
> INFO : Dag name: SELECT s.name, s2.age, sv2.contributions(Stage-1)
> ERROR : Failed to execute tez graph.
> java.lang.RuntimeException: Error caching map.xml:
> org.apache.hive.com.esotericsoftware.kryo.KryoException:
> java.util.ConcurrentModificationException
> Serialization trace:
> classes (sun.misc.Launcher$AppClassLoader)
> classloader (java.security.ProtectionDomain)
> context (java.security.AccessControlContext)
> acc (org.apache.hadoop.hive.ql.exec.UDFClassLoader)
> classLoader (org.apache.hadoop.hive.conf.HiveConf)
> conf (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics)
> metrics
> (org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope)
> openScopes (org.apache.hadoop.hive.ql.log.PerfLogger)
> perfLogger (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.MapJoinOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.FilterOperator)
> childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
> aliasToWork (org.apache.hadoop.hive.ql.plan.MapWork)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:582)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:516)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:601)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.tez.DagUtils.createVertex(DagUtils.java:1147)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:390)
> ~[hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:164)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> [hive-exec-2.1.0.2.5.0.0-1009.jar:2.1.0.2.5.0.0-1009]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSe
