[
https://issues.apache.org/jira/browse/HIVE-11681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16103506#comment-16103506
]
frank luo commented on HIVE-11681:
----------------------------------
https://issues.apache.org/jira/browse/HADOOP-13809 is a similar case.
I believe they are all related to
https://bugs.openjdk.java.net/browse/JDK-6947916, which hasn't been released.
I am able to recreate it with oracle jdk 1.8.0_131.
> sometimes when query mr job progress, stream closed exception will happen
> -------------------------------------------------------------------------
>
> Key: HIVE-11681
> URL: https://issues.apache.org/jira/browse/HIVE-11681
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
> Affects Versions: 1.2.1
> Reporter: wangwenli
>
> sometimes the hiveserver will throw below exception ,
> 2015-08-28 05:05:44,107 | FATAL | Thread-82995 | error parsing conf
> mapred-default.xml |
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2404)
> java.io.IOException: Stream closed
> at
> java.util.zip.InflaterInputStream.ensureOpen(InflaterInputStream.java:84)
> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:160)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:2902)
> at
> com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:302)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1753)
> at
> com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipChar(XMLEntityScanner.java:1426)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2807)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
> at
> com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
> at
> com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
> at
> com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
> at
> com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
> at
> com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:243)
> at
> com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:347)
> at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2246)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2234)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2305)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2258)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2175)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:854)
> at
> org.apache.hadoop.mapred.JobConf.checkAndWarnDeprecation(JobConf.java:2069)
> at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:477)
> at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:467)
> at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:187)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:580)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:578)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1612)
> at
> org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:578)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:596)
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:289)
> at
> org.apache.hadoop.hive.ql.exec.mr.HadoopJobExecHelper.progress(HadoopJobExecHelper.java:548)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:435)
> at
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:159)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:72)
> after analysis, we found the root cause, below is step to reproduce the issue
> 1. open one beeline window, add jar
> 2. execute one sql like create table abc as select * from t1;
> 3. execute !quit, before this add a breakpoint at
> java.net.URLClassLoader.close(), so it will stop here
> 4. open another beeline window
> 5. execute one sql like select count(*) from t1, before this add one
> condition breakpoint at org.apache.hadoop.conf.Configuration.parse, the
> condition is
> url.toString().indexOf("hadoop-mapreduce-client-core-V100R001C00.jar!/mapred-default.xml")>0
> 6. when proceed to the above step, just get the stream
> 7. let the 3th step go ahead, which will close the stream
> 8. now, let sixth step go ahead, then the above exception coming
> suggest solution:
> the issue is hadppend when client use add jar + short connection , the
> client app should try best to use long connection, means reuse hive connetion
> meanwhile, here ,experts can suggest good solution for this issue
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
