[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874766#comment-13874766
]
Hudson commented on MAPREDUCE-5724:
---
FAILURE: Integrated in Hadoop-Hdfs-trunk #1647 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1647/])
MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu)
(tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1558852)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestHistoryFileManager.java
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874777#comment-13874777
]
Hudson commented on MAPREDUCE-5724:
---
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1672 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1672/])
MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu)
(tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1558852)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestHistoryFileManager.java
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unkn
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13874669#comment-13874669
]
Hudson commented on MAPREDUCE-5724:
---
SUCCESS: Integrated in Hadoop-Yarn-trunk #455 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/455/])
MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu)
(tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1558852)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestHistoryFileManager.java
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873621#comment-13873621
]
Hudson commented on MAPREDUCE-5724:
---
SUCCESS: Integrated in Hadoop-trunk-Commit #5009 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/5009/])
MAPREDUCE-5724. JobHistoryServer does not start if HDFS is not running. (tucu)
(tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1558852)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JHAdminConfig.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
*
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestHistoryFileManager.java
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown So
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873075#comment-13873075
]
Sandy Ryza commented on MAPREDUCE-5724:
---
bq. Regarding detecting the SafeModeException by cause, I've tried that at
first, the problem is that the cause is NULL
Makes sense
+1
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:185)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:671)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1722)
> at org.apache.hadoop.fs.Hdf
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873036#comment-13873036
]
Hadoop QA commented on MAPREDUCE-5724:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623303/MAPREDUCE-5724.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4322//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4322//console
This message is automatically generated.
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInsta
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873023#comment-13873023
]
Alejandro Abdelnur commented on MAPREDUCE-5724:
---
created HDFS-5787
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch,
> MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:185)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:671)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1722)
> at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:124)
> at org.apache.hadoop.fs.FileContext$14.next(FileContext.java:110
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13873003#comment-13873003
]
Sandy Ryza commented on MAPREDUCE-5724:
---
bq. Regarding removing the throw Exception from the createHistoryDirs(), not
possible because the tryCreateHistoryDirs does throw a checked exception if the
reason is other than the FS not being avail.
In that case, createHistoryDirs should just throw an IOException, no?
{code}
return ex.toString().contains("SafeModeException");
{code}
Can we get the SafeModeException from ex.getCause()?
Otherwise, LGTM.
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:185)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872910#comment-13872910
]
Hadoop QA commented on MAPREDUCE-5724:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623276/MAPREDUCE-5724.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4321//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4321//console
This message is automatically generated.
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch, MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872828#comment-13872828
]
Hadoop QA commented on MAPREDUCE-5724:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623258/MAPREDUCE-5724.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4320//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4320//console
This message is automatically generated.
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.had
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872826#comment-13872826
]
Sandy Ryza commented on MAPREDUCE-5724:
---
The approach sounds reasonable to me.
{code}
+ //DistributedFileSystem returns a RemoteException with a message stating
+ // SafeModeException in it. So this is only way to check it is because of
+ // being in safe mode.
{code}
Should use block comments like the other (even non-public) methods in the
class. Applies other places as well.
{code}
+ throw new YarnRuntimeException(
+ "Timed out waiting for FileSystem to become available");
{code}
Should report how long it waited.
{code}
+return ex.toString().contains("SafeModeException");
{code}
Two spaces between return and ex. Also, can we get the SafeModeException from
ex.getCause()?
{code}
+boolean done = false;
{code}
Nit: "succeeded" would be clearer to me than "done". Not a big deal either way.
{code}
+ void createHistoryDirs(Clock clock, long intervalCheck, long timeOut)
+ throws Exception {
...
+ Thread.sleep(intervalCheck);
{code}
Should handle interrupted exception - we don't want to fail if we hit it? Then
we can remove "throws Exception" because the only other exception in the method
isn't checked?
{code}
+ Assert.assertTrue(dfsCluster.getFileSystem().isInSafeMode());
+} catch (Exception ex) {
+ Assert.fail(ex.toString());
+}
{code}
Can asserts in other threads ever cause the test to fail?
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtil
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13872818#comment-13872818
]
Karthik Kambatla commented on MAPREDUCE-5724:
-
Would it make sense to add a config, a value of -1 would mean to wait forever
like we do with other such timeout configs? The default itself could be few
mins or -1.
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Assignee: Alejandro Abdelnur
>Priority: Critical
> Attachments: MAPREDUCE-5724.patch
>
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:185)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:671)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1722)
> at org.apache.hadoop.fs.Hdfs.ge
[jira] [Commented] (MAPREDUCE-5724) JobHistoryServer does not start if HDFS is not running
[
https://issues.apache.org/jira/browse/MAPREDUCE-5724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13871450#comment-13871450
]
Alejandro Abdelnur commented on MAPREDUCE-5724:
---
YARN-24 fixed a similar issue for the NM, we should try doing something similar
here.
> JobHistoryServer does not start if HDFS is not running
> --
>
> Key: MAPREDUCE-5724
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5724
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Alejandro Abdelnur
>Priority: Critical
>
> Starting JHS without HDFS running fails with the following error:
> {code}
> STARTUP_MSG: build = git://git.apache.org/hadoop-common.git -r
> ad74e8850b99e03b0b6435b04f5b3e9995bc3956; compiled by 'tucu' on
> 2014-01-14T22:40Z
> STARTUP_MSG: java = 1.7.0_45
> /
> 2014-01-14 16:47:40,264 INFO
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal
> handlers for [TERM, HUP, INT]
> 2014-01-14 16:47:40,883 WARN org.apache.hadoop.util.NativeCodeLoader: Unable
> to load native-hadoop library for your platform... using builtin-java classes
> where applicable
> 2014-01-14 16:47:41,101 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
> JobHistory Init
> 2014-01-14 16:47:41,710 INFO org.apache.hadoop.service.AbstractService:
> Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state
> INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error
> creating done directory:
> [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating done
> directory: [hdfs://localhost:8020/tmp/hadoop-yarn/staging/history/done]
> at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.serviceInit(HistoryFileManager.java:505)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:94)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:207)
> at
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:217)
> Caused by: java.net.ConnectException: Call From dontknow.local/172.20.10.4 to
> localhost:8020 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
> at org.apache.hadoop.ipc.Client.call(Client.java:1410)
> at org.apache.hadoop.ipc.Client.call(Client.java:1359)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:185)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:101)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:671)
> at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1722)
> at org.apache.hadoop.fs.Hdfs.getFileStatus(Hdfs.java:124)
> at org.apache.hadoop.fs.FileContext$14.next(FileContext.java:1106)
> at org.apache.hadoop.fs.FileContext$14.next(FileContext
