[jira] [Commented] (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
[ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284114#comment-13284114 ] xieguiming commented on MAPREDUCE-5: I also met the same exception, and so many. and at last the TT port(50060), 83 connections are in CLOSE_WAIT state. and can not accept any connect. 2012-05-17 15:52:42,513 INFO TaskTracker.clienttrace (TaskTracker.java:doGet(3211)) - src: 172.16.4.7:10060, dest: 172.16.4.8:47138, bytes: 458752, op: MAPRED_SHUFFLE, cliID: attempt_201205061050_13074_m_000179_0 2012-05-17 15:52:42,514 ERROR mortbay.log (Slf4jLog.java:warn(87)) - /mapOutput java.lang.IllegalStateException: Committed at org.mortbay.jetty.Response.resetBuffer(Response.java:1023) at org.mortbay.jetty.Response.sendError(Response.java:240) at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3202) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 2012-05-17 15:52:42,510 WARN mapred.TaskTracker (TaskTracker.java:doGet(3198)) - getMapOutput(attempt_201205061050_13076_m_55_0,4) failed : org.mortbay.jetty.EofException: timeout at org.mortbay.jetty.AbstractGenerator$Output.blockForOutput(AbstractGenerator.java:548) at org.mortbay.jetty.AbstractGenerator$Output.flush(AbstractGenerator.java:572) at org.mortbay.jetty.HttpConnection$Output.flush(HttpConnection.java:1012) at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:651) at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:580) at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3179) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException --- Key: MAPREDUCE-5 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5 Project:
[jira] [Commented] (MAPREDUCE-2386) TT jetty server stuck in tight loop around epoll_wait
[ https://issues.apache.org/jira/browse/MAPREDUCE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284118#comment-13284118 ] xieguiming commented on MAPREDUCE-2386: --- Adding some info: 1TT throw many exceptions of EofException and IllegalStateException: 2012-05-17 15:52:42,509 WARN mortbay.log (Slf4jLog.java:warn(76)) - Committed before 410 getMapOutput(attempt_201205061050_13074_m_000167_0,9) failed : org.mortbay.jetty.EofException: timeout at org.mortbay.jetty.AbstractGenerator$Output.blockForOutput(AbstractGenerator.java:548) at org.mortbay.jetty.AbstractGenerator$Output.flush(AbstractGenerator.java:572) at org.mortbay.jetty.HttpConnection$Output.flush(HttpConnection.java:1012) at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:651) at org.mortbay.jetty.AbstractGenerator$Output.write(AbstractGenerator.java:580) at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3179) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 2012-05-17 15:52:42,510 INFO TaskTracker.clienttrace (TaskTracker.java:doGet(3211)) - src: 172.16.4.7:10060, dest: 172.16.4.8:47139, bytes: 655360, op: MAPRED_SHUFFLE, cliID: attempt_201205061050_13074_m_000167_0 2012-05-17 15:52:42,510 ERROR mortbay.log (Slf4jLog.java:warn(87)) - /mapOutput java.lang.IllegalStateException: Committed at org.mortbay.jetty.Response.resetBuffer(Response.java:1023) at org.mortbay.jetty.Response.sendError(Response.java:240) at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3202) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 2 The 83 connections of CLOSE_WAIT status always exist, and not disappear. TT jetty server stuck in tight loop around epoll_wait - Key: MAPREDUCE-2386 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2386 Project: Hadoop Map/Reduce
[jira] [Assigned] (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
[ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming reassigned MAPREDUCE-5: -- Assignee: xieguiming Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException --- Key: MAPREDUCE-5 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.2 Environment: Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64) 10 node cluster Reporter: George Porter Assignee: xieguiming During the shuffle phase, I'm seeing a large sequence of the following actions: 1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed The map phase completes with 100%, and then the reduce phase crawls along with the above errors in each of the TaskTracker logs. None of the tasktrackers get lost. When I run non-data jobs like the 'pi' test from the example jar, everything works fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
[ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming reassigned MAPREDUCE-5: -- Assignee: (was: xieguiming) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException --- Key: MAPREDUCE-5 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.2 Environment: Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64) 10 node cluster Reporter: George Porter During the shuffle phase, I'm seeing a large sequence of the following actions: 1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed The map phase completes with 100%, and then the reduce phase crawls along with the above errors in each of the TaskTracker logs. None of the tasktrackers get lost. When I run non-data jobs like the 'pi' test from the example jar, everything works fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
[ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284130#comment-13284130 ] xieguiming commented on MAPREDUCE-5: I have analyzed this problem for one whole day, and I will show some details more. 1The TT throw the EofException and the IllegalStateExcetion for the getMapOutput. 2and then,I use the netstat command to check the http port (50060), and find 83 connections are on CLOSE_WAIT state.and the CLOSE_WAIT state do not disapper always. At least, for 24 hours. 3form the TT log, after print the exception, the TT http server do not work well. can not accept any http request(no sent out log found later). and JT add it to the blacklist. I use the curl shell command to access the http service, and client throw timeout. and the Datanode http service on the same node is ok. 4and I also find the TT CPU is 100% even when there is no any childjvm. 5and I also find the reduce task on the same node copy slower from other node . 6I restart the TT. and the TT works well. I attach the TT logs. if need other logs, tell me. but I am sorry that we have not the matched userlog, because the userlog will be delete after only 3 hours. and when we find the problem, and many hours pass. Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException --- Key: MAPREDUCE-5 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.2 Environment: Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64) 10 node cluster Reporter: George Porter Attachments: temp.rar During the shuffle phase, I'm seeing a large sequence of the following actions: 1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed The map phase completes with 100%, and then the reduce phase crawls along with the above errors in each of the TaskTracker logs. None of the tasktrackers get lost. When I run non-data jobs like the 'pi' test from the example jar, everything works fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
[ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming updated MAPREDUCE-5: --- Attachment: temp.rar Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException --- Key: MAPREDUCE-5 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.20.2 Environment: Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64) 10 node cluster Reporter: George Porter Attachments: temp.rar During the shuffle phase, I'm seeing a large sequence of the following actions: 1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_10_0,0) failed : org.mortbay.jetty.EofException 3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed The map phase completes with 100%, and then the reduce phase crawls along with the above errors in each of the TaskTracker logs. None of the tasktrackers get lost. When I run non-data jobs like the 'pi' test from the example jar, everything works fine. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2386) TT jetty server stuck in tight loop around epoll_wait
[ https://issues.apache.org/jira/browse/MAPREDUCE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13284104#comment-13284104 ] xieguiming commented on MAPREDUCE-2386: --- Hi: On my cluster, one TT also stuck. It's not responding to any HTTP connections 1 the thread stack info: 1989360587@qtp-1863318328-0 - Acceptor0 SelectChannelConnector@0.0.0.0:10060 prio=10 tid=0x7fb9fc2a6800 nid=0x612e runnable [0x7fba0015b000] java.lang.Thread.State: RUNNABLE at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69) - locked 0x7fba14758c70 (a sun.nio.ch.Util$1) - locked 0x7fba14758c58 (a java.util.Collections$UnmodifiableSet) - locked 0x7fba124d8aa8 (a sun.nio.ch.EPollSelectorImpl) at sun.nio.ch.SelectorImpl.selectNow(SelectorImpl.java:88) at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:652) at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192) at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124) at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) 2 I use netstat cmd to check the 50060 port state, and find 83 connections are on CLOSE_WAIT or SYN_RECV state. tcp0 0 172.16.4.7:50060172.16.4.6:52526SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.3:41380SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.5:41908SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.6:52495SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.8:39167SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.8:38799SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.6:52416SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.6:47010SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.5:42449SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.2:50107SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.6:52558SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.6:52402SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.6:52085SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.2:45092SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.3:41542SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.3:55977SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.4:43743SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.5:42118SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.2:44535SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.3:41890SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.3:56001SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.5:42057SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.3:56121SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.8:39173SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.8:38937SYN_RECV tcp0 0 172.16.4.7:50060172.16.4.2:44992SYN_RECV tcp 129 0 :::50060:::*LISTEN tcp 243 0 172.16.4.7:50060172.16.4.7:35878CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.7:50557CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.8:33735CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.6:40670CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.5:45702CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.3:50653CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.3:50538CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.6:48535CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.7:52049CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.5:45529CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.7:38282CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.7:51933CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.8:33008CLOSE_WAIT tcp 243 0 172.16.4.7:50060172.16.4.2:50188CLOSE_WAIT
[jira] [Commented] (MAPREDUCE-4208) The job is hanging up but never continuing until you kill the child process
[ https://issues.apache.org/jira/browse/MAPREDUCE-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13268104#comment-13268104 ] xieguiming commented on MAPREDUCE-4208: --- Hi ccw: whether the speculative is on? and the speculative is useful for your scenario? I think that the timeout mechanism is not good. because that the service will spend long time indeed. so, if kill the task by timeout mechanism, and the job is failed in fact. and is the same with killing the job. so, I suggest that it is suitable for user's application to kill the job when the job does not complete for long time. The job is hanging up but never continuing until you kill the child process Key: MAPREDUCE-4208 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4208 Project: Hadoop Map/Reduce Issue Type: Bug Environment: Hadoop 0.20.203.0 Hbase 0.90.3 Hive 0.80.1 Reporter: ccw I use the hive MR query on hbase,but the job is never end. The job is hanging but never continuing util you kill the child process 2012-04-28 18:22:33,661 Stage-1 map = 0%, reduce = 0% 2012-04-28 18:22:59,760 Stage-1 map = 25%, reduce = 0% 2012-04-28 18:23:04,782 Stage-1 map = 38%, reduce = 0% 2012-04-28 18:23:07,796 Stage-1 map = 50%, reduce = 0% 2012-04-28 18:23:08,801 Stage-1 map = 50%, reduce = 8% 2012-04-28 18:23:17,839 Stage-1 map = 50%, reduce = 17% 2012-04-28 18:23:19,848 Stage-1 map = 63%, reduce = 17% 2012-04-28 18:23:32,909 Stage-1 map = 63%, reduce = 21% 2012-04-28 18:23:57,017 Stage-1 map = 75%, reduce = 21% 2012-04-28 18:24:09,075 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:25:09,397 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:26:09,688 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:27:09,980 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:28:10,262 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:29:10,522 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:30:10,742 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:31:10,985 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:32:11,238 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:33:11,467 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:34:11,731 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:35:11,968 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:36:12,213 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:37:12,508 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:38:12,747 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:39:12,970 Stage-1 map = 75%, reduce = 25% 2012-04-28 18:40:13,205 Stage-1 map = 75%, reduce = 25% I checked the TT log, 2012-04-28 18:31:53,879 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:31:56,883 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:31:59,887 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:02,892 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:05,897 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:08,902 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:11,906 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:14,910 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:17,915 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:20,920 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:23,924 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:26,929 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:29,934 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:32,938 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:35,943 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:38,948 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:41,953 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:44,957 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:47,961 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201204281725_0002_m_02_0 0.0% 2012-04-28 18:32:50,966 INFO org.apache.hadoop.mapred.TaskTracker:
[jira] [Commented] (MAPREDUCE-4124) Job Priority is not changing
[ https://issues.apache.org/jira/browse/MAPREDUCE-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266491#comment-13266491 ] xieguiming commented on MAPREDUCE-4124: --- By the Yarn design goal, the AM can support multiple jobs. If like this, the priority also need pass to the AM. Job Priority is not changing - Key: MAPREDUCE-4124 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4124 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 3.0.0 Reporter: Nishan Shetty Priority: Critical 1.Submit job 2.Change the job priority using setPriority() or CLI command ./mapred job-set-priority job-id priority Observe that Job priority is not changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13267126#comment-13267126 ] xieguiming commented on MAPREDUCE-3873: --- Hi Vinod: Thanks for your testCase. TestResourceTrackerService is very good. At the beginning, I only focus on the NodesListManager:isValidNode function. Nodemanager is not getting decommisioned if the absolute ip is given in exclude file. - Key: MAPREDUCE-3873 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Nishan Shetty Assignee: xieguiming Priority: Minor Attachments: MAPREDUCE-3873-20120502.1.txt, MAPREDUCE-3873-20120502.txt, MAPREDUCE-3873-20120502.txt, MAPREDUCE-3873.patch Configure absolute ip in yarn.resourcemanager.nodes.exclude-path and try to decommission the node. It is not getting decommisioned.But if the hostname is given, decommissioning is happening. I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in every machine the other machines ip-host mapping is specified). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming updated MAPREDUCE-3873: -- Attachment: MAPREDUCE-3873.patch Nodemanager is not getting decommisioned if the absolute ip is given in exclude file. - Key: MAPREDUCE-3873 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Nishan Shetty Priority: Minor Attachments: MAPREDUCE-3873.patch Configure absolute ip in yarn.resourcemanager.nodes.exclude-path and try to decommission the node. It is not getting decommisioned.But if the hostname is given, decommissioning is happening. I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in every machine the other machines ip-host mapping is specified). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming updated MAPREDUCE-3873: -- Status: Patch Available (was: Open) The patch will support that we config either hostname or ip in the yarn.resourcemanager.nodes.exclude-path and yarn.resourcemanager.nodes.include-path file. Nodemanager is not getting decommisioned if the absolute ip is given in exclude file. - Key: MAPREDUCE-3873 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Nishan Shetty Priority: Minor Attachments: MAPREDUCE-3873.patch Configure absolute ip in yarn.resourcemanager.nodes.exclude-path and try to decommission the node. It is not getting decommisioned.But if the hostname is given, decommissioning is happening. I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in every machine the other machines ip-host mapping is specified). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4199) Not easy to search the property because of splited into many parts.
xieguiming created MAPREDUCE-4199: - Summary: Not easy to search the property because of splited into many parts. Key: MAPREDUCE-4199 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4199 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.1 Environment: suse Reporter: xieguiming Priority: Minor The property is splited into many parts. such as: public static final String RM_NODES_EXCLUDE_FILE_PATH = RM_PREFIX + nodes.exclude-path; It is inconvenient for code reader to search the code by the property name although there are some common parts for reuse. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[
https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263322#comment-13263322
]
xieguiming commented on MAPREDUCE-3873:
---
The modified function:
public boolean isValidNode(String hostName) {
synchronized (hostsReader) {
SetString hostsList = hostsReader.getHosts();
SetString excludeList = hostsReader.getExcludedHosts();
｛color:red｝
String ip = NetUtils.normalizeHostName(hostName);
{color}
return (hostsList.isEmpty() || hostsList.contains(hostName) || hostsList
.contains(ip))
!(excludeList.contains(hostName) || excludeList.contains(ip));
}
}
and the function NetUtils.normalizeHostName(hostName) can not be mocked.
So, I think this patch do not need provide the unit testcase.
Nodemanager is not getting decommisioned if the absolute ip is given in
exclude file.
-
Key: MAPREDUCE-3873
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.0
Reporter: Nishan Shetty
Priority: Minor
Attachments: MAPREDUCE-3873.patch
Configure absolute ip in yarn.resourcemanager.nodes.exclude-path and try to
decommission the node.
It is not getting decommisioned.But if the hostname is given, decommissioning
is happening.
I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in
every machine the other machines ip-host mapping is specified).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4052) Windows eclpise can not submit the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming reassigned MAPREDUCE-4052: - Assignee: xieguiming Windows eclpise can not submit the job -- Key: MAPREDUCE-4052 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 Project: Hadoop Map/Reduce Issue Type: Bug Components: job submission Affects Versions: 0.23.1 Environment: client on the Windows, the the cluster on the suse Reporter: xieguiming Assignee: xieguiming Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch when I use the eclipse on the windows to submit the job. and the applicationmaster throw the exception: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/v2/app/MRAppMaster Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. The reasion is : class Apps addToEnvironment function, use the private static final String SYSTEM_PATH_SEPARATOR = System.getProperty(path.separator); and will result the MRApplicationMaster classpath use the ; separator. I suggest that nodemanger do the replace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming reassigned MAPREDUCE-3873: - Assignee: xieguiming Nodemanager is not getting decommisioned if the absolute ip is given in exclude file. - Key: MAPREDUCE-3873 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Nishan Shetty Assignee: xieguiming Priority: Minor Attachments: MAPREDUCE-3873.patch Configure absolute ip in yarn.resourcemanager.nodes.exclude-path and try to decommission the node. It is not getting decommisioned.But if the hostname is given, decommissioning is happening. I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in every machine the other machines ip-host mapping is specified). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3540) saveVersion.sh script fails in windows/cygwin (hadoop-yarn-common)
[
https://issues.apache.org/jira/browse/MAPREDUCE-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261771#comment-13261771
]
xieguiming commented on MAPREDUCE-3540:
---
This patch also works my windows env.
Thanks for the patch.
saveVersion.sh script fails in windows/cygwin (hadoop-yarn-common)
--
Key: MAPREDUCE-3540
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3540
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: build
Affects Versions: 0.24.0
Reporter: Alejandro Abdelnur
Fix For: 0.24.0
Attachments: MAPREDUCE-3540.patch
{code}
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec
(generate-version) on project hadoop-yarn-common: Comman
d execution failed. Cannot run program scripts\saveVersion.sh (in directory
C:\cygwin\home\tucu\src\hadoop\hadoop-mapreduce-proje
ct\hadoop-yarn\hadoop-yarn-common): CreateProcess error=2, The system cannot
find the file specified - [Help 1]
[ERROR]
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4052) Windows eclpise can not submit the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260395#comment-13260395 ] xieguiming commented on MAPREDUCE-4052: --- -1 core tests. The patch failed these unit tests: org.apache.hadoop.yarn.server.TestContainerManagerSecurity The faillure of org.apache.hadoop.yarn.server.TestContainerManagerSecurity has not relation with my testcase. Windows eclpise can not submit the job -- Key: MAPREDUCE-4052 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 Project: Hadoop Map/Reduce Issue Type: Bug Components: job submission Affects Versions: 0.23.1 Environment: client on the Windows, the the cluster on the suse Reporter: xieguiming Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch when I use the eclipse on the windows to submit the job. and the applicationmaster throw the exception: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/v2/app/MRAppMaster Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. The reasion is : class Apps addToEnvironment function, use the private static final String SYSTEM_PATH_SEPARATOR = System.getProperty(path.separator); and will result the MRApplicationMaster classpath use the ; separator. I suggest that nodemanger do the replace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4124) Job Priority is not changing
[
https://issues.apache.org/jira/browse/MAPREDUCE-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13260408#comment-13260408
]
xieguiming commented on MAPREDUCE-4124:
---
1, When jobclient create the ApplicationSubmissionContext, set the priority to
the ApplicationSubmissionContext.
{color:red}
Priority pri = Records.newRecord(Priority.class); // Job Priority.
pri.setPriority(jobConf.getInt(JobContext.PRIORITY,
YarnConfiguration.DEFAULT_PRIORITY));
appContext.setPriority(pri);
{color}
appContext.setAMContainerSpec(amContainer); // AM Container
2, Add one interface function in the ClientRMProtocol. and implement the
function:
public void setJobPriority(JobID arg0, String arg1) throws IOException,
InterruptedException {
{color:red}
// call the ClientRMProtocol.setApplicationPriority()
{color}
return;
}
Job Priority is not changing
-
Key: MAPREDUCE-4124
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4124
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0
Reporter: Nishan Shetty
Priority: Critical
1.Submit job
2.Change the job priority using setPriority() or CLI command ./mapred
job-set-priority job-id priority
Observe that Job priority is not changed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4124) Job Priority is not changing
[ https://issues.apache.org/jira/browse/MAPREDUCE-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13261290#comment-13261290 ] xieguiming commented on MAPREDUCE-4124: --- Robert, you are right. the schedulers do not use the priority currently. I think we can change the FIFOschedulers to support the priority firstly. Currently, the FIFOschedulers only sort by submiting time. and we can change to sort by priority firstly, and then by submiting time. Job Priority is not changing - Key: MAPREDUCE-4124 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4124 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 3.0.0 Reporter: Nishan Shetty Priority: Critical 1.Submit job 2.Change the job priority using setPriority() or CLI command ./mapred job-set-priority job-id priority Observe that Job priority is not changed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4052) Windows eclpise can not submit the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming updated MAPREDUCE-4052: -- Status: Open (was: Patch Available) Windows eclpise can not submit the job -- Key: MAPREDUCE-4052 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 Project: Hadoop Map/Reduce Issue Type: Bug Components: job submission Affects Versions: 0.23.1 Environment: client on the Windows, the the cluster on the suse Reporter: xieguiming Attachments: MAPREDUCE-4052.patch when I use the eclipse on the windows to submit the job. and the applicationmaster throw the exception: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/v2/app/MRAppMaster Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. The reasion is : class Apps addToEnvironment function, use the private static final String SYSTEM_PATH_SEPARATOR = System.getProperty(path.separator); and will result the MRApplicationMaster classpath use the ; separator. I suggest that nodemanger do the replace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4052) Windows eclpise can not submit the job
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xieguiming updated MAPREDUCE-4052: -- Attachment: MAPREDUCE-4052-0.patch Windows eclpise can not submit the job -- Key: MAPREDUCE-4052 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 Project: Hadoop Map/Reduce Issue Type: Bug Components: job submission Affects Versions: 0.23.1 Environment: client on the Windows, the the cluster on the suse Reporter: xieguiming Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch when I use the eclipse on the windows to submit the job. and the applicationmaster throw the exception: Exception in thread main java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/v2/app/MRAppMaster Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. The reasion is : class Apps addToEnvironment function, use the private static final String SYSTEM_PATH_SEPARATOR = System.getProperty(path.separator); and will result the MRApplicationMaster classpath use the ; separator. I suggest that nodemanger do the replace. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4052) Windows eclpise can not submit the job
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xieguiming updated MAPREDUCE-4052:
--
Status: Patch Available (was: Open)
Add one testcase
public void testReplaceWindowsPathSeparatorCall() throws Exception {
// replace the windows path separator.
}
Windows eclpise can not submit the job
--
Key: MAPREDUCE-4052
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: job submission
Affects Versions: 0.23.1
Environment: client on the Windows, the the cluster on the suse
Reporter: xieguiming
Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch
when I use the eclipse on the windows to submit the job. and the
applicationmaster throw the exception:
Exception in thread main java.lang.NoClassDefFoundError:
org/apache/hadoop/mapreduce/v2/app/MRAppMaster
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.mapreduce.v2.app.MRAppMaster
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class:
org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
The reasion is :
class Apps addToEnvironment function, use the
private static final String SYSTEM_PATH_SEPARATOR =
System.getProperty(path.separator);
and will result the MRApplicationMaster classpath use the ; separator.
I suggest that nodemanger do the replace.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
