I guess the ERROR lines in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client- jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2. TestMRJobsWithHistoryService-output.txt
led me to believe that the problem was with TestMRJobsWithHistoryService. If that's not the case then what do these messages indicate? As I say I am a complete novice and the learning curve is very steep. On 7 February 2014 14:47, Ted Yu <yuzhih...@gmail.com> wrote: > I checked out source code from > http://svn.apache.org/repos/asf/hadoop/common/branches/branch-2.3 and it > builds. > > From TestMRJobsWithHistoryService.txt, the test passed. > What led to this test being singled out among the 454 tests ? > > Thanks > > > On Thu, Feb 6, 2014 at 7:26 PM, Christopher Thomas < > christophermauricetho...@gmail.com> wrote: > >> Yes well I tried 2.3, but I have found a number of problems building it. >> I had to resort to manually applying patches that I found in the bug >> tracking lists, which did not seem to have made it into all branches. So >> for the moment I am sticking with 2.2.0 which is advertised as being >> stable. >> >> I apologise for the confusion. >> >> Here is the contents >> of >> ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt, >> though perhaps not that illuminating: >> >> >> >> ------------------------------------------------------------------------------- >> Test set: org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService >> >> ------------------------------------------------------------------------------- >> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.669 sec >> >> >> >> >> On 7 February 2014 14:12, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> The output was >>> from >>> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-output.txt >>> >>> Can you show us the contents >>> of >>> ./hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt >>> ? >>> >>> BTW hadoop 2.3 release candidate is coming up. You may consider paying >>> more attention to hadoop 2.3 >>> >>> Cheers >>> >>> >>> On Thu, Feb 6, 2014 at 5:33 PM, Christopher Thomas < >>> christophermauricetho...@gmail.com> wrote: >>> >>>> I included the last part of >>>> >>>> hadoop-mapreduce-project/hadoop-mapreduce-client/ >>>> hadoop-mapreduce-client-jobclient/target/surefire- >>>> reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService. >>>> txt >>>> >>>> in the second half of my initial posting, calling it "the output from >>>> TestMRJobsWithHistoryService". >>>> Sloppy terminology I know, sorry if I wasn't very clear. >>>> >>>> Regards >>>> >>>> Chris >>>> >>>> >>>> On 7 February 2014 11:53, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> There isn't System.exit call in TestMRJobsWithHistoryService.java >>>>> >>>>> What >>>>> did >>>>> hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire-reports/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService.txt >>>>> say ? >>>>> >>>>> Cheers >>>>> >>>>> >>>>> On Thu, Feb 6, 2014 at 4:41 PM, Christopher Thomas < >>>>> christophermauricetho...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am a complete beginner to Hadoop, trying to build 2.2.0 from source >>>>>> on a Macbook Pro running OS X Mavericks. >>>>>> >>>>>> I am following the 'instructions' at >>>>>> http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html >>>>>> such >>>>>> as they are. >>>>>> >>>>>> I get the following test failure: >>>>>> >>>>>> Forking command line: /bin/sh -c cd >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient >>>>>> && >>>>>> /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home/bin/java >>>>>> -Xmx1024m -XX:+HeapDumpOnOutOfMemoryError -jar >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefirebooter1837947962445626736.jar >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire8537321961333093423tmp >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/surefire/surefire_1668661782666182940486tmp >>>>>> Running org.apache.hadoop.mapreduce.v2.TestMROldApiJobs >>>>>> >>>>>> Results : >>>>>> >>>>>> Tests run: 454, Failures: 0, Errors: 0, Skipped: 11 >>>>>> >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [INFO] Reactor Summary: >>>>>> [INFO] >>>>>> [INFO] hadoop-mapreduce-client-jobclient ................. FAILURE >>>>>> [1:18:07.440s] >>>>>> [INFO] hadoop-mapreduce-client-hs-plugins ................ SKIPPED >>>>>> [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED >>>>>> [INFO] hadoop-mapreduce .................................. SKIPPED >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [INFO] BUILD FAILURE >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [INFO] Total time: 1:18:07.828s >>>>>> [INFO] Finished at: Thu Feb 06 23:01:45 EST 2014 >>>>>> [INFO] Final Memory: 32M/94M >>>>>> [INFO] >>>>>> ------------------------------------------------------------------------ >>>>>> [ERROR] Failed to execute goal >>>>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) >>>>>> on project hadoop-mapreduce-client-jobclient: ExecutionException; nested >>>>>> exception is java.util.concurrent.ExecutionException: >>>>>> java.lang.RuntimeException: The forked VM terminated without saying >>>>>> properly goodbye. VM crash or System.exit called ? >>>>>> >>>>>> >>>>>> and the output from TestMRJobsWithHistoryService includes: >>>>>> >>>>>> >>>>>> 2014-02-06 23:00:54,530 ERROR [Thread[Thread-252,5,main]] >>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - >>>>>> InterruptedExcpetion >>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedException: >>>>>> sleep interrupted >>>>>> 2014-02-06 23:00:54,531 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55131 >>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server listener on 55131] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55131 >>>>>> 2014-02-06 23:00:54,540 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:00:54,544 INFO [main] mortbay.log >>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>> SelectChannelConnector@Christophers-MacBook-Pro.local:0 >>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>> (JobHistory.java:serviceStop(136)) - Stopping JobHistory >>>>>> 2014-02-06 23:00:54,647 INFO [main] hs.JobHistory >>>>>> (JobHistory.java:serviceStop(138)) - Stopping History Cleaner/Move To >>>>>> Done >>>>>> 2014-02-06 23:00:54,656 INFO [main] mortbay.log >>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>> SelectChannelConnector@Christophers-MacBook-Pro.local:0 >>>>>> 2014-02-06 23:00:54,759 INFO [main] >>>>>> containermanager.ContainerManagerImpl >>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(328)) - >>>>>> Applications still running : [application_1391688017341_0001] >>>>>> 2014-02-06 23:00:54,762 INFO [main] >>>>>> containermanager.ContainerManagerImpl >>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(336)) - >>>>>> Waiting >>>>>> for Applications to be Finished >>>>>> 2014-02-06 23:00:54,762 INFO [AsyncDispatcher event handler] >>>>>> application.Application (ApplicationImpl.java:handle(451)) - Application >>>>>> application_1391688017341_0001 transitioned from RUNNING to >>>>>> APPLICATION_RESOURCES_CLEANINGUP >>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #3] >>>>>> nodemanager.DefaultContainerExecutor >>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>> path >>>>>> : >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_0/usercache/root/appcache/application_1391688017341_0001 >>>>>> 2014-02-06 23:00:54,763 INFO [DeletionService #0] >>>>>> nodemanager.DefaultContainerExecutor >>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>> path >>>>>> : >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_1/usercache/root/appcache/application_1391688017341_0001 >>>>>> 2014-02-06 23:00:54,765 INFO [AsyncDispatcher event handler] >>>>>> containermanager.AuxServices (AuxServices.java:handle(175)) - Got event >>>>>> APPLICATION_STOP for appId application_1391688017341_0001 >>>>>> 2014-02-06 23:00:54,765 INFO [DeletionService #1] >>>>>> nodemanager.DefaultContainerExecutor >>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>> path >>>>>> : >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_2/usercache/root/appcache/application_1391688017341_0001 >>>>>> 2014-02-06 23:00:54,766 INFO [AsyncDispatcher event handler] >>>>>> application.Application (ApplicationImpl.java:handle(451)) - Application >>>>>> application_1391688017341_0001 transitioned from >>>>>> APPLICATION_RESOURCES_CLEANINGUP to FINISHED >>>>>> 2014-02-06 23:00:54,767 INFO [AsyncDispatcher event handler] >>>>>> loghandler.NonAggregatingLogHandler >>>>>> (NonAggregatingLogHandler.java:handle(118)) - Scheduling Log Deletion for >>>>>> application: application_1391688017341_0001, with delay of 10800 seconds >>>>>> 2014-02-06 23:00:54,768 INFO [DeletionService #3] >>>>>> nodemanager.DefaultContainerExecutor >>>>>> (DefaultContainerExecutor.java:deleteAsUser(368)) - Deleting absolute >>>>>> path >>>>>> : >>>>>> /Users/hadoop/hadoop-2.2.0-src/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService/org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService-localDir-nm-0_3/usercache/root/appcache/application_1391688017341_0001 >>>>>> 2014-02-06 23:00:58,766 INFO [main] >>>>>> containermanager.ContainerManagerImpl >>>>>> (ContainerManagerImpl.java:cleanUpApplicationsOnNMShutDown(353)) - Done >>>>>> waiting for Applications to be Finished. Still alive: >>>>>> [application_1391688017341_0001] >>>>>> 2014-02-06 23:00:58,768 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55125 >>>>>> 2014-02-06 23:00:58,771 INFO [IPC Server listener on 55125] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55125 >>>>>> 2014-02-06 23:00:58,772 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:01:08,774 INFO [main] monitor.ContainersMonitorImpl >>>>>> (ContainersMonitorImpl.java:isEnabled(159)) - ResourceCalculatorPlugin is >>>>>> unavailable on this system. >>>>>> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl >>>>>> is disabled. >>>>>> 2014-02-06 23:01:08,784 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55126 >>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server listener on 55126] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55126 >>>>>> 2014-02-06 23:01:08,785 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:01:08,785 INFO [Public Localizer] >>>>>> localizer.ResourceLocalizationService >>>>>> (ResourceLocalizationService.java:run(728)) - Public cache exiting >>>>>> 2014-02-06 23:01:08,787 INFO [main] mortbay.log >>>>>> (Slf4jLog.java:info(67)) - Stopped >>>>>> SelectChannelConnector@Christophers-MacBook-Pro.local:0 >>>>>> 2014-02-06 23:01:08,889 ERROR [Thread[Thread-17,5,main]] >>>>>> delegation.AbstractDelegationTokenSecretManager >>>>>> (AbstractDelegationTokenSecretManager.java:run(547)) - >>>>>> InterruptedExcpetion >>>>>> recieved for ExpiredTokenRemover thread java.lang.InterruptedException: >>>>>> sleep interrupted >>>>>> 2014-02-06 23:01:08,891 INFO [main] impl.MetricsSystemImpl >>>>>> (MetricsSystemImpl.java:stop(200)) - Stopping JobHistoryServer metrics >>>>>> system... >>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>> (MetricsSystemImpl.java:stop(206)) - JobHistoryServer metrics system >>>>>> stopped. >>>>>> 2014-02-06 23:01:08,895 INFO [main] impl.MetricsSystemImpl >>>>>> (MetricsSystemImpl.java:shutdown(572)) - JobHistoryServer metrics system >>>>>> shutdown complete. >>>>>> 2014-02-06 23:01:08,896 WARN [ApplicationMaster Launcher] >>>>>> amlauncher.ApplicationMasterLauncher >>>>>> (ApplicationMasterLauncher.java:run(98)) - >>>>>> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.ApplicationMasterLauncher$LauncherThread >>>>>> interrupted. Returning. >>>>>> 2014-02-06 23:01:08,897 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55122 >>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server listener on 55122] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55122 >>>>>> 2014-02-06 23:01:08,898 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55121 >>>>>> 2014-02-06 23:01:08,898 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:01:08,902 INFO [IPC Server listener on 55121] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55121 >>>>>> 2014-02-06 23:01:08,908 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:01:08,902 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55120 >>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server listener on 55120] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55120 >>>>>> 2014-02-06 23:01:08,912 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:01:08,912 INFO [main] ipc.Server >>>>>> (Server.java:stop(2437)) - Stopping server on 55119 >>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server listener on 55119] >>>>>> ipc.Server (Server.java:run(720)) - Stopping IPC Server listener on 55119 >>>>>> 2014-02-06 23:01:08,917 INFO [Ping Checker] >>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) >>>>>> - >>>>>> NMLivelinessMonitor thread interrupted >>>>>> 2014-02-06 23:01:08,917 ERROR [ResourceManager Event Processor] >>>>>> resourcemanager.ResourceManager (ResourceManager.java:run(435)) - >>>>>> Returning, interrupted : java.lang.InterruptedException >>>>>> 2014-02-06 23:01:08,917 INFO [IPC Server Responder] ipc.Server >>>>>> (Server.java:run(866)) - Stopping IPC Server Responder >>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) >>>>>> - >>>>>> AMLivelinessMonitor thread interrupted >>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) >>>>>> - >>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.ContainerAllocationExpirer >>>>>> thread interrupted >>>>>> 2014-02-06 23:01:08,918 INFO [Ping Checker] >>>>>> util.AbstractLivelinessMonitor (AbstractLivelinessMonitor.java:run(127)) >>>>>> - >>>>>> AMLivelinessMonitor thread interrupted >>>>>> >>>>>> Has anyone seen this error before when building hadoop, and does >>>>>> anyone have any ideas? >>>>>> >>>>>> Regards >>>>>> >>>>>> Chris >>>>>> >>>>> >>>>> >>>> >>> >> >
