Thanks a lot for the clarification! I tried to find the log and history information about finished jobs. But they are not in hfs://xxx/user/myusername/output/_SUCCESS (0B). Can you please give some pointers on where the statistical/job history files are located? The hfs://xxxx/history/done only stores history files up to 7 days.
Thanks, Boyu On Mon, Sep 21, 2015 at 1:23 PM, Varun Saxena <vsaxena.va...@gmail.com> wrote: > No, you cant show them in RM UI then. > > However if you can start another daemon, you can consider using YARN > Application History/Timeline Server or MR Job History Server(only for MR > jobs) to see information about completed jobs. > You can look up Hadoop documentation to learn more about them and how to > configure them. > > Just to clarify though, the apps themselves are not lost, as in, the > output is not lost. Its just the information about them which is no longer > present on RM restart. > > Regards, > Varun Saxena. > > On Mon, Sep 21, 2015 at 10:31 PM, Boyu Zhang <boyuzhan...@gmail.com> > wrote: > >> Thanks for the answer Varun. >> >> It is the case that yarn.resourcemanager.recovery.enabled is set to be >> false. Is there a way to show the jobs that are submitted before the >> restart? We don't want to lose that data. >> >> Thanks, >> Boyu >> >> >> On Mon, Sep 21, 2015 at 12:53 PM, Varun Saxena <vsaxena.va...@gmail.com> >> wrote: >> >>> Hi Boyu, >>> >>> RM stores apps in state store if recovery is enabled. Only then they >>> will be available on restart. >>> Otherwise they are kept in memory and hence lost on restart. >>> >>> You may not have it enabled. Check config value for below config. By >>> default its false. >>> yarn.resourcemanager.recovery.enabled >>> >>> Regards, >>> Varun. >>> >>> On Mon, Sep 21, 2015 at 10:01 PM, Boyu Zhang <boyuzhan...@gmail.com> >>> wrote: >>> >>>> Hello Everyone, >>>> >>>> I have a strange error regarding the ResourceManager web UI ( >>>> http://xx.xx:8088). >>>> >>>> Someone before me set up the hadoop + yarn cluster using Pivotal HD, it >>>> was running fine. Then today, the resource manager and node manager >>>> disappeared, the logs did not record this. I restarted them, they are up >>>> and running, but the resource manger web UI does not show any jobs. We have >>>> 700+ jobs in the past, and they were showing before. >>>> >>>> If I submit MapReduce jobs, the new submitted ones show up. But the >>>> disappear again after restart the resource manger and node manager. >>>> >>>> Can anyone give any hint on where to look? >>>> >>>> Thanks in advance, >>>> Boyu >>>> >>>> >>> >> >
