YARN - level/depth of monitoring info - newbie question

2017-07-24 Thread rajila2008 .
Hi all, Does YARN provide application level info ?! For example : there is a map-reduce job persisting its outcome in a NoSql datastore by executing an INSERT command. Can YARN provide the execution time for the INSERT ? without the application itself is not logging the info anywhere. There's

FATAL mapred.MesosScheduler: Failed to initialize the TaskScheduler

2017-07-24 Thread Traiano Welcome
Hi List I'm working on configuring hadoop to use the mesos scheduler, using the procedure outlined in "Apache Mesos Essentials" here: https://pastebin.com/y1ERJZqq Currently I've a 3 node mesos cluster, with an HDFS namenode communicating successfully with two HDFS data nodes. However, when I

Re: Regarding Simulation of Hadoop

2017-07-24 Thread Ravi Prakash
Hi Vinod! Could you please describe the "Hadoop Security Framework"? I am not sure what you mean by it. What kind of tests do you want to run? You could try Amazon / Azure / GCE instances fairly cheaply. Or you could use virtual machines. In the past I have run 2 worker nodes isolated only by

subscribe

2017-07-24 Thread Traiano Welcome

Re: Kerberised JobHistory Server not starting: User jhs trying to create the /mr-history/done directory

2017-07-24 Thread Erik Krogen
Hey Kevin, Sorry, I missed your point about using auth_to_local. You're right that you should be able to use that for what you're trying to achieve. I think it's just that your rule is wrong; I believe it should be: RULE:[2:$1@$0](jhs/.*@ECS.VUW.AC.NZ)s/.*/mapred/ HTH Erik On 7/23/17,

How to make JobHistoryServer support HighAvailbility?

2017-07-24 Thread Chang.Wu
I want to configure two JobHistoryServers and use nginx as a HA.My gold is that even when on of my JobHistoryServer crashed , my oozie jobs can still work without any influence. I doubt if I can do so. If not , what can I do to make JobHistoryServer support HighAvailability? Many thanks.