Re: Running YARN on top of legacy HDFS (i.e. 0.20)

Avery, If you could take a look at what it would take, I'd be grateful. I'm hoping it isn't very much effort. thanks, Arun On Dec 6, 2011, at 10:05 AM, Avery Ching wrote: > I think it would be nice if YARN could work on existing older HDFS instances, > a lot of folks will be slow to upgrade H

Re: Running YARN on top of legacy HDFS (i.e. 0.20)

I think it would be nice if YARN could work on existing older HDFS instances, a lot of folks will be slow to upgrade HDFS with all their important data on it. I could also go that route I guess. Avery On 12/6/11 8:51 AM, Arun C Murthy wrote: Avery, They aren't 'api changes'. HDFS just has

Re: reduce tasks pending - no progress for few hours

A "Child" is basically your Map or Reduce JVM. It is spawned by the TaskTracker. I assume that there is just this job running, yes? Can you check if any of your TaskTrackers have been blacklisted? Also, can you pastebin your day's JobTracker log, grepped for this specific Job ID? You could also

Re: reduce tasks pending - no progress for few hours

Hello Harsh, 1. All tasktracker are up, they are 10 available. What do you reckon? 2. On one node, there is additionally a process called Child. It seems like a tasktracker, so I am not sure why both Child and tasktracker are running on that node? Thanks, Keren On Tue, Dec 6, 2011 at 11:26 AM,

Re: Running YARN on top of legacy HDFS (i.e. 0.20)

Avery, They aren't 'api changes'. HDFS just has a new set of apis in hadoop-0.23 (aka FileContext apis). Both the old (FileSystem apis) and new are supported in hadoop-0.23. We have used the new HDFS apis in YARN in some places. hth, Arun On Dec 5, 2011, at 10:59 PM, Avery Ching wrote: >

Re: reduce tasks pending - no progress for few hours

Keren, How many tasktrackers are you running, and are they all still up? On 06-Dec-2011, at 9:32 PM, Keren Ouaknine wrote: > Hello, > > Please find details below. I would like to resume the pending tasks (not sure > why they went into pending state in the first place). > I had a look at the lo

reduce tasks pending - no progress for few hours

Hello, Please find details below. I would like to resume the pending tasks (not sure why they went into pending state in the first place). I had a look at the logs, there were some tasks failures (3), but successful on a second attempt. * Job Setup:* Successful

Re: how to access a mapper counter in reducer

Hi Praveen, Could you share here so that we can use ? Thanks, Sent from my iPhone On Dec 6, 2011, at 6:29 AM, Praveen Sripati wrote: > Robert, > > > I have made the above thing work. > > Any plans to make it into the Hadoop framework. There had been similar > queries about it in other forum

Re: how to access a mapper counter in reducer

Robert, > I have made the above thing work. Any plans to make it into the Hadoop framework. There had been similar queries about it in other forums also. Need any help testing/documenting or anything, please let me know. Regards, Praveen On Sat, Dec 3, 2011 at 2:34 AM, Robert Evans wrote: >