Re: Job end notification does not always work (Hadoop 2.x)

2013-06-25 Thread Alejandro Abdelnur
S also. > > ** ** > > Thanks > > Devaraj k > > ** ** > > *From:* Alejandro Abdelnur [mailto:t...@cloudera.com] > *Sent:* 25 June 2013 18:05 > *To:* common-u...@hadoop.apache.org > > *Subject:* Re: Job end notification does not alw

RE: Job end notification does not always work (Hadoop 2.x)

2013-06-25 Thread Devaraj k
I agree, for getting status/counters we need HS. I mean Job can finish without HS also. Thanks Devaraj k From: Alejandro Abdelnur [mailto:t...@cloudera.com] Sent: 25 June 2013 18:05 To: common-u...@hadoop.apache.org Subject: Re: Job end notification does not always work (Hadoop 2.x) Devaraj

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-25 Thread Alejandro Abdelnur
13 21:42 > *To:* user@hadoop.apache.org > *Cc:* user@hadoop.apache.org > > *Subject:* Re: Job end notification does not always work (Hadoop 2.x) > > ** ** > > if we ought to do this in a yarn service it > should be the RM or the HS. the RM is, IMO, the natural fit. the H

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-24 Thread Prashant Kommireddi
> > > Thanks > > Devaraj K**** > > **** > > *From:* Ravi Prakash [mailto:ravi...@ymail.com ] > *Sent:* 23 June 2013 19:01 > *To:* user@hadoop.apache.org > *Subject:* Re: Job end notification does not always work (Hadoop 2.x) > > > >

RE: Job end notification does not always work (Hadoop 2.x)

2013-06-24 Thread Devaraj k
: user@hadoop.apache.org Subject: Re: Job end notification does not always work (Hadoop 2.x) if we ought to do this in a yarn service it should be the RM or the HS. the RM is, IMO, the natural fit. the HS, would be a good choice if we are concerned about the extra work this would cause in the RM. the

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-24 Thread Alejandro Abdelnur
> types of applications as well. > > > Thanks > Devaraj K > > From: Ravi Prakash [mailto:ravi...@ymail.com] > Sent: 23 June 2013 19:01 > To: user@hadoop.apache.org > Subject: Re: Job end notification does not always work (Hadoop 2.x) > > Hi Alejandro, &

RE: Job end notification does not always work (Hadoop 2.x)

2013-06-23 Thread Devaraj k
user@hadoop.apache.org Subject: Re: Job end notification does not always work (Hadoop 2.x) Hi Alejandro, Thanks for your reply! I was thinking more along the lines Prashant suggested i.e. a failure during init() should still trigger an attempt to notify (by the AM). But now that you mention it, may

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-23 Thread Ravi Prakash
2, 2013 7:37 PM Subject: Re: Job end notification does not always work (Hadoop 2.x) If the AM fails before doing the job end notification, at any stage of the execution for whatever reason, the job end notification will never be deliver. There is not way to fix this unless the notification is

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Alejandro Abdelnur
>> example when the AM OOMs), I agree with you that we can do more. If you >> feel strongly about this, please create a JIRA and possibly upload a patch. >> >> Thanks >> Ravi >> >> >> -------------- >> *From:* Prashant Kommi

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Arun C Murthy
possibly upload a patch. > > Thanks > Ravi > > > From: Prashant Kommireddi > To: "user@hadoop.apache.org" > Sent: Thursday, June 20, 2013 9:45 PM > Subject: Job end notification does not always work (Hadoop 2.x) > > Hello, > > I came across an i

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Prashant Kommireddi
op.apache.org" > *Sent:* Thursday, June 20, 2013 9:45 PM > *Subject:* Job end notification does not always work (Hadoop 2.x) > > Hello, > > I came across an issue that occurs with the job notification callbacks in > MR2. It works fine if the Application master has started, but do

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Ravi Prakash
d possibly upload a patch. Thanks Ravi From: Prashant Kommireddi To: "user@hadoop.apache.org" Sent: Thursday, June 20, 2013 9:45 PM Subject: Job end notification does not always work (Hadoop 2.x) Hello, I came across an issue that occurs with

Re: Job end notification does not always work (Hadoop 2.x)

2013-06-22 Thread Prashant Kommireddi
Following-up on this. Please let me know if this is expected/bug and if you would like me to file a JIRA> On Thu, Jun 20, 2013 at 9:45 PM, Prashant Kommireddi wrote: > Hello, > > I came across an issue that occurs with the job notification callbacks in > MR2. It works fine if the Application mas

Job end notification does not always work (Hadoop 2.x)

2013-06-20 Thread Prashant Kommireddi
Hello, I came across an issue that occurs with the job notification callbacks in MR2. It works fine if the Application master has started, but does not send a callback if the initializing of AM fails. Here is the code from MRAppMaster.java . ... // set job classloader if configure