Ok, will do! Johannes
On 08 Oct 2014, at 21:56, Bikas Saha <[email protected]> wrote: > We would ideally like to the see Tez do a good job about reporting errors. > So if you find cases where this is not happening then please open jiras > for them. Users should not have to work around Tez issues. > > Bikas > > -----Original Message----- > From: Johannes Zillmann [mailto:[email protected]] > Sent: Tuesday, October 07, 2014 11:31 PM > To: [email protected] > Subject: Re: post task hook > > Hey Jeff, > > so the reason for copying the task logs is indeed better error-diagnostic. > MapReduce/Tez usually only reported 30 to 50% of the exceptions to the > client in my experience. > So if all of TEZ-1240 is done that might not be much an issue any more. > > thanks > Johannes > > > On 08 Oct 2014, at 08:24, Jianfeng (Jeff) Zhang <[email protected]> > wrote: > >> Hi Johannes, >> >> Currently you can see the diagnostics in client if the task fails in > processor, is that what you want ? >> >> Here's the jira tracking this, > https://issues.apache.org/jira/browse/TEZ-1240 >> >> If you find any exception that is not caught, please create a ticket > under this. >> >> >> >> Best Regards, >> Jeff Zhang >> >> >> On Wed, Oct 8, 2014 at 2:08 PM, Johannes Zillmann > <[email protected]> wrote: >> Hey Mr. Zang, >> >> so main use case is fetching the tasks logs in case the task fails. I > can do that in a try-catch blog in the processor itself but it has 2 > disadvantages: >> - log might not be complete >> - in case the task fails in not user provided code (outside of the > processor), we don't capture it >> >> Johannes >> >> On 08 Oct 2014, at 01:58, Jianfeng (Jeff) Zhang <[email protected]> > wrote: >> >>> Hi Johannes, >>> >>> You can do some post task in Processor, please refer SimpleProcessor > which has method postOp(), but can do a limited things and can not do > things like accessing task logs. >>> Could you let us know your purpose on customize the post task hook ? >>> >>> >>> Best Regards, >>> Jeff Zhang >>> >>> >>> On Thu, Oct 2, 2014 at 4:20 PM, Johannes Zillmann > <[email protected]> wrote: >>> Hey guys, >>> >>> is there any post task hook in Tez (like the OutputCommitter in > MapReduce had) ? >>> Like to do certain actions (like accessing task logs) once the task > completes, whether or not the task was successful and whether or not the > user provided processor got executed or the task failed even before. >>> >>> Johannes >>> >>> >>> CONFIDENTIALITY NOTICE >>> NOTICE: This message is intended for the use of the individual or > entity to which it is addressed and may contain information that is > confidential, privileged and exempt from disclosure under applicable law. > If the reader of this message is not the intended recipient, you are > hereby notified that any printing, copying, dissemination, distribution, > disclosure or forwarding of this communication is strictly prohibited. If > you have received this communication in error, please contact the sender > immediately and delete it from your system. Thank You. >> >> >> >> CONFIDENTIALITY NOTICE >> NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified > that any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender > immediately and delete it from your system. Thank You. > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
