Hi,
Are tasks being executed multiple times due to failures? Sorry, it was not
very clear from your question.
Thanks
hemanth
On Sat, Jan 5, 2013 at 7:44 PM, David Parks <davidpark...@yahoo.com> wrote:
> Thinking here... if you submitted the task programmatically you should be
> able to capture the failure of the task and gracefully move past it to your
> next tasks.
>
> To say it in a long-winded way: Let's say you submit a job to Hadoop, a
> java jar, and your main class implements Tool. That code has the
> responsibility to submit a series of jobs to hadoop, something like this:
>
> try{
> Job myJob = new MyJob(getConf());
> myJob.submitAndWait();
> }catch(Exception uhhohh){
> //Deal with the issue and move on
> }
> Job myNextJob = new MyNextJob(getConf());
> myNextJob.submit();
>
> Just pseudo code there to demonstrate my thought.
>
> David
>
>
>
> -----Original Message-----
> From: Håvard Wahl Kongsgård [mailto:haavard.kongsga...@gmail.com]
> Sent: Saturday, January 05, 2013 4:54 PM
> To: user
> Subject: Skipping entire task
>
> Hi, hadoop can skip bad records
>
> http://devblog.factual.com/practical-hadoop-streaming-dealing-with-brittle-c
> ode.
> But it is also possible to skip entire tasks?
>
> -Håvard
>
> --
> Håvard Wahl Kongsgård
> Faculty of Medicine &
> Department of Mathematical Sciences
> NTNU
>
> http://havard.security-review.net/
>
>
