Re: muti-thread mapreduce

2012-12-26 Thread satish verma
Hi I am using a MR job on Hbase . I am getting this sort of error on a few region servers. Can someone explain what this means and what should I do ? Thanks. 12/12/26 03:03:18 INFO mapred.JobClient: Task Id : attempt_201210121702_699686_m_46_0, Status : FAILED

Re: muti-thread mapreduce

2012-12-26 Thread Harsh J
Satish, 1. Please ask HBase questions on HBase's own lists (u...@hbase.apache.org). 2. Please start a new thread for new questions of yours, and not post on an existing thread that is for some other topic. 3. The error is as it says, that your mapper could not successfully access a specific

Re: muti-thread mapreduce

2012-12-13 Thread Yu Yang
Thank you all. In fact, I don't expect that this way can help to enhance the performance. I need to process 3 different logs (with different format). I just want to sart all these 3 logs processing at the same time , all in just this one program. but I can give different separator to each

Re: muti-thread mapreduce

2012-12-13 Thread Harsh J
I suppose you could also leverage job configuration or per-input mapper impl. via MultipleInputs to do this. On Thu, Dec 13, 2012 at 5:44 PM, Yu Yang clouder...@gmail.com wrote: Thank you all. In fact, I don't expect that this way can help to enhance the performance. I need to process 3

Re: muti-thread mapreduce

2012-12-12 Thread Harsh J
Exactly - A job is already designed to be properly parallel w.r.t. its input, and this would just add additional overheads of job setup and scheduling. If your per-record processing requires threaded work, consider using the MultithreadedMapper/Reducer classes instead. On Wed, Dec 12, 2012 at