Re: Creating MapFile.Reader instance in reducer setup
Hello, sorry my mistake. Problem solved. On 06/19/2012 03:40 PM, Devaraj k wrote: Can you share the exception stack trace and piece of code where you are trying to create? Thanks Devaraj From: Ondřej Klimpera [klimp...@fit.cvut.cz] Sent: Tuesday, June 19, 2012 6:03 PM To: common-user@hadoop.apache.org Subject: Creating MapFile.Reader instance in reducer setup Hello, I'm tring to use MapFile (stored on HDFS) in my reduce task, which processes some text data. When I try to initialize MapFile.Reader in reducer configure() method, app throws NullPointerException, when the same approach is used for each reduce() method call with the same parameters, everything goes all right. But creating instance of Reader for each reduce() call creates big slow down. Do you have any idea what am I doing wrong. Thanks Ondrej Klimpera
Re: Creating MapFile.Reader instance in reducer setup
Hi,
> But creating instance of Reader for each reduce() call creates big slow
> down.
It wouldn't, if you wrapped it properly:
if (reader != null) {
// Initialize reader
reader = new MapFile.Reader(…)
}
This way it only initializes/reads/etc. once.
On Tue, Jun 19, 2012 at 6:03 PM, Ondřej Klimpera wrote:
> Hello, I'm tring to use MapFile (stored on HDFS) in my reduce task, which
> processes some text data.
>
> When I try to initialize MapFile.Reader in reducer configure() method, app
> throws NullPointerException, when the same approach is used for each
> reduce() method call with the same parameters, everything goes all right.
>
> But creating instance of Reader for each reduce() call creates big slow
> down.
>
> Do you have any idea what am I doing wrong.
>
> Thanks
> Ondrej Klimpera
>
>
--
Harsh J
RE: Creating MapFile.Reader instance in reducer setup
Can you share the exception stack trace and piece of code where you are trying to create? Thanks Devaraj From: Ondřej Klimpera [klimp...@fit.cvut.cz] Sent: Tuesday, June 19, 2012 6:03 PM To: common-user@hadoop.apache.org Subject: Creating MapFile.Reader instance in reducer setup Hello, I'm tring to use MapFile (stored on HDFS) in my reduce task, which processes some text data. When I try to initialize MapFile.Reader in reducer configure() method, app throws NullPointerException, when the same approach is used for each reduce() method call with the same parameters, everything goes all right. But creating instance of Reader for each reduce() call creates big slow down. Do you have any idea what am I doing wrong. Thanks Ondrej Klimpera
Creating MapFile.Reader instance in reducer setup
Hello, I'm tring to use MapFile (stored on HDFS) in my reduce task, which processes some text data. When I try to initialize MapFile.Reader in reducer configure() method, app throws NullPointerException, when the same approach is used for each reduce() method call with the same parameters, everything goes all right. But creating instance of Reader for each reduce() call creates big slow down. Do you have any idea what am I doing wrong. Thanks Ondrej Klimpera
