Hi Liu, Why you want to store the output in memory? You can not use the output out of reducer. Actually at the beginning the output of reducer is in memory, and the OutputFormat write these data to file system or other data store.
Jeff Zhang 2009/11/27 Liu Xianglong <[email protected]> > Hi, everyone. Is there someone who uses map-reduce to store the reduce > output in memory. I mean, now the output path of job is set and reduce > outputs are stored into files under this path.(see the comments along with > the following codes) > job.setOutputFormatClass(MyOutputFormat.class); > //can I implement my OutputFormat to store these output key-value pairs > in my data structures, or are these other ways to do it? > job.setOutputKeyClass(ImmutableBytesWritable.class); > job.setOutputValueClass(Result.class); > FileOutputFormat.setOutputPath(job, outputDir); > > Is there any way to store them in some variables or data structures? Then > how can I implement my OutputFormat? Any suggestions and codes are welcomed. > > Another question: is there some way to set the number of map task? It seems > there is no API to do this in hadoop new job APIs. I am not sure the way to > set this number. > > Thanks! > > Best Wishes! > _____________________________________________________________ > > 刘祥龙 Liu Xianglong >
