Hi, everyone. Is there someone who uses map-reduce to store the reduce output
in memory. I mean, now the output path of job is set and reduce outputs are
stored into files under this path.(see the comments along with the following
codes)
job.setOutputFormatClass(MyOutputFormat.class);
//can I implement my OutputFormat to store these output key-value pairs in
my data structures, or are these other ways to do it?
job.setOutputKeyClass(ImmutableBytesWritable.class);
job.setOutputValueClass(Result.class);
FileOutputFormat.setOutputPath(job, outputDir);
Is there any way to store them in some variables or data structures? Then how
can I implement my OutputFormat? Any suggestions and codes are welcomed.
Another question: is there some way to set the number of map task? It seems
there is no API to do this in hadoop new job APIs. I am not sure the way to set
this number.
Thanks!
Best Wishes!
_____________________________________________________________
刘祥龙 Liu Xianglong
