If your jobs are launched by separate jvm instances, the only real
persistence framework you have is hdfs.
You have to basic choices:
1. Write a summary data to a persistent store, an hdfs file being a
simple case, that your next job reads
2. Write the data you need as a job counter,
On Wed, Dec 23, 2009 at 6:55 AM, Jason Venner jason.had...@gmail.comwrote:
If your jobs are launched by separate jvm instances, the only real
persistence framework you have is hdfs.
You have to basic choices:
1. Write a summary data to a persistent store, an hdfs file being a
simple
Hi everyone,
I run multiple map/reduce jobs which are
chained together. The output of one map/reduce is the input of another.
There are also some integer valued variables which are outputted from
one map/reduce job and used as input in the consequent one. These
variables are got by summing up
