What are the methods to share dynamic data among mappers/reducers?

2014-01-02 Thread sam liu
Hi, As I know, the Distributed Cache will copy the shared data to the slaves before starting job, and won't change the shared data after that. So are there any solutions to share dynamic data among mappers/reducers? Thanks!

Re: What are the methods to share dynamic data among mappers/reducers?

2014-01-02 Thread Vinod Kumar Vavilapalli
There isn't anything natively supported for that in the framework, but you can do that yourselves by using a shared service (for e.g via HDFS files, ZooKeeper nodes) that mappers/reducers all have access to. More details on your usecase? In any case, once you start making mappers and reducers