Re: Accessing the reduce key

2014-03-20 Thread Mayur Rustagi
. In Spark, we can do something similar with map() and reduceByKey() but we have the following questions. 1. Accessing the reduce key In reduceByKey(), how do we get access to the specific key within the reduce function? 2. Equivalent of setup/cleanup Where should we instantiate and persist each

Re: Accessing the reduce key

2014-03-20 Thread Surendranauth Hiraman
. 1. Accessing the reduce key In reduceByKey(), how do we get access to the specific key within the reduce function? 2. Equivalent of setup/cleanup Where should we instantiate and persist each Bloom Filter by key? In the driver and then pass in the references to the reduce function? But if so

Re: Accessing the reduce key

2014-03-20 Thread Surendranauth Hiraman
. Accessing the reduce key In reduceByKey(), how do we get access to the specific key within the reduce function? 2. Equivalent of setup/cleanup Where should we instantiate and persist each Bloom Filter by key? In the driver and then pass in the references to the reduce function? But if so, how