Re: Usefulness of ChainMapper/ChainReducer

2012-09-28 Thread Harsh J
Hi, Modularity! I've always had the same question before. However, Tom White put that thought to rest: "It’s possible to make map and reduce functions even more composable than we have done. A mapper commonly performs input format parsing, projection (selecting the relevant fields), and filterin

Re: Usefulness of ChainMapper/ChainReducer

2012-09-28 Thread John Armstrong
On Fri 28 Sep 2012 09:39:13 AM EDT, Harsh J wrote: Modularity! Exactly! Write a mapper that operates as a filter on something about your keys, then use it in whatever jobs you want. Your job needs to operate on data subset A? chain it with the filter mapper that picks out A. Your next one