To add on, have a look at hive and pig. Those are perfect fit for similar use cases.
Regards Bejoy KS Sent from handheld, please excuse typos. -----Original Message----- From: "Bejoy KS" <bejoy.had...@gmail.com> Date: Tue, 12 Jun 2012 13:04:33 To: <mapreduce-user@hadoop.apache.org> Reply-To: bejoy.had...@gmail.com Subject: Re: Need logical help Hi Girish You can achice this using reduce side joins. Use MultipleInputFormat for parsing two different sets of log files. Regards Bejoy KS Sent from handheld, please excuse typos. -----Original Message----- From: Girish Ravi <giri...@srmtech.com> Date: Tue, 12 Jun 2012 12:59:32 To: mapreduce-user@hadoop.apache.org<mapreduce-user@hadoop.apache.org> Reply-To: mapreduce-user@hadoop.apache.org Subject: Need logical help Hi All, I am thinking of a condition where the data in two log files are to be compared, can I use Map-Reduce to do this? I have one log file (LOG1) which has user ID and dept ID and another log file (LOG2) has some rows which has user ID and dept ID and other data. Can I compare the data where LOG1.userID = LOG2.userID and LOG1.deptID = LOG2.deptID? If so any suggestion to implement the mapper for this? Regards Girish Ph: +91-9916212114
