HDFS API has changed a little, and it does not impact Chukwa integration. For mapreduce, there are some changes which requires demux job to be written differently. This is mainly more code to be written to schedule map reduce job in YARN framework. The instrumentation of collecting data from YARN, will need designs to collect useful data in context of monitoring YARN. I agree that 1.x branch seems like a better target for Chukwa 0.6.
regards, Eric On Sun, Dec 1, 2013 at 5:00 PM, Ariel Rabkin <[email protected]> wrote: > Howdy. > > I think option 1 is fine -- let's release what we have for now. My > impression is that there are still a lot of users on the 1xx branch. > > I've been a bit out of the loop on Hadoop development and don't have a > sense how much pain it is to target different Hadoop versions. Is the > issue entirely the demux jobs, or did the filesystem APIs also change > in incompatible ways? > > --Ari > > On Thu, Nov 28, 2013 at 2:15 PM, Eric Yang <[email protected]> wrote: > > Hi all, > > > > I like to collect thoughts on plan for branching for Chukwa 0.6 release. > > We have four options: > > > > 1. Chukwa 0.6 target for Hadoop 1.x branch. > > 2. Chukwa 0.6 target for Hadoop 2.x branch. > > 3. Chukwa 0.6 target to work with either Hadoop 1.x or Hadoop 2.x with > > maven profile switching. Default to Hadoop 1.x branch. > > 4. Chukwa 0.6 target to work with either Hadoop 1.x or Hadoop 2.x with > > maven profile switching. Default to Hadoop 2.x branch. > > > > Option 2 and 4 may put Chukwa on a longer development path to integrate > > with YARN for map reduce job. > > > > My personal preference is option 1 to keep everything simple to help > > release often. > > > > regards, > > Eric > > > > -- > Ari Rabkin [email protected] > Princeton Computer Science Department >
