In my mind, it's a function of how much surgery we would need to do in order to support 0.20.2-- if it's just the multiple outputs stuff, that seems like a small price to pay in terms of code duplication vs. wider support. But I don't have a feel for how deep the rabbit hole goes. If you wouldn't mind taking a crack at it and reporting back, we would be much obliged-- but if you find yourself shaving a yak, you should bail out. :)
J On Wed, Aug 15, 2012 at 3:58 PM, Jonathan Dinu <[email protected]> wrote: > Hi, > > I have been using Crunch and starting to love it more and more. I have been > using it with Hadoop-cdh3u4 and it has been swell but as Josh points out > (http://mail-archives.apache.org/mod_mbox/incubator-crunch-user/201207.mbox/%3CCAH29n6N3GXkQ6B-NT3w3JwK8aMo7vDr9ip2=1xx0uf3kikj...@mail.gmail.com%3E) > it depends on Hadoop being able to write multiple output files. I have > tried my code with multiple versions of hadoop and it seems like 1.0.1 is the > earliest version that I got my tests to successful complete with. 0.20.x is > not compatible because of the file output issue. > > I was curious if it is a goal of Crunch as it goes through Apache incubation > to become compatible with the 0.20.x versions of Hadoop (or more versions), > if it is actually possible to make it compatible with these versions, and if > it is something that is desired by others. > > I have not yet contributed to Crunch, but I would be interested in helping to > make it compatible with the 0.20.x versions of Hadoop if the rest of the > community thinks this is a worthwhile endeavor. Just wanted to get the > opinion of the community and creators of Crunch. > > Best, > Jonathan
