I can put it through the paces w/cdh4, hadoop-1.0.3, and hadoop-2.0.0-alpha, verify that tests pass, and then integrate the pull req.
J On Tue, Jun 12, 2012 at 3:55 PM, Christian Tzolov <[email protected]> wrote: > I am afraid I have pulled the Avro upgrade request far too soon. I should > have checked my mailbox first. > > We have been using avro-1.7.0-rc0 for a week now to resolve AVRO-1046 issue. > No problems were observed so far but we have not tested it with any other > Hadoop version but chd3. > > Cheers, > Chris > > On Tue, Jun 12, 2012 at 10:47 PM, Tom White <[email protected]> wrote: >> >> Funny - I was just writing an email about potential dragons... >> >> Before upgrading someone should check that Avro 1.7.0 works with >> released versions of Hadoop. In the past there have been problems with >> Avro and dependencies like Jackson conflicting with versions that >> Hadoop uses. In particular, the MR classpath can be controlled via the >> configuration property mapreduce.user.classpath.first and the env >> property HADOOP_USER_CLASSPATH_FIRST. By setting these to true MR will >> use the newer Avro libraries, however there is a risk that Hadoop will >> not work with the newer versions. >> >> Cheers, >> Tom >> >> On Tue, Jun 12, 2012 at 3:36 PM, Josh Wills <[email protected]> wrote: >> > On Tue, Jun 12, 2012 at 1:28 PM, Gabriel Reid <[email protected]> >> > wrote: >> >> On Tue, Jun 12, 2012 at 10:14 PM, Josh Wills <[email protected]> >> >> wrote: >> >>> >> >>> I saw Chris' tweet that Avro 1.7.0 was released-- thoughts on >> >>> upgrading? Does it buy us any good stuff we want? >> >> >> >> The fix of ReflectDatumReader not working correctly with Specific >> >> Records (https://issues.apache.org/jira/browse/AVRO-1046) would allow >> >> us to remove a fair bit of Avro code that works around that bug -- if >> >> we do do the upgrade, I'd certainly volunteer to weed out those >> >> workarounds. >> >> >> >> On the other hand, I just did a quick scan of the release notes for >> >> 1.7.0 >> >> (https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310911&version=12318848) >> >> and I didn't notice anything that would really be a big reason to >> >> upgrade right away. >> >> >> >> Are there any reasons *not* to upgrade (other than risks of something >> >> else being broken)? Maybe the cleanup of the Avro code that we can do >> >> is reason enough to do the upgrade. >> > >> > +tom explicitly >> > >> > I remember having a bunch of frustrations with 1.6.0 and having to >> > stay on 1.5.4 for longer than I wanted because of some critical bugs >> > that didn't get fixed until 1.6.2, but I also think the move from >> > 1.5.4 to 1.6.0 involved a much larger rewrite than what I see from the >> > release notes for 1.7.0. Tom White is traveling across the US right >> > now, but I'm wondering if he has a feel for whether 1.7.0 is likely to >> > contain any dragons. :) >> > >> >> >> >> - Gabriel >> > >> > >> > >> > -- >> > Director of Data Science >> > Cloudera >> > Twitter: @josh_wills > > -- Director of Data Science Cloudera Twitter: @josh_wills
