Hi Jarek, I'm not using HBase on that cluster, which I've verified by hunting for binaries. Sqoop seems to be pulling it in though:
hadoop@ip-XXX-XXX-XXX-XXX:~/sqoop$ find . -iname '*hbase*.jar' ./build/ivy/lib/sqoop/hadoop100test/hbase-0.92.0-tests.jar ./build/ivy/lib/sqoop/hadoop100test/hbase-0.92.0.jar Any idea if I can prune it from Sqoop's build? Cheers, Lars On 11 October 2013 02:54, Jarek Jarcec Cecho <[email protected]> wrote: > Hi Lars, > I've seen this issue many times and I can confirm your findings - this > exception is most likely being thrown because multiple incompatible thrift > implementations are available on Sqoop classpath. > > However Sqoop itself do not ships with thrift as we are not using thrift. > Usually it's the other dependent projects that are bringing thrift as > transitive dependency. Are you by any chance also using HBase? HBase is > known to ship it's own thrift version that is usually not the same as the > one from Hive. > > Jarcec > > On Tue, Oct 08, 2013 at 05:26:52PM +1100, Lars Yencken wrote: > > Hi all, > > > > I'm having trouble with what should be a standard Sqoop import job, from > > MySQL to Hive. On a new cluster, I get this exception when I do my import > > run: > > > > java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B > > > > Full stack trace: https://gist.github.com/larsyencken/6879918 > > Environment: Sqoop 1.4.4 / Hive 0.8.1.8 / Hadoop 1.0.3 on EMR > > > > The problem only occurs with the Hive import. If I take it out and just > > import from MySQL to HDFS, the data comes through without problems. > > > > I have a feeling the problem is to do with multiple incompatible thrift > > versions (Hive has libthrift-0.7.0.jar, Sqoop uses libthrift-0.9.0.jar). > > > > Any suggestions on getting these two to play nicely together? > > > > Best, > > Lars >
