Hi, https://issues.apache.org/jira/browse/HIVE-4460 might be relevant to your issue.
Also, for curiosity's sake, could you share the build command you used to build hcatalog (I'm assuming from apache?) with hadoop 2.x artifact? On Sun, Aug 18, 2013 at 1:33 PM, Sonya Ling <[email protected]>wrote: > I am using sqoop-1.4.4 because it integrated hcatalog into it ( > https://issues.apache.org/jira/browse/SQOOP-931). Unfortunately, I cannot > get it to work because the HCatlog Incompatible issue. ( I have Hadoop > 2.0.0-cdh4.3.0 installed) > > > First, I got > > Exception in thread "main" java.lang.IncompatibleClassChangeError: Found > interface org.apache.hadoop.mapreduce.JobContext, but class was > expected > at > org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:94) > > It is expecting Hadoop 1.0.x but I have Hadoop 2.0.x. I took advice from > Sqoop developer Venkat Ranganathan and built Hcatalog with Hadoop 2.0 > artifact and replaced those jars under > /usr/lib/hive/hcatalog/share/hcatalog and webhcat with jars newly built. > > The error went away. I created a transaction3 table using hcat first then > ran > > sqoop import --connect jdbc:mysql://<host>/<database> --username <user> > --password <password> --table transaction3 --where "date >= '2013-08-12'" > --split-by date --hcatalog-table transaction3 > > I got the following errors. > 2013-08-18 12:08:23,087 INFO [AsyncDispatcher event handler] > org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics > report from attempt_1376849374287_0002_m_000002_3: Error: > java.lang.RuntimeException: Failed to instantiate: > org.apache.hcatalog.shims.HCatHadoopShims23 > at > org.apache.hcatalog.shims.HCatHadoopShims$Instance.selectShim(HCatHadoopShims.java:67) > at > org.apache.hcatalog.shims.HCatHadoopShims$Instance.<clinit>(HCatHadoopShims.java:49) > at > org.apache.hadoop.mapred.HCatMapRedUtil.createTaskAttemptContext(HCatMapRedUtil.java:39) > at > org.apache.hadoop.mapred.HCatMapRedUtil.createTaskAttemptContext(HCatMapRedUtil.java:29) > > : > Caused by: java.lang.ClassNotFoundException: > org.apache.hcatalog.shims.HCatHadoopShims23 > > My hcatalog-core-0.11.0.jar has HCatHadoopShims20S instead. > > I read source code HCatHadoopShims. It have codes like the followings, > Is any way I can get this to work (override major version etc.)? Any help > would be appreciated? THANKS. > > > private static HCatHadoopShims selectShim() { > // piggyback on Hive's detection logic > String major = ShimLoader.getMajorVersion(); > String shimFQN = > "org.apache.hcatalog.shims.HCatHadoopShims20S"; > if (major.startsWith("0.23")) { > shimFQN = "org.apache.hcatalog.shims.HCatHadoopShims23"; > } > try { > Class<? extends HCatHadoopShims> clasz = > Class.forName(shimFQN) > .asSubclass(HCatHadoopShims.class); > return clasz.newInstance(); > } catch (Exception e) { > throw new RuntimeException("Failed to instantiate: " + > shimFQN, e); > } > }
