[
https://issues.apache.org/jira/browse/CASSANDRA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Coverston updated CASSANDRA-5201:
------------------------------------------
Attachment: hadoopCompat.patch
Poking around at other projects this generally gets solved in one of two ways:
Ship two versions of their Hadoop integrations (one compiled for the old, and
one compiled for the new), or use a little reflection to make things work
across the board.
I'm attaching a patch that uses the hadoopCompat subproject of elephantbird.
This will allow us to compile a single binary and run with the new and old
context objects.
I've tested this patch with HDP 2.0, and Apache Hadoop 1.0.4 and it works fine
with both (including Hive in DSE). With Pig I needed to compile our (optional)
pig dependency with:
bq. ant clean jar-withouthadoop -Dhadoopversion=23
Only really needed if you're using one of the current versions of thrift with
the new JobContext.
> Cassandra/Hadoop does not support current Hadoop releases
> ---------------------------------------------------------
>
> Key: CASSANDRA-5201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5201
> Project: Cassandra
> Issue Type: Bug
> Components: Hadoop
> Affects Versions: 1.2.0
> Reporter: Brian Jeltema
> Assignee: Dave Brosius
> Attachments: 5201_a.txt, hadoopCompat.patch
>
>
> Using Hadoop 0.22.0 with Cassandra results in the stack trace below.
> It appears that version 0.21+ changed org.apache.hadoop.mapreduce.JobContext
> from a class to an interface.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:103)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:357)
> at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1045)
> at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1042)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1042)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1062)
> at MyHadoopApp.run(MyHadoopApp.java:163)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at MyHadoopApp.main(MyHadoopApp.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:192)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
