[
https://issues.apache.org/jira/browse/HBASE-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354120#comment-14354120
]
Sean Busbey commented on HBASE-13149:
-------------------------------------
{quote}
If the jackson 1.8 appears before jackson 1.9 in the classpath when the job is
submitted, we will have a problem.
Did a little more investigation. 'hbase' script puts hbaes lib/* in front of
hadoop classpath. hbase lib/* is also in front of user provided HBASE_CLASSPATH.
But hadoop RunJar (or 'hadoop jar') puts hadoop jars and dependency jars in
front of user jars (unless HADOOP_USER_CLASSPATH_FIRST is specified').
Therefore using 'hadoop jar' to run hbase MR class works.
Well, it is a messy classpath issue. The hbase lib jars will need to appear
AFTER the hadoop lib jars.
We probably can not ask user to use 'hadoop jar' to run HBase MR always.
Also it turns out that hbase-server jar only expose limited main classes via
org.apache.hadoop.hbase.mapreduce.Driver.
{quote}
So it sounds like we can document a work around that sets HADOOP_CLASSPATH
equal to the output of {{hbase mapredcp}} and then uses {{hadoop jar}} to run
things. Do I have that right?
Actually, looking at the output of {{hbase mapredcp}}, I don't see jackson
listed at all. Could you include the command you're running to start the job?
> HBase MR is broken on Hadoop 2.5+ Yarn
> --------------------------------------
>
> Key: HBASE-13149
> URL: https://issues.apache.org/jira/browse/HBASE-13149
> Project: HBase
> Issue Type: Bug
> Affects Versions: 1.0.0, 2.0.0, 0.98.10.1
> Reporter: Jerry He
> Priority: Critical
> Attachments: HBASE-13149-0.98.patch,
> jackson-core-asl-compat_report.html, jackson-jaxrs-compat_report.html,
> jackson-mapper-asl-compat_report.html, jackson-xc-compat_report.html
>
>
> Running the server MR tools is not working on Yarn version 2.5+.
> Running org.apache.hadoop.hbase.mapreduce.Export:
> {noformat}
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.codehaus.jackson.map.ObjectMapper.setSerializationInclusion(Lorg/codehaus/jackson/map/annotate/JsonSerialize$Inclusion;)Lorg/codehaus/jackson/map/ObjectMapper;
> at
> org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider.configObjectMapper(YarnJacksonJaxbJsonProvider.java:59)
> at
> org.apache.hadoop.yarn.util.timeline.TimelineUtils.<clinit>(TimelineUtils.java:47)
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceInit(YarnClientImpl.java:166)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.serviceInit(ResourceMgrDelegate.java:102)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
> org.apache.hadoop.mapred.ResourceMgrDelegate.<init>(ResourceMgrDelegate.java:96)
> at org.apache.hadoop.mapred.YARNRunner.<init>(YARNRunner.java:112)
> at
> org.apache.hadoop.mapred.YarnClientProtocolProvider.create(YarnClientProtocolProvider.java:34)
> at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:82)
> at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:75)
> at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1266)
> at org.apache.hadoop.mapreduce.Job$9.run(Job.java:1262)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> at org.apache.hadoop.mapreduce.Job.connect(Job.java:1261)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1290)
> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1314)
> at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:189)
> {noformat}
> The problem seems to be the jackson jar version. HADOOP-10104 updated
> jackson version to 1.9.13. YARN-2092 reported a problem as well.
> HBase is using jackson 1.8.8. This version of the jar in the classpath seem
> to cause the problem.
> Should we upgrade to jackson 1.9.13?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
