[
https://issues.apache.org/jira/browse/HBASE-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624087#comment-13624087
]
Devaraj Das commented on HBASE-8258:
------------------------------------
{quote}
When it comes out we deal with it.
The isolation of the particular relevant yarn configs demonstrate that the vmem
stuff is red herring and unrelated as of hadoop 2.0.2.
{quote}
Are you saying that we fix the problems in HBase assuming hadoop-2.0.2-alpha as
the underlying hadoop version? So far, the assumption I have been working with
is that we wanted to get HBase working with 2.0.4-alpha (and this whole thread
started with trying to make HBase work with hadoo-2.0.3-alpha, ref HBASE-7904).
bq. copyConfigurationValues seems overly broad and doesn't expose the root
cause.
Hmm.. One can dump all the yarn configurations (yarn.* config keys) before and
after the copyConfigurationValues call, and figure out what changed. That might
be a worthwhile experiment. So let's say that we identify two or three configs.
We should then change copyConfigurationValues to be more restrictive in copying?
bq. You also skipped the last two questions – without getting to the root, this
still doesn't explain why it works for other hbase-mr tests and not here, and
how to make it so that any other mr tests doesn't have to have a special
copyConfiguraitonValues equivalent call
Sorry about missing these earlier...
On the first one, as per the last hadoopQA run without the fixes with
hadoop-2.0.4-SNAPSHOT
(https://issues.apache.org/jira/browse/HBASE-8258?focusedCommentId=13621696&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13621696),
almost all of the mapreduce tests failed. I haven't looked at all the tests
that passed but one quick check on one of the passing test in the mapreduce
package like TestTableSplit.java doesn't submit jobs, and maybe that's why they
passed.
On the other question, if the approach of calling copyConfigurationValues in
the common part of the test code makes the tests work (as per the last 1 or 2
hadoopqa runs), I think we should make a note of this in some doc. We can get
to this detail once we all agree on the proposed patch.
bq. My fear is that there are fixes we need in upstream so lets figure them out
sooner rather than later (figuring them out after 2.0.4-alpha means hbase 0.96
can't release till 2.0.5-alpha/beta whenever that is).
Stack, are you suggesting that we should depend on 2.0.4-alpha sooner rather
than later, maybe even check with 2.0.4-SNAPSHOT on a periodic basis until
2.0.4-alpha is released?
> Make mapreduce tests pass on hadoop2
> ------------------------------------
>
> Key: HBASE-8258
> URL: https://issues.apache.org/jira/browse/HBASE-8258
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Reporter: stack
> Priority: Blocker
> Fix For: 0.95.1
>
> Attachments: 8258-plain.txt, 8258-v1-hadoop-2.0.txt,
> 8258-v2-hadoop-2.0.txt, 8258-v4-hadoop-2.0.txt, hbase-8258-simple.patch
>
>
> HBASE-7904 was a first attempt at making this work but it got lost in the
> weeds.
> This is a new attempt at making hbase mapreduce jobs run on hadoop2 (w/o
> breaking mapreduce on hadoop1)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
