[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504506#comment-14504506
]
Gera Shegalov commented on MAPREDUCE-6293:
--
+1 for 003.patch
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch,
MAPREDUCE-6293.003.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503892#comment-14503892
]
Gera Shegalov commented on MAPREDUCE-6293:
--
I don't mind switching to implicit toString for classloader at other places for
consistency even though they are guarded from NPE's.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504245#comment-14504245
]
Sangjin Lee commented on MAPREDUCE-6293:
The test failures appear unrelated. Locally I'm unable to reproduce them with
the patch or on the trunk. Just kicked off another build to see if it is
transient.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch,
MAPREDUCE-6293.003.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504293#comment-14504293
]
Hadoop QA commented on MAPREDUCE-6293:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726717/MAPREDUCE-6293.003.patch
against trunk revision d52de61.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5424//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5424//console
This message is automatically generated.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch,
MAPREDUCE-6293.003.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504205#comment-14504205
]
Hadoop QA commented on MAPREDUCE-6293:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12726717/MAPREDUCE-6293.003.patch
against trunk revision 44872b7.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.TestLargeSort
org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5421//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5421//console
This message is automatically generated.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch,
MAPREDUCE-6293.003.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503295#comment-14503295
]
Sangjin Lee commented on MAPREDUCE-6293:
One reason I went with classLoader.getClass().getName() is
classLoader.toString() would just use Object.toString() and print out the
hashCode which I thought was unnecessary detail. I used this pattern elsewhere
(e.g. MRApps).
I'm OK with switching to the implicit toString(). I acknowledge the point about
TCCL ever being null, although it would be very unlikely. But if we do, I think
it'd be better to switch to it in all places where I log the classloader.
Thoughts?
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502223#comment-14502223
]
Gera Shegalov commented on MAPREDUCE-6293:
--
Thanks for v2, [~sjlee0]!
Can we just use implicit {{toString}} ?
{code}
152 // note the current TCCL
153 LOG.info(Context classloader of thread + eventHandler +
154 : + eventHandler.getContextClassLoader());
{code}
Then we would not need to worry whether
eventHandler.setContextClassLoader(null) was ever called.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497664#comment-14497664
]
Hadoop QA commented on MAPREDUCE-6293:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12725778/MAPREDUCE-6293.002.patch
against trunk revision 1b89a3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5407//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5407//console
This message is automatically generated.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch, MAPREDUCE-6293.002.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497480#comment-14497480
]
Gera Shegalov commented on MAPREDUCE-6293:
--
In such a case It would be good to be able to reason what classloader is used
when job classloader is not set. Since we are already using the same approach
in other places where setCCL is called, I am +1. However, I think it will be be
helpful for debugging to add a LOG.info with thread and {{getCCL}} for
{{jobclassloader == nul}} cases as well. In {{CommitterEventHandler}} we don't
log at all.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497430#comment-14497430
]
Gera Shegalov commented on MAPREDUCE-6293:
--
Thanks for the patch, [~sjlee0]! It looks good.
minor comment for {{LocalContainerLauncher#serviceStart}}. No null check is
needed when calling {{eventHandler.setContextClassLoader}} because {{null}} is
handled appropriately within that method.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497445#comment-14497445
]
Sangjin Lee commented on MAPREDUCE-6293:
Thanks [~jira.shegalov] for the review!
I checked for null there in the abundance of caution because if
{{jobClassLoader}} was null {{setContextClassLoader()}} would also set the
thread's context classloader to null. If the inherited context classloader was
not null initially, then it would have an effect of nulling out that TCCL. So
to be safe, we should not be calling {{setContextClassLoader()}} unless we're
truly overriding it, no?
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14497537#comment-14497537
]
Sangjin Lee commented on MAPREDUCE-6293:
JVM normally sets the default classloader (sun.misc.Launcher$AppClassLoader) as
the TCCL on the main thread (from which other threads inherit it), so in most
cases the TCCL is just the default classloader. But I think it's a good idea to
add a logging statement to print out the TCCL here. I'll update the patch
shortly.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14494343#comment-14494343
]
Sangjin Lee commented on MAPREDUCE-6293:
[~djp], I haven't tracked them all down yet. I'll report what I find later.
Meanwhile, I would greatly appreciate your review on this. Thanks!
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14483691#comment-14483691
]
Junping Du commented on MAPREDUCE-6293:
---
Thanks [~sjlee0] for reporting the issue and delivering a patch to fix it. I
will help to review here.
I just found MAPREDUCE-6189 track the failure for TestMRTimelineEventHandling.
Do we have other JIRAs to track other test failures? If not, I think we should
file a separated JIRA to fix them.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388358#comment-14388358
]
Hadoop QA commented on MAPREDUCE-6293:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707416/MAPREDUCE-6293.001.patch
against trunk revision 85dc3c1.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestMRTimelineEventHandling
org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
org.apache.hadoop.mapred.pipes.TestPipeApplication
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5356//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5356//console
This message is automatically generated.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388803#comment-14388803
]
Sangjin Lee commented on MAPREDUCE-6293:
The failures seem to be existing failures on trunk. I can reproduce each of
these failures locally on trunk. Let me see if there are any open JIRAs against
these tests.
In the meantime, I would appreciate your reviews. Thanks!
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387192#comment-14387192
]
Sangjin Lee commented on MAPREDUCE-6293:
Restarted the jenkins build after the fix for HADOOP-11754:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5355/
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14387694#comment-14387694
]
Hadoop QA commented on MAPREDUCE-6293:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707416/MAPREDUCE-6293.001.patch
against trunk revision 90e07d5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestMRTimelineEventHandling
org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
org.apache.hadoop.mapred.pipes.TestPipeApplication
org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
The following test timeouts occurred in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
org.apache.hadoop.mapreduce.TestLargeSort
org.apache.hadoop.mapreduce.TestMRJobClient
org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution
org.apache.hadoop.mapreduce.v2.TestMRJobs
org.apache.hadoop.mapreduce.v2.TestUberAM
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapred.TestMiniMRChildTask
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5355//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5355//console
This message is automatically generated.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388019#comment-14388019
]
Sangjin Lee commented on MAPREDUCE-6293:
Kicking off the jenkins build one more time. I suspect these test failures are
spurious, as there are no failures related with classloading issues (they're
mostly timeouts and other varierty). The patch is a no-op if the job
classloader is not set.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382336#comment-14382336
]
Sangjin Lee commented on MAPREDUCE-6293:
Github link:
https://github.com/sjlee/hadoop/commit/259766e63d8fd739a02ee0c59990270bda78c7d4
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14382044#comment-14382044
]
Sangjin Lee commented on MAPREDUCE-6293:
The test failures are caused by HADOOP-11754.
I would greatly appreciate your review on the patch. Thanks!
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381083#comment-14381083
]
Sangjin Lee commented on MAPREDUCE-6293:
The root cause is this.
Cascading loads HadoopTupleMapFactory using the thread context classloader. The
thread on which the task runs us uber-SubtaskRunner. This thread is created
(directly or indirectly) through LocalContainerLauncher's service start. At
that point, the job classloader exists but is not yet set as the TCCL. Thus,
these threads do not inherit the job classloader as the TCCL (TCCLs are
inherited from the parent thread that created the thread).
It should be fixed in the same manner as what we did with the committer event
handler. I'll post a patch shortly.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381356#comment-14381356
]
Hadoop QA commented on MAPREDUCE-6293:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12707416/MAPREDUCE-6293.001.patch
against trunk revision 44809b8.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.jobhistory.TestJobHistoryEventHandler
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapred.TestMerge
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapred.TestMRIntermediateDataEncryption
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapreduce.TestLargeSort
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
org.apache.hadoop.mapreduce.v2.TestRMNMInfo
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
org.apache.hadoop.mapreduce.security.ssl.TestEncryptedShuffle
org.apache.hadoop.mapred.TestJobSysDirWithDFS
org.apache.hadoop.ipc.TestMRCJCSocketFactory
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapreduce.v2.TestMROldApiJobs
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapreduce.v2.TestMRAMWithNonNormalizedCapabilities
org.apache.hadoop.mapreduce.v2.TestNonExistentJob
org.apache.hadoop.mapreduce.v2.TestMRJobs
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapreduce.v2.TestMRAppWithCombiner
org.apache.hadoop.mapreduce.v2.TestUberAM
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapreduce.TestMRJobClient
org.apache.hadoop.mapred.TestMiniMRBringup
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.mapreduce.security.TestMRCredentials
org.apache.hadoop.mapred.TestMRTimelineEventHandling
org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser
org.apache.hadoop.mapreduce.v2.TestMRJobsWithHistoryService
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestMiniMRClientCluster
org.apache.hadoop.mapred.TestNetworkedJob
org.apache.hadoop.mapred.TestMiniMRChildTask
org.apache.hadoop.mapreduce.TestChild
org.apache.hadoop.mapreduce.security.TestBinaryTokenFile
org.apache.hadoop.mapred.TestJobCounters
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5337//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5337//console
This message is automatically generated.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class.
[jira] [Commented] (MAPREDUCE-6293) uberized job fails with the job classloader enabled
[
https://issues.apache.org/jira/browse/MAPREDUCE-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14381361#comment-14381361
]
Sangjin Lee commented on MAPREDUCE-6293:
The test failures are highly likely to be existing issues on the trunk
(HADOOP-10670). Basically the RM in a mini YARN cluster fails to start. I see
these failures with a clean trunk build.
uberized job fails with the job classloader enabled
---
Key: MAPREDUCE-6293
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6293
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mr-am
Affects Versions: 2.6.0
Reporter: Sangjin Lee
Assignee: Sangjin Lee
Attachments: MAPREDUCE-6293.001.patch
An uberized job fails if the job classloader is enabled and the job needs to
use the thread context classloader to load a class. Some example error in the
log:
{quote}
2015-03-23 23:28:34,675 INFO [main\]
org.apache.hadoop.mapreduce.v2.util.MRApps: Creating job classloader
...
2015-03-23 23:28:42,096 ERROR [uber-SubtaskRunner\]
cascading.provider.ServiceLoader: unable to find service class:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory, with exception:
java.lang.ClassNotFoundException:
cascading.tuple.hadoop.collect.HadoopTupleMapFactory
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
