[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306653#comment-14306653
]
bc Wong commented on MAPREDUCE-4052:
Filed https://issues.cloudera.org/browse/DISTRO-689
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14306652#comment-14306652
]
bc Wong commented on MAPREDUCE-4052:
Filed https://issues.cloudera.org/browse/DISTRO-689
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303763#comment-14303763
]
Siddhartha Azad commented on MAPREDUCE-4052:
I am using CDH 5.2.0 jars. The job gets submitted to linux from windows but
doesn't complete. Same job works fine from Linux. I have the following property
settings in mapred-site.xml:
mapreduce.app-submission.cross-platform
true
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303561#comment-14303561
]
bc Wong commented on MAPREDUCE-4052:
Which minor version are you using? This patch is included in CDH 5.2. You
should file a jira on issues.cloudera.org for CDH specific problems.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303560#comment-14303560
]
bc Wong commented on MAPREDUCE-4052:
Which minor version are you using? This patch is included in CDH 5.2. You
should file a jira on issues.cloudera.org for CDH specific problems.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303482#comment-14303482
]
Siddhartha Azad commented on MAPREDUCE-4052:
Any luck? I am using CDH5 and having the same issue submitting from Windows.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303481#comment-14303481
]
Siddhartha Azad commented on MAPREDUCE-4052:
I am using CDH5 while submitting from Windows to Linux. The job gets submitted
and starts running. There are split files created (during the map phase I
believe) but then there is an error and the client exits. If submitted from
linux to linux the job runs fine. Is this patch available in the CDH5 version
or does it need to be explicitly applied?
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:196)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090954#comment-14090954 ] Jay commented on MAPREDUCE-4052: I am still having the same issue after upgrading to hadoop2.4 from 2.2. I assume 2.4 has this fix in it. I am running hadoop on local windows box. Trying to use spring client from Eclipse to run the wordcount example. My cluster is configured correctly as I can run wordcount from the hdfs command line. I get in to issue when I try to run a mapreduce job from Eclipse on Windows. I have modified mapred--site.xml to use the proeprty mapreduce.app-submission.cross-platform to be true. I also removed yarn.application.classpath entry from my yarn-site.xml. I tried adding it back fro Windows. All scenarios seem to not work. Is there a documentation on what needs to take place to make it work. Will appreciate any help. jsaraiya( &container_1407515618489_0002_01_01╤╣ ♠stderr ♦1440java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at java.net.URLClassLoader.access$000(URLClassLoader.java:58) at java.net.URLClassLoader$1.run(URLClassLoader.java:197) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) ... 12 more Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. Exception in thread "main" ♠stdout ☺0( &container_1407515618489_0002_02_01╤╣ ♠stderr ♦1440java.lang.NoClassDefFoundError: org/apache/hadoop/service/CompositeService at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632) at java.lang.ClassLoader.defineClass(ClassLoader.java:616) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) at java.net.URLClassLoader.defineClass(URLClassLoader.java:283) at java.net.URLClassLoader.access$000(URLClassLoader.java:58) at java.net.URLClassLoader$1.run(URLClassLoader.java:197) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.service.CompositeService at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:248) ... 12 more Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. > Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop > cluster. > --- > > Key: MAPREDUCE-4052 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 0.23.1, 2.2.0 > Environment: client on the Windows, the the cluster on the suse >Reporter: xieguiming >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch, > MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch, > MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPR
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089028#comment-14089028
]
Donggeun Lee commented on MAPREDUCE-4052:
-
i also have been trouble due to this issue, however, i found reasons about
error
when you used cross-platform configuration, you should check "hadoop-env.sh"
included export JAVA_HOME="your java installation directory within this" and
conf.set("mapreduce.app-submission.cross-platform", "true"); in your remote
driver file. and then i will works.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Fix For: 2.4.0
>
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937892#comment-13937892 ] Hudson commented on MAPREDUCE-4052: --- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1729 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1729/]) YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1578135) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestMapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/ssl/TestEncryptedShuffle.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop > cluster. > --- > > Key: MAPREDUCE-4052 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 0.23.1, 2.2.0 > Environment: client on the Windows, the the cluster on the suse >Reporter: xieguiming >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch, > MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch, > MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch, > MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch > > > when I use the eclipse on the windows to submit the job. and the > applicationmaster throw the exception: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/ha
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937833#comment-13937833 ] Hudson commented on MAPREDUCE-4052: --- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1704 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1704/]) YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1578135) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestMapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/ssl/TestEncryptedShuffle.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop > cluster. > --- > > Key: MAPREDUCE-4052 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 0.23.1, 2.2.0 > Environment: client on the Windows, the the cluster on the suse >Reporter: xieguiming >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch, > MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch, > MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch, > MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch > > > when I use the eclipse on the windows to submit the job. and the > applicationmaster throw the exception: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/mapre
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937678#comment-13937678 ] Hudson commented on MAPREDUCE-4052: --- FAILURE: Integrated in Hadoop-Yarn-trunk #512 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/512/]) YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1578135) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestMapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/ssl/TestEncryptedShuffle.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop > cluster. > --- > > Key: MAPREDUCE-4052 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 0.23.1, 2.2.0 > Environment: client on the Windows, the the cluster on the suse >Reporter: xieguiming >Assignee: Jian He > Fix For: 2.4.0 > > Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch, > MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch, > MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch, > MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch > > > when I use the eclipse on the windows to submit the job. and the > applicationmaster throw the exception: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/mapredu
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937265#comment-13937265 ] Hudson commented on MAPREDUCE-4052: --- SUCCESS: Integrated in Hadoop-trunk-Commit #5336 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5336/]) YARN-1824. Improved NodeManager and clients to be able to handle cross platform application submissions. Contributed by Jian He. MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. Contributed by Jian He. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1578135) * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestMapReduceChildJVM.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/main/java/org/apache/hadoop/mapred/YARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMiniMRChildTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/security/ssl/TestEncryptedShuffle.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationConstants.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/ApplicationMaster.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java > Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop > cluster. > --- > > Key: MAPREDUCE-4052 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: job submission >Affects Versions: 0.23.1, 2.2.0 > Environment: client on the Windows, the the cluster on the suse >Reporter: xieguiming >Assignee: Jian He > Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch, > MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch, > MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch, > MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch > > > when I use the eclipse on the windows to submit the job. and the > applicationmaster throw the exception: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/mapreduce/v2/app/MRAppMaster > C
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937256#comment-13937256
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-4052:
This looks good to me +1. Checking it in.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935898#comment-13935898
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634855/MAPREDUCE-4052.8.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4430//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4430//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.8.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934708#comment-13934708
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634647/MAPREDUCE-4052.7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4424//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4424//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.7.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934574#comment-13934574
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634640/MAPREDUCE-4052.6.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4423//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.6.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934544#comment-13934544
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634615/MAPREDUCE-4052.5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.mapreduce.v2.app.job.impl.TestMapReduceChildJVM
org.apache.hadoop.mapreduce.v2.util.TestMRApps
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4422//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4422//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.5.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13934353#comment-13934353
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-4052:
Patch looks fine overall, some review comments on the MR part of the patch
- TestMapReduceChildJVM.envVar can be removed.
- MAPREDUCE_CROSS_PLATFORM_APP_SUBMISSION ->
MAPREDUCE_APP_SUBMISSION_CROSS_PLATFORM,
"mapreduce.cross-platform.app-submission" ->
"mapreduce.app-submission.cross-platform". Similarly the name of the default
constant too.
- The default for the above config should be false for compatibility
- Given that, fix the description of mapreduce.application.classpath in
mapred-default.xml
- Add the new config in mapred-default.xml
- Cannot rename MRJobConfig.DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH for
compatibility
- MRJobConfig.DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH_CROSS_ENV and
DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH_CLIENT_ENV also to be renamed in line
with the YARN constant renames.
- Fix their javadoc too to refer to cross-platform.
- There are still some usages of yarn.util.Apps. They are not intended?
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931384#comment-13931384
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634086/MAPREDUCE-4052.4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4413//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4413//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.4.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931180#comment-13931180
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12634016/MAPREDUCE-4052.3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.mapreduce.v2.app.job.impl.TestMapReduceChildJVM
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4411//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4411//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4411//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931108#comment-13931108
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-4052:
Jian, can you file a YARN counterpart for this JIRA?
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13931016#comment-13931016
]
Jian He commented on MAPREDUCE-4052:
uploaded a patch that makes new command syntax configurable as suggested by
Chris
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.3.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13929739#comment-13929739
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-4052:
bq. For this specific patch, I suppose another other option is to gate use of
the new command syntax behind a client-side config flag.
This makes sense. In a way, the protocol with the NodeManager is already
optional - if there are named tags then NM will substitute them, otherwise
nothing to do. Let's see if it can be optional in MapReduce too.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13926115#comment-13926115
]
Chris Nauroth commented on MAPREDUCE-4052:
--
Thanks, [~kkambatl]. I can start that thread later today to clarify policy and
update the text accordingly.
For this specific patch, I suppose another other option is to gate use of the
new command syntax behind a client-side config flag. Then, clients can turn on
the flag only after they know their clusters have been upgraded. That's a
compatible solution that avoids protocol impacts entirely, but it pushes some
complexity on to the end user to turn on the flag.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13926093#comment-13926093
]
Karthik Kambatla commented on MAPREDUCE-4052:
-
(Haven't looked at the patch at all).
My interpretation of the Wire-Compatibility policies is that new clients in the
same major release should work old servers and vice-versa. If that is not
clear, we should probably update the policies to clarify it. [~cnauroth],
[~jianhe] - do any of you want to start a thread in the dev lists to discuss
the same?
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13926084#comment-13926084
]
Chris Nauroth commented on MAPREDUCE-4052:
--
bq. btw, Chris Nauroth, is the use case that upgraded-client with non-upgraded
NM important ?
I brought this up, because I've been in situations where someone wanted to pick
up a client-side bug fix ahead of the cluster's upgrade schedule. It looks to
me like this is a gray area in our policies though.
http://hadoop.apache.org/docs/r2.3.0/hadoop-project-dist/hadoop-common/Compatibility.html#Wire_compatibility
>From the content in that page, we've made a specific commitment that old
>clients continue to work with new servers. As Jian said, that part is fine
>with this patch. What is less clear is whether or not we've made a commitment
>for new clients to work with old servers. Of course, it's best to strive for
>it, and forward compatibility is one of our motivations in the protobuf
>messages, but I can't tell from that policy statement if we've made a
>commitment to it. This is probably worth some wider discussion before
>changing the patch.
If we do need to achieve that kind of compatibility, then it's going to be a
more challenging patch. I think we'd end up needing to add an optional version
number or at least a flag on the {{Container}} returned in the
{{AllocateResponse}}. This would tell the client whether or not the container
can accept the new syntax, and then the client could use the old code path as a
fallback path for compatibility with old servers that don't set this version
number or flag. That would work for containers submitted by an AM. I can't
think of a similar solution that would work for the initial AM container
though, because it seems to me like the RPC sequence there doesn't have as
clear of a way for indicating capabilities inside the container that's going to
run the AM before its submission.
Like I said, please do discuss wider before pursuing this. I'd hate to send
you down an unnecessary rathole if the current patch is fine. :-) Thanks,
Jian.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: Jian He
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.2.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924513#comment-13924513
]
Jian He commented on MAPREDUCE-4052:
btw, [~cnauroth], is the use case that upgraded-client with non-upgraded NM
important ?
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924496#comment-13924496
]
Jian He commented on MAPREDUCE-4052:
Yes, An upgraded client with this patch is not able to successfully submit to
non-upgraded NM. But non-upgraded client is able to successfully submit to
upgraded NMs.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924488#comment-13924488
]
Chris Nauroth commented on MAPREDUCE-4052:
--
Hi, [~jianhe]. If I understand correctly, this change would not be
backwards-compatible. An upgraded YARN/MR client with this patch would not be
able to successfully submit containers to a non-upgraded NodeManager. Is that
correct? If so, I just want to check that this is intentional. Thanks!
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924482#comment-13924482
]
Jian He commented on MAPREDUCE-4052:
The main problem here is that client is constructing the commands and
environments for container launch depending on the client machine OS, but if
server(NM) is running a different OS, NM is not able to parse the commands and
environments.
The patch creates two constants in ApplicationConstants for PARAMETER_EXPANSION
and CLASS_PATH_SEPARATOR as known that they are different between Windows and
Linux. User application is supposed to use these constants to construct their
commands and environments when constructing the ContainerLaunchContext. When
NM tries to launch the container, it will first replace the constants with the
correct characters depending on its own OS. Changed MR and DistributedShell to
do so.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.1.patch,
> MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921332#comment-13921332
]
Jian He commented on MAPREDUCE-4052:
Hi Xieguiming, are you still working on this? I'd like to take it over.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855737#comment-13855737
]
Attila Pados commented on MAPREDUCE-4052:
-
Hi,
as i found, the patch attached to this jira ticket, is not working for 2.2.0,
Hadoop2 GA.
i made another jira issue: https://issues.apache.org/jira/browse/MAPREDUCE-5655
and also added a patch that fixes it. Please keep in mind, that this is
generaly a workaround.
Attila
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13855452#comment-13855452
]
Hadoop Developer commented on MAPREDUCE-4052:
-
Is this patch applied to Hadoop2 GA?
I am getting same error while executing WordCount on hadoop2.2 on Windows 7 (64
bit).
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838918#comment-13838918
]
Attila Pados commented on MAPREDUCE-4052:
-
I checked the patches here, i was curious about alternative solutions to the
similar (or same) problem with 2.2.0.
However, i missed the point in either the patches or in the problem
description, that the java call from the shell script included the right
$JAVA_HOME for linux cluster, or it was %JAVA_HOME% as i have experienced on
2.2.0 version?
(I didn't investigated the case with 0.23.x at all, i have downloaded only the
2.2.0 source and binary, so probably there are more differences between the
processes)
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (MAPREDUCE-4052) Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster.
[
https://issues.apache.org/jira/browse/MAPREDUCE-4052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836766#comment-13836766
]
Hadoop QA commented on MAPREDUCE-4052:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12523800/MAPREDUCE-4052-0.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/4236//console
This message is automatically generated.
> Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop
> cluster.
> ---
>
> Key: MAPREDUCE-4052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: job submission
>Affects Versions: 0.23.1, 2.2.0
> Environment: client on the Windows, the the cluster on the suse
>Reporter: xieguiming
>Assignee: xieguiming
> Attachments: MAPREDUCE-4052-0.patch, MAPREDUCE-4052.patch
>
>
> when I use the eclipse on the windows to submit the job. and the
> applicationmaster throw the exception:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/v2/app/MRAppMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class:
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit.
> The reasion is :
> class Apps addToEnvironment function, use the
> private static final String SYSTEM_PATH_SEPARATOR =
> System.getProperty("path.separator");
> and will result the MRApplicationMaster classpath use the ";" separator.
> I suggest that nodemanger do the replace.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
