YarnClient to get the running applications list in java
I am working with YarnClient for the 1st time. My goal is to get and
display the applications running on Yarn using Java. My project setup is as
follows:
public static void main(String[] args) throws IOException, YarnException {
// Create yarnClient
YarnConfiguration conf = new YarnConfiguration();
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
try {
List applications = yarnClient.getApplications();
System.err.println(yarn client : + applications.size());
} catch (YarnException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I get the following exception when i run the program:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)Caused by:
java.lang.NoClassDefFoundError:
org/apache/hadoop/HadoopIllegalArgumentException
at projects.HelloWorld.main(HelloWorld.java:16)... 6 moreCaused by:
java.lang.ClassNotFoundException:
org.apache.hadoop.HadoopIllegalArgumentException
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
The POM file is as follows:
?xml version=1.0 encoding=UTF-8?project
xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
modelVersion4.0.0/modelVersion
groupIdBigContent/groupId
artifactIdManagementServer/artifactId
version1.0-SNAPSHOT/version
properties
project.build.sourceEncodingUTF-8/project.build.sourceEncoding
hadoop.version2.4.0/hadoop.version
spark.version1.2.1/spark.version
/properties
build
plugins
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-compiler-plugin/artifactId
version3.2/version
configuration
source1.7/source
target1.7/target
/configuration
/plugin
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-war-plugin/artifactId
version2.3/version
executions
execution
!-- First step is to disable the default-war
build step. --
iddefault-war/id
phasenone/phase
/execution
execution
!-- Second step is to create an exploded war.
Done in prepare-package --
idwar-exploded/id
phaseprepare-package/phase
goals
goalexploded/goal
/goals
/execution
execution
!-- Last step is to make sure that the war is
built in the package phase --
idcustom-war/id
phasepackage/phase
goals
goalwar/goal
/goals
/execution
/executions
configuration
webXmlsrc/main/webapp/WEB-INF/web.xml/webXml
webResources
resource
!-- this is relative to the pom.xml directory --
directoryresource2/directory
/resource
/webResources
/configuration
/plugin
/plugins
/build
dependencies
dependency
groupIdorg.apache.spark/groupId
artifactIdspark-streaming_2.10/artifactId
version${spark.version}/version
scopeprovided/scope
/dependency
dependency
groupIdcom.sun.jersey/groupId
artifactIdjersey-core/artifactId
version1.9.1/version
/dependency
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
exclusions
exclusion
groupIdjavax.servlet/groupId
artifactId*/artifactId
RE: YarnClient to get the running applications list in java
Simple way to meet your goal , you can add hadoop jars into project classpath.
I.e If you have hadoop package, extract it and add all the jars into project
classpath.
Then you change java code below
YarnConfiguration conf = new YarnConfiguration();
conf.set(yarn.resourcemanager.address, rm-ip:port); // Running RM
address
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
yarnClient.start(); // you need to start YarnClient service
// code to getApplications()
}
Thanks Regards
Rohith Sharma K S
From: Mouzzam Hussain [mailto:monibab...@gmail.com]
Sent: 26 February 2015 16:23
To: user@hadoop.apache.org
Subject: YarnClient to get the running applications list in java
I am working with YarnClient for the 1st time. My goal is to get and display
the applications running on Yarn using Java. My project setup is as follows:
public static void main(String[] args) throws IOException, YarnException {
// Create yarnClient
YarnConfiguration conf = new YarnConfiguration();
YarnClient yarnClient = YarnClient.createYarnClient();
yarnClient.init(conf);
try {
List applications = yarnClient.getApplications();
System.err.println(yarn client : + applications.size());
} catch (YarnException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
I get the following exception when i run the program:
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError:
org/apache/hadoop/HadoopIllegalArgumentException
at projects.HelloWorld.main(HelloWorld.java:16)
... 6 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.HadoopIllegalArgumentException
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
The POM file is as follows:
?xml version=1.0 encoding=UTF-8?
project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
modelVersion4.0.0/modelVersion
groupIdBigContent/groupId
artifactIdManagementServer/artifactId
version1.0-SNAPSHOT/version
properties
project.build.sourceEncodingUTF-8/project.build.sourceEncoding
hadoop.version2.4.0/hadoop.version
spark.version1.2.1/spark.version
/properties
build
plugins
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-compiler-plugin/artifactId
version3.2/version
configuration
source1.7/source
target1.7/target
/configuration
/plugin
plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-war-plugin/artifactId
version2.3/version
executions
execution
!-- First step is to disable the default-war build
step. --
iddefault-war/id
phasenone/phase
/execution
execution
!-- Second step is to create an exploded war. Done in
prepare-package --
idwar-exploded/id
phaseprepare-package/phase
goals
goalexploded/goal
/goals
/execution
execution
!-- Last step is to make sure that the war is built in
the package phase --
idcustom-war/id
phasepackage/phase
goals
goalwar/goal
/goals
/execution
/executions
configuration
webXmlsrc/main/webapp/WEB-INF/web.xml/webXml
webResources
resource
!-- this is relative to the pom.xml directory --
directoryresource2/directory
