We are using the below code for for integration test. You need to wait for
the process state.
.startApplication(
new Listener {
override def infoChanged(handle: SparkAppHandle): Unit = {
println("*** info changed * ", handle.getAppId,
handle.getState)
}
override def stateChanged(handle: SparkAppHandle): Unit = {
println("*** state changed *", handle.getAppId,
handle.getState)
}
})
// Initial state goes to unknown
// To avoid the UNKNOWN state check below.
Thread.sleep(1);
def waitTillComplete(handler: SparkAppHandle): Unit = {
while (!handler.getState.isFinal && handler.getState !=
SparkAppHandle.State.UNKNOWN) {
println("State :%s".format(handler.getState()))
Thread.sleep(5000)
}
}
On Thu, May 18, 2017 at 2:17 AM, Nipun Arora
wrote:
> Hi,
>
> I am trying to get a simple spark application to run programatically. I
> looked at http://spark.apache.org/docs/2.1.0/api/java/index.
> html?org/apache/spark/launcher/package-summary.html, at the following
> code.
>
>public class MyLauncher {
> public static void main(String[] args) throws Exception {
>SparkAppHandle handle = new SparkLauncher()
> .setAppResource("/my/app.jar")
> .setMainClass("my.spark.app.Main")
> .setMaster("local")
> .setConf(SparkLauncher.DRIVER_MEMORY, "2g")
> .startApplication();
>// Use handle API to monitor / control application.
> }
>}
>
>
> I don't have any errors in running this for my application, but I am
> running spark in local mode and the launcher class immediately exits after
> executing this function. Are we supposed to wait for the process state etc.
>
> Is there a more detailed example of how to monitor inputstreams etc. any
> github link or blogpost would help.
>
> Thanks
> Nipun
>
