Dear fellow Spark users,
I have been trying to use *SparkLauncher.startApplication()* to launch a
Spark app from within java code, but unable to do so. However, same piece
of code is working if I use *SparkLauncher.launch()*.
Here are the corresponding code snippets :
*SparkAppHandle handle = new SparkLauncher()*
*
.setSparkHome("/Users/miqbal1/DISTRIBUTED_WORLD/UNPACKED/spark-1.6.1-bin-hadoop2.6")*
*
.setJavaHome("/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home")*
*
.setAppResource("/Users/miqbal1/wc.jar").setMainClass("org.myorg.WC").setMaster("local")*
* .setConf("spark.dynamicAllocation.enabled",
"true").startApplication(); System.out.println(handle.getAppId());*
* System.out.println(handle.getState());*
This prints *null* and *UNKNOWN *as output.
*Process spark = new SparkLauncher()*
*
.setSparkHome("/Users/miqbal1/DISTRIBUTED_WORLD/UNPACKED/spark-1.6.1-bin-hadoop2.6")*
*
.setJavaHome("/Library/Java/JavaVirtualMachines/jdk1.8.0_92.jdk/Contents/Home")*
*
.setAppResource("/Users/miqbal1/wc.jar").setMainClass("org.myorg.WC").setMaster("local").launch();*
* spark.waitFor();*
* InputStreamReaderRunnable inputStreamReaderRunnable = new
InputStreamReaderRunnable(spark.getInputStream(),*
* "input");*
* Thread inputThread = new Thread(inputStreamReaderRunnable,
"LogStreamReader input");*
* inputThread.start();*
* InputStreamReaderRunnable errorStreamReaderRunnable = new
InputStreamReaderRunnable(spark.getErrorStream(),*
* "error");*
* Thread errorThread = new Thread(errorStreamReaderRunnable,
"LogStreamReader error");*
* errorThread.start();*
* System.out.println("Waiting for finish...");*
* int exitCode = spark.waitFor();*
* System.out.println("Finished! Exit code:" + exitCode);*
While this works perfectly fine.
Any pointers would be really helpful.
Thank you!
[image: http://]
Tariq, Mohammad
about.me/mti
[image: http://]
<http://about.me/mti>
[image: --]
Tariq, Mohammad
[image: https://]about.me/mti
<https://about.me/mti?promo=email_sig&utm_source=email_sig&utm_medium=external_link&utm_campaign=chrome_ext>
