Hi there,
we're using with the java-api (1.6.0) a ScheduledExecutor that
continuously executes a SparkJob to a standalone cluster.
After each job we close the JavaSparkContext and create a new one.
But sometimes the Scheduling JVM crashes with:
24.03.2016-08:30:27:375# error - Application has been killed. Reason:
All masters are unresponsive! Giving up.
24.03.2016-08:30:27:398# error - Error initializing SparkContext.
java.lang.IllegalStateException: Cannot call methods on a stopped
SparkContext.
This stopped SparkContext was created at:
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
io.datapath.spark.AbstractSparkJob.createJavaSparkContext(AbstractSparkJob.java:53)
io.datapath.measurement.SparkJobMeasurements.work(SparkJobMeasurements.java:130)
io.datapath.measurement.SparkMeasurementScheduler.lambda$submitSparkJobMeasurement$30(SparkMeasurementScheduler.java:117)
io.datapath.measurement.SparkMeasurementScheduler$$Lambda$17/1568787282.run(Unknown
Source)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
The currently active SparkContext was created at:
(No active SparkContext.)
at
org.apache.spark.SparkContext.org$apache$spark$SparkContext$$assertNotStopped(SparkContext.scala:106)
at
org.apache.spark.SparkContext.getSchedulingMode(SparkContext.scala:1578)
at
org.apache.spark.SparkContext.postEnvironmentUpdate(SparkContext.scala:2179)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:579)
at
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59)
at
io.datapath.spark.AbstractSparkJob.createJavaSparkContext(AbstractSparkJob.java:53)
at
io.datapath.measurement.SparkJobMeasurements.work(SparkJobMeasurements.java:130)
at
io.datapath.measurement.SparkMeasurementScheduler.lambda$submitSparkJobMeasurement$30(SparkMeasurementScheduler.java:117)
at
io.datapath.measurement.SparkMeasurementScheduler$$Lambda$17/1568787282.run(Unknown
Source)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
24.03.2016-08:30:27:402# info - SparkMeasurement - finished.
Any guess?
--
*Max Schmidt, Senior Java Developer* | m...@datapath.io
<mailto:m...@datapath.io> | LinkedIn
<https://www.linkedin.com/in/maximilian-schmidt-9893b7bb/>
Datapath.io
Decreasing AWS latency.
Your traffic optimized.
Datapath.io GmbH
Mainz | HRB Nr. 46222
Sebastian Spies, CEO
