Hi all,
I'm trying to load streaming context from a checkpoint: "val ctx = new
StreamingContext("\\Temp\\spark-checkpoint")" and I'm getting:
"Spark Streaming cannot be used without setting spark.cleaner.ttl; set this
property before creating a SparkContext (use SPARK_JAVA_OPTS for the shell)"
Apparently MetadataCleaner delay is set to default 3600 when streaming
context is created as "new StreamingContext("local", "cpc-stub",
Seconds(10))" but not when "new
StreamingContext("\\Temp\\spark-checkpoint")"
https://github.com/mesos/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala#L546
Is there a reason for spark.cleaner.ttl to be set to a default value in one
constructor but not in another? If it is just an omission, is github's
mesos/spark still preferred way to send pull request?
Vadim.
--
>From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is
explicitly specified
