Hello,
This is about SPARK-3276 and I want to make MIN_REMEMBER_DURATION (that is
now a constant) a variable (configurable, with a default value). Before
spending effort on developing something and creating a pull request, I
wanted to consult with the core developers to see which approach makes
Approach 2 is definitely better :)
Can you tell us more about the use case why you want to do this?
TD
On Wed, Apr 8, 2015 at 1:44 AM, Emre Sevinc emre.sev...@gmail.com wrote:
Hello,
This is about SPARK-3276 and I want to make MIN_REMEMBER_DURATION (that is
now a constant) a variable
Tathagata,
Thanks for stating your preference for Approach 2.
My use case and motivation are similar to the concerns raised by others in
SPARK-3276. In previous versions of Spark, e.g. 1.1.x we had the ability
for Spark Streaming applications to process the files in an input directory
that
+1 for this feature
In our use case, we probably wouldn’t use this feature in production, but it
can be useful during prototyping and algorithm development to repeatedly
perform the same streaming operation on a fixed, already existing set of files.
-
jeremyfreeman.net
