Are you accidentally recovering from checkpoint files which has 10 second as the batch interval?
On Thu, Sep 3, 2015 at 7:34 AM, Dmitry Goldenberg <dgoldenberg...@gmail.com> wrote: > I'm seeing an oddity where I initially set the batchdurationmillis to 1 > second and it works fine: > > JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, > Durations.milliseconds(batchDurationMillis)); > > Then I tried changing the value to 10 seconds. The change didn't seem to > take. I've bounced the Spark workers and the consumers and now I'm seeing > RDD's coming in once around 10 seconds (not always 10 seconds according to > the logs). > > However, now I'm trying to change the value to 20 seconds and it's just > not taking. I've bounced Spark master, workers, and consumers and the value > seems "stuck" at 10 seconds. > > Any ideas? We're running Spark 1.3.0 built for Hadoop 2.4. > > Thanks. > > - Dmitry > >
