Good tip. I will try that. Thank you.
On Wed, Sep 2, 2015 at 6:54 PM, Cody Koeninger <c...@koeninger.org> wrote: > Yeah, in general if you're changing the jar you can't recover the > checkpoint. > > If you're just changing parameters, why not externalize those in a > configuration file so your jar doesn't change? I tend to stick even my > app-specific parameters in an external spark config so everything is in one > place. > > On Wed, Sep 2, 2015 at 4:48 PM, Ricardo Luis Silva Paiva < > ricardo.pa...@corp.globo.com> wrote: > >> Hi, >> >> Is there a way to submit an app code change, keeping the checkpoint data >> or do I need to erase the checkpoint folder every time I re-submit the >> spark app with a new jar? >> >> I have an app that count pageviews streaming from Kafka, and deliver a >> file every hour from the past 24 hours. I'm using reduceByKeyAndWindow with >> the reduce and inverse functions set. >> >> I'm doing some code improvements and would like to keep the data from the >> past hours, so when I re-submit a code change, I would keep delivering the >> pageviews aggregation without need to wait for 24 hours of new data. >> Sometimes I'm just changing the submission parameters, like number of >> executors, memory and cores. >> >> Many thanks, >> >> Ricardo >> >> -- >> Ricardo Paiva >> Big Data / Semântica >> *globo.com* <http://www.globo.com> >> > > -- Ricardo Paiva Big Data / Semântica *globo.com* <http://www.globo.com>
