Thanks a lot. Sent from my iPad
> On Mar 7, 2015, at 8:26 AM, Sean Owen <so...@cloudera.com> wrote: > >> On Sat, Mar 7, 2015 at 4:17 PM, Jean-Pascal Billaud <j...@tellapart.com> >> wrote: >> So given this let's go a bit further. Imagine my static factory provides a >> stats collector that my various map() code would use to export some metrics >> while mapping tuples. This stats collector comes with a timer that flush the >> stats buffer once every 5mn for instance. Given that executor JVM should not >> be reinitialized over the life of the DStream in practice, it is totally >> reasonable to assume that this timer will be able to do his job. Right? > > Ignoring failures, the executors should not be restarted over the life > of the streaming app, yes. You will have, potentially, many executors > at once of course. > >> Given what you said earlier that totally makes sense. In general is there >> any spark architecture documentation other than the code that gives a good >> overview of the thing we talked about? > > http://spark.apache.org/docs/latest/cluster-overview.html > > This is entirely applicable to streaming too. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
