Hi - just wondering what other folks do when a Samza job needs to be shutdown purposefully, such as during a deployment. Seems like in this case you'd want all of the job's Samza containers to cleanly stop processing messages, commit offsets/changelog, etc instead of just SIGKILL the jvms which will result in out-of-date checkpoints, etc.
I noticed TaskCoordinator has a shutdown method, which the StreamTasks in a container can use to signal a clean shutdown. Does everyone use this, with some kind of shutdown message sent to the StreamTasks using e.g. jmx or a kafka topic, as in this thread? http://www.mail-archive.com/dev%40samza.incubator.apache.org/msg01248.html We're actively developing Samza support for Mesos, so maybe this is specific to that world, but would it make sense for a jvm shutdown hook in the container to catch a SIGTERM and somehow signal to the container that it should shutdown? I can sorta make this work, but not in a very clean way. Just curious what others are doing about graceful shutdown. Thanks, Zach
