Can you take them down sequentially? Like, say, with a Kubernetes StatefulSet <https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#ordered-pod-termination> .
On Wed, Feb 8, 2017 at 2:15 PM, Elias Levy <[email protected]> wrote: > What are folks doing to cleanly shutdown a Streams job comprised of > multiple workers? > > Right now I am doing sys.addShutdownHook(streams.close()) but that is not > working well to shutdown a fleet of workers. When I signal the fleet to > shutdown by sending them all a SIGTERM, some of them will shutdown, but > some will persist. It appears that there is a race condition between the > shutdown signal and a rebalancing occurring as a result of other workers > shutting down. If a worker has not started shutting down before the > rebalancing starts, the rebalancing will cause the worker to not shutdown. > > Others seeing the same thing? >
