Hi Romain,
I see. Thanks for the feedback, it's much appreciated. Since the only way to force it is through JMX I think we'll continue to use our current method. Best, Jerome ________________________________ From: Romain Hardouin <romainh...@yahoo.fr> Sent: Monday, September 5, 2016 1:30:24 PM To: user@cassandra.apache.org Subject: Re: Is it possible to replay hints after running nodetool drain? Hi, You don't have to worry about that unless you write with CL = ANY. The sole method to force hints that I know is to invoke scheduleHintDelivery on "org.apache.cassandra.db:type=HintedHandoffManager" via JMX but it takes an endpoint as argument. If you have lots of nodes and several DCs, make sure to properly set hinted_handoff_throttle_in_kb and max_hints_delivery_threads. Best, Romain Le Samedi 3 septembre 2016 2h59, jerome <jeromefroel...@hotmail.com> a ?crit : Hi Matija, Thanks for your help! The downtime is minimal, usually less than five minutes. Since it is so short we're not so concerned about the node that's down missing data, we just want to make sure that before it goes down it replays all the hints it has so that there won't be any gaps in data on any other nodes for the hints it has while it's down. Thanks, Jerome ________________________________ From: Matija Gobec <matija0...@gmail.com> Sent: Friday, September 2, 2016 6:05:01 PM To: user@cassandra.apache.org Subject: Re: Is it possible to replay hints after running nodetool drain? Hi Jerome, The node being drained stops listening to requests but the other nodes being coordinators for given requests will store hints for that downed node for a configured period of time (max_hint_window_in_ms is 3 hours by default). If the downed node is back online in this time window it will receive hints from other nodes in the cluster and eventually catch up. What is your typical maintenance downtime? Regards, Matija On Fri, Sep 2, 2016 at 10:53 PM, jerome <jeromefroel...@hotmail.com<mailto:jeromefroel...@hotmail.com>> wrote: Hello, As part of routine maintenance for our cluster, my colleagues and I will run a nodetool drain before stopping a Cassandra node, performing maintenance, and bringing it back up. We run maintenance as a cron-job with a lock stored in a different cluster to ensure only node is ever down at a time. We would like to make sure the node has replayed all its hints before bringing it down to minimize the potential window in which users might read out-of-date data (we read at a consistency level of ONE). Is it possible to replay hints after performing a nodetool drain? The documentation leads me to believe its not since Cassandra will stop listening for connections from other nodes, but I was unable to find anything definitive either way. If a node won't replay hints after a nodetool drain, is there perhaps another way to tell Cassandra to stop listening for client connections but continue to replay hints to other nodes. Thanks, Jerome
