Incremental bootstrap patch changed the logic here. A node can act as a coordinator before it's fully joined. It's ... decidedly non-ideal and probably needs to be changed. https://issues.apache.org/jira/browse/CASSANDRA-8942
On Wed, May 6, 2020 at 9:57 PM Erick Ramirez <erick.rami...@datastax.com> wrote: > The fact that a new node is acting as a coordinator suggests that (1) you > are adding a node to a DC that is taking traffic from the app, and (2) you > are likely adding the node using nodetool rebuild instead of the standard > bootstrap method. > > If you are adding a node using the rebuild option (with auto_bootstrap > set to false in cassandra.yaml), the node joins the cluster as a normal > node except it doesn't have any data to serve read requests but it will > accept writes. This isn't a recommended way of adding nodes to a DC that is > actively serving requests from the app. > > As I understand only coordinator nodes generates hints files. Is >> cluster uses this node as coordinator before complete join? >> Or this process is normal for joining and seen as repair? >> > > The fact that the new node is storing hints is a concern because it > indicates that other nodes in your cluster are unresponsive or down. You > need to investigate why that is the case. > > >> By the way, files not deleted after 3 hour period. > > > The node will collect hints for other nodes for 3 hours (default). If the > replica has not come back online after 3 hours, hints will no longer be > stored but it doesn't delete the hints but instead hand it off to the > respective replica when it comes back online. Cheers! >
