[ https://issues.apache.org/jira/browse/CASSANDRA-10485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15001310#comment-15001310 ]
Paulo Motta commented on CASSANDRA-10485: ----------------------------------------- bq. If we submit a hint to an endpoint that left, when will the hint be cleaned up and discarded? Is there a race there? The race window is very small, the node needs to be removed exactly between getting the ID from TokenMetadata and the hint being actually written by the HintsManager. If that happens, on 2.1 and 2.2 the hint will expire by ttl after gc_grace_seconds. On 3.0+, a hint file might be created for the removed node, and needs to be removed manually or via nodetool truncatehints. Fixed minor nit and removed {{isMemberJoining}} assertion. Tests were resubmitted and seem OK. ||2.1||2.2||3.0||trunk|| |[branch|https://github.com/apache/cassandra/compare/cassandra-2.1...pauloricardomg:2.1-10485-ultimate]|[branch|https://github.com/apache/cassandra/compare/cassandra-2.2...pauloricardomg:2.2-10485-ultimate]|[branch|https://github.com/apache/cassandra/compare/cassandra-3.0...pauloricardomg:3.0-10485-ultimate]|[branch|https://github.com/apache/cassandra/compare/trunk...pauloricardomg:trunk-10485-ultimate]| |[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-10485-ultimate-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10485-ultimate-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10485-ultimate-testall/lastCompletedBuild/testReport/]|[testall|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10485-ultimate-testall/lastCompletedBuild/testReport/]| |[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.1-10485-ultimate-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-2.2-10485-ultimate-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-3.0-10485-ultimate-dtest/lastCompletedBuild/testReport/]|[dtests|http://cassci.datastax.com/view/Dev/view/paulomotta/job/pauloricardomg-trunk-10485-ultimate-dtest/lastCompletedBuild/testReport/]| > Missing host ID on hinted handoff write > --------------------------------------- > > Key: CASSANDRA-10485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10485 > Project: Cassandra > Issue Type: Bug > Reporter: Paulo Motta > Assignee: Paulo Motta > Fix For: 2.1.x, 2.2.x, 3.0.x > > > when I restart one of them I receive the error "Missing host ID": > {noformat} > WARN [SharedPool-Worker-1] 2015-10-08 13:15:33,882 > AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread > Thread[SharedPool-Worker-1,5,main]: {} > java.lang.AssertionError: Missing host ID for 63.251.156.141 > at > org.apache.cassandra.service.StorageProxy.writeHintForMutation(StorageProxy.java:978) > ~[apache-cassandra-2.1.3.jar:2.1.3] > at > org.apache.cassandra.service.StorageProxy$6.runMayThrow(StorageProxy.java:950) > ~[apache-cassandra-2.1.3.jar:2.1.3] > at > org.apache.cassandra.service.StorageProxy$HintRunnable.run(StorageProxy.java:2235) > ~[apache-cassandra-2.1.3.jar:2.1.3] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > ~[na:1.8.0_60] > at > org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164) > ~[apache-cassandra-2.1.3.jar:2.1.3] > at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) > [apache-cassandra-2.1.3.jar:2.1.3] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60] > {noformat} > If I made nodetool status, the problematic node has ID: > {noformat} > UN 10.10.10.12 1.3 TB 1 ? > 4d5c8fd2-a909-4f09-a23c-4cd6040f338a rack3 > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)