Hints not delivered possibly because of pagination issue --------------------------------------------------------
Key: CASSANDRA-3935 URL: https://issues.apache.org/jira/browse/CASSANDRA-3935 Project: Cassandra Issue Type: Bug Affects Versions: 1.1.0 Reporter: B. Todd Burruss I'm testing hinted handoff in 1.1 beta1 and cannot seem to get a hint delivered. 3 node cluster, RF = 3, writing with CL = ONE. killed a host then did the write using the CLI on another node. I can see hint waiting using CLI and I see the log messages at the end of this email. this suggests the hints exist bu are not being delivered (and I'll see the log messages over and over.) I did see tracing with debugger and see that in HintedHandoffManager.deliverHintsToEndpointInternal, this line will remove the hint because of the Integer.MAX_VALUE ColumnFamily hintsPage = ColumnFamilyStore.removeDeleted(hintStore.getColumnFamily(filter), Integer.MAX_VALUE); I'm not sure I quite understand while MAX is used when the same "remove" is done in getColumnFamily(filter). regardless if it is useful or not, it prevents the hints from delivery. any thoughts? [default@unknown] use system; Authenticated to keyspace: system [default@system] list hintscolumnfamily; Using default limit of 100 ------------------- RowKey: 00 => (super_column=493ecfa05c1411e100000da23097c7ff, (column=6b6579, value=6b35, timestamp=1329777799580, ttl=86400) (column=6d75746174696f6e, value=000662746f64646200026b3500000001000003e801000003e87fffffff80000000000000000000000100026335000004b96d055fd1300000000c68696e746564207772697465, timestamp=1329777799579, ttl=86400) (column=7461626c65, value=62746f646462, timestamp=1329777799580, ttl=86400) (column=76657273696f6e, value=00000004, timestamp=1329777799580, ttl=86400)) 1 Row Returned. Elapsed time: 58 msec(s). INFO [HintedHandoff:1] 2012-02-20 14:44:53,811 HintedHandOffManager.java (line 296) Started hinted handoff for token: 0 with IP: /192.168.56.1 INFO [HintedHandoff:1] 2012-02-20 14:44:53,815 HintedHandOffManager.java (line 373) Finished hinted handoff of 0 rows to endpoint /192.168.56.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira