[
https://issues.apache.org/jira/browse/CASSANDRA-3832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201042#comment-13201042
]
Peter Schuller commented on CASSANDRA-3832:
-------------------------------------------
What I'd like in this case is to send a request and register a callback with
the response that just puts a task on the migration stage that does the schema
merge. That, and keep track of outstanding requests and alter the logic for "is
ready to bootstrap".
I'm still fuzzy on the overall design of messaging; is it me or is there no way
to send a request and wait for a response without hogging a thread for it? I
want an on-delivery callback, not be waiting for a synchronous get(). A hacky
work-around would be to just keep track of all outstanding and get() them in
order - it would work, but would be a bit unclean to have one slow request
block others from being de-ref:ed.
> gossip stage backed up due to migration manager future de-ref
> --------------------------------------------------------------
>
> Key: CASSANDRA-3832
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3832
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Priority: Blocker
> Fix For: 1.1
>
> Attachments: CASSANDRA-3832-trunk-dontwaitonfuture.txt
>
>
> This is just bootstrapping a ~ 180 trunk cluster. After a while, a
> node I was on was stuck with thinking all nodes are down, because
> gossip stage was backed up, because it was spending a long time
> (multiple seconds or more, I suppose RPC timeout maybe) doing the
> following. Cluster-wide restart -> back to normal. I have not
> investigated further.
> {code}
> "GossipStage:1" daemon prio=10 tid=0x00007f9d5847a800 nid=0xa6fc waiting on
> condition [0x000000004345f000]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000005029ad1c0> (a
> java.util.concurrent.FutureTask$Sync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at
> org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:364)
> at
> org.apache.cassandra.service.MigrationManager.rectifySchema(MigrationManager.java:132)
> at
> org.apache.cassandra.service.MigrationManager.onAlive(MigrationManager.java:75)
> at org.apache.cassandra.gms.Gossiper.markAlive(Gossiper.java:802)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:918)
> at
> org.apache.cassandra.gms.GossipDigestAckVerbHandler.doVerb(GossipDigestAckVerbHandler.java:68)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
