[ https://issues.apache.org/jira/browse/USERGRID-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Todd Nine updated USERGRID-909: ------------------------------- Comment: was deleted (was: Usergrid currently performs 5 shard lookups on write. We'll change the following *Shard allocation* ) > Evaluate removing cache from edge shard management > -------------------------------------------------- > > Key: USERGRID-909 > URL: https://issues.apache.org/jira/browse/USERGRID-909 > Project: Usergrid > Issue Type: Story > Components: Stack > Reporter: Todd Nine > Assignee: Todd Nine > > Currently, using caches with the proposal phase for new shards has caused > large numbers of tombstones. Review the algorithm, and determine the cause > of the additional shard proposals. > Usergrid currently performs 5 shard lookups on write and most miss the cache. > We'll change the following > *Shard allocation* > Shard allocation will not perform a propose + check phase. The following > algorithm will occur. > Any node will propose a new shard pivot. This pivot will have compacted set > to false, and will have a column timestamp of a new timeuuid in micros > The proposing node will then read back all compacted = false shard pivots. > The shard pivot with the minimum timestamp will be retained, all others will > be deleted as proposals. This will function similarly to distributed read > locks in Cassandra. Lowest proposed value always wins. > Allocation is complete > *Shard compaction* > After a successful proposal allocation, the allocating node will copy all > edges from the previously compacted shard into the new shard. It will > continue to copy until no new edges are returned on read. > Once read has been completed, all edges that were copied can be deleted from > the previous shard. > This target shard will be marked as compacted > *Consistency* > When performing consistency, we will need a higher consistency level. To > ensure that shards exist worldwide, proposal + selection should occur at > EACH_QUORUM. Standard reads can occur as LOCAL_QUORUM -- This message was sent by Atlassian JIRA (v6.3.4#6332)