so first limits are good, the unlimited row count of a user can eventually eat you, which I suspect it is here, you maybe better off partitioning your data with some reasonable limits, but this is a bigger domain modeling conversation. Second, tombstone overflowing is typically a canary for a data model that no longer fits the application's needs.
Typical options for tombstones are: 1. gc_grace_seconds to a much lower number. I'm not a huge fan of this strategy as it means you can easily introduce inconsistency if you don't handle repairs before gc_grace_seconds. 2. partition data in a way that makes it easier to manage tombstones, if there is a logical way to allocate data, either by problem domain or time then you can at some point safely truncate 'aged out' data. http://lostechies.com/ryansvihla/2014/10/20/domain-modeling-around-deletes-or-using-cassandra-as-a-queue-even-when-you-know-better/ I will say I'm not a huge fan of the soft delete pattern in Cassandra, it's like a permanent tombstone. On Wed, Dec 17, 2014 at 6:38 AM, Jens Rantil <jens.ran...@tink.se> wrote: > > Hi, > > I have a table with composite primary id ((userid), id). Some patterns > about my table: > * Each user generally has 0-3000 rows. But there is currently no upper > limit. > * Deleting rows for a user is extremely rare, but when done it can be > done thousands of rows at a time. > * The absolutely most common query is to select all rows for a user. > > Recently I saw a user that previously had 65000 tombstones when querying > for all his rows. system.log was printing TombstoneOverwhelmingException. > > What are my options to avoid this overwhelming tombstone exception? I am > willing to have slower queries than actually not being able to query at > all. I see a couple of options: > * Using an anti-column to mark rows as deleted. I could then control the > rate of which I am writing tombstones by occasionally deleting > anti-columns/rows with their equivalent rows. > * Simply raise tombstone_failure_threshold. AFAIK, this will eventually > make me run into possible GC issues. > * Use fetchSize to limit the number of rows paged through. This would > make every single query slower, and would not entirely avoid the > possibility of getting TombstoneOverwhelmingException. > > Have I missed any alternatives here? > > In the best of worlds, the fetchSize property would also honour the > number of tombstones, but I don’t think that would be possible, right? > > Thanks, > Jens > > ——— Jens Rantil Backend engineer Tink AB Email: jens.ran...@tink.se > Phone: +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter > -- [image: datastax_logo.png] <http://www.datastax.com/> Ryan Svihla Solution Architect [image: twitter.png] <https://twitter.com/foundev> [image: linkedin.png] <http://www.linkedin.com/pub/ryan-svihla/12/621/727/> DataStax is the fastest, most scalable distributed database technology, delivering Apache Cassandra to the world’s most innovative enterprises. Datastax is built to be agile, always-on, and predictably scalable to any size. With more than 500 customers in 45 countries, DataStax is the database technology and transactional backbone of choice for the worlds most innovative companies such as Netflix, Adobe, Intuit, and eBay.