Agree with JEFF in twcs. Also look At https://github.com/paradoxical-io/cassieq for reference. Good ideas for a queue on Cassandra.
Rahul Singh Chief Executive Officer m 202.905.2818 Anant Corporation 1010 Wisconsin Ave NW, Suite 250 Washington, D.C. 20007 We build and manage digital business technology platforms. On Nov 28, 2018, 5:33 PM -0500, Adam Smith <adamsmith8...@gmail.com>, wrote: > Thanks for the excellent advice, this was extremely helpful! Did not know > about TWCS... curing a lot of headache. > > Adam > > > Am Mi., 28. Nov. 2018 um 20:47 Uhr schrieb Jeff Jirsa <jji...@gmail.com>: > > > Probably fine as long as there’s some concept of time in the partition > > > key to keep them from growing unbounded. > > > > > > Use TWCS, TTLs and something like 5-10 minute buckets. Don’t use RF=1, > > > but you can write at CL ONE. TWCS will largely just drop whole sstables > > > as they expire (especially with 3.11 and the more aggressive expiration > > > logic there) > > > > > > > > > > > > -- > > > Jeff Jirsa > > > > > > > > > > On Nov 28, 2018, at 11:24 AM, Adam Smith <adamsmith8...@gmail.com> > > > > wrote: > > > > > > > > Hi All, > > > > > > > > I need to use C* somehow as fluent data storage - maybe this is > > > > different to the queue antipattern? Lots of data come in > > > > (10MB/sec/node), remains for e.g. 1 hour and should then be evicted. It > > > > is somehow not critical when data would occasionally disappear/get lost. > > > > > > > > Thankful for any advice! > > > > > > > > Is this nowadays possible without suffering too much from compactation? > > > > I would not have ranged tombstones, and depending on a possible > > > > solution only using point deletes (PK+CK). There is only one CK, could > > > > also be empty. > > > > > > > > 1) The data is usually 1 MB. Can I just update with empty data? PK + CK > > > > would remain, but I would not carry about that. Would this create > > > > tombstones or is equivalent to a DELETE? > > > > > > > > 2) Like 1) and later then set a TTL == small amount of data to be > > > > deleted then? And hopefully small compactation? > > > > > > > > 3) Simply setting TTL 1h and hoping the best, because I am wrong with > > > > my worries? > > > > > > > > 4) Any optimization strategies like setting the RF to 1? Which > > > > compactation strategy is advised? > > > > > > > > 5) Are there any recent performance benchmarks for one of the scenarios? > > > > > > > > What else could I do? > > > > > > > > Thanks a lot! > > > > Adam > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > > > For additional commands, e-mail: user-h...@cassandra.apache.org > > >
