[ https://issues.apache.org/jira/browse/CASSANDRA-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806901#comment-13806901 ]
Aleksey Yeschenko commented on CASSANDRA-6134: ---------------------------------------------- bq. Do you mean time jumping, if operator forcibly changes time on machine or some other scenario ? Yup. That's a minor concern though. bq. Using it without COMPACT STORAGE will add 2x to memory and disk. How so? And yeah, having the ability to add a map or a set with some extra metadata there is useful. While it hasn't been done to the batchlog, we've done it for other system cf-s (system.schema_columnfamilies for one) and were burnt by COMPACT with system.schema_keyspaces (can't switch rf options to a map and have to keep the ghetto-json b/c can't add a map) (see CASSANDRA-4603). > More efficient BatchlogManager > ------------------------------ > > Key: CASSANDRA-6134 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6134 > Project: Cassandra > Issue Type: Improvement > Reporter: Oleg Anastasyev > Priority: Minor > Attachments: BatchlogManager.txt > > > As we discussed earlier in CASSANDRA-6079 this is the new BatchManager. > It stores batch records in > {code} > CREATE TABLE batchlog ( > id_partition int, > id timeuuid, > data blob, > PRIMARY KEY (id_partition, id) > ) WITH COMPACT STORAGE AND > CLUSTERING ORDER BY (id DESC) > {code} > where id_partition is minute-since-epoch of id uuid. > So when it scans for batches to replay ot scans within a single partition for > a slice of ids since last processed date till now minus write timeout. > So no full batchlog CF scan and lot of randrom reads are made on normal > cycle. > Other improvements: > 1. It runs every 1/2 of write timeout and replays all batches written within > 0.9 * write timeout from now. This way we ensure, that batched updates will > be replayed to th moment client times out from coordinator. > 2. It submits all mutations from single batch in parallel (Like StorageProxy > do). Old implementation played them one-by-one, so client can see half > applied batches in CF for a long time (depending on size of batch). > 3. It fixes a subtle racing bug with incorrect hint ttl calculation -- This message was sent by Atlassian JIRA (v6.1#6144)