Timo, Your email again brought up the topic of fixing this issue within Basho's leveldb. Previously there had always been a "bigger problem" and we did not worry about tombstone (delete) purging. Today, we have all but one of the "bigger problems" coded. I therefore have created a plan to track tombstones and automatically perform background compactions to purge the old data and the tombstone.
This tombstone initiated, background compaction will occur at a lower priority than ingest and directory size compactions. The current plan would only "know about" tombstones created after the new code is released. It will not magically search your entire data set to find existing tombstones. But over time, any prior tombstones found in a regular compaction will lead to a grooming compaction of those tombstones. Matthew On Sep 25, 2013, at 3:31 PM, Timo Gatsonides <t...@me.com> wrote: > > Matthew, thanks very much for your feedback. I may bite the bullet and try > your suggested "last resort method" on one of the nodes. I'll let you know > how that goes. All nodes are 1.3 or newer. > > As to your question: my key is a bit of both and is quite long for typical > Riak usage I guess. The first part is a sort of device-ID indicating where > the data came from (typical Riak-generated random key generated when a new > device is added to the system), there are around 500 devices in the system > now. The second part is typical timestamp in yyyymmdd notation. As an > example: qvKKWKrCHLzRTYZBnDNkL6aaRFU20120619098357011462. This allows for > selection of a certain date range for a single device using 2i on $key which > actually works very well and has decent performance (the same timestamp is > also stored as an _int index to select across devices). > > Regards, > Timo > >> >> Here is an important quote from the Goggle document mentioned in the prior >> thread: >> >> ... >> One question for you: What is the nature of your "key"? Is it random, date >> based (increasing), etc.? >> > > > _______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
