Antonio, A Riak delete operation happens in these steps:
- Riak writes a “tombstone” value for the key to the N vnodes that contain it (this is a new record) - Riak by default, waits 3 seconds to verify all vnodes agree to the tombstone/delete - Riak issues an actual delete operation against the key to leveldb - leveldb creates its own tombstone - the leveldb tombstone “floats” through level-0 and level-1 as part of normal compactions - upon reaching level-2, leveldb will initiate immediate compaction and propagation of tombstones in .sst table files containing 1000 or more tombstones. This is when disk space recovery begins. Yes, this means that initially the leveldb vnodes will grow in size until enough stuff (new data and/or tombstones) forces the tombstones to level-2 via normal compaction operations. “Enough stuff” to fill levels 0 and 1 is about 4.2Gbytes of compressed Riak objects. The “get” operation you mentioned is something that happens internally. A manual “get” by your code will not influence the operation. Matthew > On Jul 14, 2015, at 1:01 PM, Antonio Teixeira <eagle.anto...@gmail.com> wrote: > > Hi Matthew, > > We will be removing close to 1 TB of data from the node , and since we are > short on "disk space" when we saw that the disk space was actually rising we > halted the data removal. > > Now according to some docs I have read, if after a deletion ( a few seconds ) > we make a .get() it force the release of the diskspace , is this true ? > > For us it's not possible to move data to another node, is there any way even > manually to release the space ? or at least to force : > " The actual release occurs significantly later (days, weeks, or even months > later) when the tombstone record merges with the actual data in a background > compaction." > > Thanks for all the help, > Antonio > > > 2015-07-14 17:43 GMT+01:00 Matthew Von-Maszewski <matth...@basho.com > <mailto:matth...@basho.com>>: > Antonio, > > Here is a detailed discussion of the Riak / leveldb delete scenario: > > https://github.com/basho/leveldb/wiki/mv-aggressive-delete > <https://github.com/basho/leveldb/wiki/mv-aggressive-delete> > > Pay close attention to the section titled “Update April 6, 2014”. This > explains why as much as 4.2G bytes per vnode might remain within leveldb > after deleting all keys. There is no mechanism to override the logic that > causes the disk space retention. > > One workaround is to use Riak’s handoff mechanism to transfer vnodes from one > physical server to another. The vnode transfer will remove all deletion > tombstones on the destination. The last step of the transfer then deletes > all leveldb files on the original server, recovering the space. > > Matthew > > > > > On Jul 14, 2015, at 12:32 PM, Antonio Teixeira <eagle.anto...@gmail.com > > <mailto:eagle.anto...@gmail.com>> wrote: > > > > Hello, > > > > We have been migrating our Riak Database to another infrastructure through > > a "streaming" process and right now we should have somewhere around 2Gb of > > free space the Hard Disk, however those 2Gb are still being used by Riak. > > After some research I believe the problem is the Objects are only being > > marked for deletion and not actually deleted at runtime. What we need is a > > way to aggressively deleted those keys or some way to force Riak to delete > > those marked keys and subsequently release the Disk Space. The Riak version > > we are using is v2.0.2 > > _______________________________________________ > > riak-users mailing list > > riak-users@lists.basho.com <mailto:riak-users@lists.basho.com> > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > <http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com> > >
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
