Re: Riak LevelDB Deletion Problem

2015-07-16 Thread Antonio Teixeira
Timo, After one hour it spitted the data we needed , for the record : curl YOURIP:8098/buckets/YOURBUCKET/index/\$bucket/_ Cheers, Antonio 2015-07-15 21:51 GMT+01:00 Timo Gatsonides : > From: Antonio Teixeira > To: Matthew Von-Maszewski > Cc: riak-users > Subject: Re: Riak Leve

Re: Riak LevelDB Deletion Problem

2015-07-15 Thread Timo Gatsonides
> From: Antonio Teixeira > To: Matthew Von-Maszewski > Cc: riak-users > Subject: Re: Riak LevelDB Deletion Problem > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > Hello Matthew, > > Space is reducing slowly , now we

Re: Riak LevelDB Deletion Problem

2015-07-15 Thread Antonio Teixeira
Hello Matthew, Space is reducing slowly , now we are faced with another problem : Alot of applications store data on this node and we don't have the bucket keys (they are uuid4) so : We are using listkeys ( I know its bad ) on the erlang client and we also tried with curl using both blocking

Re: Riak LevelDB Deletion Problem

2015-07-14 Thread Matthew Von-Maszewski
Antonio, Someone reminded me that you could make temporary space on your servers by deactivating active_anti_entropy, then deleting its data. Of course, this assumes you are running “anti_entropy = active” in your riak.conf file. I will send you some better notes if you think this is worth res

Re: Riak LevelDB Deletion Problem

2015-07-14 Thread Antonio Teixeira
Ok Matthew, We will proceed with the deletion , will monitor the disk space and will come back with further reports to the list. Thanks for your time, Antonio 2015-07-14 18:32 GMT+01:00 Matthew Von-Maszewski : > Antonio, > > A Riak delete operation happens in these steps: > > - Riak writes a “t

Re: Riak LevelDB Deletion Problem

2015-07-14 Thread Matthew Von-Maszewski
Antonio, A Riak delete operation happens in these steps: - Riak writes a “tombstone” value for the key to the N vnodes that contain it (this is a new record) - Riak by default, waits 3 seconds to verify all vnodes agree to the tombstone/delete - Riak issues an actual delete operation against

Re: Riak LevelDB Deletion Problem

2015-07-14 Thread Antonio Teixeira
Hi Matthew, We will be removing close to 1 TB of data from the node , and since we are short on "disk space" when we saw that the disk space was actually rising we halted the data removal. Now according to some docs I have read, if after a deletion ( a few seconds ) we make a .get() it force the

Re: Riak LevelDB Deletion Problem

2015-07-14 Thread Matthew Von-Maszewski
Antonio, Here is a detailed discussion of the Riak / leveldb delete scenario: https://github.com/basho/leveldb/wiki/mv-aggressive-delete Pay close attention to the section titled “Update April 6, 2014”. This explains why as much as 4.2G bytes per vnode might remain within leveldb after d

Riak LevelDB Deletion Problem

2015-07-14 Thread Antonio Teixeira
Hello, We have been migrating our Riak Database to another infrastructure through a "streaming" process and right now we should have somewhere around 2Gb of free space the Hard Disk, however those 2Gb are still being used by Riak. After some research I believe the problem is the Objects are only b