Re: Truncated bit-cask files

2017-02-14 Thread Arun Rajagopalan
Thanks Matthew. I will try one of those solutions On Tue, Feb 14, 2017 at 3:51 PM, Matthew Von-Maszewski wrote: > Arun, > > You are running out of RAM for the leveldb AAE. There are several ways to > fix that: > > - reduce memory allocated to bitcask > - more memory per

Re: Truncated bit-cask files

2017-02-14 Thread Matthew Von-Maszewski
Arun, You are running out of RAM for the leveldb AAE. There are several ways to fix that: - reduce memory allocated to bitcask - more memory per server - more servers of same memory - reduce the ring size from 64 to 8, and rebuild data within the cluster from scratch - lie to leveldb and give

Re: Truncated bit-cask files

2017-02-14 Thread Matthew Von-Maszewski
Arun, The AAE code uses leveldb for its storage of anti-entropy data, no matter which backend holds the user data. Therefore the error below suggests corruption within leveldb files (which is not impossible, but becoming really rare except with bad hardware or full disks). Before wiping out

Re: Truncated bit-cask files

2017-02-14 Thread Magnus Kessler
On 14 February 2017 at 14:46, Arun Rajagopalan wrote: > Hi Magnus > > RIAK crashes on startup when I have trucated bitcask file > > It also crashes when the AAE files are bad too I think. Example below > > 2017-02-13 21:18:30 =CRASH REPORT > > crasher: > >

Re: Truncated bit-cask files

2017-02-14 Thread Arun Rajagopalan
Hi Magnus RIAK crashes on startup when I have trucated bitcask file It also crashes when the AAE files are bad too I think. Example below 2017-02-13 21:18:30 =CRASH REPORT crasher: initial call: riak_kv_index_hashtree:init/1 pid: <0.6037.0> registered_name: []

Truncated bit-cask files

2017-02-13 Thread Arun Rajagopalan
Hello Riak Users We have situations where we dont or cant gracefully stop riak. When that happens we occasionally get a truncated last-record in bitcask files If I delete those bitcask dir and the anti_entropy directory, Riak rebuilds those bitcask files correctly Is there a way to rectify