Re: bitcask dead byte and fragment for riak-cs

2015-10-29 Thread Daniel Abrahamsson
Hi Mohamad. To keep your data available, Riak duplicates data and stores it on multiple nodes. The default number of replicas for a stored value is 3. If you have 4TB of image data, you should expect it to take up 12TB of space (+ size of metadata) in your Riak cluster. What is the total amount o

cuttlefish

2015-10-29 Thread Michael Martin
Hi folks, Sorry if this is the wrong venue for cuttlefish questions - please 302 me to the right place if need be. I would like to use cuttlefish in my application, but can't see how to hook it in. Doing the schemas is quite obvious - I have a small set of mappings ready to go. I just don't

Re: cuttlefish

2015-10-29 Thread Luke Bakken
Hi Michael, This wiki page may be of interest: https://github.com/basho/cuttlefish/wiki/Cuttlefish-for-Erlang-Developers After cuttlefish loads and processes your schemas, the variables and values are available via Erlang's standard application:get_env() function as though they were loaded from t

Re: Ownership handoff timed out

2015-10-29 Thread Vladyslav Zakhozhai
Hi, I want to make small update. Jon your hint about problems on sender side is correct. As I've already told there problems with available resources on sender nodes. There are no enough available RAM which is a cause of swapiness and load on disks. Restarting of sender nodes helps me (at least te

Re: Ownership handoff timed out

2015-10-29 Thread Vladyslav Zakhozhai
Hi, Average size of objects in Riak - 300 Kb. This objects are images. This data updates very very rearly (there almost no updates). I have GC turned on and works: root@python:~# riak-cs-gc status There is no garbage collection in progress The current garbage collection interval is: 900 The c

Re: cuttlefish

2015-10-29 Thread Luke Bakken
Hi Michael, I'm figuring this out as I go as well. I searched for "riak.conf" (https://github.com/basho/riak/search?q=riak.conf) and got a hit here: https://github.com/basho/riak/blob/develop/rel/rebar.config I suspect this configuration is what causes "riak.conf" to be generated at build time,

Re: Ownership handoff timed out

2015-10-29 Thread Matthew Von-Maszewski
Hi, There was a known eleveldb bug with handoff receiving that could cause a timeout. But it does not sound like bug fits your symptoms. However, I am willing to verify my diagnosis. I would need you to gather the LOG files from all vnodes on the RECEIVING side (or at least from the vnode th

Re: cuttlefish

2015-10-29 Thread Luke Bakken
Hi Michael, Please use "Reply All" so that riak-users is included in this discussion. I am assuming that the important part, for Riak, is here: https://github.com/basho/riak/blob/develop/rel/vars.config#L55-L59 Is your application open-source? I would be interested in getting relx support into

Re: cuttlefish

2015-10-29 Thread Michael Martin
It's not open source yet. It needs quite a bit more testing before I'm comfortable releasing it into the wild, but that will happen soon, I hope. On 10/29/2015 11:42 AM, Luke Bakken wrote: Hi Michael, Please use "Reply All" so that riak-users is included in this discussion. I am assuming tha

Re: Ownership handoff timed out

2015-10-29 Thread Matthew Von-Maszewski
Sad to say your LOG files suggest the same bug as seen elsewhere and fixed by recent changes in the leveldb code. The tougher issue is that the fixes are currently only available for our 2.0 product series. A backport would be non-trivial due to the number of places changed between 1.4 and 2.0

Re: Ownership handoff timed out

2015-10-29 Thread Vladyslav Zakhozhai
Matthew can you describe the bug more detail? My plan was to migrate to eleveldb and only then to migrate to Riak 2.0. It seems that I need to change my plans to migrate to Riak 2.0 first. It is sad. Is it safe to migrate Riak 1.4.12/Riak CS 1.5.0 to Riak 2.0 on production environment? According

Re: Ownership handoff timed out

2015-10-29 Thread Matthew Von-Maszewski
I queried Basho’s Client Services team. They tell me the upgrade / coexist should be no problem. Matthew > On Oct 29, 2015, at 1:38 PM, Vladyslav Zakhozhai > wrote: > > Matthew can you describe the bug more detail? > > My plan was to migrate to eleveldb and only then to migrate to Riak 2.0.