riak key sharding

2013-12-10 Thread Georgio Pandarez
Hi, I have noticed that Riak-CS can shard (that is split) large keys automatically across nodes. I would like to achieve a similar outcome with Riak itself. Is there any best practice to achieve this? Could a portion of Riak-CS be used or should I just bite the bullet and use Riak-CS? Latency is

Upgrade from 1.3.1 to 1.4.2 = high IO

2013-12-10 Thread Simon Effenberg
Hi @list, I'm trying to upgrade our Riak cluster from 1.3.1 to 1.4.2 .. after upgrading the first node (out of 12) this node seems to do many merges. the sst_* directories changes in size rapidly and the node is having a disk utilization of 100% all the time. I know that there is something like

Re: Upgrade from 1.3.1 to 1.4.2 = high IO

2013-12-10 Thread Simon Effenberg
Hi Matthew, see inline.. On Tue, 10 Dec 2013 10:38:03 -0500 Matthew Von-Maszewski matth...@basho.com wrote: The sad truth is that you are not the first to see this problem. And yes, it has to do with your 950GB per node dataset. And no, nothing to do but sit through it at this time.

Re: Stalled handoffs on a prod cluster after server crash

2013-12-10 Thread Simon Effenberg
I had something like that once but with version 1.2 or 1.3 .. a rolling restart helped in my case. /s On Mon, 9 Dec 2013 09:48:12 -0500 Ivaylo Panitchkov ipanitch...@hibernum.com wrote: Hello, We have a prod cluster of four machines running riak (1.1.4 2012-06-19) Debian x86_64. Two days

Re: Upgrade from 1.3.1 to 1.4.2 = high IO

2013-12-10 Thread Matthew Von-Maszewski
The sad truth is that you are not the first to see this problem. And yes, it has to do with your 950GB per node dataset. And no, nothing to do but sit through it at this time. While I did extensive testing around upgrade times before shipping 1.4, apparently there are data configurations I

Re: Upgrade from 1.3.1 to 1.4.2 = high IO

2013-12-10 Thread Matthew Von-Maszewski
2i is not my expertise, so I had to discuss you concerns with another Basho developer. He says: Between 1.3 and 1.4, the 2i query did change but not the 2i on-disk format. You must wait for all nodes to update if you desire to use the new 2i query. The 2i data will properly write/update on

Re: Upgrade from 1.3.1 to 1.4.2 = high IO

2013-12-10 Thread Simon Effenberg
Hi Matthew, thanks!.. that answers my questions! Cheers Simon On Tue, 10 Dec 2013 11:08:32 -0500 Matthew Von-Maszewski matth...@basho.com wrote: 2i is not my expertise, so I had to discuss you concerns with another Basho developer. He says: Between 1.3 and 1.4, the 2i query did change

2i stopped working on LevelDB with multi backend

2013-12-10 Thread Chris Read
We just rebuilt our test environment (something we do every month) and suddenly we get the following error when trying to use 2i: {error,{error,{indexes_not_supported,riak_kv_multi_backend}}} But looking at the properties of the bucket it's set to use leveldb: # curl -k

Re: riak nagios script

2013-12-10 Thread Hector Castro
Hello Kathleen, Have you executed the `make encrypt` target to build the `check_node` binary? [0] From there, I copied it to the Riak node and invoked it like this: $ /usr/lib/riak/erts-5.9.1/bin/escript check_node --node riak@127.0.0.1 riak_kv_up OKAY: riak_kv is running on riak@127.0.0.1 I

Re: riak nagios script

2013-12-10 Thread kzhang
Thanks Hector. Here is how I executed the script. I downloaded and installed the erlang shell from http://www.erlang.org/documentation/doc-5.3/doc/getting_started/getting_started.html started erlang OTP: root@MYRIAKNODE otp_src_R16B02]# erl -s toolbar Erlang R16B02 (erts-5.10.3) [source]

Riak Recap for December 4 - 9

2013-12-10 Thread Mark Phillips
Morning, Afternoon, Evening to All - Here's today's Recap. Enjoy. Also, if you're around Raleigh/Durham and want to have drinks next week, let me know. Mark twitter.com/pharkmillups --- Riak Recap for December 4 - 9 == The recording of last

Re: riak nagios script

2013-12-10 Thread Alex Moore
Hi Kathleen, If you’d like to run riak_nagios from the erl command line, you’ll need to compile everything in src and include it in the path along with the getopt library.    You can compile everything with a simple call to make, and then include it in the path with erl -pa deps/*/ebin ebin”.

Re: Stalled handoffs on a prod cluster after server crash

2013-12-10 Thread Jeppe Toustrup
What does riak-admin transfers tell you? Are there any transfers in progress? You can try to set the amount of allowed transfers per host to 0 and then back to 2 (the default) or whatever you want, in order to restart any transfers which may be in progress. You can do that with the riak-admin

Re: Stalled handoffs on a prod cluster after server crash

2013-12-10 Thread Ivaylo Panitchkov
Hello, Below is the transfers info: ~# riak-admin transfers Attempting to restart script through sudo -u riak 'r...@ccc.ccc.ccc.ccc' waiting to handoff 7 partitions 'r...@bbb.bbb.bbb.bbb' waiting to handoff 7 partitions 'r...@aaa.aaa.aaa.aaa' waiting to handoff 5 partitions ~# riak-admin

Re: Stalled handoffs on a prod cluster after server crash

2013-12-10 Thread Jeppe Toustrup
Try to take a look at this thread from November where I experienced a similar problem: http://lists.basho.com/pipermail/riak-users_lists.basho.com/2013-November/014027.html The following mails in the thread mentions things you try to correct the problem, and what I ended up doing with the help of

Re: riak nagios script

2013-12-10 Thread kzhang
Hi Alex, Thanks. I am completely new to erlang. When googling how to run an erlang program, I came across http://www.erlang.org/documentation/doc-5.3/doc/getting_started/getting_started.html . That's how I got started. To run the script using escript, based on

Re: Stalled handoffs on a prod cluster after server crash

2013-12-10 Thread Mark Phillips
Hi Ivaylo, Is there anything useful in console.log of any (or all) the nodes? If so, throw it in a gist and we'll take a look at it. Mark On Tue, Dec 10, 2013 at 1:13 PM, Jeppe Toustrup je...@falconsocial.com wrote: Try to take a look at this thread from November where I experienced a similar