Re: power outage
There is no background process to read all keys in the cluster in this version of Riak. The Active Anti Entropy(AAE)feature added in future versions odes precisely this. It notices any replicas that are out of sync and forces read repair. If the manual repair procedure is not working you might consider upgrading to 1.4.8(most recent stable release) and letting AAE take care of the repair. On Fri, Apr 11, 2014 at 2:09 AM, Gerlach, Sebastian < sebastian.gerl...@immonet.de> wrote: > Hi Brian, Hi mailing list, > > Is ist possible to read all keys from the cluster an trigger in this way > a read repay on the failed nodes? > > Thanks and best regards > Sebastian > > > Von: Brian Sparrow > Datum: Donnerstag, 10. April 2014 20:19 > An: Sebastian Gerlach > Cc: riak-users > Betreff: Re: power outage > > Sebastian, > > Those errors are normal following an outage of this sort. Hintfiles will > be regenerated during the next bitcask merge and any data file that is > incomplete or has invalid entries will be truncated. This does result in a > loss of replicas but as long as at least 1 replica of the data is not lost > the replicas can be repaired by manual repair or read-repair. The manual > repair procedure is outlined here: > http://docs.basho.com/riak/1.2.0/cookbooks/Repairing-KV-Indexes/ . > > I recommend that you manually repair any partition that discarded data > file entries but other than that your cluster should be working fine. > > Thanks, > Brian > > > > > On Thu, Apr 10, 2014 at 1:44 PM, Gerlach, Sebastian < > sebastian.gerl...@immonet.de> wrote: > >> Dear mailing list, >> >> Our prod riak cluster goes down with an power outage. We restartet riak an >> found a lot of the following messages: >> >> [error] <0.22012.0> Hintfile >> >> '/var/lib/riak/bitcask/174124218510127105489110888272838406638695088128/142 >> .bitcask.hint' has bad CRC 28991324 expected 0 >> [error] <0.5063.0> Bad datafile entry, discarding(3450/46927 bytes) >> >> '/var/lib/riak/bitcask/302576510853663494784356625523292968913142284288/140 >> .bitcask.hint' contains pointer 949927320 13881 that is greater than total >> data size 949927936 >> >> >> Our Cluster contains 12 Server. >> We run version 1.2.0-1 on Debian 6 >> >> We are not sure if the error are fixed by riak itself. We found an old >> mailing list entry with the hint to run >> "riak_kv:repair(22835963083295358096932575511191922182123945984)." >> >> We also checked the Dokumentaion and found: >> http://docs.basho.com/riak/1.2.0/cookbooks/Repairing-KV-Indexes/ "> >> [riak_kv_vnode:repair(P) || P <- Partitions]." >> >> Both hints doesn't work on our system. >> >> Have anyone some hints for me. >> >> Thanks a lot >> >> Sebastian >> >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Command line suggestions.
Hi Guido, Thanks a lot for the suggestions. I've passed them on internally. -- Luke Bakken CSE lbak...@basho.com On Fri, Apr 11, 2014 at 2:03 AM, Guido Medina wrote: > Hi, > > Base on the documentation and tricks I have seen to fix/repair stuff in Riak > I would suggest the following approach: > > riak-admin [cluster] (if cluster is specified then run the action > at every node) > > and I know most of the commands are designed like that but adding to them > for specific actions will help admin to keep things tidy up: > > riak-admin [cluster] repair-partitions - Instead of listing partitions per > node via Erlang and running it at every node. > riak-admin [cluster] repair-2i - This is already part of riak-admin without > the cluster level. > riak-admin [cluster] aae-cleanup - Stops AAE, deletes anti_entropy > information and starts AAE, again; if cluster is specified, run it at every > node. > riak-admin [cluster] aae-start | aae-stop - Handy when doing lots of > hand-off, should be start-aae and stop-aae from the English perspective but > then from the alphabetical perspective when documenting it can be confusing. > > I do believe the 5 actions listed should all have a local vs cluster > counterpart, don't get me wrong, Erlang is nice (I have plans to do deep > Erlang stuff integrated into Riak and RabbitMQ for our application) but from > the DevOps perspective, for scripting having each command line possible can > save lot of time and avoid mistakes, like, what node am I, etc. > > Regards, > > Guido. > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RE: RIAK best practices for storage
Not any deeper. But a node can have a couple of thousand children in some cases. Query pattern is usually fetch all children for a parent. Thanks, Meghna -Original Message- From: Sargun Dhillon [mailto:sar...@sargun.me] Sent: Friday, April 11, 2014 10:25 AM To: Sapre, Meghna A Cc: riak-users@lists.basho.com Subject: Re: RIAK best practices for storage How much deeper does your tree go? What's the average number of children a node has? What is your query pattern (fetch a parent, and all of its children?)? On Fri, Apr 11, 2014 at 10:13 AM, Sapre, Meghna A wrote: > Hi all, > > Most of my data is in parent-child format (1:n). > > For read/write performance, is it better to store parents in a > separate bucket and reference with IDs in children, or store complete > copies of parents within children? > > > > Thanks, > > Meghna > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: RIAK best practices for storage
How much deeper does your tree go? What's the average number of children a node has? What is your query pattern (fetch a parent, and all of its children?)? On Fri, Apr 11, 2014 at 10:13 AM, Sapre, Meghna A wrote: > Hi all, > > Most of my data is in parent-child format (1:n). > > For read/write performance, is it better to store parents in a separate > bucket and reference with IDs in children, or store complete copies of > parents within children? > > > > Thanks, > > Meghna > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RIAK best practices for storage
Hi all, Most of my data is in parent-child format (1:n). For read/write performance, is it better to store parents in a separate bucket and reference with IDs in children, or store complete copies of parents within children? Thanks, Meghna ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
reversing node removal?
I have a 5-node cluster (riak 1.4.0, freebsd9) that is being used in production and miscalculated the disk space being used by the cluster as a whole. Yesterday I told the cluster to remove two nodes, leaving just three, but I need four active to cover the usage. One node left successfully before I became aware of the problem, and disk filled up completely on the other three. I added the one that left back to the cluster, but data is not being moved to it. Is there any way to 'abort' the cluster leave issued to the node that is still trying to leave, or some other way to straighten this out without losing (much) data? ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Command line suggestions.
Hi, Base on the documentation and tricks I have seen to fix/repair stuff in Riak I would suggest the following approach: riak-admin [cluster] (if cluster is specified then run the action at every node) and I know most of the commands are designed like that but adding to them for specific actions will help admin to keep things tidy up: * riak-admin [cluster] repair-partitions - Instead of listing partitions per node via Erlang and running it at every node. * riak-admin [cluster] repair-2i - This is already part of riak-admin without the cluster level. * riak-admin [cluster] aae-cleanup - Stops AAE, deletes anti_entropy information and starts AAE, again; if cluster is specified, run it at every node. * riak-admin [cluster] aae-start | aae-stop - Handy when doing lots of hand-off, should be start-aae and stop-aae from the English perspective but then from the alphabetical perspective when documenting it can be confusing. I do believe the 5 actions listed should all have a local vs cluster counterpart, don't get me wrong, Erlang is nice (I have plans to do deep Erlang stuff integrated into Riak and RabbitMQ for our application) but from the DevOps perspective, for scripting having each command line possible can save lot of time and avoid mistakes, like, what node am I, etc. Regards, Guido. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Rebuilding AAE hashes - small question
> Date: Thu, 10 Apr 2014 14:42:23 +0100 > From: Guido Medina > To: Engel Sanchez , Luke Bakken > Cc: riak-users > Subject: Re: Rebuilding AAE hashes - small question > Message-ID: <53469fbf.1000...@temetra.com> > Content-Type: text/plain; charset="iso-8859-1"; Format="flowed" > > Thanks Engel, > > That approach looks very accurate, I would only suggest to have a > riak-admin cluster stop-aae and similar for start, for the dummies ;-) > I agree with Guido that would be the best solution for the dummies. But a good interim solution would be if Basho puts up a “definitive guide to enabling and disabling AAE” somewhere. I have now seen a lot of different commands to achieve this. In this thread and in the “RIAK 1.4.6 - Mass key deletion” thread and some older threads. There is the per-host approach and there seem to be several cluster-wide / rpc approaches. I have seen: a. rpc:multicall(riak_kv_entropy_manager, disable, []). rpc:multicall(riak_kv_entropy_manager, cancel_exchanges, []). z. (with questions about the a. and z. ) and there is: riak_core_util:rpc_every_member_ann(riak_kv_entropy_manager, disable, [], 6). And similar to enable. Can someone please advise what is the “authoritative” way to enable/disable AAE? Kind regards, Timo ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
CfP: NoSQL Databases and Linked Data Applications (Deadline: April 18)
[Apologies for crossposting] Call for industrial and research papers for the 1st International Workshop on NoSQL Databases and Linked Data Applications (part of DEXA 2014 in Munich). For industrial papers: We are looking for experience reports on NoSQL databases and linked open data in industrial applications and production environments. For research papers: This workshop is intended to be a meeting point for the research communities of NoSQL data management and Linked Data. It aims to instigate discussion and new collaboration between researchers of the two communities. Submission deadline for full papers: April 18, 2014 (EXTENDED) Workshop date: September 1 or 2, 2014 More information: http://www.ksi.mff.cuni.cz/NoSQL-Net2014/cfp.html ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
