Re: Getting error: Riak-cs user please reduce your request rate
Hi Rahul, I'm bringing this back on list in case anyone else has this issue. Did you create an admin user as in step 4 listed here? http://docs.basho.com/riakcs/latest/tutorials/fast-track/Building-a-Local-Test-Environment/#Installing-Additional-Nodes -Andrew On Tue, Jun 18, 2013 at 1:44 AM, Rahul Bongirwar < bongirwar.rahul...@gmail.com> wrote: > Hi, > > All process are running on my node ( riak, riak-cs, stanchion ) but still > getting same error. > > I tried this on atleast 3 different node but facing same problem. > > Thanks, > Rahul > > > On Thu, Jun 13, 2013 at 9:36 PM, Andrew Stone wrote: > >> Hi Rahul, >> >> That error message is misleading. In general it can be treated as a 500 >> error. My guess is that you do not have stanchion running. Stanchion >> provides a serialization layer for creating users and buckets. See >> http://docs.basho.com/riakcs/latest/tutorials/fast-track/Building-a-Local-Test-Environment/for >> more details. >> >> -Andrew >> >> >> On Thu, Jun 13, 2013 at 6:20 AM, Rahul Bongirwar < >> bongirwar.rahul...@gmail.com> wrote: >> >>> Hi, >>> >>> I am getting error while creating riak-cs user as "Please reduce your >>> request rate" >>> What is the solution for it? >>> >>> Thanks, >>> Rahul >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Getting error: Riak-cs user please reduce your request rate
Did you restart Stanchion and Riak-CS after changing the config? Do you have {anonymous_user_creation, true} in riak-cs app.config ? -Andrew On Thu, Jun 20, 2013 at 7:59 AM, Rahul Bongirwar < bongirwar.rahul...@gmail.com> wrote: > Hi, > > Actually i am getting this error while creating a first user after all > configuration done. So not able to create admin user also. > > Thanks, > Rahul > > > > On Thu, Jun 20, 2013 at 5:25 PM, Andrew Stone wrote: > >> Hi Rahul, I'm bringing this back on list in case anyone else has this >> issue. >> >> Did you create an admin user as in step 4 listed here? >> >> http://docs.basho.com/riakcs/latest/tutorials/fast-track/Building-a-Local-Test-Environment/#Installing-Additional-Nodes >> >> -Andrew >> >> >> On Tue, Jun 18, 2013 at 1:44 AM, Rahul Bongirwar < >> bongirwar.rahul...@gmail.com> wrote: >> >>> Hi, >>> >>> All process are running on my node ( riak, riak-cs, stanchion ) but >>> still getting same error. >>> >>> I tried this on atleast 3 different node but facing same problem. >>> >>> Thanks, >>> Rahul >>> >>> >>> On Thu, Jun 13, 2013 at 9:36 PM, Andrew Stone wrote: >>> Hi Rahul, That error message is misleading. In general it can be treated as a 500 error. My guess is that you do not have stanchion running. Stanchion provides a serialization layer for creating users and buckets. See http://docs.basho.com/riakcs/latest/tutorials/fast-track/Building-a-Local-Test-Environment/for more details. -Andrew On Thu, Jun 13, 2013 at 6:20 AM, Rahul Bongirwar < bongirwar.rahul...@gmail.com> wrote: > Hi, > > I am getting error while creating riak-cs user as "Please reduce your > request rate" > What is the solution for it? > > Thanks, > Rahul > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > >>> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
How long should it take to set bucket properties?
I have objects in buckets that need to be updated based on their previous value. As per the documentation, I've implemented a sibling-resolution strategy that can deal with the writes happening in different orders, and implemented it on the client side. This of course requires me to set "allow_mult":true on each bucket. Setting that single property takes between 10 and 20 seconds per bucket. Is this expected? Here is a command I can run to reproduce the very long delay (where riak.lumi points to our Riak server): curl -X PUT http://riak.lumi:8098/riak/test-bucket -H "Content-Type: application/json" -d '{"props":{"allow_mult":true}}' -- Rob ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: question about riak siblings
Jeremy, What you are running into sounds a lot like the problems described in [1]. Jordan [1] https://github.com/ricardobcl/Dotted-Version-Vectors On Tue, Jun 18, 2013 at 12:58 PM, Jeremy Ong wrote: > > > > On Tue, Jun 18, 2013 at 12:57 PM, Jeremy Ong wrote: > >> Suppose I have two sources writing to the database, source A and source B. >> >> Next, suppose A and B both issue a write to the database so they spawn >> two siblings: [a1, b1]. >> >> Finally, suppose A writes another object using the metadata of the riak >> object returned when writing a2. This will lead to the following situation: >> > Sorry typo. This should read: > > " > Finally, suppose A writes another object using the metadata of the riak > object returned when writing a1" > > > >> [a1, b1, a2]. >> >> Upon the next read, the client will be presented with these three >> siblings. Is there a publicly exposed way to establish a strict ordering >> between a1 and a2? >> >> Thanks, >> Jeremy >> > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: question about riak siblings
Hi thanks, I was actually coming to the conclusion that to get this to work in my situation, I would need to embed a client side vector. On Thu, Jun 20, 2013 at 5:34 PM, Jordan West wrote: > Jeremy, > > What you are running into sounds a lot like the problems described in [1]. > > Jordan > > [1] https://github.com/ricardobcl/Dotted-Version-Vectors > > > On Tue, Jun 18, 2013 at 12:58 PM, Jeremy Ong wrote: > >> >> >> >> On Tue, Jun 18, 2013 at 12:57 PM, Jeremy Ong wrote: >> >>> Suppose I have two sources writing to the database, source A and source >>> B. >>> >>> Next, suppose A and B both issue a write to the database so they spawn >>> two siblings: [a1, b1]. >>> >>> Finally, suppose A writes another object using the metadata of the riak >>> object returned when writing a2. This will lead to the following situation: >>> >> Sorry typo. This should read: >> >> " >> Finally, suppose A writes another object using the metadata of the riak >> object returned when writing a1" >> >> >> >>> [a1, b1, a2]. >>> >>> Upon the next read, the client will be presented with these three >>> siblings. Is there a publicly exposed way to establish a strict ordering >>> between a1 and a2? >>> >>> Thanks, >>> Jeremy >>> >> >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Riak does not work on ec2
Hi, I am trying to get riak to work on ubuntu 12.04 and is proving not to be a friendly install on ec2. 1) I am using riak_1.3.1-1_amd64.deb and have libssl0.9.8 installed 2) I stop riak then run riak-admin cluster replace riak@127.0.0.1 r...@xxx.xxx.xxx.xxx riak-admin cluster plan riak-admin cluster commit Attempting to restart script through sudo -H -u riak Node is not running! At the very bottom are the config files. I change all of 127.0.0.1 to the public ip addres of the machine on ec2. Kinda hard to mess up there I am using chef. node[:ec2][:public_ipv4] 3) The logs are not being very helpful. What does the below mean in english? root@domU-12-31-39-0C-59-1D:/home/ubuntu# riak console Attempting to restart script through sudo -H -u riak Exec: /usr/lib/riak/erts-5.9.1/bin/erlexec -boot /usr/lib/riak/releases/1.3.1/riak -embedded -config /etc/riak/app.config -pa /usr/lib/riak/lib/basho-patches -args_file /etc/riak/vm.args -- console Root: /usr/lib/riak Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:1:1] [async-threads:64] [kernel-poll:true] /usr/lib/riak/lib/os_mon-2.2.9/priv/bin/memsup: Erlang has closed. Erlang has closed {"Kernel pid terminated",application_controller,"{application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}"} Crash dump was written to: /var/log/riak/erl_crash.dump Kernel pid terminated (application_controller) ({application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}) So..what could the issue be? Is there a missing manual? What did I miss from the documentation on http://docs.basho.com/riak/latest/cookbooks/Basic-Cluster-Setup/ %% Riak Client APIs config {riak_api, [ %% pb_backlog is the maximum length to which the queue of pending %% connections may grow. If set, it must be an integer >= 0. %% By default the value is 5. If you anticipate a huge number of %% connections being initialised *simultaneously*, set this number %% higher. %% {pb_backlog, 64}, %% pb_ip is the IP address that the Riak Protocol Buffers interface %% will bind to. If this is undefined, the interface will not run. {pb_ip, "xxx.xxx.xxx.xxx" }, %% pb_port is the TCP port that the Riak Protocol Buffers interface %% will bind to {pb_port, 8087 } ]}, %% Riak Core config {riak_core, [ %% Default location of ringstate {ring_state_dir, "/var/lib/riak/ring"}, %% Default ring creation size. Make sure it is a power of 2, %% e.g. 16, 32, 64, 128, 256, 512 etc %{ring_creation_size, 64}, %% http is a list of IP addresses and TCP ports that the Riak %% HTTP interface will bind. {http, [ {"xxx.xxx.xxx.xxx", 8098 } ]}, %% https is a list of IP addresses and TCP ports that the Riak %% HTTPS interface will bind. %{https, [{ "xxx.xxx.xxx.xxx", 8098 }]}, ## Name of the riak node -name r...@xxx.xxx.xxx.xxx ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak does not work on ec2
Hi David, Sorry to hear you're having difficulties. From what I can tell, you're simply looking to spin up a new cluster, correct? If so, riak-admin cluster replace is not the command you want. If you're attempting to rename your riak nodes and bind them to an AWS EIP, you can follow these instructions [0] from the docs. Cheers, Tom [0] http://docs.basho.com/riak/latest/cookbooks/Performance-Tuning-AWS/#Dealing-with-IP-addresses On Fri, Jun 21, 2013 at 4:55 AM, David Montgomery wrote: > Hi, > > I am trying to get riak to work on ubuntu 12.04 and is proving not to be a > friendly install on ec2. > > > 1) I am using riak_1.3.1-1_amd64.deb and have libssl0.9.8 installed > > 2) I stop riak then run > > riak-admin cluster replace riak@127.0.0.1 r...@xxx.xxx.xxx.xxx > riak-admin cluster plan > riak-admin cluster commit > Attempting to restart script through sudo -H -u riak > Node is not running! > > At the very bottom are the config files. I change all of 127.0.0.1 to the > public ip addres of the machine on ec2. Kinda hard to mess up there I am > using chef. node[:ec2][:public_ipv4] > > 3) The logs are not being very helpful. What does the below mean in > english? > > root@domU-12-31-39-0C-59-1D:/home/ubuntu# riak console > Attempting to restart script through sudo -H -u riak > Exec: /usr/lib/riak/erts-5.9.1/bin/erlexec -boot > /usr/lib/riak/releases/1.3.1/riak -embedded -config > /etc/riak/app.config -pa > /usr/lib/riak/lib/basho-patches -args_file /etc/riak/vm.args -- > console > Root: /usr/lib/riak > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:1:1] [async-threads:64] > [kernel-poll:true] > > /usr/lib/riak/lib/os_mon-2.2.9/priv/bin/memsup: Erlang has closed. >Erlang > has closed > > {"Kernel pid > terminated",application_controller,"{application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}"} > > Crash dump was written to: /var/log/riak/erl_crash.dump > Kernel pid terminated (application_controller) > ({application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}) > > > > > So..what could the issue be? Is there a missing manual? What did I miss > from the documentation on > http://docs.basho.com/riak/latest/cookbooks/Basic-Cluster-Setup/ > > > > %% Riak Client APIs config > {riak_api, [ > %% pb_backlog is the maximum length to which the queue of > pending > %% connections may grow. If set, it must be an integer >= 0. > %% By default the value is 5. If you anticipate a huge number > of > %% connections being initialised *simultaneously*, set this > number > %% higher. > %% {pb_backlog, 64}, > > %% pb_ip is the IP address that the Riak Protocol Buffers > interface > %% will bind to. If this is undefined, the interface will not > run. > {pb_ip, "xxx.xxx.xxx.xxx" }, > > %% pb_port is the TCP port that the Riak Protocol Buffers > interface > %% will bind to > {pb_port, 8087 } > ]}, > > %% Riak Core config > {riak_core, [ > %% Default location of ringstate > {ring_state_dir, "/var/lib/riak/ring"}, > > %% Default ring creation size. Make sure it is a power of 2, > %% e.g. 16, 32, 64, 128, 256, 512 etc > %{ring_creation_size, 64}, > > %% http is a list of IP addresses and TCP ports that the Riak > %% HTTP interface will bind. > {http, [ {"xxx.xxx.xxx.xxx", 8098 } ]}, > > %% https is a list of IP addresses and TCP ports that the > Riak > %% HTTPS interface will bind. > %{https, [{ "xxx.xxx.xxx.xxx", 8098 }]}, > > > > > > > > ## Name of the riak node > -name r...@xxx.xxx.xxx.xxx > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak does not work on ec2
David, Riak does indeed work on EC2, we have many customers who deploy exclusively to EC2. We also happened to do some large cluster testing on ec2 this week so I know nothing has suddenly gone awry. Your problem seems to be you using your external IP rather than your internal IP in your configuration. Try changing to internal IP. For AWS, here are a couple of pages on our docs that will probably be really helpful. http://docs.basho.com/riak/latest/tutorials/installation/Installing-on-AWS-Marketplace/ http://docs.basho.com/riak/latest/cookbooks/Performance-Tuning-AWS/ Good luck, Jared On Thu, Jun 20, 2013 at 9:55 PM, David Montgomery wrote: > Hi, > > I am trying to get riak to work on ubuntu 12.04 and is proving not to be a > friendly install on ec2. > > > 1) I am using riak_1.3.1-1_amd64.deb and have libssl0.9.8 installed > > 2) I stop riak then run > > riak-admin cluster replace riak@127.0.0.1 r...@xxx.xxx.xxx.xxx > riak-admin cluster plan > riak-admin cluster commit > Attempting to restart script through sudo -H -u riak > Node is not running! > > At the very bottom are the config files. I change all of 127.0.0.1 to the > public ip addres of the machine on ec2. Kinda hard to mess up there I am > using chef. node[:ec2][:public_ipv4] > > 3) The logs are not being very helpful. What does the below mean in > english? > > root@domU-12-31-39-0C-59-1D:/home/ubuntu# riak console > Attempting to restart script through sudo -H -u riak > Exec: /usr/lib/riak/erts-5.9.1/bin/erlexec -boot > /usr/lib/riak/releases/1.3.1/riak -embedded -config > /etc/riak/app.config -pa > /usr/lib/riak/lib/basho-patches -args_file /etc/riak/vm.args -- > console > Root: /usr/lib/riak > Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:1:1] [async-threads:64] > [kernel-poll:true] > > /usr/lib/riak/lib/os_mon-2.2.9/priv/bin/memsup: Erlang has closed. >Erlang > has closed > > {"Kernel pid > terminated",application_controller,"{application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}"} > > Crash dump was written to: /var/log/riak/erl_crash.dump > Kernel pid terminated (application_controller) > ({application_start_failure,riak_core,{shutdown,{riak_core_app,start,[normal,[]]}}}) > > > > > So..what could the issue be? Is there a missing manual? What did I miss > from the documentation on > http://docs.basho.com/riak/latest/cookbooks/Basic-Cluster-Setup/ > > > > %% Riak Client APIs config > {riak_api, [ > %% pb_backlog is the maximum length to which the queue of > pending > %% connections may grow. If set, it must be an integer >= 0. > %% By default the value is 5. If you anticipate a huge number > of > %% connections being initialised *simultaneously*, set this > number > %% higher. > %% {pb_backlog, 64}, > > %% pb_ip is the IP address that the Riak Protocol Buffers > interface > %% will bind to. If this is undefined, the interface will not > run. > {pb_ip, "xxx.xxx.xxx.xxx" }, > > %% pb_port is the TCP port that the Riak Protocol Buffers > interface > %% will bind to > {pb_port, 8087 } > ]}, > > %% Riak Core config > {riak_core, [ > %% Default location of ringstate > {ring_state_dir, "/var/lib/riak/ring"}, > > %% Default ring creation size. Make sure it is a power of 2, > %% e.g. 16, 32, 64, 128, 256, 512 etc > %{ring_creation_size, 64}, > > %% http is a list of IP addresses and TCP ports that the Riak > %% HTTP interface will bind. > {http, [ {"xxx.xxx.xxx.xxx", 8098 } ]}, > > %% https is a list of IP addresses and TCP ports that the > Riak > %% HTTPS interface will bind. > %{https, [{ "xxx.xxx.xxx.xxx", 8098 }]}, > > > > > > > > ## Name of the riak node > -name r...@xxx.xxx.xxx.xxx > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Mismatched object counts
I've just inserted some data into a six node Riak 1.3.1 EE cluster. The keys are all SHA256s. The bucket previously had somewhere in the vicinity of 1 million objects. A MR job using the $key 2i with a range of '0' to 'Z', which should cover all possible SHA256s, and using both riak_kv_mapreduce:: reduce_count_inputs and streaming the keys using reduce_identity and counting client side, both returned a count around 750K, but that is now somewhat suspect. The objects I inserted overlap somewhat with the previously existing objects, but not completely. Overlapping objects were merged. I inserted 2,521,799 objects. When I execute the MR count job against it reports 1,604,783 objects, using both techniques (reduce_count_inputs and reduce_identity plus client side counting). Given the discrepancy I queried the bucket for the 2,521,799 objects I thought I inserted and I verified the system thinks they are there. What gives? Why is MR returning incorrect result? Does the 2i query somehow miss some possible keys? This is what the job looks like in Ruby: Riak::MapReduce.new(client). index(bucket_name, "$key", '0'..'Z'). reduce(['riak_kv_mapreduce', 'reduce_count_inputs'], :keep => true, :arg => { "reduce_phase_batch_size" => 1000, "do_prereduce" => true } ). timeout(8640). run As a side question, does do_prereduce here have any effect? I am thinking it does not. The docs indicate do_prereduce is a map phase argument, not a reduce phase one. That begs the question of how to enable prereduce for a MR job without a map phase, other than setting mapred_always_prereduce = true in the config file. Elias ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com