Re: Re[5]: Get all keys from an bucket
Hi Markus, You may want to subscribe to the mailing list to avoid that messages from you are held in the moderation queue. Only subscribed members can post directly to this list. Also, please always reply to the list address, not to individual subscribers to keep the discussion public. Russel just beat me to an answer. Please let us know if this solved your issue. Kind Regards, Magnus On 26 February 2016 at 17:17, Markus Geck <zerebo...@mail.ru> wrote: > > Anyone? > > Saturday, January 30, 2016 1:42 AM +03:00 from Markus Geck < > zerebo...@mail.ru>: > > Do you have an example how to stream them? The url I've posted in my > initial mail explains how to use that index, but not how to stream the > results. Unfortunately accessing the keys that way overloads the node. > > > Friday, January 29, 2016 2:06 PM UTC from Russell Brown < > russell.br...@me.com <https://e.mail.ru/compose?To=russell.br...@me.com>>: > > > With leveldb you can use the special $bucket index. You can also stream > the keys, and paginate them, meaning you can get them in smaller lumps, > hopefully this will appear faster and avoid the timeout you're seeing. > > > On 29 Jan 2016, at 14:03, Markus Geck <zerebo...@mail.ru > <https://e.mail.ru/compose/?mailto=mailto%3azerebo...@mail.ru>> wrote: > > Yes, sorry I forgot to mention that. > > > Monday, January 25, 2016 10:10 AM UTC from Russell Brown < > russell.br...@me.com > <https://e.mail.ru/compose/?mailto=mailto%3arussell.br...@me.com>>: > > Hi Markus, > Are you using leveldb backend? > > Russell > > On 22 Jan 2016, at 19:05, Markus Geck <zerebo...@mail.ru> wrote: > > > Hello, > > > > is there any way to get all keys from an bucket? > > > > I've already tried this guide: > http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But > riak always wents unresponsive with a huge server load. > > > > and "GET /buckets/bucket/keys?keys=stream" returns an timeout error. > > > > Is there any other way? > > ___ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > -- Magnus Kessler Client Services Engineer Basho Technologies Limited Registered Office - 8 Lincoln’s Inn Fields London WC2A 3BP Reg 07970431 ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Get all keys from an bucket
Use “max_size” to set a limit on how many results you receive, as documented here: http://docs.basho.com/riak/latest/dev/using/2i/#Querying pass the parameter “stream=true” to stream. Do you want the object bodies back too? There is a super secret interface in PB to achieve this, have a look at https://github.com/basho/riak-erlang-client/blob/master/src/riakc_pb_socket.erl#L. It at least illustrates the use of the query parameters for a $bucket index that uses streaming, pagination, etc Russell On 26 Feb 2016, at 17:17, Markus Geck <zerebo...@mail.ru> wrote: > > Anyone? > > Saturday, January 30, 2016 1:42 AM +03:00 from Markus Geck > <zerebo...@mail.ru>: > > Do you have an example how to stream them? The url I've posted in my initial > mail explains how to use that index, but not how to stream the results. > Unfortunately accessing the keys that way overloads the node. > > > Friday, January 29, 2016 2:06 PM UTC from Russell Brown > <russell.br...@me.com>: > > With leveldb you can use the special $bucket index. You can also stream the > keys, and paginate them, meaning you can get them in smaller lumps, hopefully > this will appear faster and avoid the timeout you're seeing. > > > On 29 Jan 2016, at 14:03, Markus Geck <zerebo...@mail.ru> wrote: > >> Yes, sorry I forgot to mention that. >> >> >> Monday, January 25, 2016 10:10 AM UTC from Russell Brown >> <russell.br...@me.com>: >> >> Hi Markus, >> Are you using leveldb backend? >> >> Russell >> >> On 22 Jan 2016, at 19:05, Markus Geck <zerebo...@mail.ru> wrote: >> >> > Hello, >> > >> > is there any way to get all keys from an bucket? >> > >> > I've already tried this guide: >> > http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But >> > riak always wents unresponsive with a huge server load. >> > >> > and "GET /buckets/bucket/keys?keys=stream" returns an timeout error. >> > >> > Is there any other way? >> > ___ >> > riak-users mailing list >> > riak-users@lists.basho.com >> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re[5]: Get all keys from an bucket
Anyone? >Saturday, January 30, 2016 1:42 AM +03:00 from Markus Geck <zerebo...@mail.ru>: > >Do you have an example how to stream them? The url I've posted in my initial >mail explains how to use that index, but not how to stream the results. >Unfortunately accessing the keys that way overloads the node. > > >>Friday, January 29, 2016 2:06 PM UTC from Russell Brown < >>russell.br...@me.com >: >> >>With leveldb you can use the special $bucket index. You can also stream the >>keys, and paginate them, meaning you can get them in smaller lumps, hopefully >>this will appear faster and avoid the timeout you're seeing. >> >> >>On 29 Jan 2016, at 14:03, Markus Geck < zerebo...@mail.ru > wrote: >> >>>Yes, sorry I forgot to mention that. >>> >>> >>>>Monday, January 25, 2016 10:10 AM UTC from Russell Brown < >>>>russell.br...@me.com >: >>>> >>>>Hi Markus, >>>>Are you using leveldb backend? >>>> >>>>Russell >>>> >>>>On 22 Jan 2016, at 19:05, Markus Geck < zerebo...@mail.ru > wrote: >>>> >>>>> Hello, >>>>> >>>>> is there any way to get all keys from an bucket? >>>>> >>>>> I've already tried this guide: >>>>> http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But >>>>> riak always wents unresponsive with a huge server load. >>>>> >>>>> and "GET /buckets/bucket/keys?keys=stream" returns an timeout error. >>>>> >>>>> Is there any other way? >>>>> ___ >>>>> riak-users mailing list >>>>> riak-users@lists.basho.com >>>>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>>> >>> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Re[2]: Get all keys from an bucket
With leveldb you can use the special $bucket index. You can also stream the keys, and paginate them, meaning you can get them in smaller lumps, hopefully this will appear faster and avoid the timeout you're seeing. On 29 Jan 2016, at 14:03, Markus Geck <zerebo...@mail.ru> wrote: > Yes, sorry I forgot to mention that. > > > Monday, January 25, 2016 10:10 AM UTC from Russell Brown > <russell.br...@me.com>: > > Hi Markus, > Are you using leveldb backend? > > Russell > > On 22 Jan 2016, at 19:05, Markus Geck <zerebo...@mail.ru> wrote: > > > Hello, > > > > is there any way to get all keys from an bucket? > > > > I've already tried this guide: > > http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But > > riak always wents unresponsive with a huge server load. > > > > and "GET /buckets/bucket/keys?keys=stream" returns an timeout error. > > > > Is there any other way? > > ___ > > riak-users mailing list > > riak-users@lists.basho.com > > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Get all keys from an bucket
Hello, is there any way to get all keys from an bucket? I've already tried this guide: http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But riak always wents unresponsive with a huge server load. and "GET /buckets/bucket/keys?keys=stream" returns an timeout error. Is there any other way?___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Get all keys from an bucket
Hi Markus, Are you using leveldb backend? Russell On 22 Jan 2016, at 19:05, Markus Geck <zerebo...@mail.ru> wrote: > Hello, > > is there any way to get all keys from an bucket? > > I've already tried this guide: > http://www.paperplanes.de/2011/12/13/list-all-of-the-riak-keys.html But riak > always wents unresponsive with a huge server load. > > and "GET /buckets/bucket/keys?keys=stream" returns an timeout error. > > Is there any other way? > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
