Re: Scaling Riak CS to hundreds or thousands of servers

2013-07-27 Thread Andrew Stone
Hi Andre, The blocks are going to be spread across some subset of that 100 servers. Since Riak CS stores the chunks inside Riak they are hashed based on primary key. Currently there is no way to co-locate chunks in Riak CS. You can read more about How riak manages storage here: http://docs.basho.c

Mapreduce limits / scalability

2013-07-27 Thread Y N
Hi, I have recently started using Riak and had a question about mapreduce queries is there a limit on the number of queries that could be run concurrently (per node / cluster etc)? Also, does Riak optimize concurrent requests for the same mapreduce query / queries by returning results from

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Guido Medina
I get you now, I was confused by the fact that he asked explicitly about 2i, so when you mentioned indexes I thought you were referring to 2i, I would categorize Riak Search and Yokozuna as an alternative due to extra effort, schema definition and post-commits hooks, so for the straight questio

Re: Scaling Riak CS to hundreds or thousands of servers

2013-07-27 Thread Andre Lohmann
Hi Again and thanks for the answers. Now I have another questions. If chunk size is always 1mb and i have a file of 100mb within a cluster of 100 servers. Are the chunks then split over all 100 servers or am i able to manage the chunks of one file to be saved at one place, to prevent to much ne

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Jeremiah Peschka
No, I never said or hinted that you could join 2i indices using Riak Search. You can, however, query against multiple indexed fields with Riak Search. Here's what that would look like in CorrugatedIron: var req = new RiakSearchRequest { Query = new RiakFluentSearch("r

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Antonio Rohman Fernandez
Can you provide a real example on how to do that with Riak Search? Thanks, Rohman On 27.07.2013 17:44, Guido Medina wrote: > Are you saying that you can join two 2i indexes? Let's say you have a 2i named "date" and another named "category" so you could get a list of modified objects on a d

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Guido Medina
Are you saying that you can join two 2i indexes? Let's say you have a 2i named "date" and another named "category" so you could get a list of modified objects on a date and for a specific category with Riak Search? Guido. On 27/07/13 16:32, Jeremiah Peschka wrote: Both Riak Search (currently)

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Jeremiah Peschka
Both Riak Search (currently) and Yokozuna (future) provide the ability to perform joins of multiple indices on the same bucket objects. I find it best to think of 2i as an extension of what Riak KV already does well - fast key based lookups. Search and Yokozuna provide complex query semantics. -

Re: 2i timeouts in 1.4

2013-07-27 Thread Russell Brown
Hi Sean, This one really is a bug with 2i pagination on Equals term queries[1]. I've fixed it already[2], I'm very sorry that this one got out into the wild. It is something that worked then stopped working at some point in a merge. It was a pretty bad oversight that we didn't have a test for t

Re: Querying multiple 2i via Intersections

2013-07-27 Thread Guido Medina
Rohman, I think the reason for this is that the cluster will have to do the whole intersection in memory, 2i only provides queries for 1 single index and then return that result to either the client streaming or not, intersection indeed will require a MapReduce job to get a hold of both lists

Querying multiple 2i via Intersections

2013-07-27 Thread Antonio Rohman Fernandez
Hi there, I didn't fully check the new release notes yet, but I guess that Riak 1.4 doesn't have the ability to query multiple 2i at the same time... how difficult may be to implement a multiple 2i query via their intersecting items in their arrays? Maybe we could have this in future versions?