Re: map reduce on multiple buckets
One possibility would be to get the intermediate results from each bucket, and then compute the final results on the client. How much data would be involved in the initial MR, and at the point where you would have to combine the results? --Alex On Thu, Jan 28, 2016 at 12:10 PM, Eugene Shubin wrote: > I want to keep daily and weekly data in different buckets, and generate > monthly report using map reduce on these two. > so 2 buckets and MR operations are custom. > > Best, > Eugene > > 2016-01-28 17:29 GMT+01:00 Alex Moore : > >> Hi Eugene, >> >> MR is limited to one bucket for inputs, and the Solr inputs to a map >> phase have this restriction too. >> >> How many buckets are you trying to MR across, and also what type of MR >> operation are you trying to do? There may be another way to get around this >> restriction. >> >> Thanks, >> Alex >> >> On Thu, Jan 28, 2016 at 8:10 AM, Eugene Shubin >> wrote: >> >>> Is it possible to run mapreduce job on two or more buckets? >>> I see from documentation that it might be possible if I specify inputs >>> as list of {bucket, key} pairs, >>> although list of secondary index inputs causes an error: >>> riakc_pb_socket:mapred(P, [ >>> {index, Bucket1, Index1, From, To}, >>> {index, Bucket2, Index2, From, To} >>> ], ... >>> {error,<<"{inputs,{\"Inputs target tuples must be {B,K} or >>> {{B,K},KeyData}:\",\n >>> >>> Is it possible using Solr (riak search) indexes? >>> >>> Evgenii Shubin >>> >>> ___ >>> riak-users mailing list >>> riak-users@lists.basho.com >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >>> >>> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: map reduce on multiple buckets
I want to keep daily and weekly data in different buckets, and generate monthly report using map reduce on these two. so 2 buckets and MR operations are custom. Best, Eugene 2016-01-28 17:29 GMT+01:00 Alex Moore : > Hi Eugene, > > MR is limited to one bucket for inputs, and the Solr inputs to a map phase > have this restriction too. > > How many buckets are you trying to MR across, and also what type of MR > operation are you trying to do? There may be another way to get around this > restriction. > > Thanks, > Alex > > On Thu, Jan 28, 2016 at 8:10 AM, Eugene Shubin wrote: > >> Is it possible to run mapreduce job on two or more buckets? >> I see from documentation that it might be possible if I specify inputs as >> list of {bucket, key} pairs, >> although list of secondary index inputs causes an error: >> riakc_pb_socket:mapred(P, [ >> {index, Bucket1, Index1, From, To}, >> {index, Bucket2, Index2, From, To} >> ], ... >> {error,<<"{inputs,{\"Inputs target tuples must be {B,K} or >> {{B,K},KeyData}:\",\n >> >> Is it possible using Solr (riak search) indexes? >> >> Evgenii Shubin >> >> ___ >> riak-users mailing list >> riak-users@lists.basho.com >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: map reduce on multiple buckets
Hi Eugene, MR is limited to one bucket for inputs, and the Solr inputs to a map phase have this restriction too. How many buckets are you trying to MR across, and also what type of MR operation are you trying to do? There may be another way to get around this restriction. Thanks, Alex On Thu, Jan 28, 2016 at 8:10 AM, Eugene Shubin wrote: > Is it possible to run mapreduce job on two or more buckets? > I see from documentation that it might be possible if I specify inputs as > list of {bucket, key} pairs, > although list of secondary index inputs causes an error: > riakc_pb_socket:mapred(P, [ > {index, Bucket1, Index1, From, To}, > {index, Bucket2, Index2, From, To} > ], ... > {error,<<"{inputs,{\"Inputs target tuples must be {B,K} or > {{B,K},KeyData}:\",\n > > Is it possible using Solr (riak search) indexes? > > Evgenii Shubin > > ___ > riak-users mailing list > riak-users@lists.basho.com > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
map reduce on multiple buckets
Is it possible to run mapreduce job on two or more buckets? I see from documentation that it might be possible if I specify inputs as list of {bucket, key} pairs, although list of secondary index inputs causes an error: riakc_pb_socket:mapred(P, [ {index, Bucket1, Index1, From, To}, {index, Bucket2, Index2, From, To} ], ... {error,<<"{inputs,{\"Inputs target tuples must be {B,K} or {{B,K},KeyData}:\",\n Is it possible using Solr (riak search) indexes? Evgenii Shubin ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com