Does anybody have any insight on my questions below? Thanks, Jacques
On Tue, Jun 7, 2011 at 6:12 PM, Jacques <[email protected]> wrote: > I've been working on this. I have it working with an anonymous javascript > function. I was hoping to move it to using the "map_object_value" built-in > erlang function. However, when I attempt to use this function, I get > failures if any of my keys don't exist. Is there a way to construct my map > phase so that it gracefully handles not founds and just returns what it > finds? > > Also, using this function, is there a way to return the bucket and key > names? Or can one assume the response order is identical to the order of > inputs? > > I have my current constructions below. > > Thanks for any help. > > Jacques > > > > *Working Function Definition (including not founds)* > private static final OtpErlangAtom FUNCTION_TYPE = new > OtpErlangAtom("jsanon"); > private static final OtpErlangBinary FUNCTION_VALUE = new > OtpErlangBinary("function(v){ return [v]; }".getBytes()); > private static final OtpErlangTuple MAP_REDUCE_FUNCTION = new > OtpErlangTuple(new OtpErlangObject[] { FUNCTION_TYPE, FUNCTION_VALUE }); > > *Non-working Function Definition (works fine if all input list exists. > Fails on not founds.)* > private static final OtpErlangAtom FUNCTION_TYPE = new > OtpErlangAtom("modfun"); > private static final OtpErlangAtom FUNCTION_MODULE = new > OtpErlangAtom("riak_kv_mapreduce"); > private static final OtpErlangAtom FUNCTION_NAME = new > OtpErlangAtom("map_object_value"); > private static final OtpErlangTuple MAP_REDUCE_FUNCTION = new > OtpErlangTuple(new OtpErlangObject[] { FUNCTION_TYPE, FUNCTION_MODULE, > FUNCTION_NAME }); > > > *Final construction of Query object for submission (using either of > above).* > private static final OtpErlangTuple MAP_REDUCE_PHASE = new > OtpErlangTuple(new OtpErlangObject[]{ATOM_MAP, MAP_REDUCE_FUNCTION, > ATOM_NONE, KEEP_TRUE}); > private static final OtpErlangList MAP_REDUCE_PHASES = new > OtpErlangList(new OtpErlangTuple[]{MAP_REDUCE_PHASE}); > private static final OtpErlangTuple FULL_QUERY = new OtpErlangTuple(new > OtpErlangObject[] { ATOM_QUERY, MAP_REDUCE_PHASES }); > > > > On Tue, Jun 7, 2011 at 7:57 AM, Russell Brown <[email protected]>wrote: > >> >> On 7 Jun 2011, at 15:29, Jacques wrote: >> >> Have you had any success with reading the server response? >> >> >> Yes. Sorry I didn't post a reply. >> >> It was trivial (but fraught, see below), I just used the OtpInputStream to >> deserialize the byte array returned from pbc.MapReduceResponse. >> >> Like: >> >> ByteString bs = resp.getContent(); >> OtpInputStream is = new OtpInputStream(bs.toByteArray()); >> OtpErlangObject result = is.read_any(); >> // and then all sorts of looping, sniffing types, unpacking etc >> >> >> Caveats: >> >> You have a pain unpacking any reasonably complex result. >> The ETF specification drops this doozy about "strings": >> http://www.erlang.org/doc/apps/erts/erl_ext_dist.html#id85596<http://www.erlang.org/doc/apps/erts/erl_ext_dist.html>. >> My first test actually returned a list of [0,1,2,3...200] and Jinterface >> helpfully turned that into a string for me. >> >> That aside it is certainly feasible to use Jinterface to >> serialize/deserialize Map/Reduce jobs/results. >> >> Cheers >> >> Russell >> >> >> Thanks, >> Jacques >> >> On Sat, Jun 4, 2011 at 1:19 PM, Russell Brown <[email protected]>wrote: >> >>> >>> On 4 Jun 2011, at 18:22, Jacques wrote: >>> >>> I like the sound of option 3 also. I'll have a look at it this weekend >>>> and get back to you. >>>> >>> >>> Awesome! Thanks. If you can give me a point in the right direction >>> regarding the correct typing approach and what not, I'm up for giving it a >>> shot as well. >>> >>> >>> Ok, I have a half working hack. It isn't pretty 'cos the Jinterface API >>> is verbose. I've hacked the pbc.MapReduceBuilder to encode the job as >>> "application/x-erlang-binary" and submit that, but really this code should >>> be in a separate class, maybe using the output from >>> MapReduceBuilder.getJSON() as input. That way you can get the feature >>> without patching the client. >>> >>> What I haven't done is decode the response from Riak yet. If you want a >>> pointer here is a gist of the (unclean) hack. It could use a lot of work, >>> but it proves the concept: >>> >>> https://gist.github.com/1008293 >>> >>> The gist is just the diff so you can apply it as a patch >>> to src/main/java/com/basho/riak/pbc/mapreduce/MapReduceBuilder.java if you >>> want to play with it. >>> >>> You'll have to add Jinterface to your pom too. >>> >>> <!-- jinterface --> >>> <dependency> >>> <groupId>org.erlang.otp</groupId> >>> <artifactId>jinterface</artifactId> >>> <version>1.5.4</version> >>> </dependency> >>> >>> I think it is best to put the result decoding outside the library too. >>> I'm going to hack up a poc for that now, but I thought I'd post what I have >>> thus far. >>> >>> Cheers >>> >>> Russell >>> >>> >>> Thanks again, >>> Jacques >>> >>> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
