Hello My PC is a stable ubuntu linux that contains
Erlang R13B03 (erts-5.7.4) regards >----Messaggio originale---- >Da: [email protected] >Data: 08/09/2010 18.00 >A: <[email protected]> >Ogg: riak-users Digest, Vol 14, Issue 11 > >Send riak-users mailing list submissions to > [email protected] > >To subscribe or unsubscribe via the World Wide Web, visit > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >or, via email, send a message with subject or body 'help' to > [email protected] > >You can reach the person managing the list at > [email protected] > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of riak-users digest..." > > >Today's Topics: > > 1. Is there difference between "MapReduce" and "MapReduce in > RIAK"? (Ken Matsumoto) > 2. sane way to edit json map reduce functions? (Bryan Berry) > 3. Re: Is there difference between "MapReduce" and "MapReduce in > RIAK"? (Kevin Smith) > 4. Re: sane way to edit json map reduce functions? (Sean Cribbs) > 5. riak-erlang-client: riakc_pb_socket.erl:72: type ctx() > undefined ([email protected]) > 6. Re: riak-erlang-client: riakc_pb_socket.erl:72: type ctx() > undefined (Jon Meredith) > 7. Re: Is there difference between "MapReduce" and "MapReduce in > RIAK"? (Les Mikesell) > > >---------------------------------------------------------------------- > >Message: 1 >Date: Tue, 07 Sep 2010 19:52:08 -0700 >From: Ken Matsumoto <[email protected]> >To: [email protected] >Subject: Is there difference between "MapReduce" and "MapReduce in > RIAK"? >Message-ID: <[email protected]> >Content-Type: text/plain; charset=UTF-8; format=flowed > >Hi all, > >Is there difference between "MapReduce in Hadoop" and "MapReduce in RIAK"? >I tested these performances and there were difference even in the same >machine, same data. >I'd like to know the difference like: > >- Version? Which is newer or branched? >- Does RIAK build in on source level or binary? >- Number of performance tuning parameters? >- What else? > >Regards, > >Ken. > >-- >Ken Matsumoto >VP / Research & Development >Nomura Research Institute America, Inc. >NRI Pacific >1400 Fashion Island Blvd., Suite 1010 >San Mateo, CA 94404, U.S.A. > >PLEASE READ?This e-mail is confidential and intended for the named >recipient only. If you are not an intended recipient, please notify the >sender and delete this e-mail. > > > > >------------------------------ > >Message: 2 >Date: Wed, 8 Sep 2010 10:44:06 +0200 >From: Bryan Berry <[email protected]> >To: [email protected] >Subject: sane way to edit json map reduce functions? >Message-ID: > <[email protected]> >Content-Type: text/plain; charset="iso-8859-1" > >hey riak buddies, I find that I have to write my mapreduce functions in one >single line w/out tabs or line breaks. >Is there an easier way to do it? This makes my code pretty unreadable. > >I am using emacs btw, w/ js2-mode >-------------- next part -------------- >An HTML attachment was scrubbed... >URL: <http://lists.basho.com/pipermail/riak-users_lists.basho. com/attachments/20100908/0f90f7dd/attachment-0001.html> > >------------------------------ > >Message: 3 >Date: Wed, 8 Sep 2010 07:58:43 -0400 >From: Kevin Smith <[email protected]> >To: Ken Matsumoto <[email protected]> >Cc: [email protected] >Subject: Re: Is there difference between "MapReduce" and "MapReduce in > RIAK"? >Message-ID: <[email protected]> >Content-Type: text/plain; charset=utf-8 > >Ken - > >Riak contains its own implementation of a MapReduce API. This API is written in Erlang and C and shares no code with Hadoop. We have, on occasion, described our API as "Hadoop like" to help people understand the differences between our implementation and others. For example: > >* CouchDB's MapReduce incrementally builds a BTree thru the execution of views. Riak doesn't cache or store MapReduce results. Each job executes "from scratch" and returns results to caller. It is up to the caller to cache or store these results if needed. > >* Hadoop MapReduce jobs use HDFS or HDFS-adapted resources. Riak MapReduce jobs use data stored in a Riak cluster. > >* Riak MapReduce jobs take advantage of their execution environment. Map functions exploit data locality. This means map functions are sent to the node hosting the required data rather than streaming the data to a central coordinating node. The most recent version of Riak also includes improvements which has significantly boosted the efficiency of mapping over entire buckets of data. > >* Riak supports writing MapReduce functions in two languages: Erlang and Javascript. Erlang is the absolute fastest, in terms of raw speed, but Javascript runs a close second and is easier for most people to use. > >Depending on what your jobs are doing and which language, Erlang or Javascript, there are a few tunable parameters we can tweak to improve performance. More information about what your job is doing, how your cluster is set up, and the kinds of performance you're seeing would help us debug the situation. > >--Kevin >On Sep 7, 2010, at 10:52 PM, Ken Matsumoto wrote: > >> Hi all, >> >> Is there difference between "MapReduce in Hadoop" and "MapReduce in RIAK"? >> I tested these performances and there were difference even in the same machine, same data. >> I'd like to know the difference like: >> >> - Version? Which is newer or branched? >> - Does RIAK build in on source level or binary? >> - Number of performance tuning parameters? >> - What else? >> >> Regards, >> >> Ken. >> >> -- >> Ken Matsumoto >> VP / Research & Development >> Nomura Research Institute America, Inc. >> NRI Pacific >> 1400 Fashion Island Blvd., Suite 1010 >> San Mateo, CA 94404, U.S.A. >> >> PLEASE READ?This e-mail is confidential and intended for the named recipient only. If you are not an intended recipient, please notify the sender and delete this e-mail. >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > >------------------------------ > >Message: 4 >Date: Wed, 8 Sep 2010 08:23:19 -0400 >From: Sean Cribbs <[email protected]> >To: Bryan Berry <[email protected]> >Cc: [email protected] >Subject: Re: sane way to edit json map reduce functions? >Message-ID: <[email protected]> >Content-Type: text/plain; charset=us-ascii > >Bryan, > >That should only be an issue if you are putting the source of the function directly in your query. Instead, try saving the function in a bucket/key and refer to it that way. Here's what a phase using bucket/key instead of source would look like: > >{"map":{"language":"javascript","bucket":"customMR","key":" extractSomeFields","keep":false}} > >You can use curl or any one of the the language-specific clients to store the function, just specify "application/javascript" as the content-type. > >I use this technique in the "yakriak" demo app: http://github. com/seancribbs/yakriak > >Sean Cribbs <[email protected]> >Developer Advocate >Basho Technologies, Inc. >http://basho.com/ > >On Sep 8, 2010, at 4:44 AM, Bryan Berry wrote: > >> hey riak buddies, I find that I have to write my mapreduce functions in one single line w/out tabs or line breaks. >> Is there an easier way to do it? This makes my code pretty unreadable. >> >> I am using emacs btw, w/ js2-mode >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > >------------------------------ > >Message: 5 >Date: Wed, 8 Sep 2010 15:53:22 +0200 (CEST) >From: "[email protected]" <[email protected]> >To: [email protected] >Subject: riak-erlang-client: riakc_pb_socket.erl:72: type ctx() > undefined >Message-ID: > <29648517.662641283954002450.javamail.defaultu...@defaulthost> >Content-Type: text/plain;charset="UTF-8" > >HEllo to all, > >I'm using riak for my erlang web crawler ebot (http://www.redaelli. org/matteo- >blog/projects/ebot/) >In these latest weeks I cannot compile riak-erlang-client > >Below my logs. Any suggestions? > >Thanks in advance > >Matteo >http://www.redaelli.org/matteo > > > >mat...@nowar:/opt$ hg clone http://hg.basho.com/riak-erlang-client -u >riakc-1.0.0 >destination directory: riak-erlang-client >requesting all changes >adding changesets >adding manifests >adding file changes >added 98 changesets with 124 changes to 15 files >updating to branch default >15 files updated, 0 files merged, 0 files removed, 0 files unresolved > >mat...@nowar:/opt$ cd riak-erlang-client/ > >mat...@nowar:/opt/riak-erlang-client$ make >./rebar get-deps >==> riak-erlang-client (get-deps) >Pulling protobuffs from {hg,"http://bitbucket.org/basho/protobuffs";, > "protobuffs-0.5.0"} >requesting all changes >adding changesets >adding manifests >adding file changes >added 91 changesets with 243 changes to 93 files (+2 heads) >29 files updated, 0 files merged, 0 files removed, 0 files unresolved >==> protobuffs (get-deps) >./rebar compile >==> protobuffs (compile) >Compiled src/pokemon_pb.erl >Compiled src/protobuffs_parser.erl >Compiled src/protobuffs_compile.erl >Compiled src/protobuffs.erl >==> riak-erlang-client (compile) >Compiling src/riakclient.proto >src/riakc_pb_socket.erl:72: type ctx() undefined >src/riakc_pb_socket.erl:72: type rpb_req() undefined >src/riakc_pb_socket.erl:77: type option() undefined >src/riakc_pb_socket.erl:78: Warning: type option() is unused >src/riakc_pb_socket.erl:86: Warning: type rpb_req() is unused >src/riakc_pb_socket.erl:87: Warning: type ctx() is unused >make: *** [compile] Error 1 > > > >------------------------------ > >Message: 6 >Date: Wed, 08 Sep 2010 08:11:02 -0600 >From: Jon Meredith <[email protected]> >To: [email protected] >Subject: Re: riak-erlang-client: riakc_pb_socket.erl:72: type ctx() > undefined >Message-ID: <[email protected]> >Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Hi Matteo, > >The tip (fd76e53acac2) compiles for me under R13B04 - what version of >erlang are you using so I can investigate? > >Jon > >On 9/8/10 7:53 AM, [email protected] wrote: >> HEllo to all, >> >> I'm using riak for my erlang web crawler ebot (http://www.redaelli. org/matteo- >> blog/projects/ebot/) >> In these latest weeks I cannot compile riak-erlang-client >> >> Below my logs. Any suggestions? >> >> Thanks in advance >> >> Matteo >> http://www.redaelli.org/matteo >> >> >> >> mat...@nowar:/opt$ hg clone http://hg.basho.com/riak-erlang-client -u >> riakc-1.0.0 >> destination directory: riak-erlang-client >> requesting all changes >> adding changesets >> adding manifests >> adding file changes >> added 98 changesets with 124 changes to 15 files >> updating to branch default >> 15 files updated, 0 files merged, 0 files removed, 0 files unresolved >> >> mat...@nowar:/opt$ cd riak-erlang-client/ >> >> mat...@nowar:/opt/riak-erlang-client$ make >> ./rebar get-deps >> ==> riak-erlang-client (get-deps) >> Pulling protobuffs from {hg,"http://bitbucket.org/basho/protobuffs";, >> "protobuffs-0.5.0"} >> requesting all changes >> adding changesets >> adding manifests >> adding file changes >> added 91 changesets with 243 changes to 93 files (+2 heads) >> 29 files updated, 0 files merged, 0 files removed, 0 files unresolved >> ==> protobuffs (get-deps) >> ./rebar compile >> ==> protobuffs (compile) >> Compiled src/pokemon_pb.erl >> Compiled src/protobuffs_parser.erl >> Compiled src/protobuffs_compile.erl >> Compiled src/protobuffs.erl >> ==> riak-erlang-client (compile) >> Compiling src/riakclient.proto >> src/riakc_pb_socket.erl:72: type ctx() undefined >> src/riakc_pb_socket.erl:72: type rpb_req() undefined >> src/riakc_pb_socket.erl:77: type option() undefined >> src/riakc_pb_socket.erl:78: Warning: type option() is unused >> src/riakc_pb_socket.erl:86: Warning: type rpb_req() is unused >> src/riakc_pb_socket.erl:87: Warning: type ctx() is unused >> make: *** [compile] Error 1 >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > >------------------------------ > >Message: 7 >Date: Wed, 08 Sep 2010 10:07:47 -0500 >From: Les Mikesell <[email protected]> >To: [email protected] >Subject: Re: Is there difference between "MapReduce" and "MapReduce in > RIAK"? >Message-ID: <[email protected]> >Content-Type: text/plain; charset=UTF-8; format=flowed > >On 9/8/2010 6:58 AM, Kevin Smith wrote: >> Ken - >> >> Riak contains its own implementation of a MapReduce API. This API is written in Erlang and C and shares no code with Hadoop. We have, on occasion, described our API as "Hadoop like" to help people understand the differences between our implementation and others. For example: >> >> * CouchDB's MapReduce incrementally builds a BTree thru the execution of views. Riak doesn't cache or store MapReduce results. Each job executes "from scratch" and returns results to caller. It is up to the caller to cache or store these results if needed. >> >> * Hadoop MapReduce jobs use HDFS or HDFS-adapted resources. Riak MapReduce jobs use data stored in a Riak cluster. >> >> * Riak MapReduce jobs take advantage of their execution environment. Map functions exploit data locality. This means map functions are sent to the node hosting the required data rather than streaming the data to a central coordinating node. The most recent version of Riak also includes improvements which has significantly boosted the efficiency of mapping over entire buckets of data. >> >> * Riak supports writing MapReduce functions in two languages: Erlang and Javascript. Erlang is the absolute fastest, in terms of raw speed, but Javascript runs a close second and is easier for most people to use. >> >> Depending on what your jobs are doing and which language, Erlang or Javascript, there are a few tunable parameters we can tweak to improve performance. More information about what your job is doing, how your cluster is set up, and the kinds of performance you're seeing would help us debug the situation. > >Are there any 'high level' tools to integrate riak data into other >systems like the hive jdbc driver for hadoop and pentaho's project to do >reporting on top of that? > >-- > Les Mikesell > [email protected] > > > > > >------------------------------ > >_______________________________________________ >riak-users mailing list >[email protected] >http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > >End of riak-users Digest, Vol 14, Issue 11 >****************************************** > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
