Slightly offtopic: Anyone saw Simon Peyton Jones talking about making a generic mapreduce-like in haskell? http://tinyurl.com/537apv
On Sat, Nov 22, 2008 at 3:53 PM, Chris Anderson <[EMAIL PROTECTED]> wrote: > On Sat, Nov 22, 2008 at 12:25 PM, maddiin <[EMAIL PROTECTED]> wrote: > > > > Do you have any advice what I am doing wrong and how I could speed this > up? > > > I'm curious how long it takes with reduce=false (should be limited > basically by IO). > > I'm almost certain (please correct me if I'm wrong) that reduce > requests must call the JavaScript interpreter at least once per > request, to rereduce the btree inner-nodes that fit in that request > range. This means for group=true requests, the rereduce function must > run once per unique key (at minimum). That would be the source of your > slowness. It sounds like you are building a tag-cloud. The smart money > would be on caching the results of that operation, which is standard > practice with SQL based tag clouds as well. > > If you're not doing a tag cloud, maybe there's a way you can get the > needed results using map only? > > Also, I'm not sure, but perhaps it would be possible for CouchDB to > cache final reduce values in the btree as well, so that group=true > queries can save the cost of the final rereduce (and make subsequent > queries fast...) > > Chris > > > -- > Chris Anderson > http://jchris.mfdz.com >
