On Mon, Aug 30, 2010 at 10:58 AM, Norman Barker <[email protected]> wrote: > Bob, > > I can and have been testing the multiview at this scale, it is ok > (fast enough), but I think being able to test inclusion of a document > id in a view without having to loop would be a considerable speed > improvement. If you have any ideas let me know. >
I just want to bump this thread, as I think this is a useful feature. I don't expect to be able to test it in the coming weeks, but if I did I would. Is anyone besides Norman using this? Has anyone used it at scale? Cheers, Chris > thanks, > > Norman > > On Mon, Aug 30, 2010 at 10:49 AM, Robert Newson <[email protected]> > wrote: >> I'm sorry, I've had no time to play with this at scale. >> >> On Mon, Aug 30, 2010 at 5:35 PM, Norman Barker <[email protected]> >> wrote: >>> Hi, >>> >>> are there any more comments on this, if not can you describe the >>> process (in particular how to obtain a wiki and jira account for >>> couchdb which I have been unable to do) and I will start documenting >>> this so we can put this into the trunk. >>> >>> Bob, were you able to do any more testing with large views, are there >>> any suggestions on how to speed up the document id inclusion test as >>> described below? >>> >>> thanks, >>> >>> Norman >>> >>> On Mon, Aug 23, 2010 at 9:22 AM, Norman Barker <[email protected]> >>> wrote: >>>> Bob, >>>> >>>> thanks for the feedback and for taking a look at the code. Guidelines >>>> on when to use a supervisor within couchdb with a gen_server would be >>>> appreciated, currently I have a supervisor and a gen_server, but if >>>> couchdb has a supervision process I could remove that layer. >>>> >>>> I think plugins is a great idea, however intersection of views is such >>>> as common request, perhaps there needs to plugin system and if a >>>> plugin is rated enough it goes into trunk as a core feature. >>>> >>>> the four (or slightly more) summary is here >>>> >>>> http://github.com/normanb/couchdb/raw/trunk/src/couchdb/couch_query_ring.erl >>>> >>>> % >>>> % send an id from the start list to the next node in the ring, if the >>>> id is in adjacent node then the this node sends to the next ring node >>>> .... >>>> % if the id gets all round the ring and back to the start node then is >>>> has intersected all queries and should be included. The nodes in the >>>> ring >>>> % should be sorted in size from small to large for this to be effective >>>> % >>>> % In addition send the initial id list round in parallel >>>> >>>> it really needs some eyes from the core couchdb coders to see how to >>>> speed up the inclusion testing, looping is bad even if it is done in >>>> parallel. >>>> >>>> Multiview is usable, I am using it with some pretty big mega-views (as >>>> per the raindrop) model, I am also available to add features to this >>>> as this is core part of our work and we want to give it to couch as a >>>> contribution. >>>> >>>> thanks, >>>> >>>> Norman >>>> >>>> On Mon, Aug 23, 2010 at 5:05 AM, Robert Dionne >>>> <[email protected]> wrote: >>>>> Hi Norman, >>>>> >>>>> I took a peek at multiview. I haven't followed this too closely on the >>>>> mailing list but this is *view intersection*? Is there a 5 line summary >>>>> of what this does somewhere? >>>>> >>>>> I'm curious as to why the daemon needs to be a supervisor, most if not >>>>> all of the other daemons are gen_servers. OTP allows this but I think >>>>> this is a good area where some CouchDB guidelines on plugins would apply. >>>>> >>>>> It strikes me that views, the use of map/reduce, etc. are one of the >>>>> trickier aspects of using CouchDB, particularly for new users coming from >>>>> the SQL world. People are also reporting issues with performance of >>>>> views, I guess often because reduce functions go out of control. >>>>> >>>>> I think the project would be better served if features like this were >>>>> available as plugins. I would put GeoCouch in the same category. Its very >>>>> neat and timely (given everyone wants to know where everyone else is >>>>> using their telephone but without talking other than asynchronously), but >>>>> a server plugin architecture that would allow this to be done cleanly >>>>> should come first. >>>>> >>>>> This is just my opinion. I'd love to see some of the project founders >>>>> and committers weigh in on this and set some direction. >>>>> >>>>> Best regards, >>>>> >>>>> Bob >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Aug 22, 2010, at 5:45 PM, Norman Barker wrote: >>>>> >>>>>> I would like to take this multiview code and have it added to trunk if >>>>>> possible, what are the next steps? >>>>>> >>>>>> thanks, >>>>>> >>>>>> Norman >>>>>> >>>>>> On Wed, Aug 18, 2010 at 11:44 AM, Norman Barker >>>>>> <[email protected]> wrote: >>>>>>> I have made >>>>>>> >>>>>>> http://github.com/normanb/couchdb >>>>>>> >>>>>>> which is a fork of the latest couchdb trunk with the multiview code >>>>>>> and tests added. >>>>>>> >>>>>>> If geocouch is available then it can still be used. >>>>>>> >>>>>>> There are a couple of questions about the multiview on the user /dev >>>>>>> list so I will be adding some more test cases during today. >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Norman >>>>>>> >>>>>>> On Tue, Aug 17, 2010 at 9:23 PM, Norman Barker >>>>>>> <[email protected]> wrote: >>>>>>>> this is possible, I forked geocouch since I use it, but I have already >>>>>>>> separated the geocouch dependencies from the trunk. >>>>>>>> >>>>>>>> I can do this tomorrow, certainly be interested in any feedback. >>>>>>>> >>>>>>>> thanks, >>>>>>>> >>>>>>>> Norman >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Tue, Aug 17, 2010 at 7:49 PM, Volker Mische >>>>>>>> <[email protected]> wrote: >>>>>>>>> On 08/18/2010 03:26 AM, J Chris Anderson wrote: >>>>>>>>>> >>>>>>>>>> On Aug 16, 2010, at 4:38 PM, Norman Barker wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I have made the changes as recommended, adding a test case >>>>>>>>>>> multiview.js and also adding the userCtx to open the db. >>>>>>>>>>> >>>>>>>>>>> I have also forked geocouch and this is available here >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> this patch seems important (especially as people are already asking >>>>>>>>>> for >>>>>>>>>> help using it on user@) >>>>>>>>>> >>>>>>>>>> to get it committed, it either must remove the dependency on >>>>>>>>>> GeoCouch, or >>>>>>>>>> become part of CouchDB when (and if) GeoCouch becomes part of >>>>>>>>>> CouchDB. >>>>>>>>>> >>>>>>>>>> Is it possible / useful to make a version that doesn't use GeoCouch? >>>>>>>>>> And >>>>>>>>>> then to make the GeoCouch capabilities part GeoCouch for now? >>>>>>>>>> >>>>>>>>>> Chris >>>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Norman, >>>>>>>>> >>>>>>>>> if the patch is ready for trunk, I'd be happy to move the GeoCouch >>>>>>>>> bits to >>>>>>>>> GeoCouch itself (as GeoCouch isn't ready for trunk yet). >>>>>>>>> >>>>>>>>> Lately I haven't been that responsive when it comes to GeoCouch, but >>>>>>>>> that >>>>>>>>> will change (in about a month) after holidays and FOSS4G. >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Volker >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>> >>> >> > -- Chris Anderson http://jchrisa.net http://couch.io
