On Mon, May 4, 2009 at 11:58 AM, Chris Anderson <[email protected]> wrote: > On Mon, May 4, 2009 at 7:23 AM, Zachary Zolton <[email protected]> > wrote: >> @janl >> >> Perhaps he's asking why there's no activity on the other processor? >> >> I think his expectation here is that Map-Reduce would be parallelized. >> Correct me if I'm wrong, but CouchDB does not yet exploit parallelism >> in view indexing yet, right? >> > > Correct. And the JavaScript view server doesn't make it easy to do so > without starting multiple OS processes per map-function. I can see > this being nifty down the road, but it's an optimization. > > However, interest is growing in an Erlang view server, and > http://github.com/mmcdanie/erlview points out that the view process > architecture could use some changes to make Erlang views parallel. > Those changes will in turn make it more trivial to parallelize JS view > computation, in the cases that need it. So I can see us getting to > parallel JS views, but I think the cleaner way to get there is to > start with proper Erlang views. >
I'd also throw in that the biggest bottleneck for JS views is probably the JSON serialization. I've been asked to try getting eep0018 in to a copy of trunk as a configuration parameter. I'll get a branch on github up by the end of the week to have a reference for that. My gut feeling is that once we get that sorted out, we'll probably see that disk I/O becomes the bottleneck. I could see maybe eeking out some extra performance by streaming data through the map servers as opposed to the serialized delegation method we're using now, but then it becomes a cost/benfit of complexity. We'll see how it goes. Paul Davis >> —zdzolton >> >> On Mon, May 4, 2009 at 7:46 AM, Jan Lehnardt <[email protected]> wrote: >>> Beam and couchjs pipe data back and forth during view generation. While the >>> one works, the other waits. The scheduler is smart enough to keep the >>> processes local to a single CPU. Otherwise it's be even more expensive. >>> >>> Cheers >>> Jan >>> -- >>> >>> On 04.05.2009, at 12:50, Elf <[email protected]> wrote: >>> >>>> Hello. >>>> I'm using couchdb-0.9 (erlang 13.2) on my linux server with 4 CPUs >>>> (Core 2 Quad). >>>> Every time couchdb needs to reindex views, i see 2 serious processes >>>> in htop - beam and couchjs. Each of them eats < 100% of 1 cpu, and sum >>>> of their usage is about (but not greater) 100% of 1 cpu (80/20, 70/30 >>>> and so on). >>>> Can somebody explain, why that 2 different processes (they have >>>> differend PIDs) doesn't used different cpus - (one process for cpu)? >>>> >>>> >>>> -- >>>> ---------------- >>>> Best regards >>>> Elf >>>> mailto:[email protected] >>>> >>> >> > > > > -- > Chris Anderson > http://jchrisa.net > http://couch.io >
