Hi Willi, It's OK. I see 3.0 introduces "sync replication" and "binary format". That's exciting.
And, about collection level lock, I tried to separate their processing by binding the collection to a specific dispatcher thread. I re-run the test in a more powerful machine(two YCSB clients writing two different collections), the result is quite good. One collection's load won't impact another collection. In this way, ArangoDB is able to scale up with multiple collections. On Tuesday, May 17, 2016 at 7:03:23 PM UTC+8, Wilfried Gösgens wrote: > > Hi Yemin, > we're still investigating on this. > As you may have noticed, we're currently working hard to get 3.0 into its > final shape, so please bear with us. > > Cheers, > Willi > > On Friday, May 6, 2016 at 1:43:54 AM UTC+2, Yemin wrote: >> >> Hi Frank, >> I uploaded my code to github. >> https://github.com/smallsmallc/YCSB/tree/arangodb/arangodb >> TestCase: >> YCSB: 100%insert, with 48 threads. >> Arango: schedule.threads=1, v8.threads=1, server.threads=1/2 >> If your machine is powerful enough, the dispatcher thread will be more >> 95%CPU under 1 dispatcher thread. And If using 2 dispatcher threads, the >> CPU of two dispatcher threads won't more than 100%CPU. It means that there >> is a bottleneck here. >> >> More info: >> We found there is a collection level lock for multiple >> dispatcher threads. And we tried two YCSB clients writing two different >> collections, but the problem still exists. Further more, we found one >> dispatcher thread will handle tasks from different collections. So, we >> separate them. We change the ArangoDB code, making all tasks from same >> collection handled in same dispatcher thread. However the problem still >> exists. >> So, besides collection level lock, there must be some other kind >> of lock. >> >> >> >> >> On Thursday, May 5, 2016 at 4:13:08 PM UTC+8, Frank Celler wrote: >>> >>> Hi Yemin, >>> >>> is your tests open-source? Can you send me / create a gist / put it on >>> github? That we I can try to understand what is going on. >>> >>> Thanks >>> >>> >>> Am Donnerstag, 5. Mai 2016 01:19:49 UTC+2 schrieb Yemin: >>>> >>>> Hi Frank, >>>> I use the java driver from ArangoDB and the network link is >>>> persistent. Maybe because the collection locker? In my test case, all >>>> clients are writing same collection. >>>> >>>> >>>> On Wednesday, May 4, 2016 at 11:42:42 PM UTC+8, Frank Celler wrote: >>>>> >>>>> Hi Yemin, >>>>> >>>>> what kind of tests are used in the YCSB? Are you using the Java driver >>>>> from ArangoDB or direct HTTP requests? Sometimes I have seen, that in >>>>> various languages HTTP are not persistent (by default or at all). Could >>>>> you >>>>> see during your test, if there are persistent HTTP connections or not? >>>>> >>>>> A dispatcher thread keeps active for a few milli-seconds, before >>>>> waiting on a condition variable. This gives better performance on some >>>>> systems comapred to directly going to sleep. >>>>> >>>>> Best >>>>> >>>>> >>>>> Am Mittwoch, 4. Mai 2016 17:17:17 UTC+2 schrieb Yemin: >>>>>> >>>>>> Hi, recently I'm doing perf test of ArangoDB using YCSB and found >>>>>> the dispatcher threads problems. >>>>>> Case: pure inserting operations, measuring throughput. 10 scheduler >>>>>> threads, 48 clients. >>>>>> For throughput and total CPU cost of dispatcher threads, one >>>>>> dispatcher thread has no difference with multiple dispatcher threads. It >>>>>> looks like there is some kind of "global lock". >>>>>> Questions: >>>>>> 1. Why the throughput wasn't improved by adding more dispatcher >>>>>> threads. >>>>>> 2. We found many nanosleeps in dispatcher thread, maybe it's one of >>>>>> reasons? And why using nanosleep instead of wait-signal? >>>>>> >>>>>> Thx! >>>>>> >>>>> -- You received this message because you are subscribed to the Google Groups "ArangoDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
