Yes, that worked wonders! Thanks so much. I think there is an underlying issue with Neo4j trying to fsync too often, but at least now I can continue to develop on my machine which is all that really matters. Really appreciate your time.
Percentage of the requests served within a certain time (ms) 50% 3 66% 3 75% 4 80% 4 90% 4 95% 5 98% 9 99% 11 100% 28 (longest request) On Thursday, January 16, 2014 7:50:19 PM UTC-8, Wes Freeman wrote: > > Added it to my travis script: > https://github.com/wfreeman/cq/blob/master/.travis.yml > > With it I get: > Percentage of the requests served within a certain time (ms) > 50% 3 > 66% 3 > 75% 3 > 80% 3 > 90% 3 > 95% 4 > 98% 4 > 99% 5 > 100% 11 (longest request) > > Wes > > On Thu, Jan 16, 2014 at 9:02 PM, Michael Hunger < > michael...@neopersistence.com <javascript:>> wrote: > >> Bill, >> >> Wes gave me an idea, so I created a version of the Neo4j server that uses >> our testing-in-memory graph database, >> care to try it? >> >> https://github.com/jexp/neo4j-in-memory-server >> >> Cheers >> >> Michael >> >> Am 17.01.2014 um 02:52 schrieb Michael Hunger < >> michael...@neopersistence.com <javascript:>>: >> >> Good question, not sure about that. >> >> Indexing uses lucene behind the scenes, and adds some overhead. >> >> Michael >> >> Am 17.01.2014 um 02:42 schrieb Bill Scheidel <bi...@bunkat.com<javascript:> >> >: >> >> So does that mean multiple fsync operations are done? I'm trying to >> figure out what would cause the extra 110ms of delay between no indexing >> and indexing one property. >> >> On Thursday, January 16, 2014 5:39:32 PM UTC-8, Michael Hunger wrote: >>> >>> Because it is transactional. >>> >>> So during your tx whenever things are changed / created that correspond >>> to the configured auto-indexing properties they are written to the index. >>> >>> Michael >>> >>> Am 17.01.2014 um 02:35 schrieb Bill Scheidel <bi...@bunkat.com>: >>> >>> Hmm... turning off node_auto_indexing drops it from 150ms to 40ms. Why >>> would auto indexing block a request? >>> >>> On Thursday, January 16, 2014 5:07:48 PM UTC-8, Bill Scheidel wrote: >>>> >>>> My hdparm results are 118 MB/sec which isn't horrible, but it seems >>>> like disk latency is the only thing that matters. I guess I'll try going >>>> back to the stock settings and moving it over to an SSD and see what >>>> happens. >>>> >>>> On Thursday, January 16, 2014 4:34:30 PM UTC-8, Wes Freeman wrote: >>>>> >>>>> My macbook's virtualbox (running centos) got good results too (99% >>>>> <20ms, 50% <7ms). Was hoping for some weirdness. It is running on an ssd >>>>> (vintage 2011 macbook pro 13"), hdparm 250MB/sec, so not a great >>>>> comparison. Only has 800MB allocated for the VM RAM, using Neo4j stock >>>>> settings. >>>>> >>>>> Wes >>>>> >>>>> On Thu, Jan 16, 2014 at 7:08 PM, Bill Scheidel <bi...@bunkat.com> >>>>> wrote: >>>>> >>>>>> Yeah, definitely not great. Odd though since I never had a problem >>>>>> when working with Postgres or Mongo and they force things to disk as >>>>>> well. >>>>>> Never had local requests take more than a couple of ms and then I >>>>>> switch >>>>>> over to Neo4j and its almost unusable. There are no flags to change the >>>>>> behavior for dev/test machines? >>>>>> >>>>>> >>>>>> On Thursday, January 16, 2014 4:00:10 PM UTC-8, Michael Hunger wrote: >>>>>> >>>>>>> Not so good latency during the test, or? >>>>>>> >>>>>>> Here is my ioping (cool never heard of that one, nice tool). >>>>>>> >>>>>>> w/o ab >>>>>>> --- /tmp/ (hfs /dev/disk0s2) ioping statistics --- >>>>>>> 11 requests completed in 10.2 s, 32.1 k iops, 125.3 mb/s >>>>>>> min/avg/max/mdev = 21 us / 31 us / 50 us / 7 us >>>>>>> >>>>>>> w ab >>>>>>> 29 requests completed in 29.0 s, 33.1 k iops, 129.3 mb/s >>>>>>> min/avg/max/mdev = 20 us / 30 us / 109 us / 17 us >>>>>>> >>>>>>> Am 17.01.2014 um 00:54 schrieb Bill Scheidel <bi...@bunkat.com>: >>>>>>> >>>>>>> > And this is ioping without the ab test running: >>>>>>> > >>>>>>> > 31 requests completed in 30.5 s, 3.2 k iops, 12.5 mb/s >>>>>>> > min/avg/max/mdev = 190 us / 312 us / 477 us / 63 us >>>>>>> > >>>>>>> > On Thursday, January 16, 2014 3:51:32 PM UTC-8, Bill Scheidel >>>>>>> wrote: >>>>>>> > I ran vmstat while running the ab test: >>>>>>> > >>>>>>> > procs -----------memory---------- ---swap-- -----io---- -system-- >>>>>>> ----cpu---- >>>>>>> > r b swpd free buff cache si so bi bo in cs >>>>>>> us sy id wa >>>>>>> > 0 1 0 2429284 161572 2133284 0 0 22 75 98 >>>>>>> 354 9 8 80 2 >>>>>>> > >>>>>>> > I also ran ioping to check disk latency while the ab test was >>>>>>> running: >>>>>>> > >>>>>>> > 190 requests completed in 3.3 min, 37 iops, 151.9 kb/s >>>>>>> > min/avg/max/mdev = 192 us / 26.3 ms / 178.1 ms / 26.5 ms >>>>>>> > >>>>>>> > Results from the ab run: >>>>>>> > >>>>>>> > Percentage of the requests served within a certain time (ms) >>>>>>> > 50% 150 >>>>>>> > 66% 158 >>>>>>> > 75% 166 >>>>>>> > 80% 168 >>>>>>> > 90% 209 >>>>>>> > 95% 276 >>>>>>> > 98% 300 >>>>>>> > 99% 324 >>>>>>> > 100% 366 (longest request) >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > On Thursday, January 16, 2014 3:33:55 PM UTC-8, Michael Hunger >>>>>>> wrote: >>>>>>> > Let's continue this discussion here. >>>>>>> > >>>>>>> > To collect the other information so far: http://stackoverflow.com/ >>>>>>> questions/21145723/neo4j-2-0-0-poor-performance-for-dev-test- >>>>>>> in-a-virtual-machine >>>>>>> > The GH issue you raised with Wes' and my answers: >>>>>>> https://github.com/neo4j/neo4j/issues/1829 >>>>>>> > >>>>>>> > My "ab" tests: https://gist.github.com/jexp/8452037 >>>>>>> > Wes' numbers: https://github.com/neo4j/neo4j/issues/1829# >>>>>>> issuecomment-32564561 >>>>>>> > >>>>>>> > Your messages.log looks good to me. >>>>>>> > >>>>>>> > So it might be related to disk performance, could you run vmstat >>>>>>> or similar while running the ab test? >>>>>>> > >>>>>>> > I think it is related to the forced fsync at commit which can be >>>>>>> hit by a higher disk latency? >>>>>>> > >>>>>>> > Michael >>>>>>> > >>>>>>> > -- >>>>>>> > You received this message because you are subscribed to the Google >>>>>>> Groups "Neo4j" group. >>>>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to neo4j+un...@googlegroups.com. >>>>>>> > For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Neo4j" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to neo4j+un...@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>> >>>>> >>>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Neo4j" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to neo4j+un...@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to neo4j+un...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to neo4j+un...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to neo4j+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
