Hi, I recently test YCSB benchmark too. But contrary to my expectation, PG (9.5) is slower than MongoDB 3.2. Paul said that making table with no logging option improved the performance, and it might be equal to MongoDB's behavior. But in MongoDB documentation, it writes journal log too. So I think turning off no logging option is not fair. Am I wrong about MongoDB's behavior?
[http://webmail.bitnine.co.kr/images/?attachID=cmVjcnVpdHxiaXRuaW5lX0NJXzYucG5n&cid=381831a483be2a2d269bb9df50ec4d49&date=20140911&adminId=andrewpark] (C)Bitnine, Kisung Kim, Ph.D https://sites.google.com/site/kisungresearch/ E-mail : ks...@bitnine.net<mailto:ks...@bitnine.net> Office phone : 070-4800-5890, 408-606-8602 US Mobile phone : 408-805-2192 2016-03-19 5:05 GMT+09:00 <p...@cmicdo.com<mailto:p...@cmicdo.com>>: On Tuesday, March 15, 2016 7:39 PM, "p...@cmicdo.com<mailto:p...@cmicdo.com>" <p...@cmicdo.com<mailto:p...@cmicdo.com>> wrote: > Your results are close enough to mine, I think, to prove the point. > And, I agree that the EDB benchmark is not necessary reflective of a > real-world scenario. > > However, the cache I'm referring to is PG's shared_buffer cache. > You can see the first run of the select causing a lot of disk reads. > The second identical run, reads purely from shared_buffers. > > What I don't understand is, why does a slightly different select from > the *same* table during the same session cause shared_buffers to be > blown out and re-read?? > > I will see if I can try YCSB next week (I'm in workshops all week...) > > Thanks! I was able to try YCSB today on both PG 9.5.1 and Mongo 3.2. At first, PG was running 4 times slower than Mongo. Then I remembered about unlogged tables (which I think is the way Mongo is all the time.), and remade the PG table as UNLOGGED. In a 50/50 read/update test over 1M records, PG ran in 0.62 of the time of Mongo. PG Load: -------- [OVERALL], RunTime(ms), 104507.0 [OVERALL], Throughput(ops/sec), 9568.737022400413 [CLEANUP], Operations, 1.0 [CLEANUP], AverageLatency(us), 293.0 [CLEANUP], MinLatency(us), 293.0 [CLEANUP], MaxLatency(us), 293.0 [CLEANUP], 95thPercentileLatency(us), 293.0 [CLEANUP], 99thPercentileLatency(us), 293.0 [INSERT], Operations, 1000000.0 [INSERT], AverageLatency(us), 101.329235 [INSERT], MinLatency(us), 88.0 [INSERT], MaxLatency(us), 252543.0 [INSERT], 95thPercentileLatency(us), 121.0 [INSERT], 99thPercentileLatency(us), 141.0 [INSERT], Return=OK, 1000000 PG Run: ------- [OVERALL], RunTime(ms), 92763.0 [OVERALL], Throughput(ops/sec), 10780.16019318047 [READ], Operations, 499922.0 [READ], AverageLatency(us), 79.1722428698877 [READ], MinLatency(us), 69.0 [READ], MaxLatency(us), 19935.0 [READ], 95thPercentileLatency(us), 94.0 [READ], 99thPercentileLatency(us), 112.0 [READ], Return=OK, 499922 [CLEANUP], Operations, 1.0 [CLEANUP], AverageLatency(us), 222.0 [CLEANUP], MinLatency(us), 222.0 [CLEANUP], MaxLatency(us), 222.0 [CLEANUP], 95thPercentileLatency(us), 222.0 [CLEANUP], 99thPercentileLatency(us), 222.0 [UPDATE], Operations, 500078.0 [UPDATE], AverageLatency(us), 98.96430156895525 [UPDATE], MinLatency(us), 83.0 [UPDATE], MaxLatency(us), 26655.0 [UPDATE], 95thPercentileLatency(us), 127.0 [UPDATE], 99thPercentileLatency(us), 158.0 [UPDATE], Return=OK, 500078 Mongo Load: ----------- [OVERALL], RunTime(ms), 133308.0 [OVERALL], Throughput(ops/sec), 7501.425270801452 [CLEANUP], Operations, 1.0 [CLEANUP], AverageLatency(us), 1822.0 [CLEANUP], MinLatency(us), 1822.0 [CLEANUP], MaxLatency(us), 1822.0 [CLEANUP], 95thPercentileLatency(us), 1822.0 [CLEANUP], 99thPercentileLatency(us), 1822.0 [INSERT], Operations, 1000000.0 [INSERT], AverageLatency(us), 130.830678 [INSERT], MinLatency(us), 90.0 [INSERT], MaxLatency(us), 7147519.0 [INSERT], 95thPercentileLatency(us), 159.0 [INSERT], 99thPercentileLatency(us), 226.0 [INSERT], Return=OK, 1000000 Mongo Run: --------- [OVERALL], RunTime(ms), 149150.0 [OVERALL], Throughput(ops/sec), 6704.65973851827 [READ], Operations, 500837.0 [READ], AverageLatency(us), 98.13153980237084 [READ], MinLatency(us), 69.0 [READ], MaxLatency(us), 28271.0 [READ], 95thPercentileLatency(us), 166.0 [READ], 99thPercentileLatency(us), 186.0 [READ], Return=OK, 500837 [CLEANUP], Operations, 1.0 [CLEANUP], AverageLatency(us), 2387.0 [CLEANUP], MinLatency(us), 2386.0 [CLEANUP], MaxLatency(us), 2387.0 [CLEANUP], 95thPercentileLatency(us), 2387.0 [CLEANUP], 99thPercentileLatency(us), 2387.0 [UPDATE], Operations, 499163.0 [UPDATE], AverageLatency(us), 195.21505600375028 [UPDATE], MinLatency(us), 118.0 [UPDATE], MaxLatency(us), 4513791.0 [UPDATE], 95thPercentileLatency(us), 211.0 [UPDATE], 99thPercentileLatency(us), 252.0 [UPDATE], Return=OK, 499163 > > > On Monday, March 14, 2016 3:34 AM, Dmitry Dolgov > <9erthali...@gmail.com<mailto:9erthali...@gmail.com>> wrote: > > > Hi, Paul > > I agree with Oleg, EDB benchmarks are strange sometimes. I did the same > benchmarks several months ago. I never noticed the cache influence back > then, so I tried to reproduce your situation now (on a 5*10^6 records > although). I started to play with db cache (using `echo 3 > > /proc/sys/vm/drop_cache`), and I see difference in time execution for two > subsequent queries, but `explain` info are almost identical, e.g. `shared > hit & read`: > > ....
