* Anthony Liguori <[EMAIL PROTECTED]> [2008-09-26 13:37]: > Ryan Harper wrote: > >* Anthony Liguori <[EMAIL PROTECTED]> [2008-09-26 11:03]:
> >kvm: cache=on posix-aio w/o patch |127.0 | 115.78 | 9.19 > >kvm: cache=on posix-aio w/ patch |126.0 | 67.35 | 9.30 > > > > It looks like 127mb/s is pretty close to the optimal cached write time. > When using caching, writes can complete almost immediately so it's not > surprising that submission latency is so low (even though it's blocking > during submission). > > I am surprised that w/patch has a latency that's so high. I think that > suggests that requests are queuing up. I bet increasing the aio_num > field would reduce this number. Yeah, there is plenty of room to twiddle with the threads and number of outstanding ios, but that'll take quite a bit of time to generate the data and compare. > >------------ new results > >----------+------+------------------+------------------ > >kvm:cache=off posix-aio fd_pool[16]| 33.5 | 14.28 | 49.19 > >kvm:cache=off posix-aio fd_pool[64]| 51.1 | 14.86 | 23.66 > > > > I assume you tried to bump from 64 to something higher and couldn't make > up the lost bandwidth? Very slightly, switching to 128 threads/fds gave another 1MB/s. > >16k write 1 thread, 74 iodepth | MB/s | avg sub lat (us) | avg comp > >lat (ms) > >-----------------------------------+------+------------------+------------------ > >baremetal (O_DIRECT, aka cache=off)|128.1 | 10.90 | 9.45 > >kvm: cache=off posix-aio w/o patch | 5.1 | 3152.00 | 231.06 > >kvm: cache=off linux-aio |130.0 | 83.83 | 8.99 > >kvm: cache=on posix-aio w/o patch |184.0 | 80.46 | 6.35 > >kvm: cache=on posix-aio w/ patch |165.0 | 70.90 | 7.09 > >------------ new results > >----------+------+------------------+------------------ > >kvm:cache=off posix-aio fd_pool[16]| 78.2 | 58.24 | 15.43 > >kvm:cache=off posix-aio fd_pool[64]|129.0 | 71.62 | 9.11 > > > > That's a nice result. We could probably improve the latency by tweaking > the queue sizes. Yeah, I was quite pleased to see a simpler solution perform so well. > > Very nice work! Thanks for doing the thorough analysis. Thanks, very happy to see a signficant improvement in IO here. -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 [EMAIL PROTECTED] -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
