One thing that we noticed while testing for storage challenge was that
(and everyone correct me if I'm wrong here) enabling the data-sync
causes a flush/sync to occur after every sizeof(FlowBuffer) bytes had
been written. I can imagine how this would help a SAN, but I'm
perplexed how it helps localdisk, what buffer size are you playing with?
We found that unless we were using HUGE (~size of cache on storage
controller) flowbuffers that this caused way too many syncs/seeks on the
disks and hurt performance quite a bit, maybe even as bad as 50%
performance because things were not being optimized for our disk
subsystems and we were issuing many small ops instead of fewer large ones.
Granted I havent been able to get 2.6.0 building properly yet to test
the latest out, but this was definitely the case for us on the 2.5
releases.
You are definitely right about the data sync option causing a flush/sync
on every sizeof(FLowBuffer). I don't really have a good explanation for
why this doesn't seem to burn us anymore on local disk. Our settings
are standard, except for:
- 512KB flow buffer size
- alt aio method
- 512KB tcp buffers (with larger /proc tcp settings)
This testing was done on some version prior to 2.6.0 also (I think it
was a merge of some in-between release, so it is hard to pin down a
version number).
It may also have something to do with the controller and local disks
being used? All of our local disk configurations are actually hardware
raid 5 with some variety of the megaraid controller, and these are
fairly new boxes.
-Phil
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers