I've been debugging why the metadata server calls fdatasync() five
times during a single create operation. (IO server separate and
not considered here.)
In fs.conf, I had these StorageHints settings
TroveSyncMeta no
TroveSyncData no
CoalescingHighWatermark infinity
CoalescingLowWatermark 1
(defaults from pvfs2-genconfig with trovesync off).
The login in dbpf-sync.c goes like this:
if (!metadata_sync)
++coalesce_count
if (high_watermark > 0 && coalesce_count >= high_watermark)
coalesce_count = 0
sync
if (num_pending_TROVE_SYNC_operations < low_watermark)
coalesce_count = 0
sync
No matter how low the low watermark, any trove operation marked as
TROVE_SYNC will cause a full sync. Changing
"CoalescingLowWatermark" to 0 fixed that---no syncs.
Do I understand this correctly? Is setting the low WM to zero what
was intended? Any non-zero value of low WM will always cause
immediate sync after every TROVE_SYNC operation---was this planned?
I'd like to have it sync every 5--10 ops, or from a timeout. Is
there some sort of idea that these TROVE_SYNC operations are so
special that they must run immediately, every time?
The five syncing MD operations in a create, for those keeping score,
are:
create dspace_create (sync)
setattr metafile distribution (sync)
setattr dspace_setattr (sync)
crdirent write_directory_entry (sync)
crdirent dspace_setattr (sync)
That's a lot of sync on both dspace and keyval dbs. The total sync
time adds 45 ms to the overall operation on a SATA disk.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
