Could you break down what the app is doing at a little bit higher level
in this time frame? (ie, how many writes is it posting, how many reads
is it posting, which are concurrent, when it calls wait for each).
From what I can tell, it looks like there are 30 total isys_io's
posted; the first 15 are writes (triggered by pvfs2_aio_flush) and the
last 15 are reads (triggered by pvfs2_aio_fill). It doesn't look like
there are any waits in between the two, though, if I am reading it right.
Are you calling wait() (or some other variant) between the writes and
the reads? The pvfs2 system interface doesn't order any of the
operations, so it might just be that some of your reads happen to be
hitting the server before your writes have put the data there.
This is different from the standard posix aio; I think their api
automatically orders every I/O operation at least at a file descriptor
level. The PVFS system interface doesn't do anything like that to
prevent I/O operations from getting out of order once they are posted.
-Phil
Troy Benjegerdes wrote:
I have an IO shim for a fortran application that uses pvfs_isys_io and
pvfs_sys_wait directly, and when using the IB BMI module, I issue a
whole bunch of writes, but it looks like the last one doesn't actually
make it all the way to disk. I get a failure where I go back to read the
last 32M block of the file, and the total_completed value is 10mb
instead of the 32MB it's supposed to be.
I've got debugging output (PVFS2_DEBUGMASK=all) for both a single
server, and a single client at:
http://scl.ameslab.gov/~troy/pvfs/corruption/pvfs2-server-da13.log
http://scl.ameslab.gov/~troy/pvfs/corruption/qtest.err
Where should I be looking for problems?
When the file already exists, I don't get this error, but I seem to get
data that was already in the file, instead of what I just wrote.
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
