On 01/26/2016 09:32 AM, Dave Love wrote:
Rob Latham <r...@mcs.anl.gov> writes:
We didn't need to deploy PLFS at Argonne: GPFS handled writing N-to-1
files just fine (once you line up the block sizes), so I'm beholden to
PLFS communities for ROMIO support.
I guess GPFS has improved in that respect, as I think it benefited
originally. Is it known whether Lustre has improved sufficiently too?
Lustre is still the same but the MPI-IO implementations have gotten
better at getting performance out of lustre. Once MPI-IO
implementations figured out that constructing i/o requests such that a
client only talks to one OST resulted in a lot less lock revocation, the
Lustre story got a lot better.
[I wonder why ANL apparently dumped PVFS but I probably shouldn't ask.]
No harm in asking. This is getting a bit off topic for an OpenMPI list
but OMPIO and ROMIO both support PVFS so perhaps it's interesting to some.
ANL still adores PVFS. It's a fantastic file system for exploring
research ideas. Maintaining a production-quality research file system
(the "production quality" part, I mean) took a lot of work, though, and
when our long-time collaborators at Clemson wanted to spin up a
commercial support entity for PVFS we said "yes please that sounds great!".
Maybe you are curious why ALCF, our compute facility, moved away from
PVFS over time? PVFS on Blue Gene worked pretty well, but it wasn't
PVFS's best environment -- we could not use key PVFS optimizations like
"datatype-aware I/O" and "scalable open" with the stripped down Blue
Gene kernel and instead had to treat PVFS like a regular unix file
system -- which it unashamedly is not. MPI-IO applications worked fine,
but we had a non-trivial amount of applications that wanted to do things
(e.g. mmap a file) that PVFS supported poorly if at all.
Parallel file systems are complex beasts, and in order to provide good
performance in one part of the design space, need to make trade-offs in
another. PVFS had a good 5 year run, but it ended up being easier for
ALCF to deal with the GPFS quirks (don't create N files in a directory!)
than to educate users about quirks inherent in two different parallel
file systems.
A single file system also meant the storage budget -- which is never
large enough, says the storage and I/O guy -- need not be split across
two file systems.
PVFS still lives, now branded "OrangeFS". The folks at Clemson are
working towards getting PVFS support into the linux kernel.
(http://lwn.net/Articles/657333/ : Linus says "the code doesn't look
obviously horrible" -- how's *that* for an endorsement)
==rob
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfoW.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2016/01/28386.php
