On Fri, Jul 31, 2009 at 10:53 PM, Roman Shaposhnik<r...@sun.com> wrote:
> What are your clients running? Linux > What are their requirements as > far as POSIX is concerned? 10,000 machines, working on a single app, must have access to a common file store with full posix semantics and it all has to work like it were one machine (their desktop, of course). This gets messy. It turns into an exercise of attempting to manage a competing set of race conditions. It's like tuning a multi-carburated enging from years gone by, assuming we ever had an engine with 10,000 cylinders. > How much storage are talking about? In round numbers, for the small clusters, usually a couple hundred T. For anyhing else, more. > > I'd be interested in discussing some aspects of what you're trying to > accomplish with 9P for the HPC guys. The request: for each of the (lots of) compute nodes, have them mount over 9p to, say 100x fewer io nodes, each of those to run lustre. Which tells you right away that our original dreams for lustre did not quite work out. In all honesty, however, the 20K node Jaguar machine at ORNL claims to run lustre and have it all "just work". I know as many people who have de-installed lustre as use it, however. ron
