On 17/04/2020 14:15, Aaron Knister wrote:
Yeah, I had similar experiences in the past (over a decade ago) with Lustre and was heavily heavily anti-Lustre. That said, I just finished several weeks of what I’d call grueling testing of DDN Lustre and GPFS on the same hardware and I’m reasonably convinced much of that is behind us now (things like stability, metadata performance, random I/O performance just don’t appear to be issues anymore and in some cases these operations are now faster in Lustre).
Several weeks testing frankly does not cut the mustard to demonstrate stability. Our Lustre would run for months on end then boom, metadata server kernel panics. Sometimes but not always this would introduce the incorrectable file system corruption. You are going to need to have several years behind it to claim it is now stable.
At this point I would note that basically a fsck on Lustre is not possible. Sure there is a somewhat complicated procedure for it, but firstly it is highly likely to take weeks to run, and even then it might not be able to actually fix the problem.
Full disclosure, I work for DDN, but the source of my paycheck has relatively little bearing on my technical opinions. All I’m saying is for me to honestly believe Lustre is worth another shot after the experiences I had years ago is significant. I do think it’s key to have a vendor behind you, vs rolling your own. I have seen that make a difference. I’m happy to take any further conversation/questions offline, I’m in no way trying to turn this into a marketing campaign.
Lustre is as of two years ago still behind GPFS 3.0 in terms of features and stability in my view. The idea it has caught up to GPFS 5.x in the last two years is in my view errant nonsense, software development just does not work like that.
Let me put it another way, in our experience the loss of compute capacity from the downtime of Lustre exceeded the cost of GPFS licenses. That excludes the wage costs of researches twiddling their thumbs whilst the system was restored to working order.
If I am being cynical if you can afford DDN storage in the first place stop winging about GPFS license costs.
JAB. -- Jonathan A. Buzzard Tel: +44141-5483420 HPC System Administrator, ARCHIE-WeSt. University of Strathclyde, John Anderson Building, Glasgow. G4 0NG _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
