Hi Michael, As Carlos said the SSDs do not improve meta data rates to much due to the bottle necks in Lustre code itself (at least this is the case in Lustre-1.8)
If you have a limited budget the DRBD solution seems like a good choice. I have been running production Lustre filesystem which uses DRBD mirrored MDT for 4 years now and never had problems with it. However if the budget is not tight and your are thinking about adding more filesystems in the future then you may consider an external RAID disk array for example with 24 small 2.5" SAS 15Kdisks. My bigger production Lustre filesystems use that type of storage and it does the job very well. For the information on both configurations and some matadata performance figures please read through my white papers: http://i.dell.com/sites/content/shared-content/solutions/en/Documents/lustre-storage-brick-white-paper.pdf http://i.dell.com/sites/content/business/solutions/hpcc/en/Documents/Lustre-HPC-Whitepaper-10082011.pdf Best regards, Wojciech On 22 January 2012 19:55, Michael Kluge <michael.kl...@tu-dresden.de> wrote: > Hi Carlos, > > > In my experience SSDs didn't help much, since the MDS bottleneck is not > > only a disk problem rather than the entire lustre metadata mechanism. > > Yes, but one does not need much space on the MDS and four SSDs (as MDT) > are way cheaper than a RAID controller with 10 15K disks. So the > question is basically how the DRDB latency will influence the MDT > performance. I know sync/async makes a big difference here, but I have > no idea about the performance impact of both or how the reliability is > influenced. > > > One remark about DRDB: I've seen customers using it, but IMHO, if > > Active/standby HA type configuration would be more reliable and will > > provide you a better resilience. Again, don't know about your uptime and > > reliability needs, but the customers I've worked with that requires > > minimum downtime on production, always go for RAID controllers rather > than > > DRDB replication. > > OK, thanks. That is a good information. So SSD+DRDB are considered to be > the "cheap" solution. Even for small clusters? > > > Regards, Michael > > > > > Regards, > > Carlos. > > > > > > -- > > Carlos Thomaz | Systems Architect > > Mobile: +1 (303) 519-0578 > > ctho...@ddn.com | Skype ID: carlosthomaz > > DataDirect Networks, Inc. > > 9960 Federal Dr., Ste 100 Colorado Springs, CO 80921 > > ddn.com<http://www.ddn.com/> | Twitter: @ddn_limitless > > <http://twitter.com/ddn_limitless> | 1.800.TERABYTE > > > > > > > > > > > > On 1/22/12 12:04 PM, "Michael Kluge"<michael.kl...@tu-dresden.de> > wrote: > > > >> Hi, > >> > >> I have been asked, which one of the two I would chose for two MDS > >> servers (active/passive). Whether I would like to have SSDs, maybe two > >> (mirrored) in both servers and DRDB for synching, or a RAID controller > >> that has a 15K disks. I have not done benchmarks on this topic myself > >> and would like to ask if anyone has an idea or numbers? The cluster will > >> be pretty small, about 50 clients. > >> > >> > >> Regards, Michael > >> > >> -- > >> Dr.-Ing. Michael Kluge > >> > >> Technische Universität Dresden > >> Center for Information Services and > >> High Performance Computing (ZIH) > >> D-01062 Dresden > >> Germany > >> > >> Contact: > >> Willersbau, Room WIL A 208 > >> Phone: (+49) 351 463-34217 > >> Fax: (+49) 351 463-37773 > >> e-mail: michael.kl...@tu-dresden.de > >> WWW: http://www.tu-dresden.de/zih > >> _______________________________________________ > >> Lustre-discuss mailing list > >> Lustre-discuss@lists.lustre.org > >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > > > > -- > Dr.-Ing. Michael Kluge > > Technische Universität Dresden > Center for Information Services and > High Performance Computing (ZIH) > D-01062 Dresden > Germany > > Contact: > Willersbau, Room WIL A 208 > Phone: (+49) 351 463-34217 > Fax: (+49) 351 463-37773 > e-mail: michael.kl...@tu-dresden.de > WWW: http://www.tu-dresden.de/zih > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > -- Wojciech Turek Senior System Architect High Performance Computing Service University of Cambridge Email: wj...@cam.ac.uk Tel: (+)44 1223 763517
_______________________________________________ Lustre-discuss mailing list Lustre-discuss@lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
