Oleg thanks for your excellent answers. To my understanding (My knowledge on Linux is not broad) We will have stable NFS export capabilities both for 1.4x and 1.6.x with in weeks? (Project will be istalled in februrary and NFS Gateways might wait till the end of fabruary)
Basically we will attach a Sun 25k to Lustre partition. I plan to use 2x Gateway server each with IB to Lustre and 10 Gbit ethernet to 25k . PS: BTW do HP support IB loadbalancing with their version of Lustre? regards Mertol Ozyoney Storage Practice - Sales Manager Sun Microsystems, TR Istanbul TR Phone +902123352200 Mobile +905339310752 Fax +902123352222 Email [EMAIL PROTECTED] -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Oleg Drokin Sent: Friday, December 14, 2007 5:18 AM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [email protected]; [EMAIL PROTECTED] Subject: Re: [Lustre-discuss] Lustre NFs expport Hello! On Dec 13, 2007, at 7:50 PM, Mertol Ozyoney wrote: > Is there any experiance about exporting Lustre via NFS to outside > world? Any whitepapers, how to guides, schemas etc.. There are some people who do this, also HP support this with their Lustre spin-off and I think they are one of the biggest users at the moment. Their latest tests indicate that lustre reexported with NFS achieves around 90% of the lustre link speed (this was tested with gigabit connections, but I see nothing that would prevent us from this on higher connection speeds provided there is enough CPU and bus bandwidth). These results were achieved with 2.6 kernel on (2.4 kernels perform significantly worse, like 2x worse due to very small i/o submitted to lustre). To obtain this sort of speed special patch (bug 13371 in our bugzilla) was used on lustre 1.4.11, that is not part of released lustre yet, though it will likely be included into 1.4.12. Port of the patch is also underway for 1.6 lustre releases. Metadata speed is mostly around 1/2 or 2/3 compared to NFS exported off ext3 if lustre transport is tcp. For lower-latency network (like infiniband) we do better, but I do not have any numbers at hand. Exporting lustre via NFS is not unlike any other filesystem, you just write /etc/exports entry for fs you want to export and you are mostly done (for some older releases you also need to specify fsid parameter). Some people (HP) also export one lustre filesystem via several NFS servers (to many clients) that allows to achieve better speed when lustre link is significantly faster than NFS side link (also for load-balancing?), for this case normally you need to have identical fsid= setting in your /etc/exports, though current 1.6 release eliminate this requirement. Also in this case you cannot have consistent posix locking unless you use 2.6.22 kernel or above. (also sles9 kernels have special patch that allows this. The patch was dropped in sles10, though). Bye, Oleg _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss _______________________________________________ Lustre-discuss mailing list [email protected] https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
