Orlando, give that there are so many emails/interest on this topic in recent month, let me share some personal expertise on this :-)
any stock Samba or CTDB version you will find any distro is not sufficient and it doesn't matter which you choose (SLES, RHEL or any form of debian and any version of all of them). the reason is that Samba doesn't have the GPFS header and library files included in its source, and at compile time it dynamically enables/disables all GPFS related things based on the availability of the GPFS packages . as non of the distros build machines have GPFS installed all this packages end up with binaries in their rpms which don't have the required code enabled to properly support GPFS and non of the vfs modules get build either. the only way to get something working (don't get confused with officially Supported) is to recompile the CTDB src packages AND the Samba src packages on a node that has GPFS already installed. also the inclusion of CTDB into Samba will not address this, its just a more convenient packaging. Only if the build happens on such a node things like the vfs modules for GPFS are build and included in the package. said all this the binaries alone are only part of the Solution, after you have the correct packages, you need to properly configuration the system and setting all the right options (on GPFS as well as on CTDB and smbd.conf), which unfortunate are very System configuration specific, as otherwise you still can end up with data corruption if not set right. also some people in the past have used a single instance of Samba to export shares over CIFS as they believe its a safe alternative to a more complex CTDB setup. also here a word of caution, even if you have a single instance of Samba running on top of GPFS you are exposed to potential data corruption if you don't use the proper Samba version (explained above) and the proper configuration, you can skip CTDB For that, but you still require a proper compiled version of Samba with GPFS code installed on the build machine. to be very clear the problem is not GPFS, its that Samba does locking AND caching on top of the Filesystem without GPFS knowledge if you don't use the right code/config to 'tell' GPFS about it, so GPFS can not ensure data consistency, not even on the same physical node for data thats shared over CIFS. there are unfortunate no shortcuts. i also have to point out that if you recompile the packages and configure everything correctly this is most likely to work, but you won't get official support for the CIFS part of this setup from IBM. This email is not an official Statement/Response of IBM, see it as personal 'AS-IS' Information sharing. Sven From: Orlando Richards <orlando.richa...@ed.ac.uk> To: gpfsug-discuss@gpfsug.org Date: 12/13/2013 07:35 AM Subject: Re: [gpfsug-discuss] GPFS and both Samba and NFS Sent by: gpfsug-discuss-boun...@gpfsug.org On 13/12/13 15:31, Jonathan Buzzard wrote: > On Fri, 2013-12-13 at 15:15 +0000, Orlando Richards wrote: > > [SNIP] > >> Hi Lindsay, >> >> We rebuild ctdb from the (git) source (in the 1.2.40 branch currently), >> after running into performance problems with the sernet bundled version >> (1.0.114). It's easy to build: > > Interestingly the RHEL7 beta is shipping ctdb 2.1 in combination with > Samba 4.1 > > http://ftp.redhat.com/pub/redhat/rhel/beta/7/x86_64/os/Packages/ > > JAB. > The samba team are currently working to bring ctdb into the main samba source tree - so hopefully this will become a moot point soon! -- -- Dr Orlando Richards Information Services IT Infrastructure Division Unix Section Tel: 0131 650 4994 skype: orlando.richards The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss