Jez, the other replies to my email where kind of unexpected as i was indeed intending to help, i will continue to provide details and try to answer serious questions and comments.
also as you point out there is no secret master plan to spread misinformation, the topic is simply complicated and there are reasons that are hard to explain why there is no full official support outside the current shipping IBM products and i will not dive into discussions around it. a few words of clarification, on the controversial point, compile ctdb yes/no. while its technically correct, that you don't have to recompile ctdb and it has no code that links/depends in any form on GPFS (samba does), you still have to compile one if the one shipped with your distro is out of date and while i didn't explicitly wrote it this way, this is what i intended to say. the version RH ships is very old and since has gotten many fixes which i highly recommend people to use unless the distros will pickup more recent version like RHEL7 will do. i probably wouldn't recommend a daily git build, but rather the packaged versions that are hosted on ftp.samba.org like : https://ftp.samba.org/pub/ctdb/packages/redhat/RHEL6/ so the proposed order of things would be to install gpfs, pull the src package of ctdb, compile and install ctdb and the devel packages, then pull a recent samba src package , install all the dependencies and build samba on this same host with gpfs and ctdb packages already installed. the resulting rpm's should contain the proper code to continue. after you got your versions compiled, installed and the basics of ctdb running, you should use the following smb.conf as a starting point : [global] netbios name = cluster fileid:mapping = fsname gpfs:sharemodes = yes gpfs:leases = yes gpfs:dfreequota = yes gpfs:hsm = yes syncops:onmeta = no kernel oplocks = no level2 oplocks = yes notify:inotify = no vfs objects = shadow_copy2 syncops gpfs fileid shadow:snapdir = .snapshots shadow:fixinodes = yes shadow:snapdirseverywhere = yes shadow:sort = desc wide links = no async smb echo handler = yes smbd:backgroundqueue = False use sendfile = no strict locking = yes posix locking = yes force unknown acl user = yes nfs4:mode = simple nfs4:chown = yes nfs4:acedup = merge nfs4:sidmap = /etc/samba/sidmap.tdb gpfs:winattr = yes store dos attributes = yes map readonly = yes map archive = yes map system = yes map hidden = yes ea support = yes dmapi support = no unix extensions = no socket options = TCP_NODELAY SO_KEEPALIVE TCP_KEEPCNT=4 TCP_KEEPIDLE=240 TCP_KEEPINTVL=15 strict allocate = yes gpfs:prealloc = yes if you don't configure using the registry you need to maintain the smb.conf file on all your nodes and i am not diving into how to setup the registry, but for the people who care, michael adam's presentation is a good starting point http://www.samba.org/~obnox/presentations/linux-kongress-2008/lk2008-obnox.pdf also depending on your authentication/idmap setup , there are multiple changes/additions that needs to be made. on the gpfs side you should set the following config parameters : cifsBypassShareLocksOnRename=yes syncSambaMetadataOps=yes cifsBypassTraversalChecking=yes allowSambaCaseInsensitiveLookup=no your filesystem should have the following settings configured : -D nfs4 File locking semantics in effect -k nfs4 ACL semantics in effect -o nfssync Additional mount options --fastea Yes Fast external attributes enabled? -S relatime Suppress atime mount option the -S is a performance optimization, but you need to check if your backup/av or other software can deal with it, it essentially reduces the frequency of atime updates to once every 24 hours which is the new default on nfs mounts and others as well. a lot of the configuration parameters above are also above and beyond locking and data integrity, they are for better windows compatibility and should be checked if applicable for your environment. i would also recommend to run on GPFS 3.5 TL3 or newer to get the proper GPFS level of fixes for this type of configurations. i would like to repeat that i don't write this email to encourage people to all go start installing/ configuring samba on top of GPFS as i pointed out that you are kind on your own unless you can read source code and/or have somebody who does and is able to help as soon as you run into a problem. the main point of sharing this information is to clarify a lot of misinformation out there and provide the people who already have setups that are incorrect the information to at least not run into data corruption issues do to wrong configuration. Sven From: Chair <ch...@gpfsug.org> To: gpfsug-discuss@gpfsug.org Date: 12/16/2013 03:31 AM Subject: Re: [gpfsug-discuss] GPFS and both Samba and NFS Sent by: gpfsug-discuss-boun...@gpfsug.org Allo Just jumping in here a minute: > It is unworthy of an IBM employee to spread such inaccurate misinformation. Whilst this may be inaccurate - I very, very, much doubt that IBM or their employees have a secret master plan to spread misinformation (!) In the spirit of this group, let's work together to technically look at such issues. Sven, if that is the case, perhaps you could crib the lines of code / show your methodology that supports your views / experience. Regards, Jez -- UG Chair On 16/12/13 11:21, Jonathan Buzzard wrote: > On Fri, 2013-12-13 at 10:14 -0800, Sven Oehme wrote: > > [SNIP] > >> the only way to get something working (don't get confused with >> officially Supported) is to recompile the CTDB src packages AND the >> Samba src packages on a node that has GPFS already installed. also the >> inclusion of CTDB into Samba will not address this, its just a more >> convenient packaging. >> >> Only if the build happens on such a node things like the vfs modules >> for GPFS are build and included in the package. >> > That is a factually inaccurate statement. There is nothing in CTDB that > is GPFS specific. Trust me I have examined the code closely to determine > if this is the case. So unless this has changed recently you are flat > out wrong. > > Consequently there is no requirement whatsoever to rebuild CTDB to get > the vfs_gpfs module. In addition there is also no requirement to > actually have GPFS installed to build the vfs_gpfs module either. What > you need to have is the GPFS GPL header files and nothing else. As it is > a loadable VFS module linking takes place at load time not compile time. > > It is unworthy of an IBM employee to spread such inaccurate > misinformation. > > [SNIP] > >> said all this the binaries alone are only part of the Solution, after >> you have the correct packages, you need to properly configuration the >> system and setting all the right options (on GPFS as well as on CTDB >> and smbd.conf), which unfortunate are very System configuration >> specific, as otherwise you still can end up with data corruption if >> not set right. > Indeed. However I know not only what those options are, but also what > they do despite IBM's refusal to tell us anything about them. > > I would also point out that there are sites that where running Samba on > top of GPFS for many years before IBM began offering their > SONAS/Storwize Unifed products. > > > JAB. > _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at gpfsug.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss