On Thu, 2017-02-09 at 16:18 +0000, Gary Lloyd wrote: > Was just reading the small file section of the 3.9 release notes: > > http://blog.gluster.org/2016/11/announcing-gluster-3-9/ > > Setting these options does seem to increase transfer speeds on small files by > quite alot: > # gluster volume set <volname> features.cache-invalidation on > # gluster volume set <volname> features.cache-invalidation-timeout 600 > # gluster volume set <volname> performance.stat-prefetch on #This one > seemed to have the > biggest impact in small file performance for me > # gluster volume set <volname> performance.cache-invalidation on > # gluster volume set <volname> performance.md-cache-timeout 600 > > Setting # gluster volume set <volname> performance.cache-samba-metadata on # > Only for SMB > access. Results in my client to keep losing the state of the server and the > shares often disappear > / become inaccessible and I can only get them back if I logon / logoff the > machine, this is with > distro Samba 4.4.4. >
This is something which needs to be analyzed further.. We would need more info on volume config, glusterfs client logs from Samba(/var/log/samba/glusterfs- <volname>.<IP/hostname>.log) and your smb.conf? > Has anyone here had the same issue, does the version of samba need to be > newer to support the > feature ? > To my knowledge, version of Samba should not be a problem here. At least I am not aware of any such issues with v4.4.6. > Thanks > > Gary Lloyd > ________________________________________________ > I.T. Systems:Keele University > Finance & IT Directorate > Keele:Staffs:IC1 Building:ST5 5NB:UK > +44 1782 733063 > ________________________________________________ > > On 8 February 2017 at 11:49, Дмитрий Глушенок <gl...@jet.msk.su> wrote: > > For _every_ file copied samba performs readdir() to get all entries of the > > destination folder. > > Then the list is searched for filename (to prevent name collisions as SMB > > shares are not case > > sensitive). More files in folder, more time it takes to perform readdir(). > > It is a lot worse for > > Gluster because single folder contents distributed among many servers and > > Gluster has to join > > many directory listings (requested via network) to form one and return it > > to caller. > > > > Rsync does not perform readdir(), it just checks file existence with stat() > > IIRC. And as modern > > Gluster versions has default setting to check for file only at its > > destination (when volume is > > balanced) - the check performs relatively fast. > > > > You can hack samba to prevent such checks if your goal is to get files > > copied not so slow (as > > you sure the files you are copying are not exists at destination). But try > > to perform 'ls -l' on > > _not_ cached folder with thousands of files - it will take tens of seconds. > > This is time your > > users will waste browsing shares. > > > > > 8 февр. 2017 г., в 13:17, Gary Lloyd <g.ll...@keele.ac.uk> написал(а): > > > > > > Thanks for the reply > > > > > > I've just done a bit more testing. If I use rsync from a gluster client > > > to copy the same files > > > to the mount point it only takes a couple of minutes. > > > For some reason it's very slow on samba though (version 4.4.4). > > > > > > I have tried various samba tweaks / settings and have yet to get > > > acceptable write speed on > > > small files. > > > > > > > > > Gary Lloyd > > > ________________________________________________ > > > I.T. Systems:Keele University > > > Finance & IT Directorate > > > Keele:Staffs:IC1 Building:ST5 5NB:UK > > > +44 1782 733063 > > > ________________________________________________ > > > > > > On 8 February 2017 at 10:05, Дмитрий Глушенок <gl...@jet.msk.su> wrote: > > > > Hi, > > > > > > > > There is a number of tweaks/hacks to make it better, but IMHO overall > > > > performance with small > > > > files is still unacceptable for such folders with thousands of entries. > > > > > > > > If your shares are not too large to be placed on single filesystem and > > > > you still want to use > > > > Gluster - it is possible to run VM on top of Gluster. Inside that VM > > > > you can create ZFS/NTFS > > > > to be shared. > > > > > > > > > 8 февр. 2017 г., в 12:10, Gary Lloyd <g.ll...@keele.ac.uk> написал(а): > > > > > > > > > > Hi > > > > > > > > > > I am currently testing gluster 3.9 replicated/distrbuted on centos > > > > > 7.3 with samba/ctdb. > > > > > I have been able to get it all up and running, but writing small > > > > > files is really slow. > > > > > > > > > > If I copy large files from gluster backed samba I get almost wire > > > > > speed (We only have 1Gb > > > > > at the moment). I get around half that speed if I copy large files to > > > > > the gluster backed > > > > > samba system, which I am guessing is due to it being replicated (This > > > > > is acceptable). > > > > > > > > > > Small file write performance seems really poor for us though: > > > > > As an example I have an eclipse IDE workspace folder that is 6MB in > > > > > size that has around > > > > > 6000 files in it. A lot of these files are <1k in size. > > > > > > > > > > If I copy this up to gluster backed samba it takes almost one hour to > > > > > get there. > > > > > With our basic samba deployment it only takes about 5 minutes. > > > > > > > > > > Both systems reside on the same disks/SAN. > > > > > > > > > > > > > > > I was hoping that we would be able to move away from using a > > > > > proprietary SAN to house our > > > > > network shares and use gluster instead. > > > > > > > > > > Does anyone have any suggestions of anything I could tweak to make it > > > > > better ? > > > > > > > > > > Many Thanks > > > > > > > > > > > > > > > Gary Lloyd > > > > > ________________________________________________ > > > > > I.T. Systems:Keele University > > > > > Finance & IT Directorate > > > > > Keele:Staffs:IC1 Building:ST5 5NB:UK > > > > > ________________________________________________ > > > > > _______________________________________________ > > > > > Gluster-users mailing list > > > > > Gluster-users@gluster.org > > > > > http://lists.gluster.org/mailman/listinfo/gluster-users > > > > > > > > -- > > > > Dmitry Glushenok > > > > Jet Infosystems > > > > > > > > > > > > -- > > Dmitry Glushenok > > Jet Infosystems > > > > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
