PS: I just tested between these 2: mount -t glusterfs -o negative-timeout=1,use-readdirp=no,log-level=WARNING,log-file=/var/log/glusterxxx.log 192.168.140.41:/www /var/www mount -t glusterfs -o use-readdirp=no,log-level=WARNING,log-file=/var/log/glusterxxx.log 192.168.140.41:/www /var/www So it means only 1 second negative timeout... In this particular test: ./smallfile_cli.py --top /var/www/test --host-set 192.168.140.41 --threads 8 --files 50000 --file-size 64 --record-size 64
The result is about 4 seconds with the negative timeout of 1 second defined and many many minutes without the negative timeout (I quit after 15 minutes of waiting) I will go over to some real world tests now to see how it performs there. Regards Jo -----Original message----- From:Jo Goossens <jo.gooss...@hosted-power.com> Sent:Tue 11-07-2017 18:23 Subject:Re: [Gluster-users] Gluster native mount is really slow compared to nfs To:Vijay Bellur <vbel...@redhat.com>; CC:gluster-users@gluster.org; Hello Vijay, What do you mean exactly? What info is missing? PS: I already found out that for this particular test all the difference is made by : negative-timeout=600 , when removing it, it's much much slower again. Regards Jo -----Original message----- From:Vijay Bellur <vbel...@redhat.com> Sent:Tue 11-07-2017 18:16 Subject:Re: [Gluster-users] Gluster native mount is really slow compared to nfs To:Jo Goossens <jo.gooss...@hosted-power.com>; CC:gluster-users@gluster.org; Joe Julian <j...@julianfamily.org>; On Tue, Jul 11, 2017 at 11:39 AM, Jo Goossens <jo.gooss...@hosted-power.com <mailto:jo.gooss...@hosted-power.com> > wrote: Hello Joe, I just did a mount like this (added the bold): mount -t glusterfs -o attribute-timeout=600,entry-timeout=600,negative-timeout=600,fopen-keep-cache,use-readdirp=no,log-level=WARNING,log-file=/var/log/glusterxxx.log 192.168.140.41:/www /var/www Results: root@app1:~/smallfile-master# ./smallfile_cli.py --top /var/www/test --host-set 192.168.140.41 --threads 8 --files 5000 --file-size 64 --record-size 64 smallfile version 3.0 hosts in test : ['192.168.140.41'] top test directory(s) : ['/var/www/test'] operation : cleanup files/thread : 5000 threads : 8 record size (KB, 0 = maximum) : 64 file size (KB) : 64 file size distribution : fixed files per dir : 100 dirs per dir : 10 threads share directories? : N filename prefix : filename suffix : hash file number into dir.? : N fsync after modify? : N pause between files (microsec) : 0 finish all requests? : Y stonewall? : Y measure response times? : N verify read? : Y verbose? : False log to stderr? : False ext.attr.size : 0 ext.attr.count : 0 permute host directories? : N remote program directory : /root/smallfile-master network thread sync. dir. : /var/www/test/network_shared starting all threads by creating starting gate file /var/www/test/network_shared/starting_gate.tmp host = 192.168.140.41,thr = 00,elapsed = 1.232004,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 01,elapsed = 1.148738,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 02,elapsed = 1.130913,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 03,elapsed = 1.183088,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 04,elapsed = 1.220752,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 05,elapsed = 1.228039,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 06,elapsed = 1.216787,files = 5000,records = 0,status = ok host = 192.168.140.41,thr = 07,elapsed = 1.229036,files = 5000,records = 0,status = ok total threads = 8 total files = 40000 100.00% of requested files processed, minimum is 70.00 1.232004 sec elapsed time 32467.428972 files/sec root@app1:~/smallfile-master# ./smallfile_cli.py --top /var/www/test --host-set 192.168.140.41 --threads 8 --files 50000 --file-size 64 --record-size 64 smallfile version 3.0 hosts in test : ['192.168.140.41'] top test directory(s) : ['/var/www/test'] operation : cleanup files/thread : 50000 threads : 8 record size (KB, 0 = maximum) : 64 file size (KB) : 64 file size distribution : fixed files per dir : 100 dirs per dir : 10 threads share directories? : N filename prefix : filename suffix : hash file number into dir.? : N fsync after modify? : N pause between files (microsec) : 0 finish all requests? : Y stonewall? : Y measure response times? : N verify read? : Y verbose? : False log to stderr? : False ext.attr.size : 0 ext.attr.count : 0 permute host directories? : N remote program directory : /root/smallfile-master network thread sync. dir. : /var/www/test/network_shared starting all threads by creating starting gate file /var/www/test/network_shared/starting_gate.tmp host = 192.168.140.41,thr = 00,elapsed = 4.242312,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 01,elapsed = 4.250831,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 02,elapsed = 3.771269,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 03,elapsed = 4.060653,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 04,elapsed = 3.880653,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 05,elapsed = 3.847107,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 06,elapsed = 3.895537,files = 50000,records = 0,status = ok host = 192.168.140.41,thr = 07,elapsed = 3.966394,files = 50000,records = 0,status = ok total threads = 8 total files = 400000 100.00% of requested files processed, minimum is 70.00 4.250831 sec elapsed time 94099.245073 files/sec root@app1:~/smallfile-master# As you can see it's now crazy fast, I think close to or faster than nfs !! What the hell!??! I'm so exited I already post. Any suggestions for those parameters? I will do additional testing over here , because this is ridiculous. That woud mean defaults or no good at all... Would it be possible to profile the client [1] with defaults and the set of options used now? That could help in understanding the performance delta better. Thanks, Vijay [1] https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Performance%20Testing/#client-side-profiling _______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users