On 05/17/17 02:02, Pranith Kumar Karampuri wrote:
On Tue, May 16, 2017 at 9:38 PM, Joe Julian <j...@julianfamily.org
<mailto:j...@julianfamily.org>> wrote:
On 04/13/17 23:50, Pranith Kumar Karampuri wrote:
On Sat, Apr 8, 2017 at 10:28 AM, Ravishankar N
<ravishan...@redhat.com <mailto:ravishan...@redhat.com>> wrote:
Hi Pat,
I'm assuming you are using gluster native (fuse mount). If it
helps, you could try mounting it via gluster NFS (gnfs) and
then see if there is an improvement in speed. Fuse mounts are
slower than gnfs mounts but you get the benefit of avoiding a
single point of failure. Unlike fuse mounts, if the gluster
node containing the gnfs server goes down, all mounts done
using that node will fail). For fuse mounts, you could try
tweaking the write-behind xlator settings to see if it helps.
See the performance.write-behind and
performance.write-behind-window-size options in `gluster
volume set help`. Of course, even for gnfs mounts, you can
achieve fail-over by using CTDB.
Ravi,
Do you have any data that suggests fuse mounts are slower
than gNFS servers?
Pat,
I see that I am late to the thread, but do you happen to
have "profile info" of the workload?
I have done actual testing. For directory ops, NFS is faster due
to the default cache settings in the kernel. For raw throughput,
or ops on an open file, fuse is faster.
I have yet to test this but I expect with the newer caching
features in 3.8+, even directory op performance should be similar
to nfs and more accurate.
We are actually comparing fuse+gluster and kernel NFS (n the same
brick. Did you get a chance to do this test at any point?
No, that's not comparing like to like and I've rarely had a use case to
which a single-store NFS was the answer.
You can follow
https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Monitoring%20Workload/
<https://gluster.readthedocs.io/en/latest/Administrator%20Guide/Monitoring%20Workload/>
to get the information.
Thanks,
Ravi
On 04/08/2017 12:07 AM, Pat Haley wrote:
Hi,
We noticed a dramatic slowness when writing to a gluster
disk when compared to writing to an NFS disk. Specifically
when using dd (data duplicator) to write a 4.3 GB file of zeros:
* on NFS disk (/home): 9.5 Gb/s
* on gluster disk (/gdata): 508 Mb/s
The gluser disk is 2 bricks joined together, no replication
or anything else. The hardware is (literally) the same:
* one server with 70 hard disks and a hardware RAID card.
* 4 disks in a RAID-6 group (the NFS disk)
* 32 disks in a RAID-6 group (the max allowed by the card,
/mnt/brick1)
* 32 disks in another RAID-6 group (/mnt/brick2)
* 2 hot spare
Some additional information and more tests results (after
changing the log level):
glusterfs 3.7.11 built on Apr 27 2016 14:09:22
CentOS release 6.8 (Final)
RAID bus controller: LSI Logic / Symbios Logic MegaRAID
SAS-3 3108 [Invader] (rev 02)
*Create the file to /gdata (gluster)*
[root@mseas-data2 gdata]# dd if=/dev/zero of=/gdata/zero1
bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 1.91876 s, *546 MB/s*
*Create the file to /home (ext4)*
[root@mseas-data2 gdata]# dd if=/dev/zero of=/home/zero1
bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 0.686021 s, *1.5 GB/s - *3
times as fast*
Copy from /gdata to /gdata (gluster to gluster)
*[root@mseas-data2 gdata]# dd if=/gdata/zero1 of=/gdata/zero2
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB) copied, 101.052 s, *10.4 MB/s* -
realllyyy slooowww
*Copy from /gdata to /gdata* *2nd time *(gluster to gluster)**
[root@mseas-data2 gdata]# dd if=/gdata/zero1 of=/gdata/zero2
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB) copied, 92.4904 s, *11.3 MB/s* -
realllyyy slooowww again
*Copy from /home to /home (ext4 to ext4)*
[root@mseas-data2 gdata]# dd if=/home/zero1 of=/home/zero2
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB) copied, 3.53263 s, *297 MB/s *30
times as fast
*Copy from /home to /home (ext4 to ext4)*
[root@mseas-data2 gdata]# dd if=/home/zero1 of=/home/zero3
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB) copied, 4.1737 s, *251 MB/s* - 30
times as fast
As a test, can we copy data directly to the xfs mountpoint
(/mnt/brick1) and bypass gluster?
Any help you could give us would be appreciated.
Thanks
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Pat Haley Email:pha...@mit.edu
<mailto:pha...@mit.edu>
Center for Ocean Engineering Phone: (617) 253-6824
Dept. of Mechanical Engineering Fax: (617) 253-8125
MIT, Room 5-213http://web.mit.edu/phaley/www/
77 Massachusetts Avenue
Cambridge, MA 02139-4301
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org <mailto:Gluster-users@gluster.org>
http://lists.gluster.org/mailman/listinfo/gluster-users
<http://lists.gluster.org/mailman/listinfo/gluster-users>
_______________________________________________ Gluster-users
mailing list Gluster-users@gluster.org
<mailto:Gluster-users@gluster.org>
http://lists.gluster.org/mailman/listinfo/gluster-users
<http://lists.gluster.org/mailman/listinfo/gluster-users>
--
Pranith
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org <mailto:Gluster-users@gluster.org>
http://lists.gluster.org/mailman/listinfo/gluster-users
<http://lists.gluster.org/mailman/listinfo/gluster-users>
_______________________________________________ Gluster-users
mailing list Gluster-users@gluster.org
<mailto:Gluster-users@gluster.org>
http://lists.gluster.org/mailman/listinfo/gluster-users
<http://lists.gluster.org/mailman/listinfo/gluster-users>
--
Pranith
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users
