Re: [Gluster-users] write-behind / write-back caching (asked again - nobody can help?)
On Tuesday 30 August 2011 08:36 PM, Christian wrote: Hello to all, I'm currently testing glusterfs ( version 3.1.4, 3.1.6, 3.2.2, 3.2.3 and 3.3beta ) for the following situation / behavior: I want to create a replicated storage via internet / wan with two storage nodes. The first node is located in office A and the other one is in office B. If I try to write a file to the mounted glusterfs (mounted via glusterfs or nfs), the write performance is as poor as the upload speed (~ 1 mbit - adjusted manually using tc). I tested several cache-options (see below) with the following effect: The copy process of a file is done very fast (~40 mbyte/sec), but the application (rsync, mc copy, cp) is waiting at 100% for the final sync of the storage. The process is not finished before glusterfs has written the file to the 2nd node. With a replicate config, this is what you can expect. The increased write-behind cache is holding your file giving you the boosted throughput, but on close, it will have to sync the data to both nodes. The behavior I am looking for is to store files locally first and then sync the content to the second node in the background. Is there a way for this? I think you are better off using geo-replication rather than the traditional replicate configuration for the above requirement of yours. The following link should help you configure geo-rep - http://www.gluster.com/community/documentation/index.php/Gluster_3.2_Filesystem_Administration_Guide Look for the geo-replication section there. It also gives you a comparison of replicated volumes vs geo-replication. HTH, Pavan ** volume info: Volume Name: gl5 Type: Replicate Status: Started Number of Bricks: 2 Transport-type: tcp Bricks: Brick1: 192.168.42.130:/gl5 Brick2: 192.168.42.7:/gl5 Options Reconfigured: nfs.disable: off nfs.trusted-sync: on nfs.trusted-write: on performance.flush-behind: off performance.write-behind-window-size: 200MB performance.cache-max-file-size: 200MB ** tested mount options: mount.nfs 127.0.0.1:gl5 /mnt/gluster/ -v -o mountproto=tcp -o async mount -t glusterfs 127.0.0.1:gl5 /mnt/gluster -o async Thanks a lot, Christian ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] add bricks on distributed replicated volume failed
Hi, I'm using gluster 3.2.2 on 10 nodes. Each node have 2x2 TB disk for gluster. I manage to configure a distributed and replicated volume on 4 nodes : [root@node07 ~]# gluster volume info cluster Volume Name: cluster Type: Distributed-Replicate Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: node09:/gluster1 Brick2: node10:/gluster1 Brick3: node09:/gluster2 Brick4: node10:/gluster2 Brick5: node07:/gluster1 Brick6: node08:/gluster1 Brick7: node07:/gluster2 Brick8: node08:/gluster2 But I can't add new nodes to this volume [root@node07 ~]# gluster peer status Number of Peers: 5 Hostname: node10 Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0 State: Peer in Cluster (Connected) Hostname: node08 Uuid: dd114546-5b94-4a62-9301-260703bf5707 State: Peer in Cluster (Connected) Hostname: node09 Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04 State: Peer in Cluster (Connected) Hostname: node06 Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a State: Peer in Cluster (Connected) Hostname: node05 Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b State: Peer in Cluster (Connected) [root@node07 ~]# gluster volume add-brick cluster node05:/gluster1 node06:/gluster1 node05:/gluster2 node06:/gluster2 Operation failed on node05 I try to detach nodes 5 and 6, restart glusterd do the probe and the add-brick but still nothing ... Did some one have any idea to fix this ? Thanks in advance, Laurent. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] add bricks on distributed replicated volume failed
Hi Laurent. Since your configuration specifies replication, you must add bricks in the same number as your number of replicas. For instance - if you have 2 replicas (most normal case), you would need to do something like this: gluster volume add-brick cluster node05:/gluster1 node06:/gluster1 James Burnash Unix Engineer Knight Capital Group -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Laurent DOUCHY Sent: Wednesday, August 31, 2011 12:49 PM To: gluster-users@gluster.org Subject: [Gluster-users] add bricks on distributed replicated volume failed Hi, I'm using gluster 3.2.2 on 10 nodes. Each node have 2x2 TB disk for gluster. I manage to configure a distributed and replicated volume on 4 nodes : [root@node07 ~]# gluster volume info cluster Volume Name: cluster Type: Distributed-Replicate Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: node09:/gluster1 Brick2: node10:/gluster1 Brick3: node09:/gluster2 Brick4: node10:/gluster2 Brick5: node07:/gluster1 Brick6: node08:/gluster1 Brick7: node07:/gluster2 Brick8: node08:/gluster2 But I can't add new nodes to this volume [root@node07 ~]# gluster peer status Number of Peers: 5 Hostname: node10 Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0 State: Peer in Cluster (Connected) Hostname: node08 Uuid: dd114546-5b94-4a62-9301-260703bf5707 State: Peer in Cluster (Connected) Hostname: node09 Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04 State: Peer in Cluster (Connected) Hostname: node06 Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a State: Peer in Cluster (Connected) Hostname: node05 Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b State: Peer in Cluster (Connected) [root@node07 ~]# gluster volume add-brick cluster node05:/gluster1 node06:/gluster1 node05:/gluster2 node06:/gluster2 Operation failed on node05 I try to detach nodes 5 and 6, restart glusterd do the probe and the add-brick but still nothing ... Did some one have any idea to fix this ? Thanks in advance, Laurent. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] add bricks on distributed replicated volume failed
Hi, I try to add 2 bricks or 4 bricks for the same effect. I try to reinstall gluster without success. Cheers, Laurent DOUCHY. System Administrator ISDC Data Centre for Astrophysics On 8/31/11 8:07 PM, Burnash, James wrote: Hi Laurent. Since your configuration specifies replication, you must add bricks in the same number as your number of replicas. For instance - if you have 2 replicas (most normal case), you would need to do something like this: gluster volume add-brick cluster node05:/gluster1 node06:/gluster1 James Burnash Unix Engineer Knight Capital Group -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Laurent DOUCHY Sent: Wednesday, August 31, 2011 12:49 PM To: gluster-users@gluster.org Subject: [Gluster-users] add bricks on distributed replicated volume failed Hi, I'm using gluster 3.2.2 on 10 nodes. Each node have 2x2 TB disk for gluster. I manage to configure a distributed and replicated volume on 4 nodes : [root@node07 ~]# gluster volume info cluster Volume Name: cluster Type: Distributed-Replicate Status: Started Number of Bricks: 4 x 2 = 8 Transport-type: tcp Bricks: Brick1: node09:/gluster1 Brick2: node10:/gluster1 Brick3: node09:/gluster2 Brick4: node10:/gluster2 Brick5: node07:/gluster1 Brick6: node08:/gluster1 Brick7: node07:/gluster2 Brick8: node08:/gluster2 But I can't add new nodes to this volume [root@node07 ~]# gluster peer status Number of Peers: 5 Hostname: node10 Uuid: 212ce5a0-de51-4a98-9262-ae071c2d63a0 State: Peer in Cluster (Connected) Hostname: node08 Uuid: dd114546-5b94-4a62-9301-260703bf5707 State: Peer in Cluster (Connected) Hostname: node09 Uuid: f73fee83-8d47-4f07-bfac-b8a8592eff04 State: Peer in Cluster (Connected) Hostname: node06 Uuid: 3142fb9a-0a6b-46ec-9262-ede95e8f798a State: Peer in Cluster (Connected) Hostname: node05 Uuid: 13ffcf87-6e8d-4c6b-814a-cbc14d15d88b State: Peer in Cluster (Connected) [root@node07 ~]# gluster volume add-brick cluster node05:/gluster1 node06:/gluster1 node05:/gluster2 node06:/gluster2 Operation failed on node05 I try to detach nodes 5 and 6, restart glusterd do the probe and the add-brick but still nothing ... Did some one have any idea to fix this ? Thanks in advance, Laurent. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
