Re: [Gluster-users] Should the gluster host access files via gluster?
Yes. All access to gluster data must be over the gluster mount. Regards, Tejas. - Original Message - From: "Morgan O'Neal" To: gluster-users@gluster.org Sent: Thursday, July 29, 2010 5:12:54 AM Subject: [Gluster-users] Should the gluster host access files via gluster? I have a gluster host server with a few clients connected via a lan network. When I make a lot of changes to the files on the host directly the clients cant access them sometimes. Should the host machine access the files via a gluster mount to fix this problem? -- Morgan O'Neal mon...@alpineinternet.com http://twitter.com/morganoneal Schedule me - http://tungle.me/morganoneal 901-4MONEAL (901-466-6325) ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] Should the gluster host access files via gluster?
I have a gluster host server with a few clients connected via a lan network. When I make a lot of changes to the files on the host directly the clients cant access them sometimes. Should the host machine access the files via a gluster mount to fix this problem? -- Morgan O'Neal mon...@alpineinternet.com http://twitter.com/morganoneal Schedule me - http://tungle.me/morganoneal 901-4MONEAL (901-466-6325) ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Mirror volumes with odd number of servers
seems like there is some problem in sending attachments. Please find the configuration below, server.vol: === # server1 spec file ### Export volume "brick" with the contents of "/home/export" directory. volume posix1 type storage/posix# POSIX FS translator option directory /data/export/1# Export this directory end-volume ### Add POSIX record locking support to the storage brick volume brick1 type features/posix-locks option mandatory on # enables mandatory locking on all files subvolumes posix1 end-volume ### Add network serving capability to above brick. volume server-1 type protocol/server option transport-type tcp # For TCP/IP transport option transport.socket.listen-port 6996 # Default is 6996 # option client-volume-filename /etc/glusterfs/glusterfs-client.vol subvolumes brick1 option auth.addr.brick1.allow * # access to "brick" volume end-volume #= # server2 spec file volume posix2 type storage/posix# POSIX FS translator option directory /data/export/2# Export this directory end-volume ### Add POSIX record locking support to the storage brick volume brick2 type features/posix-locks option mandatory on # enables mandatory locking on all files subvolumes posix2 end-volume ### Add network serving capability to above brick. volume server-2 type protocol/server option transport-type tcp # For TCP/IP transport option transport.socket.listen-port 6997 # Default is 6996 subvolumes brick2 option auth.addr.brick2.allow * # Allow access to "brick" volume end-volume #= # server3 spec file volume posix3 type storage/posix# POSIX FS translator option directory /data/export/3# Export this directory end-volume ### Add POSIX record locking support to the storage brick volume brick3 type features/posix-locks option mandatory on # enables mandatory locking on all files subvolumes posix3 end-volume ### Add network serving capability to above brick. volume server-3 type protocol/server option transport-type tcp # For TCP/IP transport option transport.socket.listen-port 6998 # Default is 6996 subvolumes brick3 option auth.addr.brick3.allow * # access to "brick" volume end-volume #= # server4 spec file ### Export volume "brick" with the contents of "/home/export" directory. volume posix4 type storage/posix# POSIX FS translator option directory /data/export/4# Export this directory end-volume ### Add POSIX record locking support to the storage brick volume brick4 type features/posix-locks opti
Re: [Gluster-users] Mirror volumes with odd number of servers
previously sent volume specification files were of a replicated setup of 3 servers. The ones attached with this mail are examples of a distributed-replicated setup of 5 servers. regards, - Original Message - From: "Raghavendra G" To: "Gluster General Discussion List" Sent: Thursday, July 29, 2010 9:28:33 AM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Hi James, Please find example volume specification files attached with this mail. regards, - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 8:41:08 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Thanks Tejas. If an actual example of the glusterfs.vol was available showing this setup was available, that would be a valuable sanity check against what I will build. James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Tejas N. Bhise Sent: Wednesday, July 28, 2010 12:26 PM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers James, You can do that, but you will have to hand craft the volume file. volgen looks for even numbers as you have already noticed. With hand crafting of volume files, you can even have a replicate distribute .. so e.g. 2 copies of file, and each replicate node of the graph can have a differing number of distribute servers under it. something like this can be done to have a replica count of two with 5 servers... | R1 |--R1D1 mount--- |---|--R1D2 | | R2 |---R2D1 |---|---R2D2 |---R2D3 The design of translators is so modular that they can be used in any combination. This however used to lead to confusion and hence we developed volgen that produced easy to use and default best-fit configuration. >From the perspective of "official" support, we typically only support configs >that volgen produces. For other configs, we do fix the bugs, but how fast >depends on the config and translator seeing the bug. Let me know if you have more questions about this. Regards, Tejas Bhise. - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 6:41:02 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Carl - could you possibly provide an example of a configuration using an odd number of servers? glusterfs-volgen is unhappy when you don't give an even number. Thanks! James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Craig Carl Sent: Wednesday, July 28, 2010 12:16 AM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Brock - It is completely possible using Gluster File System. We haven't exposed that option via the Gluster Storage Platform GUI yet. Craig -- Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "brock brown" To: gluster-users@gluster.org Sent: Tuesday, July 27, 2010 12:51:19 PM Subject: [Gluster-users] Mirror volumes with odd number of servers All the documentation refers to using multiples of 2 when setting up mirrored volumes. Also the Gluster Storage Platform will not allow it. Is this impossible and if so why? Thanks, Brock _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NO
Re: [Gluster-users] Mirror volumes with odd number of servers
Hi James, Please find example volume specification files attached with this mail. regards, - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 8:41:08 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Thanks Tejas. If an actual example of the glusterfs.vol was available showing this setup was available, that would be a valuable sanity check against what I will build. James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Tejas N. Bhise Sent: Wednesday, July 28, 2010 12:26 PM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers James, You can do that, but you will have to hand craft the volume file. volgen looks for even numbers as you have already noticed. With hand crafting of volume files, you can even have a replicate distribute .. so e.g. 2 copies of file, and each replicate node of the graph can have a differing number of distribute servers under it. something like this can be done to have a replica count of two with 5 servers... | R1 |--R1D1 mount--- |---|--R1D2 | | R2 |---R2D1 |---|---R2D2 |---R2D3 The design of translators is so modular that they can be used in any combination. This however used to lead to confusion and hence we developed volgen that produced easy to use and default best-fit configuration. >From the perspective of "official" support, we typically only support configs >that volgen produces. For other configs, we do fix the bugs, but how fast >depends on the config and translator seeing the bug. Let me know if you have more questions about this. Regards, Tejas Bhise. - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 6:41:02 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Carl - could you possibly provide an example of a configuration using an odd number of servers? glusterfs-volgen is unhappy when you don't give an even number. Thanks! James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Craig Carl Sent: Wednesday, July 28, 2010 12:16 AM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Brock - It is completely possible using Gluster File System. We haven't exposed that option via the Gluster Storage Platform GUI yet. Craig -- Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "brock brown" To: gluster-users@gluster.org Sent: Tuesday, July 27, 2010 12:51:19 PM Subject: [Gluster-users] Mirror volumes with odd number of servers All the documentation refers to using multiples of 2 when setting up mirrored volumes. Also the Gluster Storage Platform will not allow it. Is this impossible and if so why? Thanks, Brock _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users
Re: [Gluster-users] Adding volumes - How redistribute existing data
Hi Michael, Sorry for the confusion on 'scale-n-defrag.sh' script. To make sure the script does defrag, you need to have two options set in distribute volume. 'option unhashed-sticky-bit on' 'option lookup-unhashed on' Without these options it will not move the data files in backend. If you don't want to bring down the current mount point to run the defrag, you can have another mount point with changed volume file, and run defrag over it. Let us know if you have any more questions regarding defrag process. Regards, Amar On Wed, Jul 28, 2010 at 9:37 PM, Moore, Michael wrote: > Hi, > > I am trying to add several new backend volumes to an existing GlusterFS > setup. I am running GlusterFS 3.0.4 using the distribute translator. I've > tried running the "scale-n-defrag.sh" script to redistribute the data across > the additional volumes, but after running for a significant time, nothing > was significantly redistributed. What are the proper steps to do to > redistribute the data? Do I need to clean up the links GlusterFS makes on > the backends before I run scale-n-defrag? > > I am running GlusterFS 3.0.4 on top of CentOS 5.4. This is not running > GlusterSP. > ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] glusterfsd crashes when using quota translator
Hi Peter, We opened a bug for this issue - http://bugs.gluster.com/cgi-bin/bugzilla3/show_bug.cgi?id=1243 Regards, Tejas. - Original Message - From: "Tejas N. Bhise" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 9:45:40 PM Subject: Re: [Gluster-users] glusterfsd crashes when using quota translator Hi Peter, Please go ahead and open a bug for this. I will have someone take a look. The volume quota translator is not officially supported, though some users use it with success. We do however try to actively fix defects in it even though not officially supported. Does it crash every time you create a file after a restart ? Regards, Tejas. - Original Message - From: "Peter Mueller" To: gluster-users@gluster.org Sent: Wednesday, July 28, 2010 8:39:44 PM Subject: [Gluster-users] glusterfsd crashes when using quota translator Hi, i tried to use the quota translator to limit the storage for a specific volume. I can mount the volume and see the limit with df. But as soon as i try to create a file on my mounted volume, the glusterfsd crashes. Configuration and Logs: Version : glusterfs 3.0.0 built on Jul 23 2010 16:45:59 git: 2.0.1-886-g8379edd Starting Time: 2010-07-25 19:27:56 Command line : /sbin/glusterfsd -f /etc/glusterfs/glusterfsd.vol PID : 27113 System name : Linux Nodename : inst-209.install.de Kernel Release : 2.6.18-194.el5 Hardware Identifier: x86_64 Given volfile: +--+ 1: 2: volume v8705-posix 3: type storage/posix 4: option directory /data/v8705 5: end-volume 6: 7: volume v8705-quota 8: type features/quota 9: option disk-usage-limit 100GB 10: subvolumes v8705-posix 11: end-volume 12: 13: volume v8705-brick 14: type features/locks 15: subvolumes v8705-quota 16: end-volume 17: 18: volume v8704-posix 19: type storage/posix 20: option directory /data/v8704 21: end-volume 22: 23: volume v8704-quota 24: type features/quota 25: option disk-usage-limit 100GB 26: subvolumes v8704-posix 27: end-volume 28: 29: volume v8704-brick 30: type features/locks 31: subvolumes v8704-quota 32: end-volume 33: 34: 35: volume server 36: type protocol/server 37: option transport-type tcp 38: option listen-port 6996 39: option bind-address 10.0.1.122 40: 41: option auth.login.v8705-brick.allow v8705 42: option auth.login.v8705.password 43: 44: option auth.login.v8704-brick.allow v8704 45: option auth.login.v8704.password 46: 47: subvolumes v8705-brick v8704-brick 48: 49: end-volume +--+ [2010-07-25 19:27:56] W [xlator.c:655:validate_xlator_volume_options] server: option 'bind-address' is deprecated, preferred is 'transport.socket.bind-address', continuing with correction [2010-07-25 19:27:56] W [xlator.c:655:validate_xlator_volume_options] server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2010-07-25 19:27:56] N [glusterfsd.c:1361:main] glusterfs: Successfully started [2010-07-25 19:27:58] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.145:1023 [2010-07-25 19:27:58] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.145:1022 [2010-07-25 19:28:34] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.122:1022 [2010-07-25 19:28:34] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.122:1023 pending frames: frame : type(1) op(UNLINK) frame : type(1) op(UNLINK) patchset: 2.0.1-886-g8379edd signal received: 11 time of crash: 2010-07-25 19:28:49 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.0.0 /lib64/libc.so.6[0x3b870302d0] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink_cbk+0xd5)[0x2ae3d57b4235] /usr/local/lib/libglusterfs.so.0[0x2ae3d46c9604] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink_cbk+0x95)[0x2ae3d538a955] /usr/local/lib/glusterfs/3.0.0/xlator/storage/posix.so(posix_unlink+0x1f3)[0x2ae3d517e613] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink_stat_cbk+0xd4)[0x2ae3d5389234] /usr/local/lib/glusterfs/3.0.0/xlator/storage/posix.so(posix_stat+0x137)[0x2ae3d517f9b7] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink+0xf1)[0x2ae3d538a501] /usr/local/lib/libglusterfs.so.0(default_unlink+0xcb)[0x2ae3d46c954b] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink_resume+0xd1)[0x2ae3d57b45a1] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_done+0x30)[0x2ae3d57b4e20] /usr/local/lib/
Re: [Gluster-users] Inconsistent volume
On 07/26/2010 04:37 PM, Andy Pace wrote: I too would like to know how to "sync up" a replicated pair of bricks. Right now I've got a slight difference between the 2... Scale-n-defrag.sh didn't do much either. Looking forward to some help :) 13182120616 139057220 12362648984 2% /export Vs 13181705324 139057208 12362233500 2% /export Granted, it's a very small amount (and the total availalble is slightly different), but the amount used should be the same, no? -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Steve Wilson Sent: Monday, July 26, 2010 3:35 PM To: gluster-users@gluster.org Subject: [Gluster-users] Inconsistent volume I have a volume that is distributed and replicated. While deleting a directory structure on the mounted volume, I also restarted the GlusterFS daemon on one of the replicated servers. After the "rm -rf" command completed, it complained that it couldn't delete a directory because it wasn't empty. But from the perspective of the mounted volume it appeared empty. Looking at the individual bricks, though, I could see that there were files remaining in this directory. My question: what is the proper way to correct this problem and bring the volume back to a consistent state? I've tried using the "ls -alR" command to force a self-heal but for some reason this always causes the volume to become unresponsive from any client after 10 minutes or so. Some clients/servers are running version 3.0.4 while the others are running 3.0.5. Thanks! Steve -- Steven M. Wilson, Systems and Network Manager Markey Center for Structural Biology Purdue University (765) 496-1946 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users All, Does anyone have any hints on how to proceed on this? I do now have all three servers running at version 3.0.5. Running a "du -bs" on each of the bricks shows the following: Replication pair #1: 693999264161/gluster/jiang-scratch-1a 693998682461/gluster/jiang-scratch-1b Replication pair #2: 231056270208/gluster/jiang-scratch-2a 231049560706/gluster/jiang-scratch-2b Replication pair #3: 228227559462/gluster/jiang-scratch-3a 228839698590/gluster/jiang-scratch-3b Is there something I can do manually (and safely) that will bring my volume back to a consistent state? Thanks, Steve -- Steven M. Wilson, Systems and Network Manager Markey Center for Structural Biology Purdue University (765) 496-1946 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Mirror volumes with odd number of servers
ok, will request someone from my team to send it .. but you will need to remember the caveat about the inadequate testing and non availability of quick support for the non-standard volume configs :-) - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 10:11:08 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Thanks Tejas. If an actual example of the glusterfs.vol was available showing this setup was available, that would be a valuable sanity check against what I will build. James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Tejas N. Bhise Sent: Wednesday, July 28, 2010 12:26 PM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers James, You can do that, but you will have to hand craft the volume file. volgen looks for even numbers as you have already noticed. With hand crafting of volume files, you can even have a replicate distribute .. so e.g. 2 copies of file, and each replicate node of the graph can have a differing number of distribute servers under it. something like this can be done to have a replica count of two with 5 servers... | R1 |--R1D1 mount--- |---|--R1D2 | | R2 |---R2D1 |---|---R2D2 |---R2D3 The design of translators is so modular that they can be used in any combination. This however used to lead to confusion and hence we developed volgen that produced easy to use and default best-fit configuration. >From the perspective of "official" support, we typically only support configs >that volgen produces. For other configs, we do fix the bugs, but how fast >depends on the config and translator seeing the bug. Let me know if you have more questions about this. Regards, Tejas Bhise. - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 6:41:02 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Carl - could you possibly provide an example of a configuration using an odd number of servers? glusterfs-volgen is unhappy when you don't give an even number. Thanks! James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Craig Carl Sent: Wednesday, July 28, 2010 12:16 AM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Brock - It is completely possible using Gluster File System. We haven't exposed that option via the Gluster Storage Platform GUI yet. Craig -- Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "brock brown" To: gluster-users@gluster.org Sent: Tuesday, July 27, 2010 12:51:19 PM Subject: [Gluster-users] Mirror volumes with odd number of servers All the documentation refers to using multiples of 2 when setting up mirrored volumes. Also the Gluster Storage Platform will not allow it. Is this impossible and if so why? Thanks, Brock _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://g
Re: [Gluster-users] Mirror volumes with odd number of servers
Thanks Tejas. If an actual example of the glusterfs.vol was available showing this setup was available, that would be a valuable sanity check against what I will build. James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Tejas N. Bhise Sent: Wednesday, July 28, 2010 12:26 PM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers James, You can do that, but you will have to hand craft the volume file. volgen looks for even numbers as you have already noticed. With hand crafting of volume files, you can even have a replicate distribute .. so e.g. 2 copies of file, and each replicate node of the graph can have a differing number of distribute servers under it. something like this can be done to have a replica count of two with 5 servers... | R1 |--R1D1 mount--- |---|--R1D2 | | R2 |---R2D1 |---|---R2D2 |---R2D3 The design of translators is so modular that they can be used in any combination. This however used to lead to confusion and hence we developed volgen that produced easy to use and default best-fit configuration. >From the perspective of "official" support, we typically only support configs >that volgen produces. For other configs, we do fix the bugs, but how fast >depends on the config and translator seeing the bug. Let me know if you have more questions about this. Regards, Tejas Bhise. - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 6:41:02 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Carl - could you possibly provide an example of a configuration using an odd number of servers? glusterfs-volgen is unhappy when you don't give an even number. Thanks! James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Craig Carl Sent: Wednesday, July 28, 2010 12:16 AM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Brock - It is completely possible using Gluster File System. We haven't exposed that option via the Gluster Storage Platform GUI yet. Craig -- Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "brock brown" To: gluster-users@gluster.org Sent: Tuesday, July 27, 2010 12:51:19 PM Subject: [Gluster-users] Mirror volumes with odd number of servers All the documentation refers to using multiples of 2 when setting up mirrored volumes. Also the Gluster Storage Platform will not allow it. Is this impossible and if so why? Thanks, Brock _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Mirror volumes with odd number of servers
James, You can do that, but you will have to hand craft the volume file. volgen looks for even numbers as you have already noticed. With hand crafting of volume files, you can even have a replicate distribute .. so e.g. 2 copies of file, and each replicate node of the graph can have a differing number of distribute servers under it. something like this can be done to have a replica count of two with 5 servers... | R1 |--R1D1 mount--- |---|--R1D2 | | R2 |---R2D1 |---|---R2D2 |---R2D3 The design of translators is so modular that they can be used in any combination. This however used to lead to confusion and hence we developed volgen that produced easy to use and default best-fit configuration. >From the perspective of "official" support, we typically only support configs >that volgen produces. For other configs, we do fix the bugs, but how fast >depends on the config and translator seeing the bug. Let me know if you have more questions about this. Regards, Tejas Bhise. - Original Message - From: "James Burnash" To: "Gluster General Discussion List" Sent: Wednesday, July 28, 2010 6:41:02 PM Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Carl - could you possibly provide an example of a configuration using an odd number of servers? glusterfs-volgen is unhappy when you don't give an even number. Thanks! James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Craig Carl Sent: Wednesday, July 28, 2010 12:16 AM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Brock - It is completely possible using Gluster File System. We haven't exposed that option via the Gluster Storage Platform GUI yet. Craig -- Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "brock brown" To: gluster-users@gluster.org Sent: Tuesday, July 27, 2010 12:51:19 PM Subject: [Gluster-users] Mirror volumes with odd number of servers All the documentation refers to using multiples of 2 when setting up mirrored volumes. Also the Gluster Storage Platform will not allow it. Is this impossible and if so why? Thanks, Brock _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] glusterfsd crashes when using quota translator
Hi Peter, Please go ahead and open a bug for this. I will have someone take a look. The volume quota translator is not officially supported, though some users use it with success. We do however try to actively fix defects in it even though not officially supported. Does it crash every time you create a file after a restart ? Regards, Tejas. - Original Message - From: "Peter Mueller" To: gluster-users@gluster.org Sent: Wednesday, July 28, 2010 8:39:44 PM Subject: [Gluster-users] glusterfsd crashes when using quota translator Hi, i tried to use the quota translator to limit the storage for a specific volume. I can mount the volume and see the limit with df. But as soon as i try to create a file on my mounted volume, the glusterfsd crashes. Configuration and Logs: Version : glusterfs 3.0.0 built on Jul 23 2010 16:45:59 git: 2.0.1-886-g8379edd Starting Time: 2010-07-25 19:27:56 Command line : /sbin/glusterfsd -f /etc/glusterfs/glusterfsd.vol PID : 27113 System name : Linux Nodename : inst-209.install.de Kernel Release : 2.6.18-194.el5 Hardware Identifier: x86_64 Given volfile: +--+ 1: 2: volume v8705-posix 3: type storage/posix 4: option directory /data/v8705 5: end-volume 6: 7: volume v8705-quota 8: type features/quota 9: option disk-usage-limit 100GB 10: subvolumes v8705-posix 11: end-volume 12: 13: volume v8705-brick 14: type features/locks 15: subvolumes v8705-quota 16: end-volume 17: 18: volume v8704-posix 19: type storage/posix 20: option directory /data/v8704 21: end-volume 22: 23: volume v8704-quota 24: type features/quota 25: option disk-usage-limit 100GB 26: subvolumes v8704-posix 27: end-volume 28: 29: volume v8704-brick 30: type features/locks 31: subvolumes v8704-quota 32: end-volume 33: 34: 35: volume server 36: type protocol/server 37: option transport-type tcp 38: option listen-port 6996 39: option bind-address 10.0.1.122 40: 41: option auth.login.v8705-brick.allow v8705 42: option auth.login.v8705.password 43: 44: option auth.login.v8704-brick.allow v8704 45: option auth.login.v8704.password 46: 47: subvolumes v8705-brick v8704-brick 48: 49: end-volume +--+ [2010-07-25 19:27:56] W [xlator.c:655:validate_xlator_volume_options] server: option 'bind-address' is deprecated, preferred is 'transport.socket.bind-address', continuing with correction [2010-07-25 19:27:56] W [xlator.c:655:validate_xlator_volume_options] server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2010-07-25 19:27:56] N [glusterfsd.c:1361:main] glusterfs: Successfully started [2010-07-25 19:27:58] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.145:1023 [2010-07-25 19:27:58] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.145:1022 [2010-07-25 19:28:34] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.122:1022 [2010-07-25 19:28:34] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.122:1023 pending frames: frame : type(1) op(UNLINK) frame : type(1) op(UNLINK) patchset: 2.0.1-886-g8379edd signal received: 11 time of crash: 2010-07-25 19:28:49 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.0.0 /lib64/libc.so.6[0x3b870302d0] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink_cbk+0xd5)[0x2ae3d57b4235] /usr/local/lib/libglusterfs.so.0[0x2ae3d46c9604] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink_cbk+0x95)[0x2ae3d538a955] /usr/local/lib/glusterfs/3.0.0/xlator/storage/posix.so(posix_unlink+0x1f3)[0x2ae3d517e613] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink_stat_cbk+0xd4)[0x2ae3d5389234] /usr/local/lib/glusterfs/3.0.0/xlator/storage/posix.so(posix_stat+0x137)[0x2ae3d517f9b7] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink+0xf1)[0x2ae3d538a501] /usr/local/lib/libglusterfs.so.0(default_unlink+0xcb)[0x2ae3d46c954b] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink_resume+0xd1)[0x2ae3d57b45a1] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_done+0x30)[0x2ae3d57b4e20] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_all+0xaf)[0x2ae3d57b575f] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve+0x7f)[0x2ae3d57b569f] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_all+0xa8)[0x2ae3d57b5758] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(serve
[Gluster-users] Adding volumes - How redistribute existing data
Hi, I am trying to add several new backend volumes to an existing GlusterFS setup. I am running GlusterFS 3.0.4 using the distribute translator. I've tried running the "scale-n-defrag.sh" script to redistribute the data across the additional volumes, but after running for a significant time, nothing was significantly redistributed. What are the proper steps to do to redistribute the data? Do I need to clean up the links GlusterFS makes on the backends before I run scale-n-defrag? I am running GlusterFS 3.0.4 on top of CentOS 5.4. This is not running GlusterSP. Thank you. Michael Moore Staff Engineer - Network SOLiD Advanced Research and Collaborations 500 Cummings Center Suite 2400 Beverly, MA 01915 T: 978-232-7886 michael.mo...@lifetech.com mike.mo...@appliedbiosystems.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
[Gluster-users] glusterfsd crashes when using quota translator
Hi, i tried to use the quota translator to limit the storage for a specific volume. I can mount the volume and see the limit with df. But as soon as i try to create a file on my mounted volume, the glusterfsd crashes. Configuration and Logs: Version : glusterfs 3.0.0 built on Jul 23 2010 16:45:59 git: 2.0.1-886-g8379edd Starting Time: 2010-07-25 19:27:56 Command line : /sbin/glusterfsd -f /etc/glusterfs/glusterfsd.vol PID : 27113 System name : Linux Nodename : inst-209.install.de Kernel Release : 2.6.18-194.el5 Hardware Identifier: x86_64 Given volfile: +--+ 1: 2: volume v8705-posix 3: type storage/posix 4: option directory /data/v8705 5: end-volume 6: 7: volume v8705-quota 8: type features/quota 9: option disk-usage-limit 100GB 10: subvolumes v8705-posix 11: end-volume 12: 13: volume v8705-brick 14: type features/locks 15: subvolumes v8705-quota 16: end-volume 17: 18: volume v8704-posix 19: type storage/posix 20: option directory /data/v8704 21: end-volume 22: 23: volume v8704-quota 24: type features/quota 25: option disk-usage-limit 100GB 26: subvolumes v8704-posix 27: end-volume 28: 29: volume v8704-brick 30: type features/locks 31: subvolumes v8704-quota 32: end-volume 33: 34: 35: volume server 36: type protocol/server 37: option transport-type tcp 38: option listen-port 6996 39: option bind-address 10.0.1.122 40: 41: option auth.login.v8705-brick.allow v8705 42: option auth.login.v8705.password 43: 44: option auth.login.v8704-brick.allow v8704 45: option auth.login.v8704.password 46: 47: subvolumes v8705-brick v8704-brick 48: 49: end-volume +--+ [2010-07-25 19:27:56] W [xlator.c:655:validate_xlator_volume_options] server: option 'bind-address' is deprecated, preferred is 'transport.socket.bind-address', continuing with correction [2010-07-25 19:27:56] W [xlator.c:655:validate_xlator_volume_options] server: option 'listen-port' is deprecated, preferred is 'transport.socket.listen-port', continuing with correction [2010-07-25 19:27:56] N [glusterfsd.c:1361:main] glusterfs: Successfully started [2010-07-25 19:27:58] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.145:1023 [2010-07-25 19:27:58] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.145:1022 [2010-07-25 19:28:34] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.122:1022 [2010-07-25 19:28:34] N [server-protocol.c:5809:mop_setvolume] server: accepted client from 10.0.1.122:1023 pending frames: frame : type(1) op(UNLINK) frame : type(1) op(UNLINK) patchset: 2.0.1-886-g8379edd signal received: 11 time of crash: 2010-07-25 19:28:49 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.0.0 /lib64/libc.so.6[0x3b870302d0] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink_cbk+0xd5)[0x2ae3d57b4235] /usr/local/lib/libglusterfs.so.0[0x2ae3d46c9604] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink_cbk+0x95)[0x2ae3d538a955] /usr/local/lib/glusterfs/3.0.0/xlator/storage/posix.so(posix_unlink+0x1f3)[0x2ae3d517e613] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink_stat_cbk+0xd4)[0x2ae3d5389234] /usr/local/lib/glusterfs/3.0.0/xlator/storage/posix.so(posix_stat+0x137)[0x2ae3d517f9b7] /usr/local/lib/glusterfs/3.0.0/xlator/features/quota.so(quota_unlink+0xf1)[0x2ae3d538a501] /usr/local/lib/libglusterfs.so.0(default_unlink+0xcb)[0x2ae3d46c954b] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink_resume+0xd1)[0x2ae3d57b45a1] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_done+0x30)[0x2ae3d57b4e20] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_all+0xaf)[0x2ae3d57b575f] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve+0x7f)[0x2ae3d57b569f] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_all+0xa8)[0x2ae3d57b5758] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_entry+0x5a)[0x2ae3d57b55fa] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve+0x55)[0x2ae3d57b5675] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_resolve_all+0x88)[0x2ae3d57b5738] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(resolve_and_resume+0x14)[0x2ae3d57b5784] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(server_unlink+0xa6)[0x2ae3d57a7e36] /usr/local/lib/glusterfs/3.0.0/xlator/protocol/server.so(protocol_server_pollin+0x90)[0x2ae3d57a5660] /usr/local/lib/glusterfs/
Re: [Gluster-users] Mirror volumes with odd number of servers
Carl - could you possibly provide an example of a configuration using an odd number of servers? glusterfs-volgen is unhappy when you don't give an even number. Thanks! James Burnash, Unix Engineering T. 201-239-2248 jburn...@knight.com | www.knight.com 545 Washington Ave. | Jersey City, NJ -Original Message- From: gluster-users-boun...@gluster.org [mailto:gluster-users-boun...@gluster.org] On Behalf Of Craig Carl Sent: Wednesday, July 28, 2010 12:16 AM To: Gluster General Discussion List Subject: Re: [Gluster-users] Mirror volumes with odd number of servers Brock - It is completely possible using Gluster File System. We haven't exposed that option via the Gluster Storage Platform GUI yet. Craig -- Craig Carl Gluster, Inc. Cell - (408) 829-9953 (California, USA) Gtalk - craig.c...@gmail.com From: "brock brown" To: gluster-users@gluster.org Sent: Tuesday, July 27, 2010 12:51:19 PM Subject: [Gluster-users] Mirror volumes with odd number of servers All the documentation refers to using multiples of 2 when setting up mirrored volumes. Also the Gluster Storage Platform will not allow it. Is this impossible and if so why? Thanks, Brock _ The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4 ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users DISCLAIMER: This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this in error, please immediately notify me and permanently delete the original and any copy of any e-mail and any printout thereof. E-mail transmission cannot be guaranteed to be secure or error-free. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. NOTICE REGARDING PRIVACY AND CONFIDENTIALITY Knight Capital Group may, at its discretion, monitor and review the content of all e-mail communications. http://www.knight.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
