Re: [Gluster-users] [Gluster-devel] inode list memory usage
On 03/02/2010 05:27 AM, Mike Terzo wrote: On Fri, Feb 26, 2010 at 2:41 PM, Harshavardhanahar...@gluster.com wrote: I'm guessing you are talking about this patch: http://patches.gluster.com/patch/2659/ or is there another one? Yes thats the patch can you patch it and use it?. Tests are looking good so far, I haven't done a long test yet. I'll let you know how that goes, I'm gonna let it run for a few days. My test code, is simply create a bunch of directories, remove a bunch of directories, over and over and over. The memory of process stays consistent, however for some reason directories are not being removed completely. After I do a pass of removing a bunch of directories the next pass of create fails, because the previous directories weren't removed. --mike terzo It could be a fuse bug. Are you getting Device Resource or Busy or File Exists errors. What is the fuse version under use? dmesg | grep -i fuse Regards -- Harshavardhana http://www.gluster.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] error: Transport endpoint is not connected and Stale NFS file handle
José, I guess my question is why? If these are local drives you can use software raid (which will be MUCH faster) to do raid0 or raid5. If these are not local drives what benefit do you think you are getting by adding an additional layer of replication? As I see it yo will just kill your performance, as it seems to me that network FSs are about 10-25% of the speed of local file systems. Adding the 2nd layer would mean 1-2.5% of the speed of the original. ^C José Manuel Canelas wrote: Hi, Since no one replies to this, i'll reply to myself :) I just realized I assumed that it is possible to replicate distributed volumes. I am wrong? In my setup bellow I was trying to make Replicated Distributed Storage, the inverse of what is described in http://www.gluster.com/community/documentation/index.php/Distributed_Replicated_Storage. Trying to draw a picture: replicated -| 3 replicas presented as one volume replica1 replica2 replica3 ---|-|---| - 4 volumes, distributed, to make up 4vols 4vols 4volseach of the 3 volumes to be replicated Is this dumb or is there a better way? thanks, José Canelas On 02/26/2010 03:55 PM, José Manuel Canelas wrote: Hello, everyone. We're setting up GlusterFS for some testing and having some trouble with the configuration. We have 4 nodes as clients and servers, 4 disks each. I'm trying to setup 3 replicas across all those 16 disks, configured at the client side, for high availability and optimal performance, in a way that makes it easy to add new disks and nodes. The best way I thought doing it was to put disks together from different nodes into 3 distributed volumes and then use each of those as a replica of the top volume. I'd like your input on this too, so if you look at the configuration and something looks wrong or dumb, it probably is, so please let me know :) Now the server config looks like this: volume posix1 type storage/posix option directory /srv/gdisk01 end-volume volume locks1 type features/locks subvolumes posix1 end-volume volume brick1 type performance/io-threads option thread-count 8 subvolumes locks1 end-volume [4 more identical bricks and...] volume server-tcp type protocol/server option transport-type tcp option auth.addr.brick1.allow * option auth.addr.brick2.allow * option auth.addr.brick3.allow * option auth.addr.brick4.allow * option transport.socket.listen-port 6996 option transport.socket.nodelay on subvolumes brick1 brick2 brick3 brick4 end-volume The client config: volume node01-1 type protocol/client option transport-type tcp option remote-host node01 option transport.socket.nodelay on option transport.remote-port 6996 option remote-subvolume brick1 end-volume [repeated for every brick, until node04-4] ### Our 3 replicas volume repstore1 type cluster/distribute subvolumes node01-1 node02-1 node03-1 node04-1 node04-4 end-volume volume repstore2 type cluster/distribute subvolumes node01-2 node02-2 node03-2 node04-2 node02-2 end-volume volume repstore3 type cluster/distribute subvolumes node01-3 node02-3 node03-3 node04-3 node03-3 end-volume volume replicate type cluster/replicate subvolumes repstore1 repstore2 repstore3 end-volume [and then the performance bits] When starting the glusterfs server, everything looks fine. I then mount the filesystem with node01:~# glusterfs --debug -f /etc/glusterfs/glusterfs.vol /srv/gluster-export and it does not complain and shows up as properly mounted. When accessing the content, it gives back an error, that the Transport endpoint is not connected. The log has a Stale NFS file handle warning. See bellow: [...] [2010-02-26 14:56:01] D [dht-common.c:274:dht_revalidate_cbk] repstore3: mismatching layouts for / [2010-02-26 14:56:01] W [fuse-bridge.c:722:fuse_attr_cbk] glusterfs-fuse: 9: LOOKUP() / = -1 (Stale NFS file handle) node01:~# mount /dev/cciss/c0d0p1 on / type ext3 (rw,errors=remount-ro) tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755) proc on /proc type proc (rw,noexec,nosuid,nodev) sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) procbususb on /proc/bus/usb type usbfs (rw) udev on /dev type tmpfs (rw,mode=0755) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620) fusectl on /sys/fs/fuse/connections type fusectl (rw) /dev/cciss/c0d1 on /srv/gdisk01 type ext3 (rw,errors=remount-ro) /dev/cciss/c0d2 on /srv/gdisk02 type ext3 (rw,errors=remount-ro) /dev/cciss/c0d3 on /srv/gdisk03 type ext3 (rw,errors=remount-ro) /dev/cciss/c0d4 on /srv/gdisk04 type ext3 (rw,errors=remount-ro) /etc/glusterfs/glusterfs.vol on /srv/gluster-export type fuse.glusterfs (rw,allow_other,default_permissions,max_read=131072) node01:~# ls /srv/gluster-export ls: cannot access /srv/gluster-export: Transport endpoint is not connected node01:~# The
[Gluster-users] crash when using the cp command to copy files off a striped gluster dir but not when using rsync
Hi, I've got this strange problem where a striped endpoint will crash when I try to use cp to copy files off of it but not when I use rsync to copy files off: [u...@gluster5 user]$ cp -r Python-2.6.4/ ~/tmp/ cp: reading `Python-2.6.4/Lib/lib2to3/tests/data/fixers/myfixes/__init__.py': Software caused connection abort cp: closing `Python-2.6.4/Lib/lib2to3/tests/data/fixers/myfixes/__init__.py': Transport endpoint is not connected pending frames: frame : type(1) op(READ) frame : type(1) op(READ) patchset: 2.0.1-886-g8379edd signal received: 11 time of crash: 2010-03-02 11:06:40 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.0.0 /lib64/libc.so.6[0x3a66a30280] /lib64/libpthread.so.0(pthread_spin_lock+0x2)[0x3a6760b0d2] /usr/lib64/libglusterfs.so.0(iobref_merge+0x2f)[0x37af83fe71] /usr/lib64/glusterfs/3.0.0/xlator/cluster/stripe.so(stripe_readv_cbk+0x1ee)[0x2b55b16c1b68] /usr/lib64/glusterfs/3.0.0/xlator/performance/stat-prefetch.so(sp_readv_cbk+0xf5)[0x2b55b14a39d2] /usr/lib64/glusterfs/3.0.0/xlator/performance/quick-read.so(qr_readv+0x6a6)[0x2b55b128c209] /usr/lib64/glusterfs/3.0.0/xlator/performance/stat-prefetch.so(sp_readv+0x256)[0x2b55b14a3c4c] /usr/lib64/glusterfs/3.0.0/xlator/cluster/stripe.so(stripe_readv+0x5fc)[0x2b55b16c28cd] /usr/lib64/glusterfs/3.0.0/xlator/mount/fuse.so[0x2b55b18d2665] /usr/lib64/glusterfs/3.0.0/xlator/mount/fuse.so[0x2b55b18d88ff] /lib64/libpthread.so.0[0x3a67606367] /lib64/libc.so.6(clone+0x6d)[0x3a66ad2f7d] - Here's the client configuration: volume client-stripe-1 type protocol/client option transport-type ib-verbs option remote-host gluster1 option remote-subvolume iothreads end-volume volume client-stripe-2 type protocol/client option transport-type ib-verbs option remote-host gluster2 option remote-subvolume iothreads end-volume volume client-stripe-3 type protocol/client option transport-type ib-verbs option remote-host gluster3 option remote-subvolume iothreads end-volume volume client-stripe-4 type protocol/client option transport-type ib-verbs option remote-host gluster4 option remote-subvolume iothreads end-volume volume client-stripe-5 type protocol/client option transport-type ib-verbs option remote-host gluster5 option remote-subvolume iothreads end-volume volume readahead-gluster1 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-1 end-volume volume readahead-gluster2 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-2 end-volume volume readahead-gluster3 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-3 end-volume volume readahead-gluster4 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-4 end-volume volume readahead-gluster5 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-5 end-volume volume writebehind-gluster1 type performance/write-behind option flush-behind on subvolumes readahead-gluster1 end-volume volume writebehind-gluster2 type performance/write-behind option flush-behind on subvolumes readahead-gluster2 end-volume volume writebehind-gluster3 type performance/write-behind option flush-behind on subvolumes readahead-gluster3 end-volume volume writebehind-gluster4 type performance/write-behind option flush-behind on subvolumes readahead-gluster4 end-volume volume writebehind-gluster5 type performance/write-behind option flush-behind on subvolumes readahead-gluster5 end-volume volume quick-read-gluster1 type performance/quick-read subvolumes writebehind-gluster1 end-volume volume quick-read-gluster2 type performance/quick-read subvolumes writebehind-gluster2 end-volume volume quick-read-gluster3 type performance/quick-read subvolumes writebehind-gluster3 end-volume volume quick-read-gluster4 type performance/quick-read subvolumes writebehind-gluster4 end-volume volume quick-read-gluster5 type performance/quick-read subvolumes writebehind-gluster5 end-volume volume stat-prefetch-gluster1 type performance/stat-prefetch subvolumes quick-read-gluster1 end-volume volume stat-prefetch-gluster2 type performance/stat-prefetch subvolumes quick-read-gluster2 end-volume volume stat-prefetch-gluster3 type performance/stat-prefetch subvolumes quick-read-gluster3 end-volume volume stat-prefetch-gluster4
Re: [Gluster-users] [Gluster-devel] inode list memory usage
On Tue, Mar 2, 2010 at 10:02 PM, Mike Terzo mte...@gmail.com wrote: On Tue, Mar 2, 2010 at 8:51 AM, Harshavardhana har...@gluster.com wrote: On 03/02/2010 05:27 AM, Mike Terzo wrote: On Fri, Feb 26, 2010 at 2:41 PM, Harshavardhanahar...@gluster.com wrote: I'm guessing you are talking about this patch: http://patches.gluster.com/patch/2659/ or is there another one? Yes thats the patch can you patch it and use it?. Tests are looking good so far, I haven't done a long test yet. I'll let you know how that goes, I'm gonna let it run for a few days. My test code, is simply create a bunch of directories, remove a bunch of directories, over and over and over. The memory of process stays consistent, however for some reason directories are not being removed completely. After I do a pass of removing a bunch of directories the next pass of create fails, because the previous directories weren't removed. --mike terzo It could be a fuse bug. Are you getting Device Resource or Busy or File Exists errors. What is the fuse version under use? [2010-02-26 13:51:56] N [fuse-bridge.c:3017:fuse_init] glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.11 The problem only occurs with the patch. What is the patch are you referring here? From the protocol versions of FUSE it is evident that you have 7.11. Are you saying the patch for inode invalidation has problem?. Can you attach all the server and client logs?. Regards dmesg | grep -i fuse Regards -- Harshavardhana http://www.gluster.com ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] crash when using the cp command to copy files off a striped gluster dir but not when using rsync
Also, I cannot duplicate the crash when I copy using cp off the distributed endpoint, i.e. cp works fine there. ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] error: Transport endpoint is not connected and Stale NFS file handle
Jose, I would request you to use volgen. A suggestion on the config, why not group disks on two nodes ( simple e.g. group1 from server1 and server2, group2 from server2 and server4 ) with replicate. Then have distribute translator on top of the two replicated volumes. This would give you a single GlusterFS volume to be mounted on each of the clients. Would'nt something simple like that work for you ? I gave a representative example of 2 replicas this can be easily extended to 3 replicas also. Regards, Tejas. - Original Message - From: José Manuel Canelas jcane...@co.sapo.pt To: gluster-users@gluster.org Sent: Tuesday, March 2, 2010 9:38:32 PM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [Gluster-users] error: Transport endpoint is not connected and Stale NFS file handle Hi, Since no one replies to this, i'll reply to myself :) I just realized I assumed that it is possible to replicate distributed volumes. I am wrong? In my setup bellow I was trying to make Replicated Distributed Storage, the inverse of what is described in http://www.gluster.com/community/documentation/index.php/Distributed_Replicated_Storage. Trying to draw a picture: replicated -| 3 replicas presented as one volume replica1 replica2 replica3 ---|-|---| - 4 volumes, distributed, to make up 4vols 4vols 4volseach of the 3 volumes to be replicated Is this dumb or is there a better way? thanks, José Canelas On 02/26/2010 03:55 PM, José Manuel Canelas wrote: Hello, everyone. We're setting up GlusterFS for some testing and having some trouble with the configuration. We have 4 nodes as clients and servers, 4 disks each. I'm trying to setup 3 replicas across all those 16 disks, configured at the client side, for high availability and optimal performance, in a way that makes it easy to add new disks and nodes. The best way I thought doing it was to put disks together from different nodes into 3 distributed volumes and then use each of those as a replica of the top volume. I'd like your input on this too, so if you look at the configuration and something looks wrong or dumb, it probably is, so please let me know :) Now the server config looks like this: volume posix1 type storage/posix option directory /srv/gdisk01 end-volume volume locks1 type features/locks subvolumes posix1 end-volume volume brick1 type performance/io-threads option thread-count 8 subvolumes locks1 end-volume [4 more identical bricks and...] volume server-tcp type protocol/server option transport-type tcp option auth.addr.brick1.allow * option auth.addr.brick2.allow * option auth.addr.brick3.allow * option auth.addr.brick4.allow * option transport.socket.listen-port 6996 option transport.socket.nodelay on subvolumes brick1 brick2 brick3 brick4 end-volume The client config: volume node01-1 type protocol/client option transport-type tcp option remote-host node01 option transport.socket.nodelay on option transport.remote-port 6996 option remote-subvolume brick1 end-volume [repeated for every brick, until node04-4] ### Our 3 replicas volume repstore1 type cluster/distribute subvolumes node01-1 node02-1 node03-1 node04-1 node04-4 end-volume volume repstore2 type cluster/distribute subvolumes node01-2 node02-2 node03-2 node04-2 node02-2 end-volume volume repstore3 type cluster/distribute subvolumes node01-3 node02-3 node03-3 node04-3 node03-3 end-volume volume replicate type cluster/replicate subvolumes repstore1 repstore2 repstore3 end-volume [and then the performance bits] When starting the glusterfs server, everything looks fine. I then mount the filesystem with node01:~# glusterfs --debug -f /etc/glusterfs/glusterfs.vol /srv/gluster-export and it does not complain and shows up as properly mounted. When accessing the content, it gives back an error, that the Transport endpoint is not connected. The log has a Stale NFS file handle warning. See bellow: [...] [2010-02-26 14:56:01] D [dht-common.c:274:dht_revalidate_cbk] repstore3: mismatching layouts for / [2010-02-26 14:56:01] W [fuse-bridge.c:722:fuse_attr_cbk] glusterfs-fuse: 9: LOOKUP() / = -1 (Stale NFS file handle) node01:~# mount /dev/cciss/c0d0p1 on / type ext3 (rw,errors=remount-ro) tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755) proc on /proc type proc (rw,noexec,nosuid,nodev) sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) procbususb on /proc/bus/usb type usbfs (rw) udev on /dev type tmpfs (rw,mode=0755) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev) devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620) fusectl on /sys/fs/fuse/connections type fusectl (rw) /dev/cciss/c0d1 on /srv/gdisk01 type ext3 (rw,errors=remount-ro)
Re: [Gluster-users] GlusterFS 3.0.2 small file read performance benchmark
Ed, oplocks are implemented by SAMBA and it would not be a part of GlusterFS per se till we implement a native SAMBA translator ( something that would replace the SAMBA server itself with a thin SAMBA kind of a layer on top of GlusterFS itself ). We are doing that for NFS by building an NFS translator. At some point, it would be interesting to explore, clustered SAMBA using ctdb, where two GlusterFS clients can export the same volume. ctdb itself seems to be coming up well now. Regards, Tejas. - Original Message - From: Ed W li...@wildgooses.com To: Gluster Users gluster-users@gluster.org Sent: Wednesday, March 3, 2010 12:10:47 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [Gluster-users] GlusterFS 3.0.2 small file readperformance benchmark On 01/03/2010 20:44, Ed W wrote: I believe samba (and probably others) use a two way lock escalation facility to mitigate a similar problem. So you can read-lock or phrased differently, express your interest in caching some files/metadata and then if someone changes what you are watching the lock break is pushed to you to invalidate your cache. Seems NFS v4 implements something similar via delegations (not believed implemented in linux NFSv4 though...) In samba the equivalent are called op locks I guess this would be a great project for someone interested to work on - op-lock translator for gluster Ed W ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] GlusterFS 3.0.2 small file read performance benchmark
Well, oplocks are an SMB definition, but the basic concept of opportunistic locking is independent of the filesystem. For example it appears that oplocks now appear in the NFS v4 standard under the name delegations (I would assume some variation of oplocks also exists in GFS and OCFS, but I'm not familiar with them) The basic concept would potentially provide a huge performance boost for glusterfs because it allows cache coherent writeback caching. In fact lets cut to the chase - what we desire is cache coherent writeback caching, ie reads to one server can be served from local client cache, but if the file is changed elsewhere then instantly our cache here is invalidated, and likewise we can write at will to a local copy of the file and allow it to get out of sync with the other servers, but as soon as some other server tries to read/write to our file then we must be notified and flush our cache (and request alternative locks or fall back to sync reads/writes) How do we do this? Well in NFS v3 and before and I believe in Glusterfs there is implemented only a cache and hope option, which caches data for a second or so and hopes the file doesn't change under us. The improved algorithm is opportunistic locking where the client indicates to the server the desire to work with some data locally and get it out of sync with the server - the server then tracks that reservation and if some other client wants to access the data it pushes a lock break to the original client and informs it that it needs to fsync and run without the oplock I believe that an oplock service this could be implemented via a new translator which works in conjunction with the read and writeback caching. Effectively it would be a two way lock manager, but it's job is somewhat simpler in that all it needs do is vary the existing caches on a per file basis. So for example if we read some attributes for some files then at present they are blindly cached for X ms and then dropped, but our oplock translator will instead allow the attributes to be cached indefinitely until we get a push notification from the server side that our cache must be invalidated. Same also with writes - we can use writeback cache as long as no one else has tried to read or write to our file, but as soon as someone else touches it we need to fsync and run without cache I have had a very quick glance at the current locks module and it's quite a bit more complex than I might have guessed... I had wondered if it might not be possible to make the locks module talk to the cache module and add server side lock breaking through that module? Essentially it's the addition of the push lock breaking which helps, so if we are reading away and some other client modifies a file then we need a feedback loop to invalide our read cache Perhaps this is all implemented in glusterfs already though and I'm just missing the point... Cheers Ed W On 02/03/2010 18:52, Tejas N. Bhise wrote: Ed, oplocks are implemented by SAMBA and it would not be a part of GlusterFS per se till we implement a native SAMBA translator ( something that would replace the SAMBA server itself with a thin SAMBA kind of a layer on top of GlusterFS itself ). We are doing that for NFS by building an NFS translator. At some point, it would be interesting to explore, clustered SAMBA using ctdb, where two GlusterFS clients can export the same volume. ctdb itself seems to be coming up well now. Regards, Tejas. - Original Message - From: Ed Wli...@wildgooses.com To: Gluster Usersgluster-users@gluster.org Sent: Wednesday, March 3, 2010 12:10:47 AM GMT +05:30 Chennai, Kolkata, Mumbai, New Delhi Subject: Re: [Gluster-users] GlusterFS 3.0.2 small file readperformance benchmark On 01/03/2010 20:44, Ed W wrote: I believe samba (and probably others) use a two way lock escalation facility to mitigate a similar problem. So you can read-lock or phrased differently, express your interest in caching some files/metadata and then if someone changes what you are watching the lock break is pushed to you to invalidate your cache. Seems NFS v4 implements something similar via delegations (not believed implemented in linux NFSv4 though...) In samba the equivalent are called op locks I guess this would be a great project for someone interested to work on - op-lock translator for gluster Ed W ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] Can gluster export be nfs exported too?
No, its not supported to export glusterfs backend directories using NFS. On Tue, Mar 2, 2010 at 8:43 AM, Jeremy Enos je...@ncsa.uiuc.edu wrote: If I have a single system exporting gluster, can I also export that same directory via NFS w/o Gluster in the loop, or is it different somehow? I assume I definitely can't do that on any striped setup for obvious reasons. (I realize I could NFS export the gluster mounted volume, but I'm talking about the gluster export volume here) thx- Jeremy ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users -- Raghavendra G ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Re: [Gluster-users] crash when using the cp command to copy files off a striped gluster dir but not when using rsync
On 03/02/2010 10:43 PM, Sabuj Pattanayek wrote: Hi, I've got this strange problem where a striped endpoint will crash when I try to use cp to copy files off of it but not when I use rsync to copy files off: [u...@gluster5 user]$ cp -r Python-2.6.4/ ~/tmp/ cp: reading `Python-2.6.4/Lib/lib2to3/tests/data/fixers/myfixes/__init__.py': Software caused connection abort cp: closing `Python-2.6.4/Lib/lib2to3/tests/data/fixers/myfixes/__init__.py': Transport endpoint is not connected pending frames: frame : type(1) op(READ) frame : type(1) op(READ) patchset: 2.0.1-886-g8379edd signal received: 11 time of crash: 2010-03-02 11:06:40 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.0.0 /lib64/libc.so.6[0x3a66a30280] /lib64/libpthread.so.0(pthread_spin_lock+0x2)[0x3a6760b0d2] /usr/lib64/libglusterfs.so.0(iobref_merge+0x2f)[0x37af83fe71] /usr/lib64/glusterfs/3.0.0/xlator/cluster/stripe.so(stripe_readv_cbk+0x1ee)[0x2b55b16c1b68] /usr/lib64/glusterfs/3.0.0/xlator/performance/stat-prefetch.so(sp_readv_cbk+0xf5)[0x2b55b14a39d2] /usr/lib64/glusterfs/3.0.0/xlator/performance/quick-read.so(qr_readv+0x6a6)[0x2b55b128c209] /usr/lib64/glusterfs/3.0.0/xlator/performance/stat-prefetch.so(sp_readv+0x256)[0x2b55b14a3c4c] /usr/lib64/glusterfs/3.0.0/xlator/cluster/stripe.so(stripe_readv+0x5fc)[0x2b55b16c28cd] /usr/lib64/glusterfs/3.0.0/xlator/mount/fuse.so[0x2b55b18d2665] /usr/lib64/glusterfs/3.0.0/xlator/mount/fuse.so[0x2b55b18d88ff] /lib64/libpthread.so.0[0x3a67606367] /lib64/libc.so.6(clone+0x6d)[0x3a66ad2f7d] - Here's the client configuration: volume client-stripe-1 type protocol/client option transport-type ib-verbs option remote-host gluster1 option remote-subvolume iothreads end-volume volume client-stripe-2 type protocol/client option transport-type ib-verbs option remote-host gluster2 option remote-subvolume iothreads end-volume volume client-stripe-3 type protocol/client option transport-type ib-verbs option remote-host gluster3 option remote-subvolume iothreads end-volume volume client-stripe-4 type protocol/client option transport-type ib-verbs option remote-host gluster4 option remote-subvolume iothreads end-volume volume client-stripe-5 type protocol/client option transport-type ib-verbs option remote-host gluster5 option remote-subvolume iothreads end-volume volume readahead-gluster1 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-1 end-volume volume readahead-gluster2 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-2 end-volume volume readahead-gluster3 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-3 end-volume volume readahead-gluster4 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-4 end-volume volume readahead-gluster5 type performance/read-ahead option page-count 4 # 2 is default option force-atime-update off # default is off subvolumes client-stripe-5 end-volume volume writebehind-gluster1 type performance/write-behind option flush-behind on subvolumes readahead-gluster1 end-volume volume writebehind-gluster2 type performance/write-behind option flush-behind on subvolumes readahead-gluster2 end-volume volume writebehind-gluster3 type performance/write-behind option flush-behind on subvolumes readahead-gluster3 end-volume volume writebehind-gluster4 type performance/write-behind option flush-behind on subvolumes readahead-gluster4 end-volume volume writebehind-gluster5 type performance/write-behind option flush-behind on subvolumes readahead-gluster5 end-volume volume quick-read-gluster1 type performance/quick-read subvolumes writebehind-gluster1 end-volume volume quick-read-gluster2 type performance/quick-read subvolumes writebehind-gluster2 end-volume volume quick-read-gluster3 type performance/quick-read subvolumes writebehind-gluster3 end-volume volume quick-read-gluster4 type performance/quick-read subvolumes writebehind-gluster4 end-volume volume quick-read-gluster5 type performance/quick-read subvolumes writebehind-gluster5 end-volume volume stat-prefetch-gluster1 type performance/stat-prefetch subvolumes quick-read-gluster1 end-volume volume stat-prefetch-gluster2 type performance/stat-prefetch subvolumes quick-read-gluster2 end-volume volume stat-prefetch-gluster3 type
Re: [Gluster-users] Can gluster export be nfs exported too?
Second question: Even if it's not supported, is it theoretically feasible? Jeremy On 3/2/2010 10:16 PM, Raghavendra G wrote: No, its not supported to export glusterfs backend directories using NFS. On Tue, Mar 2, 2010 at 8:43 AM, Jeremy Enos je...@ncsa.uiuc.edu mailto:je...@ncsa.uiuc.edu wrote: If I have a single system exporting gluster, can I also export that same directory via NFS w/o Gluster in the loop, or is it different somehow? I assume I definitely can't do that on any striped setup for obvious reasons. (I realize I could NFS export the gluster mounted volume, but I'm talking about the gluster export volume here) thx- Jeremy ___ Gluster-users mailing list Gluster-users@gluster.org mailto:Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users -- Raghavendra G ___ Gluster-users mailing list Gluster-users@gluster.org http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
