Re: [Gluster-devel] atomic operations fails_
Fredrik Widlund wrote: Can you reproduce this behavior on your end? We've been able to reproduce this, and are looking into it. Vikas -- Engineer - Gluster, Inc. ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
RE: [Gluster-devel] atomic operations fails_
Can you reproduce this behavior on your end? Regards, Fredrik -Original Message- From: gluster-devel-bounces+fredrik.widlund=qbrick@nongnu.org [mailto:gluster-devel-bounces+fredrik.widlund=qbrick@nongnu.org] On Behalf Of Fredrik Widlund Sent: den 10 februari 2010 10:27 To: Anand Avati Cc: gluster-devel@nongnu.org Subject: RE: [Gluster-devel] atomic operations fails_ "--entry-timeout=0" did not make any visible change. Regards, Fredrik -Original Message- From: anand.av...@gmail.com [mailto:anand.av...@gmail.com] On Behalf Of Anand Avati Sent: den 10 februari 2010 05:00 To: Fredrik Widlund Cc: gluster-devel@nongnu.org Subject: Re: [Gluster-devel] atomic operations fails_ On Wed, Feb 10, 2010 at 12:48 AM, Fredrik Widlund wrote: > > Hi, > > > > I'll try to make myself clearer than in the earlier thread, since I need > some help here. I'm not sure if I am missing something. > > > > I am not able to use atomic operations using glusterfs 3.0.2, on a Arch > Linux 2.6.32.7-1 x86_64 server. > > > > I've stripped everything down to the most simple configuration > > Server: a posix store with locks > > Client: a tcp client > > > > I mount the glusterfs on /gfs, and use a "writer" and a "reader" as below: > > [writer]r...@frustration> while :; do date > /gfs/.file; mv > /gfs/.file /gfs/file; done > > [reader] r...@frustration> while :; do cat /gfs/file > > /dev/null; done > > > > Leaving this running, I get a flood of > > cat: /gfs/file: No such file or directory > > > > This should not happen on a filesystem that supports atomic operations?! Please try turning off FUSE entry caching by passing --entry-timeout=0 to your mount command on the client side and let us know what difference that makes. Thanks, Avati ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
RE: [Gluster-devel] atomic operations fails_
"--entry-timeout=0" did not make any visible change. Regards, Fredrik -Original Message- From: anand.av...@gmail.com [mailto:anand.av...@gmail.com] On Behalf Of Anand Avati Sent: den 10 februari 2010 05:00 To: Fredrik Widlund Cc: gluster-devel@nongnu.org Subject: Re: [Gluster-devel] atomic operations fails_ On Wed, Feb 10, 2010 at 12:48 AM, Fredrik Widlund wrote: > > Hi, > > > > I'll try to make myself clearer than in the earlier thread, since I need > some help here. I'm not sure if I am missing something. > > > > I am not able to use atomic operations using glusterfs 3.0.2, on a Arch > Linux 2.6.32.7-1 x86_64 server. > > > > I've stripped everything down to the most simple configuration > > Server: a posix store with locks > > Client: a tcp client > > > > I mount the glusterfs on /gfs, and use a "writer" and a "reader" as below: > > [writer]r...@frustration> while :; do date > /gfs/.file; mv > /gfs/.file /gfs/file; done > > [reader] r...@frustration> while :; do cat /gfs/file > > /dev/null; done > > > > Leaving this running, I get a flood of > > cat: /gfs/file: No such file or directory > > > > This should not happen on a filesystem that supports atomic operations?! Please try turning off FUSE entry caching by passing --entry-timeout=0 to your mount command on the client side and let us know what difference that makes. Thanks, Avati ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] atomic operations fails_
On Wed, Feb 10, 2010 at 12:48 AM, Fredrik Widlund wrote: > > Hi, > > > > I’ll try to make myself clearer than in the earlier thread, since I need > some help here. I’m not sure if I am missing something. > > > > I am not able to use atomic operations using glusterfs 3.0.2, on a Arch > Linux 2.6.32.7-1 x86_64 server. > > > > I’ve stripped everything down to the most simple configuration > > Server: a posix store with locks > > Client: a tcp client > > > > I mount the glusterfs on /gfs, and use a “writer” and a “reader” as below: > > [writer] r...@frustration> while :; do date > /gfs/.file; mv > /gfs/.file /gfs/file; done > > [reader] r...@frustration> while :; do cat /gfs/file > > /dev/null; done > > > > Leaving this running, I get a flood of > > cat: /gfs/file: No such file or directory > > > > This should not happen on a filesystem that supports atomic operations?! Please try turning off FUSE entry caching by passing --entry-timeout=0 to your mount command on the client side and let us know what difference that makes. Thanks, Avati ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] atomic operations fails_
Haha, I love how you named your server! I think a lot of us feel the same way :) On Tue, Feb 9, 2010 at 11:18 AM, Fredrik Widlund wrote: > > Hi, > > > > I’ll try to make myself clearer than in the earlier thread, since I need > some help here. I’m not sure if I am missing something. > > > > I am not able to use atomic operations using glusterfs 3.0.2, on a Arch > Linux 2.6.32.7-1 x86_64 server. > > > > I’ve stripped everything down to the most simple configuration > > Server: a posix store with locks > > Client: a tcp client > > > > I mount the glusterfs on /gfs, and use a “writer” and a “reader” as below: > > [writer]r...@frustration> while :; do date > /gfs/.file; > mv /gfs/.file /gfs/file; done > > [reader] r...@frustration> while :; do cat /gfs/file > > /dev/null; done > > > > Leaving this running, I get a flood of > > cat: /gfs/file: No such file or directory > > > > This should not happen on a filesystem that supports atomic operations?! > > > > What this means is that I am unable to replace a file as an atomic > operation, creating a race-condition where the file is sometimes not > present. Furthermore this actually “desyncs” the server in a more complex > setup, requiring a restart of the server part to make “desynced” files > available again. > > > > This reproduces using XFS as well as EXT2 as the underlying fs. > > > > [glusterfs.vol] > > volume media0 > > type protocol/client > > option transport-type tcp > > option remote-host 127.0.0.1 > > option remote-subvolume media0 > > end-volume > > > > mounted with “glusterfs -f glusterfs.vol /gfs” > > > > [glusterfsd.vol] > > (se below debug log) > > > > -x- > > > > [2010-02-09 18:18:23] D [glusterfsd.c:424:_get_specfp] glusterfs: loading > volume file glusterfsd.vol > > [2010-02-09 18:18:23] D [xlator.c:739:xlator_set_type] xlator: > dlsym(notify) on /usr/lib/glusterfs/3.0.2/xlator/features/locks.so: > undefined symbol: notify -- neglecting > > > > > Version : glusterfs 3.0.2 built on Feb 8 2010 15:23:34 > > git: v3.0.2 > > Starting Time: 2010-02-09 18:18:23 > > Command line : glusterfs -f glusterfsd.vol --debug > > PID : 19486 > > System name : Linux > > Nodename : n100.p1.x0.local > > Kernel Release : 2.6.32-ARCH > > Hardware Identifier: x86_64 > > > > Given volfile: > > > +--+ > > 1: volume media0-store > > 2: type storage/posix > > 3: option directory /mnt > > 4: end-volume > > 5: > > 6: volume media0 > > 7: type features/locks > > 8: subvolumes media0-store > > 9: end-volume > > 10: > > 11: volume server > > 12: type protocol/server > > 13: option transport-type tcp > > 14: option auth.addr.media0.allow * > > 15: subvolumes media0 > > 16: end-volume > > > > > +--+ > > [2010-02-09 18:18:23] D [glusterfsd.c:1370:main] glusterfs: running in pid > 19486 > > [2010-02-09 18:18:23] D [transport.c:145:transport_load] transport: attempt > to load file /usr/lib/glusterfs/3.0.2/transport/socket.so > > [2010-02-09 18:18:23] N [glusterfsd.c:1396:main] glusterfs: Successfully > started > > [2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = "*", > received addr = "127.0.0.1" > > [2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: > accepted client from 127.0.0.1:1023 > > [2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = "*", > received addr = "127.0.0.1" > > [2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: > accepted client from 127.0.0.1:1022 > > [2010-02-09 18:18:58] D [server-resolve.c:238:resolve_path_deep] media0: > RESOLVE RENAME() seeking deep resolution of /file > > [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) > @key=0x7f2a38a9f79b > > [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) > @key=0x7f2a38a9f7b3 > > [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) > @key=0x7f2a38a9f7cb > > [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) > @key=0x7f2a38a9f79b > > [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) > @key=0x7f2a38a9f7b3 > > [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) > @key=0x7f2a38a9f7cb > > [2010-02-09 18:19:15] D [inode.c:916:inode_path] media0/inode: no dentry > for non-root inode 502725 > > [2010-02-09 18:19:34] D [inode.c:916:inode_path] media0/inode: no dentry > for non-root inode 502734 > > [2010-02-09 18:19:56] D [inode.c:916:inode_path] media0/inode: no dentry > for non-root inode 502734 > > […] repeated many times. Frequency of the “no dentry” entries depends on > how aggressive the reader is. > > > > ___ > Gluster-devel mailing list > Gluster-devel@nongnu.org > http://lists.nongnu.org/mailman/listinf