Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-18 Thread Emmanuel Dreyfus
On Sat, Sep 13, 2014 at 01:34:58PM -0700, Anand Avati wrote: > How does the NetBSD nfs server provide stable directory offsets, for the > NFS client to resume reading from at a later point in time? Very similar > problems are present in that scenario and it might be helpful to see what > approaches

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-15 Thread Emmanuel Dreyfus
J. Bruce Fields wrote: > Again, this is a requirement for any filesystem that wants to be > exported over NFS. Not sure. We operate at readdir level, which is not even the system call level. >What filesystem exactly are you testing on? FFS > What is the NetBSD NFS server doing? I do not

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-15 Thread J. Bruce Fields
On Mon, Sep 15, 2014 at 07:57:21PM +0200, Emmanuel Dreyfus wrote: > J. Bruce Fields wrote: > > > 4) Report this as a bug to NetBSD. > > > > You may be correct about the letter of the spec, but specs don't > > necessarily capture all requirements. And as others say NFS server code > > at least w

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-15 Thread Emmanuel Dreyfus
J. Bruce Fields wrote: > 4) Report this as a bug to NetBSD. > > You may be correct about the letter of the spec, but specs don't > necessarily capture all requirements. And as others say NFS server code > at least will require that these actually work across reboots. > Filesystems go to great l

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-15 Thread J. Bruce Fields
On Sat, Sep 13, 2014 at 09:02:55PM +0200, Emmanuel Dreyfus wrote: > In <1lrx1si.n8tms1igmi5pm%m...@netbsd.org> I explained why NetBSD > currently fails self-heald.t, but since the subjet is burried deep in a > thread, it might be worth starting a new one to talk about how to fix. > > In 3 places w

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-14 Thread Emmanuel Dreyfus
Pranith Kumar Karampuri wrote: > If we hit a dead-end there we should make this change in self-heald.c > Keep us posted. Sure. In the meantime I am stil linterested by your change about non anonymous fd in features/index -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz m...@netbsd.org __

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-14 Thread Pranith Kumar Karampuri
On 09/14/2014 10:41 PM, Emmanuel Dreyfus wrote: 'Pranith Kumar Karampuri wrote: I can do that. That will teach me about that anonymous fd. Reading the code it seems afr-self-heald.c code does opendir and use the fd for readdir syncop, which suggest underlying xlator will use the same DIR *,

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-14 Thread Emmanuel Dreyfus
'Pranith Kumar Karampuri wrote: > I can do that. That will teach me about that anonymous fd. Reading the code it seems afr-self-heald.c code does opendir and use the fd for readdir syncop, which suggest underlying xlator will use the same DIR *, but logging in index.c I can see it calls opendir

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-14 Thread Pranith Kumar Karampuri
On 09/14/2014 10:39 AM, Emmanuel Dreyfus wrote: Pranith Kumar Karampuri wrote: Just to make sure I understand the problem, the issue is happening because self-heal-daemon uses anonymous fds to perform readdirs? i.e. there is no explicit opendir on the directory. Everytime there is a readdir i

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-14 Thread Emmanuel Dreyfus
Pranith Kumar Karampuri wrote: > I believe posix xlator doesn't have this problem for > non-anonymous fds where the DIR* stream is open till the final unref on > the fd. You must be right since reading a directory in a glusterfs volume does not cause an infinite loop on NetBSD [1]. A quick insp

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-13 Thread Emmanuel Dreyfus
Pranith Kumar Karampuri wrote: > Just to make sure I understand the problem, the issue is happening > because self-heal-daemon uses anonymous fds to perform readdirs? i.e. > there is no explicit opendir on the directory. Everytime there is a > readdir it may lead to opendir/seekdir/readdir/clos

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-13 Thread Pranith Kumar Karampuri
On 09/14/2014 12:32 AM, Emmanuel Dreyfus wrote: In <1lrx1si.n8tms1igmi5pm%m...@netbsd.org> I explained why NetBSD currently fails self-heald.t, but since the subjet is burried deep in a thread, it might be worth starting a new one to talk about how to fix. In 3 places within glusterfs code (fea

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-13 Thread Anand Avati
How does the NetBSD nfs server provide stable directory offsets, for the NFS client to resume reading from at a later point in time? Very similar problems are present in that scenario and it might be helpful to see what approaches are taken there (which are probably more tried and tested) Thanks

Re: [Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-13 Thread Joe Julian
Personally, I like the third option provided that doesn't cause memory issues. In fact, read the whole thing, transfer it to the client and let the client handle the posix syntax. Optionally add a path cache timeout client side that stores the directory listing for a period of time to mitigate

[Gluster-devel] How to fix wrong telldir/seekdir usage

2014-09-13 Thread Emmanuel Dreyfus
In <1lrx1si.n8tms1igmi5pm%m...@netbsd.org> I explained why NetBSD currently fails self-heald.t, but since the subjet is burried deep in a thread, it might be worth starting a new one to talk about how to fix. In 3 places within glusterfs code (features/index, features/snapview-server and storage/p