Re: struct file reference at VFS level

2016-04-29 Thread Emmanuel Dreyfus
On Fri, Apr 29, 2016 at 10:13:13AM +0200, J. Hannken-Illjes wrote: > Advertised locks are already implemented inside the file systems and > we have VOP_ADVLOCK() with ?op == F_GETLK?. What else is needed to > make the decision here? In advisory locks, the only place where the filesystem checks loc

Re: struct file reference at VFS level

2016-04-29 Thread J. Hannken-Illjes
> On 28 Apr 2016, at 10:19, Emmanuel Dreyfus wrote: > >> Wouldn't it be better to put the mandatory locking at the file->vnode >> level, operations vn_read() and vn_write()? > > Do you mean that instead of providing support so that filesystems > can implement mandatory locks, we should just im

Re: struct file reference at VFS level

2016-04-28 Thread Emmanuel Dreyfus
On Thu, Apr 28, 2016 at 09:54:05AM +0200, J. Hannken-Illjes wrote: > In your example process 2 writes without locking, is it part of your > proposed semantics to add an implicit lock/unlock to read and write > operations? The given scenario is a test case for mandatory lock support. Process 2 doe

Re: struct file reference at VFS level

2016-04-28 Thread J. Hannken-Illjes
> On 22 Apr 2016, at 17:33, Emmanuel Dreyfus wrote: > > On Fri, Apr 22, 2016 at 10:42:10AM -0400, Greg Troxel wrote: >> I still don't understand why this is about FUSE. What if a file were >> opened without O_NONBLOCK and then the same file were opened with? Why >> isn't that the same situatio

Re: struct file reference at VFS level

2016-04-28 Thread Emmanuel Dreyfus
On Thu, Apr 28, 2016 at 04:57:28PM +1000, matthew green wrote: > i don't know, but not in kauth_cred_t. don't go linking systems > together like that. don't abuse kauth to shove some random other > data you need simply because that's a place you found you could > change. the usage of it in VFS i

re: struct file reference at VFS level

2016-04-27 Thread matthew green
Emmanuel Dreyfus writes: > matthew green wrote: > > > you do realise that kauth_cred_t is shared across processes in > > some cases but not all, right? > > VFS uses a kauth_cred_t from struct file f_cred field, which is tied to > the struct file. The value is initialized as curlwp->l_cred in >

Re: struct file reference at VFS level

2016-04-27 Thread Emmanuel Dreyfus
matthew green wrote: > you do realise that kauth_cred_t is shared across processes in > some cases but not all, right? VFS uses a kauth_cred_t from struct file f_cred field, which is tied to the struct file. The value is initialized as curlwp->l_cred in fd_allocfile(). My proposed patch repla

re: struct file reference at VFS level

2016-04-27 Thread matthew green
> > Can you please start with a consistent proposal of what the end result > > should look like before adding random pieces? I'm sure, but this feels > > like a very adhoc hack to cover a few corner cases for some strange out > > of tree target without a clean idea on how it affects the long term >

Re: struct file reference at VFS level

2016-04-27 Thread Emmanuel Dreyfus
Joerg Sonnenberger wrote: > Can you please start with a consistent proposal of what the end result > should look like before adding random pieces? I'm sure, but this feels > like a very adhoc hack to cover a few corner cases for some strange out > of tree target without a clean idea on how it aff

Re: struct file reference at VFS level

2016-04-27 Thread Joerg Sonnenberger
On Wed, Apr 27, 2016 at 03:58:43PM +, Emmanuel Dreyfus wrote: > On Sun, Apr 24, 2016 at 07:11:37PM +, David Holland wrote: > > Since you said fuse has a way to do that but it doesn't work for our > > fuse, I guess the right way forward is to make it work in our fuse. > > What's required? Ju

Re: struct file reference at VFS level

2016-04-27 Thread Emmanuel Dreyfus
David Holland wrote: > ok, I don't understand: even assuming that stashing this information > in the credentials is the right place (which I don't think I agree > with but haven't really formed a coherent opinion about yet)... why on > earth should it be conditional? Because if the filesystem do

Re: struct file reference at VFS level

2016-04-27 Thread David Holland
On Wed, Apr 27, 2016 at 03:58:43PM +, Emmanuel Dreyfus wrote: > On Sun, Apr 24, 2016 at 07:11:37PM +, David Holland wrote: > > Since you said fuse has a way to do that but it doesn't work for our > > fuse, I guess the right way forward is to make it work in our fuse. > > What's required

Re: struct file reference at VFS level

2016-04-27 Thread Emmanuel Dreyfus
On Sun, Apr 24, 2016 at 07:11:37PM +, David Holland wrote: > Since you said fuse has a way to do that but it doesn't work for our > fuse, I guess the right way forward is to make it work in our fuse. > What's required? Just send an arbitrary ID associated with the open > through puffs to userla

Re: struct file reference at VFS level

2016-04-24 Thread Emmanuel Dreyfus
David Holland wrote: > Ah, I see -- you were talking about access to the flags of the struct > file, and that isn't necessary. Yes, sorry for that confusing part. In fact, the file flags are quite useful to spot the problem in a test case and this is why I insisted on them, but indeed they are

Re: struct file reference at VFS level

2016-04-24 Thread David Holland
On Sun, Apr 24, 2016 at 04:40:43AM +0200, Emmanuel Dreyfus wrote: > > If something in fuse is causing these cases to share the same open and > > thus the same struct file, fuse is broken. Fix fuse first. > > > > If that isn't what's happening, the next possible problem is that > > puffs/refus

Re: struct file reference at VFS level

2016-04-23 Thread Emmanuel Dreyfus
David Holland wrote: > If something in fuse is causing these cases to share the same open and > thus the same struct file, fuse is broken. Fix fuse first. > > If that isn't what's happening, the next possible problem is that > puffs/refuse/whatnot is the losing IO_NDELAY flag of VOP_WRITE. If so

Re: struct file reference at VFS level

2016-04-23 Thread David Holland
On Sat, Apr 23, 2016 at 09:20:28PM +0200, Emmanuel Dreyfus wrote: > > If something in fuse is causing these cases to share the same open and > > thus the same struct file, fuse is broken. Fix fuse first. > > The NetBSD VFS interface does not let underlying filesystem distinguish > different s

Re: struct file reference at VFS level

2016-04-23 Thread Emmanuel Dreyfus
David Holland wrote: > If something in fuse is causing these cases to share the same open and > thus the same struct file, fuse is broken. Fix fuse first. The NetBSD VFS interface does not let underlying filesystem distinguish different struct file for a given vnode. That information is just not

Re: struct file reference at VFS level

2016-04-23 Thread David Holland
On Fri, Apr 22, 2016 at 09:10:23AM +, Emmanuel Dreyfus wrote: > I talked to the glusterFS developer that hit the problem about the > requirement. This is to iplement mandatory locks, a feature not available > in UFS. UFS isn't relevant. > Quooted below is the scenario chere the problem

Re: struct file reference at VFS level

2016-04-23 Thread Greg Troxel
Joerg Sonnenberger writes: > On Fri, Apr 22, 2016 at 10:42:10AM -0400, Greg Troxel wrote: >> I still don't understand why this is about FUSE. What if a file were >> opened without O_NONBLOCK and then the same file were opened with? > > O_NONBLOCK is pretty much pointless for regular files. It o

Re: struct file reference at VFS level

2016-04-22 Thread Emmanuel Dreyfus
Emmanuel Dreyfus wrote: > Now let us see the locking scenario: > - process 1 opens file foo O_RDWR > - process 2 opens file foo O_RDWR|O_NONBLOCK > - process 1 locks a part of the file > - process 2 atteemps to write to the locked part > > With advisory locks (what we have in our filesystems),

Re: struct file reference at VFS level

2016-04-22 Thread Joerg Sonnenberger
On Fri, Apr 22, 2016 at 10:42:10AM -0400, Greg Troxel wrote: > I still don't understand why this is about FUSE. What if a file were > opened without O_NONBLOCK and then the same file were opened with? O_NONBLOCK is pretty much pointless for regular files. It only really changes something for sock

Re: struct file reference at VFS level

2016-04-22 Thread Emmanuel Dreyfus
On Fri, Apr 22, 2016 at 10:42:10AM -0400, Greg Troxel wrote: > I still don't understand why this is about FUSE. What if a file were > opened without O_NONBLOCK and then the same file were opened with? Why > isn't that the same situation, and the same bug? The filesystem below NetBSD VFS cannot m

Re: struct file reference at VFS level

2016-04-22 Thread Taylor R Campbell
Date: Fri, 22 Apr 2016 14:33:14 + From: Emmanuel Dreyfus On Fri, Apr 22, 2016 at 02:07:22PM +, Taylor R Campbell wrote: >I talked to the glusterFS developer that hit the problem about the >requirement. This is to iplement mandatory locks, a feature not available

Re: struct file reference at VFS level

2016-04-22 Thread Greg Troxel
Emmanuel Dreyfus writes: >> (...) proposed mandatory lock feature checks for locks during >> every data modifying fops. And hence we make use of fd flags too. Let >> me explain the scenario from my test case: >> >> Process 1 opens file 'foo' with O_RDWR and acquires a shared byte range >> lock.

Re: struct file reference at VFS level

2016-04-22 Thread Emmanuel Dreyfus
On Fri, Apr 22, 2016 at 02:07:22PM +, Taylor R Campbell wrote: >I talked to the glusterFS developer that hit the problem about the >requirement. This is to iplement mandatory locks, a feature not available >in UFS. > > Why is this file-system-specific? It is not filesystem-speci

Re: struct file reference at VFS level

2016-04-22 Thread Taylor R Campbell
Date: Fri, 22 Apr 2016 09:10:23 + From: Emmanuel Dreyfus On Wed, Apr 20, 2016 at 07:09:12AM -0400, Greg Troxel wrote: > I am not following this. My basic issue is that while I see your point > about open files in FUSE being different, I don't see how that's > different fro

Re: struct file reference at VFS level

2016-04-22 Thread Emmanuel Dreyfus
On Wed, Apr 20, 2016 at 07:09:12AM -0400, Greg Troxel wrote: > I am not following this. My basic issue is that while I see your point > about open files in FUSE being different, I don't see how that's > different from multiple open files in any other filesystem. > > What specific flags are you t

Re: struct file reference at VFS level

2016-04-20 Thread Greg Troxel
m...@netbsd.org (Emmanuel Dreyfus) writes: > FUSE methods on open files use a pointer on a file handle. The > filesystem uses it to track attribute about the open file. For instance, > if process 1 opens a file with a given flag and process 2 opens it > without it, the filesystem may need to act

struct file reference at VFS level

2016-04-20 Thread Emmanuel Dreyfus
Hello I have hit an implementation problem with FUSE on NetBSD which seems difficult to implement with current VFS and PUFFS interfaces. FUSE methods on open files use a pointer on a file handle. The filesystem uses it to track attribute about the open file. For instance, if process 1 opens a fil