[systemd-devel] Suggestion for a lowlevel fsnotify change daemon.
Hi all, for some time I have been looking at the issue why fsnotify does not work with network filesystems and FUSE (with a shared backend). I've found out that changes initiated on the localhost, on the filesystem are supported by the fs change subsystems on Linux, and events initiated at the backend (from another host with network fs) are not detected. This is because the filesystem are not aware a watch has been set on an inode, and thus cannot act on it. (if they act if they are aware is another question). I've tried to tackle this in the kernel. I've made this working with a FUSE: - when a watch is set on a FUSE fs, a message is forwarded to the userspace daemon containing the inode and the mask. I had to add a opcode FUSE_FSNOTIFY. - the fuse fs has to react in it, by setting a watch on the backend. I wrote a simple overlay fs, and setting a watch on the backend is simple - I had to add some calls to the fuse library to push changes to the VFS where there is no direct related call from the VFS. (files are added and/or files are changed) - the FUSE kernel module in VFS has to trigger fsnotify call when events are pushed to the VFS by the userspace daemon. This worked but is I think not the best way to deal with it. My suggestion it to write a fs notify change service which does all the watching for clients, like there are already services for desktops right now. This service should also work with a console app like mc, but also with desktop environments like Gnome and KDE. It should also be able to forward a watch to a filesystem like FUSE and cifs and nfs, so that they know a watch has been set. They can act then on it, by forwarding the watch to the backend. SMB does upport this, NFS4 also, and you can make FUSE also support it(depending the protocol). When the fs receives an event, it can send it back to the fs notify change service, which informs the client(s). This way the filesystem also stays up to date. To forward a watch and to read to incoming fsevents, a socket/filedescriptor is required. A FUSE fs can easily connect to it at startup, the in kernel filesystems need some extra. Via mountoptions parse the fd to the kernel? Is this something what can be added to systemd? Please let me know what you think of it. Stef ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Suggestion for a lowlevel fsnotify change daemon.
At first look, this seems very similar to FAM (which even supported NFSv3, using custom notifications over SunRPC). Later I remember GNOME replaced it with Gamin and finally with local-only inotify inside glib/gvfs. It might be useful to revive it, both inotify and fanotify have problems. But I guess security would be a problem (how to determine which users may receive which events). -- Mantas Mikulėnas On Jul 28, 2015 18:46, Stef Bon stef...@gmail.com wrote: Hi all, for some time I have been looking at the issue why fsnotify does not work with network filesystems and FUSE (with a shared backend). I've found out that changes initiated on the localhost, on the filesystem are supported by the fs change subsystems on Linux, and events initiated at the backend (from another host with network fs) are not detected. This is because the filesystem are not aware a watch has been set on an inode, and thus cannot act on it. (if they act if they are aware is another question). I've tried to tackle this in the kernel. I've made this working with a FUSE: - when a watch is set on a FUSE fs, a message is forwarded to the userspace daemon containing the inode and the mask. I had to add a opcode FUSE_FSNOTIFY. - the fuse fs has to react in it, by setting a watch on the backend. I wrote a simple overlay fs, and setting a watch on the backend is simple - I had to add some calls to the fuse library to push changes to the VFS where there is no direct related call from the VFS. (files are added and/or files are changed) - the FUSE kernel module in VFS has to trigger fsnotify call when events are pushed to the VFS by the userspace daemon. This worked but is I think not the best way to deal with it. My suggestion it to write a fs notify change service which does all the watching for clients, like there are already services for desktops right now. This service should also work with a console app like mc, but also with desktop environments like Gnome and KDE. It should also be able to forward a watch to a filesystem like FUSE and cifs and nfs, so that they know a watch has been set. They can act then on it, by forwarding the watch to the backend. SMB does upport this, NFS4 also, and you can make FUSE also support it(depending the protocol). When the fs receives an event, it can send it back to the fs notify change service, which informs the client(s). This way the filesystem also stays up to date. To forward a watch and to read to incoming fsevents, a socket/filedescriptor is required. A FUSE fs can easily connect to it at startup, the in kernel filesystems need some extra. Via mountoptions parse the fd to the kernel? Is this something what can be added to systemd? Please let me know what you think of it. Stef ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Suggestion for a lowlevel fsnotify change daemon.
On 28/07/15 17:28, Mantas Mikulėnas wrote: At first look, this seems very similar to FAM (which even supported NFSv3, using custom notifications over SunRPC). Later I remember GNOME replaced it with Gamin and finally with local-only inotify inside glib/gvfs. What GLib actually uses is an abstraction with multiple backends, including inotify (the default on Linux) and FAM; so in principle it could have a backend for some new thing, or even use inotify for normal local filesystems and the new backend for other mounts. However... On Jul 28, 2015 18:46, Stef Bon stef...@gmail.com wrote: - I had to add some calls to the fuse library to push changes to the VFS where there is no direct related call from the VFS. (files are added and/or files are changed) - the FUSE kernel module in VFS has to trigger fsnotify call when events are pushed to the VFS by the userspace daemon. If you're adding a monitoring/notification call to FUSE, would it be out of the question for the user-space API to it to be exactly use inotify? (Or fanotify, or whatever is believed to be the right user-space API for file monitoring these days.) It should also be able to forward a watch to a filesystem like FUSE and cifs and nfs, so that they know a watch has been set. They can act then on it, by forwarding the watch to the backend. SMB does upport this, NFS4 also, and you can make FUSE also support it(depending the protocol). If the in-kernel implementation of NFS or CIFS isn't enhanced to support monitoring, this can't work. If the in-kernel implementation of NFS or CIFS *is* enhanced to support monitoring, is there any reason for the kernel not to present the resulting information to user-space via inotify? In other words, is there a reason why a user-space service is necessary? (I realise that one possible reason for a user-space service is so that it can aggregate all the periodic polling, on filesystems that don't have anything better you can do - that's why FAM had a daemon, if I remember correctly.) -- Simon McVittie Collabora Ltd. http://www.collabora.com/ ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Suggestion for a lowlevel fsnotify change daemon.
2015-07-28 19:20 GMT+02:00 Simon McVittie simon.mcvit...@collabora.co.uk: On 28/07/15 17:28, Mantas Mikulėnas wrote: At first look, this seems very similar to FAM (which even supported NFSv3, using custom notifications over SunRPC). Later I remember GNOME replaced it with Gamin and finally with local-only inotify inside glib/gvfs. No, what I propose is a fs notify change daemon which is able to forward a watch to individual filesystems, and can listen to events coming from these filesystems from the backend. Gamin can't do that. It's not maintained anymore and difficult to understand. What GLib actually uses is an abstraction with multiple backends, including inotify (the default on Linux) and FAM; so in principle it could have a backend for some new thing, or even use inotify for normal local filesystems and the new backend for other mounts. I'm afraid I do not understand you here. - the FUSE kernel module in VFS has to trigger fsnotify call when events are pushed to the VFS by the userspace daemon. If you're adding a monitoring/notification call to FUSE, would it be out of the question for the user-space API to it to be exactly use inotify? (Or fanotify, or whatever is believed to be the right user-space API for file monitoring these days.) I've written that I've tried this, and also that I stopped this idea, cause it's not the way to go. But for your understanding, the underlying subsystem is fsnotify, and it does handle inotify and/or fanotify, whatever is used. So userspace should use the fsnotify api. It should also be able to forward a watch to a filesystem like FUSE and cifs and nfs, so that they know a watch has been set. They can act then on it, by forwarding the watch to the backend. SMB does upport this, NFS4 also, and you can make FUSE also support it(depending the protocol). If the in-kernel implementation of NFS or CIFS isn't enhanced to support monitoring, this can't work. At this moment this can't work, but for CIFS in the past it worked (with dnotiify). See: See line 6459 in cifssmb.c in /fs/cifs in the kernel. It is disabled for now. But support in SMB (SMB servers do support it). I know that the NFS4 protocol also supports it. You must understand that the protocols do support it, but it does not work with Linux, cause nobody has tried it yet, and fsnotify does not let the individual filesystems know that there is a watch set. There are network filesystems/protocols like webdav which do not support the setting of a watch and getting fsevents from the backend. Webdav is build upon HTTP, and the current version does not support the pushing of events from the server to the client. We have to wait for version 2 of the HTTP protocol, as the main developer of the webdav proto told me Joe Orton. If the in-kernel implementation of NFS or CIFS *is* enhanced to support monitoring, is there any reason for the kernel not to present the resulting information to user-space via inotify? In other words, is there a reason why a user-space service is necessary? Like I mentioned in the first post and here again, the filesystems like FUSE, NFS and CIFS (and other) are not contacted by fsnotify about a watch. This is by design. So if the kernel does not do this (and beleive me there are good reasons for), you have to do this in userspace. Stef Bon (I realise that one possible reason for a user-space service is so that it can aggregate all the periodic polling, on filesystems that don't have anything better you can do - that's why FAM had a daemon, if I remember correctly.) Yes that's one reason. But there are more reasons to do this in userspace, see above. Stef ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
