Hi All, I'm trying to use the new framework to speed up lookups/attr/xattr operations by split functionality between fast/slow execution paths. I'd highly appreciate if you could suggest experiments to evaluate the performance improvement.
As I mentioned in my previous email, I'm caching replies from fuse daemon (hashed key/value blobs) in the kernel so that for the same key (e.g., <parent ino, child name> in case of FUSE_LOOKUP), the reply (e.g., fuse_entry_out) is served from the kernel itself and no call is delivered to user-space. While this may seem redundant due to entry_timeout/attr_timeout caching that already exists in FUSE, this design provides more control to the user-space daemon over when/what to invalidate. For instance, entry_timeout caching is only valid until a timeout or until the kernel removes dentry from its dcache. For invalidation, fuse_lowlevel_notify_inval_entry() can also remove entries from the hash table. Please refer to the figure attached in my last email. Thanks, Riya On Tue, Apr 3, 2018 at 1:45 PM, riya khanna <riyakhanna1...@gmail.com> wrote: > I'm attaching a figure that depicts the architecture of my optimized fuse > framework. Kindly let me know if you have any questions. > > On Mon, Apr 2, 2018 at 10:57 AM, riya khanna <riyakhanna1...@gmail.com> > wrote: > >> Thanks Amar! Please see my answers inline. >> >> On Mon, Apr 2, 2018 at 5:41 AM, Amar Tumballi <atumb...@redhat.com> >> wrote: >> >>> Hi Riya, >>> >>> Thanks for writing to us. Some questions before we start on this. >>> >>> * Where can we see your work of modifying the fuse module to cache the >>> calls? Some reference would help us to provide more specific pointers. (or >>> ask better questions). >>> >>> I've created a fast path framework for FUSE, where the user space daemon >> can load a module and register handlers for file operations (lookup, open, >> r/w, etc.) that must be handled in the kernel itself without an up call to >> the user space. I call them fast path handlers. This design also retains >> the regular FUSE handlers for file system operations in upser-space (slow >> path). The fast path and slow path can communicate with each other over >> shared memory or using syscalls to enable/invalidate caching of data >> structs (e.g., results of getattr, getxattr, etc.) >> >> There's a process I need to follow in order to make the code available >> publicly. I've already started, but will take some time. I will try to do >> this asap. >> >> * If the caching happened in fuse module, and it expects the regular >>> arguments as the parameters, then there may not be any work required at all >>> in glusterfs, as it works on low-level fuse api. >>> >>> >> The fast handlers expect same interface and args (fuse_args) as the >> regular user-space daemon. The fast handler code is fs-specific, therefore, >> must come from glusterfs. Changes are also needed in glusterfs code to >> communicate with the fast path for enabling/invalidating caching. >> >> >>> * Also, how to invalidate caches from userspace program? because >>> GlusterFS could be accessed from multiple clients, so it becomes an >>> important piece to have. >>> >>> >> Server invalidate can trigger a system call into the fast path framework >> to invalidate caches. >> >> >>> For referring at the codebase to look at integration with fuse module, >>> please check the directory 'xlators/mount/fuse/src/' and mostly the file >>> 'fuse-bridge.c'. >>> >>> Thanks for your interest in project, would be great to collaborate on >>> this effort, as it can enhance the performance of glusterfs in many >>> usecases. >>> >> >> I'm still going through gluster developer documentation, but it'd be >> helpful if you could mention what kind of use cases does the fast/slow >> split FUSE framework enable? i've already applied the framework to >> accelerate multiple FUSE-based stackable file systems, but want the >> interface to be generic enough for all FUSE file systems to take advantage >> of it. >> >> >>> Regards, >>> Amar >>> >>> >>> >>> >>> On Mon, Apr 2, 2018 at 6:34 AM, riya khanna <riyakhanna1...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> I've modified FUSE framework to take a part of user-space daemon code >>>> and moves it into the kernel fuse driver to minimize user-kernel-user >>>> switches during file system operations. An example would be caching >>>> getattr/getxattr/lookup/security checks etc. This design, therefore, >>>> create fast (served directly from the kernel) and a slow (regular fuse) >>>> execution paths. The fast and slow paths can also communicate with each >>>> other using shared memory. >>>> >>>> I was wondering if it is possible to accelerate glusterfs using this >>>> design. What pieces could (should) be easily moved to kernel space? >>>> Any pointers would be highly appreciated. Thanks! >>>> >>>> -Riya >>>> >>>> _______________________________________________ >>>> Gluster-devel mailing list >>>> Gluster-devel@gluster.org >>>> http://lists.gluster.org/mailman/listinfo/gluster-devel >>>> >>> >>> >>> >>> -- >>> Amar Tumballi (amarts) >>> >> >> >
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel