> From: Aurelien RAINONE [mailto:aurelien.rain...@gmail.com] > Sent: Friday, December 15, 2017 9:30 AM > To: Nfs-ganesha-devel@lists.sourceforge.net > Subject: [Nfs-ganesha-devel] Implement a FSAL for S3-compatible storage > > Hi, > > I'm new to nfs-ganesha and on this mailing-list. I'm currently reading a > maximum of documentation and code in order to understand where to start > the implementation of a new FSAL. > > The main objectives of my development are: > - connect one fsal export to exactly one S3 Bucket. Requests to the S3 host > will be performed with libs3 (github.com/bji/libs3). > - make proper use of cache in order to not overload the network connection > to the S3 host. > - no need for full ACL support, maybe also no ACL at all, at least at the > beggining. > > My new FSAL would be based on v2.5.4 stable branch. > > Here are my surely very basic questions: > > 1) I read that FSAL API has undergone major refactorings in the last > months/years. Is the 2.5.4 branch recent enough for my needs?
Should be reasonable, though you might want to develop against next. I strongly encourage development in tree. > 2) I intent to start developing the new FSAL over FSAL_RGW? Is my choice > correct ? I'm not quite sure what you mean by that. > I read those recommendations in a recent thread on this mailing-list. > Somebody wanted to implement a new FSAL and this was answered: > > - You need to be able to form a handle (preferably no larger than about 40 > bytes) which is persistent across Ganesha process restarts and server > reboots. > > - You need to be able to find a file using this handle > > Can you explain why the need for handle persistence? If an NFS server crashes while clients are mounted, the clients will have inodes cached (possibly with open files) with handles cached. The client may no longer be able to do a lookup by name, the only reference it has to the file is the file handle. If that file handle no longer is usable to reference the file, then the client and the applications are basically out of luck. > 3) Neither the S3 API nor libs3 have the concept of file handle, I mean the > only way to identify a file with S3 API and libs3 is through its name, a > string of > characters that may go over 40 bytes. A hash may be a workable solution, the trick is how to handle collisions reasonably. Persistence may be challenging with collisions (what happens if the client has cached the file handle from the first instance. Now someone accesses a file that produces a collision. Now the server crashes, and in the meantime, the first file is removed. The first client's handle now would reference a different file (assuming you have persisted something), and the second client's handle now doesn't point to anything. A persistent mapping database might solve the issue. > Does hashing the filename and keeping a map of hashes <-> paths seems like > a good idea to you? > > 4) Neither the S3 API nor libs3 have the concept of directory. Directories are > just implied when a path contains some slashes or finishes with a slash. Any > recommandations/ideas about how to treat this specific point For FSAL_RGW, the RGW library has synthetized the directory structure, that is a good way to go if you have a significant number of files. Frank --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
