[ https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13606335#comment-13606335 ]
Matthew Farrellee commented on HADOOP-9371: ------------------------------------------- [~ste...@apache.org] Does delete(path, true) need to be atomic? My research suggests that only the HDFS implementation is atomic. (Note: current = r2.0.3-alpha on 2013-02-15 19:41) http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html FilterFileSystem - delegates w/o locking ChecksumFileSystem - delegates w/o locking LocalFileSystem - inherits, delegates to RawLocalFileSystem HarFileSystem - not implemented FTPFileSystem - FTPClient.removeDirectory w/o locking KosmosFileSystem - (not on trunk) no locking NativeS3FileSystem - no locking (even createParent()s to avoid errors, weird) RawLocalFileSystem - uses File.delete (if isFile) and FileUtil.fullyDelete w/o locking S3FileSystem - no locking ViewFileSystem - partial eval, no locking on top level - ChRootFileSystem uses RawLocalFileSystem http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/AbstractFileSystem.html AbstractFileSystem (uses FileContext.delete) FilterFs - delegates w/o locking ChecksumFs - delegates w/o locking LocalFs - inherits, delegates to RawLocalFs DelegateToFileSystem - delegates w/o locking RawLocalFs - inherits, delegates to RawLocalFileSystem FtpFs - inherits, delegates to FTPFileSystem ViewFs - partial eval, no locking at top level FileContext.delete - no hint of atomic requirement, delegates to AbstractFileSystem Side note - it's interesting to see how many FS implementations make their way back to RawLocalFileSystem, sometimes through 3+ layers of indirection. > Define Semantics of FileSystem and FileContext more rigorously > -------------------------------------------------------------- > > Key: HADOOP-9371 > URL: https://issues.apache.org/jira/browse/HADOOP-9371 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs > Affects Versions: 1.2.0, 3.0.0 > Reporter: Steve Loughran > Assignee: Steve Loughran > Attachments: HADOOP-9361.2.patch, HADOOP-9361.patch, > HadoopFilesystemContract.pdf > > Original Estimate: 48h > Remaining Estimate: 48h > > The semantics of {{FileSystem}} and {{FileContext}} are not completely > defined in terms of > # core expectations of a filesystem > # consistency requirements. > # concurrency requirements. > # minimum scale limits > Furthermore, methods are not defined strictly enough in terms of their > outcomes and failure modes. > The requirements and method semantics should be defined more strictly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira