[GitHub] [hadoop] steveloughran commented on pull request #2280: HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-14 Thread GitBox 


steveloughran commented on pull request #2280:
URL: https://github.com/apache/hadoop/pull/2280#issuecomment-691930699


   This patch is incomplete. Reverted and will create a new one.  
   
   Issues
   * cleanup on a multipart fail will tombstone everything; no information at 
that point as to what is/isn't a dir
   * delete needs more coverage
   
   I think I may want to push this delete/no-delete logic into s3guard -it can 
do the check rather than relying on the layer above. But: we will need to pass 
down for each path whether a path is a dir or a file, so that s3guard doesn't 
do a DDB query on every single path entry to work that out for itself



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] steveloughran commented on pull request #2280: HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-10 Thread GitBox 


steveloughran commented on pull request #2280:
URL: https://github.com/apache/hadoop/pull/2280#issuecomment-690402713


   repeatedly tested against london with options showing the error and the 
-Dkeep option *and unguarded*. Some transient failures related to local network 
issues, addressed in HADOOP-17181.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] steveloughran commented on pull request #2280: HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-09 Thread GitBox 


steveloughran commented on pull request #2280:
URL: https://github.com/apache/hadoop/pull/2280#issuecomment-689484740


   Note: I ran the hdfs contract tests in my IDE to verify it was happy too; 
all good. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] steveloughran commented on pull request #2280: HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-08 Thread GitBox 


steveloughran commented on pull request #2280:
URL: https://github.com/apache/hadoop/pull/2280#issuecomment-688851531


   broke the local fs tests
   ```
   [ERROR] Failures: 
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryAsEmptyDirectory:1192
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryAsFile:1241
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryAsNonEmptyDirectory:1216
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryToItself:1112
 Renamed directory to itself
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameDirectoryToNonExistentParent:1135
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameFileAsExistingDirectory:1093
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameFileAsExistingFile:1072
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameFileToDestinationWithParentFile:1018
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameFileToItself:1047
 Renamed file to itself
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameFileToNonExistentDirectory:994
 Expected exception was not thrown
   [ERROR]   
TestLocalFSFileContextMainOperations>FileContextMainOperationsBaseTest.testRenameNonExistentPath:971
 Should throw FileNotFoundException
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameDirectoryAsEmptyDirectory:1192
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameDirectoryAsFile:1241
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameDirectoryAsNonEmptyDirectory:1216
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameDirectoryToItself:1112
 Renamed directory to itself
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameDirectoryToNonExistentParent:1135
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameFileAsExistingDirectory:1093
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameFileAsExistingFile:1072
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameFileToDestinationWithParentFile:1018
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameFileToItself:1047
 Renamed file to itself
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameFileToNonExistentDirectory:994
 Expected exception was not thrown
   [ERROR]   
TestFcMainOperationsLocalFs>FileContextMainOperationsBaseTest.testRenameNonExistentPath:971
 Should throw FileNotFoundException
   [INFO] 
   [ERROR] Tests run: 4439, Failures: 22, Errors: 0, Skipped: 165
   ```



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[GitHub] [hadoop] steveloughran commented on pull request #2280: HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

2020-09-07 Thread GitBox 


steveloughran commented on pull request #2280:
URL: https://github.com/apache/hadoop/pull/2280#issuecomment-688475543


   Tests in progress



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org