[GitHub] [tez] shameersss1 commented on a change in pull request #60: TEZ-3363: Delete intermediate data at the vertex level for Shuffle Handler

[GitBox]

shameersss1 commented on a change in pull request #60:
URL: https://github.com/apache/tez/pull/60#discussion_r821871579



##########
File path: 
tez-plugins/tez-aux-services/src/main/java/org/apache/tez/auxservices/ShuffleHandler.java
##########
@@ -1256,6 +1280,29 @@ private String getBaseLocation(String jobId, String 
dagId, String user) {
       return baseStr;
     }
 
+    /**
+     * Delete shuffle data in task directories belonging to a vertex.
+     */
+    private void deleteTaskDirsOfVertex(String jobId, String dagId, String 
vertexId, String user) throws IOException {
+      String baseStr = getBaseLocation(jobId, dagId, user);
+      FileContext lfc = FileContext.getLocalFSFileContext();
+      for(Path dagPath : lDirAlloc.getAllLocalPathsToRead(baseStr, conf)) {
+        RemoteIterator<FileStatus> status = lfc.listStatus(dagPath);
+        final JobID jobID = JobID.forName(jobId);
+        String taskDirPrefix = "attempt" + jobID.toString().replace("job", "") 
+

Review comment:
       ack. I will fix this in next revision.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@tez.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org