[ https://issues.apache.org/jira/browse/DRILL-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251940#comment-15251940 ]
ASF GitHub Bot commented on DRILL-2100: --------------------------------------- Github user vdiravka commented on a diff in the pull request: https://github.com/apache/drill/pull/454#discussion_r60585364 --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java --- @@ -554,7 +572,18 @@ public BatchGroup mergeAndSpill(LinkedList<BatchGroup> batchGroups) throws Schem c1.buildSchema(BatchSchema.SelectionVectorMode.NONE); c1.setRecordCount(count); - String outputFile = Joiner.on("/").join(dirs.next(), fileName, spillCount++); + String spillDir = dirs.next(); + Path currSpillPath = new Path(Joiner.on("/").join(spillDir, fileName)); + currSpillDirs.add(currSpillPath); + String outputFile = Joiner.on("/").join(currSpillPath, spillCount++); + try { + fs.deleteOnExit(currSpillPath); + } catch (IOException e) { + // since this is meant to be used in a batches's spilling, we don't propagate the exception + final String message = "Unable to mark spill directory \"" + currSpillPath + "\" to delete on exit"; + System.err.println(message + '.'); --- End diff -- Updated > Drill not deleting spooling files > --------------------------------- > > Key: DRILL-2100 > URL: https://issues.apache.org/jira/browse/DRILL-2100 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Relational Operators > Affects Versions: 0.8.0 > Reporter: Abhishek Girish > Assignee: Deneche A. Hakim > Fix For: 1.7.0 > > > Currently, after forcing queries to use an external sort by switching off > hash join/agg causes spill-to-disk files accumulating. > This causes issues with disk space availability when the spill is configured > to be on the local file system (/tmp/drill). Also not optimal when configured > to use DFS (custom). > Drill must clean up all temporary files created after a query completes or > after a drillbit restart. -- This message was sent by Atlassian JIRA (v6.3.4#6332)