[ https://issues.apache.org/jira/browse/HIVE-22832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marton Bod updated HIVE-22832: ------------------------------ Description: Inside Utilities::handleDirectInsertTableFinalPath, the cleanDirectInsertDirectories method is called sequentially for each element of the directInsertDirectories list, which might have a large number of elements depending on how many partitions were written. This current sequential execution could be improved by parallelising the clean up process. (was: Inside Utilities::handleMmTableFinalPath, the cleanMmDirectories method is called sequentially for each element of the mmDirectory list, which might have a large number of elements depending on how many partitions were written. This current sequential execution could be improved by parallelising the clean up process. ) > Parallelise direct insert directory cleaning process > ---------------------------------------------------- > > Key: HIVE-22832 > URL: https://issues.apache.org/jira/browse/HIVE-22832 > Project: Hive > Issue Type: Improvement > Reporter: Marton Bod > Assignee: Marton Bod > Priority: Major > Attachments: HIVE-22832.1.patch, HIVE-22832.2.patch > > > Inside Utilities::handleDirectInsertTableFinalPath, the > cleanDirectInsertDirectories method is called sequentially for each element > of the directInsertDirectories list, which might have a large number of > elements depending on how many partitions were written. This current > sequential execution could be improved by parallelising the clean up process. -- This message was sent by Atlassian Jira (v8.3.4#803005)