[ https://issues.apache.org/jira/browse/HIVE-22411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16963676#comment-16963676 ]
Hive QA commented on HIVE-22411: -------------------------------- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 12s{color} | {color:red} /data/hiveptest/logs/PreCommit-HIVE-Build-19222/patches/PreCommit-HIVE-Build-19222.patch does not apply to master. Rebase required? Wrong Branch? See http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-19222/yetus.txt | | Powered by | Apache Yetus http://yetus.apache.org | This message was automatically generated. > Performance degradation on single row inserts > --------------------------------------------- > > Key: HIVE-22411 > URL: https://issues.apache.org/jira/browse/HIVE-22411 > Project: Hive > Issue Type: Bug > Components: Hive > Reporter: Attila Magyar > Assignee: Attila Magyar > Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22411.1.patch, Screen Shot 2019-10-17 at 8.40.50 > PM.png > > > Executing single insert statements on a transactional table effects write > performance on a s3 file system. Each insert creates a new delta directory. > After each insert hive calculates statistics like number of file in the table > and total size of the table. In order to calculate these, it traverses the > directory recursively. During the recursion for each path a separateĀ > listStatus call is executed. In the end the more delta directory you have the > more time it takes to calculate the statistics. > Therefore insertion time goes up linearly: > !Screen Shot 2019-10-17 at 8.40.50 PM.png|width=601,height=436! > The fix is to useĀ fs.listFiles(path, /**recursive**/ true) instead the > handcrafter recursive method/ -- This message was sent by Atlassian Jira (v8.3.4#803005)