[jira] [Updated] (HIVE-23140) Optimise file move in CTAS
[ https://issues.apache.org/jira/browse/HIVE-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23140: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Thanks [~ashutoshc]. Committed to master. > Optimise file move in CTAS > --- > > Key: HIVE-23140 > URL: https://issues.apache.org/jira/browse/HIVE-23140 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-23140.1.patch > > > FileSinkOperator can be optimized to run file move operation (/_tmp.-ext --> > /-ext-) in parallel fashion. Currently it invokes > {{Utilities.moveSpecifiedFileStatus}} and renames in sequential mode causing > delays in cloud storage. FS rename can be used (S3A internally has parallel > rename operation). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23140) Optimise file move in CTAS
[ https://issues.apache.org/jira/browse/HIVE-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23140: Description: FileSinkOperator can be optimized to run file move operation (/_tmp.-ext --> /-ext-) in parallel fashion. Currently it invokes {{Utilities.moveSpecifiedFileStatus}} and renames in sequential mode causing delays in cloud storage. FS rename can be used (S3A internally has parallel rename operation). (was: FileSinkOperator can be optimized to run file move operation (/_tmp.-ext --> /-ext-10002) in parallel fashion. Currently it invokes {{Utilities.moveSpecifiedFileStatus}} and renames in sequential mode causing delays in cloud storage. FS rename can be used (S3A internally has parallel rename operation). ) > Optimise file move in CTAS > --- > > Key: HIVE-23140 > URL: https://issues.apache.org/jira/browse/HIVE-23140 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-23140.1.patch > > > FileSinkOperator can be optimized to run file move operation (/_tmp.-ext --> > /-ext-) in parallel fashion. Currently it invokes > {{Utilities.moveSpecifiedFileStatus}} and renames in sequential mode causing > delays in cloud storage. FS rename can be used (S3A internally has parallel > rename operation). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23140) Optimise file move in CTAS
[ https://issues.apache.org/jira/browse/HIVE-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23140: Attachment: HIVE-23140.1.patch > Optimise file move in CTAS > --- > > Key: HIVE-23140 > URL: https://issues.apache.org/jira/browse/HIVE-23140 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-23140.1.patch > > > FileSinkOperator can be optimized to run file move operation (/_tmp.-ext --> > /-ext-10002) in parallel fashion. Currently it invokes > {{Utilities.moveSpecifiedFileStatus}} and renames in sequential mode causing > delays in cloud storage. FS rename can be used (S3A internally has parallel > rename operation). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23140) Optimise file move in CTAS
[ https://issues.apache.org/jira/browse/HIVE-23140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23140: Status: Patch Available (was: Open) > Optimise file move in CTAS > --- > > Key: HIVE-23140 > URL: https://issues.apache.org/jira/browse/HIVE-23140 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-23140.1.patch > > > FileSinkOperator can be optimized to run file move operation (/_tmp.-ext --> > /-ext-10002) in parallel fashion. Currently it invokes > {{Utilities.moveSpecifiedFileStatus}} and renames in sequential mode causing > delays in cloud storage. FS rename can be used (S3A internally has parallel > rename operation). -- This message was sent by Atlassian Jira (v8.3.4#803005)