vikramahuja1001 commented on PR #5890: URL: https://github.com/apache/hive/pull/5890#issuecomment-3000316790
@Aggarwal-Raghav , thanks for testing out the patch HIVE-28956 with along with HIVE-28909 and HIVE-28972. When i was working on the fix and tested the same, the 2 patches were not merged. From the above tests, post HIVE-28972 it seems that the JDO performance is much better than the current DirectSql Implementation. > @Aggarwal-Raghav, do we know what change in [HIVE-28956](https://issues.apache.org/jira/browse/HIVE-28956) introduced a perf degradation: batching or `alterPartitions` API? cc [vikramahuja1001](https://github.com/vikramahuja1001) @deniskuzZ , in HIVE-28956, i had moved from alterPartition API to alterPartitions API with batching. As is the case currently, alterPartition does not support DirectSql. > @Aggarwal-Raghav, @dengzhhu653, do we have an agreement on [HIVE-28956](https://issues.apache.org/jira/browse/HIVE-28956) revert? If yes, I can do that. > > Also, do we have a benchmark in metastore-tools for `alter table add column cascade`? Or how do you find the regression post DataNucleus upgrade, private setup ? Though reverting the patch HIVE-28956 would definitely improve the performance of alter table add column cascade operation, but it would still not fix the issue of DirectSql being slower than JDO in the alterPartitions API which currently is being used in other flows as well, for instance: alter table rename operation. This use case of rename table should also potentially suffer from the same issue, based on the analysis in the above comments. HIVE-17969 is doing the similar change of moving from alterPartition API to alterPartitions API in the alter table rename flow. We should also perhaps look and compare the performance of the rename operation as well in JDO vs DirectSql and then take a call. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
