mdayakar commented on PR #6651: URL: https://github.com/apache/hive/pull/6651#issuecomment-5129398575
> Check the context of [HIVE-24428](https://issues.apache.org/jira/browse/HIVE-24428), the Metastore server won't delete the created directory upon dynamic partitions from different clients collide, and this seems to be a in-process lock, do we really need it to coordinate clients? Thanks for the review @dengzhhu653. As per my understanding, [here](https://github.com/apache/hive/blob/eb0ea730493e9a5ee8db47ac7def6188bc07775e/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/AddPartitionsHandler.java#L350) partition folder in the storage is getting created and they maintain Map<PartValEqWrapperLite, Boolean> to store whether the partition folder is successfully created or not (value attribute in the map). In case of any exception while adding the partition, [here](https://github.com/apache/hive/blob/eb0ea730493e9a5ee8db47ac7def6188bc07775e/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/handler/AddPartitionsHandler.java#L203) the directory is getting cleaned based on the boolean value present in the map for that particular partition. There will be below two scenarios, 1. If multiple clients connecting to different HMS instances, in this case only the client which created the data folder can delete it in case of any errors/exceptions. 2. If multiple clients connecting to same HMS instance, in this case if don't provide synchronization logic then there can be a chance that one client can delete the data folder even though partition is added successfully. Please let me know whether my understanding is correct or I am missing something. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
