[ https://issues.apache.org/jira/browse/IMPALA-12812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822631#comment-17822631 ]
Csaba Ringhofer edited comment on IMPALA-12812 at 3/1/24 4:14 PM: ------------------------------------------------------------------ I was wrong about this one: " It also reloads the table (similarly to other DDLs) and detects new files in existing partitions. " At the moment no refresh is done on partitions that already exist in HMS. A valid workaround is to call both REFRESH after ALTER TABLE RECOVER PARTITIONS - REFRESH will both detect new files and send the reload event. Closing the issue as it wouldn't be that useful. was (Author: csringhofer): I was wrong about this one: "An HMS event is created for the new partitions but there is no event that would indicate that there are new files in existing partitions. " At the moment no refresh is done on partitions that already exist in HMS. A valid workaround is to call both REFRESH after ALTER TABLE RECOVER PARTITIONS - REFRESH will both detect new files and send the reload event. Closing the issue as it wouldn't be that useful. > Send reload event after ALTER TABLE RECOVER PARTITIONS > ------------------------------------------------------ > > Key: IMPALA-12812 > URL: https://issues.apache.org/jira/browse/IMPALA-12812 > Project: IMPALA > Issue Type: Improvement > Reporter: Csaba Ringhofer > Priority: Major > > IMPALA-11808 added support for sending reload events after REFRESH to allow > other Impala cluster connecting to the same HMS to also reload their tables. > REFRESH is often used when in external tables the files are written directly > to filesystem without notifying HMS, so Impala needs to update its cache and > can't rely on HMS notifications. > The same could be useful for ALTER TABLE RECOVER PARTITIONS. It detects > partition directories that were only created in the FS but not in HMS and > creates them in HMS too. It also reloads the table (similarly to other DDLs) > and detects new files in existing partitions. An HMS event is created for the > new partitions but there is no event that would indicate that there are new > files in existing partitions. As ALTER TABLE RECOVER PARTITIONS is called > when the user expects changes in the filesystem (similarly to REFRESH), it > could be useful to send a reload event after it is finished. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org