[ https://issues.apache.org/jira/browse/IMPALA-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16888331#comment-16888331 ]
Anurag Mantripragada edited comment on IMPALA-8489 at 7/18/19 9:23 PM: ----------------------------------------------------------------------- [~bharathv] - As per your gerrit comments, I reproduced this locally and here is the sequence of steps to reproduce. {code:java} CREATE TABLE foo (c int) PARTITIONED BY (i int, p string); INSERT INTO TABLE foo PARTITION(i=1, p='p1') VALUES(1); -- Create a partition (i=002, p='p2') directory manually and add a file there -- ALTER TABLE foo RECOVER PARTITIONS; INVALIDATE METADATA; INSERT INTO TABLE test_recover_partitions PARTITION(i=002, p='p2') VALUES(4);{code} The final insert throws the exception as the partition ids have changed after a drop + recreate of partition (i=002, p='p2'). Catalogd logs below: {code:java} I0718 13:52:47.996079 20026 jni-util.cc:288] java.lang.IllegalArgumentException: no such partition id 6 at org.apache.impala.catalog.HdfsTable.loadPartitions(HdfsTable.java:343) at org.apache.impala.service.CatalogOpExecutor.createInsertEvents(CatalogOpExecutor.java:3885) at org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:3843) at org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:322) I0718 13:52:48.129122 20026 status.cc:124] IllegalArgumentException: no such partition id 6{code} was (Author: anuragmantri): [~bharathv] - As per your gerrit comments, I reproduced this locally and here is the sequence of steps to reproduce. {code:java} CREATE TABLE foo (c int) PARTITIONED BY (i int, p string); INSERT INTO TABLE foo PARTITION(i=1, p='p1') VALUES(1); -- Create a partition directory manually and add a file there ---- ALTER TABLE foo RECOVER PARTITIONS; INVALIDATE METADATA; INSERT INTO TABLE test_recover_partitions PARTITION(i=002, p='p2') VALUES(4);{code} The final insert throws the exception as the partition ids have changed after a drop + recreate of partition (i=002, p='p2'). Catalogd logs below: {code:java} I0718 13:52:47.996079 20026 jni-util.cc:288] java.lang.IllegalArgumentException: no such partition id 6 at org.apache.impala.catalog.HdfsTable.loadPartitions(HdfsTable.java:343) at org.apache.impala.service.CatalogOpExecutor.createInsertEvents(CatalogOpExecutor.java:3885) at org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:3843) at org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:322) I0718 13:52:48.129122 20026 status.cc:124] IllegalArgumentException: no such partition id 6{code} > TestRecoverPartitions.test_post_invalidate fails with IllegalStateException > when HMS polling is enabled > ------------------------------------------------------------------------------------------------------- > > Key: IMPALA-8489 > URL: https://issues.apache.org/jira/browse/IMPALA-8489 > Project: IMPALA > Issue Type: Bug > Components: Catalog > Affects Versions: Impala 3.3.0 > Reporter: Tim Armstrong > Assignee: Anurag Mantripragada > Priority: Critical > Labels: catalog-v2 > > {noformat} > metadata/test_recover_partitions.py:279: in test_post_invalidate > "INSERT INTO TABLE %s PARTITION(i=002, p='p2') VALUES(4)" % FQ_TBL_NAME) > common/impala_test_suite.py:620: in wrapper > return function(*args, **kwargs) > common/impala_test_suite.py:628: in execute_query_expect_success > result = cls.__execute_query(impalad_client, query, query_options, user) > common/impala_test_suite.py:722: in __execute_query > return impalad_client.execute(query, user=user) > common/impala_connection.py:180: in execute > return self.__beeswax_client.execute(sql_stmt, user=user) > beeswax/impala_beeswax.py:187: in execute > handle = self.__execute_query(query_string.strip(), user=user) > beeswax/impala_beeswax.py:364: in __execute_query > self.wait_for_finished(handle) > beeswax/impala_beeswax.py:385: in wait_for_finished > raise ImpalaBeeswaxException("Query aborted:" + error_log, None) > E ImpalaBeeswaxException: ImpalaBeeswaxException: > E Query aborted:IllegalArgumentException: no such partition id 6244 > {noformat} > The failure is reproducible for me locally with catalog v2. -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org