[ https://issues.apache.org/jira/browse/IMPALA-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16704101#comment-16704101 ]
Tim Armstrong commented on IMPALA-7733: --------------------------------------- Also seen here: {noformat} custom_cluster.test_parquet_page_index.TestHdfsParquetTableIndexWriter.test_ctas_tables[protocol: beeswax | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'debug_action': None, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] (from pytest) Failing for the past 1 build (Since Failed#226 ) Took 1 min 19 sec. add description Error Message CalledProcessError: Command '['hdfs', 'dfs', '-get', '/test-warehouse/test_ctas_tables_7a125d10.db/test_hdfs_parquet_table_writer/', '/tmp/pytest-of-jenkins/pytest-2/test_ctas_tables_protocol__bee0/functional_parquet.widetable_1000_cols']' returned non-zero exit status 1 Stacktrace custom_cluster/test_parquet_page_index.py:330: in test_ctas_tables "functional_parquet.widetable_1000_cols", tmpdir) custom_cluster/test_parquet_page_index.py:266: in _ctas_table_and_verify_index self._validate_parquet_page_index(hdfs_path, tmpdir.join(source_table)) custom_cluster/test_parquet_page_index.py:227: in _validate_parquet_page_index row_group_indexes = self._get_row_groups_from_hdfs_folder(hdfs_path, tmpdir) custom_cluster/test_parquet_page_index.py:108: in _get_row_groups_from_hdfs_folder check_call(['hdfs', 'dfs', '-get', hdfs_path, tmpdir.strpath]) /usr/lib64/python2.7/subprocess.py:542: in check_call raise CalledProcessError(retcode, cmd) E CalledProcessError: Command '['hdfs', 'dfs', '-get', '/test-warehouse/test_ctas_tables_7a125d10.db/test_hdfs_parquet_table_writer/', '/tmp/pytest-of-jenkins/pytest-2/test_ctas_tables_protocol__bee0/functional_parquet.widetable_1000_cols']' returned non-zero exit status 1 {noformat} > TestInsertParquetQueries.test_insert_parquet is flaky in S3 due to rename > ------------------------------------------------------------------------- > > Key: IMPALA-7733 > URL: https://issues.apache.org/jira/browse/IMPALA-7733 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure > Affects Versions: Impala 3.1.0 > Reporter: Vuk Ercegovac > Assignee: Tianyi Wang > Priority: Blocker > Labels: broken-build, flaky > > I see two examples in the past two months or so where this test fails due to > a rename error on S3. The test's stacktrace looks like this: > {noformat} > query_test/test_insert_parquet.py:112: in test_insert_parquet > self.run_test_case('insert_parquet', vector, unique_database, > multiple_impalad=True) > common/impala_test_suite.py:408: in run_test_case > result = self.__execute_query(target_impalad_client, query, user=user) > common/impala_test_suite.py:625: in __execute_query > return impalad_client.execute(query, user=user) > common/impala_connection.py:160: in execute > return self.__beeswax_client.execute(sql_stmt, user=user) > beeswax/impala_beeswax.py:176: in execute > handle = self.__execute_query(query_string.strip(), user=user) > beeswax/impala_beeswax.py:350: in __execute_query > self.wait_for_finished(handle) > beeswax/impala_beeswax.py:371: in wait_for_finished > raise ImpalaBeeswaxException("Query aborted:" + error_log, None) > E ImpalaBeeswaxException: ImpalaBeeswaxException: > E Query aborted:Error(s) moving partition files. First error (of 1) was: > Hdfs op (RENAME > s3a://<removed>/test_insert_parquet_968f37fe.db/orders_insert_table/_impala_insert_staging/4e45cd68bcddd451_3c7156ed00000000/.4e45cd68bcddd451-3c7156ed00000002_803672621_dir/4e45cd68bcddd451-3c7156ed00000002_448261088_data.0.parq > TO > s3a://<removed>/test-warehouse/test_insert_parquet_968f37fe.db/orders_insert_table/4e45cd68bcddd451-3c7156ed00000002_448261088_data.0.parq) > failed, error was: > s3a://<removed>/test-warehouse/test_insert_parquet_968f37fe.db/orders_insert_table/_impala_insert_staging/4e45cd68bcddd451_3c7156ed00000000/.4e45cd68bcddd451-3c7156ed00000002_803672621_dir/4e45cd68bcddd451-3c7156ed00000002_448261088_data.0.parq > E Error(5): Input/output error{noformat} > Since we know this happens once in a while, some ideas to deflake it: > * retry > * check for this specific issue... if we think its platform flakiness, then > we should skip it. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org