[ https://issues.apache.org/jira/browse/IMPALA-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tianyi Wang resolved IMPALA-3887. --------------------------------- Resolution: Fixed IMPALA-3887: Wait for HDFS replication in data loading When the data loading finishes, it is possible for some HDFS blocks to be under replicated. If impala gets the metadata before the replication is done, some tests may fail. This patch adds a replication waiting step in the data loading script. Resubmitted with filesystem type check. Change-Id: I64d9a8ea1d0a32b40047321b50a7139a8f48eac8 Reviewed-on: http://gerrit.cloudera.org:8080/8916 Reviewed-by: Vuk Ercegovac <vercego...@cloudera.com> Reviewed-by: Alex Behm <alex.b...@cloudera.com> Tested-by: Impala Public Jenkins > Planner tests failing due to metadata loading race with HDFS, fewer #hosts > than expected > ---------------------------------------------------------------------------------------- > > Key: IMPALA-3887 > URL: https://issues.apache.org/jira/browse/IMPALA-3887 > Project: IMPALA > Issue Type: Bug > Components: Infrastructure > Affects Versions: Impala 2.6.0, Impala 2.11.0 > Reporter: Jim Apple > Assignee: Tianyi Wang > Priority: Blocker > Labels: broken-build, flaky > Fix For: Impala 2.12.0 > > > {noformat} > 11:22:33.349 =================================== FAILURES > =================================== > 11:22:33.349 TestCorruptTableStats.test_compute_stats[exec_option: > {'disable_codegen': False, 'abort_on_error': 1, > 'exec_single_node_rows_threshold': 100, 'batch_size': 0, 'num_nodes': 0} | > table_format: text/none] > 11:22:33.349 [gw0] linux2 -- Python 2.6.6 > /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python > 11:22:33.349 metadata/test_compute_stats.py:40: in test_compute_stats > 11:22:33.349 self.run_test_case('QueryTest/compute-stats', vector, > unique_database) > 11:22:33.349 common/impala_test_suite.py:323: in run_test_case > 11:22:33.349 pytest.config.option.update_results) > 11:22:33.349 common/test_result_verifier.py:357: in verify_raw_results > 11:22:33.349 VERIFIER_MAP[verifier](expected, actual) > 11:22:33.349 common/test_result_verifier.py:203: in > verify_query_result_is_equal > 11:22:33.349 assert expected_results == actual_results > 11:22:33.349 E assert Comparing QueryTestResults (expected vs actual): > 11:22:33.349 E 100,3,'7.73KB','NOT CACHED','NOT > CACHED','TEXT','false',regex:.* != 100,2,'7.73KB','NOT CACHED','NOT > CACHED','TEXT','false','hdfs://localhost:20500/test-warehouse/test_compute_stats_42383876.db/alltypesnopart' > {noformat} > http://sandbox.jenkins.cloudera.com/job/impala-cdh5-2.6.0_5.8.0-exhaustive-release/57/consoleFull -- This message was sent by Atlassian JIRA (v6.4.14#64029)