[
https://issues.apache.org/jira/browse/IMPALA-3887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tianyi Wang resolved IMPALA-3887.
---------------------------------
Resolution: Fixed
IMPALA-3887: Wait for HDFS replication in data loading
When the data loading finishes, it is possible for some HDFS blocks to
be under replicated. If impala gets the metadata before the replication
is done, some tests may fail. This patch adds a replication waiting step
in the data loading script.
Resubmitted with filesystem type check.
Change-Id: I64d9a8ea1d0a32b40047321b50a7139a8f48eac8
Reviewed-on: http://gerrit.cloudera.org:8080/8916
Reviewed-by: Vuk Ercegovac <vercego...@cloudera.com>
Reviewed-by: Alex Behm <alex.b...@cloudera.com>
Tested-by: Impala Public Jenkins
> Planner tests failing due to metadata loading race with HDFS, fewer #hosts
> than expected
> ----------------------------------------------------------------------------------------
>
> Key: IMPALA-3887
> URL: https://issues.apache.org/jira/browse/IMPALA-3887
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.6.0, Impala 2.11.0
> Reporter: Jim Apple
> Assignee: Tianyi Wang
> Priority: Blocker
> Labels: broken-build, flaky
> Fix For: Impala 2.12.0
>
>
> {noformat}
> 11:22:33.349 =================================== FAILURES
> ===================================
> 11:22:33.349 TestCorruptTableStats.test_compute_stats[exec_option:
> {'disable_codegen': False, 'abort_on_error': 1,
> 'exec_single_node_rows_threshold': 100, 'batch_size': 0, 'num_nodes': 0} |
> table_format: text/none]
> 11:22:33.349 [gw0] linux2 -- Python 2.6.6
> /data/jenkins/workspace/impala-umbrella-build-and-test/repos/Impala/bin/../infra/python/env/bin/python
> 11:22:33.349 metadata/test_compute_stats.py:40: in test_compute_stats
> 11:22:33.349 self.run_test_case('QueryTest/compute-stats', vector,
> unique_database)
> 11:22:33.349 common/impala_test_suite.py:323: in run_test_case
> 11:22:33.349 pytest.config.option.update_results)
> 11:22:33.349 common/test_result_verifier.py:357: in verify_raw_results
> 11:22:33.349 VERIFIER_MAP[verifier](expected, actual)
> 11:22:33.349 common/test_result_verifier.py:203: in
> verify_query_result_is_equal
> 11:22:33.349 assert expected_results == actual_results
> 11:22:33.349 E assert Comparing QueryTestResults (expected vs actual):
> 11:22:33.349 E 100,3,'7.73KB','NOT CACHED','NOT
> CACHED','TEXT','false',regex:.* != 100,2,'7.73KB','NOT CACHED','NOT
> CACHED','TEXT','false','hdfs://localhost:20500/test-warehouse/test_compute_stats_42383876.db/alltypesnopart'
> {noformat}
> http://sandbox.jenkins.cloudera.com/job/impala-cdh5-2.6.0_5.8.0-exhaustive-release/57/consoleFull
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
