Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/12639 )
Change subject: IMPALA-7995: part 1: fixes for e2e dockerised impala tests ...................................................................... IMPALA-7995: part 1: fixes for e2e dockerised impala tests This fixes all core e2e tests running on my local dockerised minicluster build. I do not yet have a CI job or script running but I wanted to get feedback on these changes sooner. The second part of the change will include the CI script and any follow-on fixes required for the exhaustive tests. The following fixes were required: * Detect docker_network from TEST_START_CLUSTER_ARGS * get_webserver_port() does not depend on the caller passing in the default webserver port. It failed previously because it relied on start-impala-cluster.py setting -webserver_port for *all* processes. * Add SkipIf markers for tests that don't make sense or are non-trivial to fix for containerised Impala. * Support loading Impala-lzo plugin from host for tests that depend on it. * Fix some tests that had 'localhost' hardcoded - instead it should be $INTERNAL_LISTEN_HOST, which defaults to localhost. * Fix bug with sorting impala daemons by backend port, which is the same for all dockerised impalads. Testing: I ran tests locally as follows after having set up a docker network and starting other services: ./buildall.sh -noclean -notests -ninja ninja -j $IMPALA_BUILD_THREADS docker_images export TEST_START_CLUSTER_ARGS="--docker_network=impala-cluster" export FE_TEST=false export BE_TEST=false export JDBC_TEST=false export CLUSTER_TEST=false ./bin/run-all-tests.sh Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Reviewed-on: http://gerrit.cloudera.org:8080/12639 Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M bin/start-impala-cluster.py M docker/daemon_entrypoint.sh M docker/impala_base/Dockerfile M testdata/workloads/functional-query/queries/QueryTest/kudu_alter.test M testdata/workloads/functional-query/queries/QueryTest/kudu_create.test M tests/catalog_service/test_catalog_service_client.py M tests/common/custom_cluster_test_suite.py M tests/common/environ.py M tests/common/impala_cluster.py M tests/common/impala_test_suite.py M tests/common/skip.py M tests/conftest.py M tests/custom_cluster/test_jvm_mem_tracking.py M tests/custom_cluster/test_krpc_mem_usage.py M tests/custom_cluster/test_rpc_timeout.py M tests/custom_cluster/test_udf_concurrency.py M tests/hs2/test_fetch_first.py M tests/hs2/test_hs2.py M tests/hs2/test_json_endpoints.py M tests/metadata/test_compute_stats.py M tests/metadata/test_ddl.py M tests/observability/test_log_fragments.py M tests/query_test/test_hash_join_timer.py M tests/query_test/test_hdfs_caching.py M tests/query_test/test_insert.py M tests/query_test/test_insert_behaviour.py M tests/query_test/test_kudu.py M tests/query_test/test_lifecycle.py M tests/query_test/test_local_fs.py M tests/query_test/test_mem_usage_scaling.py M tests/query_test/test_queries.py M tests/query_test/test_udfs.py M tests/run-tests.py M tests/statestore/test_statestore.py M tests/stress/test_mini_stress.py M tests/webserver/test_web_pages.py 36 files changed, 181 insertions(+), 80 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/12639 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iee86cbd2c4631a014af1e8cef8e1cd523a812755 Gerrit-Change-Number: 12639 Gerrit-PatchSet: 18 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: David Knupp <dkn...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
