Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20803 )
Change subject: IMPALA-12629: Fix Iceberg V2 tests in non-HDFS environment ...................................................................... IMPALA-12629: Fix Iceberg V2 tests in non-HDFS environment The UPDATE part 2 patch (https://gerrit.cloudera.org/#/c/20677/) added new ranger tests that verifies that row filtering and column masking are working on Iceberg V2 tables with delete files. The Iceberg V2 tables we load during data loading are already written and the position delete files have hard-coded file URIs that refer to the data files. Unfortunately these URIs start with 'hdfs://...'. Therefore these tables cannot be used in non-HDFS environment. To quickly fix this, this patch introduces the 'IS_HDFS_ONLY' test section. Tests that are annotated with it are only executed when the underlying storage system is HDFS. This new test section can be also used in the future to make it easier to write Impala/Hive interop tests, because currently we can only run Hive in an HDFS environment. So we could just annotate the tests that have HIVE_QUERY with IS_HDFS_ONLY, and we could also add a HIVE_RESULTS section to check the results Hive produces. Testing: * Ran the tests with '-s' option in HDFS environment and verified that the IS_HDFS_ONLY tests are executed * Ran the ranger tests in Ozone environment and they passed Change-Id: Ifa72041e281e3d5083adaee2f8a41bb1c2c8d19f Reviewed-on: http://gerrit.cloudera.org:8080/20803 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking.test M testdata/workloads/functional-query/queries/QueryTest/ranger_row_filtering.test M tests/common/impala_test_suite.py M tests/util/test_file_parser.py 4 files changed, 6 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20803 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ifa72041e281e3d5083adaee2f8a41bb1c2c8d19f Gerrit-Change-Number: 20803 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
