Joe McDonnell has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/19149 )
Change subject: IMPALA-11524: Bump up the CDP GBN to 33375775 and remove workarounds ...................................................................... IMPALA-11524: Bump up the CDP GBN to 33375775 and remove workarounds This patch bumps up the GBN to 33375775, which contains the fix for HADOOP-18410 as well as HADOOP-18456, which should help with the stability of S3 tests. This removes the workaround for HADOOP-18410 introduced in IMPALA-11514. This also picks up newer versions of Ozone, Iceberg, Hive, etc. After HIVE-26071, hive-standalone-metastore starts relying on jetty-servlet, which in turn requires jetty-security. Since Impala bans all jetty-related dependencies unless otherwise specified, this patch adds jetty-servlet, jetty-security, and jetty-util-ajax as allowed dependencies in order to compile Impala with Hive. Hive Metastore introduced several new APIs in their interface, so this adds implementations for these (e.g. HIVE-25303, HIVE-22782). For example, HIVE-26149 introduced a new HMS API drop_database_req, so we need to implement this API in CatalogServerHandler as well. This patch also fixes a bug that the third parameter in drop_database was treated as "ignoreUnknownDb", however it should be "cascade". The number of files produced on systems with 32GB of memory changed. Until we track down the specific cause, this tunes the YARN memory slightly higher to avoid changes in the number of files produced for systems with 32GB of memory. The content in this patch comes from several developers: Yu-Wen Lai, Kishen Das, and Fang-Yu Rao. Testing: - Ran core jobs - Ran CatalogHmsSyncToLatestEventIdTest tests - Hand tested on a machine with 32GB of memory Change-Id: Iea7e1785f5c93f61179cc336968c3a86c53e9ed1 --- M bin/impala-config.sh M fe/pom.xml M fe/src/main/java/org/apache/impala/catalog/metastore/CatalogMetastoreServiceHandler.java M fe/src/main/java/org/apache/impala/catalog/metastore/HmsApiNameEnum.java M fe/src/main/java/org/apache/impala/catalog/metastore/MetastoreServiceHandler.java M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.py M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py 7 files changed, 221 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/19149/10 -- To view, visit http://gerrit.cloudera.org:8080/19149 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Iea7e1785f5c93f61179cc336968c3a86c53e9ed1 Gerrit-Change-Number: 19149 Gerrit-PatchSet: 10 Gerrit-Owner: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
