bharos opened a new pull request, #8755:
URL: https://github.com/apache/gravitino/pull/8755
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): support xxx"
- "[#233] fix: check null before access result in xxx"
- "[MINOR] refactor: fix typo in variable name"
- "[MINOR] docs: fix typo in README"
- "[#255] test: fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
Spark client on connecting to Icebeg Rest catalog in Gravitino gives the
error:
```
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Exception in thread "main"
org.apache.iceberg.exceptions.ServiceFailureException: Server error: null: {
"servlet":"org.glassfish.jersey.servlet.ServletContainer-602f8f94",
"message":"org.glassfish.jersey.server.ContainerException:
java.lang.NoSuchMethodError: 'void
org.apache.hadoop.security.HadoopKerberosName.setRuleMechanism(java.lang.String)'",
"url":"/iceberg/v1/config",
"status":"500"
}
```
Looking at the jars on Gravitino server, seems like there's a mix of hadoop
2.x and 3.x which is causing the class conflicts
```
kubectl exec deployment/gravitino -n hms3 -- find /r
oot -name "*hadoop*" -type f | head -10
/root/gravitino/iceberg-rest-server/libs/hadoop-auth-2.10.2.jar
/root/gravitino/iceberg-rest-server/libs/hadoop-hdfs-client-2.10.2.jar
/root/gravitino/iceberg-rest-server/libs/hadoop-hdfs-2.10.2.jar
/root/gravitino/iceberg-rest-server/libs/hadoop-common-2.10.2.jar
/root/gravitino/iceberg-rest-server/libs/gcs-connector-hadoop2-2.2.18-shaded.jar
/root/gravitino/iceberg-rest-server/libs/hadoop-mapreduce-client-core-2.10.2.jar
/root/gravitino/iceberg-rest-server/libs/hadoop-annotations-2.10.2.jar
/root/gravitino/authorizations/ranger/libs/hadoop-client-api-3.3.1.jar
/root/gravitino/authorizations/ranger/libs/hadoop-client-runtime-3.3.1.jar
/root/gravitino/catalogs/lakehouse-paimon/libs/hadoop-auth-2.10.2.jar
```
### Why are the changes needed?
Fix: #(issue)
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Testing on spark-client, after the fix , is able to connect to the Gravitino
server without the error stack.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
