Kirill created ZEPPELIN-6183:
--------------------------------
Summary: ClassNotFoundException
org.apache.hadoop.conf.Configuration when LDAP password is stored in keystore
Key: ZEPPELIN-6183
URL: https://issues.apache.org/jira/browse/ZEPPELIN-6183
Project: Zeppelin
Issue Type: Bug
Components: zeppelin-server
Affects Versions: 0.12.0
Environment: Kubernetes, official {{apache/zeppelin:0.12.0}} docker
image, {{securityManager.realms = org.apache.zeppelin.realm.LdapRealm}} set in
{{shiro.ini}}, {{systemPassword}} is not set directly in config but is
referenced from keystore:
{code}
ldapRealm.hadoopSecurityCredentialPath =
jceks:///opt/zeppelin/conf/keystore/zeppelin.jceks
{code}
Keystore itself has been created by following the example in {{shiro.ini}}
template:
{code}
hadoop credential create ldapRealm.systemPassword -provider
jceks://file/user/zeppelin/conf/zeppelin.jceks
{code}
Just after start zeppelin throws the following error in console:
{noformat}
ERROR [2025-04-25 15:53:45,075] ({main}
EnvironmentLoader.java[initEnvironment]:156) - Shiro environment initialization
failed
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
at
org.apache.zeppelin.realm.LdapRealm.getSystemPassword(LdapRealm.java:231)
at org.apache.zeppelin.realm.LdapRealm.onInit(LdapRealm.java:223)
at
org.apache.shiro.realm.AuthenticatingRealm.init(AuthenticatingRealm.java:398)
{noformat}
and later:
{noformat}
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.conf.Configuration
at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
{noformat}
Full log attached.
After that the server responds with 503 to all requests.
If {{ldapRealm.contextFactory.systemPassword}} is directly set in {{shiro.ini}}
and {{ldapRealm.hadoopSecurityCredentialPath}} is commented out, there is no
such issue.
As a *workaround* the following works:
- modify server start command to fetch the libraries. Due to lack of write
permissions to {{/opt/zeppelin/lib}} by user Zeppelin-server runs from,
libraries are downloaded to {{/tmp/libs}}
{code}
command:
- sh
- -c
- mkdir /tmp/libs && wget -O /tmp/libs/hadoop-common-3.4.1.jar
https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/3.4.1/hadoop-common-3.4.1.jar
&& wget -O /tmp/libs/woodstox-core-7.1.0.jar
https://repo1.maven.org/maven2/com/fasterxml/woodstox/woodstox-core/7.1.0/woodstox-core-7.1.0.jar
&& wget -O /tmp/libs/stax2-api-4.2.2.jar
https://repo1.maven.org/maven2/org/codehaus/woodstox/stax2-api/4.2.2/stax2-api-4.2.2.jar
&& wget -O /tmp/libs/hadoop-client-runtime-3.4.1.jar
https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client-runtime/3.4.1/hadoop-client-runtime-3.4.1.jar
&& wget -O /tmp/libs/hadoop-auth-3.4.1.jar
https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/3.4.1/hadoop-auth-3.4.1.jar
&& $(ZEPPELIN_HOME)/bin/zeppelin.sh
{code}
- Set environment variable to include custom dir into classpath:
{code}
env:
- name: ZEPPELIN_CLASSPATH_OVERRIDES
value: /tmp/libs/*
{code}
I guess the missing libraries should either be included in docker image or
there should be a note in {{shiro.ini}} example about need for these libraries.
Reporter: Kirill
Attachments: trace.log
Zeppelin is deployed in k8s with official 0.12.0 image and provided
k8s-manifests.
LDAP authentication is enabled in {{shiro.ini}} with {{securityManager.realms =
org.apache.zeppelin.realm.LdapRealm}},
{{ldapRealm.hadoopSecurityCredentialPath}} is defined.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
