classnotfoundexception using zeppelin with spark authentication
I am having trouble using zeppelin in a spark cluster that has spark node
authentication turned on (e.g. with spark.authenticate=true,
spark.authenticate.secret=...)
Notebook code that calls built-in spark functions (or other things on
executor classpath) work fine, but functions defined in the notebook
(anonymous or named) throw ClassNotFoundExceptions when called from an
executor.
For example, this code works:
val rdd = sc.textFile("hdfs://my-text-file")
rdd.take(1).foreach(println)
rdd.saveAsTextFile("hdfs:///my-other-text-file")
but code like this...
rdd.filter(_.contains("my data"))
fails with
Caused by: java.lang.ClassNotFoundException:
$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1
at
org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:84)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
...
I get the same kind of error if the filter function is defined as a named
function in the notebook, or as a member of singleton object defined in the
notebook.
When I look at the executor's log output, I see this error:
16/06/27 21:36:23 ERROR repl.ExecutorClassLoader: Failed to check existence
of class $line31.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1
on REPL class server at https://172.30.54.30:34980
java.lang.NullPointerException
at
org.apache.spark.repl.ExecutorClassLoader.getClassFileInputStreamFromHttpServer(ExecutorClassLoader.scala:113)
at
org.apache.spark.repl.ExecutorClassLoader.findClassLocally(ExecutorClassLoader.scala:146)
at
org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:76)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
...
If I disable spark authentication, everything works as expected. I am
running zeppelin 0.5.6 on spark 1.6.1 with yarn.
Has anyone been able to get zeppelin working with spark authentication?
Re: classnotfoundexception using zeppelin with spark authentication
Hyung,
Yes, here they are.
zeppelin-env.sh:
export ZEPPELIN_PORT=8890
export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf
export ZEPPELIN_LOG_DIR=/var/log/zeppelin
export ZEPPELIN_PID_DIR=/var/run/zeppelin
export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid
export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook
export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps
export MASTER=yarn-client
export SPARK_HOME=/usr/lib/spark
export HADOOP_CONF_DIR=/etc/hadoop/conf
export
CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/*
:/usr/share/aws/emr/emrfs/auxlib/*"
export JAVA_HOME=/usr/lib/jvm/java-1.8.0
export ZEPPELIN_NOTEBOOK_S3_BUCKET=mybucket
export ZEPPELIN_NOTEBOOK_S3_USER=zeppelin
export
ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo
spark-defaults.conf:
spark.master yarn
spark.driver.extraClassPath
/etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf
:/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/*
spark.driver.extraLibraryPath
/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native
spark.executor.extraClassPath
/etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf
:/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/*
spark.executor.extraLibraryPath
/usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native
spark.eventLog.enabled true
spark.eventLog.dir hdfs:///var/log/spark/apps
spark.history.fs.logDirectoryhdfs:///var/log/spark/apps
spark.yarn.historyServer.address ip-172-30-54-30.ec2.internal:18080
spark.history.ui.port18080
spark.shuffle.service.enabledtrue
spark.driver.extraJavaOptions
-Dlog4j.configuration=file:///etc/spark/conf/log4j.properties
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70
-XX:MaxHeapFreeRatio=70
-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=512M
-XX:OnOutOfMemoryError='kill -9 %p'
spark.dynamicAllocation.enabled true
spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:+CM
SClassUnloadingEnabled -XX:OnOutOfMemoryError='kill -9 %p'
spark.executor.memory8640m
spark.executor.cores 7
spark.authenticate.enableSaslEncryption true
spark.driver.memory 1g
spark.network.sasl.serverAlwaysEncrypt true
spark.driver.cores 1
spark.ssl.protocol TLSv1.2
spark.ssl.keyStorePassword password
spark.yarn.maxAppAttempts1
spark.ssl.keyStore /etc/emr/security/keystore.jks
spark.authenticate true
spark.ssl.keyPasswordpassword
spark.ssl.enabledtrue
spark.ssl.enabledAlgorithms TLS_RSA_WITH_AES_256_CBC_SHA
spark.ssl.trustStore /etc/emr/security/truststore.jks
spark.authenticate.secretsecret
spark.ssl.trustStorePassword password
On Mon, Jun 27, 2016 at 7:33 PM, Hyung Sung Shim wrote:
> Hi.
> Could you share your conf/zeppelin-env.sh and spark-defaults.conf ?
>
> 2016-06-28 8:52 GMT+09:00 Jonathan Esterhazy >:
>
>> I am having trouble using zeppelin in a spark cluster that has spark node
>> authentication turned on (e.g. with spark.authenticate=true,
>> spark.authenticate.secret=...)
>>
>> Notebook code that calls built-in spark functions (or other things on
>> executor classpath) work fine, but functions defined in the notebook
>> (anonymous or named) throw ClassNotFoundExceptions when called from an
>> executor.
>>
>> For example, this code works:
>>
>> val rdd = sc.textFile("hdfs://my-text-file")
>> rdd.take(1).foreach(println)
>>
>> rdd.saveAsTextFile("hdfs:///my-other-text-file")
>>
>> but code like this...
>>
>> rdd.filter(_.contains("my data"))
>>
>> fails with
>>
>> Caused by: java.lang.ClassNotFoundException:
>> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1
>> at
>> org.apache.spark.repl.ExecutorClassLoader.findClass(ExecutorClassLoader.scala:84)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:348)
>> ...
>>
>>
>> I get the same kind of error if the filter function is defined as a named
>> function in the notebook, or as a member of singleton object defined in the
>> notebook.
>>
>> When I lo
Re: classnotfoundexception using zeppelin with spark authentication
Hyung, did you have any luck w/ zeppelin + spark authentication? I'm quite stumped. thx. On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim wrote: > Thank you. > Let me try. > > 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy < > jonathan.esterh...@gmail.com>: > >> Hyung, >> >> Yes, here they are. >> >> zeppelin-env.sh: >> >> export ZEPPELIN_PORT=8890 >> export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf >> export ZEPPELIN_LOG_DIR=/var/log/zeppelin >> export ZEPPELIN_PID_DIR=/var/run/zeppelin >> export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid >> export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook >> export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps >> export MASTER=yarn-client >> export SPARK_HOME=/usr/lib/spark >> export HADOOP_CONF_DIR=/etc/hadoop/conf >> export >> CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/* >> :/usr/share/aws/emr/emrfs/auxlib/*" >> export JAVA_HOME=/usr/lib/jvm/java-1.8.0 >> export ZEPPELIN_NOTEBOOK_S3_BUCKET=mybucket >> export ZEPPELIN_NOTEBOOK_S3_USER=zeppelin >> export >> ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo >> >> spark-defaults.conf: >> >> spark.master yarn >> spark.driver.extraClassPath >> >> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >> spark.driver.extraLibraryPath >> /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native >> spark.executor.extraClassPath >> >> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >> spark.executor.extraLibraryPath >> /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native >> spark.eventLog.enabled true >> spark.eventLog.dir hdfs:///var/log/spark/apps >> spark.history.fs.logDirectoryhdfs:///var/log/spark/apps >> spark.yarn.historyServer.address ip-172-30-54-30.ec2.internal:18080 >> spark.history.ui.port18080 >> spark.shuffle.service.enabledtrue >> spark.driver.extraJavaOptions >> -Dlog4j.configuration=file:///etc/spark/conf/log4j.properties >> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 >> -XX:MaxHeapFreeRatio=70 >> -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=512M >> -XX:OnOutOfMemoryError='kill -9 %p' >> spark.dynamicAllocation.enabled true >> spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails >> -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC >> -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:+CM >> SClassUnloadingEnabled -XX:OnOutOfMemoryError='kill -9 %p' >> spark.executor.memory8640m >> spark.executor.cores 7 >> spark.authenticate.enableSaslEncryption true >> spark.driver.memory 1g >> spark.network.sasl.serverAlwaysEncrypt true >> spark.driver.cores 1 >> spark.ssl.protocol TLSv1.2 >> spark.ssl.keyStorePassword password >> spark.yarn.maxAppAttempts1 >> spark.ssl.keyStore /etc/emr/security/keystore.jks >> spark.authenticate true >> spark.ssl.keyPassword password >> spark.ssl.enabledtrue >> spark.ssl.enabledAlgorithms TLS_RSA_WITH_AES_256_CBC_SHA >> spark.ssl.trustStore /etc/emr/security/truststore.jks >> spark.authenticate.secretsecret >> spark.ssl.trustStorePassword password >> >> >> >> On Mon, Jun 27, 2016 at 7:33 PM, Hyung Sung Shim >> wrote: >> >>> Hi. >>> Could you share your conf/zeppelin-env.sh and spark-defaults.conf ? >>> >>> 2016-06-28 8:52 GMT+09:00 Jonathan Esterhazy < >>> jonathan.esterh...@gmail.com>: >>> >>>> I am having trouble using zeppelin in a spark cluster that has spark >>>> node authentication turned on (e.g. with spark.authenticate=true, >>>> spark.authenticate.secret=...) >>>> >>>> Notebook code that calls built-in spark functions (or other things on >>>> executor classpath) work fine, but functions defined in the notebook >>>> (anonymous or named) thro
Re: classnotfoundexception using zeppelin with spark authentication
yes it does. i only see this problem in zeppelin. On Thu, Jun 30, 2016 at 7:05 AM, Hyung Sung Shim wrote: > Hi Jonathan. > It's not easy to build the test environments but I am working on this. > I have question for you. > Does your code working well on spark-shell in the spark.authenticate mode? > > 2016-06-30 22:47 GMT+09:00 Jonathan Esterhazy < > jonathan.esterh...@gmail.com>: > >> Hyung, did you have any luck w/ zeppelin + spark authentication? I'm >> quite stumped. >> >> thx. >> >> On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim >> wrote: >> >>> Thank you. >>> Let me try. >>> >>> 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy < >>> jonathan.esterh...@gmail.com>: >>> >>>> Hyung, >>>> >>>> Yes, here they are. >>>> >>>> zeppelin-env.sh: >>>> >>>> export ZEPPELIN_PORT=8890 >>>> export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf >>>> export ZEPPELIN_LOG_DIR=/var/log/zeppelin >>>> export ZEPPELIN_PID_DIR=/var/run/zeppelin >>>> export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid >>>> export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook >>>> export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps >>>> export MASTER=yarn-client >>>> export SPARK_HOME=/usr/lib/spark >>>> export HADOOP_CONF_DIR=/etc/hadoop/conf >>>> export >>>> CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/* >>>> :/usr/share/aws/emr/emrfs/auxlib/*" >>>> export JAVA_HOME=/usr/lib/jvm/java-1.8.0 >>>> export ZEPPELIN_NOTEBOOK_S3_BUCKET=mybucket >>>> export ZEPPELIN_NOTEBOOK_S3_USER=zeppelin >>>> export >>>> ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo >>>> >>>> spark-defaults.conf: >>>> >>>> spark.master yarn >>>> spark.driver.extraClassPath >>>> >>>> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >>>> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >>>> spark.driver.extraLibraryPath >>>> /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native >>>> spark.executor.extraClassPath >>>> >>>> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >>>> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >>>> spark.executor.extraLibraryPath >>>> /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native >>>> spark.eventLog.enabled true >>>> spark.eventLog.dir hdfs:///var/log/spark/apps >>>> spark.history.fs.logDirectoryhdfs:///var/log/spark/apps >>>> spark.yarn.historyServer.address ip-172-30-54-30.ec2.internal:18080 >>>> spark.history.ui.port18080 >>>> spark.shuffle.service.enabledtrue >>>> spark.driver.extraJavaOptions >>>> -Dlog4j.configuration=file:///etc/spark/conf/log4j.properties >>>> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 >>>> -XX:MaxHeapFreeRatio=70 >>>> -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=512M >>>> -XX:OnOutOfMemoryError='kill -9 %p' >>>> spark.dynamicAllocation.enabled true >>>> spark.executor.extraJavaOptions -verbose:gc -XX:+PrintGCDetails >>>> -XX:+PrintGCDateStamps -XX:+UseConcMarkSweepGC >>>> -XX:CMSInitiatingOccupancyFraction=70 -XX:MaxHeapFreeRatio=70 -XX:+CM >>>> SClassUnloadingEnabled -XX:OnOutOfMemoryError='kill -9 %p' >>>> spark.executor.memory8640m >>>> spark.executor.cores 7 >>>> spark.authenticate.enableSaslEncryption true >>>> spark.driver.memory 1g >>>> spark.network.sasl.serverAlwaysEncrypt true >>>> spark.driver.cores 1 >>>> spark.ssl.protocol TLSv1.2 >>>> spark.ssl.keyStorePassword password >>>> spark.yarn.maxAppAttempts1 >>>> spark.ssl.keyStore /etc/emr/security/keystore.jks >>>> spark.authenticate true >>>> spark.ssl.keyPasswordpassword
Re: classnotfoundexception using zeppelin with spark authentication
I added this and still get the same exception. The same property is also set in spark-defaults.conf. After that didn't work, I also tried adding --conf spark.authenticate=true and --conf spark.authenticate.enableSaslEncryption=true, to match the other related settings in spark-defaults.conf. Still get the same classnotfoundexception. On Thu, Jun 30, 2016 at 10:45 AM, Hyung Sung Shim wrote: > Please add the *export SPARK_SUBMIT_OPTIONS="--conf > spark.authenticate.secret=secret"* in conf/zeppelin-env.sh and restart > zeppelin and retry your code. > > > 2016-06-30 23:34 GMT+09:00 Jonathan Esterhazy < > jonathan.esterh...@gmail.com>: > >> yes it does. i only see this problem in zeppelin. >> >> On Thu, Jun 30, 2016 at 7:05 AM, Hyung Sung Shim >> wrote: >> >>> Hi Jonathan. >>> It's not easy to build the test environments but I am working on this. >>> I have question for you. >>> Does your code working well on spark-shell in the spark.authenticate >>> mode? >>> >>> 2016-06-30 22:47 GMT+09:00 Jonathan Esterhazy < >>> jonathan.esterh...@gmail.com>: >>> >>>> Hyung, did you have any luck w/ zeppelin + spark authentication? I'm >>>> quite stumped. >>>> >>>> thx. >>>> >>>> On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim >>>> wrote: >>>> >>>>> Thank you. >>>>> Let me try. >>>>> >>>>> 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy < >>>>> jonathan.esterh...@gmail.com>: >>>>> >>>>>> Hyung, >>>>>> >>>>>> Yes, here they are. >>>>>> >>>>>> zeppelin-env.sh: >>>>>> >>>>>> export ZEPPELIN_PORT=8890 >>>>>> export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf >>>>>> export ZEPPELIN_LOG_DIR=/var/log/zeppelin >>>>>> export ZEPPELIN_PID_DIR=/var/run/zeppelin >>>>>> export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid >>>>>> export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook >>>>>> export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps >>>>>> export MASTER=yarn-client >>>>>> export SPARK_HOME=/usr/lib/spark >>>>>> export HADOOP_CONF_DIR=/etc/hadoop/conf >>>>>> export >>>>>> CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/* >>>>>> :/usr/share/aws/emr/emrfs/auxlib/*" >>>>>> export JAVA_HOME=/usr/lib/jvm/java-1.8.0 >>>>>> export ZEPPELIN_NOTEBOOK_S3_BUCKET=mybucket >>>>>> export ZEPPELIN_NOTEBOOK_S3_USER=zeppelin >>>>>> export >>>>>> ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo >>>>>> >>>>>> spark-defaults.conf: >>>>>> >>>>>> spark.master yarn >>>>>> spark.driver.extraClassPath >>>>>> >>>>>> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >>>>>> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >>>>>> spark.driver.extraLibraryPath >>>>>> /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native >>>>>> spark.executor.extraClassPath >>>>>> >>>>>> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >>>>>> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >>>>>> spark.executor.extraLibraryPath >>>>>> /usr/lib/hadoop/lib/native:/usr/lib/hadoop-lzo/lib/native >>>>>> spark.eventLog.enabled true >>>>>> spark.eventLog.dir hdfs:///var/log/spark/apps >>>>>> spark.history.fs.logDirectoryhdfs:///var/log/spark/apps >>>>>> spark.yarn.historyServer.address ip-172-30-54-30.ec2.internal:18080 >>>>>> spark.history.ui.port18080 >>>>>> spark.shuffle.service.enabledtrue >>>>>> spark.driver.extraJavaOptions >>>>>> -Dlog4j.configuration=file:///etc/spark/conf/log4j.properties >>>>>
Re: classnotfoundexception using zeppelin with spark authentication
Hyung, thx for your help. I opened these: https://issues.apache.org/jira/browse/ZEPPELIN-1096 (this scala problem) https://issues.apache.org/jira/browse/ZEPPELIN-1097 (similar looking python problem) LMK if I can provide more info or help in some way. On Fri, Jul 1, 2016 at 5:08 AM, Hyung Sung Shim wrote: > Hi Jonathan. > Unfortunately I got same error in my test bed. > Do you mind create an jira issue for this? > > 2016-07-01 3:19 GMT+09:00 Jonathan Esterhazy >: > >> I added this and still get the same exception. The same property is also >> set in spark-defaults.conf. >> >> After that didn't work, I also tried adding --conf >> spark.authenticate=true and --conf >> spark.authenticate.enableSaslEncryption=true, to match the other related >> settings in spark-defaults.conf. Still get the same classnotfoundexception. >> >> >> On Thu, Jun 30, 2016 at 10:45 AM, Hyung Sung Shim >> wrote: >> >>> Please add the *export SPARK_SUBMIT_OPTIONS="--conf >>> spark.authenticate.secret=secret"* in conf/zeppelin-env.sh and restart >>> zeppelin and retry your code. >>> >>> >>> 2016-06-30 23:34 GMT+09:00 Jonathan Esterhazy < >>> jonathan.esterh...@gmail.com>: >>> >>>> yes it does. i only see this problem in zeppelin. >>>> >>>> On Thu, Jun 30, 2016 at 7:05 AM, Hyung Sung Shim >>>> wrote: >>>> >>>>> Hi Jonathan. >>>>> It's not easy to build the test environments but I am working on this. >>>>> I have question for you. >>>>> Does your code working well on spark-shell in the spark.authenticate >>>>> mode? >>>>> >>>>> 2016-06-30 22:47 GMT+09:00 Jonathan Esterhazy < >>>>> jonathan.esterh...@gmail.com>: >>>>> >>>>>> Hyung, did you have any luck w/ zeppelin + spark authentication? I'm >>>>>> quite stumped. >>>>>> >>>>>> thx. >>>>>> >>>>>> On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim >>>>>> wrote: >>>>>> >>>>>>> Thank you. >>>>>>> Let me try. >>>>>>> >>>>>>> 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy < >>>>>>> jonathan.esterh...@gmail.com>: >>>>>>> >>>>>>>> Hyung, >>>>>>>> >>>>>>>> Yes, here they are. >>>>>>>> >>>>>>>> zeppelin-env.sh: >>>>>>>> >>>>>>>> export ZEPPELIN_PORT=8890 >>>>>>>> export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf >>>>>>>> export ZEPPELIN_LOG_DIR=/var/log/zeppelin >>>>>>>> export ZEPPELIN_PID_DIR=/var/run/zeppelin >>>>>>>> export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid >>>>>>>> export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook >>>>>>>> export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps >>>>>>>> export MASTER=yarn-client >>>>>>>> export SPARK_HOME=/usr/lib/spark >>>>>>>> export HADOOP_CONF_DIR=/etc/hadoop/conf >>>>>>>> export >>>>>>>> CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/usr/share/aws/emr/emrfs/lib/* >>>>>>>> :/usr/share/aws/emr/emrfs/auxlib/*" >>>>>>>> export JAVA_HOME=/usr/lib/jvm/java-1.8.0 >>>>>>>> export ZEPPELIN_NOTEBOOK_S3_BUCKET=mybucket >>>>>>>> export ZEPPELIN_NOTEBOOK_S3_USER=zeppelin >>>>>>>> export >>>>>>>> ZEPPELIN_NOTEBOOK_STORAGE=org.apache.zeppelin.notebook.repo.S3NotebookRepo >>>>>>>> >>>>>>>> spark-defaults.conf: >>>>>>>> >>>>>>>> spark.master yarn >>>>>>>> spark.driver.extraClassPath >>>>>>>> >>>>>>>> /etc/hadoop/conf:/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf >>>>>>>> :/usr/share/aws/emr/emrfs/lib/*:/usr/share/aws/emr/emrfs/auxlib/* >>>>>>>> spark.driver.extraLibraryPath >>>>>>>>
Re: classnotfoundexception using zeppelin with spark authentication
I just tried this with using zeppelin master, and the classnotfound error is still there. On Thu, Jul 7, 2016 at 7:07 AM, Hyung Sung Shim wrote: > Hi. Jonathan. > If it is possible, could you use the zeppelin master on the github? > Zeppelin is working well with spark authentication when i tested. > > > > 2016-07-02 3:05 GMT+09:00 Jonathan Esterhazy >: > >> Hyung, thx for your help. I opened these: >> >> https://issues.apache.org/jira/browse/ZEPPELIN-1096 (this scala problem) >> https://issues.apache.org/jira/browse/ZEPPELIN-1097 (similar looking >> python problem) >> >> LMK if I can provide more info or help in some way. >> >> On Fri, Jul 1, 2016 at 5:08 AM, Hyung Sung Shim >> wrote: >> >>> Hi Jonathan. >>> Unfortunately I got same error in my test bed. >>> Do you mind create an jira issue for this? >>> >>> 2016-07-01 3:19 GMT+09:00 Jonathan Esterhazy < >>> jonathan.esterh...@gmail.com>: >>> >>>> I added this and still get the same exception. The same property is >>>> also set in spark-defaults.conf. >>>> >>>> After that didn't work, I also tried adding --conf >>>> spark.authenticate=true and --conf >>>> spark.authenticate.enableSaslEncryption=true, to match the other related >>>> settings in spark-defaults.conf. Still get the same classnotfoundexception. >>>> >>>> >>>> On Thu, Jun 30, 2016 at 10:45 AM, Hyung Sung Shim >>>> wrote: >>>> >>>>> Please add the *export SPARK_SUBMIT_OPTIONS="--conf >>>>> spark.authenticate.secret=secret"* in conf/zeppelin-env.sh and >>>>> restart zeppelin and retry your code. >>>>> >>>>> >>>>> 2016-06-30 23:34 GMT+09:00 Jonathan Esterhazy < >>>>> jonathan.esterh...@gmail.com>: >>>>> >>>>>> yes it does. i only see this problem in zeppelin. >>>>>> >>>>>> On Thu, Jun 30, 2016 at 7:05 AM, Hyung Sung Shim >>>>>> wrote: >>>>>> >>>>>>> Hi Jonathan. >>>>>>> It's not easy to build the test environments but I am working on >>>>>>> this. >>>>>>> I have question for you. >>>>>>> Does your code working well on spark-shell in the spark.authenticate >>>>>>> mode? >>>>>>> >>>>>>> 2016-06-30 22:47 GMT+09:00 Jonathan Esterhazy < >>>>>>> jonathan.esterh...@gmail.com>: >>>>>>> >>>>>>>> Hyung, did you have any luck w/ zeppelin + spark authentication? >>>>>>>> I'm quite stumped. >>>>>>>> >>>>>>>> thx. >>>>>>>> >>>>>>>> On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim >>>>>>> > wrote: >>>>>>>> >>>>>>>>> Thank you. >>>>>>>>> Let me try. >>>>>>>>> >>>>>>>>> 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy < >>>>>>>>> jonathan.esterh...@gmail.com>: >>>>>>>>> >>>>>>>>>> Hyung, >>>>>>>>>> >>>>>>>>>> Yes, here they are. >>>>>>>>>> >>>>>>>>>> zeppelin-env.sh: >>>>>>>>>> >>>>>>>>>> export ZEPPELIN_PORT=8890 >>>>>>>>>> export ZEPPELIN_CONF_DIR=/etc/zeppelin/conf >>>>>>>>>> export ZEPPELIN_LOG_DIR=/var/log/zeppelin >>>>>>>>>> export ZEPPELIN_PID_DIR=/var/run/zeppelin >>>>>>>>>> export ZEPPELIN_PID=$ZEPPELIN_PID_DIR/zeppelin.pid >>>>>>>>>> export ZEPPELIN_NOTEBOOK_DIR=/var/lib/zeppelin/notebook >>>>>>>>>> export ZEPPELIN_WAR_TEMPDIR=/var/run/zeppelin/webapps >>>>>>>>>> export MASTER=yarn-client >>>>>>>>>> export SPARK_HOME=/usr/lib/spark >>>>>>>>>> export HADOOP_CONF_DIR=/etc/hadoop/conf >>>>>>>>>> export >>>>>>>>>> CLASSPATH=":/etc/hive/conf:/usr/lib/hadoop-lzo/lib/*:/usr/lib/hadoop/hadoop-aws.jar:/usr/share/aws/aws-java-sdk/*:/usr/share/aws/emr/emrfs/conf:/
Re: classnotfoundexception using zeppelin with spark authentication
yes, this PR fixes my classnotfoundexceptions. awesome! any idea when it might make it into a release? On Wed, Jul 27, 2016 at 11:25 PM, Jeff Zhang wrote: > I think it is fixed in https://github.com/apache/zeppelin/pull/1237, > please try that. > > On Thu, Jul 28, 2016 at 1:58 AM, Jonathan Esterhazy < > jonathan.esterh...@gmail.com> wrote: > >> I just tried this with using zeppelin master, and the classnotfound error >> is still there. >> >> >> On Thu, Jul 7, 2016 at 7:07 AM, Hyung Sung Shim >> wrote: >> >>> Hi. Jonathan. >>> If it is possible, could you use the zeppelin master on the github? >>> Zeppelin is working well with spark authentication when i tested. >>> >>> >>> >>> 2016-07-02 3:05 GMT+09:00 Jonathan Esterhazy < >>> jonathan.esterh...@gmail.com>: >>> >>>> Hyung, thx for your help. I opened these: >>>> >>>> https://issues.apache.org/jira/browse/ZEPPELIN-1096 (this scala >>>> problem) >>>> https://issues.apache.org/jira/browse/ZEPPELIN-1097 (similar looking >>>> python problem) >>>> >>>> LMK if I can provide more info or help in some way. >>>> >>>> On Fri, Jul 1, 2016 at 5:08 AM, Hyung Sung Shim >>>> wrote: >>>> >>>>> Hi Jonathan. >>>>> Unfortunately I got same error in my test bed. >>>>> Do you mind create an jira issue for this? >>>>> >>>>> 2016-07-01 3:19 GMT+09:00 Jonathan Esterhazy < >>>>> jonathan.esterh...@gmail.com>: >>>>> >>>>>> I added this and still get the same exception. The same property is >>>>>> also set in spark-defaults.conf. >>>>>> >>>>>> After that didn't work, I also tried adding --conf >>>>>> spark.authenticate=true and --conf >>>>>> spark.authenticate.enableSaslEncryption=true, to match the other related >>>>>> settings in spark-defaults.conf. Still get the same >>>>>> classnotfoundexception. >>>>>> >>>>>> >>>>>> On Thu, Jun 30, 2016 at 10:45 AM, Hyung Sung Shim >>>>>> wrote: >>>>>> >>>>>>> Please add the *export SPARK_SUBMIT_OPTIONS="--conf >>>>>>> spark.authenticate.secret=secret"* in conf/zeppelin-env.sh and >>>>>>> restart zeppelin and retry your code. >>>>>>> >>>>>>> >>>>>>> 2016-06-30 23:34 GMT+09:00 Jonathan Esterhazy < >>>>>>> jonathan.esterh...@gmail.com>: >>>>>>> >>>>>>>> yes it does. i only see this problem in zeppelin. >>>>>>>> >>>>>>>> On Thu, Jun 30, 2016 at 7:05 AM, Hyung Sung Shim >>>>>>> > wrote: >>>>>>>> >>>>>>>>> Hi Jonathan. >>>>>>>>> It's not easy to build the test environments but I am working on >>>>>>>>> this. >>>>>>>>> I have question for you. >>>>>>>>> Does your code working well on spark-shell in the >>>>>>>>> spark.authenticate mode? >>>>>>>>> >>>>>>>>> 2016-06-30 22:47 GMT+09:00 Jonathan Esterhazy < >>>>>>>>> jonathan.esterh...@gmail.com>: >>>>>>>>> >>>>>>>>>> Hyung, did you have any luck w/ zeppelin + spark authentication? >>>>>>>>>> I'm quite stumped. >>>>>>>>>> >>>>>>>>>> thx. >>>>>>>>>> >>>>>>>>>> On Tue, Jun 28, 2016 at 9:11 PM, Hyung Sung Shim < >>>>>>>>>> hss...@nflabs.com> wrote: >>>>>>>>>> >>>>>>>>>>> Thank you. >>>>>>>>>>> Let me try. >>>>>>>>>>> >>>>>>>>>>> 2016-06-28 22:18 GMT+09:00 Jonathan Esterhazy < >>>>>>>>>>> jonathan.esterh...@gmail.com>: >>>>>>>>>>> >>>>>>>>>>>> Hyung, >>>>>>>>>>>> >>>>>>>>>>>> Yes, here they are. >>>>>>>>>>&g
