Thank you Ted for the reference. I am going through it in detail.
Thank you Marco for your suggestion. I created a properties file with these two lines spark.driver.extraJavaOptions -Djsse.enableSNIExtension=false spark.executor.extraJavaOptions -Djsse.enableSNIExtension=false and gave this file as input to spark-shell with --properties-file. I still get the same error. Do you recommend giving these flags differently? Thank you again! On Mon, Jun 6, 2016 at 4:02 PM, Marco Mistroni <mmistr...@gmail.com> wrote: > HI > have you tried to add this flag? > > -Djsse.enableSNIExtension=false > > i had similar issues in another standalone application when i switched to > java8 from java7 > hth > marco > > On Mon, Jun 6, 2016 at 9:58 PM, Koert Kuipers <ko...@tresata.com> wrote: > >> mhh i would not be very happy if the implication is that i have to start >> maintaining separate spark builds for client clusters that use java 8... >> >> On Mon, Jun 6, 2016 at 4:34 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> Please see: >>> https://spark.apache.org/docs/latest/security.html >>> >>> w.r.t. Java 8, probably you need to rebuild 1.5.2 using Java 8. >>> >>> Cheers >>> >>> On Mon, Jun 6, 2016 at 1:19 PM, verylucky...@gmail.com < >>> verylucky...@gmail.com> wrote: >>> >>>> Thank you for your response. >>>> >>>> I have seen this and couple of other similar ones about java ssl in >>>> general. However, I am not sure how it applies to Spark and specifically to >>>> my case. >>>> >>>> This error I mention above occurs when I switch from java 7 to java 8 >>>> by changing the env variable JAVA_HOME. >>>> >>>> The error occurs seems to occur at the time of starting Jetty >>>> HTTPServer. >>>> >>>> Can you please point me to resources that help me understand how >>>> security is managed in Spark and how changing from java 7 to 8 can mess up >>>> these configurations? >>>> >>>> >>>> Thank you! >>>> >>>> On Mon, Jun 6, 2016 at 2:37 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> Have you seen this ? >>>>> >>>>> >>>>> http://stackoverflow.com/questions/22423063/java-exception-on-sslsocket-creation >>>>> >>>>> On Mon, Jun 6, 2016 at 12:31 PM, verylucky Man <verylucky...@gmail.com >>>>> > wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I have a cluster (Hortonworks supported system) running Apache spark >>>>>> on 1.5.2 on Java 7, installed by admin. Java 8 is also installed. >>>>>> >>>>>> I don't have admin access to this cluster and would like to run spark >>>>>> (1.5.2 and later versions) on java 8. >>>>>> >>>>>> I come from HPC/MPI background. So I naively copied all executables >>>>>> of spark "/usr/hdp/current/spark-client/" into my root folder. >>>>>> >>>>>> When I run spark-shell from my copied folder, it runs as expected on >>>>>> java 7. >>>>>> >>>>>> When I change $JAVA_HOME to point to java 8, and run spark-shell, I >>>>>> get the following error. >>>>>> >>>>>> Could you please help me fix this error? >>>>>> >>>>>> Exception in thread "main" java.security.NoSuchAlgorithmException: >>>>>> Error constructing implementation (algorithm: Default, provider: >>>>>> SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext) >>>>>> at java.security.Provider$Service.newInstance(Provider.java:1617) at >>>>>> sun.security.jca.GetInstance.getInstance(GetInstance.java:236) at sun >>>>>> .security.jca.GetInstance.getInstance(GetInstance.java:164) at javax. >>>>>> net.ssl.SSLContext.getInstance(SSLContext.java:156) at javax.net.ssl. >>>>>> SSLContext.getDefault(SSLContext.java:96) at org.apache.spark. >>>>>> SSLOptions.liftedTree1$1(SSLOptions.scala:122) at org.apache.spark. >>>>>> SSLOptions.<init>(SSLOptions.scala:114) at org.apache.spark. >>>>>> SSLOptions$.parse(SSLOptions.scala:199) at org.apache.spark. >>>>>> SecurityManager.<init>(SecurityManager.scala:243) at org.apache.spark >>>>>> .repl.SparkIMain.<init>(SparkIMain.scala:118) at org.apache.spark. >>>>>> repl.SparkILoop$SparkILoopInterpreter.<init>(SparkILoop.scala:187) >>>>>> at org.apache.spark.repl.SparkILoop.createInterpreter(SparkILoop. >>>>>> scala:217) at org.apache.spark.repl. >>>>>> SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1. >>>>>> apply$mcZ$sp(SparkILoop.scala:949) at org.apache.spark.repl. >>>>>> SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply >>>>>> (SparkILoop.scala:945) at org.apache.spark.repl. >>>>>> SparkILoop$$anonfun$org$apache$spark$repl$SparkILoop$$process$1.apply >>>>>> (SparkILoop.scala:945) at scala.tools.nsc.util.ScalaClassLoader$. >>>>>> savingContextLoader(ScalaClassLoader.scala:135) at org.apache.spark. >>>>>> repl.SparkILoop.org$apache$spark$repl$SparkILoop$$process(SparkILoop. >>>>>> scala:945) at org.apache.spark.repl.SparkILoop.process(SparkILoop. >>>>>> scala:1059) at org.apache.spark.repl.Main$.main(Main.scala:31) at org >>>>>> .apache.spark.repl.Main.main(Main.scala) at sun.reflect. >>>>>> NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect. >>>>>> NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at >>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke( >>>>>> DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method. >>>>>> invoke(Method.java:497) at org.apache.spark.deploy.SparkSubmit$. >>>>>> org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:685) >>>>>> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala >>>>>> :180) at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit. >>>>>> scala:205) at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit. >>>>>> scala:120) at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit. >>>>>> scala) Caused by: java.io.EOFException at java.io.DataInputStream. >>>>>> readInt(DataInputStream.java:392) at sun.security.provider. >>>>>> JavaKeyStore.engineLoad(JavaKeyStore.java:653) at sun.security. >>>>>> provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:56) at sun. >>>>>> security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java >>>>>> :225) at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad( >>>>>> JavaKeyStore.java:70) at java.security.KeyStore.load(KeyStore.java: >>>>>> 1445) at sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore( >>>>>> TrustManagerFactoryImpl.java:226) at sun.security.ssl. >>>>>> SSLContextImpl$DefaultSSLContext.getDefaultTrustManager( >>>>>> SSLContextImpl.java:767) at sun.security.ssl. >>>>>> SSLContextImpl$DefaultSSLContext.<init>(SSLContextImpl.java:733) at >>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >>>>>> at sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>> NativeConstructorAccessorImpl.java:62) at sun.reflect. >>>>>> DelegatingConstructorAccessorImpl.newInstance( >>>>>> DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect. >>>>>> Constructor.newInstance(Constructor.java:422) at java.security. >>>>>> Provider$Service.newInstance(Provider.java:1595) ... 28 more >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> >
