[
https://issues.apache.org/jira/browse/SPARK-42511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SHOBHIT SHUKLA updated SPARK-42511:
-----------------------------------
Description:
When we are running Spark by setting up below SSL configurations, Spark
masterwebui and workerwebui is fail to start.
{quote}
"spark.ssl.enabled":"true",
"spark.ssl.keyStore":"/opt/ibm/jdk/conf/security/nss.fips.cfg",
"spark.ssl.keyStorePassword":"changeit",
"spark.ssl.keyStoreType":"PKCS11"
{quote}
*Errors :*
{quote}23/02/21 12:29:43 INFO Master: Running Spark version 3.3.1
23/02/21 12:29:43 ERROR MasterWebUI: Failed to bind MasterWebUI
java.security.KeyStoreException: PKCS11 not found
at java.base/java.security.KeyStore.getInstance(KeyStore.java:878)
at
org.sparkproject.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:46)
at
org.sparkproject.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1203)
at
org.sparkproject.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:322)
at
org.sparkproject.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:244)
at
org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.sparkproject.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:97)
at
org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.sparkproject.jetty.server.AbstractConnector.doStart(AbstractConnector.java:323)
at
org.sparkproject.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at
org.sparkproject.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
at
org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.spark.ui.JettyUtils$.newConnector$1(JettyUtils.scala:303)
at org.apache.spark.ui.JettyUtils$.sslConnect$1(JettyUtils.scala:322)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4(JettyUtils.scala:326)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4$adapted(JettyUtils.scala:326)
at
org.apache.spark.util.Utils$.$anonfun$startServiceOnPort$2(Utils.scala:2401)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2393)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2(JettyUtils.scala:326)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2$adapted(JettyUtils.scala:315)
at scala.Option.map(Option.scala:230)
at
org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:315)
at org.apache.spark.ui.WebUI.initServer(WebUI.scala:144)
at org.apache.spark.ui.WebUI.bind(WebUI.scala:153)
at org.apache.spark.deploy.master.Master.onStart(Master.scala:138)
at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:120)
at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:213)
at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
at
org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
at
org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:839)
Caused by: java.security.NoSuchAlgorithmException: PKCS11 KeyStore not available
at
java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
at java.base/java.security.Security.getImpl(Security.java:719)
at java.base/java.security.KeyStore.getInstance(KeyStore.java:875)
... 37 more
{quote}
was:
When we are running Spark by setting up below SSL configurations, Spark
masterwebui and workerwebui is fail to start.
```
"spark.ssl.enabled":"true",
"spark.ssl.keyStore":"/opt/ibm/jdk/conf/security/nss.fips.cfg",
"spark.ssl.keyStorePassword":"changeit",
"spark.ssl.keyStoreType":"PKCS11"
```
Errors :
```
23/02/21 12:29:43 INFO Master: Running Spark version 3.3.1
23/02/21 12:29:43 ERROR MasterWebUI: Failed to bind MasterWebUI
java.security.KeyStoreException: PKCS11 not found
at java.base/java.security.KeyStore.getInstance(KeyStore.java:878)
at
org.sparkproject.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:46)
at
org.sparkproject.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1203)
at
org.sparkproject.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:322)
at
org.sparkproject.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:244)
at
org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.sparkproject.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:97)
at
org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.sparkproject.jetty.server.AbstractConnector.doStart(AbstractConnector.java:323)
at
org.sparkproject.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
at
org.sparkproject.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
at
org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.spark.ui.JettyUtils$.newConnector$1(JettyUtils.scala:303)
at org.apache.spark.ui.JettyUtils$.sslConnect$1(JettyUtils.scala:322)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4(JettyUtils.scala:326)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4$adapted(JettyUtils.scala:326)
at
org.apache.spark.util.Utils$.$anonfun$startServiceOnPort$2(Utils.scala:2401)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2393)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2(JettyUtils.scala:326)
at
org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2$adapted(JettyUtils.scala:315)
at scala.Option.map(Option.scala:230)
at
org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:315)
at org.apache.spark.ui.WebUI.initServer(WebUI.scala:144)
at org.apache.spark.ui.WebUI.bind(WebUI.scala:153)
at org.apache.spark.deploy.master.Master.onStart(Master.scala:138)
at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:120)
at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:213)
at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
at
org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
at
org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:839)
Caused by: java.security.NoSuchAlgorithmException: PKCS11 KeyStore not available
at
java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
at java.base/java.security.Security.getImpl(Security.java:719)
at java.base/java.security.KeyStore.getInstance(KeyStore.java:875)
... 37 more
```
> Spark MasterWebUI and WorkerWebUI fail to start when NSSDB used as keystore.
> ----------------------------------------------------------------------------
>
> Key: SPARK-42511
> URL: https://issues.apache.org/jira/browse/SPARK-42511
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, Spark Submit
> Affects Versions: 3.3.0, 3.3.1, 3.3.2
> Reporter: SHOBHIT SHUKLA
> Priority: Major
>
> When we are running Spark by setting up below SSL configurations, Spark
> masterwebui and workerwebui is fail to start.
> {quote}
> "spark.ssl.enabled":"true",
> "spark.ssl.keyStore":"/opt/ibm/jdk/conf/security/nss.fips.cfg",
> "spark.ssl.keyStorePassword":"changeit",
> "spark.ssl.keyStoreType":"PKCS11"
> {quote}
> *Errors :*
> {quote}23/02/21 12:29:43 INFO Master: Running Spark version 3.3.1
> 23/02/21 12:29:43 ERROR MasterWebUI: Failed to bind MasterWebUI
> java.security.KeyStoreException: PKCS11 not found
> at java.base/java.security.KeyStore.getInstance(KeyStore.java:878)
> at
> org.sparkproject.jetty.util.security.CertificateUtils.getKeyStore(CertificateUtils.java:46)
> at
> org.sparkproject.jetty.util.ssl.SslContextFactory.loadKeyStore(SslContextFactory.java:1203)
> at
> org.sparkproject.jetty.util.ssl.SslContextFactory.load(SslContextFactory.java:322)
> at
> org.sparkproject.jetty.util.ssl.SslContextFactory.doStart(SslContextFactory.java:244)
> at
> org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
> org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at
> org.sparkproject.jetty.server.SslConnectionFactory.doStart(SslConnectionFactory.java:97)
> at
> org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at
> org.sparkproject.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
> at
> org.sparkproject.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
> at
> org.sparkproject.jetty.server.AbstractConnector.doStart(AbstractConnector.java:323)
> at
> org.sparkproject.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81)
> at
> org.sparkproject.jetty.server.ServerConnector.doStart(ServerConnector.java:234)
> at
> org.sparkproject.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
> at org.apache.spark.ui.JettyUtils$.newConnector$1(JettyUtils.scala:303)
> at org.apache.spark.ui.JettyUtils$.sslConnect$1(JettyUtils.scala:322)
> at
> org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4(JettyUtils.scala:326)
> at
> org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$4$adapted(JettyUtils.scala:326)
> at
> org.apache.spark.util.Utils$.$anonfun$startServiceOnPort$2(Utils.scala:2401)
> at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:158)
> at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:2393)
> at
> org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2(JettyUtils.scala:326)
> at
> org.apache.spark.ui.JettyUtils$.$anonfun$startJettyServer$2$adapted(JettyUtils.scala:315)
> at scala.Option.map(Option.scala:230)
> at
> org.apache.spark.ui.JettyUtils$.startJettyServer(JettyUtils.scala:315)
> at org.apache.spark.ui.WebUI.initServer(WebUI.scala:144)
> at org.apache.spark.ui.WebUI.bind(WebUI.scala:153)
> at org.apache.spark.deploy.master.Master.onStart(Master.scala:138)
> at org.apache.spark.rpc.netty.Inbox.$anonfun$process$1(Inbox.scala:120)
> at org.apache.spark.rpc.netty.Inbox.safelyCall(Inbox.scala:213)
> at org.apache.spark.rpc.netty.Inbox.process(Inbox.scala:100)
> at
> org.apache.spark.rpc.netty.MessageLoop.org$apache$spark$rpc$netty$MessageLoop$$receiveLoop(MessageLoop.scala:75)
> at
> org.apache.spark.rpc.netty.MessageLoop$$anon$1.run(MessageLoop.scala:41)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:839)
> Caused by: java.security.NoSuchAlgorithmException: PKCS11 KeyStore not
> available
> at
> java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
> at java.base/java.security.Security.getImpl(Security.java:719)
> at java.base/java.security.KeyStore.getInstance(KeyStore.java:875)
> ... 37 more
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
