[
https://issues.apache.org/jira/browse/SPARK-44102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
shamim updated SPARK-44102:
---------------------------
Description:
Getting below error while creating Spark Session using JAVA
Error in executing job:
org.apache.spark.SparkException: Could not parse Master URL:
'k8s://[https://kubernetes.default.svc.cluster.local:443|https://kubernetes.default.svc.cluster.local/]'
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2982)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:563)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2704)
at
org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:953)
at scala.Option.getOrElse(Option.scala:189)
This was working fine with spark version 3.0.1, after upgrading spark version
to 3.3.0. its starting throwing above error.
Below are the Dependency version we used.
Spark Core 3.3.0
Spark Kubernetes Jar 3.3.0
io.fabrics dependency with version 5.12.2
We are not using spark-submit rather submitting spark application using java
code
was:
Getting below error while creating Spark Session using JAVA
Error in executing job:
org.apache.spark.SparkException: Could not parse Master URL:
'k8s://https://kubernetes.default.svc.cluster.local:443'
at
org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2982)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:563)
at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2704)
at
org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:953)
at scala.Option.getOrElse(Option.scala:189)
> Unable to create spark session when master URL set to k8s in spark 3.3.0
> ------------------------------------------------------------------------
>
> Key: SPARK-44102
> URL: https://issues.apache.org/jira/browse/SPARK-44102
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.3.0
> Reporter: shamim
> Priority: Major
>
> Getting below error while creating Spark Session using JAVA
>
> Error in executing job:
> org.apache.spark.SparkException: Could not parse Master URL:
> 'k8s://[https://kubernetes.default.svc.cluster.local:443|https://kubernetes.default.svc.cluster.local/]'
> at
> org.apache.spark.SparkContext$.org$apache$spark$SparkContext$$createTaskScheduler(SparkContext.scala:2982)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:563)
> at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2704)
> at
> org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:953)
> at scala.Option.getOrElse(Option.scala:189)
>
> This was working fine with spark version 3.0.1, after upgrading spark version
> to 3.3.0. its starting throwing above error.
>
> Below are the Dependency version we used.
> Spark Core 3.3.0
> Spark Kubernetes Jar 3.3.0
> io.fabrics dependency with version 5.12.2
>
> We are not using spark-submit rather submitting spark application using java
> code
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
