Hello,
We are facing issue with ingress during spark-submit with kubernetes cluster
1.24.x .
We are using spark 3.3.0 to do spark-submit.
# kubectl version
WARNING: This version information is deprecated and will be replaced with the
output from kubectl version --short. Use --output=yaml|json to get the full
version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9",
GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean",
BuildDate:"2022-12-08T10:15:09Z", GoVersion:"go1.18.9", Compiler:"gc",
Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9",
GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean",
BuildDate:"2022-12-08T10:08:06Z", GoVersion:"go1.18.9", Compiler:"gc",
Platform:"linux/amd64"}
Error:
{"type":"log", "level":"WARN", "time":"2022-12-23T10:04:57.536Z",
"timezone":"UTC", "log":"The client is using resource type 'ingresses' with
unstable version 'v1beta1'"}
Exception in thread "main"
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST
at: https://xx.xx.xx.xx:6443/apis/extensions/v1beta1/namespaces/test/ingresses.
Message: Not Found.
>From spark code we see that spark 3.3.0 is using
>"<kubernetes-client.version>5.12.2</kubernetes-client.version>"
We also tried changing this to 6.x as in PR :
https://github.com/apache/spark/pull/37990/files
Still we facing same issue.
Let us know if spark is tested/working with k8s 1.24.x cluster and what could
be the solution to resolve this.
Thanks.
Regards
Thimme Gowda
