Re: Issue with Spark Session Initialization in Kubernetes Deployment

2024-01-05 Thread Mich Talebzadeh
Hi, I personally do not use the Spark operator. Anyhow, the Spark Operator automates the deployment and management of Spark applications within Kubernetes. However, it does not eliminate the need to configure Spark sessions for proper communication with the k8 cluster. So specifying the master

Issue with Spark Session Initialization in Kubernetes Deployment

2024-01-04 Thread Atul Patil
Hello Team, I am currently working on initializing a Spark session using Spark Structure Streaming within a Kubernetes deployment managed by the Spark Operator. During the initialization process, I encountered an error message indicating the necessity to set a master URL: *"Caused by: