Viewing UI for spark jobs running on K8s

2023-05-31 Thread Nikhil Goyal
Hi folks,
Is there an equivalent of the Yarn RM page for Spark on Kubernetes. We can
port-forward the UI from the driver pod for each but this process is
tedious given we have multiple jobs running. Is there a clever solution to
exposing all Driver UIs in a centralized place?

Thanks
Nikhil


Re: Viewing UI for spark jobs running on K8s

2023-05-31 Thread Qian Sun
Hi Nikhil

Spark operator supports ingress for exposing all UIs of running spark
applications.

reference:
https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/master/docs/quick-start-guide.md#driver-ui-access-and-ingress

On Thu, Jun 1, 2023 at 6:19 AM Nikhil Goyal  wrote:

> Hi folks,
> Is there an equivalent of the Yarn RM page for Spark on Kubernetes. We can
> port-forward the UI from the driver pod for each but this process is
> tedious given we have multiple jobs running. Is there a clever solution to
> exposing all Driver UIs in a centralized place?
>
> Thanks
> Nikhil
>

-- 
Regards,
Qian Sun