Hey Holden I can probably help you with that. We should probably pass the same configs to the application also,for the reverse proxy to work.
Try setting the spark.ui.reverseProxy=true while running your application. I hope this will solve your issue. Best Regards Ankit Prakash Gupta On Thu, 26 Aug, 2021, 2:48 am Holden Karau, <hol...@pigscanfly.ca> wrote: > So I tried turning on the Spark exec UI proxy but it broke the Spark UI > (in 3.1.2) and regardless of what URL I requested everything came back as > text/html of the jobs page. Is anyone actively using this feature in prod? > > On Sun, Aug 22, 2021 at 5:58 PM Holden Karau <hol...@pigscanfly.ca> wrote: > >> Oh cool. I’ll have to dig down into why that’s not working with my K8s >> deployment then. >> >> On Sat, Aug 21, 2021 at 11:54 PM Gengliang Wang <ltn...@gmail.com> wrote: >> >>> Hi Holden, >>> >>> FYI there are already some related features in Spark: >>> >>> - Spark Master UI to reverse proxy Application and Workers UI >>> <https://github.com/apache/spark/pull/13950> >>> - Support Spark UI behind front-end reverse proxy using a path >>> prefix Revert proxy URL <https://github.com/apache/spark/pull/29820> >>> >>> Not sure if they are helpful to you. >>> >>> On Sat, Aug 21, 2021 at 3:16 PM Mich Talebzadeh < >>> mich.talebza...@gmail.com> wrote: >>> >>>> Yes I can see your point. >>>> >>>> Will that work in kubernetes deployment? >>>> >>>> Mich >>>> >>>> >>>> view my Linkedin profile >>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>> >>>> >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> >>>> On Sat, 21 Aug 2021 at 00:02, Holden Karau <hol...@pigscanfly.ca> >>>> wrote: >>>> >>>>> Hi Folks, >>>>> >>>>> I'm wondering what people think about the idea of having the Spark UI >>>>> (optionally) act as a proxy to the executors? This could help with exec UI >>>>> access in some deployment environments. >>>>> >>>>> Cheers, >>>>> >>>>> Holden :) >>>>> >>>>> -- >>>>> Twitter: https://twitter.com/holdenkarau >>>>> Books (Learning Spark, High Performance Spark, etc.): >>>>> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >>>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >>>>> >>>> -- >> Twitter: https://twitter.com/holdenkarau >> Books (Learning Spark, High Performance Spark, etc.): >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau >> > > > -- > Twitter: https://twitter.com/holdenkarau > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >
