Re: Is there a way to get the final web URL from an active Spark context

To answer my own question, it turns out what I was after is the YARN ResourceManager URL for the Spark application. As alluded to in SPARK-20458 , it's possible to use the YARN API client to get this value. Here is a gist that shows how it can

Is there a way to get the final web URL from an active Spark context

Given a session/context, we can get the UI web URL like this: sparkSession.sparkContext.uiWebUrl This gives me something like http://node-name.cluster-name:4040. If opening this from outside the cluster (ex: my laptop), this redirects via HTTP 302 to something like