[jira] [Resolved] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-20044. Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29820 [https://github.com/apache/spark/pull/29820] > Support Spark UI behind front-end reverse proxy using a path prefix > --- > > Key: SPARK-20044 > URL: https://issues.apache.org/jira/browse/SPARK-20044 > Project: Spark > Issue Type: Improvement > Components: Web UI >Affects Versions: 2.1.0, 2.2.0, 2.3.0, 2.4.0, 3.0.0 >Reporter: Oliver Koeth >Assignee: Gengliang Wang >Priority: Minor > Labels: reverse-proxy, sso > Fix For: 3.1.0 > > > Purpose: allow to run the Spark web UI behind a reverse proxy with URLs > prefixed by a context root, like www.mydomain.com/spark. In particular, this > allows to access multiple Spark clusters through the same virtual host, only > distinguishing them by context root, like www.mydomain.com/cluster1, > www.mydomain.com/cluster2, and it allows to run the Spark UI in a common > cookie domain (for SSO) with other services. > [SPARK-15487] introduced some support for front-end reverse proxies by > allowing all Spark UI requests to be routed through the master UI as a single > endpoint and also added a spark.ui.reverseProxyUrl setting to define a > another proxy sitting in front of Spark. However, as noted in the comments on > [SPARK-15487], this mechanism does not currently work if the reverseProxyUrl > includes a context root like the examples above: Most links generated by the > Spark UI result in full path URLs (like /proxy/app-"id"/...) that do not > account for a path prefix (context root) and work only if the Spark UI "owns" > the entire virtual host. In fact, the only place in the UI where the > reverseProxyUrl seems to be used is the back-link from the worker UI to the > master UI. > The discussion on [SPARK-15487] proposes to open a new issue for the problem, > but that does not seem to have happened, so this issue aims to address the > remaining shortcomings of spark.ui.reverseProxyUrl > The problem can be partially worked around by doing content rewrite in a > front-end proxy and prefixing src="/..." or href="/..." links with a context > root. However, detecting and patching URLs in HTML output is not a robust > approach and breaks down for URLs included in custom REST responses. E.g. the > "allexecutors" REST call used from the Spark 2.1.0 application/executors page > returns links for log viewing that direct to the worker UI and do not work in > this scenario. > This issue proposes to honor spark.ui.reverseProxyUrl throughout Spark UI URL > generation. Experiments indicate that most of this can simply be achieved by > using/prepending spark.ui.reverseProxyUrl to the existing spark.ui.proxyBase > system property. Beyond that, the places that require adaption are > - worker and application links in the master web UI > - webui URLs returned by REST interfaces > Note: It seems that returned redirect location headers do not need to be > adapted, since URL rewriting for these is commonly done in front-end proxies > and has a well-defined interface -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-20044) Support Spark UI behind front-end reverse proxy using a path prefix
[ https://issues.apache.org/jira/browse/SPARK-20044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20044. --- Resolution: Won't Fix > Support Spark UI behind front-end reverse proxy using a path prefix > --- > > Key: SPARK-20044 > URL: https://issues.apache.org/jira/browse/SPARK-20044 > Project: Spark > Issue Type: Improvement > Components: Web UI >Affects Versions: 2.1.0 >Reporter: Oliver Koeth >Priority: Minor > Labels: reverse-proxy, sso > > Purpose: allow to run the Spark web UI behind a reverse proxy with URLs > prefixed by a context root, like www.mydomain.com/spark. In particular, this > allows to access multiple Spark clusters through the same virtual host, only > distinguishing them by context root, like www.mydomain.com/cluster1, > www.mydomain.com/cluster2, and it allows to run the Spark UI in a common > cookie domain (for SSO) with other services. > [SPARK-15487] introduced some support for front-end reverse proxies by > allowing all Spark UI requests to be routed through the master UI as a single > endpoint and also added a spark.ui.reverseProxyUrl setting to define a > another proxy sitting in front of Spark. However, as noted in the comments on > [SPARK-15487], this mechanism does not currently work if the reverseProxyUrl > includes a context root like the examples above: Most links generated by the > Spark UI result in full path URLs (like /proxy/app-"id"/...) that do not > account for a path prefix (context root) and work only if the Spark UI "owns" > the entire virtual host. In fact, the only place in the UI where the > reverseProxyUrl seems to be used is the back-link from the worker UI to the > master UI. > The discussion on [SPARK-15487] proposes to open a new issue for the problem, > but that does not seem to have happened, so this issue aims to address the > remaining shortcomings of spark.ui.reverseProxyUrl > The problem can be partially worked around by doing content rewrite in a > front-end proxy and prefixing src="/..." or href="/..." links with a context > root. However, detecting and patching URLs in HTML output is not a robust > approach and breaks down for URLs included in custom REST responses. E.g. the > "allexecutors" REST call used from the Spark 2.1.0 application/executors page > returns links for log viewing that direct to the worker UI and do not work in > this scenario. > This issue proposes to honor spark.ui.reverseProxyUrl throughout Spark UI URL > generation. Experiments indicate that most of this can simply be achieved by > using/prepending spark.ui.reverseProxyUrl to the existing spark.ui.proxyBase > system property. Beyond that, the places that require adaption are > - worker and application links in the master web UI > - webui URLs returned by REST interfaces > Note: It seems that returned redirect location headers do not need to be > adapted, since URL rewriting for these is commonly done in front-end proxies > and has a well-defined interface -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
