HyukjinKwon commented on code in PR #58599:
URL: https://github.com/apache/spark/pull/58599#discussion_r3994455180
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config/package.scala:
##########
@@ -301,6 +301,32 @@ package object config extends Logging {
.intConf
.createWithDefault(1)
+ private[spark] val AM_TRUST_PROXY_USER_COOKIE =
+ ConfigBuilder("spark.yarn.am.trustProxyUserCookie")
+ .doc("When true (default), the YARN AM UI filter uses the 'proxy-user'
cookie set by the " +
+ "YARN RM web proxy to determine the user for the AM UI view/modify
ACLs. This forwarded " +
+ "cookie is not cryptographically signed; fully guaranteeing its
integrity would require " +
+ "the YARN RM web proxy to sign it (a Hadoop-side change), so this
option is an interim " +
+ "workaround until then. The AM always installs its own filter first,
so this option is " +
+ "meant for client mode, where a separate authentication filter set in
spark.ui.filters " +
+ "runs after it. The last request wrapper in the chain determines the
user: an " +
+ "authentication filter that wraps the request overrides the cookie
principal on its own, " +
+ "so this option changes nothing there; but a filter that authenticates
without wrapping " +
+ "the request (for example an IP or network allowlist, a token/header
check that only " +
+ "accepts or rejects, or an SSO filter that passes whitelisted paths
straight through) " +
+ "leaves the AM's own request wrapper outermost, so getRemoteUser() is
still the " +
+ "unverified cookie value. That is the case this option is for: setting
it to false makes " +
Review Comment:
Agreed, and thanks for withdrawing the round-1 suggestion. I took the
fail-closed route in 7970d7d: when the cookie is not trusted, `AmIpFilter` now
wraps the request with a sentinel principal (`UNTRUSTED_PROXY_USER`, the empty
string) that is in no ACL, so `SecurityManager` denies proxied requests at the
ACL check rather than passing them as a null user. A downstream authentication
filter in `spark.ui.filters` that wraps the request still replaces the sentinel
with the real user, so the client-mode use is unaffected; without such a filter
(e.g. cluster mode) proxied requests are denied while ACLs are enabled. The
config doc and `running-on-yarn.md` are rewritten accordingly.
##########
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala:
##########
@@ -695,7 +695,20 @@ private[spark] class ApplicationMaster(
/** Add the Yarn IP filter that is required for properly securing the UI. */
private def addAmIpFilter(driver: Option[RpcEndpointRef], proxyBase: String)
= {
val amFilter = classOf[AmIpFilter].getName
- val params = client.getAmIpFilterParams(yarnConf, proxyBase)
+ val baseParams = client.getAmIpFilterParams(yarnConf, proxyBase)
+ val trustProxyUserCookie = sparkConf.get(AM_TRUST_PROXY_USER_COOKIE)
+ // Refuse to arm the option silently: when the cookie is not trusted and
no other UI filter
+ // will run to establish the user (cluster mode replaces spark.ui.filters
with the AM's own
+ // filter; client mode may also have no other filter), proxied requests
carry no user and pass
+ // every view and modify ACL check.
+ if (!trustProxyUserCookie && (driver.isEmpty ||
sparkConf.get(UI_FILTERS).isEmpty)) {
Review Comment:
Done in 7970d7d. The warning now fires whenever the option is off and states
the effect -- proxied requests are treated as an unauthenticated user and are
denied by the AM UI view/modify ACLs unless another authentication filter
establishes the user -- instead of inspecting `UI_FILTERS`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
