This is an automated email from the ASF dual-hosted git repository.
RexXiong pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 9a088699ba [CELEBORN-2474] Send WWW-Authenticate challenge when no
auth header present
9a088699ba is described below
commit 9a088699ba1689136c4247947176de29fb6b74fb
Author: senthh <[email protected]>
AuthorDate: Mon Oct 12 22:23:00 2026 +0800
[CELEBORN-2474] Send WWW-Authenticate challenge when no auth header present
### What changes were proposed in this pull request?
When `AuthenticationFilter.doFilter` receives a request with no
`Authorization` header (the first request in a SPNEGO/Kerberos handshake),
respond `401` with `WWW-Authenticate` headers listing all registered auth
schemes, so a SPNEGO client (e.g. `curl --negotiate`) can receive the challenge
and respond with its Kerberos token. If an `Authorization` header is present
but matches no handler, the original error message is preserved.
Also fix the scheme name in the `supportedSchemes` documentation
(`celeborn.master.http.auth.supportedSchemes` /
`celeborn.worker.http.auth.supportedSchemes`): the Kerberos scheme was listed
as `SPNEGO` but the enumeration value is `NEGOTIATE`. Updated
`CelebornConf.scala`, `docs/configuration/master.md` and
`docs/configuration/worker.md` for consistency with the `ConfigurationSuite`
golden-file check.
Files:
-
`service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala`
- `common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala`
- `docs/configuration/master.md`
- `docs/configuration/worker.md`
### Why are the changes needed?
Previously the filter returned a bare `401` with no `WWW-Authenticate`
header, so the SPNEGO handshake was broken at the very first step (error: "No
auth scheme matched for null"). A SPNEGO client never receives the challenge
and cannot send its Kerberos token.
### Does this PR resolve a correctness bug?
- [ ] Yes
### Does this PR introduce _any_ user-facing change?
- [ ] Yes
### How was this patch tested?
Verified with `curl --negotiate` against a Celeborn server configured with:
- `celeborn.master.http.auth.supportedSchemes NEGOTIATE`
- `celeborn.master.http.spnego.keytab /path/to/keytab`
- `celeborn.master.http.spnego.principal HTTP/hostREALM`
Before: `401` without `WWW-Authenticate` header, curl never sends the token.
After: `401` with `WWW-Authenticate: NEGOTIATE`, curl completes the SPNEGO
handshake and returns `200`.
Closes #3863 from senthh/CELEBORN-9999.
Authored-by: senthh <[email protected]>
Signed-off-by: Shuang <[email protected]>
---
.../org/apache/celeborn/common/CelebornConf.scala | 4 ++--
docs/configuration/master.md | 2 +-
docs/configuration/worker.md | 2 +-
.../server/common/http/HttpAuthUtils.scala | 16 ++++++++++++++
.../http/authentication/AuthenticationFilter.scala | 24 +++++++++++++++++----
.../BasicAuthenticationHandler.scala | 4 +++-
.../http/ApiBaseResourceAuthenticationSuite.scala | 25 ++++++++++++++++++++++
7 files changed, 68 insertions(+), 9 deletions(-)
diff --git
a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
index 3eef83205e..3391e3d169 100644
--- a/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
+++ b/common/src/main/scala/org/apache/celeborn/common/CelebornConf.scala
@@ -2655,7 +2655,7 @@ object CelebornConf extends Logging {
.version("0.6.0")
.doc("A comma-separated list of master http auth supported schemes." +
"<ul>" +
- " <li>SPNEGO: Kerberos/GSSAPI authentication.</li>" +
+ " <li>NEGOTIATE: Kerberos/GSSAPI authentication.</li>" +
" <li>BASIC: User-defined password authentication, the concreted
implementation is" +
" configurable via `celeborn.master.http.auth.basic.provider`.</li>" +
" <li>BEARER: User-defined bearer token authentication, the concreted
implementation is" +
@@ -3649,7 +3649,7 @@ object CelebornConf extends Logging {
.version("0.6.0")
.doc("A comma-separated list of worker http auth supported schemes." +
"<ul>" +
- " <li>SPNEGO: Kerberos/GSSAPI authentication.</li>" +
+ " <li>NEGOTIATE: Kerberos/GSSAPI authentication.</li>" +
" <li>BASIC: User-defined password authentication, the concreted
implementation is" +
" configurable via `celeborn.worker.http.auth.basic.provider`.</li>" +
" <li>BEARER: User-defined bearer token authentication, the concreted
implementation is" +
diff --git a/docs/configuration/master.md b/docs/configuration/master.md
index 86a4158668..8e5b27c3f0 100644
--- a/docs/configuration/master.md
+++ b/docs/configuration/master.md
@@ -53,7 +53,7 @@ license: |
| celeborn.master.http.auth.administers | | false | A comma-separated list of
users who have admin privileges, Note, when
celeborn.master.http.auth.supportedSchemes is not set, everyone is treated as
administrator. | 0.6.0 | |
| celeborn.master.http.auth.basic.provider |
org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl |
false | User-defined password authentication implementation of
org.apache.celeborn.spi.authentication.PasswdAuthenticationProvider | 0.6.0 |
|
| celeborn.master.http.auth.bearer.provider |
org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl |
false | User-defined token authentication implementation of
org.apache.celeborn.spi.authentication.TokenAuthenticationProvider | 0.6.0 | |
-| celeborn.master.http.auth.supportedSchemes | | false | A comma-separated
list of master http auth supported schemes.<ul> <li>SPNEGO: Kerberos/GSSAPI
authentication.</li> <li>BASIC: User-defined password authentication, the
concreted implementation is configurable via
`celeborn.master.http.auth.basic.provider`.</li> <li>BEARER: User-defined
bearer token authentication, the concreted implementation is configurable via
`celeborn.master.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
+| celeborn.master.http.auth.supportedSchemes | | false | A comma-separated
list of master http auth supported schemes.<ul> <li>NEGOTIATE: Kerberos/GSSAPI
authentication.</li> <li>BASIC: User-defined password authentication, the
concreted implementation is configurable via
`celeborn.master.http.auth.basic.provider`.</li> <li>BEARER: User-defined
bearer token authentication, the concreted implementation is configurable via
`celeborn.master.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
| celeborn.master.http.host | <localhost> | false | Master's http host.
| 0.4.0 |
celeborn.metrics.master.prometheus.host,celeborn.master.metrics.prometheus.host
|
| celeborn.master.http.idleTimeout | 30s | false | Master http server idle
timeout. | 0.5.0 | |
| celeborn.master.http.maxWorkerThreads | 200 | false | Maximum number of
threads in the master http worker thread pool. | 0.5.0 | |
diff --git a/docs/configuration/worker.md b/docs/configuration/worker.md
index f33a169936..dd0aa15a55 100644
--- a/docs/configuration/worker.md
+++ b/docs/configuration/worker.md
@@ -118,7 +118,7 @@ license: |
| celeborn.worker.http.auth.administers | | false | A comma-separated list of
users who have admin privileges, Note, when
celeborn.worker.http.auth.supportedSchemes is not set, everyone is treated as
administrator. | 0.6.0 | |
| celeborn.worker.http.auth.basic.provider |
org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl |
false | User-defined password authentication implementation of
org.apache.celeborn.common.authentication.PasswdAuthenticationProvider | 0.6.0
| |
| celeborn.worker.http.auth.bearer.provider |
org.apache.celeborn.common.authentication.AnonymousAuthenticationProviderImpl |
false | User-defined token authentication implementation of
org.apache.celeborn.common.authentication.TokenAuthenticationProvider | 0.6.0 |
|
-| celeborn.worker.http.auth.supportedSchemes | | false | A comma-separated
list of worker http auth supported schemes.<ul> <li>SPNEGO: Kerberos/GSSAPI
authentication.</li> <li>BASIC: User-defined password authentication, the
concreted implementation is configurable via
`celeborn.worker.http.auth.basic.provider`.</li> <li>BEARER: User-defined
bearer token authentication, the concreted implementation is configurable via
`celeborn.worker.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
+| celeborn.worker.http.auth.supportedSchemes | | false | A comma-separated
list of worker http auth supported schemes.<ul> <li>NEGOTIATE: Kerberos/GSSAPI
authentication.</li> <li>BASIC: User-defined password authentication, the
concreted implementation is configurable via
`celeborn.worker.http.auth.basic.provider`.</li> <li>BEARER: User-defined
bearer token authentication, the concreted implementation is configurable via
`celeborn.worker.http.auth.bearer.provider`.</li></ul> | 0.6.0 | |
| celeborn.worker.http.host | <localhost> | false | Worker's http host.
| 0.4.0 |
celeborn.metrics.worker.prometheus.host,celeborn.worker.metrics.prometheus.host
|
| celeborn.worker.http.idleTimeout | 30s | false | Worker http server idle
timeout. | 0.5.0 | |
| celeborn.worker.http.maxWorkerThreads | 200 | false | Maximum number of
threads in the worker http worker thread pool. | 0.5.0 | |
diff --git
a/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
b/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
index 61d028bee9..47072c9d19 100644
---
a/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
+++
b/service/src/main/scala/org/apache/celeborn/server/common/http/HttpAuthUtils.scala
@@ -21,6 +21,7 @@ import java.util.{Map => JMap}
import scala.collection.JavaConverters._
+import org.apache.celeborn.common.authentication.HttpAuthSchemes
import
org.apache.celeborn.server.common.http.authentication.AuthenticationFilter
import org.apache.celeborn.spi.authentication.Credential
@@ -29,6 +30,21 @@ object HttpAuthUtils {
val WWW_AUTHENTICATE_HEADER = "WWW-Authenticate"
// HTTP header used by the client endpoint during an authentication sequence.
val AUTHORIZATION_HEADER = "Authorization"
+ // Realm advertised in the Basic authentication challenge (RFC 7617 requires
a realm).
+ val BASIC_AUTH_REALM = "Celeborn"
+
+ /**
+ * Build the value of a single WWW-Authenticate challenge header for an auth
scheme.
+ *
+ * The scheme token is case-insensitive on the wire, so we emit the
enumeration name (e.g.
+ * NEGOTIATE / BEARER) for most schemes, but RFC 7617 mandates a realm for
Basic, so BASIC is
+ * emitted with one: `Basic realm="Celeborn"`. This is the challenge a
client (curl, HTTP
+ * auth handlers) uses to decide it can start an auth handshake.
+ */
+ def challengeHeaderValue(scheme: HttpAuthSchemes.HttpAuthScheme): String =
scheme match {
+ case HttpAuthSchemes.BASIC => s"""Basic realm="$BASIC_AUTH_REALM""""
+ case _ => scheme.toString
+ }
def getCredentialExtraInfo: JMap[String, String] = {
Map(Credential.CLIENT_IP_KEY ->
diff --git
a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
index e0a9a8bbac..99d9cee6e8 100644
---
a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
+++
b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/AuthenticationFilter.scala
@@ -30,6 +30,7 @@ import
org.apache.celeborn.common.authentication.HttpAuthSchemes
import org.apache.celeborn.common.authentication.HttpAuthSchemes.HttpAuthScheme
import org.apache.celeborn.common.internal.Logging
import org.apache.celeborn.server.common.Service
+import org.apache.celeborn.server.common.http.HttpAuthUtils
import
org.apache.celeborn.server.common.http.HttpAuthUtils.AUTHORIZATION_HEADER
import org.apache.celeborn.server.common.http.RestAuditLogger
@@ -162,10 +163,25 @@ class AuthenticationFilter(conf: CelebornConf,
serviceName: String) extends Filt
try {
if (matchedHandler == null) {
logDebug(s"No auth scheme matched for url:
${httpRequest.getRequestURL}")
- httpResponse.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
- httpResponse.sendError(
- HttpServletResponse.SC_UNAUTHORIZED,
- s"No auth scheme matched for $authorization")
+ if (authorization == null || authorization.isEmpty) {
+ // First request with no Authorization header. Send WWW-Authenticate
+ // challenge(s) to initiate auth handshake (e.g. SPNEGO/Kerberos
+ // requires a 401+WWW-Authenticate to start the handshake).
+ authSchemeHandlers.keys.foreach { scheme =>
+ httpResponse.addHeader(
+ HttpAuthUtils.WWW_AUTHENTICATE_HEADER,
+ HttpAuthUtils.challengeHeaderValue(scheme))
+ }
+ httpResponse.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
+ httpResponse.setContentType("text/html;charset=iso-8859-1")
+ httpResponse.getWriter.write(
+ "<html><body><h2>HTTP ERROR 401 Unauthorized</h2></body></html>")
+ httpResponse.getWriter.flush
+ } else {
+ httpResponse.sendError(
+ HttpServletResponse.SC_UNAUTHORIZED,
+ s"No auth scheme matched for $authorization")
+ }
} else {
HTTP_AUTH_TYPE.set(matchedHandler.authScheme.toString)
HTTP_CLIENT_IDENTIFIER.set(matchedHandler.authenticate(httpRequest,
httpResponse))
diff --git
a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
index e5d8ba5399..6d17fe6e0a 100644
---
a/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
+++
b/service/src/main/scala/org/apache/celeborn/server/common/http/authentication/BasicAuthenticationHandler.scala
@@ -81,7 +81,9 @@ class BasicAuthenticationHandler(providerClass: String)
extends AuthenticationHa
authUser =
creds.take(1).headOption.filterNot(_.isEmpty).getOrElse("anonymous")
} else {
if (creds.size < 2 || creds(0).trim.isEmpty || creds(1).trim.isEmpty) {
- response.setHeader(WWW_AUTHENTICATE_HEADER, authScheme.toString)
+ response.setHeader(
+ WWW_AUTHENTICATE_HEADER,
+ HttpAuthUtils.challengeHeaderValue(authScheme))
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED)
} else {
val Seq(user, password) = creds.toSeq.take(2)
diff --git
a/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
b/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
index 8fb95c7ef7..677b6b1707 100644
---
a/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
+++
b/service/src/test/scala/org/apache/celeborn/server/common/http/ApiBaseResourceAuthenticationSuite.scala
@@ -22,6 +22,8 @@ import java.util.Base64
import javax.servlet.http.HttpServletResponse
import javax.ws.rs.core.MediaType
+import scala.collection.JavaConverters._
+
import org.apache.celeborn.common.CelebornConf
import org.apache.celeborn.common.authentication.HttpAuthSchemes
import org.apache.celeborn.common.network.TestHelper
@@ -151,4 +153,27 @@ abstract class ApiBaseResourceAuthenticationSuite extends
HttpTestHelper {
}
}
}
+
+ test("request without Authorization header gets 401 with WWW-Authenticate
challenges") {
+ // The suite configures BASIC + BEARER schemes; a request with no
Authorization header must be
+ // challenged with one WWW-Authenticate value per registered scheme. The
Basic challenge carries
+ // a realm (RFC 7617), other schemes use their bare token.
+ val response =
webTarget.path("shuffle").request(MediaType.TEXT_PLAIN).get()
+ assert(HttpServletResponse.SC_UNAUTHORIZED == response.getStatus)
+
+ val challenges =
+
response.getStringHeaders.get(HttpAuthUtils.WWW_AUTHENTICATE_HEADER).asScala.toSet
+ assert(challenges.contains("Basic realm=\"Celeborn\""))
+ assert(challenges.contains(HttpAuthSchemes.BEARER.toString))
+ }
+
+ test("request with unmatched Authorization scheme keeps 401 and original
error") {
+ // An Authorization header that matches no configured handler must still
401 and must NOT be
+ // challenged (no new WWW-Authenticate added for it).
+ val response = webTarget.path("shuffle")
+ .request(MediaType.TEXT_PLAIN)
+ .header(AUTHORIZATION_HEADER, "Digest realm=\"x\"")
+ .get()
+ assert(HttpServletResponse.SC_UNAUTHORIZED == response.getStatus)
+ }
}
