[
https://issues.apache.org/jira/browse/KAFKA-14496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Endre Vig updated KAFKA-14496:
------------------------------
Description:
Currently our team is setting up a blueprint for our Kafka consumers/producers
to provide guidelines on how to connect to our broker using the OIDC security
mechanism. The blueprint is written in Java using the latest 3.3.1 Kafka
library dependencies managed by Spring Boot 3.0.0.
While trying to use the new built-in
{{org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler}}
introduced by [KIP-768: Extend SASL/OAUTHBEARER with Support for
OIDC|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=186877575],
we've noticed that some calls to retrieve the token work out well, while some
of them (seemingly randomly) are failing with 401 Unauthorized.
After some debugging we've got to the conclusion that the faulty behavior is
caused by
{{org.apache.kafka.common.security.oauthbearer.secured.HttpAccessTokenRetriever#formatAuthorizationHeader:}}
{code:java}
static String formatAuthorizationHeader(String clientId, String clientSecret) {
clientId = sanitizeString("the token endpoint request client ID parameter",
clientId);
clientSecret = sanitizeString("the token endpoint request client secret
parameter", clientSecret);
String s = String.format("%s:%s", clientId, clientSecret);
String encoded = Base64.getUrlEncoder().encodeToString(Utils.utf8(s));
return String.format("Basic %s", encoded);
} {code}
The above code is using {{java.util.Base64#getUrlEncoder}} on line 311 to
encode the authorization header value, which is using the alphabet described in
[section 5 of the RFC|https://www.rfc-editor.org/rfc/rfc4648#section-5] during
the encoding algorithm. As stated by the Basic Authentication Scheme
[definition|https://www.rfc-editor.org/rfc/rfc7617#section-2] however, [section
4 of the RFC|https://www.rfc-editor.org/rfc/rfc4648#section-4] should be used:
??4. and obtains the basic-credentials by encoding this octet sequence using
Base64 ([RFC4648], Section 4) into a sequence of US-ASCII characters
([RFC0020]).??
The difference between the 2 alphabets are only on two characters (62: '+' vs.
'-' and 63: '/' vs. '_'), that's why the 401 Unauthorized response arises only
for certain credential values.
Here's a concrete example use case:
{code:java}
String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234",
"9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getUrlEncoder().encodeToString(Utils.utf8(s))); {code}
would print out:
{code:java}
U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl-dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ==
{code}
while
{code:java}
String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234",
"9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); {code}
would give:
{code:java}
U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl+dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ==
{code}
Please notice the '-' vs. '+' characters.
The 2 code snippets above would not behave differently for other credentials,
where the encoded result doesn't use the 62nd character of the alphabet:
{code:java}
String s = String.format("%s:%s", "SHORT_USER_01234",
"9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); {code}
{code:java}
U0hPUlRfVVNFUl8wMTIzNDo5UXwwYDhpfnV0ZS1uOWtzakxXYlw1MCJBWEBVVUVENUU=
{code}
As a *conclusion* I would suggest that line 311 of {{HttpAccessTokenRetriever}}
should be modified to use {{Base64.getEncoder().encodeToString(...)}} instead
of {{Base64.getUrlEncoder().encodeToString(...).}}
I'm attaching a short sample application with tests proving that the above
encoding method is rejected by the standard Spring Security HTTP basic
authentication as well.
was:
Currently our team is setting up a blueprint for our Kafka consumers/producers
to provide guidelines on how to connect to our broker using the OIDC security
mechanism. The blueprint is written in Java using the latest 3.3.1 Kafka
library dependencies managed by Spring Boot 3.0.0.
While trying to use the new built-in
{{org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler}}
introduced by [KIP-768: Extend SASL/OAUTHBEARER with Support for
OIDC|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=186877575],
we've noticed that some calls to retrieve the token work out well, while some
of them (seemingly randomly) are failing with 401 Unauthorized.
After some debugging we've got to the conclusion that the faulty behavior is
caused by
{{org.apache.kafka.common.security.oauthbearer.secured.HttpAccessTokenRetriever#formatAuthorizationHeader:}}
{code:java}
static String formatAuthorizationHeader(String clientId, String clientSecret) {
clientId = sanitizeString("the token endpoint request client ID parameter",
clientId);
clientSecret = sanitizeString("the token endpoint request client secret
parameter", clientSecret); String s = String.format("%s:%s", clientId,
clientSecret);
String encoded = Base64.getUrlEncoder().encodeToString(Utils.utf8(s));
return String.format("Basic %s", encoded);
} {code}
The above code is using {{java.util.Base64#getUrlEncoder}} on line 311 to
encode the authorization header value, which is using the alphabet described in
[section 5 of the RFC|https://www.rfc-editor.org/rfc/rfc4648#section-5] during
the encoding algorithm. As stated by the Basic Authentication Scheme
[definition|https://www.rfc-editor.org/rfc/rfc7617#section-2] however, [section
4 of the RFC|https://www.rfc-editor.org/rfc/rfc4648#section-4] should be used:
??4. and obtains the basic-credentials by encoding this octet sequence using
Base64 ([RFC4648], Section 4) into a sequence of US-ASCII characters
([RFC0020]).??
The difference between the 2 alphabets are only on two characters (62: '+' vs.
'-' and 63: '/' vs. '_'), that's why the 401 Unauthorized response arises only
for certain credential values.
Here's a concrete example use case:
{code:java}
String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234",
"9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getUrlEncoder().encodeToString(Utils.utf8(s))); {code}
would print out:
{code:java}
U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl-dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ==
{code}
while
{code:java}
String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234",
"9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); {code}
would give:
{code:java}
U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl+dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ==
{code}
Please notice the '-' vs. '+' characters.
The 2 code snippets above would not behave differently for different
credentials, where the encoded result doesn't use the 62th character of the
alphabet:
{code:java}
String s = String.format("%s:%s", "SHORT_USER_01234",
"9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); {code}
{code:java}
U0hPUlRfVVNFUl8wMTIzNDo5UXwwYDhpfnV0ZS1uOWtzakxXYlw1MCJBWEBVVUVENUU=
{code}
So as a *conclusion* I would suggest that line 311 of
{{HttpAccessTokenRetriever}} should be modified to use
{{Base64.getEncoder().encodeToString(...)}} instead of
{{Base64.getUrlEncoder().encodeToString(...).}}
I'm attaching a short sample application with tests proving that the above
encoding method is rejected by the standard Spring Security HTTP basic
authentication as well.
> Wrong Base64 encoder used by OIDC OAuthBearerLoginCallbackHandler
> -----------------------------------------------------------------
>
> Key: KAFKA-14496
> URL: https://issues.apache.org/jira/browse/KAFKA-14496
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 3.3.1
> Reporter: Endre Vig
> Priority: Major
> Attachments: base64test.zip
>
>
> Currently our team is setting up a blueprint for our Kafka
> consumers/producers to provide guidelines on how to connect to our broker
> using the OIDC security mechanism. The blueprint is written in Java using the
> latest 3.3.1 Kafka library dependencies managed by Spring Boot 3.0.0.
> While trying to use the new built-in
> {{org.apache.kafka.common.security.oauthbearer.secured.OAuthBearerLoginCallbackHandler}}
> introduced by [KIP-768: Extend SASL/OAUTHBEARER with Support for
> OIDC|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=186877575],
> we've noticed that some calls to retrieve the token work out well, while
> some of them (seemingly randomly) are failing with 401 Unauthorized.
> After some debugging we've got to the conclusion that the faulty behavior is
> caused by
> {{org.apache.kafka.common.security.oauthbearer.secured.HttpAccessTokenRetriever#formatAuthorizationHeader:}}
> {code:java}
> static String formatAuthorizationHeader(String clientId, String clientSecret)
> {
> clientId = sanitizeString("the token endpoint request client ID
> parameter", clientId);
> clientSecret = sanitizeString("the token endpoint request client secret
> parameter", clientSecret);
>
> String s = String.format("%s:%s", clientId, clientSecret);
> String encoded = Base64.getUrlEncoder().encodeToString(Utils.utf8(s));
> return String.format("Basic %s", encoded);
> } {code}
> The above code is using {{java.util.Base64#getUrlEncoder}} on line 311 to
> encode the authorization header value, which is using the alphabet described
> in [section 5 of the RFC|https://www.rfc-editor.org/rfc/rfc4648#section-5]
> during the encoding algorithm. As stated by the Basic Authentication Scheme
> [definition|https://www.rfc-editor.org/rfc/rfc7617#section-2] however,
> [section 4 of the RFC|https://www.rfc-editor.org/rfc/rfc4648#section-4]
> should be used:
> ??4. and obtains the basic-credentials by encoding this octet sequence using
> Base64 ([RFC4648], Section 4) into a sequence of US-ASCII characters
> ([RFC0020]).??
> The difference between the 2 alphabets are only on two characters (62: '+'
> vs. '-' and 63: '/' vs. '_'), that's why the 401 Unauthorized response arises
> only for certain credential values.
> Here's a concrete example use case:
>
> {code:java}
> String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234",
> "9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
> System.out.println(Base64.getUrlEncoder().encodeToString(Utils.utf8(s)));
> {code}
> would print out:
> {code:java}
> U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl-dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ==
> {code}
> while
> {code:java}
> String s = String.format("%s:%s", "SOME_RANDOM_LONG_USER_01234",
> "9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
> System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); {code}
> would give:
> {code:java}
> U09NRV9SQU5ET01fTE9OR19VU0VSXzAxMjM0OjlRfDBgOGl+dXRlLW45a3NqTFdiXDUwIkFYQFVVRUQ1RQ==
> {code}
> Please notice the '-' vs. '+' characters.
>
> The 2 code snippets above would not behave differently for other credentials,
> where the encoded result doesn't use the 62nd character of the alphabet:
> {code:java}
> String s = String.format("%s:%s", "SHORT_USER_01234",
> "9Q|0`8i~ute-n9ksjLWb\\50\"AX@UUED5E");
> System.out.println(Base64.getEncoder().encodeToString(Utils.utf8(s))); {code}
> {code:java}
> U0hPUlRfVVNFUl8wMTIzNDo5UXwwYDhpfnV0ZS1uOWtzakxXYlw1MCJBWEBVVUVENUU=
> {code}
>
> As a *conclusion* I would suggest that line 311 of
> {{HttpAccessTokenRetriever}} should be modified to use
> {{Base64.getEncoder().encodeToString(...)}} instead of
> {{Base64.getUrlEncoder().encodeToString(...).}}
>
> I'm attaching a short sample application with tests proving that the above
> encoding method is rejected by the standard Spring Security HTTP basic
> authentication as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
