This is an automated email from the ASF dual-hosted git repository.
CalvinKirs pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new fccaa75d465 [fix](arrow-flight) Stop writing bearer tokens to fe.log
(#66572)
fccaa75d465 is described below
commit fccaa75d465da2af1615e98d8751f82591c304d2
Author: Calvin Kirs <[email protected]>
AuthorDate: Tue Aug 11 10:10:04 2026 +0800
[fix](arrow-flight) Stop writing bearer tokens to fe.log (#66572)
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Arrow Flight SQL bearer tokens are written to `fe.log` in cleartext.
`FlightTokenManagerImpl` logs the token verbatim at INFO when it is
minted, evicted from either cache, and invalidated, and it also puts the
token into the `IllegalArgumentException` messages that
`FlightBearerTokenAuthenticator.validateBearer` logs at ERROR:
```java
LOG.info("Created flight token for user: {}, token: {}", username, token);
```
A bearer token is a complete credential until it expires —
`arrow_flight_token_alive_time_second` defaults to 86400s. So anyone who
can read `fe.log`, or the log aggregation platform it is shipped to, or
a backup of either, can take a live token, send it as `Authorization:
Bearer <token>` to the Arrow Flight SQL port (`arrow_flight_sql_port`,
default 8070), and run queries as that user without ever knowing their
password. Logs routinely reach a much wider audience than the credential
store does, which is what makes this worth fixing even though the log
file itself is not world readable.
**What this PR does**
Adds `org.apache.doris.common.util.TokenMasker`, which offers the two
renderings a secret can reasonably have in a message:
- `tokenId(t)` → `sha256:1a2b3c4d`, a truncated SHA-256. It is stable,
so a log line and the error message returned to the client still point
at the same token and can be matched up, but no part of the secret
survives in it. This is what the flight token paths now use. The
existing "search for this token in fe.log to see the evict reason" hint
therefore still works — it now says *token id*, and the id appears both
in the client's error and in the log.
- `maskPrefix(t)` → `abc***`, revealing only a short leading prefix, for
the case where a human has to recognize *which* configured secret was
involved (token rotation). This is the helper that already existed
privately in `MetaService`; it is moved into the utility and reused
rather than duplicated.
Every token-valued site in the Arrow Flight path is converted: the four
`LOG.info` calls in `FlightTokenManagerImpl`, the four
`IllegalArgumentException` messages in
`validateToken`/`getTokenDetails`, the one in
`FlightSessionsWithTokenManager.createConnectContext`, and the teardown
warning in `FlightSqlConnectPoolMgr.unregisterConnection`. That last one
is worth spelling out: a Flight SQL `ConnectContext`'s **`peerIdentity`
is the bearer token itself** —
`FlightBearerTokenAuthenticator.createAuthResultWithBearerToken` returns
the token as the peer identity, and `FlightSqlConnectPoolMgr` keys its
`flightToken2ConnectionId` map by it — so `ctx.getPeerIdentity()` in a
log line leaks a live token under a name that does not look like one.
Two more credentials with the same problem, found while auditing for
other instances:
- `Env` logs the cluster token adopted from a helper node at INFO (`get
token from helper node. token={}`). That token authenticates metadata
access between FE nodes, so it gets `maskPrefix`, consistent with how
`MetaService` already renders the same token.
- `Auth` echoes `initial_root_password` into a WARN — and it does so
from the branch that runs when the configured value failed 2-staged
SHA-1 validation, which is exactly the case where an operator put a
plaintext password in the config. The value is simply dropped from the
message; it adds nothing to the diagnosis that the config key name does
not already give.
Finally, a checkstyle rule rejects a value whose name says it holds a
token/password/secret/peer identity being passed straight into a
`LOG.x(...)` call, as a parameter or concatenated into the message. It
matches across lines, because the credential argument frequently sits on
a continuation line — that is true of the `FlightSqlConnectPoolMgr` case
above, which a line-based rule silently misses.
It is a backstop, not a substitute for review, and the honest limitation
is that it only knows the naming convention: `peerIdentity` had to be
taught to it by hand once it turned out to be a token, and any other
alias would be equally invisible. It reports **no violation anywhere in
`fe/`** after this PR, so it lands without a single suppression.
### Release note
Arrow Flight SQL bearer tokens are no longer written to `fe.log`. Log
lines and error messages now carry a non-reversible token id (`sha256:`
prefix) instead of the token itself.
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
`TokenMaskerTest` covers that the token id is a digest and cannot
contain any part of the token, that it is stable for the same token and
differs across tokens, the empty/null handling, and `maskPrefix`
including its too-short-to-reveal branch.
The checkstyle rule was verified to actually fire, not just to be quiet:
re-adding the original `LOG.info(..., username, token)` line, and
separately un-masking the multi-line `FlightSqlConnectPoolMgr` call,
each fail the build at that line with the new message; restoring them
goes back to green.
- Behavior changed:
- [ ] No.
- [x] Yes. <!-- Explain the behavior change -->
The text of some Arrow Flight error messages changes: where they used to
echo the bearer token, they now carry `token id: sha256:...`. Anything
that parsed the token out of an error message or out of `fe.log` would
need to use the id instead. No API, wire format or configuration
changes.
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
---
fe/check/checkstyle/checkstyle.xml | 19 ++++++
.../main/java/org/apache/doris/catalog/Env.java | 5 +-
.../org/apache/doris/common/util/TokenMasker.java | 79 ++++++++++++++++++++++
.../org/apache/doris/httpv2/meta/MetaService.java | 24 +------
.../org/apache/doris/mysql/privilege/Auth.java | 5 +-
.../sessions/FlightSessionsWithTokenManager.java | 6 +-
.../sessions/FlightSqlConnectPoolMgr.java | 5 +-
.../arrowflight/tokens/FlightTokenManagerImpl.java | 40 ++++++-----
.../apache/doris/common/util/TokenMaskerTest.java | 63 +++++++++++++++++
9 files changed, 202 insertions(+), 44 deletions(-)
diff --git a/fe/check/checkstyle/checkstyle.xml
b/fe/check/checkstyle/checkstyle.xml
index 2b87546548b..c97fbeed110 100644
--- a/fe/check/checkstyle/checkstyle.xml
+++ b/fe/check/checkstyle/checkstyle.xml
@@ -66,6 +66,25 @@ under the License.
<property name="message" value="Trailing whitespace found."/>
<property name="fileExtensions" value=".java"/>
</module>
+ <!--
+ Best effort guard against writing credentials to the log: catches a
value whose name says it
+ holds a token/password/secret/peer identity being passed straight to a
log call, either as a
+ parameter or concatenated into the message. Logs are routinely shipped
off the host, so a
+ credential in a log line has a far wider audience than the credential
store itself.
+
+ Matching is multiline because the argument often sits on a continuation
line, and [^;] bounds
+ each match to a single statement. It only recognizes the naming
convention, so it is a
+ backstop and not a substitute for review: an alias that does not carry
the noun (peerIdentity
+ needed to be added by hand once it turned out to be the Arrow Flight
bearer token) is invisible
+ to it.
+ -->
+ <module name="RegexpMultiline">
+ <property name="format"
+
value="LOG\.(?:info|warn|error|debug|trace)\([^;]*[,+]\s*(?![Mm]asked)(?:[A-Za-z0-9_]+\.)*(?:get)?[A-Za-z0-9_]*(?:[Tt]oken|[Pp]assword|[Ss]ecret|[Pp]eerIdentity)(?:Value|Str|String|Text)?(?:\(\))?\s*[,)+]"/>
+ <property name="message"
+ value="Do not log credentials. Mask the value with
org.apache.doris.common.util.TokenMasker (or log only a non-reversible id); if
the value is not a secret, name it so."/>
+ <property name="fileExtensions" value=".java"/>
+ </module>
<module name="TreeWalker">
<!-- filter -->
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index 2d0ccd486bc..50c1bfc2171 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -91,6 +91,7 @@ import org.apache.doris.common.util.PropertyAnalyzer;
import org.apache.doris.common.util.SmallFileMgr;
import org.apache.doris.common.util.SqlUtils;
import org.apache.doris.common.util.TimeUtils;
+import org.apache.doris.common.util.TokenMasker;
import org.apache.doris.common.util.Util;
import org.apache.doris.connector.ConnectorFactory;
import org.apache.doris.connector.ConnectorPluginManager;
@@ -1491,7 +1492,9 @@ public class Env {
}
String remoteToken =
conn.getHeaderField(MetaBaseAction.TOKEN);
if (token == null && remoteToken != null) {
- LOG.info("get token from helper node. token={}.",
remoteToken);
+ // Masked: the cluster token authenticates meta
access, so it must not
+ // reach fe.log. The prefix is enough to tell which
token was adopted.
+ LOG.info("get token from helper node. token={}.",
TokenMasker.maskPrefix(remoteToken));
token = remoteToken;
storage.writeClusterIdAndToken();
storage.reload();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/TokenMasker.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/TokenMasker.java
new file mode 100644
index 00000000000..cd6f47503c1
--- /dev/null
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/TokenMasker.java
@@ -0,0 +1,79 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.util;
+
+import com.google.common.base.Strings;
+import com.google.common.hash.Hashing;
+
+import java.nio.charset.StandardCharsets;
+
+/**
+ * Helpers to render a secret (bearer token, auth token, ...) in log lines and
error messages
+ * without writing the secret itself, since logs are routinely shipped to
places that are far
+ * less protected than the credential store.
+ *
+ * <p>Two different renderings are offered, pick by what the reader of the
message needs:
+ * {@link #tokenId} when the reader only needs to correlate messages about the
same secret, and
+ * {@link #maskPrefix} when a human needs to recognize <i>which</i> configured
secret was used.
+ */
+public class TokenMasker {
+ public static final String EMPTY_TOKEN = "<empty>";
+
+ // A truncated digest: long enough that two live tokens are very unlikely
to collide,
+ // short enough that it is useless as a credential.
+ private static final int TOKEN_ID_LEN = 8;
+ private static final String TOKEN_ID_PREFIX = "sha256:";
+
+ // Minimum token length required before we reveal a masked prefix. Shorter
tokens would
+ // leak too large a fraction of the secret, so they are hidden entirely
with only a length hint.
+ private static final int MIN_TOKEN_LEN_FOR_PREFIX = 8;
+ private static final int TOKEN_PREFIX_LEN = 3;
+
+ private TokenMasker() {
+ }
+
+ /**
+ * Returns a stable, non-reversible handle for a token, e.g. {@code
sha256:1a2b3c4d}. The same
+ * token always renders to the same handle, so log lines and the error
message handed back to
+ * the client can be matched up, while no part of the secret is disclosed.
+ */
+ public static String tokenId(String token) {
+ if (Strings.isNullOrEmpty(token)) {
+ return EMPTY_TOKEN;
+ }
+ return TOKEN_ID_PREFIX + Hashing.sha256().hashString(token,
StandardCharsets.UTF_8).toString()
+ .substring(0, TOKEN_ID_LEN);
+ }
+
+ /**
+ * Masks a token by revealing only a short leading prefix (e.g. {@code
abc***}), so that a
+ * token mismatch is diagnosable during rotation, while never logging the
full secret. Empty
+ * tokens and tokens too short to safely show a prefix are hidden. Prefer
{@link #tokenId}
+ * unless the reader really needs to recognize the secret by sight.
+ */
+ public static String maskPrefix(String token) {
+ if (Strings.isNullOrEmpty(token)) {
+ return EMPTY_TOKEN;
+ }
+ if (token.length() < MIN_TOKEN_LEN_FOR_PREFIX) {
+ // Too short to reveal any prefix without leaking a large fraction
of the secret.
+ return "<hidden, token length " + token.length() + " < " +
MIN_TOKEN_LEN_FOR_PREFIX + ">";
+ }
+ return token.substring(0, TOKEN_PREFIX_LEN) + "***";
+ }
+}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
b/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
index 4a8acdce33d..fe88e854d35 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
@@ -22,6 +22,7 @@ import org.apache.doris.common.Config;
import org.apache.doris.common.DdlException;
import org.apache.doris.common.util.HttpURLUtil;
import org.apache.doris.common.util.NetUtils;
+import org.apache.doris.common.util.TokenMasker;
import org.apache.doris.ha.FrontendNodeType;
import
org.apache.doris.httpv2.controller.BaseController.ActionAuthorizationInfo;
import org.apache.doris.httpv2.entity.ResponseEntityBuilder;
@@ -97,7 +98,7 @@ public class MetaService extends RestBaseController {
LOG.warn("reject meta request with invalid token. client: {},
{}, request from: {}, "
+ "expected: {}, actual: {}",
clientHost, clientPort, request.getRemoteAddr(),
- maskToken(clusterToken), maskToken(requestToken));
+ TokenMasker.maskPrefix(clusterToken),
TokenMasker.maskPrefix(requestToken));
throw unauthorized(clientHost, clientPort, request);
}
}
@@ -108,27 +109,6 @@ public class MetaService extends RestBaseController {
+ ", request from " + request.getRemoteAddr());
}
- // Minimum token length required before we reveal a masked prefix in logs.
Shorter tokens would
- // leak too large a fraction of the secret, so they are hidden entirely
with only a length hint.
- private static final int MIN_TOKEN_LEN_FOR_PREFIX = 8;
- private static final int TOKEN_PREFIX_LEN = 3;
-
- /**
- * Masks a token for logging: reveals only a short leading prefix (e.g.
"abc***") so that a
- * token mismatch is diagnosable during rotation, while never logging the
full secret. Empty
- * tokens and tokens too short to safely show a prefix are hidden.
- */
- private static String maskToken(String token) {
- if (Strings.isNullOrEmpty(token)) {
- return "<empty>";
- }
- if (token.length() < MIN_TOKEN_LEN_FOR_PREFIX) {
- // Too short to reveal any prefix without leaking a large fraction
of the secret.
- return "<hidden, token length " + token.length() + " < " +
MIN_TOKEN_LEN_FOR_PREFIX + ">";
- }
- return token.substring(0, TOKEN_PREFIX_LEN) + "***";
- }
-
@RequestMapping(path = "/image", method = RequestMethod.GET)
public Object image(HttpServletRequest request, HttpServletResponse
response) {
checkFromValidFe(request);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Auth.java
b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Auth.java
index 3d03be862fd..75fcced4f2a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Auth.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/Auth.java
@@ -1745,8 +1745,9 @@ public class Auth implements Writable {
scramble = MysqlPassword.checkPassword(initialRootPassword);
} catch (AnalysisException e) {
// Skip set root password if `initial_root_password` is not valid
2-staged SHA-1 encrypted
- LOG.warn("initial_root_password [{}] is not valid 2-staged SHA-1
encrypted, ignore it",
- initialRootPassword);
+ // Do not echo the configured value: this branch is reached
precisely when it is not a
+ // 2-staged SHA-1 hash, which usually means a plaintext password
was configured.
+ LOG.warn("initial_root_password is not valid 2-staged SHA-1
encrypted, ignore it");
return;
}
UserIdentity rootUser = new UserIdentity(ROOT_USER, "%");
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
index 8001998a66e..02e1389111d 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
@@ -18,6 +18,7 @@
package org.apache.doris.service.arrowflight.sessions;
import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.util.TokenMasker;
import org.apache.doris.common.util.Util;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.qe.ConnectScheduler;
@@ -59,8 +60,9 @@ public class FlightSessionsWithTokenManager implements
FlightSessionsManager {
final FlightTokenDetails flightTokenDetails =
flightTokenManager.validateToken(peerIdentity);
if (flightTokenDetails.getCreatedSession()) {
flightTokenManager.invalidateToken(peerIdentity);
- throw new IllegalArgumentException("UserSession expire after
access, try reconnect, bearer token: "
- + peerIdentity + ", a peerIdentity(bearer token) can only
create a ConnectContext once. "
+ throw new IllegalArgumentException("UserSession expire after
access, try reconnect, bearer token id: "
+ + TokenMasker.tokenId(peerIdentity)
+ + ", a peerIdentity(bearer token) can only create a
ConnectContext once. "
+ "if ConnectContext is deleted without operation for a
long time, it needs to be reconnected "
+ "(at the same time obtain a new bearer token).");
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSqlConnectPoolMgr.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSqlConnectPoolMgr.java
index 04982a7fedc..c8854507e00 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSqlConnectPoolMgr.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSqlConnectPoolMgr.java
@@ -17,6 +17,7 @@
package org.apache.doris.service.arrowflight.sessions;
+import org.apache.doris.common.util.TokenMasker;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.qe.ConnectContext.ConnectType;
import org.apache.doris.qe.ConnectPoolMgr;
@@ -67,8 +68,10 @@ public class FlightSqlConnectPoolMgr extends ConnectPoolMgr {
// RootAllocator.close() marks the allocator closed before it
reports outstanding
// bytes. The error is actionable, but session teardown must
still release the
// coordinator, transaction and pool/token bookkeeping below.
+ // For an Arrow Flight SQL connection the peer identity IS the
bearer token, so it is
+ // logged as a masked id, the same one FlightTokenManagerImpl
uses.
LOG.warn("failed to close Flight SQL channel while
unregistering connection {}, peer identity {}",
- ctx.getConnectionId(), ctx.getPeerIdentity(), t);
+ ctx.getConnectionId(),
TokenMasker.tokenId(ctx.getPeerIdentity()), t);
}
}
// Finalize any Arrow Flight query whose coordinator was kept alive
across the
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/tokens/FlightTokenManagerImpl.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/tokens/FlightTokenManagerImpl.java
index 85d1a0bce35..a3c76da9f05 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/tokens/FlightTokenManagerImpl.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/tokens/FlightTokenManagerImpl.java
@@ -21,6 +21,7 @@ package org.apache.doris.service.arrowflight.tokens;
import org.apache.doris.catalog.Env;
import org.apache.doris.common.CustomThreadFactory;
+import org.apache.doris.common.util.TokenMasker;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.service.ExecuteEnv;
import org.apache.doris.service.arrowflight.auth2.FlightAuthResult;
@@ -79,12 +80,13 @@ public class FlightTokenManagerImpl implements
FlightTokenManager {
if (context != null) {
ExecuteEnv.getInstance().getScheduler().getFlightSqlConnectPoolMgr()
.unregisterConnection(context);
- LOG.info("evict bearer token: " + token + " from
tokenCache, reason: "
- + notification.getCause()
+ LOG.info("evict bearer token: " +
TokenMasker.tokenId(token) + " from tokenCache, "
+ + "reason: " + notification.getCause()
+ ", and unregister flight connection
context after evict bearer token");
} else {
- LOG.info("evict bearer token: " + token + " from
tokenCache, reason: "
- + notification.getCause() + ", and flight
connection context not exist");
+ LOG.info("evict bearer token: " +
TokenMasker.tokenId(token) + " from tokenCache, "
+ + "reason: " + notification.getCause()
+ + ", and flight connection context not
exist");
}
usersTokenLRU.get(tokenDetails.getUsername()).invalidate(token);
}
@@ -128,7 +130,7 @@ public class FlightTokenManagerImpl implements
FlightTokenManager {
public void onRemoval(@NotNull
RemovalNotification<String, Integer> notification) {
// TODO: broadcast this message to other FE
assert notification.getKey() != null;
- LOG.info("evict bearer token: " +
notification.getKey()
+ LOG.info("evict bearer token: " +
TokenMasker.tokenId(notification.getKey())
+ " from usersTokenLRU, reason: "
+ notification.getCause());
tokenCache.invalidate(notification.getKey());
}
@@ -141,7 +143,9 @@ public class FlightTokenManagerImpl implements
FlightTokenManager {
}));
}
usersTokenLRU.get(username).put(token, 1);
- LOG.info("Created flight token for user: {}, token: {}", username,
token);
+ // Never log the token itself: fe.log is routinely shipped off the FE
host, and the token is
+ // accepted as a full credential until it expires. The id is enough to
trace its lifecycle.
+ LOG.info("Created flight token for user: {}, token id: {}", username,
TokenMasker.tokenId(token));
return flightTokenDetails;
}
@@ -149,25 +153,28 @@ public class FlightTokenManagerImpl implements
FlightTokenManager {
public FlightTokenDetails validateToken(final String token) throws
IllegalArgumentException {
final FlightTokenDetails value = getTokenDetails(token);
if (value.getToken().equals("")) {
- throw new IllegalArgumentException("invalid bearer token: " + token
+ throw new IllegalArgumentException("invalid bearer token, token
id: " + TokenMasker.tokenId(token)
+ ", try reconnect, bearer token may not be created, or
may have been evict, search for this "
- + "token in fe.log to see the evict reason. currently in
fe.conf, `arrow_flight_max_connections`="
- + this.cacheSize + ",
`arrow_flight_token_alive_time_second`=" + this.cacheExpiration);
+ + "token id in fe.log to see the evict reason. currently
in fe.conf, "
+ + "`arrow_flight_max_connections`=" + this.cacheSize
+ + ", `arrow_flight_token_alive_time_second`=" +
this.cacheExpiration);
}
if (System.currentTimeMillis() >= value.getExpiresAt()) {
tokenCache.invalidate(token);
- throw new IllegalArgumentException("bearer token expired: " +
token + ", try reconnect, "
- + "currently in fe.conf,
`arrow_flight_token_alive_time_second`=" + this.cacheExpiration);
+ throw new IllegalArgumentException("bearer token expired, token
id: " + TokenMasker.tokenId(token)
+ + ", try reconnect, currently in fe.conf,
`arrow_flight_token_alive_time_second`="
+ + this.cacheExpiration);
}
if (usersTokenLRU.containsKey(value.getUsername())) {
try {
usersTokenLRU.get(value.getUsername()).get(token);
} catch (ExecutionException ignored) {
- throw new IllegalArgumentException("usersTokenLRU not exist
bearer token: " + token);
+ throw new IllegalArgumentException(
+ "usersTokenLRU not exist bearer token, token id: " +
TokenMasker.tokenId(token));
}
} else {
- throw new IllegalArgumentException(
- "bearer token not created: " + token + ", username: " +
value.getUsername());
+ throw new IllegalArgumentException("bearer token not created,
token id: " + TokenMasker.tokenId(token)
+ + ", username: " + value.getUsername());
}
LOG.info("Validated bearer token for user: {}", value.getUsername());
return value;
@@ -175,7 +182,7 @@ public class FlightTokenManagerImpl implements
FlightTokenManager {
@Override
public void invalidateToken(final String token) {
- LOG.info("Invalidate bearer token, {}", token);
+ LOG.info("Invalidate bearer token, token id: {}",
TokenMasker.tokenId(token));
tokenCache.invalidate(token);
}
@@ -185,7 +192,8 @@ public class FlightTokenManagerImpl implements
FlightTokenManager {
try {
value = tokenCache.getUnchecked(token);
} catch (CacheLoader.InvalidCacheLoadException ignored) {
- throw new IllegalArgumentException("InvalidCacheLoadException,
invalid bearer token: " + token);
+ throw new IllegalArgumentException(
+ "InvalidCacheLoadException, invalid bearer token, token
id: " + TokenMasker.tokenId(token));
}
return value;
diff --git
a/fe/fe-core/src/test/java/org/apache/doris/common/util/TokenMaskerTest.java
b/fe/fe-core/src/test/java/org/apache/doris/common/util/TokenMaskerTest.java
new file mode 100644
index 00000000000..fbd4e924ea9
--- /dev/null
+++ b/fe/fe-core/src/test/java/org/apache/doris/common/util/TokenMaskerTest.java
@@ -0,0 +1,63 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.common.util;
+
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+public class TokenMaskerTest {
+
+ // Same length and alphabet as the tokens FlightTokenManagerImpl mints
(130 random bits in
+ // base 32), but deliberately low entropy and readable, so that no secret
scanner has to
+ // decide whether a random-looking 26 character literal in the tree is a
real credential.
+ private static final String TOKEN = "notarealtokennotarealtoken";
+
+ @Test
+ public void testTokenIdHidesTheSecret() {
+ String id = TokenMasker.tokenId(TOKEN);
+ Assertions.assertFalse(id.contains(TOKEN));
+ // The id is a truncated digest and nothing else, so no part of the
token can survive in it.
+ Assertions.assertTrue(id.matches("sha256:[0-9a-f]{8}"), "unexpected
token id: " + id);
+ }
+
+ @Test
+ public void testTokenIdIsStableAndDistinguishing() {
+ // Same token always renders identically, so a log line and the error
message returned to
+ // the client can be matched up.
+ Assertions.assertEquals(TokenMasker.tokenId(TOKEN),
TokenMasker.tokenId(TOKEN));
+ Assertions.assertNotEquals(TokenMasker.tokenId(TOKEN),
TokenMasker.tokenId(TOKEN + "x"));
+ Assertions.assertEquals("sha256:",
TokenMasker.tokenId(TOKEN).substring(0, 7));
+ Assertions.assertEquals(15, TokenMasker.tokenId(TOKEN).length());
+ }
+
+ @Test
+ public void testTokenIdOfEmptyToken() {
+ Assertions.assertEquals(TokenMasker.EMPTY_TOKEN,
TokenMasker.tokenId(null));
+ Assertions.assertEquals(TokenMasker.EMPTY_TOKEN,
TokenMasker.tokenId(""));
+ }
+
+ @Test
+ public void testMaskPrefix() {
+ Assertions.assertEquals("not***", TokenMasker.maskPrefix(TOKEN));
+ Assertions.assertEquals(TokenMasker.EMPTY_TOKEN,
TokenMasker.maskPrefix(null));
+ Assertions.assertEquals(TokenMasker.EMPTY_TOKEN,
TokenMasker.maskPrefix(""));
+ // Too short to show a prefix: hidden entirely, only the length is
reported.
+ Assertions.assertEquals("<hidden, token length 7 < 8>",
TokenMasker.maskPrefix("1234567"));
+ Assertions.assertEquals("123***", TokenMasker.maskPrefix("12345678"));
+ }
+}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
