Harrison Sheinblatt created KNOX-3515:
-----------------------------------------
Summary: Add trusted oidc issuer cache TTL/eventual consistency
for HA configurations
Key: KNOX-3515
URL: https://issues.apache.org/jira/browse/KNOX-3515
Project: Apache Knox
Issue Type: Sub-task
Components: JWT
Reporter: Harrison Sheinblatt
Knox nodes in an HA configuration will not all see changes made on other nodes
within a known time limit, and some will never be updated.
Currently there are a few cache layers used with Trusted OIDC Issuers:
# Registry snapshot in JdbcTrustedOidcIssuerService
# Discovery document cache in OIDCDiscoveryHelper
# JWKS source cache in DefaultTokenAuthorityService (this is preexisting)
The registry snapshot never expires and is only mutated by the local admin API
calls. See reloadRegistrySnapshot. The discovery document cache has a TTL, but
won't be invalidated when the table data is changed.
To enable HA configurations to work properly, the registry snapshot needs to be
invalidated within a configurable time limit, ideally by a background/scheduled
thread with jitter. This will be the same as the revocation timeout and the
time before a newly registered issuer can be safely used for token exchange.
To make the full cache stack work, when the registry snapshot cache is
invalidated, the discovery cache must also be invalidated:
discoveryHelper.invalidate(issuerUrl).
There are other options to allow HA configurations to work, but they have
drawbacks, including but not limited to:
# Removing the registry cache exposes a DOS surface for invalid issuers
## Could use a negative cache with rate limiting to mitigate the DOS threat
# One could optimize for reducing add issuer staleness: on issuer not found,
have a faster interval to read through to the database. Likely would still need
rate limiting.
# Operator could call each node in the HA set. This is prone to error and may
not be possible if the operator is not colocated in the cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
