[
https://issues.apache.org/jira/browse/CODEC-348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Mclaughlin updated CODEC-348:
---------------------------------
Attachment: commons-codec-1.23-strict-repro.zip
> Base64 STRICT decoding in 1.23.0 rejects valid input when the codec has a
> line length (breaking change from 1.22.1)
> -------------------------------------------------------------------------------------------------------------------
>
> Key: CODEC-348
> URL: https://issues.apache.org/jira/browse/CODEC-348
> Project: Commons Codec
> Issue Type: Bug
> Affects Versions: 1.23.0
> Environment: OpenJDK 21.0.12
> Reporter: Dan Mclaughlin
> Priority: Major
> Attachments: commons-codec-1.23-strict-repro.zip
>
>
> Component: {{org.apache.commons.codec.binary.Base64}} / {{BaseNCodec}}
> Works in: 1.22.1 and earlier (STRICT since 1.15)
> h2. Summary
> Commit {{8b506a1c}} ("Base32 and Base64 STRICT decoding now require the
> encoder's canonical alphabet, padding, and line separators, and validate
> streams through EOF") changed what {{CodecPolicy.STRICT}} means for decoding.
> A STRICT {{Base64}} instance constructed with a line length now accepts only
> byte-for-byte what that instance would encode, including a trailing line
> separator. Previously valid input that decoded correctly in 1.22.1 now throws
> {{IllegalArgumentException}}.
> This is a behavior change in a minor release that breaks existing callers who
> use one STRICT instance for both encoding and decoding. Shibboleth's
> {{Base64Support}} (used by the Shibboleth IdP, SP integrations and OpenSAML
> for all SAML/XML-DSig base64) is one such caller: with 1.23.0 it can no
> longer decode its own encoded output. This is tracked downstream as
> Shibboleth [JSSH-84|https://shibboleth.atlassian.net/browse/JSSH-84]
> (priority Highest), where it breaks SAML message, signature and certificate
> decoding; in our deployment it broke SAML metadata signature validation.
> Because the IdP does not control which commons-codec version deployers'
> builds resolve, every Shibboleth deployment whose dependency management moves
> to 1.23.0 is affected.
> h2. Steps to reproduce
> Attached {{CodecOnlyRepro.java}} (commons-codec only):
> {code:java}
> Base64 strict76 = new Base64(76, new byte[] {'\n'}, false,
> CodecPolicy.STRICT);
> String exact = strict76.encodeToString(data);
> strict76.decode(exact); // exact encoder output
> strict76.decode(exact.trim()); // same, trailing '\n'
> removed
> strict76.decode(singleLineEncodingOf(data)); // same bytes, no line breaks
> strict76.decode(exact.replace("\n", "\r\n")); // same bytes, CRLF
> separators
> {code}
> h2. Expected
> All four decode to the original bytes, as in 1.22.1. Each input is valid
> base64 for the same data, with canonical alphabet, padding and trailing bits.
> h2. Actual (1.23.0)
> ||Input||20 bytes||256 bytes||
> |Exact encoder output|OK|OK|
> |Encoder output, trailing {{\n}} trimmed|Missing or incomplete final line
> separator|Missing or incomplete final line separator|
> |Single line, no separators|Missing or incomplete final line
> separator|Invalid line separator or line length|
> |76-char lines, CRLF|Unexpected character or data after padding|Invalid line
> separator or line length|
> In 1.22.1 every case returns the correct bytes.
> h2. Why this is a compatibility problem
> * Through 1.22.1, {{BaseNCodec.isStrictDecoding()}} documented STRICT
> narrowly: decoding raises {{IllegalArgumentException}} "if trailing bits are
> not part of a valid encoding." Callers chose STRICT for that guarantee, not
> for line-structure validation.
> * {{CodecPolicy.STRICT}}'s Javadoc is unchanged in 1.23.0 ("Data that causes
> a codec to fail should throw an exception"), so nothing at the API level
> signals the broader meaning.
> * The changelog entry lists it under fixed bugs, and the release notes
> describe 1.23.0 as a "feature and maintenance release," so downstream
> consumers and automated update tools treat it as a safe minor bump.
> * Requiring a trailing separator even for input shorter than one line is
> stricter than RFC 2045 and RFC 4648 practice, and rejects the common pattern
> of trimming encoded output.
> * In CODEC-346 (resolved 2026-10-03), a request to make default decoding
> stricter was declined because "changing the default would introduce another
> compatibility break." Changing what STRICT itself accepts is the same kind of
> break for callers who already opted into STRICT.
> h2. Requests (any one would resolve it)
> # Make line-structure validation opt-in (for example a builder option such as
> {{setStrictLineValidation(true)}}), keeping 1.22.x STRICT semantics as the
> default.
> # Apply line-structure validation only when explicitly requested, and
> continue to accept input without separators (or with a missing final
> separator) for instances that have a line length.
> # If the new behavior stays, document it as a breaking change in the release
> notes and in the {{CodecPolicy.STRICT}} and {{isStrictDecoding()}} Javadoc,
> and state that an instance with a line length should not be used to decode
> arbitrary input.
> h2. Workaround for affected users
> Decode with an instance that has no line length, after removing whitespace:
> {code:java}
> new Base64(0, new byte[] {'\n'}, false,
> CodecPolicy.STRICT).decode(input.replaceAll("\\s+", ""));
> {code}
> This decodes all the inputs above in both 1.22.1 and 1.23.0, and still
> rejects non-canonical trailing bits.
> h2. Attachments
> {{commons-codec-1.23-strict-repro}} project: {{CodecOnlyRepro.java}},
> {{run.sh}}, {{run-output.txt}} (output for 1.22.1 and 1.23.0). Also includes
> the Shibboleth {{Base64Support}} and OpenSAML reproductions showing
> downstream impact.
> Related: Shibboleth [JSSH-84|https://shibboleth.atlassian.net/browse/JSSH-84]
> (downstream impact), CODEC-346 (compatibility rationale), commit {{8b506a1c}}
> (the change).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
