[
https://issues.apache.org/jira/browse/HTTPCLIENT-2430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18106702#comment-18106702
]
Jonny Carter commented on HTTPCLIENT-2430:
------------------------------------------
Something worth a note here: I did see some other errors in my environment that
seem to be down to {{Brotli4jRuntime.available()}} using class presence rather
than {{Brotli4jLoader#isAvailable}} ([method
source|https://github.com/hyperxpro/Brotli4j/blob/main/brotli4j/src/main/java/com/aayushatharva/brotli4j/Brotli4jLoader.java#L98]).
I can work around this by calling
{{{}Brotli4jLoader.ensureAvailability(){}}}in my own environment, but
HttpClient may want to change to the more robust Brotli4jLoader which will
check for successful linking of the native libraries. I'll grant there may be
context there as to why they aren't, just noting it for the record.
> Brotli decompression fails for large payloads
> ---------------------------------------------
>
> Key: HTTPCLIENT-2430
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2430
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient (async)
> Affects Versions: 5.6.1
> Reporter: Jonny Carter
> Assignee: Arturo Bernal
> Priority: Major
> Fix For: 5.6.5, 5.7-alpha2
>
>
> h2. Symptom
> With transparent async content decompression (new in 5.6) and brotli4j on the
> classpath, any response with {{Content-Encoding: br}} whose *compressed* body
> is larger than 8 KB never completes.
> {{InflatingBrotliDataConsumer.consume()}} stops making progress once the
> decoder's fixed 8 KB input buffer is full: the {{xfer == 0}} branch calls
> {{decoder.push(0)}} / {{pump()}} in a loop that can neither advance nor fail,
> so {{while (src.hasRemaining())}} never exits.
> In a real async client the consequences are severe and silent: the exchange
> never completes, no response timeout fires, and the pooled connection is
> never released. Under load the pool drains connection by connection until
> every request to that route hangs. There is no exception and no log line
> anywhere.
> We found this on 5.6.4; the code is unchanged on the 5.6.x branch head.
> HTTPCLIENT-2428 (fixed for 5.7-alpha1) touches the same class but only
> replaces the capacity plumbing; the {{consume()}} loop is untouched, so
> 5.7-alpha1 appears to be affected as well.
> A second, smaller defect in the same class: a single spurious byte after a
> complete, valid brotli stream fails the exchange with {{{}IOException: Brotli
> stream corrupted{}}}, although the content is fully recoverable and the
> trailer could be ignored.
> h2. How to reproduce
> Failing unit tests against {{InflatingBrotliDataConsumerTest}} are attached
> and available on a GitHub branch in my fork
> [https://github.com/jonnybot0/httpcomponents-client/tree/bugfix-brotli]
> (commit 402e689774fe76ae9385e9b2e7b603a3ac11f5bc).
> * {{inflateBrotliLargerThanInputBufferSingleBuffer}} – well-formed brotli
> stream, ~64 KB of incompressible (random) payload so the compressed form
> exceeds 8 KB, delivered as one buffer. Fails with {{{}execution timed out
> after 5000 ms{}}}; the timeout stack pins the busy-loop at
> {{InflatingBrotliDataConsumer.consume}} ({{{}decoder.push{}}} in the {{xfer
> == 0}} branch).
> * {{inflateBrotliLargerThanInputBufferChunked}} – the same stream in 1 KB
> chunks, i.e. identical in shape to the existing passing test; the only
> difference is the compressed size. Same 5 s timeout.
> * {{inflateBrotliWithTrailingByte}} – valid small stream plus one trailing
> byte; fails with {{{}IOException: Brotli stream corrupted{}}}.
> Note the existing happy-path test passes only because its highly compressible
> payload stays well under the 8 KB buffer.
> Core of the repro (the harness drives the consumer directly, no network):
> {code:java}
> final byte[] original = new byte[64 * 1024];
> new java.util.Random(42).nextBytes(original); // incompressible =>
> compressed > 8 KB
> final byte[] compressed = Encoder.compress(original,
> new Encoder.Parameters().setQuality(6).setWindow(22));
> final InflatingBrotliDataConsumer inflating = new
> InflatingBrotliDataConsumer(rawByteCollector);
> inflating.consume(ByteBuffer.wrap(compressed)); // never returns
> inflating.streamEnd(Collections.emptyList());
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
