[
https://issues.apache.org/jira/browse/HTTPCLIENT-2420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Barry updated HTTPCLIENT-2420:
--------------------------------------
Description:
This was something I asked about a long time ago, and it did not exist. So I
ended up creating it and it has worked well for us. I see continued interest in
the forums for something like this, so I created a sample/test project and
threw it up in github in hopes is could be adopted in the official client.
*Apache HttpClient 5 does not currently provide built-in mechanisms to:*
# *Abort a response mid-stream* if the body exceeds a configured byte limit
# *Abort a response mid-stream* if the server is delivering bytes too slowly
(slow-drip attack)
# *Enforce a size limit during decompression* to prevent decompression bomb
attacks — where a small compressed payload expands to an enormous decompressed
body
The 5.6 release added transparent async decompression, which is a welcome
addition. However, it does not support a max response size or other protections.
h3. Why is a 2 stage check needed on compressed responses?
* *Stage 1 alone* is insufficient: a 500 KB gzip payload passes any
reasonable wire-size limit, but could decompress to 500 MB.
* *Stage 2 alone* is insufficient: without Stage 1, the full compressed body
must be buffered before decompression begins, wasting memory on oversized raw
responses.
Here is what I propose:
[https://github.com/patrickjamesbarry/apache-client-decompression]
was:
This was something I asked about a long time ago, and it did not exist. So I
ended up creating it and it has worked well for us. I see continued interest in
the forums for something like this, so I created a sample/test project and
threw it up in github in hopes is could be adopted in the official client.
*Apache HttpClient 5 does not currently provide built-in mechanisms to:*
# *Abort a response mid-stream* if the body exceeds a configured byte limit
# *Abort a response mid-stream* if the server is delivering bytes too slowly
(slow-drip attack)
# *Enforce a size limit during decompression* to prevent decompression bomb
attacks — where a small compressed payload expands to an enormous decompressed
body
The 5.6 release added transparent async decompression, which is a welcome
addition. However, because decompression now happens inside the library before
bytes reach the caller's entity consumer, it is no longer possible to inject a
size check mid-decompression without owning the decompression step entirely.
h3. Why both stages are needed
* *Stage 1 alone* is insufficient: a 500 KB gzip payload passes any
reasonable wire-size limit, but could decompress to 500 MB.
* *Stage 2 alone* is insufficient: without Stage 1, the full compressed body
must be buffered before decompression begins, wasting memory on oversized raw
responses.
Here is what I propose:
[https://github.com/patrickjamesbarry/apache-client-decompression]
> Support max response size and handle slow responders
> ----------------------------------------------------
>
> Key: HTTPCLIENT-2420
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2420
> Project: HttpComponents HttpClient
> Issue Type: New Feature
> Reporter: Patrick Barry
> Priority: Major
>
> This was something I asked about a long time ago, and it did not exist. So I
> ended up creating it and it has worked well for us. I see continued interest
> in the forums for something like this, so I created a sample/test project and
> threw it up in github in hopes is could be adopted in the official client.
> *Apache HttpClient 5 does not currently provide built-in mechanisms to:*
> # *Abort a response mid-stream* if the body exceeds a configured byte limit
> # *Abort a response mid-stream* if the server is delivering bytes too slowly
> (slow-drip attack)
> # *Enforce a size limit during decompression* to prevent decompression bomb
> attacks — where a small compressed payload expands to an enormous
> decompressed body
> The 5.6 release added transparent async decompression, which is a welcome
> addition. However, it does not support a max response size or other
> protections.
> h3. Why is a 2 stage check needed on compressed responses?
> * *Stage 1 alone* is insufficient: a 500 KB gzip payload passes any
> reasonable wire-size limit, but could decompress to 500 MB.
> * *Stage 2 alone* is insufficient: without Stage 1, the full compressed body
> must be buffered before decompression begins, wasting memory on oversized raw
> responses.
> Here is what I propose:
> [https://github.com/patrickjamesbarry/apache-client-decompression]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
