httpasyncclient BasicAsyncRequestProducer prevents HttpRequestInterceptor from replacing HttpEntity of the request

[Sven Zethelius]

I am trying to use a HttpRequestInterceptor to adapt a request sent to a 
org.apache.http.impl.nio.client.CloseableHttpAsyncClient.  I want to do request 
body compression (e.g. Content-Encoding: gzip for HttpRequest, not 
Accept-Encoding) following the example here: 
https://hc.apache.org/httpcomponents-client-4.2.x/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java
 .  The example is for response compression, but it looked like I could do the 
same thing to do request compression, and I need to handle response compression 
too.

After much debugging I realized the HttpRequestInterceptor would allow me to 
change the headers, but not the HttpEntity that was sent.  The entity sent is 
always the original one passed into the AsyncHttpClient.  On the server, I see 
the "Content-Type: gzip" and a "Content-Length" header for the gzipped length 
(Content-Length was set by the internal interceptors), but the bytes sent are 
the original values from the NByteArrayEntity, which leads to exceptions since 
Content-Length doesn't match actual length after GZIP.

I tracked it down to the constructor for 
org.apache.http.nio.protocol.BasicAsyncRequestProducer.  The 
BasicAsyncRequestProducer is created before the interceptors run.  In the 
producer constructor, the entity from the request is stored as the producer.  
This occurs before the HttpRequestInterceptor has a chance to manipulate the 
entity.

My idea for a patch would be to delay load the producer field until after the 
interceptors have run.  Looking at the class, that would have to be in 
produceContent, before it checks for producer==null. Since the request is also 
a member, this would allow the producer to be set based on what's in the 
request when we actually produce the output, rather than what was in it when 
the whole processing chain started.

Is patching the BasicAsyncRequestProducer the only way to do this or is there a 
simpler way that I am missing?

Re: httpasyncclient BasicAsyncRequestProducer prevents HttpRequestInterceptor from replacing HttpEntity of the request

[Oleg Kalnichevski]

On Wed, 2016-04-20 at 15:03 +, Sven Zethelius wrote:
> I am trying to use a HttpRequestInterceptor to adapt a request sent to a 
> org.apache.http.impl.nio.client.CloseableHttpAsyncClient.  I want to do 
> request body compression (e.g. Content-Encoding: gzip for HttpRequest, not 
> Accept-Encoding) following the example here: 
> https://hc.apache.org/httpcomponents-client-4.2.x/httpclient/examples/org/apache/http/examples/client/ClientGZipContentCompression.java
>  .  The example is for response compression, but it looked like I could do 
> the same thing to do request compression, and I need to handle response 
> compression too.
> 
> After much debugging I realized the HttpRequestInterceptor would allow me to 
> change the headers, but not the HttpEntity that was sent.  The entity sent is 
> always the original one passed into the AsyncHttpClient.  On the server, I 
> see the "Content-Type: gzip" and a "Content-Length" header for the gzipped 
> length (Content-Length was set by the internal interceptors), but the bytes 
> sent are the original values from the NByteArrayEntity, which leads to 
> exceptions since Content-Length doesn't match actual length after GZIP.
> 
> I tracked it down to the constructor for 
> org.apache.http.nio.protocol.BasicAsyncRequestProducer.  The 
> BasicAsyncRequestProducer is created before the interceptors run.  In the 
> producer constructor, the entity from the request is stored as the producer.  
> This occurs before the HttpRequestInterceptor has a chance to manipulate the 
> entity.
> 
> My idea for a patch would be to delay load the producer field until after the 
> interceptors have run.  Looking at the class, that would have to be in 
> produceContent, before it checks for producer==null. Since the request is 
> also a member, this would allow the producer to be set based on what's in the 
> request when we actually produce the output, rather than what was in it when 
> the whole processing chain started.
> 
> Is patching the BasicAsyncRequestProducer the only way to do this or is there 
> a simpler way that I am missing?
> 
> 

Hi Sven

Building a custom request producer is currently the only way to
implement content compression. 

HttpAsyncClient presently does not support transparent content
compression. This feature is tentatively planned for 5.0.

Oleg  



-
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org