Hi Oleg,
Thank you for the input and information provided. There is another
observation we are seeing related to POST data.
When we have small POST data everything works fine perfectly, but when we
have huge POST data (for example 64 KB), it seems OutPutstream (internally
using ContentOutputStream a wrapper over SharedOutputBuffer of initial size
1024) is not able to write data (We are using temporary buffer block of 4kb
but still it is not able to write 4KB of data). Is there any limitation or
some configuration problem?
*Code being used*
SharedOutputBuffer buffer = new SharedOutputBuffer(1024);
produceData(contentType, new ContentOutputStream(buffer));
protected void produceData(ContentType contentType, OutputStream
outputStream) throws IOException {
byte[] tmp = new byte[4 * 1024];
int l = -1;
while ((l = inputStream.read(tmp)) != -1) {
* outputStream.write(tmp, 0, l);*
}
}
*HTTPClient has last line as below*
2023-03-02 21:53:52,710 DEBUG
[org.apache.hc.client5.http.impl.async.HttpAsyncMainClientExec]
ex-0000000004: produce request data
Another observation is even after connecting to a server, the request is
not getting timed out even after 30 minutes as well in this scenario (i.e.,
Large POST Data). DefaultRequestConfig ResponseTimeout is being set to 5
seconds and even DefaultConnectionConfig ConnectionTimeout and
SocketTimeout also being set to 180000 milliseconds (ie., 3 minutes) and
also same value is being set to IOReactorConfig SoTimeout as well. Is
there any other setting which is missing which is causing this behavior?
*HTTPClient log extract*
2023-03-02 21:31:51,187 DEBUG
[org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager]
ep-0000000001 connecting endpoint to https://server.us.com (180000
MILLISECONDS)
2023-03-02 21:31:55,964 DEBUG
[org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager]
ep-0000000001 connection c-0000000000 can be kept alive for 3 MINUTES
Regards,
Sreenivas
On Fri, Feb 24, 2023 at 6:06 PM Oleg Kalnichevski <ol...@apache.org> wrote:
> On Thu, 2023-02-23 at 17:38 +0530, sreenivas somavarapu wrote:
> > Hi Oleg,
> >
> > Agreed. Is there a way to achieve that instead of adding method
> > condition
> > check explicitly (In this case sample code is not initiator of
> > request,
> > instead that sample code is acting as proxy)? Also is there a way to
> > suppress that ContentLength header with 0 value generation alone by
> > httpclient?
> >
>
> The problem is not HttpClient APIs but rather Servlet APIs which
> provide no reliable way of distinguishing requests with without a
> request entity from requests with zero length request body.
>
> Oleg
>
>
>
> > Earlier in 4.x we had HttpEntityEnclosingRequestBase based on which
> > we
> > could determine what method accepts body and what method doesn't, but
> > with
> > 5.x (Both Synchronous and Asynchronous) I am unable to find any way.
> >
> > Regards,
> > Sreenivas
> >
> > On Thu, Feb 23, 2023 at 2:31 AM Oleg Kalnichevski <ol...@apache.org>
> > wrote:
> >
> > > On Thu, 2023-02-23 at 02:11 +0530, sreenivas somavarapu wrote:
> > > > Hi Oleg,
> > > >
> > > > Real problem was in our customized AbstractClassicEntityProducer
> > > > class produce method, buffer flush was missing which was causing
> > > > the
> > > > issue (*Connection
> > > > Closed* in case of GET requests and *Connection forcibly closed*
> > > > in
> > > > case of
> > > > POST requests). If we don't add a request method check for
> > > > skipping
> > > > adding
> > > > of entity to request, httpclient produces ContentLength header
> > > > with 0
> > > > (In
> > > > *RequestContent.java* process method there is a check for entity
> > > > and
> > > > if
> > > > entity present it adds ContentLength header by default).
> > > > Currently I
> > > > see
> > > > the only way to avoid not adding ContentLength header is to not
> > > > set
> > > > an
> > > > entity to request in specific conditions (i.e., GET requests
> > > > etc.,).
> > > > Is
> > > > there any other way we can make httpclient skip adding
> > > > ContentLength
> > > > header
> > > > with 0?
> > > >
> > > > Here is sample code with below modifications to actual code being
> > > > used
> > > > - ContentType, ProtocolVersion and inputstream (i.e., Content) we
> > > > get
> > > > as
> > > > input from servlet
> > > > - AbstractClassicEntityProducer has been customized to generate
> > > > content
> > > > length as well and is not using any executor instead produceData
> > > > API
> > > > is
> > > > predefined in that custom class
> > > >
> > > > *Code*
> > > > HttpHost host = new HttpHost("host", "/printenv.pl", 80);
> > > > InputStream content = new BufferedInputStream(<Stream
> > > > from
> > > > servlet>);
> > > > AsyncRequestProducer request =
> > > > AsyncRequestBuilder.get().setHttpHost(host).setEntity(
> > > > new AbstractClassicEntityProducer(1024,
> > > > ContentType.TEXT_XML, Executors.newSingleThreadExecutor()) {
> > > > @Override
> > > > protected void produceData(ContentType ct,
> > > > OutputStream
> > > > os) throws IOException {
> > > > if (content != null) {
> > > > byte[] b = new byte[1024];
> > > > int bytesRead = 0;
> > > > while (content.read(b) != -1) {
> > > > os.write(b, bytesRead, b.length);
> > > > bytesRead += b.length;
> > > > }
> > > > }
> > > > }
> > > > }).build();
> > > >
> > >
> > > This is where your code is wrong: if the request is not supposed to
> > > have an entity it should not have a entity producer, _at all_.
> > >
> > > Oleg
> > >
> > >
> > > -------------------------------------------------------------------
> > > --
> > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> > > For additional commands, e-mail:
> > > httpclient-users-h...@hc.apache.org
> > >
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>
>
--
Cheers,
S. Sreenivas
