On Mon, 2023-03-06 at 21:03 +0530, sreenivas somavarapu wrote:
> Hi Oleg,
>
> My sample program (it's a kind of unit test for SharedOutputBuffer to
> demonstrate that it is not able to write single byte if input byte
> array
> size is greater than 1023) doesn't have flush as my intention is to
> show
> the issue with SharedOutputBuffer alone and it is not a complete
> program
> which sends the data to the actual server and get response back. Main
> issue
> lies while writing data greater than 1023 bytes at first go itself
> to SharedOutputBuffer from the byte array.
>
> I think the below conditions in SharedOutputBuffer is causing this
> behavior
>
> while (src.hasRemaining()) {
> // always buffer small chunks
> *if (src.remaining() < 1024 && buffer().remaining() >
> src.remaining()) {*
> buffer().put(src);
> } else {
> * if (buffer().position() > 0 || dataStreamChannel
> ==
> null) {*
> waitFlush();
> }
>
What behavior? This code blocks if it is unable to store input data
until the internal buffer gets flushed and space in it frees up, which
is what is it supposed to do.
If you think SharedOutputBuffer is wrong, please just use your own
custom buffer implementation.
Oleg
> Regards,
> Sreenivas
>
> On Mon, Mar 6, 2023 at 8:52 PM Oleg Kalnichevski <ol...@apache.org>
> wrote:
>
> > On Mon, 2023-03-06 at 20:20 +0530, sreenivas somavarapu wrote:
> > > Hi Oleg,
> > >
> > > It looks like some issue with SharedOutputBuffer implementation
> > > and
> > > it is
> > > also not expandable as we have to allocate complete size during
> > > initialization itself. If I change below two lines from my
> > > program it
> > > proceeds further and completes the request (Basically
> > > SharedOutputBuffer is
> > > not able to write 1024 or above bytes of data at single go (If we
> > > declare
> > > temp block size as 1023 also it works)). It looks like there is
> > > an
> > > explicit 1024 length check in SharedOutputBuffer which is causing
> > > this
> > > behavior.
> > >
> > > SharedOutputBuffer buffer = new
> > > SharedOutputBuffer(req.contentLength() +
> > > 1024);
> >
> > What you are doing is _completely_ wrong. SharedOutputBuffer is not
> > meant to be expandable.
> >
> > You cannot just write into the buffer and expect it to work by
> > magic.
> > You also must flush the buffer.
> >
> > Oleg
> >
> > >
> > > byte[] tmp = new byte[512];
> > >
> > > Earlier those 2 lines were as below
> > >
> > > SharedOutputBuffer buffer = new SharedOutputBuffer(1024);
> > >
> > > byte[] tmp = new byte[4 * 1024];
> > >
> > > Below is my sample program on SharedOutputBuffer which
> > > demonstrated
> > > this
> > > behavior
> > >
> > > final StringBuilder sbuffer = new StringBuilder("123");
> > > System.out.println("O/P: " + sbuffer.toString());
> > > for (long i = 0; i < (64 * 1000); ++i) {
> > > sbuffer.append('A');
> > > }
> > >
> > > String postData = sbuffer.toString();
> > > System.out.println("O/P1: " + postData);
> > >
> > > byte[] inputData =
> > > postData.getBytes(StandardCharsets.UTF_8);
> > > ByteArrayInputStream inputStream = new
> > > ByteArrayInputStream(inputData);
> > > SharedOutputBuffer outputStream = new
> > > SharedOutputBuffer(100
> > > *
> > > 1024); *<-- If we keep this as 4 * 1024 also it hangs.*
> > > *byte[] tmp = new byte[1024]; <-- If we change this to
> > > 1023
> > > then
> > > it works*
> > > int l = -1;
> > > while ((l = inputStream.read(tmp)) != -1) {
> > > System.out.println("Writing Data");
> > > System.out.println(l);
> > > outputStream.write(tmp, 0, l);
> > > System.out.println("Wrote Data");
> > > }
> > >
> > > System.out.println("Done Writing Data");
> > > outputStream.writeCompleted();
> > > System.out.println(outputStream.toString());
> > >
> > > Regards,
> > > Sreenivas
> > >
> > > On Mon, Mar 6, 2023 at 7:17 PM Oleg Kalnichevski
> > > <ol...@apache.org>
> > > wrote:
> > >
> > > > On Mon, 2023-03-06 at 16:26 +0530, sreenivas somavarapu wrote:
> > > > > Hi Team,
> > > > >
> > > > > We are using AbstractClassicEntityProducer with a
> > > > > customization
> > > > > of
> > > > > storing
> > > > > / computing contentLength as well which works fine if we have
> > > > > small
> > > > > POST
> > > > > data, but when POST data is big (for example 64 KB), it seems
> > > > > OutPutstream
> > > > > (internally using ContentOutputStream a wrapper over
> > > > > SharedOutputBuffer of
> > > > > initial size 1024) is not able to write data (We are using
> > > > > temporary
> > > > > buffer
> > > > > block of 4kb but still it is not able to write 4KB of data).
> > > > > Is
> > > > > there
> > > > > any
> > > > > limitation or some configuration problem?
> > > > >
> > > > > *Code being used*
> > > > > SharedOutputBuffer buffer = new
> > > > > SharedOutputBuffer(1024);
> > > > >
> > > > > produceData(contentType, new
> > > > > ContentOutputStream(buffer));
> > > > >
> > > > > protected void produceData(ContentType contentType,
> > > > > OutputStream
> > > > > outputStream) throws IOException {
> > > > > byte[] tmp = new byte[4 * 1024];
> > > > > int l = -1;
> > > > > while ((l = inputStream.read(tmp)) != -1) {
> > > > > * outputStream.write(tmp, 0, l); <---- This is
> > > > > where
> > > > > request
> > > > > hangs / waits*
> > > >
> > > > This is likely due to a bug in your code that prevents the
> > > > shared
> > > > buffer from being flushed.
> > > >
> > > > Oleg
> > > >
> > > >
> > > > ---------------------------------------------------------------
> > > > ----
> > > > --
> > > > To unsubscribe, e-mail:
> > > > httpclient-users-unsubscr...@hc.apache.org
> > > > For additional commands, e-mail:
> > > > httpclient-users-h...@hc.apache.org
> > > >
> > > >
> > >
> >
> >
> > -------------------------------------------------------------------
> > --
> > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
> > For additional commands, e-mail:
> > httpclient-users-h...@hc.apache.org
> >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
For additional commands, e-mail: httpclient-users-h...@hc.apache.org
