Hi Oleg,
*public final class SharedOutputBuffer extends ExpandableBuffer implements
ContentOutputBuffer*
Above method definition (*extends ExpandableBuffer*) made me think that
SharedOutputBuffer is expandable.
Regards,
Sreenivas
On Thu, Mar 9, 2023 at 8:20 PM sreenivas somavarapu <
soma.sreeni...@gmail.com> wrote:
> Hi Oleg,
>
> *if (src.remaining() < 1024 && buffer().remaining() > src.remaining()) {*
>
> Problem with this above condition is that even though buffer has capacity
> to store complete contents of src, this condition will not satisfy if src
> has 1024 bytes or more data (may well be in limit of actual size provided
> during creation of buffer) to be written to buffer
>
> Simple way to reproduce the problem is (Same has been demonstrated with
> sample program in previous mail)
> - Create SharedOutputBuffer with 4096 bytes of capacity
> - Now try to write 1024 bytes of data to above created buffer
>
> second step still will cause in a hang as first if condition is not
> satisfied due to *src.remaining() < 1024* even though there is enough
> capacity to store data and it proceeds to second condition which will cause
> the program to wait on a condition for flush (even though this is the first
> write we are performing)
>
>
> *if (buffer().position() > 0 || dataStreamChannel == null) {*
>
> * waitFlush();*
> *}*
>
> Regards,
> Sreenivas
>
> On Mon, Mar 6, 2023 at 9:12 PM Oleg Kalnichevski <ol...@apache.org> wrote:
>
>> On Mon, 2023-03-06 at 21:03 +0530, sreenivas somavarapu wrote:
>> > Hi Oleg,
>> >
>> > My sample program (it's a kind of unit test for SharedOutputBuffer to
>> > demonstrate that it is not able to write single byte if input byte
>> > array
>> > size is greater than 1023) doesn't have flush as my intention is to
>> > show
>> > the issue with SharedOutputBuffer alone and it is not a complete
>> > program
>> > which sends the data to the actual server and get response back. Main
>> > issue
>> > lies while writing data greater than 1023 bytes at first go itself
>> > to SharedOutputBuffer from the byte array.
>> >
>> > I think the below conditions in SharedOutputBuffer is causing this
>> > behavior
>> >
>> > while (src.hasRemaining()) {
>> > // always buffer small chunks
>> > *if (src.remaining() < 1024 && buffer().remaining() >
>> > src.remaining()) {*
>> > buffer().put(src);
>> > } else {
>> > * if (buffer().position() > 0 || dataStreamChannel
>> > ==
>> > null) {*
>> > waitFlush();
>> > }
>> >
>>
>> What behavior? This code blocks if it is unable to store input data
>> until the internal buffer gets flushed and space in it frees up, which
>> is what is it supposed to do.
>>
>> If you think SharedOutputBuffer is wrong, please just use your own
>> custom buffer implementation.
>>
>> Oleg
>>
>>
>> > Regards,
>> > Sreenivas
>> >
>> > On Mon, Mar 6, 2023 at 8:52 PM Oleg Kalnichevski <ol...@apache.org>
>> > wrote:
>> >
>> > > On Mon, 2023-03-06 at 20:20 +0530, sreenivas somavarapu wrote:
>> > > > Hi Oleg,
>> > > >
>> > > > It looks like some issue with SharedOutputBuffer implementation
>> > > > and
>> > > > it is
>> > > > also not expandable as we have to allocate complete size during
>> > > > initialization itself. If I change below two lines from my
>> > > > program it
>> > > > proceeds further and completes the request (Basically
>> > > > SharedOutputBuffer is
>> > > > not able to write 1024 or above bytes of data at single go (If we
>> > > > declare
>> > > > temp block size as 1023 also it works)). It looks like there is
>> > > > an
>> > > > explicit 1024 length check in SharedOutputBuffer which is causing
>> > > > this
>> > > > behavior.
>> > > >
>> > > > SharedOutputBuffer buffer = new
>> > > > SharedOutputBuffer(req.contentLength() +
>> > > > 1024);
>> > >
>> > > What you are doing is _completely_ wrong. SharedOutputBuffer is not
>> > > meant to be expandable.
>> > >
>> > > You cannot just write into the buffer and expect it to work by
>> > > magic.
>> > > You also must flush the buffer.
>> > >
>> > > Oleg
>> > >
>> > > >
>> > > > byte[] tmp = new byte[512];
>> > > >
>> > > > Earlier those 2 lines were as below
>> > > >
>> > > > SharedOutputBuffer buffer = new SharedOutputBuffer(1024);
>> > > >
>> > > > byte[] tmp = new byte[4 * 1024];
>> > > >
>> > > > Below is my sample program on SharedOutputBuffer which
>> > > > demonstrated
>> > > > this
>> > > > behavior
>> > > >
>> > > > final StringBuilder sbuffer = new StringBuilder("123");
>> > > > System.out.println("O/P: " + sbuffer.toString());
>> > > > for (long i = 0; i < (64 * 1000); ++i) {
>> > > > sbuffer.append('A');
>> > > > }
>> > > >
>> > > > String postData = sbuffer.toString();
>> > > > System.out.println("O/P1: " + postData);
>> > > >
>> > > > byte[] inputData =
>> > > > postData.getBytes(StandardCharsets.UTF_8);
>> > > > ByteArrayInputStream inputStream = new
>> > > > ByteArrayInputStream(inputData);
>> > > > SharedOutputBuffer outputStream = new
>> > > > SharedOutputBuffer(100
>> > > > *
>> > > > 1024); *<-- If we keep this as 4 * 1024 also it hangs.*
>> > > > *byte[] tmp = new byte[1024]; <-- If we change this to
>> > > > 1023
>> > > > then
>> > > > it works*
>> > > > int l = -1;
>> > > > while ((l = inputStream.read(tmp)) != -1) {
>> > > > System.out.println("Writing Data");
>> > > > System.out.println(l);
>> > > > outputStream.write(tmp, 0, l);
>> > > > System.out.println("Wrote Data");
>> > > > }
>> > > >
>> > > > System.out.println("Done Writing Data");
>> > > > outputStream.writeCompleted();
>> > > > System.out.println(outputStream.toString());
>> > > >
>> > > > Regards,
>> > > > Sreenivas
>> > > >
>> > > > On Mon, Mar 6, 2023 at 7:17 PM Oleg Kalnichevski
>> > > > <ol...@apache.org>
>> > > > wrote:
>> > > >
>> > > > > On Mon, 2023-03-06 at 16:26 +0530, sreenivas somavarapu wrote:
>> > > > > > Hi Team,
>> > > > > >
>> > > > > > We are using AbstractClassicEntityProducer with a
>> > > > > > customization
>> > > > > > of
>> > > > > > storing
>> > > > > > / computing contentLength as well which works fine if we have
>> > > > > > small
>> > > > > > POST
>> > > > > > data, but when POST data is big (for example 64 KB), it seems
>> > > > > > OutPutstream
>> > > > > > (internally using ContentOutputStream a wrapper over
>> > > > > > SharedOutputBuffer of
>> > > > > > initial size 1024) is not able to write data (We are using
>> > > > > > temporary
>> > > > > > buffer
>> > > > > > block of 4kb but still it is not able to write 4KB of data).
>> > > > > > Is
>> > > > > > there
>> > > > > > any
>> > > > > > limitation or some configuration problem?
>> > > > > >
>> > > > > > *Code being used*
>> > > > > > SharedOutputBuffer buffer = new
>> > > > > > SharedOutputBuffer(1024);
>> > > > > >
>> > > > > > produceData(contentType, new
>> > > > > > ContentOutputStream(buffer));
>> > > > > >
>> > > > > > protected void produceData(ContentType contentType,
>> > > > > > OutputStream
>> > > > > > outputStream) throws IOException {
>> > > > > > byte[] tmp = new byte[4 * 1024];
>> > > > > > int l = -1;
>> > > > > > while ((l = inputStream.read(tmp)) != -1) {
>> > > > > > * outputStream.write(tmp, 0, l); <---- This is
>> > > > > > where
>> > > > > > request
>> > > > > > hangs / waits*
>> > > > >
>> > > > > This is likely due to a bug in your code that prevents the
>> > > > > shared
>> > > > > buffer from being flushed.
>> > > > >
>> > > > > Oleg
>> > > > >
>> > > > >
>> > > > > ---------------------------------------------------------------
>> > > > > ----
>> > > > > --
>> > > > > To unsubscribe, e-mail:
>> > > > > httpclient-users-unsubscr...@hc.apache.org
>> > > > > For additional commands, e-mail:
>> > > > > httpclient-users-h...@hc.apache.org
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > > -------------------------------------------------------------------
>> > > --
>> > > To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> > > For additional commands, e-mail:
>> > > httpclient-users-h...@hc.apache.org
>> > >
>> > >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscr...@hc.apache.org
>> For additional commands, e-mail: httpclient-users-h...@hc.apache.org
>>
>>
>
> --
> Cheers,
> S. Sreenivas
>
--
Cheers,
S. Sreenivas
