Re: [openssl-users] How to know maximum sendable fragment size?

[Matt Caswell]

On 07/11/17 03:56, J Decker wrote:
> I've been developing this NodeJS plugin, it implements HTTPS server and
> now client.  I was having an issue with HTTPS request getting ECONNRESET
> for no apparent reason; so I implemented my own request, and ran into
> the same sort of issue.  What I was requesting was some .js files from
> the server, and apparently my most recent changes to those files made
> them larger than some magic number greater than 4096 but less than
> 6561.  The server was sending using OpenSSL (statically linked in the
> NodeJS executable) on CentOS, and it was sending the full length of the
> file as one buffer.  I'm using memory BIO to interact with the SSL
> object; The buffer was transmitted as one block. With my own client,
> (where I could add debugging) was receiving the full count of bytes from
> the server but in two blocks, the first 4096 and the second
> 2472(something like that).  Because my network read buffer was
> only 4096 So the first read was short, and caused SSL_read to fail,

What do you mean this cause SSL_read() to fail? You got a <= 0 return
value? This should not happen. It is perfectly valid to read less bytes
than OpenSSL has available.


> which I had initially treated as a failure and terminated my
> connection.  I then
> found I could (almost) check using SSL_pending before getting an
> error (really I ended up doing SSL_read( ssl, NULL, 0 ) and then
> SSL_pending(ssl)
> ).  But after receiving the full count of bytes and having nothing else
> to receive, the message never completed (read return -1, and error 2,
> pending
> returned 0 ).

I'm not sure sure what you mean by "the message never completed". Do you
mean you were expecting more bytes but they never came?

> I manually broke up the transmission to 4356 (3*1452 -29)
> bytes so it ends up sending in 3 full tcp buffer units, and that works. 
> (it's http protocol so it's got higher level gathering for the
> fragments).  It also works if I revert to using the NodeJS HTTPS request
> object instead of my own.
> 
> So - how do I know what the maximum amount of data I can send is?

TLS is a stream protocol. There is no maximum amount of data you can
send in one go. Internally the protocol breaks up the data into a number
of records. The maximum amount of plaintext data sent in a single record
is SSL3_RT_MAX_PLAIN_LENGTH (16384) bytes. This can be changed by your
application (to something smaller - not larger) but you have to
explicitly do that. However changing this should have no impact on the
functional behaviour of your application.

> 
> Node TLS object (on which HTTPS is based)
> has tlsSocket.setMaxSendFragment(size)(which defaults to 16384)  but
> that's about sending, not receiving, so I really have no idea how big
> the receive buffer is actually (same as SSL send fragment default)
> 
> I did
> find 
> https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_set_split_send_fragment.html
> 
>  but
> there's no get_for fragment size I could find.  (this would be on
> the server side that I need to know how much I can send).
> 
> how do I set how big of a fragment I can receive?

The TLS protocol defines the size of the maximum fragment so no well
behaved implementation will ever send more than that. The OpenSSL
buffers are sized accordingly. In normal operation you should never need
to play around with these settings. Currently it is not possible to
change the size of the receive fragment - the sender is always allowed
to send any size up the to the maximum, so the receive buffers must
always be at least as big as that. In 1.1.1 (in development) there is a
feature which allows the client and server to negotiate a smaller
fragment size if they wish - but this is typically only useful in
resource constrained environments.

>  Like what if I tried
> to send 100's of Meg as a single fragment?   (I guess it should auto
> fragment to like 16k)

If you send a large amount of data in an SSL_write() call then OpenSSL
will automatically break that up into a series of records containing the
maximum amount of data until all of the data has been sent.

> 
> I guess there (will be) SSL_CTX_set_default_read_buffer_len() (1.1.0)

Unless you are doing pipelining with an engine that can support it then
you do not need to call this function.

Matt
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

[openssl-users] How to know maximum sendable fragment size?

[J Decker]

I've been developing this NodeJS plugin, it implements HTTPS server
and now client.
I was having an issue with HTTPS request getting ECONNRESET for no apparent
reason; so I implemented my own request, and ran into the same sort of
issue.  What I was requesting was some .js files from the server, and
apparently
my most recent changes to those files made them larger than some magic
number greater than 4096 but less than 6561.  The server was sending using
OpenSSL (statically linked in the NodeJS executable) on CentOS, and it was
sending the full length of the file as one buffer.  I'm using memory BIO to
interact with the SSL object; The buffer was transmitted as one block. With
my own client, (where I could add debugging) was receiving the full count
of bytes from the server but in two blocks, the first 4096 and the second
2472(something like that).  Because my network read buffer was only 4096
So the first read was short, and caused SSL_read to fail, which I had
initially treated as a failure and terminated my connection.  I then
found I could (almost) check using SSL_pending before getting an error (really
I ended up doing SSL_read( ssl, NULL, 0 ) and then SSL_pending(ssl)
).  But after receiving the full count of bytes and having nothing
else to receive,
the message never completed (read return -1, and error 2, pending
returned 0 ). I manually broke up the transmission to 4356 (3*1452 -29)
bytes so it ends up sending in 3 full tcp buffer units, and that works.
(it's http protocol so it's got higher level gathering for the fragments).
It also works if I revert to using the NodeJS HTTPS request object instead
of my own.

So - how do I know what the maximum amount of data I can send is?

Node TLS object (on which HTTPS is based) has
tlsSocket.setMaxSendFragment(size)(which
defaults to 16384)  but that's about sending, not receiving, so I really
have no idea how big the receive buffer is actually (same as SSL send
fragment default)

I did find https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_set_
split_send_fragment.html but there's no get_for fragment size I could
find.  (this would be on the server side that I need to know how much I can
send).

how do I set how big of a fragment I can receive?  Like what if I tried to send
100's of Meg as a single fragment?   (I guess it should auto fragment to
like 16k)

I guess there (will be) SSL_CTX_set_default_read_buffer_len() (1.1.0)

(node's open ssl version)
#  define OPENSSL_VERSION_TEXT"OpenSSL 1.0.2l  25 May 2017"

I guess read should default to something like SSL3_RT_MAX_PLAIN_LENGTH
+ SSL3_RT_MAX_ENCRYPTED_OVERHEAD
(16704)  ?  I wonder why it doesn't.
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users