Re: Blocking on a non-blocking socket?

[Detlef Vollmann]

On 5/24/24 03:30, Wiebe Cazemier via openssl-users wrote:

Hi Matt,

- Original Message -

From: "Matt Caswell" 
To: openssl-users@openssl.org
Sent: Friday, 24 May, 2024 00:26:28
Subject: Re: Blocking on a non-blocking socket?



Not quite.

When you call SSL_read() it is because you are hoping to read
application data.

OpenSSL will go ahead and attempt to read a record from the socket. If
there is no data (and you are using a non-blocking socket), or only a
partial record available then the SSL_read() call will fail and indicate
SSL_ERROR_WANT_READ.

If a full record is available it will process it. If the record contains
application data then the SSL_read() call will return successfully and
provide the application data to the application.

If the record contains non-application data (i.e. some TLS protocol
message like a key update, or new session ticket) then, with
SSL_MODE_AUTO_RETRY on it will automatically try and read another record
(and the above process repeats).


Can you show me in the code where that is? It seems the callers of BIO_read() 
[1] are responsible for doing the retry, because the reader functions abort 
when retry is set. Those are many callers, for x509, evp, b64, etc. But, the 
code is kind of hard to trace, because it's all calls to bio_method_st.bread 
function pointers.

My main concern is, if it would get an EWOULDBLOCK, there is (almost) no sense 
in retrying


That's correct, but if I understand Matt correctly, this isn't the case.
The idea of SSL_MODE_AUTO_RETRY is that if there's data, but it isn't
application data but some kind of handshake data, then SSL_read doesn't
return (after handling the handshake data), but immediately retries.
If this retry fails with EWOULDBLOCK (or actually BIO_read returns 0),
then SSL_read returns with 0 and SSL_WANT_READ.

Without SSL_MODE_AUTO_RETRY SSL_read would return after handling the
handshake data with 0 and SSL_WANT_READ.
I'm not sure how useful this is, but this is how I understand Matt.

  Detlef

Re: SSL_accept doesn't retry BIO_write

[Detlef Vollmann]

On 5/6/24 11:48, Michael Richardson wrote:


 > Now I treat the flush as 'OpenSSL isn't interested in the result
 > of the last write anymore'.  I'm not sure this assumption is correct,
 > but it seems to work... (Well, it could cause duplicate messages

It does not sound correct.


I agree, that's why I asked.


Might be true for a read.
But, for a write, I'd think it would mean fflush().


The man page says:
"BIO_flush() normally writes out any internally buffered data,
in some cases it is used to signal EOF and that no more data
will be written."

The problem is that the protocol between OpenSSL and the BIO
is not very well documented.
As I understand it, a call to BIO_write will be repeated with
the same arguments if the first call returns <= 0.
But this doesn't seem to be the case for the last packet of
the session ticket...


What do your BIO functions do for UDP that the existing set of UDP BIO don't
do?  I have outstanding patches here.


I use ASIO  as the non-blocking framework.
And I can't use a pipe BIO as then I loose the info where
the boundaries of the packets are.

  Detlef

SSL_accept doesn't retry BIO_write

[Detlef Vollmann]

Hello,

I'm trying to write a non-blocking UDP based BIO to use
for a DTLS connection.
When the write() is called then the data is placed for transmission
and '0' is returned.  The respective OpenSSL function returns -1
with SSL_ERROR_WANT_WRITE.
When the data is actually put on the wire a function is called that 
calls the OpenSSL again that then retries the write.  This time

the BIO's write returns '1'.

Now it turns out that at the end of the DTLS handshake (even after
SSL_accept returned success, in Wireshark it looks like the
session key) OpenSSL still send data via the BIO's write, but it
looks like the last packet is never retried.  Instead OpenSSL
calls the BIO's control functionwith BIO_CTRL_FLUSH.

Now I treat the flush as 'OpenSSL isn't interested in the result
of the last write anymore'.  I'm not sure this assumption is correct,
but it seems to work... (Well, it could cause duplicate messages
if OpenSSL sends a flush but still retries the last write, but
this is UDP and both ends need to deal with duplicates (and losses)
anyways.

Is this approach ok or have I missed something?

  Detlef

OCSP verification in a non-blocking environment

[Detlef Vollmann]

I have a non-blocking DTLS server and use SSL_VERIFY_PEER.
Now I'd like to use the verify_callback that I set with
SSL_CTX_set_verify() to check via OCSP for revocation.
This works fine in a simple blocking test program, where
I can just wait for the OCSP reply and then return 0 or 1
from my verify_callback function dependent on this reply.

But in a non-blocking program I can't wait for the OCSP reply,
but I also can't return -1 from my verify_callback function
to tell the handshake code to call the callback again.

Am I right that I can only do a non-blocking OCSP check after
the handshake has finished?

  Detlef

Re: Real MTU problems with BIO pair

[Detlef Vollmann]

On 2020-08-21 19:48, Benjamin Kaduk wrote:

On Fri, Aug 21, 2020 at 05:05:51PM +0200, Detlef Vollmann wrote:

On 2020-08-20 21:44, Detlef Vollmann wrote:


Is there any way to set the maximum fragment size for
DTLS handshake with a BIO pair?

One solution is to set the MTU and the int_bio size to
exactly the same value.
Another option would be to use BIO_set_callback_ex() and send
the data to the socket after each BIO_write() into int_bio,
but the problem here is that BIO_set_data() cannot be used
as the ptr is already used for the peer address.


There's always EX_DATA...

Thanks for the pointer. Using my own hash table would also be
an option.

But in the meantime I found that I can define my own BIO_METHOD,
so this is probably my preferred option.

  Detlef

Re: Real MTU problems with BIO pair

[Detlef Vollmann]

On 2020-08-20 21:44, Detlef Vollmann wrote:

if I create a BIO pair with
   BIO_new_bio_pair(&int_bio, 0, &ext_bio_, 0);

then I tried to use SSL_set_mtu(), DTLS_set_link_mtu()
and SSL_CTX_set_max_send_fragment(ctx, 1000).
None of them gave me an error, but also none of them worked:
the ServerHello was still sent as a single packet (>1500 bytes).

It turned out that this was not true: it actually were two
packets but written to the BIO together before SSL_accept()
returned, so my side of the bio pair got on a BIO_read()
one single big packet and sent it to the socket and the wire
as one UDP packet.


If I create the BIO pair using
   BIO_new_bio_pair(&int_bio, 1000, &ext_bio_, 1000);
then the ServerHello is fragmented, but not into DTLS
handshake fragments, but just into separate UDP packets,
that neither s_client nor my own client can work with.

Is there any way to set the maximum fragment size for
DTLS handshake with a BIO pair?

One solution is to set the MTU and the int_bio size to
exactly the same value.
Another option would be to use BIO_set_callback_ex() and send
the data to the socket after each BIO_write() into int_bio,
but the problem here is that BIO_set_data() cannot be used
as the ptr is already used for the peer address.

  Detlef

Real MTU problems with BIO pair

[Detlef Vollmann]

Hello,

if I create a BIO pair with
  BIO_new_bio_pair(&int_bio, 0, &ext_bio_, 0);

then I tried to use SSL_set_mtu(), DTLS_set_link_mtu()
and SSL_CTX_set_max_send_fragment(ctx, 1000).
None of them gave me an error, but also none of them worked:
the ServerHello was still sent as a single packet (>1500 bytes).

If I create the BIO pair using
  BIO_new_bio_pair(&int_bio, 1000, &ext_bio_, 1000);
then the ServerHello is fragmented, but not into DTLS
handshake fragments, but just into separate UDP packets,
that neither s_client nor my own client can work with.

Is there any way to set the maximum fragment size for
DTLS handshake with a BIO pair?

Thanks,
  Detlef

Surprising behaviour of DTLSv1_listen

[Detlef Vollmann]

Hello,

if I do:
// ctx is setup with certificate, key and cookie callbacks
BIO *bio = BIO_new_dgram(sock, BIO_NOCLOSE);
SSL *ssl = SSL_new(ctx);
SSL_set_bio(ssl, bio, bio.get());
DTLS_set_link_mtu(ssl, 1000);
SSL_set_options(ssl, SSL_OP_COOKIE_EXCHANGE);
SSL_set_accept_state(ssl);
SSL_accept(ssl);

then the MTU setting works as expected, i.e. the ServerHello
is split into two DTLS handshake fragments.

But if I do:
BIO *bio = BIO_new_dgram(sock, BIO_NOCLOSE);
SSL *ssl = SSL_new(ctx);
SSL_set_bio(ssl, bio, bio.get());
DTLS_set_link_mtu(ssl, 1000);
SSL_set_options(ssl, SSL_OP_COOKIE_EXCHANGE);
SSL_set_accept_state(ssl);
DTLSv1_listen(ssl, addr.get());
SSL_accept(ssl);

then the ServerHello is sent as a single packet (>1500 bytes).

I think the reason is that DTLSv1_listen() internally
calls SSL_clear().
I find this pretty surprising.

I personally don't really care too much, as I'll do my
own cookie handshake without DTLSv1_listen() before I
call SSL_accept(), but I thought I'd report it anyway.

  Detlef

Re: NULL ciphers

[Detlef Vollmann]

On 2020-08-13 20:20, Benjamin Kaduk wrote:

On Thu, Aug 13, 2020 at 08:19:10PM +0200, Detlef Vollmann wrote:

Hello,

with the following commands:

openssl s_server -accept 18010 -cert srv.crt -key test.key \
-CAfile testca.crt -debug -cipher 'NULL-SHA256' -dtls1_2

openssl s_client -connect localhost:18010 -cert clnt.crt \
  -key test.key -CAfile testca.crt -debug \
  -cipher 'COMPLEMENTOFALL:eNULL' -dtls1_2

NULL ciphers work fine with OpenSSL 1.0.2g.

With OpenSSL 1.1.1g the handshake fails on the server side with
140295725053248:error:14102438:SSL routines:dtls1_read_bytes:tlsv1 \
alert internal error:../ssl/record/rec_layer_d1.c:611:SSL alert number \
80

Even on OpenSSL 1.1.1g 'openssl ciphers -v  NULL' lists NULL-SHA256.

I'm only using s_server and s_client as tests, but I have the same
problem in my application if I use
  SSL_CTX_set_cipher_list(sslCtx, "NULL-SHA256");

What can I do to get NULL ciphers for no encryption working?


-cipher 'COMPLEMENTOFALL:eNULL@SECLEVEL=0'


Wow, great :-)
Thanks a lot for this quick reply, it actually works :-)

  Detlef

NULL ciphers

[Detlef Vollmann]

Hello,

with the following commands:

openssl s_server -accept 18010 -cert srv.crt -key test.key \
-CAfile testca.crt -debug -cipher 'NULL-SHA256' -dtls1_2

openssl s_client -connect localhost:18010 -cert clnt.crt \
 -key test.key -CAfile testca.crt -debug \
 -cipher 'COMPLEMENTOFALL:eNULL' -dtls1_2

NULL ciphers work fine with OpenSSL 1.0.2g.

With OpenSSL 1.1.1g the handshake fails on the server side with
140295725053248:error:14102438:SSL routines:dtls1_read_bytes:tlsv1 \
alert internal error:../ssl/record/rec_layer_d1.c:611:SSL alert number \
80

Even on OpenSSL 1.1.1g 'openssl ciphers -v  NULL' lists NULL-SHA256.

I'm only using s_server and s_client as tests, but I have the same
problem in my application if I use
 SSL_CTX_set_cipher_list(sslCtx, "NULL-SHA256");

What can I do to get NULL ciphers for no encryption working?

  Detlef

DTLS reconnect

[Detlef Vollmann]

Hello,

section 4.2.8 "Establishing New Associations with Existing Parameters"
of RFC 6347 ()
recommends ("SHOULD") that a new ClientHello from a client IP/port
pair for which a session already exists initiates a new handshake.

I tried to test such a scenario using s_server and s_client:

As server:
openssl s_server -accept 127.0.0.1:18001 -4 -cert demoCA/srv.crt -key 
demoCA/testdev.key -CAfile demoCA/testca.crt -dtls1_2


As client:
openssl s_client -bind 127.0.0.1:19933 -connect 127.0.0.1:18001 -4 
-noservername -cert demoCA/clnt.crt -key demoCA/testdev.key -CAfile 
demoCA/testca.crt -dtls1_2


Then stop the client using CTRL-C and start it again.
tcpdump/wireshark shows the first (successful) handshake and then
ClientHellos from the same IP/port pair which are never replied to.


I tested the official Debian unstable package 1.1.1g-1 and
today's master (a3f15e237c).

So here are my questions:

 - Is this part of RFC 6347 implemented in OpenSSL?
 - If yes, how can it be used?

Note: this is not the same as what the "-reconnect' option of
s_client does.  This reconnect sends a ClientHello with a
previously established session ID for cached connections.
Section 4.2.8 of DTLS 1.2 is about a client that has no
existing session context but tries to connect from an IP address
and UDP port it has used previously.  This happens e.g. for many
embedded devices after a reset.

Thanks,
  Detlef