DTLSv1_listen calls dtls1_listen in ssl/d1_lib.c:
int dtls1_listen(SSL *s, struct sockaddr *client)
{
int ret;
SSL_set_options(s, SSL_OP_COOKIE_EXCHANGE);
s->d1->listen = 1;
ret = SSL_accept(s);
if (ret <= 0) return ret;
(void) BIO_dgram_get_peer(SSL_get_rbio(s), client);
return 1;
}
The function calls BIO_dgram_get_peer on the rbio unconditionally, even if
it is e.g. a memory BIO instead of a dgram BIO. If I'm not mistaken this
could result in undefined behaviour. Moreover, calling DTLSv1_listen
appears to be the only path to setting s->d1->listen to 1. Here is a
possible fix:
int dtls1_listen(SSL *s, struct sockaddr *client)
{
int ret;
SSL_set_options(s, SSL_OP_COOKIE_EXCHANGE);
s->d1->listen = 1;
ret = SSL_accept(s);
if (ret <= 0) return ret;
if(client != NULL)
(void) BIO_dgram_get_peer(SSL_get_rbio(s), client);
return 1;
}
That would allow calling applications not using a dgram rbio to pass NULL
in place of the client sockaddr and avoid the call to BIO_dgram_get_peer.