[openssl-users] Workaround for 'unexpected record' error during renegotiation

[hokusai]

Hello All,

 

I experience the same problem as other people described in the past. Despite reading all the postings on the topic I am still not sure if there is a usable workaround to make spontaneous message exchange between server an client work:

 

- both client and server exchange spontaneous application messages.
- both client and server use non-blocking sockets. SSL_read() and SSL_write() are serviced by one message pump thread.
- server initiates renegotiation with SSL_renegotiate()
- both client and server continue exchanging messages and renegotiation completes implicitly within the calls of SSL_read() and SSL_write()

--> sometimes (depending on timing) communication fails in client SSL_write() with 'unexpected record' because an application message is received instead of the expected renegotiation message.

So far, so good (bad)

 

My question is if suppressing SSL_write() when SSL_renegotiate_pending() returns true would reliably avoid the problem:
 
Her the pseudo-code:

server
-------

while(1) {
    select()
    if needToRenegotiate() {
        SSL_renegotiate()
    }

    SSL_read() // always read
    if hasDataToSend() && !SSL_renegotiate_pending() {
        SSL_write()
    }  
    handleWantSomethingReturnCodes()
}

client
------

while(1) {
    select()
    SSL_read() // always read
    if hasDataToSend() && !SSL_renegotiate_pending() {
        SSL_write()
    }  
    handleWantSomethingReturnCodes()
}

Since I implemented this trick the error has never happened anymore, however, I wonder if the problem is really solved ot it just happens far less frequently in race conditions, e.g. when messages arrive between the check on SSL_renegotiate_pending() and SSL_write()

My interpretation of the logic in s3_pkt.c is that it's ok to receive application data between clientHello and serverHello in client or between helloRequest and clientHello in server. But this is just a guess.

Can somebody with good knowledge of the whole OpenSSL state-machine logic give an authoritative opinion?
 
    case SSL3_RT_APPLICATION_DATA:
        /*
         * At this point, we were expecting handshake data, but have
         * application data.  If the library was running inside ssl3_read()
         * (i.e. in_read_app_data is set) and it makes sense to read
         * application data at this point (session renegotiation not yet
         * started), we will indulge it.
         */
        if (s->s3->in_read_app_data &&
            (s->s3->total_renegotiations != 0) &&
            (((s->state & SSL_ST_CONNECT) &&
              (s->state >= SSL3_ST_CW_CLNT_HELLO_A) &&
              (s->state <= SSL3_ST_CR_SRVR_HELLO_A)
             ) || ((s->state & SSL_ST_ACCEPT) &&
                   (s->state <= SSL3_ST_SW_HELLO_REQ_A) &&
                   (s->state >= SSL3_ST_SR_CLNT_HELLO_A)
             )
            )) {
            s->s3->in_read_app_data = 2;
            return (-1);
        } else {
            al = SSL_AD_UNEXPECTED_MESSAGE;
            SSLerr(SSL_F_SSL3_READ_BYTES, SSL_R_UNEXPECTED_RECORD);
            goto f_err;
        }

 

Many Thanks

Fabrizio

_______________________________________________
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users