Memory leaks during SSL handshake

[Shridhar Bhat]

Hi,

I have written a server which receives connections from
SSL clients. I accept SSL connections as well as read the 
data from the client in the following function: DoSSLRead. 
Upon the arrival of the client connection, the SSL_read()
is called 3 times (2 times for handshake data and once
for client app data). When I monitor this process through
top, I see that the process footprint increases by 8kb 
each for first two calls of SSL_read. In the last SSL_read
call, when I read app data, there is no leak. Thus, for
every client connection, my process seems to leak 16kb.
Sometimes, the first SSL_read leaks 12kb. 

I am running this app on Redhat Linux 6.2 and tried both
openssl-0.9.5 and openssl-0.9.6.  I would be grateful if
anybody explains me the reason for the leak. The same
code works fine on WindowsNT and Purify doesn't show me
any leaks. Also, the NT performance monitor doesn't show
any increase in the process footprint size.


/* Reads nSize bytes from SSL socket into pData. If there is
 * still more data in the SSL, bMoreData is set to true. If
 * we can not read the data withing uTimeout period, we return
 * false.
*/
bool DoSSLRead(SSL* pSSL, char* pData, int nSize, bool& bMoreData,
               int& nBytesRead, unsigned int uTimeout)
{
  int iRet1, iRet2, iRet3;
  bool bRet;
  unsigned int whichfds;
  int sock;

  bMoreData = false;
  nBytesRead = 0;

  while (nBytesRead < nSize)
  {
    iRet1 = SSL_read(pSSL, (pData + nBytesRead), (nSize - nBytesRead));
                
    if (iRet1 > 0)
    {
      nBytesRead += iRet1;
    }

    iRet2 = SSL_get_error(pSSL,iRet1);
    switch (iRet2)
    {
    case SSL_ERROR_NONE:
      iRet3 = SSL_pending(pSSL);
      if (iRet3)
      {
        if (nBytesRead == nSize)
        {
        /* There is more data pending but we have filled
         * up the buffer completely. Indicate to the caller
         * that more data is available and return.
        */
          bMoreData = true;
          return true;
        }
        else
        {
        /* There is more data pending and we still
         * have not filled the buffer completely.
        */
          continue;
        }
      }
      else
      { // There is no more data pending
        return true;
      }
      break;
    case SSL_ERROR_WANT_READ:
    /* The data is available but SSL wants us to call
     * SSL_read again. We should call SSL_read but only
     * if we still haven't filled up the buffer.
    */
      if (nBytesRead >= nSize)
        return true;

      sock = SSL_get_fd(pSSL);
      whichfds = 0;

      // Use select to wait till the socket becomes readable
      bRet = DoSelect(sock, READ_SELECT | EXCEPT_SELECT, whichfds, uTimeout);

      if(bRet)
      {
        if(whichfds & READ_SELECT)
          continue; // Ready to Read, go back
        else return false; // Some error or exception
      }
      else return false; // close or error

      break; // Never gets here
    case SSL_ERROR_WANT_WRITE:
      if(nBytesRead >= nSize)
        return true;
                                
      sock = SSL_get_fd(pSSL);
      whichfds = 0;

      // Use select to wait till the socket becomes writable
      bRet = DoSelect(sock, WRITE_SELECT | EXCEPT_SELECT, whichfds, uTimeout);
                                
      if(bRet)
      {
        if(whichfds & WRITE_SELECT)
          continue;
        else return false;
      }
      else return false;                                
        break;

    case SSL_ERROR_ZERO_RETURN:
      return false;
    default:
      return false; /* for now */
    }
  }
  return true;
}

Thanks in advance,

Shridhar.

______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [EMAIL PROTECTED]
Automated List Manager                           [EMAIL PROTECTED]