[openssl.org #1930] [PATCH] DTLS record buffer limitation bug

[Robin Seggelmann via RT]

Records are buffered if they arrive with a future epoch to be  
processed after finishing the corresponding handshake. There is  
currently no limitation to this buffer allowing an attacker to perform  
a DOS attack with sending records with future epochs until there is no  
memory left. This patch adds the pqueue_size() function to detemine  
the size of a buffer and limits the record buffer to 100 entries.

Thanks to Daniel Mentz for finding this bug!


--- crypto/pqueue/pqueue.c      2005-12-20 08:03:10.000000000 +0100
+++ crypto/pqueue/pqueue.c      2009-05-15 16:07:33.000000000 +0200
@@ -237,3 +237,17 @@

        return ret;
        }
+
+int
+pqueue_size(pqueue_s *pq)
+{
+       pitem *item = pq->items;
+       int count = 0;
+       
+       while(item != NULL)
+       {
+               count++;
+               item = item->next;
+       }
+       return count;
+}

--- crypto/pqueue/pqueue.h      2005-06-08 00:21:14.000000000 +0200
+++ crypto/pqueue/pqueue.h      2009-05-15 16:07:03.000000000 +0200
@@ -89,5 +89,6 @@
  pitem *pqueue_next(piterator *iter);

  void   pqueue_print(pqueue pq);
+int       pqueue_size(pqueue pq);

  #endif /* ! HEADER_PQUEUE_H */

--- ssl/d1_pkt.c        2009-04-23 18:32:40.000000000 +0200
+++ ssl/d1_pkt.c        2009-05-15 16:06:23.000000000 +0200
@@ -207,6 +207,10 @@
        DTLS1_RECORD_DATA *rdata;
        pitem *item;

+       /* Limit the size of the queue to prevent DOS attacks */
+       if (pqueue_size(queue->q) >= 100)
+               return 0;
+               
        rdata = OPENSSL_malloc(sizeof(DTLS1_RECORD_DATA));
        item = pitem_new(priority, rdata);
        if (rdata == NULL || item == NULL)

dtls-record-buffer-bug-1.0.0.patch
Description: Binary data