On Thu, Sep 20, 2012 at 8:12 AM, Mathias Krause <mini...@googlemail.com> wrote: > What still might happen is the overflow in xfrm_replay_state_esn_len() > resulting in a to small bitmap allocation for the requested replay > size. But that gets catched in xfrm_init_replay(). Little late, but > hey.
Sorry, I mixed that up. The replay_window check in xfrm_init_replay() has only little to do with the bmp_len overflow. But changing the return type of xfrm_replay_state_esn_len() to size_t and by doing so, making the all the size compares operating on positive values, we'll at least allocate enough memory to not run into memory corruptions. Though, the replay window will be much smaller, than requested -- due to the overflow. But userland should expect this. A check for some upper limit in verify_replay() could catch this early. Mathias -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
