Not sure about the speedup (we might have small sizes often enough that the constant factor becomes relevant), but the code is quite a net win with regard to obviously doing what it should do. It probably takes a whole lot more allocation calls, but at least the impact on memory is just temporary.
LGTM https://codereview.appspot.com/583390043/
