Great, that is a possible performance enhancement for the future, as very large directories can be slow to read, even when totally cached by Linux. For the immediate term, I'll talk with the aufs folks to see if directory reads can get faster.
$ time ls -U > /dev/null # cached real 0m1.471s user 0m0.530s sys 0m0.940s
