The single best performance enhancer you could do would be the external todo patch, from my point of view.
Depends on the filesystem. If the filesystem has indexed directory support, ext-todo should be sufficient. Otherwise, you need the big-ext-todo patch.
