> Any thoughts on scaling / clustering? Whether i need to use Hadoop / Carrot > etc... >
Carrot2 does search results clustering (by content), while what you probably need is server/index clustering. See the other responses in this thread for suggestions. S.