Michael, I think you might be get interested in "zoie".
zoie: real-time search and indexing system built on Apache Lucene http://code.google.com/p/zoie/ Zoie is realtime search project for lucene by Linkedin. Basically, I think it is similar technique to a Otis's trick. >>In the mean time you can use the trick of one large and less frequently >>updated core and one small and more frequently >>updated core + distributed >>search across them. >> >>Otis Genta On Sat, Feb 7, 2009 at 3:02 AM, Michael Austin <mausti...@gmail.com> wrote: > I need to find a solution for our current social application. It's low > traffic now because we are early on.. However I'm expecting and want to be > prepaired to grow. We have messages of different "types" that are > aggregated into one stream. Each of these message types have much different > data so that our main queries have a few unions and many joins. I know that > Solr would work great for searching but we need a realtime system > (twitter-like) to view user updates. I'm not interested in a few minutes > delay; I need something that will be fast updating and searchable and have n > columns per record/document. Can solor do this? what is Ocean? > > Thanks >