I have already store it in lucene index. But it is in disk and When a query come, it must seek the disk to get it. I am not familiar with lucene cache. I just want to fully use my memory that load 10GB of it in memory and a LRU stragety when cache full. To load more into memory, I want to compress it "in memory". I don't care much about disk space so whether or not it's compressed in lucene .
2010/7/14 findbestopensource <findbestopensou...@gmail.com>: > You have two options > 1. Store the compressed text as part of stored field in Solr. > 2. Using external caching. > http://www.findbestopensource.com/tagged/distributed-caching > You could use ehcache / Memcache / Membase. > > The problem with external caching is you need to synchronize the deletions > and modification. Fetching the stored field from Solr is also faster. > > Regards > Aditya > www.findbestopensource.com > > > On Wed, Jul 14, 2010 at 12:08 PM, Li Li <fancye...@gmail.com> wrote: > >> I want to cache full text into memory to improve performance. >> Full text is only used to highlight in my application(But it's very >> time consuming, My avg query time is about 250ms, I guess it will cost >> about 50ms if I just get top 10 full text. Things get worse when get >> more full text because in disk, it scatters erverywhere for a query.). >> My full text per machine is about 200GB. The memory available for >> store full text is about 10GB. So I want to compress it in memory. >> Suppose compression ratio is 1:5, then I can load 1/4 full text in >> memory. I need a Cache component for it. Has anyone faced the problem >> before? I need some advice. Is it possbile using external tools such >> as MemCached? Thank you. >> >
