If you are using MMapDirectory (default on 64 bit platforms) then they are already in filesystem cache and directly accessible like RAM to indexreader. No need to cache separately.
----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [email protected] > -----Original Message----- > From: Igor Shalyminov [mailto:[email protected]] > Sent: Tuesday, April 02, 2013 9:58 PM > To: [email protected] > Subject: Re: How to use concurrency efficiently > > These are not document hits but text hits (to be more specific, spans). > For the search result it is necessary to have the precise number of document > and text hits and a relatively small number of matched text snippets. > > I've tried several approaches to optimize the search algorithm but they didn't > help - for the specific types of queries there is indeed a great amount of > data > to be retrieved from the index. > At the moment I'm thinking about in-RAM caching of posting lists. Is it > possible in Lucene? > > -- > Igor > > 02.04.2013, 20:44, "Adrien Grand" <[email protected]>: > > On Tue, Apr 2, 2013 at 4:39 PM, Igor Shalyminov > > <[email protected]> wrote: > > > >> Yes, the number of documents is not too large (about 90 000), but the > queries are very hard. Although they're just boolean, a typical query can > produce a result with tens of millions of hits. > > > > How can there be tens of millions of hits with only 90000 docs? > > > >> Single-threadedly such a query runs ~20 seconds, which is too slow. > therefore, multithreading is vital for this task. > > > > Indeed, that's super slow. Multithreading could help a little, but > > maybe there is something to do to better index your data so that > > queries get faster? > > > > -- > > Adrien > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
