Hi,
we use Lucene to store around 300 millions of records. We use the index both
for conventional searching, but also for all the system's data - we replaced
MySQL with Lucene because it was simply not working at all with MySQL due to
the amount or records. Our problem is that we have HUGE performance
problems... whenever we search, it takes forever to return results, and Java
uses 100% CPU/RAM.
Our index fields are like this:
TYPE
PK
FOREIGN_PK
TAG
...other information depending on type...
* All fields are Field.Index.UN_TOKENIZED
* The field "TAG" always contains the value "TAG".
Whenever we search in the index, our query is "TAG:TAG" to match all
documents, and we do the search like this:
// Search
Hits h = searcher.search(q, cluCF, cluSort);
cluCF is a ChainedFilter containing all the other filters (like
FOREIGN_PK=12345, TYPE=a, etc.).
I know that the method is probably crazy because "TAG:TAG" is matching all
300M documents and then it applies filters; so that's probably why every
little query is taking 100% CPU/RAM.... but I don't know how to do it
properly.
Help ! Any advice is welcome.
- Mike
aka...@gmail.com
