Just re-read my post and I don't think it was clear.
The algorithm I ended up doing is:
for all daily data
gather hourly ids in a set
build map with placeholders for hourly values
get hourly documents from set
for all daily data
insert hourly data from documents fetched in placeholders
So the method I showed below is the one to "get hourly documents from set"
François
François Eric wrote:
>
> Hello,
>
> I have a performance problem and would need expert advice on how to go
> about fixing it:
>
> I currently have 2 indexes: Daily and Hourly. The Daily index contains
> about 1,000,000 documents and my Hourly index approximately: 24,000,000
> documents. My Daily index contains many fields and some of them are IDs
> to my Hourly Index.
> What I want to do is fetch data in one request (if possible).
> Right now I do it in many requests:
> 1- Get the matching Daily documents (say it returns 500 documents)
> 2- For each of these documents, locate the Hourly Index Id and fetch it.
>
> Therefore I make 501 requests to lucene. This causes some performance
> issues I guess because of the overhead to making a request to Lucene.
>
> Is it possible to do this in 1 request? I'm thinking no because I'm not
> sure what the result set would be but maybe I'm missing something.
>
> If not I guess it would be possible to build a query with my 500 hourly
> ids and make a OR between them to make it in 2 requests....but then I
> have to find the matching documents. Will this overflow if I have 50000
> ids in my query?
>
> Anyway, I just want advice on how one would address this situation.
>
> Thank you very much,
>
> François
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
>
--
View this message in context:
http://old.nabble.com/Query-joining-2-indexes-tp26843980p26848479.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
