On 5/5/13 7:48 PM, Mingfeng Yang wrote:
Dear Solr Users,
Does anyone know what is the best way to iterate through each document in a
Solr index with billion entries?
I tried to use select?q=*:*&start=xx&rows=500 to get 500 docs each time
and then change start value, but it got very slow after getting through
about 10 million docs.
Thanks,
Ming-
You need to use a unique and stable sort key and get documents >
sortkey. For example, if you have a unique key, retrieve documents
ordered by the unique key, and for each batch get documents > max (key)
from the previous batch
-Mike
