On Wed, 8 May 2024 at 17:53, Rajani M <[email protected]> wrote:
> >I think I have limited the documents returned in each query to 10,000 via
> the client software.
>
> Did you mean the solr query rows param is limited to 10,000? Fetching a
> large number of records can be inefficient, it should be less than 100, a
> standard page size. Go through this[1] doc. Hope it helps.
>
> https://cwiki.apache.org/confluence/display/solr/solrperformanceproblems#SolrPerformanceProblems-JavaHeap
>
>
Thank you! That looks really useful. I will read properly later.
In the meantime, I continued to get OOM issues with 256Mb so I have decided
to try limiting the rows returned by my queries to 1000:
# set a sensible limit, like 1000
- return qs.get_results(rows=50000)
+ return qs.get_results(rows=1000)
I already use a paginator [0]. The only problem with limiting the rows is
that there could be more than 1000 results.
[0] django.core.paginator
