//lucene.472066.n3.nabble.com/Estimating-peak-memory-use-for-UnInvertedField-faceting-tp4100044p4114771.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: dev-unsubscr...@luc
Hi Tom,
I believe Solr will automatically use DocValues for faceting if you've
defined them in the schema.
Otis
--
Performance Monitoring * Log Analytics * Search Analytics
Solr & Elasticsearch Support * http://sematext.com/
On Mon, Nov 11, 2013 at 11:33 AM, Tom Burton-West wrote:
> Thanks Oti
Thanks Otis,
I'm looking forward to the presentation videos.
I'll look into using DocValues.Re-indexing 200 million docs will take a
while though :).
Will Solr automatically use DocValues for faceting if you have DocValues
for the field or is there some configuration or parameter that needs
Hi Tom,
Check http://blog.sematext.com/2013/11/09/presentation-solr-for-analytics/
. It includes info about our experiment with DocValues, which clearly
shows lower heap usage, which means you'll get further without getting
this OOM. In our experiments we didn't sort, facet, or group, and I
see
Hi Yonik,
I don't know enough about JVM tuning and monitoring to do this in a clean
way, so I just tried setting the max heap at 8GB and then 6GB to force
garbage collection. With it set to 6GB it goes into a long GC loop and
then runs out of heap (See below) . The stack trace says the issue is
On Fri, Nov 8, 2013 at 1:56 PM, Tom Burton-West wrote:
> When testing an index of about 200 million documents, when we do a first
> faceting on one field (query appended below), the memory use rises from
> about 2.5 GB to 13GB. If I run GC after the query the memory use goes down
> to about 3GB a
We are considering indexing our 11 million books at a page level, which
comes to about 3 billion Solr documents.
Our subject field by necessity is multi-valued so the UnInvertedField is
used for faceting.
When testing an index of about 200 million documents, when we do a first
faceting on one fi
