On 4/2/07, Jeff Rodenburg <[EMAIL PROTECTED]> wrote:
Hoping I can get a better response with a more directed question:
I haven't answered your original question as it seems that general java memory debugging techniques would be the most useful thing here.
With facet queries and the fields used, what qualifies as a "large" number of values? The wiki uses U.S. states as an example, so the number of unique values = 50. More to the point, is there an algorithm that I can use to estimate the cache consumption rate for facet queries?
The cache consumption rate is one entry per unique value in all faceted fields, excluding fields that have faceting satisfied via FieldCache (single-valued fields with exacly one token per document). The size of each cached filter is num docs / 8 bytes, unless the number of maching docs is less than the useHashSet threshold in solrconfig.xml. Sorting requires FieldCache population, which consists of an integer per document plus the sum of the lengths of the unique values in the field (less for pure int/float fields, but I'm not sure if Solr's sint qualifies). Both faceting and sorting shouldn't consume more memory after their datastructures have been built, so it would be odd to see OOM after 48 hours if they were the cause. -Mike
