On 3/20/23 09:52, Alessio Cecchi wrote:
I'm running a mail server with Dovecot 2.3.20 and Apache Solr as FTS
backend.
It seems that iPhone/iPad Mail clients are generating some IMAP
searches, especially on header Message-ID, that are increasing our Solr
I/O load, especially during the night.
Are these kind of queries normal? What are they?
I don't know whether they are normal or not. That requires knowledge
about IMAP that I do not have. It does seem odd that a mail client
would be doing header searches unless the user is actually doing a
search. I would expect it to look messages up by the ID using other
imap commands, not with a header search. But my knowledge about IMAP is
not complete enough to say for sure.
You can find some logs below.
Those Solr logs show what SHOULD be low-impact queries on a system
that's appropriately sized.
I suspect what's happening is that the Solr server doesn't have enough
free memory (memory not allocated by ANY program) to effectively cache
the index.
If there is enough free memory for the OS disk cache and the cache is
reasonably warmed up, queries like the ones you showed should produce
very little disk I/O, because the data that they need will be sitting in
memory and an actual disk read is unnecessary.
Once the search completes, the record retrievals might result in a cache
miss, but the amount of data that gets retrieved for a dovecot search
result is quite small. The data is compressed, so a few CPU cycles are
required to decompress it.
Memory-related performance issues with Solr come up frequently enough
that I wrote a wiki article about it:
https://cwiki.apache.org/confluence/display/solr/solrperformanceproblems
Thanks,
Shawn
