Re: Solr, Dovecot & macOS / iOS

[@lbutlr via dovecot]

On 13 Aug 19, at 05:58 , James Brown  wrote:
> 
> b) does Mail.app and other mail clients on Macs or iOS devices perform 
> searches on their local copy of mail or does it just send a search request to 
> the server?

Mail.app uses spotlight on the local data, so if your users are all Mac then 
solar is pointless.

Re: Solr, Dovecot & macOS / iOS

[Jean-Daniel via dovecot]

> Le 13 août 2019 à 14:53, Sami Ketola  a écrit :
> 
> 
> 
>> On 13 Aug 2019, at 15.37, Jean-Daniel via dovecot > > wrote:
>> 
>> 
>> 
>>> Le 13 août 2019 à 14:16, Sami Ketola via dovecot >> > a écrit :
>>> 
>>> 
>>> 
 On 13 Aug 2019, at 14.58, James Brown via dovecot >>> > wrote:
 
 I’m thinking of getting Solr working with my Dovecot server. Server is new 
 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of 
 thousands of messages.
 
 But I’m not sure if:
 
 a) it will make enough of a difference and
>>> 
>>> Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 
>>> 100s of thousands of small files in same directory. If so, then use mdbox 
>>> with periodic purges.
>>> 
 
 b) does Mail.app and other mail clients on Macs or iOS devices perform 
 searches on their local copy of mail or does it just send a search request 
 to the server?
>>> 
>>> None of the apple devices use IMAP SEARCH. They ALL maintain and use their 
>>> own local search database on the device. Also they seem to refresh the 
>>> database every now and then redownloading all emails.
>> 
>> Do you have a source for that. My experience is that without server search 
>> support, iOS is very slow at returning result. Moreover, it keep only latest 
>> messages and never download message until you read them.
> 
> I'm a apple device user myself. I have couple of iPhones, couple of iPads, 
> couple if MacBooks and Mail.app on any of them is not using IMAP SEARCH.
> And I cannot find any configuration option to enable it. Only spotlight index 
> is used. On Mac OS Mail.App seems to store the indexed data to:
> 
> samik@samikworkmac:~>ls -1 Library/Mail/V6/MailData/Envelope\ Index*
> Library/Mail/V6/MailData/Envelope Index
> Library/Mail/V6/MailData/Envelope Index-shm
> Library/Mail/V6/MailData/Envelope Index-wal
> 
> if those files are removed or spotlight search for mails is disabled Mail.App 
> can't find anything anymore. It does not fall back to IMAP SEARCH.
> 

My question was more about iOS. I know that macOS Mail does not rely on any way 
on remote indexing and has it’s own local index, but as it also store all 
messages locally, it’s an easy requirement. For iOS that only download messages 
meta-data by default, I was not so sure. 

I’m accessing my mail server using Apple devices only, and see some imap SEARCH 
requests in dovecot stats, but can’t figure out where they came from though. So 
you may be right.

Re: Solr, Dovecot & macOS / iOS

[Sami Ketola via dovecot]

> On 13 Aug 2019, at 15.37, Jean-Daniel via dovecot  wrote:
> 
> 
> 
>> Le 13 août 2019 à 14:16, Sami Ketola via dovecot  a 
>> écrit :
>> 
>> 
>> 
>>> On 13 Aug 2019, at 14.58, James Brown via dovecot  
>>> wrote:
>>> 
>>> I’m thinking of getting Solr working with my Dovecot server. Server is new 
>>> 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of 
>>> thousands of messages.
>>> 
>>> But I’m not sure if:
>>> 
>>> a) it will make enough of a difference and
>> 
>> Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 
>> 100s of thousands of small files in same directory. If so, then use mdbox 
>> with periodic purges.
>> 
>>> 
>>> b) does Mail.app and other mail clients on Macs or iOS devices perform 
>>> searches on their local copy of mail or does it just send a search request 
>>> to the server?
>> 
>> None of the apple devices use IMAP SEARCH. They ALL maintain and use their 
>> own local search database on the device. Also they seem to refresh the 
>> database every now and then redownloading all emails.
> 
> Do you have a source for that. My experience is that without server search 
> support, iOS is very slow at returning result. Moreover, it keep only latest 
> messages and never download message until you read them.

I'm a apple device user myself. I have couple of iPhones, couple of iPads, 
couple if MacBooks and Mail.app on any of them is not using IMAP SEARCH.
And I cannot find any configuration option to enable it. Only spotlight index 
is used. On Mac OS Mail.App seems to store the indexed data to:

samik@samikworkmac:~>ls -1 Library/Mail/V6/MailData/Envelope\ Index*
Library/Mail/V6/MailData/Envelope Index
Library/Mail/V6/MailData/Envelope Index-shm
Library/Mail/V6/MailData/Envelope Index-wal

if those files are removed or spotlight search for mails is disabled Mail.App 
can't find anything anymore. It does not fall back to IMAP SEARCH.

Sami

Re: Solr, Dovecot & macOS / iOS

[Jean-Daniel via dovecot]

> Le 13 août 2019 à 14:16, Sami Ketola via dovecot  a 
> écrit :
> 
> 
> 
>> On 13 Aug 2019, at 14.58, James Brown via dovecot  
>> wrote:
>> 
>> I’m thinking of getting Solr working with my Dovecot server. Server is new 
>> 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands 
>> of messages.
>> 
>> But I’m not sure if:
>> 
>> a) it will make enough of a difference and
> 
> Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 
> 100s of thousands of small files in same directory. If so, then use mdbox 
> with periodic purges.
> 
>> 
>> b) does Mail.app and other mail clients on Macs or iOS devices perform 
>> searches on their local copy of mail or does it just send a search request 
>> to the server?
> 
> None of the apple devices use IMAP SEARCH. They ALL maintain and use their 
> own local search database on the device. Also they seem to refresh the 
> database every now and then redownloading all emails.

Do you have a source for that. My experience is that without server search 
support, iOS is very slow at returning result. Moreover, it keep only latest 
messages and never download message until you read them.

Re: Solr, Dovecot & macOS / iOS

[Sami Ketola via dovecot]

> On 13 Aug 2019, at 14.58, James Brown via dovecot  wrote:
> 
> I’m thinking of getting Solr working with my Dovecot server. Server is new 
> 6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands 
> of messages.
> 
> But I’m not sure if:
> 
> a) it will make enough of a difference and

Choose mailbox format wisely. sdbox preferred unless HFS+ has problems with 
100s of thousands of small files in same directory. If so, then use mdbox with 
periodic purges.

> 
> b) does Mail.app and other mail clients on Macs or iOS devices perform 
> searches on their local copy of mail or does it just send a search request to 
> the server?

None of the apple devices use IMAP SEARCH. They ALL maintain and use their own 
local search database on the device. Also they seem to refresh the database 
every now and then redownloading all emails.

> 
> I’m guessing the searches are done locally so no point in Solr?

No point if only apple client devices.

Sami

Solr, Dovecot & macOS / iOS

[James Brown via dovecot]

I’m thinking of getting Solr working with my Dovecot server. Server is new 
6-core Mac Mini, mail store of over 1/2 TB. Mailboxes with 100s of thousands of 
messages.

But I’m not sure if:

a) it will make enough of a difference and

b) does Mail.app and other mail clients on Macs or iOS devices perform searches 
on their local copy of mail or does it just send a search request to the server?

I’m guessing the searches are done locally so no point in Solr?

Does anyone know? 

Thanks, James.


smime.p7s
Description: S/MIME cryptographic signature