doveadm backup -R -m -s not picking up a new message

2020-02-05 Thread Daniel Shahaf
Good morning,

I use «doveadm backup -R -m foo -s $state» to create a local mirror of
an imapc remote folder.  When I run that command, wait for it to finish,
deliver a new message to the remote folder, and run that command again,
the new message does not get downloaded from the remote.

I'll attach the rawlog, doveconf, etc at the end.  Allow me first to
summarize what I've already checked:

- I used «openssl s_client» to issue EXAMINE on the remote mailbox
  before and after delivering a mail to the remote mailbox, and
  HIGHESTMODSEQ does increase.

- I looked at rawlog for both runs.  doveadm issues a «STATUS foo
  (UIDNEXT UIDVALIDITY)» and receives a correct response (UIDNEXT in the
  second run is one larger than in the first run).
  
- Other than that STATUS command, doveadm only issues LOGIN, LOGOUT,
  LIST, and LSUB commands.  The remote server's responses do not report
  the new HIGHESTMODSEQ value for the folder.

- The message does get mirrored locally when I run without the -s flag.

I'm running dovecot 2.3.9.2 from FreeBSD ports.  I've reviewed the
FreeBSD port¹; it does have some patches, but none of them seem relevant
to this issue.

The remote server runs Cyrus.

Debug output, etc follow:

[[[
% doveconf -n
# 2.3.9.2 (cf2918cac): /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 12.1-RELEASE amd64  
# Hostname: imapmirror.v.shahaf.local2
disable_plaintext_auth = no
first_valid_uid = 1000
imapc_features = " rfc822.size fetch-headers modseq"
imapc_host = imap.fastmail.com
imapc_password = # hidden, use -P to show it
imapc_port = 993
imapc_ssl = imaps
imapc_user = ...
last_valid_uid = 65500
mail_location = maildir:~/Maildir:UTF-8
mail_prefetch_count = 99
namespace {
  inbox = yes
  location = 
  prefix = 
  separator = /
}
passdb {
  driver = pam
}
protocols = imap lmtp
service imap {
  drop_priv_before_exec = yes
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
ssl_cert = <>: Debug: auth-master: userdb lookup(fmsync): Started 
userdb lookup
doveadm(fmsync)<966><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb: Connecting
doveadm(fmsync)<966><>: Debug: auth-master: conn 
unix:/var/run/dovecot/auth-userdb (uid=0): Client connected (fd=10)
doveadm(fmsync)<966><>: Debug: auth-master: userdb lookup(fmsync): auth USER 
input: fmsync system_groups_user=fmsync uid=1002 gid=1002 home=/home/fmsync
doveadm(fmsync)<966><>: Debug: auth-master: userdb lookup(fmsync): Finished 
userdb lookup (username=fmsync system_groups_user=fmsync uid=1002 gid=1002 
home=/home/fmsync)
doveadm(fmsync): Debug: Effective uid=1002, gid=1002, home=/home/fmsync
doveadm(fmsync): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, 
hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir:UTF-8
doveadm(fmsync): Debug: maildir++: root=/home/fmsync/Maildir, index=, 
indexpvt=, control=, inbox=/home/fmsync/Maildir, alt=
doveadm(fmsync): Debug: brain M: Imported mailbox states:
doveadm(fmsync): Debug: brain M: Mailbox $guid1 state: uidvalidity=1529153257 
uid=24288 modseq=4910711 pvt_modseq=0 messages=24117 changes_during_sync=0
doveadm(fmsync): Debug: brain M: Namespace  has location maildir:~/Maildir:UTF-8
doveadm(fmsync): Debug: Namespace : Using permissions from 
/home/fmsync/Maildir: mode=0700 gid=default
dsync(fmsync): Debug: Effective uid=1002, gid=1002, home=/home/fmsync
dsync(fmsync): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, 
hidden=no, list=yes, subscriptions=yes location=imapc:/home/fmsync/imapcdir
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Created new connection
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Looking up IP address 
(reconnect_ok=true, last_connect=1580957435)
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Connecting to 
66.111.4.135:993
dsync(fmsync): Info: imapc(imap.fastmail.com:993): Connected to 
66.111.4.135:993 (local 192.168.122.253:59074)
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Starting SSL handshake
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): SSL handshake successful
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Server capabilities: IMAP4 
IMAP4rev1 LITERAL+ ENABLE UIDPLUS SASL-IR NAMESPACE CONDSTORE SORT 
LIST-EXTENDED QRESYNC MOVE SPECIAL-USE CREATE-SPECIAL-USE IDLE AUTH=PLAIN
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Authenticating as ...
dsync(fmsync): Debug: imapc(imap.fastmail.com:993): Server capabilities: 
IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE 
UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE 
CONDSTORE ESEARCH SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID 
THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATE-EXPERIMENT-1 
METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC 
SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION 
STATUS=SIZE OBJECTID SAVEDATE 

Re: Dovecot - Upgrade Solr 7.7.2 to 8.4.1

2020-02-05 Thread Gedalya
On 2/5/20 5:55 PM, Francis Augusto Medeiros-Logeay wrote:
> I want to install fts-solr, but must tutorials are mentioning solr 7.7.0. Any 
> heads-up on what one must pay attention to when installing 8.4.0? Do I need 
> to update the version on the schemas, for example? 

I followed the instructions and used the schema for 7.7.0, for a new install of 
8.4.0 and everything went fine.



re: Dovecot - Upgrade Solr 7.7.2 to 8.4.1

2020-02-05 Thread Francis Augusto Medeiros-Logeay



On 1/23/20 7:03 AM, Domenico Pastore wrote:

So, with Dovecot is it possible to use Apache Solr 8.4?
High RAM usage is the only problem?



I'm using 8.4.0 and it works flawlessly.


I want to install fts-solr, but must tutorials are mentioning solr 
7.7.0. Any heads-up on what one must pay attention to when installing 
8.4.0? Do I need to update the version on the schemas, for example?


Best,

Francis

0xEE41D33F.asc
Description: application/pgp-keys


Re: Strategy for fts

2020-02-05 Thread Francis Augusto Medeiros-Logeay



---
Francis Augusto Medeiros-Logeay
Oslo, Norway

On 04.02.2020 22:55, Peter Chiochetti wrote:

Am 04.02.20 um 12:37 schrieb Peter Chiochetti:

Am 04.02.20 um 11:46 schrieb Francis Augusto Medeiros-Logeay:

Hi Philon,

Thanks a lot for your thoughts!

Can I ask you if using Solr improved things for you? I have a mailbox 
with 15 years of e-mail and searching things take a long time.


Here, SOLR itself searches a quarter million mails in split seconds 
and returns very good results. That is on a low memory average 
machine.




How much memory are you using, if I may ask? I have a really small 
server only with only 2GB. I am thinking about migrating it, but haven't 
done it so far, most likely to a 16GB instance.


Best,

Francis

0xEE41D33F.asc
Description: application/pgp-keys


Re: Strategy for fts

2020-02-05 Thread Francis Augusto Medeiros-Logeay


---
Francis Augusto Medeiros-Logeay
Oslo, Norway

On 04.02.2020 22:55, Peter Chiochetti wrote:

Am 04.02.20 um 12:37 schrieb Peter Chiochetti:

Am 04.02.20 um 11:46 schrieb Francis Augusto Medeiros-Logeay:

Hi Philon,

Thanks a lot for your thoughts!

Can I ask you if using Solr improved things for you? I have a mailbox 
with 15 years of e-mail and searching things take a long time.


Here, SOLR itself searches a quarter million mails in split seconds 
and returns very good results. That is on a low memory average 
machine.




How much memory are you using, if I may ask? I have a really small 
server only with only 2GB. I am thinking about migrating it, but haven't 
done it so far, most likely to a 16GB instance.


Best,

Francis

0xEE41D33F.asc
Description: application/pgp-keys


0xEE41D33F.asc
Description: application/pgp-keys


Re: auth-client via SSL?

2020-02-05 Thread Heiko Schlittermann
Hi Aki,

thank you for answering.

Aki Tuomi  (Mi 05 Feb 2020 07:59:55 CET):
> > does dovecot support tls-on-connect for AF INET based auth-client
> > sockets?
> > inet_listener auth-client {
> > name = exim
> > port = 4711
> > ssl = yes
> > }
> > }

Interestingly enough, dovecot didn't complain about an unknown option or
about ssl not being supported here.

> This is not (yet) implemented. You can probably workaround with haproxy
> / stunnel for now.

So I did, yes. I used to use socat.

--
Heiko


signature.asc
Description: PGP signature