mailbox locking
Dear Colleagues, I use exim's appendfile transport, procmail and a local mutt on my system, they all (to my knowledge) use lockfiles when working with mboxes. However, `doveconf | grep lock` says dotlock_use_excl = yes lock_method = fcntl mail_max_lock_timeout = 0 mbox_dotlock_change_timeout = 2 mins mbox_lock_timeout = 5 mins mbox_read_locks = fcntl mbox_write_locks = dotlock fcntl pop3_lock_session = no Do I need to change anything in dovecot's default locking setup? Does it still use lockfiles on mboxes or not? There are some contradictory parameters IMHO. -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/
Re: Solr
Joan, I understand and sympathize with your frustration - trying to get multiple applications to work together, particularly given the lack of documentation for some of them, can be extremely challenging. That said, I suggest you consider an alternative viewpoint. Frequently being misunderstood myself I apologize in advance if I'm reading you wrong - but it appears your view towards the situation is there is a bug in Dovecot related to this problem. That may well be - but I generally approach these matters from the assumption that *I* made the error in configuration and go from there. I'm not an official rep for any product nor claim to be any form of expert in these matters - but I do have a working setup and I'd like to help you if I can. If you're willing to - take a deep breath and let's try starting over. Looking back through your emails there were two items that stood out - your Dovecot config has two settings I don't use: "fts_decoder" and "fts_enforced". I also asked you earlier whether or not NFS is involved here and I didn't see an answer - please clarify. I suggest you try once more: delete Solr completely. Re-install per the directions and use *my* managed-schema. Also comment out the Dovecot directives for "fts_decoder" and "fts_enforced" so you're closer to my setup. Try running again and then post back - I'll do what I can. Based on the fact that Dovecot+Solr 7.5+my schema is working for me leads me to believe we can get it working for you as well. Daniel On 12/15/2018 2:42 PM, Joan Moreau wrote: here my latest schema.xml (remove the "long" type hich seems to be very deprecated in 7.x) id positionIncrementGap="0" /> autoGeneratePhraseQueries="true" positionIncrementGap="100"> ignoreCase="true"/> generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" splitOnNumerics="1" catenateWords="1" catenateNumbers="1" catenateAll="1"/> maxGramSize="15" /> protected="protwords.txt"/> ignoreCase="true" synonyms="synonyms.txt"/> ignoreCase="true"/> generateWordParts="1" generateNumberParts="1" splitOnCaseChange="1" splitOnNumerics="1" catenateWords="1" catenateNumbers="1" catenateAll="1"/> maxGramSize="15" /> protected="protwords.txt"/> stored="true"/> stored="true"/> stored="true"/> stored="true"/> On 2018-12-15 20:54, Joan Moreau wrote: Daniel, I have done that so any times (deleteing the data folders, recreating the instance, restarting etc...) But this is really not the issue The issue is 1 - fts_solr reports errors in the log file (this is a pure dovecot issue) : how to have much more details on what fts_solr sends to Slor server and what does it returns ? 2 - Solr returns properly for a few hours, then starts crashing or responding non-sense after some time Additionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Moreau via dovecot wrote: I shared the errors already so many times (check this mailinling for "solr" in teh title) Contrary to what you say, with SOlr 7.5 and Dovecot git, I had to remove the "managed-schema" to make solr respond a bit properly. It relies on schema.xml In order to create the instance, no, it copies the default config in the dovecot instance. I'm not a Solr expert by any means but I believe you are incorrect. As of Solr 5.x the managed-schema file is the primary method for configuration. The method I detailed previously for setting up a config helps automate creating new Solr instances - but as I stated you can either setup a Solr template and then create the instance from that or create an instance using the default template and then adjust it. The part that you *must* do after creating from the default template is stop the server, delete the entire "/solr/dovecot/data" folder, then install the correct managed-schema file, then restart the server. The server will not function with mismatched schema/data. If you'll try that - explicitly "rm -rf /solr/dovecot/data", copy the managed-schema file into the conf folder, and restart - things will either work or there's something else that needs correction. -- Daniel
Re: Solr
here my latest schema.xml (remove the "long" type hich seems to be very deprecated in 7.x) id On 2018-12-15 20:54, Joan Moreau wrote: Daniel, I have done that so any times (deleteing the data folders, recreating the instance, restarting etc...) But this is really not the issue The issue is 1 - fts_solr reports errors in the log file (this is a pure dovecot issue) : how to have much more details on what fts_solr sends to Slor server and what does it returns ? 2 - Solr returns properly for a few hours, then starts crashing or responding non-sense after some time Additionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Moreau via dovecot wrote: I shared the errors already so many times (check this mailinling for "solr" in teh title) Contrary to what you say, with SOlr 7.5 and Dovecot git, I had to remove the "managed-schema" to make solr respond a bit properly. It relies on schema.xml In order to create the instance, no, it copies the default config in the dovecot instance. I'm not a Solr expert by any means but I believe you are incorrect. As of Solr 5.x the managed-schema file is the primary method for configuration. The method I detailed previously for setting up a config helps automate creating new Solr instances - but as I stated you can either setup a Solr template and then create the instance from that or create an instance using the default template and then adjust it. The part that you *must* do after creating from the default template is stop the server, delete the entire "/solr/dovecot/data" folder, then install the correct managed-schema file, then restart the server. The server will not function with mismatched schema/data. If you'll try that - explicitly "rm -rf /solr/dovecot/data", copy the managed-schema file into the conf folder, and restart - things will either work or there's something else that needs correction. -- Daniel
[Fwd: Re: Upgrade to 2.3.1 has failed]
Alexander hi. Aki caught the STARTTLS issue as well, I corrected it, but it still doesn't work. Enjoy your weekend. I intend to enjoy mine! :-) Thanks again for your time. Andy Forwarded Message From: C. Andrews Lavarre To: Aki Tuomi Subject: Re: Upgrade to 2.3.1 has failed Date: Sat, 15 Dec 2018 15:08:58 -0500 Aki thank you again. If you and Alexander are stumped then surely I am too! I swear I didn't change anything, and indeed have tried going back to the backup of 10- ssl.conf, which worked under 2.2, but doesn't under 2.3 even after making the changes described in the upgrade documentation. All I did was change all the repositories to Leap 15.0 from Leap 42.3 and execute zypper dup. It took several hours to complete at which point everything works just fine, except that Dovecot was upgraded from 2.2.xxx? to 2.3.1 without my even agreeing to it... :-( This version 2.3.1 is the openSUSE repository offering for their Leap 15.0. I tried finding a rollback version yesterday—2.2.3, 2.2.9... I don't need all the bells and whistles, I just want it to work—but all had one kind of dependency hell or another... :-( What I've done in the meantime is to mount /home/alavarre/Maildir with sshfs, and then point KMail at it, so I can read and write email without dovecot, but it would be nice to fix it IDC... So maybe the right answer is to try the latest, perhaps in Tumbleweed... I'm usually allergic to self-compiling, I alway seem to find one dependency hell or another, but I'll go ahead and try anyhow. I'll let you know. In the meantime all the failed logins have put me in jail by the provider (Cox Cable) accusing me of being a spammer... :-( But for now I'll go have a gin and tonic and hit it again tomorrow... :-) Enjoy your weekend, and thank you again for your thoughts and time. Cheers, Andy On Sat, 2018-12-15 at 21:37 +0200, Aki Tuomi wrote: > There is still something wrong with your config. Btw if you are > compiling yourself you might want to use 2.3.4 > > We test the cert functionality in our ci tests so I am fairly > confident this is not a dovecot bug. > > Aki > >
Re: Upgrade to 2.3.1 has failed
Alexander, Thanks, as described before, if I include the "<" then Dovecot fails to start at all. Thank you again for your time. I have forwarded my latest to Aki to the group. Enjoy your weekend. Best regards, Andy On Sat, 2018-12-15 at 23:08 +0100, Alexander Dalloz wrote: > Am 15.12.2018 um 19:43 schrieb Aki Tuomi: > > > > > > > > I've posted te full output from dovecot -n to https://pastebin.co > > > m/F8Ra > > > C4bt > You again broke your setup. From your pastebin: > > ssl_cert = /etc/certbot/live/privustech.com/fullchain.pem > > That's missing the "<" in front of the path to the certificate file. > Proably the same mistake for the ssl_key parameter. > > Alexander >
Re: Upgrade to 2.3.1 has failed
Am 15.12.2018 um 19:43 schrieb Aki Tuomi: I've posted te full output from dovecot -n to https://pastebin.com/F8Ra C4bt You again broke your setup. From your pastebin: ssl_cert = /etc/certbot/live/privustech.com/fullchain.pem That's missing the "<" in front of the path to the certificate file. Proably the same mistake for the ssl_key parameter. Alexander
Re: Upgrade to 2.3.1 has failed
Am 15.12.2018 um 19:02 schrieb C. Andrews Lavarre: The openssl command I have tried (that used to work with Dovecot 2.2) is: openssl s_client -connect mail.privustech.com:143 I have also tried openssl s_client -connect mail.privustech.com:143 -servername mail.privustech.com Please, there is zero need to mail me personlly. Keep your replies to the list. I am following here as you can see. And to your command: it is wrong. As I guessed you are talking with SSL to the IMAP STARTTLS port. That of course cannot work. SSL here means directly doing a secure handshaking, just like HTTPS is working. The default port for IMAPS is 993, not 143. If you test against IMAP/STARTTLS on port 143, then do with openssl s_client -connect mail.privustech.com:143 -starttls imap As your "doveconf -n" does not show any special setup regarding IMAPS or IMAP/STARTTLS the case is as analyzed. Alexander
Re: Solr
Daniel, I have done that so any times (deleteing the data folders, recreating the instance, restarting etc...) But this is really not the issue The issue is 1 - fts_solr reports errors in the log file (this is a pure dovecot issue) : how to have much more details on what fts_solr sends to Slor server and what does it returns ? 2 - Solr returns properly for a few hours, then starts crashing or responding non-sense after some time Additionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Moreau via dovecot wrote: I shared the errors already so many times (check this mailinling for "solr" in teh title) Contrary to what you say, with SOlr 7.5 and Dovecot git, I had to remove the "managed-schema" to make solr respond a bit properly. It relies on schema.xml In order to create the instance, no, it copies the default config in the dovecot instance. I'm not a Solr expert by any means but I believe you are incorrect. As of Solr 5.x the managed-schema file is the primary method for configuration. The method I detailed previously for setting up a config helps automate creating new Solr instances - but as I stated you can either setup a Solr template and then create the instance from that or create an instance using the default template and then adjust it. The part that you *must* do after creating from the default template is stop the server, delete the entire "/solr/dovecot/data" folder, then install the correct managed-schema file, then restart the server. The server will not function with mismatched schema/data. If you'll try that - explicitly "rm -rf /solr/dovecot/data", copy the managed-schema file into the conf folder, and restart - things will either work or there's something else that needs correction. -- Daniel
Re: Upgrade to 2.3.1 has failed
That command is missing -starttls imap? or are you using port 993? On 15 December 2018 at 20:02 "C. Andrews Lavarre" < alava...@gmail.com> wrote: Excellent, thank you again. The openssl command I have tried (that used to work with Dovecot 2.2) is: openssl s_client -connect mail.privustech.com:143 I have also tried openssl s_client -connect mail.privustech.com:143 -servername mail.privustech.com I've posted the full output from this to https://pastebin.com/eUSarQdx I've posted te full output from dovecot -n to https://pastebin.com/F8Ra C4bt Thank you again, Andy On Sat, 2018-12-15 at 17:27 +0100, Alexander Dalloz wrote: Am 15.12.2018 um 17:16 schrieb C. Andrews Lavarre: to /etc/apparmor.d/local/usr.lib.dovecot.imap-login but still cannot login with either the mail client or with explicit openssl: it complains error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794: Hi, that error above typically means to connect with SSL to STARTTLS or vice versa. Please provide the complete command you issue using "openssl s_client" together with the corresponding dovecot logging. As well the output of "doveconf -n" would be useful to help you further. Alexander --- Aki Tuomi
special_use folders still created
dovecot-2.2.36-3.el7.x86_64
The below configuration seems to work for Thunderbird and emclient, when
moving a message to Junk. Emclient uses "Junk E-mail" and Thunderbird
uses "Junk". On the server is only the file .Junk created as expected.
When I move a message to Archive in Thunderbird. On the server the files
.Archives and .Archives.2018 are created (Thunderbird creates a
subfolder in Archives)
How is this possible? On the server there is already an .Archive folder
so it should create .Archive.2018 not?
imap_capability = +SPECIAL-USE
# drafts folders merging
mailbox Drafts {
special_use = \Drafts
auto = create
}
# archive folders merging
mailbox Archive {
special_use = \Archive
auto = create
}
mailbox Archives {
special_use = \Archive
auto = no
}
# spam folders merging
mailbox Junk {
special_use = \Junk
auto = create
}
mailbox Spam {
special_use = \Junk
auto = no
}
mailbox "Junk E-mail" {
special_use = \Junk
auto = no
}
Re: Upgrade to 2.3.1 has failed
Excellent, thank you again. The openssl command I have tried (that used to work with Dovecot 2.2) is: openssl s_client -connect mail.privustech.com:143 I have also tried openssl s_client -connect mail.privustech.com:143 -servername mail.privustech.com I've posted the full output from this to https://pastebin.com/eUSarQdx I've posted te full output from dovecot -n to https://pastebin.com/F8Ra C4bt Thank you again, Andy On Sat, 2018-12-15 at 17:27 +0100, Alexander Dalloz wrote: > Am 15.12.2018 um 17:16 schrieb C. Andrews Lavarre: > > > > to /etc/apparmor.d/local/usr.lib.dovecot.imap-login but > > still > > cannot login with either the mail client or with explicit openssl: > > it > > complains > > error:140770FC:SSL > > routines:SSL23_GET_SERVER_HELLO:unknown > > protocol:s23_clnt.c:794: > Hi, > > that error above typically means to connect with SSL to STARTTLS or > vice > versa. > > Please provide the complete command you issue using "openssl > s_client" > together with the corresponding dovecot logging. As well the output > of > "doveconf -n" would be useful to help you further. > > Alexander >
Multiple SSL certs in a virtual Domain hosting environment
I am trying to get this correct.
configuration
# 2.0.0: dovecot.conf
auth_cache_negative_ttl = 3600 s
base_dir = /var/run/dovecot/
disable_plaintext_auth = no
first_valid_uid = 100
info_log_path = /var/log/dovecot-info.log
log_path = /var/log/dovecot.log
listen = *
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c
mail_debug=yes
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_log_prefix = %Us(%u):
mdbox_rotate_size = 2048
passdb {
args = /etc/master.passwd
driver = passwd-file
}
protocols = imap pop3 lmtp
service auth {
executable = /usr/dovecot2/libexec/dovecot/auth
user = root
}
service imap-login {
chroot = login
client_limit = 256
inet_listener imap {
address = 204.209.81.1, 127.0.0.1
port = 143
}
inet_listener imaps {
address = 204.209.81.1, 127.0.0.1
port = 993
ssl = yes
}
executable = /usr/dovecot2/libexec/dovecot/imap-login
process_limit = 128
process_min_avail = 3
service_count = 1
user = dovecot
##vsz_limit = 1M
}
service imap {
executable = /usr/dovecot2/libexec/dovecot/imap
process_limit = 512
##vsz_limit = 256
}
ssl = yes
ssl_cert = https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism
Merry Christmas 2018 and Happy New Year 2019!!
Re: Upgrade to 2.3.1 has failed
Am 15.12.2018 um 17:16 schrieb C. Andrews Lavarre: to /etc/apparmor.d/local/usr.lib.dovecot.imap-login but still cannot login with either the mail client or with explicit openssl: it complains error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794: Hi, that error above typically means to connect with SSL to STARTTLS or vice versa. Please provide the complete command you issue using "openssl s_client" together with the corresponding dovecot logging. As well the output of "doveconf -n" would be useful to help you further. Alexander
Re: Upgrade to 2.3.1 has failed
Alexander good afternoon. Thank you. I have spent the day learning about AppArmor: • I've reviewed your link, found /etc/apparmor.d/ and its local/ directory. • I ran aa-logprof and it found the change in stat to old-stat that is discussed in the upgrade documentation. So I Allow (A) that. There are no other reports. • I followed the discussion on using yast to manage the profiles. I'm on ssh to the server so do not have the GUI yast, only the ncurses version and it does not contain editing, only adding, profiles. I tried creating a profile for imap-login with that method and scanned for any issues, there were none reported, but still cannot log in. • I followed the local/README to explicitly add /etc/certbot/live/privustech.com/* r, to /etc/apparmor.d/local/usr.lib.dovecot.imap-login but still cannot login with either the mail client or with explicit openssl: it complains error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:794: I check yast2 sw_single for the dovecot installation. Indeed the module dovecot23-xxx where xxx is anything that looks like "clnt" ( client?) does not exist. Is there a missing module in my installation? It lists only dovecot dovecot23 dovecot23-backend-mysql dovecot23-backend-pgsql dovecot23-backend-sqlite dovecot23-fts dovecot23-fts-squat I'll pursue this further. Thank you again. Kind regards, Andy On Fri, 2018-12-14 at 23:44 +0100, Alexander Dalloz wrote: > Am 14.12.2018 um 19:58 schrieb C. Andrews Lavarre: > > > > Thanks for the input. I've checked out your suggestions (details > > below) > > but unfortunately no joy. > > I also restored my backup 10-ssl.conf. It indeed has the "<" sign > > with > > a space before the explicit paths to the files: > > ssl_cert = > ssl_key = Hi, > > the syntax you see in the documentation is mandatory. Your issue is > really a permissions problem. > > Check your AppArmor setup. The path you use for storing the chained > certificate and the private key is certainly not known to AppArmor. > See > your /var/log/audit/audit.log for indications. > > https://doc.opensuse.org/documentation/leap/security/html/book.securi > ty/cha.apparmor.managing.html > > may help. > > Btw. permissions setting to 0777, especially for the cert and key, > is > awful, even for debugging issues. > > Alexander >
Re: Overrideing pop delete?
On 14 Dec 2018, at 17:09, Robert L Mathews wrote:
> https://wiki.dovecot.org/Plugins/Lazyexpunge
I have a question about the namespace section.
> You create only a single namespace. When a message is expunged from mailbox
> , it's moved to a mailbox in the expunge namespace. When an
> entire mailbox is deleted, it's also moved to this namespace as
> . If it already exists, their contents are merged.
>
> Example configuration:
>
> # the default namespace
>
> namespace {
> prefix =
> separator = /
> inbox = yes
> }
>
>
> # namespace for lazy_expunge plugin:
> namespace {
> prefix = .EXPUNGED/
> hidden = yes
> list = no
> separator = /
> location = maildir:~/Maildir/expunged
> }
>
> mail_plugins = $mail_plugins lazy_expunge
> plugin {
> lazy_expunge = .EXPUNGED/
>
> }
First all, that shows two namespace sections. Second, my namespace looks quite
different (maybe it’s from older config version, I am not on 2.3 yet). I have a
few users with shell access and mail in ~/Maildir but most users are in mysql.
user_query = select 89 as uid, 89 as gid, concat('/usr/local/virtual/',
maildir) as home FROM mailbox where username = '%u'
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
mailbox Archive {
auto = subscribe
special_use = \Archive
}
prefix =
}
Am I just adding the new namespace for lazy_expunge inside that? And since the
readme also says to put lazy_expunge and all in the mailplugins line, why is
this repeated here? Won’t this override the previous setting?
And, finally, is there any way to limit this to only POP3 delete instead of all
IMAP?
--
"Give a man a fire and he's warm for a day, but set fire to him an he's
warm for the rest of his life.”
Re: Overrideing pop delete?
On 14 Dec 2018, at 17:09, Robert L Mathews wrote: > On 12/14/18 3:34 PM, @lbutlr wrote: > >> Now that I think about it, even better would be a way to move the messages >> into an archive box when they are downloaded, this way they will be entirely >> invisible from the POP3 access, and I can use normal expiry functions to >> clean out that archive after backup. > > We do exactly this using the "Lazy Expunge" plugin: > > https://wiki.dovecot.org/Plugins/Lazyexpunge > > Despite the IMAP-sounding "expunge" in the name, it works for all > deletions, including POP3. Thank you! -- Let the Wookiee win.
Re: Overrideing pop delete?
> On 15 Dec 2018, at 03:11, Jochen Bern wrote: > > On 12/15/2018 12:34 AM, @lbutlr wrote: >> On 14 Dec 2018, at 16:30, @lbutlr wrote: >>> Is it possible to override the POP3 delete on download command and make >>> sure that messages stay on the server for at least X hours or X days? >>> It is important that the messages be around long enough to hit a snapshot >>> cycle (using rsnapshot to backup ever hour). >> >> Now that I think about it, even better would be a way to move the messages >> into an archive box when they are downloaded, this way they will be entirely >> invisible from the POP3 access, and I can use normal expiry functions to >> clean out that archive after backup. > > From a data flow (and privacy protection) POV, that wouldn't be much > different anymore from having *the MTA* feed a copy of (all incoming) > e-mails directly into an archiving mechanism, would it? No, but it is only needed for the times someone access the account via POP3 without “leave on server". -- Oh look, good intentions!
Re: Re: Overrideing pop delete?
On 12/15/2018 12:34 AM, @lbutlr wrote: > On 14 Dec 2018, at 16:30, @lbutlr wrote: >> Is it possible to override the POP3 delete on download command and make >> sure that messages stay on the server for at least X hours or X days? >> It is important that the messages be around long enough to hit a snapshot >> cycle (using rsnapshot to backup ever hour). > > Now that I think about it, even better would be a way to move the messages > into an archive box when they are downloaded, this way they will be entirely > invisible from the POP3 access, and I can use normal expiry functions to > clean out that archive after backup. From a data flow (and privacy protection) POV, that wouldn't be much different anymore from having *the MTA* feed a copy of (all incoming) e-mails directly into an archiving mechanism, would it? http://www.postfix.org/postconf.5.html#always_bcc Regards, -- Jochen Bern Systemingenieur www.binect.de www.facebook.de/binect smime.p7s Description: S/MIME Cryptographic Signature
