RE: The future of SIS
On Mon, 16 Oct 2023, Marc wrote: > Is this feature really useful? I can imagine if you are twitter or ig and > everyone is posting the same video this could be usefull. Are there any stats > on this available, so you know what to expect implementing deduplication. In an office where people insist on mailing documents to everyone, and using email as a document storage system, yes, it is very useful. -- --- ==== Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Using / in folder names
Maybe this is over simplifying things but can you just open the pst file in outlook and then rename the folder, then import? Sent from my iPhone > On Aug 3, 2023, at 9:03 PM, Scott wrote: > > Is this possible at all ? > > I am trying to import a PST file and it has a slash in the folder name. > Dovecot errors out with : > > APPEND "Caixa de Entrada_VIDA/S1" (\Seen) > NO [CANNOT] Invalid mailbox name: Name must not have '/' characters > > My namespace separator is the dot and I configured the listescape plugin to > escape / but it still doesn't work. > > The listescape documentation says: > > The / character is disallowed on POSIX systems. > > This plugin allows you to use all of these characters, as long as the > virtual separator (i.e. what is set by the separator setting and used as > such by the IMAP protocol) is changed to something else, which means that > the plugin does not make it possible to use the virtual separator in folder > names. > > > So it seems that as long as I use the dot, it should work ? Except it > doesn't... > > How can I do this import ? > > Thank you! > > ___ > dovecot mailing list -- dovecot@dovecot.org > To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: IMAP account can't save any email with attachment
omain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Added userdb setting: plugin/quota_rule=*:bytes=0 > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Effective uid=8, gid=8, home=/mail/mydomain.com/myuser > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, > list=yes, subscriptions=yes location=maildir:/mail/mydomain.com/myuser > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > maildir++: root=/mail/mydomain.com/myuser, index=, indexpvt=, control=, > inbox=/mail/mydomain.com/myuser, alt= > Jul 20 15:06:21 imap(myu...@mydomain.com)<2181><7W6sfewAb8VfWumz>: Debug: > Mailbox Drafts: Mailbox opened because: SELECT In the thunderbird client, I wait for ages before a popup appears saying "Your draft message was not copied to your drafts folder (Drafts) due to network or file access errors. You can retry or save the draft locally to Local Folders" I've tried searching around for information on what the problem could be, but I've not found anything that would explain this problem. Have any ideas? Chris On Thu, Jul 20, 2023 at 3:20 PM William Edwards wrote: > > > Op 20 jul. 2023 om 14:26 heeft Chris Thomas > het volgende geschreven: > > > > > > Hi, > > > > I'm getting a curious problem where if I write a draft without an > attachment and click save. It'll work without any issue at all. > > > > But if I do the same, then attach a file to the email, it'll sit there > for a couple of minutes before timing out (I'm using thunderbird), it'll > eventually give you a message saying > > > > "Your draft message was not copied to your drafts folder (Drafts) due to > network or file access errors." > > > > I've got all of dovecots verbose logging turned on. > > Cool! So … where is it? > > > I'm using dovecot as a submission server through to the postfix server > to do the actual sending. All the logging is turned on there too. But I > can't figure out what the problem is. > > > > Is there anything I can look for in the logs that will help me out? > > > > chris > > ___ > > dovecot mailing list -- dovecot@dovecot.org > > To unsubscribe send an email to dovecot-le...@dovecot.org > > ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: IMAP account can't save any email with attachment
Here is the info from dovecot -n
dovecot.mail-server and postfix.mail-server are valid dns entries for
themselves. It's running on a kubernetes cluster so those hostnames are
provided by the namespace and pod name, they work too, you can ping them
and it works for everything except emails with attachments, for some reason
# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.4 ()
# OS: Linux 4.9.0-9-amd64 x86_64 Debian 10.13 ext4
# Hostname: dovecot.mail-server.svc.cluster.local
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = yes
disable_plaintext_auth = no
first_valid_gid = 8
first_valid_uid = 8
haproxy_timeout = 5 secs
haproxy_trusted_networks = 10.0.0.0/8
hostname = s3.mydomain.com
log_path = /dev/stderr
mail_access_groups = mail
mail_debug = yes
mail_gid = mail
mail_home = /mail/%d/%n
mail_location = maildir:/mail/%d/%n
mail_plugins = " zlib"
mail_privileged_group = mail
mail_uid = mail
maildir_stat_dirs = yes
namespace inbox {
inbox = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
postmaster_address = i...@mydomain.com
protocols = " imap lmtp pop3 submission"
service auth-worker {
unix_listener auth-worker {
group = mail
mode = 0660
user = $default_internal_user
}
user = mail
}
service auth {
user = $default_internal_user
}
service dict {
unix_listener dict {
group = mail
mode = 0660
}
}
service imap-login {
inet_listener imap {
haproxy = yes
port = 143
}
inet_listener imaps {
haproxy = yes
port = 993
ssl = yes
}
}
service lmtp {
inet_listener lmtp {
haproxy = no
port = 24
}
}
service pop3-login {
inet_listener pop3 {
haproxy = yes
port = 110
}
inet_listener pop3s {
haproxy = yes
port = 995
ssl = yes
}
}
service submission-login {
inet_listener submission {
haproxy = yes
port = 587
}
}
ssl_cert =
wrote:
>
> > Op 20 jul. 2023 om 14:26 heeft Chris Thomas
> het volgende geschreven:
> >
> >
> > Hi,
> >
> > I'm getting a curious problem where if I write a draft without an
> attachment and click save. It'll work without any issue at all.
> >
> > But if I do the same, then attach a file to the email, it'll sit there
> for a couple of minutes before timing out (I'm using thunderbird), it'll
> eventually give you a message saying
> >
> > "Your draft message was not copied to your drafts folder (Drafts) due to
> network or file access errors."
> >
> > I've got all of dovecots verbose logging turned on.
>
> Cool! So … where is it?
>
> > I'm using dovecot as a submission server through to the postfix server
> to do the actual sending. All the logging is turned on there too. But I
> can't figure out what the problem is.
> >
> > Is there anything I can look for in the logs that will help me out?
> >
> > chris
> > ___
> > dovecot mailing list -- dovecot@dovecot.org
> > To unsubscribe send an email to dovecot-le...@dovecot.org
>
>
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
IMAP account can't save any email with attachment
Hi, I'm getting a curious problem where if I write a draft without an attachment and click save. It'll work without any issue at all. But if I do the same, then attach a file to the email, it'll sit there for a couple of minutes before timing out (I'm using thunderbird), it'll eventually give you a message saying "Your draft message was not copied to your drafts folder (Drafts) due to network or file access errors." I've got all of dovecots verbose logging turned on. I'm using dovecot as a submission server through to the postfix server to do the actual sending. All the logging is turned on there too. But I can't figure out what the problem is. Is there anything I can look for in the logs that will help me out? chris ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Incorrect saved dates in mailboxes
On 2023-05-17 18:28, Joseph Tam wrote: Chris Szilagyi writes: Recently, I noticed that our expunge script is not working, and I don't think it ever has on this server. On further inspection, it looks like the saved date for emails in the folders we want to expunge is set to a recent date (yesterday) for almost all messages. For example, when I run the command "doveadm -f tab fetch -u username date.saved mailbox Trash", the date for almost all of the messages is yesterday at the exact same time. I tried this on other users and they have this exact same timestamp shown for most of their messages, too. I tried checking for "date.received" and that shows correct, it is "date.saved" that is not working. Any idea of how or why this would not be showing the correct date? I've used this setup in the past on other servers with dovecot and it has worked great, no idea why we would be seeing this issue now. I recall having the same problem. I think the "date.saved" is not instantiated in the cache until you query for it. So the value you were shown is when you last dumped its value if it wasn't previously set -- your run of same values coincided when you ran "doveadm fetch". My expunge script just uses date.received instead -- it seems to work. Joseph Tam OK, that does make sense so far. Today I have checked, and the correct dates seem to be showing up now after the initial date (ever since that initial run of the "dovecot expunge" command was run on all mailboxes). So in our case I have the expunge set to delete older than 30 days. I assume if I wait 30 days from now, it will start working? Or did your issue come back repeatedly? I too thought about using date.received, but in our case date.saved is more ideal. Thank you for your reply. ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Incorrect saved dates in mailboxes
Hello: We have a server (Debian 11) using Dovecot 2.3 which has been in place for about a year. We are using mbox format for our mailboxes, and do not use MySQL, just plain user directories under /home. Recently, I noticed that our expunge script is not working, and I don't think it ever has on this server. On further inspection, it looks like the saved date for emails in the folders we want to expunge is set to a recent date (yesterday) for almost all messages. For example, when I run the command "doveadm -f tab fetch -u username date.saved mailbox Trash", the date for almost all of the messages is yesterday at the exact same time. I tried this on other users and they have this exact same timestamp shown for most of their messages, too. I tried checking for "date.received" and that shows correct, it is "date.saved" that is not working. Any idea of how or why this would not be showing the correct date? I've used this setup in the past on other servers with dovecot and it has worked great, no idea why we would be seeing this issue now. Thanks in advance. Chris ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Cannot reach documentation site
On Sun, 26 Feb 2023, Ken Bass wrote: > doc.dovecot.org does not have a DNS entry. It worked a few days ago, but not > now. [chris@newpop ~]$ host doc.dovecot.org doc.dovecot.org is an alias for talvi.dovecot.org. talvi.dovecot.org has address 94.237.105.223 talvi.dovecot.org has IPv6 address 2a04:3545:1000:720:acc1:5bff:fe5e:459 -- --- ==== Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Re: doveadm sending invalid AUTHENTICATE to uw-imap
For anyone searching in the future: The eventual cause turns out to be, as far as I can tell, something screwy in uw-imap or the base64 decode function it uses on this old Solaris server. It seems to be something odd with the number of characters in the hash. tl;dr I'm going to do my migration with 2 different master users with different length passwords, which will cover all cases and allow me to migrate all the users. Problem solved for this case. Now the long version for the archive: To diagnose the problem I eventually wrote a short perl program using IO::Socket::SSL that prints the UW-Imap banner and the '+' second prompt, logs what it receives back, and exits. This let me see that in all cases, doveadm sends the single line AUTHENTICATE command, so that wasn't the problem. It had never occurred to me to reverse the hash and see what was being sent. I had been testing from telnet / "openssl s_client" using a string "\0user@masteruser\0masterpass" . Doveadm is sending "user\0masteruser\0masterpass" . Different input, different results. Somewhere in my testing, I forgot to add the -n to echo -e "\0user@masteruser\-masterpass" | base64 (so the input to base64 had a trailing newline) and to my surprise, a user that didn't work before worked ! That mistake is how I figured out the extra character in the password was somehow making a difference. So -- since this is a one time migration, and it's repeatable, I've come up with the users that work with the padding, and the ones that work without, and will run the import each way for each list, turn off the legacy server and be done with this ! Thanks for the pointers, and if anyone else runs into this bizare situation hopefully they find this ! Of course if anyone knows why this happens I'd love to hear it. -Chris On Wed, 8 Feb 2023, Chris Candreva wrote: > > On Wed, 8 Feb 2023, Aki Tuomi wrote: > > > Can you try setting imapc_sasl_mechanisms to login, maybe it works better? > > And Stephan Bosch wrote: > > > Can you make a protocol log (tcp dump of commands sent by client and > > replies sent by server) for one of these sessions? e.g. using ngrep if > > connections aren't secured. > > > I was using imaps initially. Switching to imap over port 143 to do the > tcpdump had the side effect of switching to LOGIN authentication, > evidently uw-imap is sending different capability strings. It still > doesn't work though. Both from the error and the dump I can tell "doveadm" > is sending the user's id only without the "*masteruser" and the > master user password. > > Plain connection banner: > * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS > STARTTLS] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 16:45:22 > -0500 (EST) > > SSL Banner on 993: > * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS > AUTH=PLAIN AUTH=LOGIN] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 > 16:53:36 -0500 (EST) > > > > > > On 08/02/2023 06:24 EET Chris Candreva wrote: > > > > > > > > > I'm migrating a legacy uw-imap system to Dovecot, on a Rocky (RHEL) 8 > > > server running Dovecot 2.3.16-3 from their repos. I am using a master > > > user > > > to import all users for an imaps connection from the old server to the > > > new. On a trial run however, it worked for about half the users. Half are > > > giving an error of the form: > > > > > > dsync(user): Error: imapc(host:993): > > > Command '1 AUTHENTICATE PLAIN ' failed > > > with BAD: > > > 1 Missing or invalid argument to AUTHENTICATE > > > > > > I can't seem to get the IMAP command for the users that did work. > > > However, > > > on the face of it, that is an invalid AUTHENTICATE command. If I take > > > that > > > string and brake it up into (what I've googled is) the proper form of > > > multi-command form of > > > > > > 1 AUTHENTICATE PLAIN > > > + > > > > > > > > > then the login succeeds. I have not been able to find anyone else with > > > this problem in my search. Is this a known issue, is there a way to force > > > the multi-line AUTHENTICATE, something else I'm missing ? Any help is > > > appreciate on this! > > > > > > -Chris > > > > > > > > > > > > -- > > > --- > > > > > > Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris > > > > -- --- Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Re: doveadm sending invalid AUTHENTICATE to uw-imap
On Wed, 8 Feb 2023, Aki Tuomi wrote: > Can you try setting imapc_sasl_mechanisms to login, maybe it works better? And Stephan Bosch wrote: > Can you make a protocol log (tcp dump of commands sent by client and > replies sent by server) for one of these sessions? e.g. using ngrep if > connections aren't secured. I was using imaps initially. Switching to imap over port 143 to do the tcpdump had the side effect of switching to LOGIN authentication, evidently uw-imap is sending different capability strings. It still doesn't work though. Both from the error and the dump I can tell "doveadm" is sending the user's id only without the "*masteruser" and the master user password. Plain connection banner: * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS STARTTLS] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 16:45:22 -0500 (EST) SSL Banner on 993: * OK [CAPABILITY IMAP4REV1 I18NLEVEL=1 LITERAL+ SASL-IR LOGIN-REFERRALS AUTH=PLAIN AUTH=LOGIN] foo.com IMAP4rev1 2007e.404 at Wed, 8 Feb 2023 16:53:36 -0500 (EST) > > On 08/02/2023 06:24 EET Chris Candreva wrote: > > > > > > I'm migrating a legacy uw-imap system to Dovecot, on a Rocky (RHEL) 8 > > server running Dovecot 2.3.16-3 from their repos. I am using a master user > > to import all users for an imaps connection from the old server to the > > new. On a trial run however, it worked for about half the users. Half are > > giving an error of the form: > > > > dsync(user): Error: imapc(host:993): > > Command '1 AUTHENTICATE PLAIN ' failed with > > BAD: > > 1 Missing or invalid argument to AUTHENTICATE > > > > I can't seem to get the IMAP command for the users that did work. However, > > on the face of it, that is an invalid AUTHENTICATE command. If I take that > > string and brake it up into (what I've googled is) the proper form of > > multi-command form of > > > > 1 AUTHENTICATE PLAIN > > + > > > > > > then the login succeeds. I have not been able to find anyone else with > > this problem in my search. Is this a known issue, is there a way to force > > the multi-line AUTHENTICATE, something else I'm missing ? Any help is > > appreciate on this! > > > > -Chris > > > > > > > > -- > > --- > > ==== > > Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris > -- --- Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
doveadm sending invalid AUTHENTICATE to uw-imap
I'm migrating a legacy uw-imap system to Dovecot, on a Rocky (RHEL) 8 server running Dovecot 2.3.16-3 from their repos. I am using a master user to import all users for an imaps connection from the old server to the new. On a trial run however, it worked for about half the users. Half are giving an error of the form: dsync(user): Error: imapc(host:993): Command '1 AUTHENTICATE PLAIN ' failed with BAD: 1 Missing or invalid argument to AUTHENTICATE I can't seem to get the IMAP command for the users that did work. However, on the face of it, that is an invalid AUTHENTICATE command. If I take that string and brake it up into (what I've googled is) the proper form of multi-command form of 1 AUTHENTICATE PLAIN + then the login succeeds. I have not been able to find anyone else with this problem in my search. Is this a known issue, is there a way to force the multi-line AUTHENTICATE, something else I'm missing ? Any help is appreciate on this! -Chris -- --- ==== Chris Candreva -- ch...@westnet.com -- http://www.westnet.com/~chris
Re: ot: how to t/s TBird problems ?
Over the last several months we have seen what seems like large delays in email delivery as well, we get emails at 11AM that are time stamped at 9:10. I thought it was a networking issue, but I can’t be sure. I wish I knew more about coding, to look under the hood to examine things further. Sent from my iPhone > On Oct 23, 2022, at 7:17 AM, Voytek Eymont wrote: > > > >> On Sat, October 22, 2022 11:29 am, Joseph Tam wrote: >> >> I haven't seen anyone else replying, but there doesn't seem anything >> anomalous with the output. The session commands-repliesd is is more or >> less what I expect, although to make sense of this, you'll have to splice >> the input and output files together using timestamps to see the sequential >> flow of data. > ... >> Typically, if some resource limit is hit, one side or the other will >> create a log or notification. Your INBOX is large, but not outrageous. You >> can test it directly by creating smaller subsets of the INBOX messages and >> see if the problem goes away. > > Joseph, > > thank you very much for the follow up! > you won't believe it, literally minutes before your email I got this email > from the 'problem user' (below) > > thank you to all who responded! > > - I guess if TB debug log was enabled (as was suggested)- maybe the issue > would become apparent from TB debug log ? > > - I guess i should encourage POP users to switch to IMAP anyhow ? > > got this from problem user: > --- > Mozilla Thunderbird released an update which I just installed. > > Problem solved. > > I guess Tbird had a problem that the new release addressed. > > I'm sorry for the inconvenience. > > I'm mystified why my issue was only with one account. Perhaps it was > something to do with the size of the database. > > --- > yesterday it was > --- > I'm still experiencing a 40 second delay to retrieve emails for > xxx > > I have changed the pop port to 110 for the server but that did not > work at all. > > I have reinstalled my email client TBird but no change, anyway all the > other accounts on TBird are working ok but they are MAPI not POP. > > > Voytek >
Re: Configuring master password when using pam/openldap [was: "running alternate dovecot instances on the same server"]
So this is resolved. Turns out that getting rid of the "result_success = continue" fixed it. Which is odd, because I thought from the documentation that that was required. Anyway. Fixed. On 6/20/22 2:53 PM, Chris Hoogendyk wrote: Hoping that someone can help with this. It's a critical project that requires getting the master password to work. I've managed to get the logging at maximum level and what I'm seeing is the following. Testing with: chrisho@marlin:~$ telnet localhost 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. a login chrisho+ a NO [AUTHORIZATIONFAILED] Authorization failed * BYE Disconnected for inactivity. Connection closed by foreign host. chrisho@marlin:~$ And what I see in the log files is: Jun 17 12:16:10 marlin dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011session=#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60150#011resp (previous base64 data may contain sensitive data) Jun 17 12:16:10 marlin dovecot: auth: Debug: passwd-file(,127.0.0.1,master,): Master user lookup for login: chrisho Jun 17 12:16:10 marlin dovecot: auth: Debug: passwd-file(,127.0.0.1,master,): lookup: user= file=/etc/dovecot/passwd.masterusers Jun 17 12:16:10 marlin dovecot: auth: passwd-file(,127.0.0.1,master,): Master user logging in as chrisho Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug: pam(chrisho,127.0.0.1): lookup service=imap Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug: pam(chrisho,127.0.0.1): #1/1 style=1 msg=Password: Jun 17 12:16:12 marlin dovecot: auth-worker(9763): pam(chrisho,127.0.0.1): pam_authenticate() failed: Authentication failure (password mismatch?) (given password: ) Jun 17 12:18:53 marlin dovecot: imap-login: Disconnected: Inactivity (auth failed, 1 attempts in 163 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, I've replaced sensitive content with , , , and . BUT, note that in the next to the last line, where dovecot is attempting to authenticate the user via pam using the master password!! Why?? 1st, it shouldn't be trying to authenticate, it already has the master user authenticated. 2nd, why would it try to use what it should know is the master password to authenticate the regular user? Irrespective of those two questions, which are given primarily as what I would see as significant clues, what do I need to do to get it to work? Is this an issue with my dovecot configuration? An issue with my pam configuration? Or a bug in dovecot? Note also that I am using ldap as a user database. One of the criteria set up in pam is that the user must also exist in the local /etc/passwd, but the authentication goes to ldap. Any help very much appreciated. My doveconf output is already included in this thread, so I won't repeat it. The only thing I've changed is the debug levels. If there is anything else that would be useful to know, please let me know. --- Chris Hoogendyk - O__ Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~ - University of Massachusetts, Amherst --- Erdös 4 On 6/5/22 2:40 PM, Chris Hoogendyk wrote: I've changed the subject line on this email to more closely address the current issue. I'm hoping that with the most recent release out the door for a while there might be time to answer this. I believe I have the configuration done correctly to use a master password, but for some reason Dovecot seems to be going to ldap rather than just transferring to the requested user account. Since I don't really know what Dovecot is doing, it's hard to know what to try or whether it will actually work. In my mind it seems like Dovecot is could do something similar to a `sudo su user`, bypassing the usual authentication since it was given the master password. Details below and already on the dovecot mailing list. I've got a limited time to get this running, because central IT wants to transfer these accounts from our department server to the central IT servers this month. We have hundreds of accounts, so it doesn't scale to try to copy over individual accounts using individual's passwords. Thank you, --- Chris Hoogendyk - O__ Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~ - University of Massachusetts, Amherst --- Erdös 4 On 5/1/22 4:59 PM, Chris Hoogendyk wrote: Progress??? I realized that the error might indicate I had an empty conf paragraph or whatever you call it when d
Re: Configuring master password when using pam/openldap [was: "running alternate dovecot instances on the same server"]
Hoping that someone can help with this. It's a critical project that requires getting the master
password to work.
I've managed to get the logging at maximum level and what I'm seeing is the
following. Testing with:
chrisho@marlin:~$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
STARTTLS AUTH=PLAIN]
Dovecot ready.
a login chrisho+
a NO [AUTHORIZATIONFAILED] Authorization failed
* BYE Disconnected for inactivity.
Connection closed by foreign host.
chrisho@marlin:~$
And what I see in the log files is:
Jun 17 12:16:10 marlin dovecot: auth: Debug: client in:
AUTH#0111#011PLAIN#011service=imap#011secured#011session=#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60150#011resp
(previous base64 data may contain sensitive data)
Jun 17 12:16:10 marlin dovecot: auth: Debug:
passwd-file(,127.0.0.1,master,):
Master user lookup for login: chrisho
Jun 17 12:16:10 marlin dovecot: auth: Debug:
passwd-file(,127.0.0.1,master,):
lookup: user= file=/etc/dovecot/passwd.masterusers
Jun 17 12:16:10 marlin dovecot: auth:
passwd-file(,127.0.0.1,master,): Master
user logging in as chrisho
Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug:
pam(chrisho,127.0.0.1): lookup
service=imap
Jun 17 12:16:10 marlin dovecot: auth-worker(9763): Debug:
pam(chrisho,127.0.0.1): #1/1 style=1
msg=Password:
Jun 17 12:16:12 marlin dovecot: auth-worker(9763): pam(chrisho,127.0.0.1):
pam_authenticate()
failed: Authentication failure (password mismatch?) (given password:
)
Jun 17 12:18:53 marlin dovecot: imap-login: Disconnected: Inactivity (auth
failed, 1 attempts in
163 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1,
secured,
I've replaced sensitive content with , , , and . BUT,
note that in the next to the last line, where dovecot is attempting to authenticate the user via pam
using the master password!! Why?? 1st, it shouldn't be trying to authenticate, it already has the
master user authenticated. 2nd, why would it try to use what it should know is the master password
to authenticate the regular user?
Irrespective of those two questions, which are given primarily as what I would see as significant
clues, what do I need to do to get it to work? Is this an issue with my dovecot configuration? An
issue with my pam configuration? Or a bug in dovecot? Note also that I am using ldap as a user
database. One of the criteria set up in pam is that the user must also exist in the local
/etc/passwd, but the authentication goes to ldap.
Any help very much appreciated.
My doveconf output is already included in this thread, so I won't repeat it. The only thing I've
changed is the debug levels. If there is anything else that would be useful to know, please let me
know.
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
On 6/5/22 2:40 PM, Chris Hoogendyk wrote:
I've changed the subject line on this email to more closely address the current
issue.
I'm hoping that with the most recent release out the door for a while there might be time to
answer this.
I believe I have the configuration done correctly to use a master password, but for some reason
Dovecot seems to be going to ldap rather than just transferring to the requested user account.
Since I don't really know what Dovecot is doing, it's hard to know what to try or whether it will
actually work. In my mind it seems like Dovecot is could do something similar to a `sudo su user`,
bypassing the usual authentication since it was given the master password.
Details below and already on the dovecot mailing list.
I've got a limited time to get this running, because central IT wants to transfer these accounts
from our department server to the central IT servers this month. We have hundreds of accounts, so
it doesn't scale to try to copy over individual accounts using individual's passwords.
Thank you,
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
On 5/1/22 4:59 PM, Chris Hoogendyk wrote:
Progress???
I realized that the error might indicate I had an empty conf paragraph or whatever you call it
when dovecot failed to start. That is I commented out the driver = pam, but left the enclosing
passdb { } with no contents. That was the error. So I commented out everything in
/etc/dovecot/conf.d/auth-system.conf.ext. That worked and doveconf -n
Re: Configuring master password when using pam/openldap [was: "running alternate dovecot instances on the same server"]
I've changed the subject line on this email to more closely address the current
issue.
I'm hoping that with the most recent release out the door for a while there might be time to answer
this.
I believe I have the configuration done correctly to use a master password, but for some reason
Dovecot seems to be going to ldap rather than just transferring to the requested user account. Since
I don't really know what Dovecot is doing, it's hard to know what to try or whether it will actually
work. In my mind it seems like Dovecot is could do something similar to a `sudo su user`, bypassing
the usual authentication since it was given the master password.
Details below and already on the dovecot mailing list.
I've got a limited time to get this running, because central IT wants to transfer these accounts
from our department server to the central IT servers this month. We have hundreds of accounts, so it
doesn't scale to try to copy over individual accounts using individual's passwords.
Thank you,
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
On 5/1/22 4:59 PM, Chris Hoogendyk wrote:
Progress???
I realized that the error might indicate I had an empty conf paragraph or whatever you call it
when dovecot failed to start. That is I commented out the driver = pam, but left the enclosing
passdb { } with no contents. That was the error. So I commented out everything in
/etc/dovecot/conf.d/auth-system.conf.ext. That worked and doveconf -n gave me the right precedence
for masteruser and regular logins.
Then my login from telnet still failed. So I had two terminal windows, one with `sudo journalctl
-u dovecot.service -f` to follow the journal entries form dovecot, and another to do my telnet
localhost 143. Now, when I tried to login with the masteruser, I got an error in the journal that
said:
May 01 16:15:50 marlin dovecot[31944]: auth-worker(32577): Error:
pam(chrisho,127.0.0.1):
pam_authenticate() failed: Authentication failure (/etc/pam.d/imap missing?)
hmm. No idea what to put there. /etc/pam.d/ contains a bunch of entries, including dovecot. Then
there are some entries that start with common and include common-auth, common-session,
common-password, etc. The dovecot entry simply includes those. So, I tried just setting up a
symlink from imap pointing at the dovecot file. Then I tried logging in again via telnet. Now I
simply get:
May 01 16:26:40 marlin dovecot[31944]: imap-login: Aborted login (auth
failed, 2 attempts in 112
secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured,
session=<2BspEfndKMN/AAAB>
The entry of that line in the journal seemed to be slightly delayed. It showed up when I entered
`2 logout`.
At this point, my dovecot -n output is as follows:
chrisho@marlin:/etc/dovecot$ sudo doveconf -n
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-223-generic x86_64 Ubuntu 16.04.7 LTS
auth_master_user_separator = *
default_process_limit = 200
first_valid_gid = 98
first_valid_uid = 1000
login_access_sockets = tcpwrap
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_max_userip_connections = 8
mail_privileged_group = mail
mbox_write_locks = dotlock fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /etc/dovecot/passwd.masterusers
driver = passwd-file
master = yes
result_success = continue
}
passdb {
args = session=yes %Ls
driver = pam
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
protocols = " imap pop3"
service imap-login {
inet_listener imap {
address = localhost
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service tcpwrap {
unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
}
}
ssl = required
ssl_ca =
Aki,
Thankyou for your advice. I finally got around to this. I'm retired, working part time, and have
more to do than fits the time. Anyway . . .
I did the configuration in /etc/dovec
Re: running alternate dovecot instances on the same server
is Saturday afternoon and a bit slow). Dovecot starts up alright with the
above local.conf, but master user doesn't seem to work.
I'm testing with the following (master username and passwrd replaced):
chrisho@marlin:/etc/dovecot$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
STARTTLS AUTH=PLAIN]
Dovecot ready.
1 login chrisho*masteruser masterpassword
1 NO [AUTHORIZATIONFAILED] Authorization failed
2 exit
Connection closed by foreign host.
What I'm seeing in the logs is:
Apr 30 19:32:29 marlin auth[20859]: pam_unix(dovecot:auth): authentication
failure; logname=
uid=0 euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:29 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
Apr 30 19:32:31 marlin auth[20859]: pam_unix(imap:auth): authentication
failure; logname= uid=0
euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:31 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
The output of doveconf -n is as follows:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-223-generic x86_64 Ubuntu 16.04.7 LTS
auth_master_user_separator = *
default_process_limit = 200
first_valid_gid = 98
first_valid_uid = 1000
login_access_sockets = tcpwrap
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_max_userip_connections = 8
mail_privileged_group = mail
mbox_write_locks = dotlock fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
passdb {
args = /etc/dovecot/passwd.masterusers
driver = passwd-file
master = yes
result_success = continue
}
passdb {
args = session=yes %Ls
driver = pam
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
protocols = " imap pop3"
service imap-login {
inet_listener imap {
address = localhost
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service tcpwrap {
unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
}
}
ssl = required
ssl_ca = And, yes, Ubuntu 16.04 is EOL; however, we do have an Ubuntu Advantage account for this server. So
we do get security patches and the hope is that we can do a release upgrade this summer.
Any further guidance would be much appreciated. If any further information is needed, I can
provide it.
On 3/21/22 1:57 AM, Aki Tuomi wrote:
On 20/03/2022 22:36 Chris Hoogendyk wrote:
I'm posting to the list, but not on the list. I presume that means a reply-all to get to me as
well
as the list?
We have two servers (dovecot --version: 2.2.22 (fe789d2)) that handle email
for two different
departments.
We are transitioning mail service to the University central IT. They need to
move accounts in an
automated fashion and therefore need a master password to our dovecot servers.
However, we are
running with LDAP authentication, and I understand that a master password is
not possible in that
configuration.
Hi!
It is totally possible to use LDAP with master password, using configuration
like this:
# this must be first
passdb {
driver = static
args = password=masterpass
}
# current passdb config
# you probably already have this
userdb {
driver = ldap
args = /path/to/ldap/userdb
}
If this does not work, please send your `doveconf -n` as well.
Aki
--
---
Chris Hoogendyk
-
O__ Systems Administrator, Retired
c/ /'_ --- Biology & Geosciences Departments
(*) \(*) -- 315 Morrill Science Center III
~~ - University of Massachusetts, Amherst
---
Erdös 4
Re: running alternate dovecot instances on the same server
Aki,
Thankyou for your advice. I finally got around to this. I'm retired, working part time, and have
more to do than fits the time. Anyway . . .
I did the configuration in /etc/dovecot/local.conf, which is included in the
/etc/dovecot/dovecot.conf. However, the dovecot.conf includes the /etc/dovecot/conf.d/* before the
local.conf. I believe that means that the entries in the conf.d come first and have precedence. I
was getting authorization failures. The section of my local.conf is as follows:
#
# master passwd added 4/30/2022 based on email from Aki Tuomi on Dovecot
support list 3/21/2022,
# modified based on example from
https://doc.dovecot.org/configuration_manual/authentication/master_users/
# also added userdb at bottom of this and removed lines from
conf.d/auth-system.conf.ext to
resolve precedence.
# CGH
#
auth_master_user_separator = *
passdb {
driver = passwd-file
args = /etc/dovecot/passwd.masterusers
master = yes
result_success = continue
}
passdb {
driver = pam
args = session=yes %Ls
}
userdb {
driver = passwd
}
I had found entries in the /etc/dovecot/conf.d/auth-system.conf.ext that set drivers to pam for
passdb and passwd for userdb. I commented those two lines out since I had them covered in my
local.conf. That failed with the control process exiting with an error code. I quickly uncommented
those two lines in auth-system.conf.ext and it started just fine (I have a lot of users dependent on
this, although it is Saturday afternoon and a bit slow). Dovecot starts up alright with the above
local.conf, but master user doesn't seem to work.
I'm testing with the following (master username and passwrd replaced):
chrisho@marlin:/etc/dovecot$ telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
STARTTLS AUTH=PLAIN]
Dovecot ready.
1 login chrisho*masteruser masterpassword
1 NO [AUTHORIZATIONFAILED] Authorization failed
2 exit
Connection closed by foreign host.
What I'm seeing in the logs is:
Apr 30 19:32:29 marlin auth[20859]: pam_unix(dovecot:auth): authentication
failure; logname=
uid=0 euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:29 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
Apr 30 19:32:31 marlin auth[20859]: pam_unix(imap:auth): authentication
failure; logname= uid=0
euid=0 tty=dovecot ruser=chrisho rhost=127.0.0.1
Apr 30 19:32:31 marlin auth[20859]: pam_ldap: error trying to bind as user
"uid=chrisho,ou=People,dc=bio,dc=nsm" (Invalid credentials)
The output of doveconf -n is as follows:
# 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.13 (7b14904)
# OS: Linux 4.4.0-223-generic x86_64 Ubuntu 16.04.7 LTS
auth_master_user_separator = *
default_process_limit = 200
first_valid_gid = 98
first_valid_uid = 1000
login_access_sockets = tcpwrap
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_max_userip_connections = 8
mail_privileged_group = mail
mbox_write_locks = dotlock fcntl
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
passdb {
args = /etc/dovecot/passwd.masterusers
driver = passwd-file
master = yes
result_success = continue
}
passdb {
args = session=yes %Ls
driver = pam
}
pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
pop3_reuse_xuidl = yes
pop3_uidl_format = %08Xv%08Xu
protocols = " imap pop3"
service imap-login {
inet_listener imap {
address = localhost
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 0
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
service tcpwrap {
unix_listener login/tcpwrap {
group = $default_login_user
mode = 0600
user = $default_login_user
}
}
ssl = required
ssl_ca = And, yes, Ubuntu 16.04 is EOL; however, we do have an Ubuntu Advantage account for this server. So
we do get security patches and the hope is that we can do a release upgrade this summer.
Any further guidance would be much appreciated. If any further information is
needed, I can provide it.
On 3/21/22 1:57 AM, Aki Tuomi wrote:
On 20/03/2022 22:36 Chris
running alternate dovecot instances on the same server
I'm posting to the list, but not on the list. I presume that means a reply-all to get to me as well as the list? We have two servers (dovecot --version: 2.2.22 (fe789d2)) that handle email for two different departments. We are transitioning mail service to the University central IT. They need to move accounts in an automated fashion and therefore need a master password to our dovecot servers. However, we are running with LDAP authentication, and I understand that a master password is not possible in that configuration. Would it be possible to run an alternate dovecot process that would use local account authentication, have a master password, and use an alternate port for connecting? Ideally it would only read accounts without changing anything, and would not interfere with the operation of the other dovecot process. I'm hoping that I could copy the configuration files, make these changes, and then launch it manually without any startup scripts in /etc/inetd.conf. Oh, by the way, we are running Ubuntu 16.04 LTS and have contracts with Ubuntu Advantage for ongoing patch support. The dovecot version is from the distribution, installed with aptitude. -- ------- Chris Hoogendyk - O__ Systems Administrator, Retired c/ /'_ --- Biology & Geosciences Departments (*) \(*) -- 315 Morrill Science Center III ~~ - University of Massachusetts, Amherst --- Erdös 4
Re: email location - files or sql
On Wed, Feb 23, 2022 at 08:48:11PM -0500, Robert Moskowitz wrote: > > > I liked Maildir at the time and still do. Only a 1,000+ emails a day. My > wife keeps a lot (10K messages) on the server, I keep all of my various > boxes on the server small. So do my other users. > > > Personally, I would not use a relational database as a mail store unless > > specifically required by the mail system of your choice. Email is not > > organised in a way that benefits from a RDB. > > I was there for the beginning of RDB. Almost had NOMAD shoved down my > throat (UNIVAC) and did work with RIM (BCS) then R:Base. Was gamma release > site of DB2, where we worked out how to do a UNION which was not supported > in the original design. I have seen email systems that stuff the messages > into RDB and really wonder if the hammer really fits. > What is the opinion on repairs to problems that occur? Easier to fix in a DB? That does give a simple set of fixed points and dates to make repairs simple. I backup into a tar.gz and that preserves a timepoint, but how in the heck could a fix a problem that occurs? That seems like a nightmare problem. Does anyone have any method to do that? I'd love to know that if it exists. I love Maildir, but the backups are very large. I don't have a huge set of emails, but downloading a copy to home instead of my other server is a big task. Since I use a single ISP and once had a company shut down all servers, which makes me nervous. -- Hmm, Chris Bennett
Re: dovecot mailing list (this mailing list), DKIM, SPF and DMARC
On Sat, Feb 12, 2022 at 12:58:03PM +0100, Sebastian Nielsen wrote: > Thats a TLD ban. Meaning *.ru is banned. > > same applies for my domain for example, I ban *.xyz, *.date and a few others. > I don't understand at all why banning tld is reasonable. I'm not rich. I buy .rocks and .xyz .rocks really works well with the domain name. .xyz is short, memorable and easy to type. I can't afford to buy domain names that cost $200 a year to purchase. .com .net .info , etc. have run out of the names I wish to use. I have never ever sent a single spam email, but you would block my emails? Bluntly said, but without malice, that attitude favors the rich over the poor. I refuse to trust the BIG guys. My dad uses yahoo and gets emails yanked away while he is reading it. Also, I can't find a server company that has IP blocks that are clean enough. I truly wish I could. There are many other methods to block spam. IMHO, blocking by tld is a bit harsh. But you have the right to do whatever method you wish. I will only point out my thoughts. SPAM sucks! :-) -- Chris Bennett
Re: silly quesiton [ot]
On Mon, Jan 31, 2022 at 06:23:28AM +, Sam Kuper wrote:
> On Sun, Jan 30, 2022 at 07:49:56PM -0900, justina colmena ~biz wrote:
> > On January 30, 2022 6:30:44 PM AKST, Sam Kuper wrote:
> >> On Sun, Jan 30, 2022 at 06:17:49PM -0900, justina colmena ~biz wrote:
> >>> On January 30, 2022 5:46:53 PM AKST, dove...@ptld.com wrote:
> >>>> Storing mail in a db... at the end of the day isn't it still just a
> >>>> file (.db file) on the drive?
> >>>>
> >>>> Aren't you just adding bloat and complexity vs just storing the
> >>>> mail directly (maildir format) to a file on the drive? [...]
> >>>
> >>> You'll get better indexing and fast full text search by storing your
> >>> emails in a database rather than a flat file, hopefully after
> >>> decoding any attachments. Especially for spam scoring, analysis, and
> >>> classification. Much better performance deleting or moving specific
> >>> messages, too.
> >>
> >> Do you have evidence to back up these claims, specifically re: mail
> >> servers?
> >>
> >> Like-for-like benchmarks, for instance?
> >
> > Just ideas.
>
> OK, no then.
>
>
> > Removing or deleting a single message from near the beginning of a
> > large flat file takes an inordinate amount of time because the
> > remainder of the flat file has to be rewritten all the way from the
> > point of the deleted message to the end of the file and then
> > truncated.
>
> You might want to look up what Maildir is before making bold but
> apparently unfounded claims about it.
>
> Maildir is not a "large flat file". It is a set of conventions that
> amount to a database specification, in the traditional sense of the word
> "database": a system for storing data. (Not a relational database.)
>
Many people haven't ever had to deal with the old "database" style of
files instead of tables and columns.
Maildir does show it's age with the little complexities it has.
> DJB developed Maildir to gain performance and reliability improvements
> over mbox files. Unlike Maildirs, mbox files *are* "large flat files".
Corrupt your mbox file and bad things happen!
I also like being able to throw in some older backed up email when I
find I need a few more to fill out that important thread from 3 years
ago with Maildir.
Maildir does not have the relational database problem of needing to keep
up with updates to the database software.
And nothing works very well when you suddenly discover that the company
you are renting servers from decides to close up and turn everything
off. While you are in another country with internet cafes only and don't
even have a laptop with you! Happened to me once. 8-{
--
Chris Bennett
Re: silly quesiton [ot]
On Sun, Jan 30, 2022 at 09:46:53PM -0500, dove...@ptld.com wrote: > Storing mail in a db... at the end of the day isn't it still just a file (.db > file) on the drive? > Aren't you just adding bloat and complexity vs just storing the mail directly > (maildir format) to a file on the drive? > > What do you think you are saving? Security? > If someone can read files on your server, they can equally read a maildir or > a .db file. > K.I.S.S. I gain modularity for a system. The database is the foundation. I am working with: 1. Dovecot 2. Neomutt 3. OpenSMTPD Now, if I decide to drop or addon some new program, I can just adjust and/or add some new tables. Write a new stored procedure. Drop in a new Perl module or subroutine. 1. Dovecot 2. Neomutt 3. OpenSMTPD 4. Xyz 5. Abc 6. SuperDuperMail-ThingyPlus So what I am working for is a system that is united. Add a new user and email, CLI program, bang. All done. Change a password with a web interface. Click. All done. I'm in no rush. This is a fun side project. I have already done this type of work successfully for other kinds of projects, so it's different, but not really outside of my past experience. Secure today is wide open tomorrow. File, memory, etc. all get broken eventually. I'm much more worried about my own mistakes than that of others. :-* -- Chris Bennett
Re: silly quesiton
On Tue, Jan 25, 2022 at 03:50:12AM -0900, justina colmena ~biz wrote: > Maybe a future programming project idea: I want a system that will store all > mail messages and user account info in, say, a postgresql transactional > database, a little more manageable and reliable than ad hoc databasing with > those flat files all over the place cluttering up the system. > I am in progress moving towards something like that. As of right now, perl, dovecot for IMAP, neomutt and OpenSMTPD. Right now, .neomuttrc files *only* exist during the usage of neomutt. They have random names, cannot be written to and are immediately erased after neomutt starts (not quits). That is a very small window of threat. I would very much like to put all of the messages into PostgreSQL also instead of file folders under the user vmail. This is just a side project. As I have been advised, there is no need to even write a configuration file at all, but there are some issues with dbh that I need to solve with a different database module. If someone can read files that never exist, well... At some point you have to at least consider trusting something. That or just turn it all off and get another career. -- Chris Bennett
Re: Strategies for protecting IMAP (e.g. MFA)
On Mon, Nov 15, 2021 at 02:14:31PM -0600, Tyler Montney wrote: > I've just recently started using mailing lists, so replying is still a bit > awkward to me. (Probably be easier if we'd use forums.) Forums are a mixed bag. Some love them, some hate them. I previously ran a forum for LedgerSMB. It had some usage, but people mostly preferred the mailing lists. I was also running an add-on by the forum author that allowed posting incoming emails to boards. I never setup an outgoing to the mailing list. I wanted to. No time. However, reading and searching emails from a mailing list in a forum is *nice*. Really nice. You can see the entire thread at once. If any mailing list I'm part of wants to do a read-only posting of email only messages like that, I would put up a forum for that only, for free. I would only make something like that public if the mailing list owners wanted it done. - As far as 2FA over a phone, I hate it. When traveling, my phone only works in USA, Canada and Mexico. If I were in Guatemala, I'd have no access to my DNS server company. Ugh! -- Chris Bennett
Re: Password Schemes
On Sat, Sep 11, 2021 at 08:07:31PM -0500, John Schmerold wrote:
> My /etc/dovecot/conf.d/auth-passwdfile.conf.ext is configured to use MD5
>
>passdb {
> driver = passwd-file
> args = scheme=MD5 username_format=%n /etc/exim4/domains/%d/passwd
>}
>
>userdb {
> driver = passwd-file
> args = username_format=%n /etc/exim4/domains/%d/passwd
>}
>
> /home/account/conf/mail/domain.com/passwd has a mixture of MD5 &
> SHA512-CRYPT:
>
>
> scanner:{MD5}$1$M5QuU7QI$AE7Nnorb8KC5KMvyYfVcr0:account:mail::/home/account:0:userdb_quota_rule=*:storage=0M
>
> test:{SHA512-CRYPT}$6$towo0IVjzBgZ0htU$uTFbyJ3aPunrhsEEC2alHz6SEuPyBdL3JYDWc6Z0ZtA2cMFjFVJNqAwn04OKQfsu99DNcDGu21zkvdYbsPmgJ0:account:mail::/home/account:0:userdb_quota_rule=*:storage=0M
>
> Everything is working fine, is this by design? In other words does the {MD5}
> vs {SHA512-CRYPT} in passwd over-rule auth-passwdfile.conf.ext ?
>
If you can, I would get rid of MD5. It's no longer secure. Sending out
mountains of spam if a password gets cracked, could be problematic. :-{
I'm getting ready to drop using MD5 on secure cookies for that very
reason. Website software, not dovecot.
Hopefully that's helpful. I dropped one of my bare metal servers because
the company couldn't keep other spammers off of the IP block I was in.
They refused to do anything to clean up their blacklist, which included
me unfortunately.
Chris Bennett
Re: invalid vsize-hdr
nd trailing whitespace "
>>
>> # Most (but not all) settings can be overridden by different protocols
>> and/or
>> # source/destination IPs by placing the settings inside sections, for
>> example:
>> # protocol imap { }, local 127.0.0.1 { }, remote 10.0.0.0/8 { }
>>
>> # Default values are shown for each setting, it's not required to uncomment
>> # those. These are exceptions to this though: No sections (e.g.
>> namespace {})
>> # or plugin settings are added by default, they're listed only as examples.
>> # Paths are also just examples with the real defaults being based on
>> configure
>> # options. The paths listed here are for configure --prefix=/usr
>> # --sysconfdir=/etc --localstatedir=/var
>>
>> # Protocols we want to be serving.
>> #protocols = imap pop3 lmtp
>> protocols = imap pop3
>> # A comma separated list of IPs or hosts where to listen in for connections.
>> # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces.
>> # If you want to specify non-default ports or anything more complex,
>> # edit conf.d/master.conf.
>> #listen = *, ::
>> listen = *, ::
>>
>> # Base directory where to store runtime data.
>> #base_dir = /var/run/dovecot/
>>
>> # Name of this instance. In multi-instance setup doveadm and other commands
>> # can use -i to select which instance is used (an
>> alternative
>> # to -c ). The instance name is also added to Dovecot processes
>> # in ps output.
>> #instance_name = dovecot
>>
>> # Greeting message for clients.
>> #login_greeting = Dovecot ready.
>>
>> # Space separated list of trusted network ranges. Connections from these
>> # IPs are allowed to override their IP addresses and ports (for logging and
>> # for authentication checks). disable_plaintext_auth is also ignored for
>> # these networks. Typically you'd specify your IMAP proxy servers here.
>> #login_trusted_networks =
>> login_trusted_networks = 10.5.1.0/24
>> # Space separated list of login access check sockets (e.g. tcpwrap)
>> #login_access_sockets =
>>
>> # With proxy_maybe=yes if proxy destination matches any of these IPs,
>> don't do
>> # proxying. This isn't necessary normally, but may be useful if the
>> destination
>> # IP is e.g. a load balancer's IP.
>> #auth_proxy_self =
>>
>> # Show more verbose process titles (in ps). Currently shows user name and
>> # IP address. Useful for seeing who are actually using the IMAP processes
>> # (eg. shared mailboxes or if same uid is used for multiple accounts).
>> #verbose_proctitle = no
>>
>> # Should all processes be killed when Dovecot master process shuts down.
>> # Setting this to "no" means that Dovecot can be upgraded without
>> # forcing existing client connections to close (although that could also be
>> # a problem if the upgrade is e.g. because of a security fix).
>> #shutdown_clients = yes
>>
>> # If non-zero, run mail commands via this many connections to doveadm
>> server,
>> # instead of running them directly in the same process.
>> #doveadm_worker_count = 0
>> # UNIX socket or host:port used for connecting to doveadm server
>> #doveadm_socket_path = doveadm-server
>>
>> # Space separated list of environment variables that are preserved on
>> Dovecot
>> # startup and passed down to all of its child processes. You can also give
>> # key=value pairs to always set specific settings.
>> #import_environment = TZ
>>
>> ##
>> ## Dictionary server settings
>> ##
>>
>> # Dictionary can be used to store key=value lists. This is used by several
>> # plugins. The dictionary can be accessed either directly or though a
>> # dictionary server. The following dict block maps dictionary names to URIs
>> # when the server is used. These can then be referenced using URIs in format
>> # "proxy::".
>>
>> dict {
>> #quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
>> #expire = sqlite:/etc/dovecot/dovecot-dict-sql.conf.ext
>> }
>>
>> # Most of the actual configuration gets included below. The filenames are
>> # first sorted by their ASCII value and parsed in that order. The
>> 00-prefixes
>> # in filenames are intended to make it easier to understand the ordering.
>> !include conf.d/*.conf
>>
>> # A config file can also tried to be included without giving an error if
>> # it's not found:
>> !include_try local.conf
>> ---
>>
>> I need assistance. I appreciate the help.
>>
>> Chris
>>
>>
>>
>> --
>> Christopher Wensink
>> IS Administrator
>> Five Star Plastics, Inc
>> 1339 Continental Drive
>> Eau Claire, WI 54701
>> Office: 715-831-1682
>> Mobile: 715-563-3112
>> Fax: 715-831-6075
>> cwens...@five-star-plastics.com
>> www.five-star-plastics.com
>>
>>
>
> For what it's worth... I know less than nothing, but a quick search
> turned up an apparent issue with cpanel which sounds similar:
>
> https://forums.cpanel.net/threads/dovecot-errors.626131/
>
> John Tulp
>
>
Re: Dovecot and mutt
On Mon, Jan 25, 2021 at 08:52:14AM -0700, @lbutlr wrote: > I have tried, but failed,. To write a sieve and script to strip HTML parts of > messages and if the message is only HTML to pipe it through w3m and add the > html portion as an emo attachment (in case it has links that need clicking, > like on some 'confirm you exist' emails. I truly hate those. Most often they now require Javascript, too. I use ssh and neomutt. I'm going to write a macro to take the html attachment and put it in a website directory. I've been doing it the long hard manual way. I hate forced javascript. No excuse but sloppiness to have that on a confirm you exist page. In any case, this is just nice to vent a little steam out. I don't think we can do much except chastise users of mailing lists. Sounds like a good macro to send a polite form letter reply to evildoers. > Honestly, I do not main HTML per se, it is when the HTML specifies font size, > colors, background colors, and other garbage like that that I despise it. A > well formed HTML message is is fine, but those are very rare. +1 Thanks for the great software and long hard work to find the most miniscule hidden bugs! Chris Bennett
Re: Recommended Protocols?
On Tue, Nov 10, 2020 at 09:07:37AM -0600, Raymond Herrera wrote: > > I have arrived to a preliminary conclusion. The error that I am getting is > this: > > dovecot: imap-login: Disconnected: TLS: SSL_read() failed: SSL > routines:ssl3_read_bytes:sslv3 alert bad certificate: SSL alert number 42 > SSLv3 is no longer considered safe. You shouldn't use it. If the mail client program can't use something newer (I only allow TLSv2 and TLSv3), I would question whether it's a good choice. Because runaway spam is the kiss of death with blacklisting and IP reputation. Isn't sslv3 disallowed in the default config??? If it's only for reading and not sending mail, then just manually set it up as sslv3 allowed. Hopefully this is helpful. Debug logs can be helpful \o/, or just confusing :-[ but they are long to read through. Good luck! Chris Bennett
Move older messages to Old folder after a bunch of changes (sql + other stuff)
I'm starting to get things better done now. Moved to all virtual users over IMAP. Using a SQL database for users/passwords. Setting up folders working properly with neomutt. Right now, that leaves me with just new mail in the INBOX, and other folders. I have a bunch of old mail in each users Maildirs in their real home folders. Right now, I'd like to move all the old mail into a folder (doesn't really matter what it's called). Right now is actually a perfect time to do this by good luck in timing. How should I go about creating a new folder for those messages and then adding the old mail from system user home Maildir folders to the virtual users folders I'm using? Thanks for any help, Chris Bennett
Re: Providers running dovecot?
On Sat, Oct 10, 2020 at 04:31:07PM +0200, Olivier Cailloux wrote: > Le vendredi 09 octobre 2020 à 11:22 +0200, Piotr Auksztulewicz a > écrit : > > On Fri, Oct 09, 2020 at 10:15:10AM +0200, Olivier Cailloux wrote: > > > The real, “final” question I am interested in is, but which might be > > > slightly off-topic on this list (the reason I asked the other > > > question), is to find providers that satisfy these two conditions: > > > a) offer free e-mail accounts > > > b) implement correctly the IMAP SEARCH feature of RFC 3501. > > > > IMO this is the right question to ask, even here. > > You are probably right, in retrospect, I should have started with that > question. > > > > > > That is because my client uses the IMAP SEARCH feature, and it is > > > usually implemented incorrectly (e.g. in GMail or MS Exchange). > > > > Probably it would be more informative to describe which features you > > need that are implemented "incorrectly". > > Well, support of the IMAP SEARCH command is the specific feature I > need. See https://tools.ietf.org/html/rfc3501#section-6.4.4. > > > > > Reality check: RFCs are not government-enforced standards. There are many > > sensible RFCs that never got implemented widely, or nearly at all, are > > implemented partially, or there are widely deployed not-fully-compliant > > software systems. If your client software requires a feature that's not > > widely available, you're just limiting your audience. > > Sad but true, I believe you are completely right. I will perhaps have > to abandon my hope of finding compliant providers. > > > > > You may try to find a different way to achieve your goal using the > > features that are widely implemented. Real, successful software packages > > very often contain options to do some quirks in order to stay interoperable > > with existing noncompliant implementations. > > This is precisely the problem in my case: it is impossible to work > nicely around the lack of IMAP SEARCH feature on the side of my > software, which is client-side, because that support must be provided > server side. In a nutshell, the SEARCH command lets a client ask a > server: “give me all e-mails whose subject and date match such and such > criteria”. As a client, if the server does not implement IMAP SEARCH, I > simply can’t know which e-mails match such and such criteria, short of > downloading all e-mail headers and filtering them, which is orders of > magnitude slower if my user has many e-mails in her box. > > Admittedly, I can work around this more or less nicely, e.g. by > downloading all headers once, storing them on the device of my user, > and searching this local database, instead of re-downloading all > headers every time my software runs. (This is how Thunderbird, and, I > suppose, most MUAs out there, work.) But this creates other > inconvenience for the user: this database takes space, takes time and > bandwidth to build, has to be re-built when the user changes device, > there is a security issue with having these e-mail headers stored > locally; not talking about the fact that it will make my software much > more complex for a single feature that really should, conceptually, be > implemented server side. Hence my willingness to actively try to find > compliant providers before giving up. > > An alternative is to try to understand what exactly bugs in the > implementation of IMAP SEARCH of each of the main providers out there > (GMail; MS Exchange; and so on) and work around this on a case-by-case > basis. I suppose this has been investigated already by some developers; > if anybody knows where I could ask about this, I’d be very happy to ask > there, as I guess this discussion is becoming completely OT for this > list. > > But the general lack of support for remote search in well known > softwares such as Thunderbird (that seem to systematically perform > searches client-side, on the local database) makes me rather > pessimistic about the possibility of working around those bugs; after > all, if the server sometimes (or often) replies incorrectly, as my > tests indicate, there may be nothing the client can do to guess what > the right answer is. > -- > Olivier > > Some projects just can't get around the lack of compliance from vendors, whether it's from incompetence or their not needing/desiring to do it. Several years ago, I wanted to move lpd forward. After weeks of looking at the hardware with non-compliance, I finally just had to drop the project because it was truly a vast and hopeless situation. It was an excellent, but frustrating lesson. I wish you luck, but you might be in the same spot I found myself. Chris Bennett
Problems, virtual and alias, Dovecot, Neomutt, OpenSMTPD on OpenBSD
I'm having some problems. I'm using virtual users with real domains.
That is working, but the folders are not showing up properly in neomutt,
for both virtual users and system users.
When I use K9 on Android, the folders show up correctly.
Sending is working, but I don't have auth setup right on OpenSMTPD, so
I'm sending locally, not remotely (including from K9, which can't login
to send). That's really probably not relevant here, but I thought I'd
mention it. I'm a bit embarrassed to admit that after having to move
everything off of one server super rapidly to another already working
to save money, I realized that I had made some wrong assumptions.
Oops but +1 for me. So now I know what to fix there and why.
If anyone has a good .neomuttrc or hints, I would appreciate that help.
But I can save that problem for the neomutt mailing list.
Here are copies of my doveconf -n, dovecot/passwd, vaddr, vdomains and
vusers. Obscured of course. Plus the DNS IP's of this server and the one
other server, which isn't running dovecot yet. Everything is using
maildir on both servers. DNSSEC on all but bennettconstruction.us and
mail.bennettconstruction.us
Is there anything wrong, missing or screwed up here?
I do intend to use encrypted passwds once I get submission to OpenSMTPD
working and add the right ports for that.
So far, I really haven't found the three different dovecot logs to be
particularly helpful, but I can send excerpts of those if helpful.
Thanks. I appreciate any help. This is my first go at IMAP and someone
sent me some config info a while ago, which really helped a ton.
I really like this software, I used it previously a few years ago for
POP3. I do find the documentation missing some sample configs with some
explanations for why and how. At some point I would like to add some
more complicated things, but I don't understand what some config options
really do. I also understand that documentation vs. code work are always
in conflict. 28 hour days would really help! :-}
doveconf -n:
# 2.3.10.1 (a3d0e1171): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.10 (bf8ef1c2)
# OS: OpenBSD 6.7 amd64 ffs
# Hostname: cowboyup.xyz
auth_debug = yes
auth_debug_passwords = yes
auth_socket_path = /var/dovecot/auth-userdb
auth_verbose = yes
auth_verbose_passwords = sha1
debug_log_path = /var/log/dovecot-debug.log
first_valid_uid = 1000
hostname = consulting-diy-construction.com
imap_id_log = *
imap_id_send = *
info_log_path = /var/log/dovecot-info.log
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
lmtp_client_workarounds = whitespace-before-path mailbox-for-path
log_path = /var/log/dovecot.log
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_debug = yes
mail_location = maildir:/home/vmail/%n@%d/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext
mbox_write_locks = fcntl
mmap_disable = yes
namespace inbox {
inbox = yes
list = yes
location =
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
subscriptions = yes
}
passdb {
driver = bsdauth
}
passdb {
args = /etc/dovecot/passwd
driver = passwd-file
}
plugin {
mail_log_fields = uid box msgid size
sieve = file:~/sieve;active=~/.dovecot.sieve
}
postmaster_address = postmas...@consulting-diy-construction.com
protocols = imap lmtp sieve
service auth-worker {
user = root
}
service auth {
unix_listener auth-userdb {
group = vmail
mode = 0666
user = vmail
}
}
service dict {
unix_listener dict {
group = vmail
mode = 0600
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
service_count = 1
}
service imap {
process_limit = 1024
}
service lmtp {
unix_listener lmtp {
mode = 0666
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
inet_listener sieve_deprecated {
port = 2000
}
}
ssl_dh = # hidden, use -P to show it
ssl_prefer_server_ciphers = yes
userdb {
driver = passwd
}
userdb {
args = uid=vmail gid=vmail home=/home/vmail/%n@%d
driver = static
}
verbose_ssl = yes
protocol lmtp {
mail_plugins =
}
protocol imap {
mail_max_userip_connections = 10
mail_plugins =
}
protocol lda {
mail_plugins =
}
local 172.107.198.230 {
ssl_cert =
Thanks, got replys
I've got a mix of some receive only, some send only. I tried to slam together two different configs really fast. This helped. I'll look things over and post some configs after a bit. Thanks again! Chris Bennett
Test on sending only
I had to move off of a server to this one too fast. Having problems If this goes through, if someone could reply to ch...@bennettconstruction.us instead of on-list. Thanks, Chris Bennett
Re: Problems with userdb and mail deliveribility
On Sun, May 17, 2020 at 01:00:53PM -0600, @lbutlr wrote: > On 17 May 2020, at 11:08, Chris Bennett > wrote: > > > > I realize that I dumpimg a lot of info out there. > > But not what we need. > > doveconf -n > postfix -n (or equivalent) > > What MTA you are using (postfix, etc) > > Errors in mail.log showing the failures. > > > > I can shoot for system users, but I really want to get the virtual users > > working. This is a bit of a project. > > user=vmail delay=2h50m40s result=TempFail stat=Error (temporary failure: > "mail.lmtp: LMTP server error: 550 5.1.1 > User doesn't exist: > > Probably your LMTP lookup is malformed, probably in your MTA. > > lmtp(rodeo)<45419>: Error: > mkdir(/var/vmail//rodeo/Maildir) failed: Permission denied (euid=1000(rodeo) > egid=1000(rodeo) missing +w perm: /var, dir owned by 0:0 mode=0755) > > You have permission issues. vmail (and all files and folders in vmail) should > be owned by the vmail user, not by root. > Thanks. Sorry for such a delay in responding, but I was out in the boonies for awhile. I am using OpenSMTPD. The problems for this were indeed as you suggested and with the passwd files. This is all working now, but I have some other problems which I will work on too. I will continue to work on those and ask another thread if I cannot get that worked out. Thanks again. Chris Bennett
Re: Problems with userdb and mail deliveribility
I realize that I dumpimg a lot of info out there. Good news is that I´m online but not in production. I can shoot for system users, but I really want to get the virtual users working. This is a bit of a project. Thanks for anyone who wants to help. I was in a hotel room when I started and now on Amtrak train with connections fading in and out. Someone shared some of their configs, but I still confused. I may just scrap what Ie done and start fresh. Would if be helpful if I followed up with each file inline I was kirring then 40kb limit. Thanks, Chris Bennett
Re: http API for IMAP
You just described a more complex environment not a simplified environment. You can run code within code within code. Already we have all this abstraction and it’s leading to people not understanding and implementing solutions that make very little sense. Like what you describe. Why would you want to run an IMAP server in a browser? It’s so wrong I don’t even know where to start. If you are not talking about an IMAP server and you are talking about an IMAP client you are in the wrong forum. > On Nov 14, 2019, at 8:21 AM, Phillip Odam via dovecot > wrote: > > A HTTP API for IMAP and for that matter, POP3 and SMTP is exactly what we > built where I work. > > For anyone wonder why build such a thing? A simplified interface is an > exceptionally powerful tool. Many of our clients have encountered issues > constructing multipart http requests so if that’s an issue, good luck getting > IMAP to work. Since multipart turns out to be such an issue we’ve turned out > support for a variety of uploads including (completely non-standard) where > the request body is just the file content. > > We’ve built our API in Java and as such have simply used the JavaMail API. > Admittedly though the plan is to get off this design since when there’s 10s > of thousands messages in an IMAP maildir folder the index can grow to an > unmanageable size. > > On Thu, Nov 14, 2019 at 2:59 AM Thomas Güttler via dovecot > wrote: > > > Am 13.11.19 um 17:21 schrieb Ralph Seichter via dovecot: > > * Thomas Güttler via dovecot: > > > >> Is there a way to access mails in dovecot via https? > > > > Why on earth would that be beneficial? > > > >"The Internet Message Access Protocol, Version 4rev1 (IMAP4rev1) > > allows a client to access and manipulate electronic mail messages > > on a server." (RFC 3501) > > > > Putting it bluntly: Learn to use the protocol specifically designed for > > the task at hand, not a protocol with different design goals that you > > happen to know better. > > Stateless, http and URLs are the future. JavaScript running on in browser or > mobile phone can't connect to IMAP/SMTP. > > AFAIK you can't sent a link/URL to a mail on a shared folder to a friend. > Like "Hi bob, she loves me. See this message from here https:/./" > > Regards, >Thomas Güttler > > > > > -- > Thomas Guettler http://www.thomas-guettler.de/ > I am looking for feedback: https://github.com/guettli/programming-guidelines
Re: Dovecot eBook
On Sat, Jul 20, 2019 at 02:32:34AM -0600, LuKreme via dovecot wrote: > On Jul 19, 2019, at 19:29, Peter Fraser via dovecot > wrote: > > I have a strange question. I bought the Dovecot Book off Amazon. I can > > easily prove it with a picture and/or my receipt off Amazon. I still have > > it o my library but I don’t like to travel around with it. Is there a way > > for me to get a PDF copy? I just checked Amazon and there is still no PDF > > version available there. > > Tedious, but scan the book. I have done this with my iPhone and it resulted > in a very good copy that was fully OCRed > > I will double check, but there is software in OpenBSD (Linux too), that can quickly change ebook to pdf. No hassle and you can boot OpenBSD off of a usb stick. Chris Bennett
Email encryption and key protection
I was reading through Dovecot mail-crypt plugin documentation and I'm wondering what is the benefit of turning the encryption on if private and public keys are both stored on the server? What are the benefits and how the key can be protected (apart from file permissions). Cheers, Chris signature.asc Description: OpenPGP digital signature
haproxy + submission services -> postfix failure
Hi,
I have a nginx server which is using the proxy protocol to forward tcp
connections to dovecot. Dovecot is configured to be a submission
service for email to be sent. Then postfix should send the email
itself which is also using the ha proxy protocol. There are a few
moving parts in this problem so I'm not sure where the problem is. But
I want to ask if somebody can validate my dovecot configuration
somehow so I can start to tick off some things from the list.
Sending email fails, seems to get to postfix, then die
Receiving emails succeeds and I don't have any problem to pick them up.
I've figured out some stuff, like lmtp shouldn't use haproxy when
talking between postfix -> dovecot for receiving emails. If I enable
the protocol on lmtp, I can't receive any emails at all.
In order to get postfix to accept emails, I enabled haproxy protocol
and enabled postscreen and then postfix could access the source ip and
stop my server from being an open relay.
I've got tls certificates installed on dovecot and postfix, all
created by letsencrypt and I don't appear to have any problems with
them.
I will try to give as much information about the config as I can, I'm
not sure what other parts are good to have, but let me know if you are
missing something or want to check a value.
>> 10-master.conf:
service submission-login {
inet_listener submission {
port = 587
haproxy = yes
}
}
service lmtp {
inet_listener lmtp {
port = 24
haproxy = no
}
}
>> 20-submission.conf
submission_relay_host = postfix.mail-server
submission_relay_port = 25
submission_relay_ssl = starttls
submission_relay_ssl_verify = yes
Then because it might help to give the other side of the connection
configuration for postfix, here is the relevant information:
>> master.cf:
smtp inet n - - - 1 postscreen
smtpd pass - - - - - smtpd
>> main.cf
postscreen_upstream_proxy_protocol = haproxy
postscreen_upstream_proxy_timeout = 10s
That's it. I don't know what other information could be useful.
There are some logs, they are like this (I've got logging turned on
for pretty much every option I have:
Dovecot logs:
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
Added userdb setting: plugin/quota_rule=*:bytes=0
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
Effective uid=8, gid=8, home=/mail/__DOMAIN_COM__/__USER__
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no,
list=yes, subscriptions=yes
location=maildir:/mail/__DOMAIN_COM__/__USER__
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
maildir++: root=/mail/__DOMAIN_COM__/__USER__, index=, indexpvt=,
control=, inbox=/mail/__DOMAIN_COM__/__USER__, alt=
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-server: conn __IP_ADDR_1__:31217 [0]: Connection created
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connection created
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Looking up IP address
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: DNS lookup successful;
got 1 IPs
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connecting to
10.104.211.161:25
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connected
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Received greeting from
server: 421 4.3.2 No system resources
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connection failed: 421
4.3.2 No system resources
Apr 19 17:54:57 submission(__EMAIL__)<497>: Error:
Failed to establish relay connection: 421 4.3.2 No system resources
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Disconnected
Apr 19 17:54:57 submission(__EMAIL__)<497>: Info:
Disconnect from __IP_ADDR_1__: Failed to establish relay connection
in=0 out=22 (state=GREETING)
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-server: conn __IP_ADDR_1__:31217 [0]: Disconnected: Failed to
establish relay connection
Postfix Logs:
postfix/postscreen[525]: warning: haproxy read: time limit exceeded
If anybody could help out, I'd be grateful because I just can't see
what the problem is.
Chris
Dovecot 2.3 no longer accepts ssl_key_password
Hit a little problem when I upgraded a system from FreeBSD 10.3 to 11.2. I did not receive any errors in the upgrade. The system is running 4 jails and everything seems to work except in Dovecot dovecot-2.3.4_5 where when using the exact same configuration which worked in 10.3 with the same password protected certificate key. (doveconf -n -P shows the correct password.) ssl_ca =
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1
Hello Stephan,
I'm interested to see this get fixed (and ideally one day for it to be
really easy for people to set up Alpine so that it competes fairly with
heavier GUIs!), but I do not know that I am qualified to help (beyond
light testing). Is there someone from Alpine on this thread? The second
(askubuntu.com) link you give is my own question (and answer, based on Aki
Tuomi's earlier responses).
Thanks and best wishes,
Chris
On Sun, 6 Jan 2019, Stephan Bosch wrote:
>
> Op 31/12/2018 om 16:53 schreef Chris Barrington-Leigh:
> > I then followed the instructions in the Warning lines above and now I get:
> >
> > dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > # Hostname: cpbl-t450s
> > mail_location =
> > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > service stats {
> > unix_listener stats-writer {
> > mode = 0666
> > }
> > }
> > ssl_dh = # hidden, use -P to show it
> > verbose_proctitle = yes
> >
> >
> > However, this does not help my primary symptom:
> >
> >
> > $ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
> > Permission denied
> > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
> > SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND
> > URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED
> > I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH
> > LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY] Logged in as meuser
> >
>
> I notice that you use an explicit configuration file for your imap invocation.
> Which likely means that this is something different from the config used the
> main dovecot service running on your system. That service block needs to be in
> the configuration of the dovecot instance running the stats service (which is
> likely just the main /etc/dovecot/dovecot.conf). Given the location of your
> own config file, I'm assuming you cannot manage the main dovecot instance?
>
> BTW, there is a related thread with a different solution to this issue:
>
> https://www.dovecot.org/pipermail/dovecot/2019-January/114151.html
>
> We're still working on getting that bug fixed, so that will not help you
> immediately, unless you can patch dovecot.
>
> BTW, I find it a bit weird that Alpine would fail on this. It is not a fatal
> error, so the imap service will just work. The error is sent to /dev/stderr,
> so I'd normally expect a service like Alpine to be able to ignore output sent
> there.
>
> The question below is about the same issue and their config suggests that
> stderr is sent to a log file:
>
> https://askubuntu.com/questions/1104056/how-to-use-local-dovecot-imap-with-alpine-18-10-changes
>
> So, I really don't understand what the core of the problem is. Why does Alpine
> fail on this at all?
>
> Regards,
>
> Stephan.
>
>
>
>
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1 (fwd)
Hi Aki,
Bingo! Putting that line in dovecot.conf rather than my custom file, and
then using my old custom file, unchanged, from Alpine works perfectly.
My only remaining question, then, would be ... what was this (change)
about / or is there any documentation that I should/could have read to
know what to do without asking a lead developer?
Thank you very much.
I hope you had a great start to the New Year!
Happy 2019,
Chris
On Mon, 31 Dec 2018, Aki Tuomi wrote:
> If your client directly executes imap binary, it is not mandatory for dovecot
> to be running, but you need to put the service section into
> /etc/dovecot/dovecot.conf and restart the dovecot service.
>
> Putting it to your local config file does not affect the global service.
>
> Aki
>
> > On 31 December 2018 at 22:16 Chris BL wrote:
> >
> >
> >
> > Hi Aki,
> >
> > I am not 100% sure. I do not really know what my OS does. "ps -A" says
> > dovecot is running, but if I say "sudo service dovecot stop", my email
> > client works fine still (that is, assuming the old version of dovecot is
> > installed). ie the on-demand instances of dovecot-imapd that Alpine calls
> > are, I assume, started and stopped by Alpine. (?)
> >
> > Is my OS using dovecot for other stuff, like managing system mail (e.g.
> > root's reports of cron jobs, which still go to /var/mail, but I never read
> > them)? I am not certain. But I believed that any other use of dovecot
> > would be using a different .conf file, so I was hoping I just had to know
> > how to make my custom one (-c flag) work.
> >
> > Thanks,
> > Chris
> >
> >
> > On Mon, 31 Dec 2018, Aki Tuomi wrote:
> >
> > > Am I assuming correctly that you are not, in fact, running anything but
> > > imap process from Dovecot?
> > >
> > > Aki
> > >
> > > > On 31 December 2018 at 18:35 Chris BL
> > > > wrote:
> > > >
> > > >
> > > >
> > > > Hi Aki,
> > > >
> > > > Oops; I did not think carefully enough about your instructions. Here is
> > > > the output from
> > > >
> > > > $ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > >
> > > > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > # Pigeonhole version 0.5.2 ()
> > > > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > > > # Hostname: cpbl-t450s
> > > > doveconf: Warning: please set ssl_dh= > > > doveconf: Warning: You can generate it with: dd
> > > > if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam
> > > > -inform der > /etc/dovecot/dh.pem
> > > > mail_location =
> > > > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > > > service stats {
> > > > unix_listener stats-writer {
> > > > mode = 0666
> > > > }
> > > > }
> > > > ssl_dh = # hidden, use -P to show it
> > > > verbose_proctitle = yes
> > > >
> > > >
> > > > I then followed the instructions in the Warning lines above and now I
> > > > get:
> > > >
> > > > dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > # Pigeonhole version 0.5.2 ()
> > > > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > > > # Hostname: cpbl-t450s
> > > > mail_location =
> > > > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > > > service stats {
> > > > unix_listener stats-writer {
> > > > mode = 0666
> > > > }
> > > > }
> > > > ssl_dh = # hidden, use -P to show it
> > > > verbose_proctitle = yes
> > > >
> > > >
> > > > However, this does not help my primary symptom:
> > > >
> > > >
> > > > $ /usr/lib/dovecot/imap -c
> > > > /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > > > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer)
> > > > failed: Permission denied
> > > > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> > > > SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=OR
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1 (fwd)
Hi Aki,
I am not 100% sure. I do not really know what my OS does. "ps -A" says
dovecot is running, but if I say "sudo service dovecot stop", my email
client works fine still (that is, assuming the old version of dovecot is
installed). ie the on-demand instances of dovecot-imapd that Alpine calls
are, I assume, started and stopped by Alpine. (?)
Is my OS using dovecot for other stuff, like managing system mail (e.g.
root's reports of cron jobs, which still go to /var/mail, but I never read
them)? I am not certain. But I believed that any other use of dovecot
would be using a different .conf file, so I was hoping I just had to know
how to make my custom one (-c flag) work.
Thanks,
Chris
On Mon, 31 Dec 2018, Aki Tuomi wrote:
> Am I assuming correctly that you are not, in fact, running anything but imap
> process from Dovecot?
>
> Aki
>
> > On 31 December 2018 at 18:35 Chris BL wrote:
> >
> >
> >
> > Hi Aki,
> >
> > Oops; I did not think carefully enough about your instructions. Here is the
> > output from
> >
> > $ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> >
> > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > # Hostname: cpbl-t450s
> > doveconf: Warning: please set ssl_dh= > doveconf: Warning: You can generate it with: dd
> > if=/var/lib/dovecot/ssl-parameters.dat bs=1 skip=88 | openssl dhparam
> > -inform der > /etc/dovecot/dh.pem
> > mail_location =
> > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > service stats {
> > unix_listener stats-writer {
> > mode = 0666
> > }
> > }
> > ssl_dh = # hidden, use -P to show it
> > verbose_proctitle = yes
> >
> >
> > I then followed the instructions in the Warning lines above and now I get:
> >
> > dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
> > # Hostname: cpbl-t450s
> > mail_location =
> > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > service stats {
> > unix_listener stats-writer {
> > mode = 0666
> > }
> > }
> > ssl_dh = # hidden, use -P to show it
> > verbose_proctitle = yes
> >
> >
> > However, this does not help my primary symptom:
> >
> >
> > $ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
> > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
> > Permission denied
> > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
> > SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
> > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE
> > UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> > WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+
> > NOTIFY] Logged in as meuser
> >
> >
> > Thanks!
> > Chris
> >
> >
> >
> > On Mon, Dec 31, 2018 at 10:27 AM Aki Tuomi
> > wrote:
> > Did you remove the service section before taking doveconf -n? 'cos I
> > can't see it here..
> >
> > Also, did you restart dovecot after config change?
> >
> > Aki
> >
> > > On 31 December 2018 at 16:54 Chris BL
> > wrote:
> > >
> > >
> > > Hi Aki,
> > >
> > > Yes, the same connect error: imap(meuser,)Error:
> > net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
> > >
> > > The mail location in "doveconf -n" (belo) does not reflect my
> > preferences.
> > >
> > > In Alpine, if I acknowledge the error I get, which tells me to add
> > > novalidate-cert in my pinerc file, it prompts me for my POSIX
> > username and
> > > password. After that, I get my /var/spool/mail inbox, which is not
> > what I
> > > want.
> > >
> > >
> > > doveconf -n
> > > # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
> > > # Pigeonhole version 0.5.2 ()
> > > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
> &g
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1
Hi Aki,
(This message might not make it to the list; I have to switch back to the
other dovecot to be able to send from that address)
Oops; I did not think carefully enough about your instructions. Here is the
output from
$ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
# 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
# Hostname: cpbl-t450s
doveconf: Warning: please set ssl_dh= /etc/dovecot/dh.pem
mail_location =
maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_dh = # hidden, use -P to show it
verbose_proctitle = yes
I then followed the instructions in the Warning lines above and now I get:
dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
# 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
# Hostname: cpbl-t450s
mail_location =
maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_dh = # hidden, use -P to show it
verbose_proctitle = yes
However, this does not help my primary symptom:
$ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
Permission denied
* PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN
CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY]
Logged in as meuser
Thanks!
Chris
On Mon, Dec 31, 2018 at 10:27 AM Aki Tuomi
wrote:
> Did you remove the service section before taking doveconf -n? 'cos I can't
> see it here..
>
> Also, did you restart dovecot after config change?
>
> Aki
>
> > On 31 December 2018 at 16:54 Chris BL
> wrote:
> >
> >
> > Hi Aki,
> >
> > Yes, the same connect error: imap(meuser,)Error:
> net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
> >
> > The mail location in "doveconf -n" (belo) does not reflect my
> preferences.
> >
> > In Alpine, if I acknowledge the error I get, which tells me to add
> > novalidate-cert in my pinerc file, it prompts me for my POSIX username
> and
> > password. After that, I get my /var/spool/mail inbox, which is not what
> I
> > want.
> >
> >
> > doveconf -n
> > # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
> > # Pigeonhole version 0.5.2 ()
> > # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
> > # Hostname: myhost-t450s
> > mail_location = mbox:~/mail:INBOX=/var/mail/%u
> > mail_privileged_group = mail
> > namespace inbox {
> > inbox = yes
> > location =
> > mailbox Drafts {
> > special_use = \Drafts
> > }
> > mailbox Junk {
> > special_use = \Junk
> > }
> > mailbox Sent {
> > special_use = \Sent
> > }
> > mailbox "Sent Messages" {
> > special_use = \Sent
> > }
> > mailbox Trash {
> > special_use = \Trash
> > }
> > prefix =
> > }
> > passdb {
> > driver = pam
> > }
> > protocols = " imap"
> > ssl_cert = > ssl_client_ca_dir = /etc/ssl/certs
> > ssl_dh = # hidden, use -P to show it
> > ssl_key = # hidden, use -P to show it
> > userdb {
> > driver = passwd
> > }
> >
> >
> >
> > On Mon, 31 Dec 2018, Aki Tuomi wrote:
> >
> > > You are still getting the connect error?
> > >
> > > Can you share your doveconf -n?
> > >
> > > Aki
> > >
> > > > On 31 December 2018 at 16:42 cpblpublic+dove...@gmail.com wrote:
> > > >
> > > >
> > > > Hi Aki,
> > > >
> > > > Thanks. I put that in the .conf file I mentioned, and it made no
> > > > difference.
> > > >
> > > > Chris
> > > >
> > > > > Hi!
> > > > >
> > > > > In your case it's probably easiest to change
> > > > >
> > > > > service stats {
> > > > > unix_listener stats-writer {
> > > > > mode = 0666
> > > > > }
> > > > > }
> > > > >
> > > > > A
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1 (fwd)
Hi Aki,
Oops; I did not think carefully enough about your instructions. Here is the
output from
$ dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
# 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
# Hostname: cpbl-t450s
doveconf: Warning: please set ssl_dh= /etc/dovecot/dh.pem
mail_location =
maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_dh = # hidden, use -P to show it
verbose_proctitle = yes
I then followed the instructions in the Warning lines above and now I get:
dovecot -n -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
# 2.3.2.1 (0719df592): /home/meuser/dotfiles/auto-stanford-dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10 ext4
# Hostname: cpbl-t450s
mail_location =
maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
service stats {
unix_listener stats-writer {
mode = 0666
}
}
ssl_dh = # hidden, use -P to show it
verbose_proctitle = yes
However, this does not help my primary symptom:
$ /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-stanford-dovecot.conf
imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer) failed:
Permission denied
* PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND
URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE
UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY]
Logged in as meuser
Thanks!
Chris
On Mon, Dec 31, 2018 at 10:27 AM Aki Tuomi wrote:
Did you remove the service section before taking doveconf -n? 'cos I
can't see it here..
Also, did you restart dovecot after config change?
Aki
> On 31 December 2018 at 16:54 Chris BL
wrote:
>
>
> Hi Aki,
>
> Yes, the same connect error: imap(meuser,)Error:
net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
>
> The mail location in "doveconf -n" (belo) does not reflect my
preferences.
>
> In Alpine, if I acknowledge the error I get, which tells me to add
> novalidate-cert in my pinerc file, it prompts me for my POSIX username
and
> password. After that, I get my /var/spool/mail inbox, which is not
what I
> want.
>
>
> doveconf -n
> # 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.2 ()
> # OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
> # Hostname: myhost-t450s
> mail_location = mbox:~/mail:INBOX=/var/mail/%u
> mail_privileged_group = mail
> namespace inbox {
> inbox = yes
> location =
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Trash {
> special_use = \Trash
> }
> prefix =
> }
> passdb {
> driver = pam
> }
> protocols = " imap"
> ssl_cert = ssl_client_ca_dir = /etc/ssl/certs
> ssl_dh = # hidden, use -P to show it
> ssl_key = # hidden, use -P to show it
> userdb {
> driver = passwd
> }
>
>
>
> On Mon, 31 Dec 2018, Aki Tuomi wrote:
>
> > You are still getting the connect error?
> >
> > Can you share your doveconf -n?
> >
> > Aki
> >
> > > On 31 December 2018 at 16:42 cpblpublic+dove...@gmail.com wrote:
> > >
> > >
> > > Hi Aki,
> > >
> > > Thanks. I put that in the .conf file I mentioned, and it made no
> > > difference.
> > >
> > > Chris
> > >
> > > > Hi!
> > > >
> > > > In your case it's probably easiest to change
> > > >
> > > > service stats {
> > > > unix_listener stats-writer {
> > > > mode = 0666
> > > > }
> > > > }
> > > >
> > > > Aki
> > > >
> > > > > On 31 De
Re: Localhost imap server fails after upgrade from 2.2.33.2-1 to 1:2.3.2.1
Hi Aki,
Yes, the same connect error: imap(meuser,)Error:
net_connect_unix(/var/run/dovecot/stats-writer) failed: Permission denied
The mail location in "doveconf -n" (belo) does not reflect my preferences.
In Alpine, if I acknowledge the error I get, which tells me to add
novalidate-cert in my pinerc file, it prompts me for my POSIX username and
password. After that, I get my /var/spool/mail inbox, which is not what I
want.
doveconf -n
# 2.3.2.1 (0719df592): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.2 ()
# OS: Linux 4.18.0-13-generic x86_64 Ubuntu 18.10
# Hostname: myhost-t450s
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_privileged_group = mail
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
protocols = " imap"
ssl_cert = You are still getting the connect error?
>
> Can you share your doveconf -n?
>
> Aki
>
> > On 31 December 2018 at 16:42 cpblpublic+dove...@gmail.com wrote:
> >
> >
> > Hi Aki,
> >
> > Thanks. I put that in the .conf file I mentioned, and it made no
> > difference.
> >
> > Chris
> >
> > > Hi!
> > >
> > > In your case it's probably easiest to change
> > >
> > > service stats {
> > > unix_listener stats-writer {
> > > mode = 0666
> > > }
> > > }
> > >
> > > Aki
> > >
> > > > On 31 December 2018 at 16:05 cpblpublic+dove...@gmail.com wrote:
> > > >
> > > >
> > > >
> > > > I use a localhost dovecot imap server in order to access maildir
> > > > folders from Alpine (see 2009 instructions for my set up here
> > > > https://cpbl.wordpress.com/2009/11/07/alpine-offlineimap-and-gmail-under-ubuntu/
> > > > )
> > > >
> > > > My configuration has worked nicely since 2009, but failed after a
> > > > recent upgrade of dovecot (during basic upgrade from Ubuntu 18.04 to
> > > > 18.10).
> > > >
> > > > The version of Alpine did not change. Uninstalling dovecot-core and
> > > > dovecot-imapd, and reinstalling the older versions, fully fixes the
> > > > problem.
> > > >
> > > > I would like to know how to get my configuration working with the new
> > > > version of dovecot.
> > > >
> > > > Here's what the problem looks like:
> > > >
> > > > I have a file auto-dovecot.conf with only the following two lines.
> > > >
> > > > verbose_proctitle = yes
> > > > mail_location =
> > > > maildir:/home/meuser/imap/gmailStanford:LAYOUT=fs:INBOX=/home/meuser/imap/gmailStanford/INBOX
> > > >
> > > > Other /etc/ files are unchanged from installation defaults.
> > > >
> > > > Alpine calls imapd when it starts, with the following command:
> > > >
> > > > /usr/lib/dovecot/imap -c /home/meuser/dotfiles/auto-dovecot.conf
> > > >
> > > >
> > > > When I run that command, I get:
> > > >
> > > > imap(meuser,)Error: net_connect_unix(/var/run/dovecot/stats-writer)
> > > > failed: Permission denied
> > > > * PREAUTH [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE
> > > > SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT
> > > > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
> > > > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> > > > WITHIN
> > > > CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY LITERAL+ NOTIFY]
> > > > Logged in as meuser
> > > >
> > > >
> > > >
> > > >
> > > > so I am guessing that the first line is the issue, since when the older
> > > > version I get instead:
> > > >
> > > > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
> > > > ENABLE
> > > > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
> > > > THREAD=ORDEREDSUBJECT
> > > > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
> > > > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> > > > WITHIN
> > > > CONTEXT=SEARCH LIST-STATUS BINARY MOVE] Logged in as meuser
> > > >
> > > >
> > > > Can anyone help me? I cannot find documentation for using a localhost
> > > > server for Alpine, except my own from 2009.
> > > >
> > > > Many thanks!
> > > >
> > > > Chris
> > >
>
2.3.4 doesnt compile on FreeBSD 11.2 using clang
extract below, this has already been reported a while back but still no new patch, so this email is to serve as a reminder, if someone manually fixes it for the ports tree, I dont consider that a fix, ideally we need this fixed in the source code, as not everyone will install it from ports. Chris "clang40 -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-dns -I../../src/lib-test -I../../src/lib-settings -I../../src/lib-ssl-iostream -DPKG_RUNDIR=\""/var/run/dovecot"\" -DPKG_STATEDIR=\""/var/lib/dovecot"\" -DSYSCONFDIR=\""/etc/dovecot"\" -DBINDIR=\""/usr/bin"\" -I/usr/local/include -std=gnu99 -fdiagnostics-color -Wl,-rpath=/usr/local/llvm40/lib -O2 -pipe -march=native -fno-strict-aliasing -fstack-protector-strong -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/usr/local/include -MT test-event-stats.o -MD -MP -MF .deps/test-event-stats.Tpo -c -o test-event-stats.o test-event-stats.c clang-4.0: warning: -Wl,-rpath=/usr/local/llvm40/lib: 'linker' input unused [-Wunused-command-line-argument] test-event-stats.c:101:8: warning: implicit declaration of function 'kill' is invalid in C99 [-Wimplicit-function-declaration] (void)kill(stats_pid, SIGKILL); ^ test-event-stats.c:101:24: error: use of undeclared identifier 'SIGKILL' (void)kill(stats_pid, SIGKILL); ^ 1 warning and 1 error generated. gmake[2]: *** [Makefile:656: test-event-stats.o] Error 1 gmake[2]: Leaving directory '/usr/local/directadmin/custombuild/dovecot-2.3.4/src/lib-master' gmake[1]: *** [Makefile:565: install-recursive] Error 1 gmake[1]: Leaving directory '/usr/local/directadmin/custombuild/dovecot-2.3.4/src' gmake: *** [Makefile:683: install-recursive] Error 1"
Re: Ubuntu 18.04 (Bionic) packages now available
W dniu 23/11/2018 o 12:44, Aki Tuomi pisze: Please find instructions on how to use them at https://repo.dovecot.org/ Thank you. I was always interested why those packages cannot be in upstream, but people maintain their own repositories for them. Is it too slow-moving to get a pkg into upstream or it is in upstream already, but not up to date? Other reasons? Just curiosity, as I'll be in similar position soon, trying to maintain own package. Best regards, Chris Narkiewicz
Re: Delete vs. Expunge in Public Namespace
On Wed, 3 Oct 2018 11:04:05 +0300 Timo Sirainen wrote: > If it's sending CLOSE, that explains it. It's not just "some > servers", it's all IMAP servers. There's UNSELECT command to close > without expunging. Thank you! Sorry, didn't know that. It is sending close: ::1.38247-::1.00143: MALP9 CLOSE ::1.00143-::1.38247: MALP9 OK Close completed. ::1.38247-::1.00143: MALP10 LOGOUT ::1.00143-::1.38247: * BYE Logging out MALP10 OK Logout completed. - Chris
Delete vs. Expunge in Public Namespace
All,
my IMAP client [1] marks messages as deleted, but doesn't expunge.
Dovecot (2.2.27-3+deb9u2) expunges them. Is this because the mailbox is
part of public namespace?
::1.38247-::1.00143: MALP6 STORE
48,49,50,51,52,53,54,55,56,57,58,59,60,61 +FLAGS (\Deleted)
::1.00143-::1.38247: * 48 FETCH (FLAGS (\Deleted \Seen))
* 49 FETCH (FLAGS (\Deleted \Seen))
* 50 FETCH (FLAGS (\Deleted \Seen))
* 51 FETCH (FLAGS (\Deleted \Seen))
* 52 FETCH (FLAGS (\Deleted \Seen))
* 53 FETCH (FLAGS (\Deleted \Seen))
* 54 FETCH (FLAGS (\Deleted \Seen))
* 55 FETCH (FLAGS (\Deleted \Seen))
* 56 FETCH (FLAGS (\Deleted \Seen))
* 57 FETCH (FLAGS (\Deleted \Seen))
* 58 FETCH (FLAGS (\Deleted \Seen))
* 59 FETCH (FLAGS (\Deleted \Seen))
* 60 FETCH (FLAGS (\Deleted \Seen))
* 61 FETCH (FLAGS (\Deleted \Seen))
MALP6 OK Store completed.
::1.38247-::1.00143: MALP7 SEARCH DELETED
Oct 2 10:18:03 hyptest dovecot: imap(chris): delete:
box=MailingListen/test125, uid=180,
msgid=<740a9ba3-d637-3fa0-1b87-ff8d79491bf0@ hyp.postbox.xyz>,
size=1509 Oct 2 10:18:03 hyptest dovecot: imap(chris): expunge:
box=MailingListen/test125, uid=167,
msgid=<740a9ba3-d637-3fa0-1b87-ff8d79491bf0 @hyp.postbox.xyz>, size=1509
namespace {
list = yes
location =
maildir:/var/vmail/public/:LAYOUT=fs:INDEXPVT=%h/.public-indexes prefix
= MailingListen/ separator = /
subscriptions = no
type = public
}
- Chris
[1]
https://raw.githubusercontent.com/quentinsf/IMAPdedup/master/imapdedup.py
Re: HTTP DoveAdm API - Possible bug?
Thanks Aki,
We'll see if we can backport this internally to the version we're
currently running as a temporary fix until we can do a proper upgrade.
Regards,
Chris
On 19/09/18 07:57, Aki Tuomi wrote:
Yeah. This is a bug, there is a fix pending.
From bb200128c83610d213b2ff2e59f1e0440ecbd2c9 Mon Sep 17 00:00:00 2001
From: Aki Tuomi
Date: Tue, 18 Sep 2018 20:37:38 +0300
Subject: [PATCH] doveadm: Fix doveadm user output when called from server
Was forgotten in a13b1245bee0b6524b4aeb3c8fd9e34af648b746
---
src/doveadm/doveadm-auth-server.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/doveadm/doveadm-auth-server.c
b/src/doveadm/doveadm-auth-server.c
index 44278e467e..d9e4e01c74 100644
--- a/src/doveadm/doveadm-auth-server.c
+++ b/src/doveadm/doveadm-auth-server.c
@@ -170,6 +170,10 @@ cmd_user_list(struct auth_master_connection *conn,
const char *username, *user_mask = "*";
unsigned int i;
+ doveadm_print_init(DOVEADM_PRINT_TYPE_FORMATTED);
+ doveadm_print_formatted_set_format("%{username}\n");
+ doveadm_print_header_simple("username");
+
if (users[0] != NULL && users[1] == NULL)
user_mask = users[0];
@@ -180,7 +184,7 @@ cmd_user_list(struct auth_master_connection *conn,
break;
}
if (users[i] != NULL)
- printf("%s\n", username);
+ doveadm_print(username);
}
if (auth_master_user_list_deinit(&ctx) < 0)
i_fatal("user listing failed");
--
Delta V Technologies Limited
0 402 402www.deltav-tech.co.uk
Office: 17 Elm Close, Southampton, SO16 7DT
Company No. 11006104 Registered in England and Wales
HTTP DoveAdm API - Possible bug?
Hi all,
It seems that setting userMask in the doveadm http api's "user" command
to anything involving wildcards ?s or *s causes the API to fail.
When using the API with a userMask set to the fully qualified email
address, I get the expected fields back.
From a python shell, with an asterisk causes the following behaviour
(note that doveadm is a python module that wrappers the dovecot API into
Python's requests module).
>>> doveadm.do_query("http://[:::::]:24280";,
"API_KEY_HERE", [ ["user", { "userMask": ["*"] }, "c01"] ])
'[["doveadmResponse",,"c01"]]'
If I run the same command with a valid mailbox in the userMask Field, I
get the response I expect:
>>> doveadm.do_query("http://[[:::::]:24280";,
"API_KEY_HERE", [ ["user", { "userMask": ["t...@test.mydomain.com"] },
"c01"] ])
'[["doveadmResponse",{"t...@test.mydomain.com":{"uid":"x","gid":"y","home":"/mail/test.mydomain.com/users/test","mail":"maildir:/mail/test.mydomain.com/users/test/Maildir/:INDEX=/indexes/test.mydomain.com/test","nice":"10"}},"c01"]]'
Chatting with people in #dovecot revealed similar behaviour elsewhere so
it doesn't appear to be just me. I've also selectively edited the above
to avoid leaking data.
We're running on the following
# 2.2.34 (874deae): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.22 (22940fb7)
# OS: Linux 4.9.0-7-amd64 x86_64 Debian 9.5 ext4
If you need a full copy of the config, please let me know and I'll talk
to my client, however, I don't think this is config related.
Thanks in advance.
Regards,
Chris Malton
--
Delta V Technologies Limited
0 402 402www.deltav-tech.co.uk
Office: 17 Elm Close, Southampton, SO16 7DT
Company No. 11006104 Registered in England and Wales
Re: Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances
I’ve had the opportunity to test the same configuration with a fresh build of
the git master branch (2.4.devel) and the issue also occurs there. I see that
"mailbox_list_index = yes" is now enabled by default. It can still be disabled
via "mailbox_list_index = no" which allows the quota to be calculated correctly.
==
root@ubuntu1804:~# dovecot -n
# 2.4.devel (44282aeeb): /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 4.15.0-30-generic x86_64 Ubuntu 18.04.1 LTS
# Hostname: ubuntu1804
mail_location = maildir:~/Maildir
mail_plugins = quota
namespace inbox {
inbox = yes
location =
prefix = INBOX.
separator = .
}
passdb {
driver = pam
}
plugin {
quota = maildir:Mailbox
}
userdb {
driver = passwd
}
==
(To summarize from my previous message -- other than "mailbox_list_index =
yes", second most important part of replication is that there is at least one
email in the real inbox and at least one sub-folder named "INBOX" in maildir
format)
root@ubuntu1804:~# ls -ld
/home/myuser/Maildir/cur/1532529376.M543965P58007.centos7.local\,S\=12712627\,W\=12877782\:2\,S
/home/myuser/Maildir/.INBOX.Test/
-rw-rw-r-- 1 myuser myuser 12712627 Aug 14 18:28
'/home/myuser/Maildir/cur/1532529376.M543965P58007.centos7.local,S=12712627,W=12877782:2,S'
drwxrwxr-x 5 myuser myuser 87 Aug 14 18:56
/home/myuser/Maildir/.INBOX.Test/
=
(In the following example usage is doubled, there is only one email)
root@ubuntu1804:~# doveadm quota recalc -u myuser; doveadm quota get -u myuser
Quota name TypeValue Limit
%
MailboxSTORAGE 24830 -
0
MailboxMESSAGE 2 -
0
==
(In the following example it works correctly with mailbox_list_index disabled)
root@ubuntu1804:~# doveadm -o 'mailbox_list_index=no' quota recalc -u myuser;
doveadm quota get -u myuser
Quota name TypeValue Limit
%
MailboxSTORAGE 12415 -
0
MailboxMESSAGE 1 -
0
==
Best Regards
Re: Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances
I failed to mention previously that the behavior also exists in latest 2.2.36 built from git sources, and I believe also exists in 2.3.2. Though, I wasn't able to successfully build release-2.3.2 or master-2.3 branches from git in the minimal test environment due to an odd compilation error that I have not yet attempted to resolve, but I used an internal binary build of 2.3.2 which has a few (though probably not affecting this particular issue) patches applied to it. -- Chris Dillon Technical Analyst III Migrations Specialist cPanel, Inc. Register Now for cPanel Conference 2018 Oct 1 - 3, 2018, Houston, Texas https://conference.cpanel.com smime.p7s Description: S/MIME Cryptographic Signature
Inbox quota usage doubled when mailbox_list_index enabled, under some circumstances
Hello,
I searched through the list archives for anything that appeared to be
similar to this but I didn't find any good matches. I apologize if this
has been brought up before.
Beginning with Dovecot 2.2.34, reported quota usage of a user's inbox
can be doubled when the following criteria are met:
1) quota plugin is enabled
2) mailbox_list_index=yes
3) A sub-folder of the inbox exists whose name also matches the prefix
(for example, "INBOX.Test"), even if that folder has no contents.
(Apparently, some mail clients such as K-9 Mail on Android can create
these folders automatically)
Downgrading to 2.2.33.2 allows this configuration to work normally. I
looked through the changes in git between 2.2.33.2 and 2.2.34 and they
appeared extensive, so unfortunately I wasn't able to pinpoint a
specific change that caused this.
Reproduced on a CentOS 7 server with Dovecot 2.2.34 built from git
sources according to
https://wiki2.dovecot.org/CompilingSource#Compiling_Dovecot_From_Git
using the following minimal configuration:
==
root@centos7 [~]# dovecot -n
# 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 3.10.0-862.9.1.el7.x86_64 x86_64 CentOS Linux release
7.5.1804 (Core)
# Hostname: centos7.local
mail_location = maildir:~/Maildir
mail_plugins = quota
mailbox_list_index = yes
namespace inbox {
inbox = yes
location =
prefix = INBOX.
separator = .
}
passdb {
driver = pam
}
plugin {
quota = maildir:Mailbox
}
userdb {
driver = passwd
}
==
With the system user "myuser" and a minimal Maildir setup containing 1
email in the inbox, and one sub-folder named "INBOX.Test":
==
root@centos7 [~]# ls -lAR /home/myuser/Maildir
/home/myuser/Maildir:
total 4
drwxr-xr-x. 2 myuser myuser 77 Jul 25 09:36 cur
drwxr-xr-x. 5 myuser myuser 56 Jul 25 15:19 .INBOX.Test
-rw-r--r--. 1 myuser myuser 14 Jul 25 15:19 maildirsize
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 new
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 tmp
/home/myuser/Maildir/cur:
total 12416
-rw-r--r--. 1 myuser myuser 12712627 Jul 25 09:07
1532529376.M543965P58007.centos7.local,S=12712627,W=12877782:2,S
/home/myuser/Maildir/.INBOX.Test:
total 0
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 cur
-rw-r--r--. 1 myuser myuser 0 Jul 25 09:36 maildirfolder
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 new
drwxr-xr-x. 2 myuser myuser 6 Jul 25 09:36 tmp
/home/myuser/Maildir/.INBOX.Test/cur:
total 0
/home/myuser/Maildir/.INBOX.Test/new:
total 0
/home/myuser/Maildir/.INBOX.Test/tmp:
total 0
/home/myuser/Maildir/new:
total 0
/home/myuser/Maildir/tmp:
total 0
==
Given this configuration, when "mailbox_list_index" is enabled, after
quota recalculation, quota reports that there are 2 messages and that
double the amount of storage is used:
==
root@centos7 [~]# doveadm quota recalc -u myuser; doveadm quota get -u
myuser
Quota name Type Value
Limit %
Mailbox STORAGE 24830
- 0
Mailbox MESSAGE 2
- 0
==
When "mailbox_list_index" is disabled and quota is recalculated, it will
report the message and storage counts correctly:
==
root@centos7 [~]# doveadm -o 'mailbox_list_index=no' quota recalc -u
myuser; doveadm quota get -u myuser
Quota name Type Value
Limit %
Mailbox STORAGE 12415
- 0
Mailbox MESSAGE 1
- 0
==
I am hoping that someone with much more familiarity with the code path
involved here could take a look at this issue and possibly submit a fix
for it -- that would be greatly appreciated. Please let me know if you
need any more details regarding the replication steps, though I hope
that I didn't miss anything.
Thank you!
--
Chris Dillon
Technical Analyst III
Migrations Specialist
cPanel, Inc.
smime.p7s
Description: S/MIME Cryptographic Signature
Re: Config problem: Service #0 is missing name
> On Mar 25, 2018, at 10:37, Aki Tuomi wrote:
>
> Without looking at your config I'd say you have a nameless service section,
> like
>
> service {
>
> }
>
> Name goes after keyword service.
Yeah, I don’t seem to. This was the same config that was working before
things stopped for other reasons last month. I think I may’ve overlooked
something in my hand-coded replacement for libc functionality. I’m testing
that now.
[time goes by]
Yup. My error. Sorry for the noise on the list, it’s running now.
- Chris
Config problem: Service #0 is missing name
Okay. So following up on the problems I was discussing weeks ago, I have hand-patched dovecot to work around problems that I believe may be in libc and not dovecot. Trying to get the properly built and installed dovecot-2.2.35, however, is giving me an error I haven’t seen before. I presume that something got munged while I was hand-configuring and installing/uninstalling/reinstalling the port in recent weeks. Running on a FreeBSD 11-stable machine, when I run “service dovecot start”, it tells me: % sudo service dovecot start Starting dovecot. Fatal: Error reading configuration: Invalid settings: Service #0 is missing name /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot % Surprisingly, googling for “Service #0 is missing name” didn’t yield a suggestion. My config is the same as it had been previously, I believe. Does anyone have an idea what might be wrong here? Thanks. - Chris
Config problem: Service #0 is missing name
Okay. So following up on the problems I was discussing weeks ago, I have hand-patched dovecot to work around problems that I believe may be in libc and not dovecot. Trying to get the properly built and installed dovecot-2.2.35, however, is giving me an error I haven’t seen before. I presume that something got munged while I was hand-configuring and installing/uninstalling/reinstalling the port in recent weeks. Running on a FreeBSD 11-stable machine, when I run “service dovecot start”, it tells me: % sudo service dovecot start Starting dovecot. Fatal: Error reading configuration: Invalid settings: Service #0 is missing name /usr/local/etc/rc.d/dovecot: WARNING: failed to start dovecot % Surprisingly, googling for “Service #0 is missing name” didn’t yield a suggestion. My config is the same as it had been previously, I believe. Does anyone have an idea what might be wrong here? Thanks. - Chris
Re: Auth SEGV on sparc64, alignment problem?
> On Feb 22, 2018, at 15:21, Josef 'Jeff' Sipek wrote: > >> Loading the core file, as described >> https://www.dovecot.org/bugreport.html , shows the error in libc >> somewhere: > > I read the your other mails in this thread; can you run things as before and > do a 'bt full' on the core file with the debug-symbol-enabled libdovecot? > gdb seems to be catching the SIGTRAPs, which is making things a bit confusing. > >> (gdb) bt full >> #0 __unaligned_load ( >>p=0x617070656e640e6d , size=4) No difference there. I changed the install process to not strip things, and manually copied in all of the libs in /usr/local/lib/dovecot again with unstripped (I think libtool stripped them, I just rejiggered makefiles and install-sh). Loading a core from a SEGV shows: Loaded symbols for /libexec/ld-elf.so.1 #0 __unaligned_load ( p=0x706172736572690a , size=4) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap_align.c:45 45 val = (val << 8) | p[i]; (gdb) bt full #0 __unaligned_load ( p=0x706172736572690a , size=4) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap_align.c:45 val = 0 i = 0 #1 0x40adb7cc in __unaligned_fixup (uf=0x7fdf110) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap_align.c:78 addr = val = insn = 3254806592 sig = #2 0x40adb5b0 in __sparc_utrap (uf=0x7fdf110) at /usr/src/lib/libc/sparc64/sys/__sparc_utrap.c:100 sig = 16 #3 0x40a2c1cc in __sparc_utrap_gen () from /lib/libc.so.7 No symbol table info available. #4 0x40a2c1cc in __sparc_utrap_gen () from /lib/libc.so.7 No symbol table info available. Previous frame identical to this frame (corrupt stack?) (gdb) (Which as you note below, that address is actually “parseri\n”) > This address looks like ASCII - "append\x0em", so my theory at the moment > is: > > (1) something clobbers a pointer > (2) the CPU attempts to execute a load from the address > (3) a utrap is generated to handle unaligned load > (4) the utrap code attempts to emulate the unaligned load > (5) the CPU fails to access the address since it is bogus, and a SIGSEGV is >generated > > Now, I'm have no idea why it'd first try to work around the alignment > requirement before doing a quick sanity check and generating SIGSEGV to > begin with, but that's my theory based on the info available so far. > Hopefully, a stack trace from a core file will help. Unfortunately it seems not to have. But, good catch on the pointer value there being ASCII data. Let me know if you have any other ideas. - Chris
Re: Auth SEGV on sparc64, alignment problem?
(long gdb output, you’ve been warned)
Okay. So, the libdovecot shared library in /usr/local was stripped.
Replaced that, and got farther. gdb walk below.
It looks to me like it gets deep into the OS’s vfork/execv where it catches a
trap/crashes. Is this a problem I can catch, or something wrong with running
in gdb? I notice this is a SIGTRAP, where the binary when run out of gdb gets
a SIGSEGV, and that’s what a loaded core shows.
Thanks for any assistance.
- Chris
Breakpoint 3, master_service_exec_config (service=0x4103,
input=0x7fdf5a8) at master-service-settings.c:125
125 const char **conf_argv, *binary_path = service->argv[0];
(gdb) n
128 (void)t_binary_abspath(&binary_path);
(gdb) n
130 if (!service->keep_environment && !input->preserve_environment)
{
(gdb)
131 if (input->preserve_home)
(gdb)
133 if (input->preserve_user)
(gdb)
135 if ((service->flags & MASTER_SERVICE_FLAG_STANDALONE)
!= 0)
(gdb)
136
master_service_import_environment("LOG_STDERR_TIMESTAMP");
(gdb)
140 if (getenv(DOVECOT_PRESERVE_ENVS_ENV) == NULL)
(gdb)
146 if (input->use_sysexits)
(gdb)
150 i = 0;
(gdb)
151 argv_max_count = 11 + (service->argc + 1) + 1;
(gdb)
152 conf_argv = t_new(const char *, argv_max_count);
(gdb)
153 conf_argv[i++] = DOVECOT_CONFIG_BIN_PATH;
(gdb)
154 if (input->service != NULL) {
(gdb)
158 conf_argv[i++] = "-c";
(gdb)
159 conf_argv[i++] = service->config_path;
(gdb)
160 if (input->module != NULL) {
(gdb)
161 conf_argv[i++] = "-m";
(gdb)
162 conf_argv[i++] = input->module;
(gdb)
163 if (service->want_ssl_settings) {
(gdb)
168 if (input->parse_full_config)
(gdb)
171 conf_argv[i++] = "-e";
(gdb)
172 conf_argv[i++] = binary_path;
(gdb)
173 memcpy(conf_argv+i, service->argv + 1,
(gdb)
175 i += service->argc;
(gdb)
177 i_assert(i < argv_max_count);
(gdb)
178 execv_const(conf_argv[0], conf_argv);
(gdb) p conf_argv
$3 = (const char **) 0x41016e48
(gdb) p conf_argv[0]
$4 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) p *conf_argv
$5 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) s
execv_const (path=0x4064f6d8 "/usr/local/bin/doveconf", argv=0x41016e48)
at execv-const.c:23
23 (void)execv(path, argv_drop_const(argv));
(gdb) p parth
No symbol "parth" in current context.
(gdb) p path
$6 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) s
argv_drop_const (argv=0x41016e48) at execv-const.c:13
13 for (count = 0; argv[count] != NULL; count++) ;
(gdb) p argv
$7 = (const char * const *) 0x41016e48
(gdb) p argv[0]
$8 = 0x4064f6d8 "/usr/local/bin/doveconf"
(gdb) p argv[1]
$9 = 0x4064f708 "-c"
(gdb) p argv[2]
$10 = 0x4104 "/usr/local/etc/dovecot/dovecot.conf"
(gdb) p argv[3]
$11 = 0x4064f710 "-m"
(gdb) p argv[4]
$12 = 0x16ad70 "auth"
(gdb) p argv[5]
$13 = 0x4064f728 "-e"
(gdb) p argv[6]
$14 = 0x7fdfd18
"/usr/ports/mail/dovecot/work/stage/usr/local/libexec/dovecot/auth"
(gdb) p argv[7]
$15 = 0x0
(gdb) n
15 ret = t_new(char *, count + 1);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
17 ret[i] = t_strdup_noconst(argv[i]);
(gdb)
16 for (i = 0; i < count; i++)
(gdb)
18 return ret;
(gdb)
19 }
(gdb)
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb) b argv_drop_const
Breakpoint 4 at 0x405d50b8: file execv-const.c, line 13.
(gdb) run
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Re: Auth SEGV on sparc64, alignment problem?
Okay. Got to the next bit pretty quickly.:
Breakpoint 4, auth_settings_read (service=0x0, pool=0x4104b020,
output_r=0x7fdf6d0) at auth-settings.c:522
522 input.module = "auth";
(gdb) n
523 input.service = service;
(gdb) n
524 if (master_service_settings_read(master_service, &input,
(gdb) s
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb)
So, why did it not step into master_service_settings_read ? Trying again:
523 input.service = service;
(gdb) s
524 if (master_service_settings_read(master_service, &input,
(gdb) list
519
520 i_zero(&input);
521 input.roots = set_roots;
522 input.module = "auth";
523 input.service = service;
524 if (master_service_settings_read(master_service, &input,
525 output_r, &error) < 0)
526 i_fatal("Error reading configuration: %s", error);
527
528 pool_ref(pool);
(gdb) p input
$1 = {roots = 0x27fbd8, config_path = 0x0, preserve_environment = false,
preserve_user = false, preserve_home = false, never_exec = false,
use_sysexits = false, parse_full_config = false, module = 0x16ad70 "auth",
service = 0x0, username = 0x0, local_ip = {family = 0, u = {ip6 = {
__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0,
0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {
s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = {
__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0,
0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}},
local_name = 0x0}
(gdb) p &input
$2 = (struct master_service_settings_input *) 0x7fdf5a8
(gdb) p output_r
$3 = (struct master_service_settings_output *) 0x7fdf6d0
(gdb) p &error
$4 = (const char **) 0x7fdf598
(gdb) p error
$6 = 0x10dbd0 "@\005?\204\001"
(gdb) p master_service
$5 = (struct master_service *) 0x4103
(gdb) s
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb)
Any ideas here? I’m not sure where to look next…
- Chris
> On Feb 22, 2018, at 10:10, Chris Ross wrote:
>
> Fancy, while not fun. :-) But thanks, that does work. Doing that, n’ing
> over calls to strcmp, it failed:
>
> passdbs_init () at passdb.c:313
> 313 passdb_register_module(&passdb_ldap);
> (gdb)
> passdb_register_module (iface=0x280120) at passdb.c:33
> 33old_iface = passdb_interface_find(iface->name);
> (gdb)
> passdb_interface_find (name=0x16fe60 "ldap") at passdb.c:20
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (strcmp(iface->name, name) == 0)
> (gdb) n
> 20array_foreach(&passdb_interfaces, ifaces) {
> (gdb)
> 21struct passdb_module_interface *iface = *ifaces;
> (gdb)
> 23if (st
Re: Auth SEGV on sparc64, alignment problem?
Fancy, while not fun. :-) But thanks, that does work. Doing that, n’ing
over calls to strcmp, it failed:
passdbs_init () at passdb.c:313
313 passdb_register_module(&passdb_ldap);
(gdb)
passdb_register_module (iface=0x280120) at passdb.c:33
33 old_iface = passdb_interface_find(iface->name);
(gdb)
passdb_interface_find (name=0x16fe60 "ldap") at passdb.c:20
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
21 struct passdb_module_interface *iface = *ifaces;
(gdb)
23 if (strcmp(iface->name, name) == 0)
(gdb) n
20 array_foreach(&passdb_interfaces, ifaces) {
(gdb)
26 return NULL;
(gdb)
27 }
(gdb)
passdb_register_module (iface=0x280120) at passdb.c:34
34 if (old_iface != NULL && old_iface->verify_plain == NULL) {
(gdb)
37 } else if (old_iface != NULL) {
(gdb)
41 array_append(&passdb_interfaces, &iface, 1);
(gdb)
42 }
(gdb)
passdbs_init () at passdb.c:314
314 passdb_register_module(&passdb_sql);
(gdb)
315 passdb_register_module(&passdb_sia);
(gdb)
316 passdb_register_module(&passdb_static);
(gdb)
317 passdb_register_module(&passdb_oauth2);
(gdb)
318 }
(gdb)
main_preinit () at main.c:186
186 userdbs_init();
(gdb)
188 password_schemes_init();
(gdb)
190 services = read_global_settings();
(gdb)
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb)
Cannot find bounds of current function
(gdb)
Next step I’ll stop before that and be more careful about n’ing things, but.
Just passing on context while I have it.
Thanks. More later.
- Chris
> On Feb 22, 2018, at 02:25, Aki Tuomi wrote:
>
> Hi!
>
> Unfortunately we do not have a Sparc64 with any OS at hand. Maybe you could
>
> break main
> r
> s
>
> until it breaks?
>
> Aki
>
Re: Auth SEGV on sparc64, alignment problem?
Sadly, that doesn’t help either. Over the past day, I’ve built and installed
a different branch of the OS (stable/11, instead of release/11.1), to see if a
new compiler/libc might change things. Sadly, it does not.
In the same situation now, auth fails immediately with signal 11. Running
gdb on auth (from build dir, compiled -g -O2) shows something similar.
- Chris
# gdb work/dovecot-2.2.33.2/src/auth/.libs/auth
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "sparc64-marcel-freebsd"...
(gdb) list
372 /* ask auth master to disconnect us */
373 auth_worker_client_send_shutdown();
374 }
375 }
376
377 int main(int argc, char *argv[])
378 {
379 int c;
380
381 master_service = master_service_init("auth", 0, &argc, &argv,
"w");
(gdb) run
Starting program:
/usr/ports/mail/dovecot/work/dovecot-2.2.33.2/src/auth/.libs/auth
Program received signal SIGTRAP, Trace/breakpoint trap.
Cannot remove breakpoints because program is no longer writable.
It might be running in another process.
Further execution is probably impossible.
0x4022a380 in ?? ()
(gdb) bt
#0 0x4022a380 in ?? ()
#1 0x0008 in ?? ()
Previous frame identical to this frame (corrupt stack?)
(gdb)
> On Feb 21, 2018, at 02:01, Aki Tuomi wrote:
>
> Your core dump looks a bit broken. Since it seems to die instantly, can
> you try gdb /path/to/auth and just run it?
>
> Aki
Auth SEGV on sparc64, alignment problem?
Apologies first for using two addresses, but I can’t currently read my email
at distal.com. :-)
I was previously running dovecot2-2.2.29.1_2 on FreeBSD 11 on sparc64.
Trying to debug a problem I was having with one of my clients, I upgraded to
dovecot-2.2.33.2_4 on that same server. However, I cannot connect now, log
shows:
Feb 20 16:55:00 westeros dovecot: master: Dovecot v2.2.33.2 (d6601f4ec)
starting up for imap, pop3, lmtp
Feb 20 16:55:31 westeros dovecot: auth: Fatal: master: service(auth): child
25395 killed with signal 11 (core dumped)
Feb 20 16:55:31 westeros dovecot: master: Error: service(auth): command startup
failed, throttling for 2 secs
Feb 20 16:55:31 westeros dovecot: imap-login: Disconnected: Auth process broken
(disconnected before auth was ready, waited 0 secs): user=<>, rip=2001::xxx,
lip=2001:470:e24c:200::ae25, TLS handshaking, session=
Feb 20 16:55:33 westeros dovecot: auth: Fatal: master: service(auth): child
25398 killed with signal 11 (core dumped)
Feb 20 16:55:33 westeros dovecot: master: Error: service(auth): command startup
failed, throttling for 4 secs
Feb 20 16:55:33 westeros dovecot: imap-login: Disconnected: Auth process broken
(disconnected before auth was ready, waited 2 secs): user=<>, rip=2001::xxx,
lip=2001:470:e24c:200::ae25, session=
Feb 20 16:55:37 westeros dovecot: master: Error: service(auth): command startup
failed, throttling for 8 secs
Feb 20 16:55:37 westeros dovecot: auth: Fatal: master: service(auth): child
25400 killed with signal 11 (core dumped)
Loading the core file, as described https://www.dovecot.org/bugreport.html ,
shows the error in libc somewhere:
(gdb) bt full
#0 __unaligned_load (
p=0x617070656e640e6d , size=4)
at /usr/src/release-11.1.0/lib/libc/sparc64/sys/__sparc_utrap_align.c:45
val = 0
i = 0
#1 0x109f9f6c in __unaligned_fixup (uf=0x7fdee40)
at /usr/src/release-11.1.0/lib/libc/sparc64/sys/__sparc_utrap_align.c:78
addr =
val =
insn = 3254807616
sig =
#2 0x109f9d50 in __sparc_utrap (uf=0x7fdee40)
at /usr/src/release-11.1.0/lib/libc/sparc64/sys/__sparc_utrap.c:100
sig = 272013984
#3 0x1094a10c in __sparc_utrap_gen () from /lib/libc.so.7
No symbol table info available.
#4 0x1094a10c in __sparc_utrap_gen () from /lib/libc.so.7
No symbol table info available.
Previous frame identical to this frame (corrupt stack?)
(gdb)
As this is a sparc64, with 8-byte alignment requirements, I’m guessing that’s
the issue. Many a piece of software has failed to respect that and crashed.
But, I’m not sure.
Does anyone have any suggestions? I’ve built it locally (via ports), so if
there are compiler options I can/should try, I certainly can try.
Thanks…
- Chris
Transitioning away from mail_location = maildir:~
Hi,
I have been using this setup for years:
mail_home = /var/mail/vhosts/%d/%n
mail_location = maildir:~
I have since learned that mail_home and mail_location should be
different. I plan to use this:
mail_home = /var/mail/vhosts/%d/%n
mail_location = maildir:~/mail
I would like the transition to be transparent for my email clients. I
don't want mail/directories/sieve_scripts to disappear.
1. Should I manually create the maildir:~/mail directories?
2. Should I move files from /var/mail/vhosts/%d/%n into maildir:~/mail?
Which files? (In other words: which files are "home directory" files
and which files are "mail files"?)
Thank you,
Chris
Example home directory:
drwx-- .
drwxr-xr-x ..
drwx-- .Archives
drwx-- .Archives.2016
drwx-- .Archives.2017
drwx-- .Drafts
drwx-- .Junk
drwx-- .Notes
drwx-- .Sent
drwx-- .Sent
drwx-- .Trash
drwx-- .work
drwx-- .home
drwx-- .todo
lrwx-- .dovecot.sieve
-rw--- .dovecot.sieve.log
-rw--- .dovecot.sieve.log.0
-rw--- .dovecot.svbin
drwx-- cur
-rw--- dovecot-keywords
-rw--- dovecot-uidlist
-rw--- dovecot-uidvalidity
-r--r--r-- dovecot-uidvalidity.55411048
-rw--- dovecot.index
-rw--- dovecot.index.cache
-rw--- dovecot.index.log
-rw--- dovecot.mailbox.log
drwx-- new
drwx-- sieve
-rw--- subscriptions
drwx-- tmp
$ dovecot -n
# 2.2.27 (c0f36b0): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: FreeBSD 10.3-RELEASE-p11 amd64
lmtp_save_to_detail_mailbox = yes
mail_gid = 1002
mail_home = /var/mail/vhosts/%d/%n
mail_location = maildir:~
mail_privileged_group = vpostfix
mail_uid = 1002
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart extracttext
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
plugin {
recipient_delimiter = -
sieve = file:~/sieve;active=~/.dovecot.sieve
}
protocols = imap pop3 lmtp sieve
recipient_delimiter = -
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = postfix
mode = 0600
user = postfix
}
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0666
user = postfix
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
ssl_cert = xxx
ssl_key = xxx
userdb {
args = username_format=%u /usr/local/etc/dovecot/users
driver = passwd-file
}
protocol lmtp {
mail_plugins = " sieve"
postmaster_address = xxx
}
protocol lda {
mail_plugins = " sieve"
}
Re: Softlinks
Steffen Kaiser wrote: > On Wed, 28 Sep 2016, Chris wrote: >> I'm using Maildir. Is it possible to move all (or only some) maildirs >> with >> softlinks to another partition? > > I did this, but be prepared that you have left-overs when the user deletes > the mailbox (mail folder). Ok, thank you. I would just link top-level folders, e.g. /var/vmail/users/userA -> /data/vmail2/users/userA for some extra big mailboxes. - Chris
Softlinks
All, I'm using Maildir. Is it possible to move all (or only some) maildirs with softlinks to another partition? - Chris
Re: Panic: file auth-request.c
From: Tanstaafl To: Sent: 19/09/2016 5:44 PM Subject: Re: Panic: file auth-request.c On 9/17/2016 2:15 PM, Chris Wik wrote: > So we upgraded to a new CentOS 7 server with SSD RAID, fast CPUs and > tons of RAM. No more load problems. We compiled the latest dovecot > from source (as the version from CentOS yum repo is already quite > old, figure we might as well run the latest version since we were > upgrading anyway). Then on 9/18/2016 6:50 AM, Chris Wik wrote: > In my local source of 2.2.5, ??? Latest dovecot version is 2.2.25 - or was that (hopefully) a typo? http://www.dovecot.org/download.html Yes, typo, sorry. I have 2.2.25 sources and the line numbers don't match the diff. We'll wait for 2.2.26, unless someone from Dovecot would like us to test the patch? In which case I'll try removing the 2 lines and recompiling and see if it works. Chris
Re: Panic: file auth-request.c
From: Aki Tuomi To: Dovecot Mailing List , Chris Wik Sent: 18/09/2016 8:06 AM Subject: Re: Panic: file auth-request.c > Sep 17 19:34:57 mail dovecot: auth: Panic: file auth-request.c: line 1049 > (auth_request_lookup_credentials): assertion failed: > (request->credentials_scheme == scheme) > Sep 17 19:34:57 mail dovecot: auth: Error: Raw backtrace: > /usr/local/lib/dovecot/libdovecot.so.0(+0x89470) [0x7fa9cb8af470] -> > /usr/local/lib/dovecot/libdovecot.so.0(+0x8954e) [0x7fa9cb8af54e] -> > /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa9cb851f75] -> > dovecot/auth() [0x4165bc] -> dovecot/auth() [0x4221fb] -> dovecot/auth() > [0x41620b] -> dovecot/auth(auth_request_lookup_credentials_callback+0x58) > [0x4162f8] -> dovecot/auth(passdb_handle_credentials+0x6a) [0x4254ba] -> > dovecot/auth() [0x425b62] -> dovecot/auth() [0x41c1f8] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fa9cb8c207c] > -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xd7) > [0x7fa9cb8c3377] -> > /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7fa9cb8c2105] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7fa9cb8c22b8] -> > /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7fa9cb857f33] -> dovecot/auth(main+0x2eb ) [0x40ccdb] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fa9c9dc2b15] -> dovecot/auth() [0x40cf15] Hi! This has been fixed with https://github.com/dovecot/core/commit/6c969ac21a43cc10ee1f1a91a4f39e4864c886cb Aki Tuomi Dovecot oy Great, good to hear! In my local source of 2.2.5, the deleted lines are lines 1048-1049. In the patch the lines are 1068-1069. I think maybe we'll wait for 2.2.6 and not try to patch it ourselves, we aren't using the new features in 2.2.5 yet and 2.2.4 has been stable for us... Chris -- Chris Wik Anu Internet Services www.anu.net | www.cwik.ch
Panic: file auth-request.c
erdb {
args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
verbose_ssl = yes
protocol lmtp {
mail_fsync = optimized
}
protocol lda {
mail_fsync = optimized
}
protocol imap {
mail_max_userip_connections = 50
}
protocol pop3 {
mail_max_userip_connections = 20
}
--
Chris Wik
Anu Internet Services
www.anu.net | www.cwik.ch
news page not been updated
The url http://www.dovecot.org/doc/NEWS has not been updated for a while, I suggest either killing it or updating it again. Chris
Re: Dovecot/Rainloop 2.0.13-1_129.el5
> On 5 May 2016, at 21:03, Edgar Pettijohn wrote: > > > > On 05/05/16 14:40, Chris Smith wrote: >> I configured Dovecot on our mail server under Centos 5.3 (I think) some >> while ago now (about 2 years) and, to the best of my knowledge,it had been >> working correctly all that time. >> >> However, one of the team wanted to use webmail while away from base and >> found that, although he could receive emails OK, he was unable to reply or >> send fresh emails. When he pressed Send he got a message “Authentication >> failed" in a pop-up box. >> >> He assured me that sending mail had been working and I was certain that I >> had tested that aspect when I was setting Dovecot up, at least for local >> mail. >> >> I tried to send email from my Dovecot account and had the same problem. >> >> Our MTA is sendmail and that is not configured to authenticate on outgoing >> mail, The configuration for sendmail hasn’t changed at least since the last >> update in 16 March 2015 (sendmail 8.13.8-10.el5_11). >> >> Dovecot uses php-5.5.5 in this set up. >> >> The relevant entry in /var/log/maillog when I attempt to send an email from >> Dovecot is: >> >> May 5 16:34:29 firewall dovecot: auth: Debug: auth client connected >> (pid=13001) >> May 5 16:34:29 firewall dovecot: auth: Debug: client in: AUTH 1 >> PLAIN service=imapsecured lip=127.0.0.1 rip=127.0.0.1 lport=143 >>rport=51861 resp=AGNocmlzADB1dFkwdUcwNDkh >> May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): lookup >> service=dovecot >> May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): #1/1 >> style=1 msg=Password: >> May 5 16:34:29 firewall dovecot: auth: Debug: client out: OK 1 >> user=chris >> May 5 16:34:29 firewall dovecot: auth: Debug: master in: REQUEST >> 2607546369 13001 1 0bcf2e3a108cd9cd18eaff4b7de9c428 >> May 5 16:34:29 firewall dovecot: auth: Debug: passwd(chris,127.0.0.1): >> lookup >> May 5 16:34:29 firewall dovecot: auth: Debug: master out: USER >> 2607546369 chris system_groups_user=chrisuid=514 gid=100 >> home=/home/chris >> May 5 16:34:29 firewall dovecot: imap-login: Login: user=, >> method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=13002, secured >> May 5 16:34:29 firewall dovecot: imap(chris): Disconnected: Logged out >> bytes=11/334 >> May 5 16:34:29 firewall sendmail[13003]: u45FYTtd013003: >> localhost.localdomain [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during >> connection to MTA > Looks like a rainloop issue. Apparently it isn't speaking to sendmail > correctly. You are quite correct. I had misconfigured the domain to require authentication. All is working now. Many thanks > > >> >> >> The webmail server and our mail server are the same system. The firewall is >> open for port 143. >> >> It is only send from Dovecot that fails authentication, all other outgoing >> mail is sent correctly. >> >> Does anyone have any idea where else I can look for clues as to why there is >> this behaviour? >> >> Suplementary question: what does the “Sign me” check box on the webmail log >> in page do? >> >> Many thanks. >> >> Chris
Dovecot/Rainloop 2.0.13-1_129.el5
I configured Dovecot on our mail server under Centos 5.3 (I think) some while ago now (about 2 years) and, to the best of my knowledge,it had been working correctly all that time. However, one of the team wanted to use webmail while away from base and found that, although he could receive emails OK, he was unable to reply or send fresh emails. When he pressed Send he got a message “Authentication failed" in a pop-up box. He assured me that sending mail had been working and I was certain that I had tested that aspect when I was setting Dovecot up, at least for local mail. I tried to send email from my Dovecot account and had the same problem. Our MTA is sendmail and that is not configured to authenticate on outgoing mail, The configuration for sendmail hasn’t changed at least since the last update in 16 March 2015 (sendmail 8.13.8-10.el5_11). Dovecot uses php-5.5.5 in this set up. The relevant entry in /var/log/maillog when I attempt to send an email from Dovecot is: May 5 16:34:29 firewall dovecot: auth: Debug: auth client connected (pid=13001) May 5 16:34:29 firewall dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imapsecured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=51861 resp=AGNocmlzADB1dFkwdUcwNDkh May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): lookup service=dovecot May 5 16:34:29 firewall dovecot: auth: Debug: pam(chris,127.0.0.1): #1/1 style=1 msg=Password: May 5 16:34:29 firewall dovecot: auth: Debug: client out: OK 1 user=chris May 5 16:34:29 firewall dovecot: auth: Debug: master in: REQUEST 2607546369 13001 1 0bcf2e3a108cd9cd18eaff4b7de9c428 May 5 16:34:29 firewall dovecot: auth: Debug: passwd(chris,127.0.0.1): lookup May 5 16:34:29 firewall dovecot: auth: Debug: master out: USER 2607546369 chris system_groups_user=chrisuid=514 gid=100 home=/home/chris May 5 16:34:29 firewall dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=13002, secured May 5 16:34:29 firewall dovecot: imap(chris): Disconnected: Logged out bytes=11/334 May 5 16:34:29 firewall sendmail[13003]: u45FYTtd013003: localhost.localdomain [127.0.0.1] did not issue MAIL/EXPN/VRFY/ETRN during connection to MTA The webmail server and our mail server are the same system. The firewall is open for port 143. It is only send from Dovecot that fails authentication, all other outgoing mail is sent correctly. Does anyone have any idea where else I can look for clues as to why there is this behaviour? Suplementary question: what does the “Sign me” check box on the webmail log in page do? Many thanks. Chris
speedup doveadm
All, I've to set ACLs in public namespace for more than 5000 folders. That takes some hours. I'm calling doveadm by a perl script. Is there any way to speedup doveadm? Is it possible to set ACLs for multiple folders or users in a single call? - Chris
Re: Problem setting owner
Luca Bertoncello wrote:
> I already added:
>
> mail_uid = 1005
> mail_gid = 8
>
> in 10-mail.conf and:
>
> user_attrs = \
> =user=exim, \
> =uid=1005, \
> =gid=8, \
> =quota_rule=*:bytes=%{ldap:quotaBytes}, \
> =home=/home/mailboxes/%{ldap:sAMAccountName}/Maildir, \
> =mail=maildir:/home/mailboxes/%{ldap:sAMAccountName}/Maildir
>
> in the dovecot-ldap.conf.ext, but it seems to ignore them...
Have you tried override_fields?
- Chris
Re: fts_solr: Lookup failed: 400 Bad Request / GET null null
On Wed, Apr 6, 2016 at 9:27 PM, Timo Sirainen wrote:
>> The total request size is 31708 bytes and it contains many (hundreds?)
>> of 'OR hdrs' (side note: I wonder which client action triggers these
>> kind of requests, maybe the user selected hundreds of mails for
>> search?)
>
> I bet this is the weird iOS client stupidity where they for some weird reason
> started issuing commands like:
>
> SEARCH OR HEADER Message-ID id1 OR HEADER Message-ID id2 OR HEADER Message-ID
> id3 ...
>
> With the entire command about ~32 kB. It does it for every single message in
> the folder. Why not simply FETCH 1:* HEADER.FIELDS[Message-ID] and do the
> matching itself..
>
>> I _think_ this is a problem of the URL length / max http header size.
>> (Debian Jessie) Tomcat7 very likely does not accept more than 32kb
>> data in a request.
>>
>> I wonder if Dovecot should limit SOLR requests to a specific size and
>> deny long requests with an imap error (?)
>
> Or just issue multiple Solr requests.. In any case, troublesome.. Could those
> limits be just increased in Tomcat?
>
I've set maxHttpHeaderSize="65536" ("Connector"-definition in
/etc/tomcat7/server.xml) and at least during the last two days no
error occured anymore :-)
Chris
Re: fts_solr: Lookup failed: 400 Bad Request / GET null null
On Wed, Apr 6, 2016 at 9:27 PM, Timo Sirainen wrote:
> On 05 Apr 2016, at 11:33, Chris Laif wrote:
>>
>> I've captured some requests and they look like this (some parts
>> changed due to privacy concerns):
>>
>> GET
>> /solr/select?fl=uid,score&rows=439&sort=uid+asc&q={!lucene+q.op%3dAND}hdr:%3c56f3a6.7080...@domain.de%3e+OR+hdr:%3c664dcdx1a4facd8b7922c495...@czchows1356.prg%5c-domain.com%3e+OR+hdr:%3c00cxde3$70ad7880$52
>> ... (many many more OR hdrs)
>> &fq=%2Bbox:f696f93xx6e+%2Buser:u...@domain.de HTTP/1.1
>>
>> The total request size is 31708 bytes and it contains many (hundreds?)
>> of 'OR hdrs' (side note: I wonder which client action triggers these
>> kind of requests, maybe the user selected hundreds of mails for
>> search?)
>
> I bet this is the weird iOS client stupidity where they for some weird reason
> started issuing commands like:
>
> SEARCH OR HEADER Message-ID id1 OR HEADER Message-ID id2 OR HEADER Message-ID
> id3 ...
>
> With the entire command about ~32 kB. It does it for every single message in
> the folder. Why not simply FETCH 1:* HEADER.FIELDS[Message-ID] and do the
> matching itself..
>
>> I _think_ this is a problem of the URL length / max http header size.
>> (Debian Jessie) Tomcat7 very likely does not accept more than 32kb
>> data in a request.
>>
>> I wonder if Dovecot should limit SOLR requests to a specific size and
>> deny long requests with an imap error (?)
>
> Or just issue multiple Solr requests.. In any case, troublesome.. Could those
> limits be just increased in Tomcat?
>
Re: fts_solr: Lookup failed: 400 Bad Request / GET null null
On Sun, Apr 3, 2016 at 4:19 PM, Stephan Bosch wrote:
>> Unfortunately I do not know (yet) which client action produces these
>> log lines. "GET null null" seems not to be a sensible request in any
>> case.
>>
>> Anyone seeing the same effect?
>
>
> Did this happen only after upgrade to 2.2.23 or is this a fresh install?
>
This effect has been observed with 2.2.18 (initial install) and after
upgrading to 2.2.23 as well.
> Can you make a dump of what messages are exchanged on 10.0.0.123:8080 using
> ngrep or wireshark?
>
I've captured some requests and they look like this (some parts
changed due to privacy concerns):
GET
/solr/select?fl=uid,score&rows=439&sort=uid+asc&q={!lucene+q.op%3dAND}hdr:%3c56f3a6.7080...@domain.de%3e+OR+hdr:%3c664dcdx1a4facd8b7922c495...@czchows1356.prg%5c-domain.com%3e+OR+hdr:%3c00cxde3$70ad7880$52
... (many many more OR hdrs)
&fq=%2Bbox:f696f93xx6e+%2Buser:u...@domain.de HTTP/1.1
The total request size is 31708 bytes and it contains many (hundreds?)
of 'OR hdrs' (side note: I wonder which client action triggers these
kind of requests, maybe the user selected hundreds of mails for
search?)
I _think_ this is a problem of the URL length / max http header size.
(Debian Jessie) Tomcat7 very likely does not accept more than 32kb
data in a request.
I wonder if Dovecot should limit SOLR requests to a specific size and
deny long requests with an imap error (?)
Chris
fts_solr: Lookup failed: 400 Bad Request / GET null null
Hi, the log file shows suspicious entries as follows: Apr 3 09:14:49 xxx dovecot: imap(user@domain): Error: fts_solr: Lookup failed: 400 Bad Request The tomcat7 log shows for this request: 10.0.0.234 - - [03/Apr/2016:09:14:49 +0200] "GET null null" 400 - Dovecot is latest 2.2.23 with a pretty standard FTS config copied from the wiki: mail_plugins = " acl fts fts_solr" fts = solr fts_solr = break-imap-search url=http://10.0.0.123:8080/solr/ Unfortunately I do not know (yet) which client action produces these log lines. "GET null null" seems not to be a sensible request in any case. Anyone seeing the same effect? Chris
Deliver to Public Mailbox
Dear All, I'd like dovecot-lda (deliver) to post a message to a public mailbox. It doesn't seem to work when I just use the -m parameter with the IMAP-Mailbox path, e.g. Public/info . What parameters are required? Do I have to change the headers with formail in any way? TIA! - Chris
Block public namespace mail when quota exceeded
Dear All, is it possible to temporarily reject mails, when quota of public namespace subfolder is exceeded? I'd check folder size with a cronjob. Dovecot is using a system user, so I guess blocking post and insert in ACL isn't a solution? I don't want to set the folder to read-only, that users can still delete mails. Thank you in advance. - Chris
Script dovecot ACLs, Quota and doveadm
Dear All, I'd like to set a) Quota for mailboxes in private and public namespace b) ACLs for both. What's the easiest way to do this? Do I have to call doveadm for every mailbox (private and public)? Is it required to provide a username to doveadm? Can I use the same user for all if it is defined in global ACLs? For Cyrus there are perl modules which have the same functions as cyradm. Is there something like this for dovecot? Are there any modules available? Haven't found anything in CPAN. - Chris
Re: shared folders not working with . separator
Hugh Bragg wrote: > I've been trying this for weeks and never managed to get my mail client > to see shared folders. > Apparently imap_acl works fine as I can see these entries in the > dovecot-acl-list files and the correct entries persist in the client. > This is my first dovecot setup so I hope I've configured it right. > Everything else works beautifully, thanks. Are they shown with telnet? e.g. http://www.anta.net/misc/telnet-troubleshooting/imap.shtml
IMAP ACLs for groups
Dear All, is there any way to assign POSIX groups to Dovecot IMAP-ACLs? I've tried that today, but I could only open a public folder when my username was listed in the ACL. A group didn't work. How do you manage ACLs for intenseley used public folders with many users? What backend do you use for user management and ACLs? Is there any way to extend permissions? Are there any scripts available that set permissions or grant them when logging in? - Chris
Re: ACL and LDAP
Hi Daniel, > in doveadm_set_mailbox_acls() and imap_set_mailbox_acls() the attributes > are read and used to set them on the folders. thank you. That's interesting. They're calling doveadm directly. I probably would have tried to use an IMAP-Client library. The former seems easier. - Chris
Re: migrating servers
I worked out what the problem was Because I had been planning this move for several months I was syncing home user directories including Maildir for several months unfortunately I forgot the --delete command to rsync so while users were deleting emails on the original server they were often being copied to the destination server before being deleted. Therefore there where many more emails in the folder than expected. many quite old. by resyncing with the --delete command I was able to bring the source and destination into proper sync and all the extra emails were cleaned up Of course the pop email accounts still had the duplicates I have had to advise the user to clean out there email folders and apologised for the inconvenience. Chris On 22/08/2015 5:42 PM, chris wrote: Hi I am trying to migrate from a Centos dovecot 1.0.7 install to a debian dovecot 2.1.7 install. I thought simply replicating the maildirs would be fine (using rsync) but when I fetch mail from the new server using IMAP thunderbird wants to download all the messages again as duplicates. I searched the wiki over to see if I could find a solution and while it appeared I could use doveadm backup that seems to only work pushing changes from the new server to the old (wrong direction) and that program does not exist on the earlier version of dovecot on the old server. If I can be very blunt the documentation on the wiki assumes far too much knowledge from the reader. It is often hard to decipher exactly what is meant by many of the obtuse instructions. But that is an aside can anyone give me pointers on how to do this migration for all my users so they don't need to sort through hundreds of duplicated emails? Chris
migrating servers
Hi I am trying to migrate from a Centos dovecot 1.0.7 install to a debian dovecot 2.1.7 install. I thought simply replicating the maildirs would be fine (using rsync) but when I fetch mail from the new server using IMAP thunderbird wants to download all the messages again as duplicates. I searched the wiki over to see if I could find a solution and while it appeared I could use doveadm backup that seems to only work pushing changes from the new server to the old (wrong direction) and that program does not exist on the earlier version of dovecot on the old server. If I can be very blunt the documentation on the wiki assumes far too much knowledge from the reader. It is often hard to decipher exactly what is meant by many of the obtuse instructions. But that is an aside can anyone give me pointers on how to do this migration for all my users so they don't need to sort through hundreds of duplicated emails? Chris
Store ACL files within shared mailbox / rename shared folder problem
Hi. If $otheruser shares a mailbox to $myuser, my private index is stored in *my* Maildir, according to this template (from the dovecot docs): mail_location = maildir:%%h/Maildir:INDEXPVT=%h/Maildir/shared/%%u If $otheruser renames the shared mailbox, all my private mail flags (Seen) are lost because my mailclient couldn't find the private index anymore. Is there any way to store the private index within the shared mailbox? If $otheruser renames the mailbox, the index will be 'migrated' as well. Something like this: /data/mails/$otheruser/Maildir/.MagicSharedFolder/shared/$myuser.dovecot.index.pvt.log I see no way on how to do craft a INDEXPVT for this. Kind regards, Chris
Re: sharing INBOX with ACL -> share all folders
> On Jul 28, 2015, at 05:13, Marco Giunta wrote: > > Hi at all, > I have a problem with ACL; I want to share INBOX and Sent folder to an other > user, but when I configure ACL on INBOX, all folders are shared (Sent, Junk, > Draft, Trash, etc) Hello, Marco. Unfortunately I don’t know why you are seeing the behavior you are, and hope that someone else will be able to help. However, you seem to have accomplished something I’m wanting to do, and have as yet been unable to get working. I have a Users INBOX that I want to share to other users, but something is wrong with the way I’ve configured ACLs and sharing. Perhaps we could discuss off-list more of what your configuration looks like, and how you got there? I’m running on FreeBSD with the ports system version of dovecot2 2.2.16, currently, although I think I’m due an upgrade. You say you’re have "My Dovecot instance use a single user”, and I think that’s different than I. My Maildir directories and files are all owned by the UNIX user that owns the file. Maybe this is causing me the permissions problems I’m seeing. Is having it all running as one [UNIX] user a typical configuration for dovecot2? Or just typical of installations using ACLs? Thank you. - Chris signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Deafness
> On Jul 28, 2015, at 21:52 , Steffan Cline wrote: > > Ok, I think I have come a little further. > > When dovecot stops accepting connections, I checked netstat and found this: > > [root@hosting1 ~]# netstat -an | grep 993 > tcp0 0 0.0.0.0:993 0.0.0.0:* > LISTEN > tcp0 0 65.39.x.x:993184.101.x.x:36351 > SYN_RECV > tcp0 0 65.39.x.x:993107.212.x.x:51487 > SYN_RECV > tcp0 0 65.39.x.x:993107.212.x.x:51488 > SYN_RECV > tcp0 0 65.39.x.x:993184.101.x.x:44650 > SYN_RECV > > This told me it wasn’t too many connections causing dovecot to be > unresponsive. So then I tried via telnet. > > Dovecot seems to accept connections but then just sits there and does > nothing. I used the appropriate commands to try and initiate a login but > nothing happens. Typing any commands at all produce no response from dovecot. Actually, I think the above shows that it’s not a dovecot problem. A socket in a SYN_RECV state means that a connection request has been merely been received from the network. That means your kernel has not finished establishing the TCP connection, so dovecot (or the application level in general) is likely not even involved yet. I would suspect some sort of firewall config on your host, or perhaps some sort of overload at the network stack level. But, the latter only if the server were very heavily loaded. I hope this feedback is helpful. - Chris
Re: sharing INBOX with ACL -> share all folders
[- resend to the list from my list address -]
> On Jul 28, 2015, at 05:13, Marco Giunta wrote:
>
> Hi at all,
> I have a problem with ACL; I want to share INBOX and Sent folder to an other
> user, but when I configure ACL on INBOX, all folders are shared (Sent, Junk,
> Draft, Trash, etc)
Hello, Marco. Unfortunately I don’t know why you are seeing the behavior you
are, and hope that someone else will be able to help.
However, you seem to have accomplished something I’m wanting to do, and have
as yet been unable to get working. I have a Users INBOX that I want to share
to other users, but something is wrong with the way I’ve configured ACLs and
sharing.
Perhaps we could discuss off-list more of what your configuration looks like,
and how you got there? I’m running on FreeBSD with the ports system version of
dovecot2 2.2.16, currently, although I think I’m due an upgrade.
You say you’re have "My Dovecot instance use a single user”, and I think
that’s different than I. My Maildir directories and files are all owned by the
UNIX user that owns the file. Maybe this is causing me the permissions
problems I’m seeing. Is having it all running as one [UNIX] user a typical
configuration for dovecot2? Or just typical of installations using ACLs?
Thank you.
- Chris
Re: Shared inbox?
> On May 28, 2015, at 09:08, a...@thecsillags.com wrote:
>
> Chris,
>
> I do indeed have an acl_shared_dict set up. That may be the ticket. That
> makes it so that the IMAP server knows that you have acls on the other
> mailbox, so it can know to then look in that mailbox to find out precisely
> what the ACLs are.
Okay. I did set up an acl_shared_dict, but I’m not sure the file
ownership/permissions aren’t somewhat in the way. It appears dovecot, and
doveadm, operate as the user in question, and I think the permission problems
may be getting in my way. I eventually made ~childuser/Maildir group readable
(to a group I’m in), and then was seeing permission denied issues when I tried:
% sudo doveadm acl debug -u cross shared/childuser/INBOX
[…]
doveadm(cross): Error: open(/home/childuser/Maildir/dovecot-acl-list) failed:
Permission denied
So, I made that file readable to a group I’m in, and that went away, but cal
debug still shows no access. I think UNIX permissions are what is impeding my
progress at the moment. Is that normal? Any idea how you’re avoiding this
problem, Andy?
> On May 28, 2015, at 23:57, G H wrote:
>
> Look in to dovecot's master user feature as well.
Okay. This looks like it may help with my above problem. I haven’t looked
into it yet, but will check on that as it sounds valuable. Thanks!
- Chris
Re: Shared inbox?
On May 27, 2015, at 22:57 , Chris Ross wrote:
> On May 25, 2015, at 15:55 , a...@thecsillags.com wrote:
>> When I set them up that way, I shared the target inbox (we'll call it
>> f...@example.com) to be accessible by user b...@example.com. When I go into
>> my email client, I'll see something like: shared/f...@example.com/INBOX as a
>> folder.
>
> When you say "set them up that way", do you mean following the example
> config at SharedMailboxes/Shared ? In that example, a mail_location is set
> at the outer level, which I think I don't want to do, and overridden in the
> shared namespace. I'm assuming I don't want either of those, or at least
> that's what I'm going to try first.
>
> A question I have given your example above, is, did you set
> mail_shared_explicit_inbox ? It's only briefly described what that effects,
> so I'm not sure which setting (on or off) will cause the
> shared/f...@example.com/INBOX you describe seeing. I assume on, so I've
> turned it on.
>
>> The boxes don't have to be shared publicly if you use the "private"
>> namespace.
>
> By "use the "private" namespace", do you mean specifying a separate
> namespace block in the config, like the one is declared in the example at
> http://wiki2.dovecot.org/SharedMailboxes/Shared ? If so, that's what I've
> done. If there is a way to have a single namespace declared that's both
> private and shared, I don't see that, so I have two as presented on that wiki
> page.
Okay. Lots of trial and error and error, and I at least have some
configuration that I think includes pieces about shared mailboxes and ACLs, and
it's actually up and running. But, I'm not seeing the shared folder.
I was getting a lot of errors about separators (namespace configuration
error: All list=yes namespaces must use the same separator) and prefixes
(namespace configuration error: list=yes requires prefix=/ not to start with
separator), but have it working now. The namespace private block from the
example at http://wiki2.dovecot.org/SharedMailboxes/Public caused many of those
problems, and I didn't have a separator configured at all (ie, was still
commented out) in my main namespace, so that apparently conflicted with the
"separator = /" in the shared namespace. But, all that resolved.
The config file I've added into conf.d/15-shared.conf is:
namespace {
type = shared
separator = /
prefix = shared/%%u/
location =
maildir:/home/%%n/Maildir:INDEX=~/Maildir/shared/%%u:INDEXPVT=~/Maildir/shared/%%u
subscriptions = no
list = children
}
mail_shared_explicit_inbox = yes
protocol imap {
mail_plugins = $mail_plugins acl imap_acl
}
plugin {
acl = vfile
}
This comes from the aforementioned wiki page. I ran the doveadm acl command
you suggested (though my usernames have no domain part, like your example did
have domain parts)
Now, while dovecot is back to working, I don't see a shared folder anywhere.
"doveadm mailbox list" lists the mailboxes for users, including the INBOX I've
tried to configure an ACL to let me share.
Clearly I'm missing a piece. Andy, or anyone else, if you see what I've
missed, please let me know. I'm not seeing anything back from the server with
my mail client under "Subscription List", so I can't choose to subscribe.
Maybe I've turned off subscriptions somehow, and it's not auto-subscribing me?
Or, maybe having "list=children", and I haven't set up the ACL correctly?
I ran:
sudo doveadm acl add -u childuser INBOX user=cross lookup read write write-seen
write-deleted insert post expunge create delete admin
After removing the domains from your example, that's about what you had. I
can see the INBOX for childuser with "doveadm mailbox list -u childuser".
Ahh. I think I may be on to something. I tried the "doveadm acl add" again,
which seems to succeed, but "doveadm acl debug" then doesn't mention anything
about me, or my access to childuser's INBOX. Running the "acl add" with
"doveadm -D" shows:
doveadm(childuser): Debug: acl: No acl_shared_dict setting - shared mailbox
listing is disabled
and later:
doveadm(childuser): Debug: acl vfile: Global ACLs disabled
The first of those looks like it might be a problem. Do you have an
"acl_shared_dict" set up in your config anywhere, Andy?
Thanks. Any help appreciated.
- Chris
Re: Shared inbox?
On May 25, 2015, at 15:55 , a...@thecsillags.com wrote: > I'll assume you've seen these: > http://wiki2.dovecot.org/SharedMailboxes/Shared and > http://wiki2.dovecot.org/ACL Yup, I'd looked at those, but still left not understanding all of the options. And you gave a doveadm command, which I appreciate very much, but surprises me since the materials in the ACL wiki page are all about file contents. But I'll try your command. > When I set them up that way, I shared the target inbox (we'll call it > f...@example.com) to be accessible by user b...@example.com. When I go into > my email client, I'll see something like: shared/f...@example.com/INBOX as a > folder. When you say "set them up that way", do you mean following the example config at SharedMailboxes/Shared ? In that example, a mail_location is set at the outer level, which I think I don't want to do, and overridden in the shared namespace. I'm assuming I don't want either of those, or at least that's what I'm going to try first. A question I have given your example above, is, did you set mail_shared_explicit_inbox ? It's only briefly described what that effects, so I'm not sure which setting (on or off) will cause the shared/f...@example.com/INBOX you describe seeing. I assume on, so I've turned it on. > To give bar@ access to foo@'s INBOX, I ran something like this: > > doveadm acl add \ >-u f...@example.com \ >INBOX user=b...@example.com \ >lookup read write write-seen write-deleted insert\ >post expunge create delete admin > > http://wiki2.dovecot.org/Tools/Doveadm/ACL covers the details of what all of > the "lookup read..." bits and more do. > > The boxes don't have to be shared publicly if you use the "private" namespace. By "use the "private" namespace", do you mean specifying a separate namespace block in the config, like the one is declared in the example at http://wiki2.dovecot.org/SharedMailboxes/Shared ? If so, that's what I've done. If there is a way to have a single namespace declared that's both private and shared, I don't see that, so I have two as presented on that wiki page. And did/do you have "subscriptions=no" and "list=children" defined? Again, as their only briefly described, I'm not 100% sure I understand the difference in behavior I'll see with different settings. And I'm pretty sure I read that there are ways to have mailboxes shared such that who has or hasn't read contents of the mailboxes is stored within the mailbox, and also ways to have it stored per reading user. I'm not sure which I'll be seeing/using here. Thanks. I'll ask more questions if I have more questions after a little trial and error on my own server. :-) Thanks for your help! - Chris
Shared inbox?
I'm running dovecot 2.2.16 on my FreeBSD mail server. I've read information on the wiki about setting up shared mailboxes, but I want to do something that isn't really coved by the instructions I was reading there. My son (now 7 years old) has an account on the system, but doesn't use it directly. But, for things he's interested in like Minecraft, and/or the local zoo, we have given out his email address in a small number of places. What I would like to set up, both for now while he's not actually ever reading his email himself, and perhaps even for the future when we teach and/or allow for that, is for my wife and I to be able to "view" his inbox from our accounts. The instructions for setting up shared folders all are written so that they're secondary folders to all accounts. Is there a way to either (a) configure sharing someone elses Inbox by other accounts, or (b) setting up a separately configured shared folder to _act_ as the inbox for a single account? Thanks. Any suggestions to achieve the above described end goal would be appreciated. - Chris
Re: Problem with sieve not triggering randomly?
Once upon a time, Stephan Bosch said: > You're using Dovecot 2.2.10, which is quite old. I remembered a bug like > this, but I had to look it up. This is the original bug report: Ah, sorry for taking your time on an old bug. For most of my servers, I try to use the RHEL/CentOS-provided packages where practical (because with over 100 servers running a wide variety of services, building everything from source takes more hours than I have in a day). The hazzard of course is chasing already-fixed bugs from time to time. I also understand Red Hat's "stability" approach, where they tend to stick with a given upstream version and just patch it; few Open Source software developers have the time to maintain bugfix-only branches for the lifetime of a RHEL major version, and new features tend to mean new bugs as well. Thanks for taking the time to point out the fix. I was hoping to have time this weekend to try the latest Dovecot release, but hadn't yet done that. I have opened a Red Hat bug to try to get this patch applied. https://bugzilla.redhat.com/show_bug.cgi?id=1224496 -- Chris Adams
Re: Problem with sieve not triggering randomly?
Once upon a time, Chris Adams said:
> Okay, digging some more, it looks like something in sieve is overwriting
> the wrong thing when it gets messages with some headers (at least From:
> and Subject:) repeated. I enabled the vnd.dovecot.debug sieve plugin,
> and used this sieve script:
And I guess something is re-parsing them at some point? The following
sieve script has the problem:
# compile with "sievec /etc/dovecot/default.sieve"
require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
fileinto "Spam";
}
Doubling up the test makes it work however:
# compile with "sievec /etc/dovecot/default.sieve"
require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
fileinto "Spam";
}
if header :contains "X-Spam-Flag" "YES" {
fileinto "Spam";
}
???
--
Chris Adams
Re: Problem with sieve not triggering randomly?
Once upon a time, Chris Adams said:
> I can confirm that a message with multiple Subject: and multiple From:
> headers does not get filed correctly into the Spam folder. The
> sieve-test tools shows the correct action, but when the message comes in
> via LMTP, it goes into INBOX.
Okay, digging some more, it looks like something in sieve is overwriting
the wrong thing when it gets messages with some headers (at least From:
and Subject:) repeated. I enabled the vnd.dovecot.debug sieve plugin,
and used this sieve script:
require [ "fileinto", "variables", "vnd.dovecot.debug" ];
if header :matches "from" "*" { set "from" "${1}"; }
if header :matches "subject" "*" { set "subject" "${1}"; }
if header :matches "x-spam-flag" "*" { set "spam" "${1}"; }
debug_log "From='${from}' Subject='${subject}' Spam='${spam}'";
if header :contains "X-Spam-Flag" "YES" {
fileinto "Spam";
}
When I feed a message to dovecot-lda with single From: and Subject:
headers and X-Spam-Flag: YES set, I get this logged:
May 19 14:25:25 hosting dovecot: lda(lo...@example.com): sieve: default: line
6: DEBUG: From='t...@cmadams.net' Subject='Test' Spam='YES'
If I duplicate the From: and Subject: headers, I get:
May 19 14:25:29 hosting dovecot: lda(lo...@example.com): sieve: default: line
6: DEBUG: From='t...@cmadams.net' Subject='Test' Spam='lo...@example.com'
So, X-Spam-Flag: is somehow getting the To: address in it?
Oddly, with this debugging enabled, even though the message with the
duplicate headers appears to have the wrong value for X-Spam-Flag:, it
then procedes to actually match the YES somehow and file the message in
the Spam folder.
--
Chris Adams
Re: Problem with sieve not triggering randomly?
Once upon a time, Chris Adams said: > In the several cases I looked at, the Spam folder exists, is > getting most spam-flagged messages filed into it, it just seems to be > something about some messages (for example, got a bunch of copies of > this particular spam to different users and they all went to INBOX > instead of Spam). I can confirm that a message with multiple Subject: and multiple From: headers does not get filed correctly into the Spam folder. The sieve-test tools shows the correct action, but when the message comes in via LMTP, it goes into INBOX. -- Chris Adams
Re: Problem with sieve not triggering randomly?
Once upon a time, Stephan Bosch said: > On 5/15/2015 5:56 PM, Chris Adams wrote: > > Once upon a time, Stephan Bosch said: > >> You can check the handling of a particular message yourself using the > >> sieve-test tool (there is a man page for it). By specifying the `-t - > >> -Tlevel=matching` options, you'll get detailed information on why a > >> particular decision is made. > > Hmm, that's weird. sieve-test says it would store the message into the > > Spam folder (as expected), but it was definitely delivered to INBOX. > > BTW, did you consult your logs for any errors? If an error occurs during > Sieve processing, the default behavior is to file the message into INBOX > (e.g. when the Spam folder doesn't exist). Yes, I didn't find any errors, just the lmtp log entry for storing into INBOX. In the several cases I looked at, the Spam folder exists, is getting most spam-flagged messages filed into it, it just seems to be something about some messages (for example, got a bunch of copies of this particular spam to different users and they all went to INBOX instead of Spam). May 15 09:44:04 dovecot2 dovecot: lmtp(10157): Connect from 10.0.9.71 May 15 09:44:05 dovecot2 dovecot: lmtp(10157, localus...@foothills.net): MikHD8/YVVWtJwAAIYJ+iw: sieve: msgid=<438088200.476329351.0075518260474.javamail.r...@sjmas02.lococandles.co>: stored mail into mailbox 'INBOX' May 15 09:44:05 dovecot2 dovecot: lmtp(10157): Disconnect from 10.0.9.57: Successful quit Also, if the Spam folder didn't exist, I have it set to be autocreated in the Dovecot config. I also know that Dovecot can write to the Spam folder okay, because I did a "doveadm move" to move several "X-Spam-Flag: YES" messages to the Spam folder, and that worked. -- Chris Adams
