Re: Outlook not showing all email in inbox at random
Problem resolved. There was a spam filter on the network that access the mail server that was interfering. Regards, David Koski dko...@sutinen.com On 8/4/23 16:59, David Koski wrote: I found the problem in Dovecot. It was an illusion that Thunderbird works. Where in Outlook there are certain emails that stop synchronization, Thunderbird continues but without loading the specific emails. I have copied content from a good email to the one that errors out due to "connection reset by peer" to no avail. However, renaming the Dovecot created file name resolves the problem and the email can now be viewed. The original file name that does not work in Dovecot and renamed to: # mv 1691104011.M395237P16733.mail\,S\=2180375\,W\=2208740\:2\,S 1691104011.M395237P16733.mail Regards, David Koski dko...@sutinen.com On 8/4/23 15:35, David Koski wrote: I manage IT for a small office with about 10 workstations. Dovecot has been installed for years and is now at 2.3.4.1-5 (Debian). I just discovered that the user cannot see many emails in the inbox. I was alerted when email was not appearing to be coming in for many hours. Other users observed the same thing. On investigation, I noticed many inbox emails missing in the Outlook GUI from times past that were on the server. It appears to be a broken client (Outlook 2019) to me but troubleshooting has yielded no solid leads. The profile and OST files have been re-created many times with no change in outcome. Each time, the inbox does not synchronize about 40 most recent emails but does for the last one for the day before and many before that. It may well be an Outlook issue so I'm looking for information to work around an Outlook bug if necessary. Thunderbird works perfectly. Also, interestingly, I can copy email from Thunderbird (or move emails on the server) to a temporary directory form the inbox, then move them back and they will appear if I don't move too many at a time. Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Outlook not showing all email in inbox at random
On 8/7/23 00:40, Robert Schetterer via dovecot wrote: Am 06.08.23 um 22:15 schrieb David Koski: Note that this is not just an Outlook problem. I have discovered that Thunderbird fails on the same emails but continues forward with synchronization instead of blocking. Why would changing the file name for the email on the server make it work? (See below.) Now its looking like a Dovecot issue, no? David Koski for me it looks more like a problem with indexing on dovecot have you tried to recreate the index and/or force resync of this mailbox Yes. Index files were deleted and Dovecot recreated them. https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fdoc.dovecot.org%2f3.0%2fman%2fdoveadm-force-resync.1%2f&c=E,1,UzSwWY-vy38koQ-foG33NgsCgRfWeGW83s3ZfbnagmmjARp6zYV-VD6ONVELJfuHZDMxSuiuo9r1dGFe9ciI26ygE-xwrxsCPj5vLiWg&typo=1 On 8/5/23 02:35, Marc wrote: I have discovered a whole list of bugs in Outlook 2019 and reported them constantly at this Q&A of them, up to a point where their advised workarounds where even contradictory and I got blocked. One bug that surprised me the most (existing already for years) is that if your outlook profile has issues your emails are not saved in Sent, they just disappear. If you don't use exchange's public folders just forget about outlook. What is also super annoying is that if you tend to archive emails in folders, outlook is not able to move them to these folders after a specific time. PS I think outlook on ios/android is not even a real client, they send your credentials to the cloud and the cloud is accessing your email. I found the problem in Dovecot. It was an illusion that Thunderbird works. Where in Outlook there are certain emails that stop synchronization, Thunderbird continues but without loading the specific emails. I have copied content from a good email to the one that errors out due to "connection reset by peer" to no avail. However, renaming the Dovecot created file name resolves the problem and the email can now be viewed. The original file name that does not work in Dovecot and renamed to: # mv 1691104011.M395237P16733.mail\,S\=2180375\,W\=2208740\:2\,S 1691104011.M395237P16733.mail Regards, David Koski dko...@sutinen.com On 8/4/23 15:35, David Koski wrote: I manage IT for a small office with about 10 workstations. Dovecot has been installed for years and is now at 2.3.4.1-5 (Debian). I just discovered that the user cannot see many emails in the inbox. I was alerted when email was not appearing to be coming in for many hours. Other users observed the same thing. On investigation, I noticed many inbox emails missing in the Outlook GUI from times past that were on the server. It appears to be a broken client (Outlook 2019) to me but troubleshooting has yielded no solid leads. The profile and OST files have been re-created many times with no change in outcome. Each time, the inbox does not synchronize about 40 most recent emails but does for the last one for the day before and many before that. It may well be an Outlook issue so I'm looking for information to work around an Outlook bug if necessary. Thunderbird works perfectly. Also, interestingly, I can copy email from Thunderbird (or move emails on the server) to a temporary directory form the inbox, then move them back and they will appear if I don't move too many at a time. Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Outlook not showing all email in inbox at random
Note that this is not just an Outlook problem. I have discovered that Thunderbird fails on the same emails but continues forward with synchronization instead of blocking. Why would changing the file name for the email on the server make it work? (See below.) Now its looking like a Dovecot issue, no? David Koski On 8/5/23 02:35, Marc wrote: I have discovered a whole list of bugs in Outlook 2019 and reported them constantly at this Q&A of them, up to a point where their advised workarounds where even contradictory and I got blocked. One bug that surprised me the most (existing already for years) is that if your outlook profile has issues your emails are not saved in Sent, they just disappear. If you don't use exchange's public folders just forget about outlook. What is also super annoying is that if you tend to archive emails in folders, outlook is not able to move them to these folders after a specific time. PS I think outlook on ios/android is not even a real client, they send your credentials to the cloud and the cloud is accessing your email. I found the problem in Dovecot. It was an illusion that Thunderbird works. Where in Outlook there are certain emails that stop synchronization, Thunderbird continues but without loading the specific emails. I have copied content from a good email to the one that errors out due to "connection reset by peer" to no avail. However, renaming the Dovecot created file name resolves the problem and the email can now be viewed. The original file name that does not work in Dovecot and renamed to: # mv 1691104011.M395237P16733.mail\,S\=2180375\,W\=2208740\:2\,S 1691104011.M395237P16733.mail Regards, David Koski dko...@sutinen.com On 8/4/23 15:35, David Koski wrote: I manage IT for a small office with about 10 workstations. Dovecot has been installed for years and is now at 2.3.4.1-5 (Debian). I just discovered that the user cannot see many emails in the inbox. I was alerted when email was not appearing to be coming in for many hours. Other users observed the same thing. On investigation, I noticed many inbox emails missing in the Outlook GUI from times past that were on the server. It appears to be a broken client (Outlook 2019) to me but troubleshooting has yielded no solid leads. The profile and OST files have been re-created many times with no change in outcome. Each time, the inbox does not synchronize about 40 most recent emails but does for the last one for the day before and many before that. It may well be an Outlook issue so I'm looking for information to work around an Outlook bug if necessary. Thunderbird works perfectly. Also, interestingly, I can copy email from Thunderbird (or move emails on the server) to a temporary directory form the inbox, then move them back and they will appear if I don't move too many at a time. Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Outlook not showing all email in inbox at random
I found the problem in Dovecot. It was an illusion that Thunderbird works. Where in Outlook there are certain emails that stop synchronization, Thunderbird continues but without loading the specific emails. I have copied content from a good email to the one that errors out due to "connection reset by peer" to no avail. However, renaming the Dovecot created file name resolves the problem and the email can now be viewed. The original file name that does not work in Dovecot and renamed to: # mv 1691104011.M395237P16733.mail\,S\=2180375\,W\=2208740\:2\,S 1691104011.M395237P16733.mail Regards, David Koski dko...@sutinen.com On 8/4/23 15:35, David Koski wrote: I manage IT for a small office with about 10 workstations. Dovecot has been installed for years and is now at 2.3.4.1-5 (Debian). I just discovered that the user cannot see many emails in the inbox. I was alerted when email was not appearing to be coming in for many hours. Other users observed the same thing. On investigation, I noticed many inbox emails missing in the Outlook GUI from times past that were on the server. It appears to be a broken client (Outlook 2019) to me but troubleshooting has yielded no solid leads. The profile and OST files have been re-created many times with no change in outcome. Each time, the inbox does not synchronize about 40 most recent emails but does for the last one for the day before and many before that. It may well be an Outlook issue so I'm looking for information to work around an Outlook bug if necessary. Thunderbird works perfectly. Also, interestingly, I can copy email from Thunderbird (or move emails on the server) to a temporary directory form the inbox, then move them back and they will appear if I don't move too many at a time. Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Outlook not showing all email in inbox at random
I manage IT for a small office with about 10 workstations. Dovecot has been installed for years and is now at 2.3.4.1-5 (Debian). I just discovered that the user cannot see many emails in the inbox. I was alerted when email was not appearing to be coming in for many hours. Other users observed the same thing. On investigation, I noticed many inbox emails missing in the Outlook GUI from times past that were on the server. It appears to be a broken client (Outlook 2019) to me but troubleshooting has yielded no solid leads. The profile and OST files have been re-created many times with no change in outcome. Each time, the inbox does not synchronize about 40 most recent emails but does for the last one for the day before and many before that. It may well be an Outlook issue so I'm looking for information to work around an Outlook bug if necessary. Thunderbird works perfectly. Also, interestingly, I can copy email from Thunderbird (or move emails on the server) to a temporary directory form the inbox, then move them back and they will appear if I don't move too many at a time. Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: quotas and Trash
On 4/19/23 09:48, Benny Pedersen wrote: David Koski skrev den 2023-04-19 18:19: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } Re: "..just says that you can save additional 100M to Trash." In addition to what? What would be a practical example? If it provides 100M additional to trash, what is the total? in the above example trash can have totaly 101M, while other only in total can have 1M My observations differ using same plugin config as above. With folder usage: 4.0K ./Maildir/tmp 80K ./Maildir/.Sent 32K ./Maildir/.Sent Messages 920K ./Maildir/.Trash 28K ./Maildir/.Junk 28K ./Maildir/.Drafts 4.0K ./Maildir/new 20K ./Maildir/cur 1.3M ./Maildir 1.3M . Sending an 172kB file to accoount fails due to over quota. Debug log: Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Quota root: name=User quota backend=maildir args= Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Quota rule: root=User quota mailbox=* bytes=1048576 messages=0 Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Quota rule: root=User quota mailbox=INBOX.Trash bytes=+104857600 messages=0 Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Quota warning: bytes=996147 (95%) messages=0 reverse=no command=quota-warning 95 dko...@example.com Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Quota warning: bytes=838860 (80%) messages=0 reverse=no command=quota-warning 80 dko...@example.com Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Quota grace: root=User quota bytes=104857 (10%) Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/lib/vmail/example.com/dkoski/Maildir Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: maildir++: root=/var/lib/vmail/example.com/dkoski/Maildir, index=, indexpvt=, control=, inbox=/var/lib/vmail/example.com/dkoski/Maildir, alt= Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: quota: quota_over_flag check: quota_over_script unset - skipping Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Destination address: (source: user@hostname) Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: Debug: Mailbox INBOX: Mailbox opened because: lib-lda delivery Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: msgid=<8f99bfe7-02e5-cbc4-9af3-112fbe278...@mymail.com>: save failed to INBOX: Quota exceeded (mailbox for user is full) Apr 21 14:52:15 examplemail dovecot: lda(dko...@example.com)<2038>: msgid=<8f99bfe7-02e5-cbc4-9af3-112fbe278...@mymail.com>: rejected: Quota exceeded (mailbox for user is full) Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: quotas and Trash
On 4/19/23 09:48, Benny Pedersen wrote: David Koski skrev den 2023-04-19 18:19: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } Re: "..just says that you can save additional 100M to Trash." In addition to what? What would be a practical example? If it provides 100M additional to trash, what is the total? in the above example trash can have totaly 101M, while other only in total can have 1M Thank you! David ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: quotas and Trash
On 4/19/23 08:24, Aki Tuomi wrote: On 19/04/2023 18:15 EEST David Koski wrote: On 4/18/23 23:46, Aki Tuomi via dovecot wrote: On 19/04/2023 02:38 EEST David Koski wrote: Hello, I am testing quotas with the following configuration: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } The 1M limit works perfectly but I had to prefix Trash with "INBOX." to get it to stop complaining about an unknown namespace. But now the quota_rule2 is ignored. Once the mailbox reaches 1M, even if it is all in INBOX.Trash, it will fail to deliver due to quota limit. Regards, David Koski dko...@sutinen.com Hi! The quota rule setup you have, just says that you can save additional 100M to Trash. It does not mean that your total quota is increased. If your mailbox size is 100M due to mail being in Trash, you're still over the 1M quota for other mailboxes than Trash. Hello, I must be missing something basic. If the 100M is not in addition to the 1M total, then what is it in additional to? How does it apply and to what? It applies when you are storing mails to INBOX.Trash. Re: "..just says that you can save additional 100M to Trash." In addition to what? What would be a practical example? If it provides 100M additional to trash, what is the total? Thanks! David Thank you! David Koski Aki Aki ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: quotas and Trash
On 4/18/23 23:46, Aki Tuomi via dovecot wrote: On 19/04/2023 02:38 EEST David Koski wrote: Hello, I am testing quotas with the following configuration: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } The 1M limit works perfectly but I had to prefix Trash with "INBOX." to get it to stop complaining about an unknown namespace. But now the quota_rule2 is ignored. Once the mailbox reaches 1M, even if it is all in INBOX.Trash, it will fail to deliver due to quota limit. Regards, David Koski dko...@sutinen.com Hi! The quota rule setup you have, just says that you can save additional 100M to Trash. It does not mean that your total quota is increased. If your mailbox size is 100M due to mail being in Trash, you're still over the 1M quota for other mailboxes than Trash. Hello, I must be missing something basic. If the 100M is not in addition to the 1M total, then what is it in additional to? How does it apply and to what? Thank you! David Koski Aki ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
quotas and Trash
Hello, I am testing quotas with the following configuration: plugin { quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M quota_grace = 10%% } The 1M limit works perfectly but I had to prefix Trash with "INBOX." to get it to stop complaining about an unknown namespace. But now the quota_rule2 is ignored. Once the mailbox reaches 1M, even if it is all in INBOX.Trash, it will fail to deliver due to quota limit. Dovecot: 2.3.13+dfsg1-2+deb11u1:amd64 Debian 11 auth_mechanisms = plain login disable_plaintext_auth = no mail_plugins = quota mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_plugins = quota quota quota = maildir:User quota quota_grace = 10%% quota_rule = *:storage=1M quota_rule2 = INBOX.Trash:storage=+100M } protocols = " imap pop3" service anvil { unix_listener anvil-auth-penalty { group = vmail mode = 0660 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-client { mode = 0660 } } service imap-login { inet_listener imap { address = 172.16.0.1 } } service pop3-login { inet_listener pop3 { address = 172.16.0.1 } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl = no ssl_client_ca_dir = /etc/ssl/certs ssl_dh = # hidden, use -P to show it userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota quota postmaster_address = dko...@sutinen.com } protocol lda { mail_plugins = quota quota } protocol imap { mail_max_userip_connections = 14 mail_plugins = quota quota imap_quota } protocol pop3 { mail_plugins = quota quota } Regards, David Koski dko...@sutinen.com ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Custom SASL authentication
Hello R., Thank you for the reply. On 4/17/23 14:49, Robert Lister wrote: Interesting... I'm not sure why you would need to set : "NULL AS password, 'Y' as nopassword", but then I haven't seen the rest of your auth config. This would seem to allow any password but succeed if it matches the rest of the query. The query does not return the password, only the hash, thus "NULL AS password", as I understand it. It is documented. The 'Y' as no password is found earlier in this thread. It seemed a bit awkward to me to embed static variables in the query which are always going to be the same. In the auth- config, or 10-mail.conf I just set:- # Defaults: mail_uid = vmail mail_gid = vmail mail_home = /var/lib/vmail/%d/%n # (this is set elsewhere in 10-mail.conf.) mail_location = maildir:~/Maildir I think you are right. My eye was on migrating global settings to SQL accounts but it probably will not happen. Nevertheless, I suspect it adds little burden having static variables given it requires no database access, no? userdb can still override these if needed, but it means that the query is a lot simpler and if one of the queries doesn't return home/uid/gid etc, it's always going to be set anyway. Seems to me it is a matter of preference. What does the "crypt" bit of "IF (crypt & 1, SHA2('%w',512)" do? crypt it before running the select? Why not just set: default_pass_scheme = ? "crypt" bit 0 is a flag to encrypt passwords. I have chosen a custom hash. Can I replace it with "default_pass_scheme = ?"? Regards, David R. On 2023-04-17 18:57, dko...@sutinen.com wrote: For the archive: This MySQL configuration seems to work well. user_query = \ SELECT \ email AS user, \ '/var/lib/vmail/%d/%n' AS home, \ 'maildir:/var/lib/vmail/%d/%n/Maildir' AS mail, \ 5000 AS uid, \ 5000 AS gid \ FROM \ view_users \ WHERE \ email='%u' \ AND \ enable = '1' password_query = \ SELECT \ email AS user, \ NULL AS password, \ 'Y' as nopassword \ FROM \ view_users \ WHERE \ email='%u' \ AND \ password=IF (crypt & 1, SHA2('%w',512), password) \ AND \ enable = '1' ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: Custom SASL authentication
Fixing to not top post. On 3/18/23 10:07, Aki Tuomi wrote: On 18/03/2023 00:44 EET David Koski wrote: Hello, I'm looking for a good way to apply a custom hash to passwords. My hope is to add passwords to a (MySQL) database: INSERT INTO users (user='joblo', pass=MYHASH('plain-password').. For SASL authentication, my thought first was to apply the same hash to the issued password and compare it with the hashed password in the database. I soon discovered the sql driver supplied by Dovecot doesn't provide that ability, unless I'm missing something. I'm looking for documentation on how to implement a custom authentication script if needed. Regards, David Koski dko...@sutinen.com Hi David, see https://doc.dovecot.org/configuration_manual/authentication/lua_based_authentication/ <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fdoc.dovecot.org%2fconfiguration_manual%2fauthentication%2flua_based_authentication%2f&c=E,1,9ey3vCELwieYk48TYCRIc3sTP0NX6IAzpNYTi7oYlL4_KJcx8IMijlRF7zmvrRU1DN9FriQm24ek0MdzT44auq5mqvOhpVhQSHsjmUvBjF54WhW0tgDC&typo=1&ancr_add=1> on how to implement custom authentication. For verifying password you could use MYHASH('%w') in your passdb sql lookup. You need to include `'Y' as nopassword` in this case, and this will cause wrong password to become unknown user error. Aki I have done some testing and found the following queries to work for implementing MySQL SHA2 passwords for authentication: 1 user_query: 2 3 SELECT 4 email AS user, 5 if ( 6 (select crypt & 1 from view_users where email='%u'), 7 (select password from view_users where email='%u' and password=SHA2('%w',512)), 8 (select password from view_users where email='%u' and password='%w') 9 ) as password, 10 '/var/lib/vmail/%d/%n' AS home, 11 'maildir:/var/lib/vmail/%d/%n/Maildir' AS mail, 12 5000 AS uid, 13 5000 AS gid 14 FROM 15 view_users 16 WHERE 17 email = '%u' AND enable = '1' 18 19 20 password_query: 21 22 SELECT 23 email AS user, 24 if ( 25 (select crypt & 1 from view_users where email='%u'), 26 (select password from view_users where email='%u' and password=SHA2('%w',512)), 27 (select password from view_users where email='%u' and password='%w') 28 ) as password 29 FROM 30 view_users 31 WHERE 32 email = '%u' AND enable = '1' But it seems wasteful in the number of queries required. Looking for ideas to consolidate queries. Also, do the Dovecot query strings have to be s single query or can there be a query to set a variable, for example, to use in subsequent queries? Regards, David Koski dko...@sutinen.com
Re: Custom SASL authentication
I have done some testing and found the following queries to work for implementing MySQL SHA2 passwords for authentication: 1 user_query: 2 3 SELECT 4 email AS user, 5 if ( 6 (select crypt & 1 from view_users where email='%u'), 7 (select password from view_users where email='%u' and password=SHA2('%w',512)), 8 (select password from view_users where email='%u' and password='%w') 9 ) as password, 10 '/var/lib/vmail/%d/%n' AS home, 11 'maildir:/var/lib/vmail/%d/%n/Maildir' AS mail, 12 5000 AS uid, 13 5000 AS gid 14 FROM 15 view_users 16 WHERE 17 email = '%u' AND enable = '1' 18 19 20 password_query: 21 22 SELECT 23 email AS user, 24 if ( 25 (select crypt & 1 from view_users where email='%u'), 26 (select password from view_users where email='%u' and password=SHA2('%w',512)), 27 (select password from view_users where email='%u' and password='%w') 28 ) as password 29 FROM 30 view_users 31 WHERE 32 email = '%u' AND enable = '1' But it seems wasteful in the number of queries required. Looking for ideas to consolidate queries. Regards, David Koski dko...@sutinen.com On 3/18/23 10:07, Aki Tuomi wrote: On 18/03/2023 00:44 EET David Koski wrote: Hello, I'm looking for a good way to apply a custom hash to passwords. My hope is to add passwords to a (MySQL) database: INSERT INTO users (user='joblo', pass=MYHASH('plain-password').. For SASL authentication, my thought first was to apply the same hash to the issued password and compare it with the hashed password in the database. I soon discovered the sql driver supplied by Dovecot doesn't provide that ability, unless I'm missing something. I'm looking for documentation on how to implement a custom authentication script if needed. Regards, David Koski dko...@sutinen.com Hi David, see https://doc.dovecot.org/configuration_manual/authentication/lua_based_authentication/ <https://linkprotect.cudasvc.com/url?a=https%3a%2f%2fdoc.dovecot.org%2fconfiguration_manual%2fauthentication%2flua_based_authentication%2f&c=E,1,9ey3vCELwieYk48TYCRIc3sTP0NX6IAzpNYTi7oYlL4_KJcx8IMijlRF7zmvrRU1DN9FriQm24ek0MdzT44auq5mqvOhpVhQSHsjmUvBjF54WhW0tgDC&typo=1&ancr_add=1> on how to implement custom authentication. For verifying password you could use MYHASH('%w') in your passdb sql lookup. You need to include `'Y' as nopassword` in this case, and this will cause wrong password to become unknown user error. Aki
Custom SASL authentication
Hello, I'm looking for a good way to apply a custom hash to passwords. My hope is to add passwords to a (MySQL) database: INSERT INTO users (user='joblo', pass=MYHASH('plain-password').. For SASL authentication, my thought first was to apply the same hash to the issued password and compare it with the hashed password in the database. I soon discovered the sql driver supplied by Dovecot doesn't provide that ability, unless I'm missing something. I'm looking for documentation on how to implement a custom authentication script if needed. Regards, David Koski dko...@sutinen.com
Re: NTLM fails: dovecot: auth: Fatal: Unknown authentication mechanism 'NTLM'
Is NTLM now dead? The Readme says: 2020-10-23 16:24:09 -0400 Josef 'Jeff' Sipek (48d6f7282) auth: Remove ntlm mechanism & the LANMAN and NTLM password schemes M COPYING M configure.ac M src/Makefile.am M src/auth/Makefile.am D src/auth/mech-ntlm.c M src/auth/mech.c M src/auth/password-scheme.c M src/auth/test-libpassword.c M src/auth/test-mech.c M src/doveadm/Makefile.am D src/lib-ntlm/Makefile.am D src/lib-ntlm/ntlm-des.c D src/lib-ntlm/ntlm-des.h D src/lib-ntlm/ntlm-encrypt.c D src/lib-ntlm/ntlm-encrypt.h D src/lib-ntlm/ntlm-flags.h D src/lib-ntlm/ntlm-message.c D src/lib-ntlm/ntlm-message.h D src/lib-ntlm/ntlm-types.h D src/lib-ntlm/ntlm.h David On 1/22/22 4:22 PM, David Koski wrote: After upgrading Debian to 11 I found Dovecot at version 2.3.13 (89f716dc2). Now auth method NTLM fails and is not even listed: # doveadm pw -l SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 SHA512-CRYPT CLEAR CLEARTEXT ARGON2I ARGON2ID SSHA256 MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SHA256-CRYPT SMD5 DIGEST-MD5 LDAP-MD5 /var/log/dovecot.log Jan 22 16:20:32 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:32 master: Error: service(auth): command startup failed, throttling for 2.000 secs Jan 22 16:20:34 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:34 master: Error: service(auth): command startup failed, throttling for 4.000 secs Jan 22 16:20:38 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:38 master: Error: service(auth): command startup failed, throttling for 8.000 secs Jan 22 16:20:46 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:46 master: Error: service(auth): command startup failed, throttling for 16.000 secs # doveconf -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-11-amd64 x86_64 Debian 11.2 # Hostname: imail.khmfdbyekekelj1rmytwnfh1bc.dx.internal.cloudapp.net auth_mechanisms = plain login ntlm debug_log_path = /var/log/dovecot-debug.log info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace compat { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_plugins = " quota trash sieve" sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = " imap sieve" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-client { mode = 0660 } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert =
NTLM fails: dovecot: auth: Fatal: Unknown authentication mechanism 'NTLM'
After upgrading Debian to 11 I found Dovecot at version 2.3.13 (89f716dc2). Now auth method NTLM fails and is not even listed: # doveadm pw -l SHA1 SSHA512 SCRAM-SHA-256 BLF-CRYPT PLAIN HMAC-MD5 OTP SHA512 SHA DES-CRYPT CRYPT SSHA MD5-CRYPT PLAIN-MD4 PLAIN-MD5 SCRAM-SHA-1 SHA512-CRYPT CLEAR CLEARTEXT ARGON2I ARGON2ID SSHA256 MD5 PBKDF2 SHA256 CRAM-MD5 PLAIN-TRUNC SHA256-CRYPT SMD5 DIGEST-MD5 LDAP-MD5 /var/log/dovecot.log Jan 22 16:20:32 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:32 master: Error: service(auth): command startup failed, throttling for 2.000 secs Jan 22 16:20:34 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:34 master: Error: service(auth): command startup failed, throttling for 4.000 secs Jan 22 16:20:38 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:38 master: Error: service(auth): command startup failed, throttling for 8.000 secs Jan 22 16:20:46 auth: Fatal: Unknown authentication mechanism 'NTLM' Jan 22 16:20:46 master: Error: service(auth): command startup failed, throttling for 16.000 secs # doveconf -n # 2.3.13 (89f716dc2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.13 (cdd19fe3) # OS: Linux 5.10.0-11-amd64 x86_64 Debian 11.2 # Hostname: imail.khmfdbyekekelj1rmytwnfh1bc.dx.internal.cloudapp.net auth_mechanisms = plain login ntlm debug_log_path = /var/log/dovecot-debug.log info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace compat { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_plugins = " quota trash sieve" sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = " imap sieve" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-client { mode = 0660 } } service stats { unix_listener stats-reader { group = vmail mode = 0660 user = vmail } unix_listener stats-writer { group = vmail mode = 0660 user = vmail } } ssl_cert =
Re: [Dovecot] dovecot LDA: "temporary failure"
Forget it. I must have been crosseyed. Someone had put the log file in a non-standard place /home/vmail/dovecot-deliver.log. It told all. A plugin cmusieve was not found, possibly depreciated? I learned how to make the deliver command line work. ;) Sorry, David Koski dko...@sutinen.com > Additional information: > > I created a script to get the return code from deliver: > > 1 #!/bin/bash > 2 > 3 cat "cur/1300037582.P7332Q0M221160.brusco-mail:2,S" | > /usr/lib/dovecot/deliver -f dko...@somedomain.com -d > dko...@domainhidden.com 4 > 5 case $? in > 6 0) echo "0" > 7 ;; > 8 64) echo "64" > 9 ;; > 10 67) echo "67" > 11 ;; > 12 78) echo "78" > 13 ;; > 14 77) echo "77" > 15 ;; > 16 75) echo "75" > 17 ;; > 18 *) echo "unknown" > 19 esac > 20 > > The file it is cat'ing is a valid email in an Maildir/cur file. It returns > 75 (EX_TEMP_FAIL) every time, even run as root. The logs reveal: > > /var/log/dovecot/lda-deliver: > > 2011-03-17 23:35:39 deliver(dko...@somedomain2.com): Info: Loading modules > from directory: /usr/lib/dovecot/modules/lda > > /var/log/dovecot/deliver.info: > (nothing) > > /var/log/dovecot/deliver.err: > (nothing) > > I tried the script on another machine with Dovecot 1.0.15-2.3+lenny1 with > success if I run as root. > > Regards, > David Koski > dko...@sutinen.com > > > This really has me stumped and desperate. I upgraded Debian from 5... > >
Re: [Dovecot] dovecot LDA: "temporary failure"
Forget it. I must have been crosseyed. Someone had put the log file in a non-standard place /home/vmail/dovecot-deliver.log. It told all. A plugin cmusieve was not found, possibly depreciated? I learned how to make the deliver command line work. ;) Sorry, David Koski dko...@sutinen.com > Additional information: > > I created a script to get the return code from deliver: > > 1 #!/bin/bash > 2 > 3 cat "cur/1300037582.P7332Q0M221160.brusco-mail:2,S" | > /usr/lib/dovecot/deliver -f dko...@somedomain.com -d > dko...@domainhidden.com 4 > 5 case $? in > 6 0) echo "0" > 7 ;; > 8 64) echo "64" > 9 ;; > 10 67) echo "67" > 11 ;; > 12 78) echo "78" > 13 ;; > 14 77) echo "77" > 15 ;; > 16 75) echo "75" > 17 ;; > 18 *) echo "unknown" > 19 esac > 20 > > The file it is cat'ing is a valid email in an Maildir/cur file. It returns > 75 (EX_TEMP_FAIL) every time, even run as root. The logs reveal: > > /var/log/dovecot/lda-deliver: > > 2011-03-17 23:35:39 deliver(dko...@somedomain2.com): Info: Loading modules > from directory: /usr/lib/dovecot/modules/lda > > /var/log/dovecot/deliver.info: > (nothing) > > /var/log/dovecot/deliver.err: > (nothing) > > I tried the script on another machine with Dovecot 1.0.15-2.3+lenny1 with > success if I run as root. > > Regards, > David Koski > dko...@sutinen.com > > > This really has me stumped and desperate. I upgraded Debian from 5... > >
Re: [Dovecot] dovecot LDA: "temporary failure"
Additional information: I created a script to get the return code from deliver: 1 #!/bin/bash 2 3 cat "cur/1300037582.P7332Q0M221160.brusco-mail:2,S" | /usr/lib/dovecot/deliver -f dko...@somedomain.com -d dko...@domainhidden.com 4 5 case $? in 6 0) echo "0" 7 ;; 8 64) echo "64" 9 ;; 10 67) echo "67" 11 ;; 12 78) echo "78" 13 ;; 14 77) echo "77" 15 ;; 16 75) echo "75" 17 ;; 18 *) echo "unknown" 19 esac 20 The file it is cat'ing is a valid email in an Maildir/cur file. It returns 75 (EX_TEMP_FAIL) every time, even run as root. The logs reveal: /var/log/dovecot/lda-deliver: 2011-03-17 23:35:39 deliver(dko...@somedomain2.com): Info: Loading modules from directory: /usr/lib/dovecot/modules/lda /var/log/dovecot/deliver.info: (nothing) /var/log/dovecot/deliver.err: (nothing) I tried the script on another machine with Dovecot 1.0.15-2.3+lenny1 with success if I run as root. Regards, David Koski dko...@sutinen.com > This really has me stumped and desperate. I upgraded Debian from 5...
[Dovecot] dovecot LDA: "temporary failure"
etc/postfix/mysql-email2email.cf virtual_gid_maps = static:5000 virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf virtual_transport = dovecot virtual_uid_maps = static:5000 From master.cf: dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient} # dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0 ext3 log_path: /var/log/dovecot/deliver.err info_log_path: /var/log/dovecot/deliver.info log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:/home/vmail/%d/%n/Maildir mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: log_path: /var/log/dovecot/lda-deliver.err info_log_path: /var/log/dovecot/lda-deliver.info log_path: /home/vmail/dovecot-deliver.log auth_socket_path: /var/run/dovecot/auth-master postmaster_address: info@[another-domainhidden].com mail_plugins: cmusieve global_script_path: /home/vmail/globalsieverc auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail Regards, David Koski da...@kosmosisland.com
Re: [Dovecot] multiple namespaces, Mac OS X
Never mind. I found: http://www.dovecot.org/doc/NEWS v1.0.rc29 2007-03-28 Timo Sirainen <[EMAIL PROTECTED]> IMAP: LIST "" "" didn't return anything if there didn't exist a namespace with empty prefix. This broke some clients I upgraded using backport binaries 1.0.10-1~bpo40+1 and it works with Mac OS X mail clients now. Regards, David On Tuesday 04 March 2008 14:37, David Koski wrote: > After installing a new server with postfix/dovecot I found that the Mac OS > X mail client does not see email in subfolders. It seemed to be related to > the namespace. From dovecot.conf: > > namespace private { > separator = . > prefix = INBOX. > inbox = yes > hidden = no > } > > After googling I found that I might need to add a default namespace with an > empty prefix so I added this: > > namespace private { > separator = . > prefix = > inbox = yes > hidden = no > } > > Then I was able to access mail in subfolders using the Mac mail client. > However, using Kmail I can see both namespaces, where all the folders are > listed under the inbox at the same level, and where all folders are listed > one level below the inbox. I don't really want to keep my "INBOX." prefix > so existing accounts remain unchanged. How do I solve this? > > The current dovecot.conf is below. > > Regards, > David Koski > [EMAIL PROTECTED] > > log_timestamp: %Y-%m-%d %H:%M:%S > login_dir: /var/run/dovecot/login > login_executable: /usr/lib/dovecot/imap-login > first_valid_uid: 104 > last_valid_uid: 104 > mail_extra_groups: mail > mail_location: /var/vmail > maildir_copy_with_hardlinks: yes > mail_plugins: quota imap_quota > imap_client_workarounds: outlook-idle delay-newmail > namespace: > type: private > separator: . > prefix: INBOX. > inbox: yes > auth default: > mechanisms: PLAIN LOGIN CRAM-MD5 APOP NTLM > user: nobody > passdb: > driver: pam > passdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: sql > args: /etc/dovecot/dovecot-sql.conf > userdb: > driver: prefetch > socket: > type: listen > client: > path: /var/spool/postfix/private/auth > mode: 432 > user: postfix > group: mail > master: > path: /var/run/dovecot/auth-master > mode: 432 > user: vmail > group: mail
[Dovecot] multiple namespaces, Mac OS X
After installing a new server with postfix/dovecot I found that the Mac OS X mail client does not see email in subfolders. It seemed to be related to the namespace. From dovecot.conf: namespace private { separator = . prefix = INBOX. inbox = yes hidden = no } After googling I found that I might need to add a default namespace with an empty prefix so I added this: namespace private { separator = . prefix = inbox = yes hidden = no } Then I was able to access mail in subfolders using the Mac mail client. However, using Kmail I can see both namespaces, where all the folders are listed under the inbox at the same level, and where all folders are listed one level below the inbox. I don't really want to keep my "INBOX." prefix so existing accounts remain unchanged. How do I solve this? The current dovecot.conf is below. Regards, David Koski [EMAIL PROTECTED] log_timestamp: %Y-%m-%d %H:%M:%S login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login first_valid_uid: 104 last_valid_uid: 104 mail_extra_groups: mail mail_location: /var/vmail maildir_copy_with_hardlinks: yes mail_plugins: quota imap_quota imap_client_workarounds: outlook-idle delay-newmail namespace: type: private separator: . prefix: INBOX. inbox: yes auth default: mechanisms: PLAIN LOGIN CRAM-MD5 APOP NTLM user: nobody passdb: driver: pam passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: prefetch socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: mail master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: mail
Re: [Dovecot] warning: connect to transport dovecot : No such file or directory
On Tuesday 19 February 2008 02:02, Uldis Pakuls wrote: > David Koski wrote: > > I have installed postfix, dovecot, mysql as per: > > > > http://wiki.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL > > > > ..on Debian Etch. I thought mailboxes were created by postfixadmin or > > postfix but I only get an error: > > > > warning: connect to transport dovecot: No such file or directory > > > > ..in the postfix log. No /var/vmail/example.com is created. After > > creating the domain directory and user directory, applying correct > > ownership, still it made no difference. I checked my main.cf: > > > > virtual_mailbox_base = /var/vmail > > "Connect to transport dovecot: No such file or directory:" > Check postfix's master.cf - search for line (transport definition) > "dovecot" - correct path to dovecot's deliver binaries. For the record, the dovecot declration in master.cf started with a space and was ignored. Thanks! David Koski [EMAIL PROTECTED]
[Dovecot] warning: connect to transport dovecot: No such file or directory
I have installed postfix, dovecot, mysql as per: http://wiki.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL ..on Debian Etch. I thought mailboxes were created by postfixadmin or postfix but I only get an error: warning: connect to transport dovecot: No such file or directory ..in the postfix log. No /var/vmail/example.com is created. After creating the domain directory and user directory, applying correct ownership, still it made no difference. I checked my main.cf: virtual_mailbox_base = /var/vmail Any clues? Thanks! David Koski [EMAIL PROTECTED]