Remote IP address in dovecot
Hello, I am using a webmail, SOGo, to access Dovecot mail server in IMAP. SOGo correctly logs the remote IP address, using the standard nginx headers, especially X-Forwarded-For. However, when I check the IMAP logs, I see my server's IP address for the "remote_ip" value. I think there is a way to override this value from the environment, but I could not find it. Thanks for your help, -- André Rodier ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
recipient delimiter option in dovecot
Hello, all. Postfix allows me to configure multiple characters to be used as recipient delimiter. However, I don't see that in dovecot, it seems I can only use one character in recipient_delimiter ? Am I right ? Does dovecot would only let me use email+...@domain.com, or can I use other forms, for instance email~b...@domain.com and email&b...@domain.com ? Thanks. André -- 𝓐𝓡 - 𝐴𝑛𝑑𝑟𝑒 𝑅𝑜𝑑𝑖𝑒𝑟 ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Re: [Fail2ban-users] Postfix: running a script on authentication failure
On Thu, 2023-06-22 at 16:27 +0100, Nick Howitt via Fail2ban-users wrote: > > > On 2023-06-22 12:58, André Rodier via Fail2ban-users wrote: > > Hello, all. > > > > I just set-up a new server, running postfix, with submission(s) > > activated on standard ports (587, 465) > > > > Shortly after it has been setup, I see brute force attacks (not > > surprising) from a whole /24 network (more surprising). > > > > I carefully checked the logs, and see the modus operandi, which > > basically loop across the IP addresses in the network, > > to avoid being blacklisted by tools like fail2ban. And it is true, > > even with fail2ban activated, no IP is blacklisted. > > > > By activating verbose logging, I see multiple user names are tried, > > not only passwords. > > > > Is there any way, with postfix, to run a script on authentication > > failure, with information like the IP address and the > > username passed, for instance. > > > > I basically need features that fail2ban doesn't offer > > > > - I would like to not rely on reading logs, removing one step and > > acting more pro-actively. > > - If a script is called on authentication failure, it is fairly easy > > to use a Levenshtein distance to differentiate > > between a user having lost his password and a brute force attack. > > - If I log all the failure in a database, with the IP address, and the > > whois information, the script would take decision > > according to the whois information. > > > > What are you using on your side ? > > > > - Do you know any service, that I could use, to get the network to ban > > from an IP address reputation, something like > > crowdsec, for instance ? > > - Anyone has success with Suricata, Snort, or a tool like this ? > > > > Please, do not suggest third party hosted services, I want to be part > > of my self-hosting solution. > > > > Kind regards, > > André > > Are you sure the attacks are on port 465/587. All the big ones I used to > see were on 25 with user/pass. There is still little action on 587 as > far as I can see. There is a bit. I don't use 465. > > What I have done is come at it from a completely different angle. Don't > allow authentication on 25! If anyone needs to authenticate they have to > use 587. If you're lazy you can allow unauthenticated connections from > your LAN to save reconfiguring all internal devices, but for external > devices, port 587 only. You still leave 25 open as you need it to > receive emails. > > > ___ > Fail2ban-users mailing list > fail2ban-us...@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fail2ban-users Yes, it is definitely happening on submission. I will probably end-up using a VPN for submission and not expose these ports on internet. Kind regards, André ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Postfix: running a script on authentication failure
Hello, all. I just set-up a new server, running postfix, with submission(s) activated on standard ports (587, 465) Shortly after it has been setup, I see brute force attacks (not surprising) from a whole /24 network (more surprising). I carefully checked the logs, and see the modus operandi, which basically loop across the IP addresses in the network, to avoid being blacklisted by tools like fail2ban. And it is true, even with fail2ban activated, no IP is blacklisted. By activating verbose logging, I see multiple user names are tried, not only passwords. Is there any way, with postfix, to run a script on authentication failure, with information like the IP address and the username passed, for instance. I basically need features that fail2ban doesn't offer - I would like to not rely on reading logs, removing one step and acting more pro-actively. - If a script is called on authentication failure, it is fairly easy to use a Levenshtein distance to differentiate between a user having lost his password and a brute force attack. - If I log all the failure in a database, with the IP address, and the whois information, the script would take decision according to the whois information. What are you using on your side ? - Do you know any service, that I could use, to get the network to ban from an IP address reputation, something like crowdsec, for instance ? - Anyone has success with Suricata, Snort, or a tool like this ? Please, do not suggest third party hosted services, I want to be part of my self-hosting solution. Kind regards, André ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Postfix: running a script on authentication failure
Hello, all. I just set-up a new server, running postfix, with submission(s) activated on standard ports (587, 465) Shortly after it has been setup, I see brute force attacks (not surprising) from a whole /24 network (more surprising). I carefully checked the logs, and see the modus operandi, which basically loop across the IP addresses in the network, to avoid being blacklisted by tools like fail2ban. And it is true, even with fail2ban activated, no IP is blacklisted. By activating verbose logging, I see multiple user names are tried, not only passwords. Is there any way, with postfix, to run a script on authentication failure, with information like the IP address and the username passed, for instance. I basically need features that fail2ban doesn't offer - I would like to not rely on reading logs, removing one step and acting more pro-actively. - If a script is called on authentication failure, it is fairly easy to use a Levenshtein distance to differentiate between a user having lost his password and a brute force attack. - If I log all the failure in a database, with the IP address, and the whois information, the script would take decision according to the whois information. What are you using on your side ? - Do you know any service, that I could use, to get the network to ban from an IP address reputation, something like crowdsec, for instance ? - Anyone has success with Suricata, Snort, or a tool like this ? Please, do not suggest third party hosted services, I want to be part of my self-hosting solution. Kind regards, André ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org
Submission behaviour
Hi,
I have used fail2ban for a while, to block brute force attacks on ssh, imap(s)
or submission(s) ports.
More because I wanted to reduce the noise in the logs rather than a fear of a
broken password.
Then, with nftables, I realised you can achieve the same thing, as long as a
TCP connection isn't close. This works very
well for SSH, but I then realised it works for a modern IMAP server that
supports IDLE, since the connection is kept
open, for instance the excellent Dovecot mail server.
Here an example, of nftable ruleset, for dovecot imap(s):
---
table inet filter {
set banned_imap_ipv4 {
type ipv4_addr
flags dynamic,timeout
timeout 1d
}
set banned_imap_ipv6 {
type ipv6_addr
size 65535
flags dynamic,timeout
timeout 1d
}
chain input {
# Limit new imap connections ala fail2ban
meta nfproto ipv4 tcp dport imaps ct state new,untracked \
limit rate over 10/minute add @banned_imap_ipv4 { ip saddr }
meta nfproto ipv6 tcp dport imaps ct state new,untracked \
limit rate over 10/minute add @banned_imap_ipv6 { ip6 saddr }
# Reject the traffic explicitly
ip saddr @banned_imap_ipv4 tcp dport imaps reject with icmp type
admin-prohibited
ip6 saddr @banned_imap_ipv6 tcp dport imaps reject with icmpv6 type
admin-prohibited
tcp dport { imap, imaps } ct state new counter accept \
comment "Accept imap/imaps connections"
}
}
---
Surprisingly, this is working very well with Dovecot, and various modern
clients like Evolution or Thunderbird, as well
as K9 on Android.
There is also a way to save the rules before restarting the firewall, which
works very well as well:
---
# nft list set inet filter banned_imap_ipv4
table inet filter {
set banned_imap_ipv4 {
type ipv4_addr
size 65535
flags dynamic,timeout
timeout 1d
elements = { 162.142.125.214 timeout 1d expires 23h44m16s600ms }
}
}
---
Now, the question I have is this.
I can limit new TCP connections to a reasonable amount, like 10 per minute,
because I know I will not try to send that
amount of emails from a single IP.
However, is there an option, in Postfix, to keep the TCP connection opened for
submission(s) protocols (ports 465 or
587)
Thanks for your insights.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org
Re: [SOLVED] Permissions for dovecot logging
On 30 December 2022 22:25:09 CET, James Moe wrote: >On 2022-12-27 16:19, James Moe wrote: > >> I changed logging to use a path rather than syslog. Doing so makes it >> easier >> to work with fail2ban. >> Dovecot fails to start with the error: >> Can't open log file /data01/var/log/dovecot.log: Permission denied >> > Yes, it was apparmor. It has been enabled for a couple of months. Dovecot is >the first app that I've added that has a apparmor profile. After adding the >necessary entry to the profile, logging proceeded as expected. > >In I added: > owner /data01/var/log/dovecot/* a, > > Hi, When an application is denied access by AppArmor, you can see the logs in syslog. Anyway, if that helps, have s look here: https://github.com/progmaticltd/homebox/tree/main/roles/dovecot/templates/apparmor.d André.
Re: Self hosting solution for Christmas
On 29 December 2022 09:10:23 CET, Joachim Lindenberg wrote: >Hello André, >thanks for the explanations, appreciated, and for sure publishing a comparison >would help users to make a decision, where to "shop", and maybe also for >enthusiast the opportunity to join forces on specific topics. >I like that you support FDE, but my personal preference is to run *x as >virtual machines on Hyper-V with Bitlocker and Bitlocker Network Unlock. I >haven´t looked into Clevis & Tang yet in detail, which might be an alternative. >I decided for mailcow early 2018 where it met my requirements, but I am also >open to alternatives, especially if they are on par or close w.r.t. >functionality, ideally offering high availability via two replicating >instances (mailcow does this commercially only and didn´t offer a GDPR >compliant contract). >Thanks, >Joachim > >-Ursprüngliche Nachricht- >Von: André Rodier >Gesendet: Donnerstag, 29. Dezember 2022 08:44 >An: dovecot@dovecot.org; Joachim Lindenberg >Betreff: Re: Self hosting solution for Christmas > >On 27 December 2022 11:39:42 CET, Joachim Lindenberg >wrote: >>I have to support Marc´s question. And also - what makes HomeBox different >>from Mailcow (https://mailcow.email/)? >>Thanks, Joachim >> >>-Ursprüngliche Nachricht- >>Von: dovecot Im Auftrag von Marc >>Gesendet: Dienstag, 27. Dezember 2022 11:25 >>An: Andre Rodier ; dovecot@dovecot.org; >>postfix-us...@postfix.org; debian-u...@lists.debian.org; >>users-requ...@sogo.nu >>Betreff: RE: Self hosting solution for Christmas >> >>> >>> Here my present for Christmas: a new version of HomeBox, the self >>> hosted email solution. >>> >>> Feel free to drop comments, create issues, update the docs, etc. >>> >>> I released this quickly before going on vacation, so you may find >>> some issues. However, this is mostly stable, and the code is easy to modify. >>> >> >>That is why one should not be interested to much risk of lacking future >>support. What if your wife gets pregnant and there is no update/release for 9 >>months? ;) Obviously I admire such open source efforts. >>It is just such a pity to see so many projects initiated seemingly without >>first trying to bundle forces. This is especially visible in crm all these >>individual projects are 'shitty', I do not get why none of them try and work >>together to create a few good ones. >> >>I used to always state that there is only one real distribution you could >>use, and that is the centos one. Basically because you could always buy a >>redhat license and get the support of a billion dollar company (now even >>IBM), but with their stream direction this all becomes questionable. However >>most projects do not even have an argument other than 'this is the >>distribution I know'. >> >>The only long term alternative I see, is using containers that hardly have >>any os dependency and behave more like micro services. So you focus on the >>direct updates of suppliers. >> >> >> >> > >Hello, Joachim. > >Perhaps I need to rewrite the doc, and the readme, so I will clarify a few >points. > >Homebox is a set of Ansible scripts to install and configure an email stack on >Debian. Exactly like you would do it manually, but in an automated way. > >Once the play book has been run, you still have a Debian installed, without >any custom binary. > >Therefore, of you need any support, ask the relevant making lists, like >postfix, dovecot, sogo, Debian, clamav, rspamd, etc... > >Now, to answer your question, I had a look to mailcow, and I still prefer >Homebox to hosts my emails. > >The security of my primary concern. If you look the code carefully, you will >see a lot of decisions in this direction. From the list of authorised ciphers >and the enforced encryption, even internally, to the absence of PHP. Also, the >non-free and contrib sections are excluded. > >I also offers full disk encryption out of the box using Debian preseed with >remote drive unlocking. > >You will also see a lot of unit tests to ensure the whole stack is running as >expected. > >Finally, I trust a lot the Debian community security policies. I prefer to use >them than another community, especially with the unattended-upgrades package. > >In terms of features, again, we're definitely not on the same line. > >Homebox does not support multiple domains, and will never. > >However, I use an LDAP server for authentication, which is used for other >services, l
Re: Self hosting solution for Christmas
On 27 December 2022 11:39:42 CET, Joachim Lindenberg wrote: >I have to support Marc´s question. And also - what makes HomeBox different >from Mailcow (https://mailcow.email/)? >Thanks, Joachim > >-Ursprüngliche Nachricht- >Von: dovecot Im Auftrag von Marc >Gesendet: Dienstag, 27. Dezember 2022 11:25 >An: Andre Rodier ; dovecot@dovecot.org; >postfix-us...@postfix.org; debian-u...@lists.debian.org; users-requ...@sogo.nu >Betreff: RE: Self hosting solution for Christmas > >> >> Here my present for Christmas: a new version of HomeBox, the self >> hosted email solution. >> >> Feel free to drop comments, create issues, update the docs, etc. >> >> I released this quickly before going on vacation, so you may find some >> issues. However, this is mostly stable, and the code is easy to modify. >> > >That is why one should not be interested to much risk of lacking future >support. What if your wife gets pregnant and there is no update/release for 9 >months? ;) Obviously I admire such open source efforts. >It is just such a pity to see so many projects initiated seemingly without >first trying to bundle forces. This is especially visible in crm all these >individual projects are 'shitty', I do not get why none of them try and work >together to create a few good ones. > >I used to always state that there is only one real distribution you could use, >and that is the centos one. Basically because you could always buy a redhat >license and get the support of a billion dollar company (now even IBM), but >with their stream direction this all becomes questionable. However most >projects do not even have an argument other than 'this is the distribution I >know'. > >The only long term alternative I see, is using containers that hardly have any >os dependency and behave more like micro services. So you focus on the direct >updates of suppliers. > > > > Hello, Joachim. Perhaps I need to rewrite the doc, and the readme, so I will clarify a few points. Homebox is a set of Ansible scripts to install and configure an email stack on Debian. Exactly like you would do it manually, but in an automated way. Once the play book has been run, you still have a Debian installed, without any custom binary. Therefore, of you need any support, ask the relevant making lists, like postfix, dovecot, sogo, Debian, clamav, rspamd, etc... Now, to answer your question, I had a look to mailcow, and I still prefer Homebox to hosts my emails. The security of my primary concern. If you look the code carefully, you will see a lot of decisions in this direction. From the list of authorised ciphers and the enforced encryption, even internally, to the absence of PHP. Also, the non-free and contrib sections are excluded. I also offers full disk encryption out of the box using Debian preseed with remote drive unlocking. You will also see a lot of unit tests to ensure the whole stack is running as expected. Finally, I trust a lot the Debian community security policies. I prefer to use them than another community, especially with the unattended-upgrades package. In terms of features, again, we're definitely not on the same line. Homebox does not support multiple domains, and will never. However, I use an LDAP server for authentication, which is used for other services, like a Jabber server. The solution includes a Jabber server out of the box, with files upload and server to server communication. Next year, I will start to include a Prometheus stack, with alerts sent by xmpp. I am also planning to add more features i think can be useful to personal hosting, still using Debian repositories. For instance, a WebDAV server to share files across multiple devices. I don't pretend creating a better solution than X or Y, and I may add mailcow in the list of other solutions. However, I think some people, like me, just want to deploy a mail / xmpp server on Debian without third party packages. This is why I created this project. Kind regards, André. PS : for Marc's knowledge, I am very happy with the kids I already have. I had a surgery to ensure I won't have more. Maybe an example to follow...
Re: Strategies for protecting IMAP (e.g. MFA)
On 15/11/2021 11:52, Arjen de Korte wrote:
Citeren Benny Pedersen :
On 2021-11-14 20:26, Matthew Richardson wrote:
On Sun, 14 Nov 2021 08:12:53 -0800, Michael Peddemors wrote:-
And there are RBL's now for know IP(s) used by IMAP hackers, including
SpamRats RATS-AUTH that can assist in reducing those attacks.
Looking at https://www.spamrats.com/rats-auth.php the "Example Usage in
Dovecot" says "PLEASE UPDATE".
How would one use a DNSBL like this in Dovecot to reject IMAP
connections
from listed IPs?
submission inet n - y - - smtpd
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_delay_reject=no
-o { smtpd_client_restrictions = reject_rbl_client
auth.spamrats.com=127.0.0.39, permit }
-o { smtpd_relay_restrictions = permit_mynetworks,
permit_sasl_authenticated, reject }
This is not an answer to the question, this is Postfix syntax.
openRelay, dont do it
In what way would this create an open relay exactly? The 'permit' at the
end of the 'smtpd_client_restrictions' only means that the client is
accepted, not that other smtpd restrictions are lifted.
resolved version
submission inet n - y - - smtpd
-o smtpd_tls_security_level=encrypt
-o smtpd_sasl_auth_enable=yes
-o smtpd_delay_reject=no
-o { smtpd_relay_restrictions = reject_rbl_client
auth.spamrats.com=127.0.0.39, permit_mynetworks,
permit_sasl_authenticated, reject }
Although syntactically correct, it is confusing at best to put client
restrictions in another place than smtpd_client_restrictions. Especially
with 'smtpd_delay_reject=no' in effect you'd only reject after receiving
'RCPT TO', which is evaluated after 'smtpd_client_restrictions' and
'smtpd_helo_restrictions' during the SMTP transfer.
order do matter
Indeed.
Perhaps I was not clear in my last message. Have a look to this
documentation:
https://homebox.readthedocs.io/en/latest/email-access-monitoring/
I am available if you have any question to implement something similar
yourself. Extending the system to add a second factor authentication is
probably easy enough.
Kind regards,
André
--
𝓐𝓡 - André Rodier
Re: Strategies for protecting IMAP (e.g. MFA)
On 14/11/2021 18:03, Lefteris Tsintjelis wrote: On 14/11/2021 14:50, Kees van Vloten wrote: Apart from a really nice firewall firehol also supplies a good set of ip-blacklists. For public exposure of email ports, I am using the combination of firehol-firewall, firehol-blacklist, fail2ban and a whitelist based on geo-ip. The mail-client ports exposed are 993 and 465, because starttls is considered flawed nowadays: https://nostarttls.secvuln.info/) Full access from any IP (except firehol-blacklist and fail2ban) is possible over VPN (openvpn) with MFA (privacyidea). Privacyidea also supplies a mobile-app compatible with a.o. TOTP and HOTP but it provides a more secure way of enrollment (2-step). Thanks for pointing at crowdsec.net, will see if it can tighten security further in cooperation with the above. - Kees The problem I faced over the years, with so many IPs, was that the black listing way would reach its limits at some point. Using the classic fail2ban expiration dates and method, over time, never actually manages to get rid of them as they keep on trying and trying. I needed to expand the blacklist expiration time limits way high but that reached firewall limitations so I personally switched to a permanent white list firewalling, as I could do that, and it really got rid of a lot of my headaches with just about all my public services. Black listing would work in case of central dedicated anf large firewalls but for smaller solutions I think country white listing firewall is far better method. What would also be interesting is something similar to the spamcop combined with crowdsec reporting system so that it can be used to effectively analyze and reduce all those bots. The Spamhouse DROP list would also be a good permanent black list addition to any border routers or stand alone public services. https://www.spamhaus.org/drop/ Perhaps I was not clear in my last message. Have a look to this documentation: https://homebox.readthedocs.io/en/latest/email-access-monitoring/ I am available if you have any question to implement something similar yourself. Extending the system to add a second factor authentication is probably easy enough. Kind regards, André -- 𝓐𝓡 - André Rodier
Re: Strategies for protecting IMAP (e.g. MFA)
On 13/11/2021 23:34, lists wrote: The thing I don't like is most 2FA token generators. Ultimately you need to transfer the polynomial that generates the code. Most do that with a QR image. Well so much for security! Others have a one time emergency code. Of course we are talking evil maid attacks, which granted is an unacceptable term these days. Now Yubikey at least has my attention. But people often leave the key plugged into their notebook. Very true with the Google equivalent which I have heard from Google employees. The keys themselves aren't exactly transferable, but when you have physical access then all bets are off. If someone fool actually paid me to be sysadmin, I would use a Yubikey. Note Freeotp let's you input the code but also has the QR code fallback. The phone app however hasn't been updated in years. It does allow you to test out a TOTP scheme. It took me no time to write a script to accept the token on Linux. The tricky part if I recall correctly was setting up the script to accept the token that just expired. You would want to do that to minimize user friction. Not to get too far off track but I don't allow any web control over my email server. There is no control panel to hack. I ssh into the server and that uses PKI. I do everything via CLI. If ssh is compromised then nothing else will be secure so email would be the last of my problems. Companies such as Last pass (not an endorsement but an example) supposedly incorporate password generators. If you are going to allow users to set let alone change their own password, you might be able to write a script that generates the password. If I were to go up to the next level of security I would use mail-crypt. It is just that I see so much chatter about getting it to work. *From:* montneyty...@gmail.com *Sent:* November 13, 2021 3:03 PM *To:* dovecot@dovecot.org *Subject:* Re: Strategies for protecting IMAP (e.g. MFA) "Use strong (as in long and/or randomised and impossible to break using rainbow table attacks) password" Again, since it's just me, this is do-able. But I'm looking for something practical as well. I'm getting the feeling that people don't have an MFA implementation. "if the users are sufficiently discipline" As a Sysadmin, I can tell you they genuinely are not and they likely never will be. Hope for the best, plan for the worst. I also want to clarify that I'm not rejecting any of these suggestions, they're all good. On Sat, Nov 13, 2021 at 4:42 PM Ralph Seichter <mailto:ra...@ml.seichter.de>> wrote: * Tyler Montney: > Since this is getting increasingly complicated, I wanted to ask before > going further. What do you all do? Any recommendations? Use strong (as in long and/or randomised and impossible to break using rainbow table attacks) passwords which are used only once (!) and kept either in the user's brain or in an encrypted password store. Ensure that authentication data can only be transmitted over encrypted connections. These measures cover a lot of ground, if the users are sufficiently disciplined. Users are usually the weakest link. -Ralph I almost reached this stage with a personal / open source project I am working on. It is based on Dovecot login scripts, and ejabberd to send alerts in XMPP, from the postmaster account. The details: Custom dovecot login scripts compute a "confidence" score, with two thresholds. The first lower threshold raises a warning sent by xmpp with some details, while the second threshold simply deny the connection. A few of us have been using it for a while, both on mobile and desktop. The current stable version is based on Stretch, as Buster has too many issues, We are now working on the bullseye version. It is perfectly valid to extend the Dovecot custom script to include for instance Duo authentication. Another option would be to extend a Dovecot custom login script to wait for an answer on xmpp message sent by postmaster, or even something crazier like a HOTP / TOTP code. Good luck. Homebox: https://github.com/progmaticltd/homebox -- 𝓐𝓡 - André Rodier -- 𝓐𝓡 - André Rodier
Re: imapsieve and master user
On Wed, 2021-04-07 at 02:13 +0200, Stephan Bosch wrote: > > On 04/04/2021 22:19, André Rodier wrote: > > Hello all, > > > > I have am imap sieve script, running fine. > > > > I would like to check, if it is possible, inside the sieve script, > > if > > the current user is a master user or not. > > I believe this is probably not possible. > > > > Otherwise, if it is not possible, I would like to check for an > > external > > condition, like an existing file in /tmp or in the mail dir folder, > > for > > instance. > > > > The issue is that I want my imap sieve script behave differently if > > the > > master user is connected. > > > > Thanks for any piece of advice. > > Maybe you could fudge something by adding a sieve_env_ field to > master > user's userdb lookup and configuring the vnd.dovecot.environment > extension: > > https://github.com/dovecot/pigeonhole/blob/master/doc/extensions/vnd.dovecot.environment.txt > https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt Thanks, Stephan, I ended up using another method. The goal was to prevent email clients to copy emails in the Sent folder, but still allow import emails procedure to append emails using the APPEND verb in this folder. I used sieve include, and added "keep;stop" in the included file during the import procedure. However, thanks to you, I know know what is possible. Kind regards, André > > > Kind regards, > > André Rodier > >
Re: Sv: Sv: Sv: Sv: Avoid duplicate emails in the sent folder
Sorry for the top posting. Here the solution I found, without using a secondary Sent folder: My sieve script is like this: require [ "imapsieve", "include" ]; include :personal :optional "pre-sent-checks"; discard; During the import phase, the sieve user script "pre-sent-checks.sieve" is created with this content: keep;stop; Once the import phase is finished, the user script contains this: require ["include"]; return; This is very well enough for me. The import takes just a few seconds, I don't have race conditions for now. I prefer doing this than createing a secondary Sent folder. On Sun, 2021-04-04 at 14:43 +0200, Sebastian wrote: > Why won't it work to just, as I said, create a separate folder, like > Sent2, which you use as a "midpoint" to land all your externally > imported emails, and then you just move everything from Sent2 to > Sent. > > -Ursprungligt meddelande- > Från: dovecot-boun...@dovecot.org För > André Rodier > Skickat: den 4 april 2021 14:40 > Till: Dovecot Mailing List > Ämne: Re: Sv: Sv: Sv: Avoid duplicate emails in the sent folder > > On Sun, 2021-04-04 at 14:31 +0200, Sebastian wrote: > > Also another important thing to take in consideration: > > > > An email that is sent outgoing from an SMTP server, isn't a exact > > copy > > of the email put into the SMTP server. > > Thats why a deduplicate program/command/filter won't work, as the > > 2 > > emails aren't 100% identical. > Thanks, I see this as the main issue > > > The 2 "duplicates" arent technically 100% identical. Thats why the > > only solution is to prevent the IMAP client from writing new > > "unknown" > > emails into the Sent folder, and then instead use a separate > > folder > > for importing emails across accounts (from which you then > > COPY/MOVE > > emails from internally in the same acount, to the Sent folder). > > I continue to search on a simple solution, and I will come back to > you! > > > -Ursprungligt meddelande- > > Från: dovecot-boun...@dovecot.org > > För > > André Rodier > > Skickat: den 4 april 2021 14:10 > > Till: Dovecot Mailing List > > Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder > > > > Hello Sebastian, > > > > On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote: > > > Thats why you need the exim4 filter too. > > > > I obtained the exact same result with postfix, using +Sent and a > > sieve > > filter. Sent emails are automatically bcc'ed to the user, using the > > +Sent email address. > > > > However, I still want to be able to drag and drop emails to the > > sent > > folder, which unfortunately uses the APPEND verb. > > > > Also, there is a mail import wizard that import emails from > > gmail/yahoo/etc, and there the verb used is APPEND. > > > > I am now investigating the doveadm deduplicate program. > > > > Kind regards, > > André > > > > > Because, my version basically prohibits the IMAP client from > > > writing > > > to the Sent folder at all, and gives that job to the SMTP server > > > instead. > > > Works perfectly for me, all sent emails gets into the sent > > > folder > > > without any hitch. > > > > > > Without any duplicates since no IMAP client is permitted to > > > write > > > into the Sent folder. > > > > > > Note that only APPEND should be specifyed, not COPY, meaning that > > > if > > > you COPY from the same IMAP account to Sent folder, it will be > > > permitted, but if you APPEND (write a new email) to the Sent > > > folder, > > > it will get auto-deleted. > > > > > > -Ursprungligt meddelande- > > > Från: dovecot-boun...@dovecot.org > > > För > > > André Rodier > > > Skickat: den 4 april 2021 13:57 > > > Till: Dovecot Mailing List > > > Ämne: Re: Sv: Avoid duplicate emails in the sent folder > > > > > > Thanks, Sebastian. > > > > > > I just tried this, but it is working too well. > > > > > > If I want to import emails from another Sent folder, using drag > > > & > > > drop from my email client, the ema
Re: failed to compile a sieve script with optional include
On Mon, 2021-04-05 at 10:19 +0100, André Rodier wrote:
> Hi,
>
> I am trying to compile the following sieve script:
>
> -
> require [
> "variables",
> "include",
> "relational",
> "imapsieve"
> ];
>
> global "importing";
>
> include :personal :optional "pre-sent-checks";
>
> if string :count "eq" "${importing}" "1"
> {
> keep;
> stop;
> }
> -
>
> But I have this error:
>
> > sent-checks: line 15: error: include: personal location for
> > included
> > script `pre-sent-checks' is unavailable (contact system
> > administrator
> > for more information).
> > sent-checks: error: validation failed.
> > sievec(root): Fatal: failed to compile sieve script 'sent-
> checks.sieve'
>
> What should I do to compile a script with an optional user script,
> please ?
>
> Thanks
I realised that I don't need to compile the script myself, and that
dovecot is compiling it on demand, without error.
I hope this helps others.
André
failed to compile a sieve script with optional include
Hi,
I am trying to compile the following sieve script:
-
require [
"variables",
"include",
"relational",
"imapsieve"
];
global "importing";
include :personal :optional "pre-sent-checks";
if string :count "eq" "${importing}" "1"
{
keep;
stop;
}
-
But I have this error:
> sent-checks: line 15: error: include: personal location for included
> script `pre-sent-checks' is unavailable (contact system administrator
> for more information).
> sent-checks: error: validation failed.
> sievec(root): Fatal: failed to compile sieve script 'sent-
checks.sieve'
What should I do to compile a script with an optional user script,
please ?
Thanks
imapsieve and master user
Hello all, I have am imap sieve script, running fine. I would like to check, if it is possible, inside the sieve script, if the current user is a master user or not. I believe this is probably not possible. Otherwise, if it is not possible, I would like to check for an external condition, like an existing file in /tmp or in the mail dir folder, for instance. The issue is that I want my imap sieve script behave differently if the master user is connected. Thanks for any piece of advice. Kind regards, André Rodier
Re: Avoid duplicate emails in the sent folder
On Sun, 2021-04-04 at 06:45 -0600, @lbutlr wrote: > On 04 Apr 2021, at 06:39, André Rodier wrote: > > I continue to search on a simple solution, and I will come back to > > you! > > Seems the best solution is to get a web client that properly puts > messages into DRAFTS and not appends them to Sent, but I guess that > is too much to hope for? > > I'd certainly at least complain about this misbehavior. > I keep sending emails to the SOGo mailing, but it is very hard - if not impossible - to get an aswer. Even worst, my emails to the list are now silently dropped. I am not proficient enough in Objective-C to submit a patch, but yes, the behaviour is despicable. André
Re: Sv: Sv: Sv: Avoid duplicate emails in the sent folder
On Sun, 2021-04-04 at 14:31 +0200, Sebastian wrote:
> Also another important thing to take in consideration:
>
> An email that is sent outgoing from an SMTP server, isn't a exact
> copy of the email put into the SMTP server.
> Thats why a deduplicate program/command/filter won't work, as the 2
> emails aren't 100% identical.
Thanks, I see this as the main issue
>
> The 2 "duplicates" arent technically 100% identical. Thats why the
> only solution is to prevent the IMAP client from writing new
> "unknown" emails into the Sent folder, and then instead use a
> separate folder for importing emails across accounts (from which you
> then COPY/MOVE emails from internally in the same acount, to the Sent
> folder).
I continue to search on a simple solution, and I will come back to you!
>
> -----Ursprungligt meddelande-
> Från: dovecot-boun...@dovecot.org För
> André Rodier
> Skickat: den 4 april 2021 14:10
> Till: Dovecot Mailing List
> Ämne: Re: Sv: Sv: Avoid duplicate emails in the sent folder
>
> Hello Sebastian,
>
> On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
> > Thats why you need the exim4 filter too.
>
> I obtained the exact same result with postfix, using +Sent and a
> sieve filter. Sent emails are automatically bcc'ed to the user, using
> the
> +Sent email address.
>
> However, I still want to be able to drag and drop emails to the sent
> folder, which unfortunately uses the APPEND verb.
>
> Also, there is a mail import wizard that import emails from
> gmail/yahoo/etc, and there the verb used is APPEND.
>
> I am now investigating the doveadm deduplicate program.
>
> Kind regards,
> André
>
> > Because, my version basically prohibits the IMAP client from
> > writing
> > to the Sent folder at all, and gives that job to the SMTP server
> > instead.
> > Works perfectly for me, all sent emails gets into the sent folder
> > without any hitch.
> >
> > Without any duplicates since no IMAP client is permitted to write
> > into
> > the Sent folder.
> >
> > Note that only APPEND should be specifyed, not COPY, meaning that
> > if
> > you COPY from the same IMAP account to Sent folder, it will be
> > permitted, but if you APPEND (write a new email) to the Sent
> > folder,
> > it will get auto-deleted.
> >
> > -Ursprungligt meddelande-
> > Från: dovecot-boun...@dovecot.org
> > För
> > André Rodier
> > Skickat: den 4 april 2021 13:57
> > Till: Dovecot Mailing List
> > Ämne: Re: Sv: Avoid duplicate emails in the sent folder
> >
> > Thanks, Sebastian.
> >
> > I just tried this, but it is working too well.
> >
> > If I want to import emails from another Sent folder, using drag &
> > drop
> > from my email client, the emails are deleted in the background.
> >
> > This is why I am trying to use the duplicate filter.
> >
> > Kind regards,
> > André Rodier
> >
> > On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
> > > I have a sieve filter called "sent.sieve"
> > >
> > > Where I just do this:
> > > discard;
> > >
> > > Then in sieve settings:
> > >
> > > plugin {
> > > sieve_plugins = sieve_imapsieve sieve_extprograms
> > > sieve_global_extensions = +vnd.dovecot.execute
> > > sieve_execute_bin_dir = /usr/bin/
> > > imapsieve_mailbox2_name = Sent
> > > imapsieve_mailbox2_causes = APPEND
> > > imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve
> > > imapsieve_expunge_discarded=yes
> > > }
> > >
> > >
> > > And then in Exim4 SMTP server:
> > >
> > >
> > > sentfolder.filter:
> > > if $sender_address_domain is sebbe.eu then unseen save
> > > /var/mail/asterisk2/Sent endif
> > >
> > > in exim4 config:
> > > system_filter = /etc/exim4/sentfolder.filter
> > > system_filter_file_transport = sentfolder
> > >
> > > in transports of exim4.conf:
> > >
> > > sentfolder:
> > > debug_print = "T: Sentfolder for $local_part@$domain"
> > > headers_add = Status: RO
> > > headers_remove = x-dkim-signature
> > > headers_remove = x-spf-signature
> > > headers_remove = x-trusted-domain
> > > headers_remove = keywords
> > > headers_remove = x-priority
> > > headers_remove = x-status
> > > driver = appendfile
> > > delivery_date_add
> >
Re: Sv: Sv: Avoid duplicate emails in the sent folder
Hello Sebastian,
On Sun, 2021-04-04 at 14:00 +0200, Sebastian wrote:
> Thats why you need the exim4 filter too.
I obtained the exact same result with postfix, using +Sent and a sieve
filter. Sent emails are automatically bcc'ed to the user, using the
+Sent email address.
However, I still want to be able to drag and drop emails to the sent
folder, which unfortunately uses the APPEND verb.
Also, there is a mail import wizard that import emails from
gmail/yahoo/etc, and there the verb used is APPEND.
I am now investigating the doveadm deduplicate program.
Kind regards,
André
> Because, my version basically prohibits the IMAP client from writing
> to the Sent folder at all, and gives that job to the SMTP server
> instead.
> Works perfectly for me, all sent emails gets into the sent folder
> without any hitch.
>
> Without any duplicates since no IMAP client is permitted to write
> into the Sent folder.
>
> Note that only APPEND should be specifyed, not COPY, meaning that if
> you COPY from the same IMAP account to Sent folder, it will be
> permitted, but if you APPEND (write a new email) to the Sent folder,
> it will get auto-deleted.
>
> -Ursprungligt meddelande-
> Från: dovecot-boun...@dovecot.org För
> André Rodier
> Skickat: den 4 april 2021 13:57
> Till: Dovecot Mailing List
> Ämne: Re: Sv: Avoid duplicate emails in the sent folder
>
> Thanks, Sebastian.
>
> I just tried this, but it is working too well.
>
> If I want to import emails from another Sent folder, using drag &
> drop from my email client, the emails are deleted in the background.
>
> This is why I am trying to use the duplicate filter.
>
> Kind regards,
> André Rodier
>
> On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
> > I have a sieve filter called "sent.sieve"
> >
> > Where I just do this:
> > discard;
> >
> > Then in sieve settings:
> >
> > plugin {
> > sieve_plugins = sieve_imapsieve sieve_extprograms
> > sieve_global_extensions = +vnd.dovecot.execute
> > sieve_execute_bin_dir = /usr/bin/
> > imapsieve_mailbox2_name = Sent
> > imapsieve_mailbox2_causes = APPEND
> > imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve
> > imapsieve_expunge_discarded=yes
> > }
> >
> >
> > And then in Exim4 SMTP server:
> >
> >
> > sentfolder.filter:
> > if $sender_address_domain is sebbe.eu then unseen save
> > /var/mail/asterisk2/Sent endif
> >
> > in exim4 config:
> > system_filter = /etc/exim4/sentfolder.filter
> > system_filter_file_transport = sentfolder
> >
> > in transports of exim4.conf:
> >
> > sentfolder:
> > debug_print = "T: Sentfolder for $local_part@$domain"
> > headers_add = Status: RO
> > headers_remove = x-dkim-signature
> > headers_remove = x-spf-signature
> > headers_remove = x-trusted-domain
> > headers_remove = keywords
> > headers_remove = x-priority
> > headers_remove = x-status
> > driver = appendfile
> > delivery_date_add
> > envelope_to_add
> > return_path_add
> > use_flock_lock = yes
> > use_fcntl_lock = no
> > use_lockfile = no
> > group = mail
> > mode = 0777
> > mode_fail_narrower = false
> > current_directory = /
> >
> >
> >
> > that should solve it.
> >
> > -Ursprungligt meddelande-
> > Från: dovecot-boun...@dovecot.org
> > För
> > André Rodier
> > Skickat: den 4 april 2021 12:48
> > Till: dovecot@dovecot.org
> > Kopia: us...@sogo.nu
> > Ämne: Avoid duplicate emails in the sent folder
> >
> > Hello,
> >
> > I have a webmail program, that I cannot configure to not save sent
> > emails in the Sent folder. My emails to the SOGo team don't seem
> > to
> > reach the mailing list anymore.
> >
> > So, I have no other choice to think the problem differently, and
> > _maybe_ use Dovecot to fix this issue. Moreover, I think this could
> > be
> > useful for other email clients.
> >
> > First, I have tried to use Sieve, but no success so far:
> >
> > --
> > imapsieve_mailbox4_name = *
> > imapsieve_mailbox4_from = Sent
> > imapsieve_mailbox4_causes = APPEND, COPY
> > imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent-
> > checks.sieve
> > --
> >
> > And this sieve script:
> >
> > --
> > Message-ID header.
> > require [
> > "variables",
> > "environment",
> > "duplicate",
> > "imapsieve"
> > ];
> >
> > […]
> >
> > if duplicate :seconds 10 {
> >discard;
> > }
> > --
> >
> > Otherwise, can I configure dovecot in a certain way, for instance
> > create a virtual Sent folder. This would not save the email when
> > appended, but still returns the list of sent emails ?
> >
> > But this is not working. Any idea?
> >
> > Thanks for your help!
> >
> > André
> >
> >
>
>
Re: Sv: Avoid duplicate emails in the sent folder
Thanks, Sebastian.
I just tried this, but it is working too well.
If I want to import emails from another Sent folder, using drag & drop
from my email client, the emails are deleted in the background.
This is why I am trying to use the duplicate filter.
Kind regards,
André Rodier
On Sun, 2021-04-04 at 13:04 +0200, Sebastian wrote:
> I have a sieve filter called "sent.sieve"
>
> Where I just do this:
> discard;
>
> Then in sieve settings:
>
> plugin {
> sieve_plugins = sieve_imapsieve sieve_extprograms
> sieve_global_extensions = +vnd.dovecot.execute
> sieve_execute_bin_dir = /usr/bin/
> imapsieve_mailbox2_name = Sent
> imapsieve_mailbox2_causes = APPEND
> imapsieve_mailbox2_before = file:/etc/dovecot/sieve/sent.sieve
> imapsieve_expunge_discarded=yes
> }
>
>
> And then in Exim4 SMTP server:
>
>
> sentfolder.filter:
> if $sender_address_domain is sebbe.eu then
> unseen save /var/mail/asterisk2/Sent
> endif
>
> in exim4 config:
> system_filter = /etc/exim4/sentfolder.filter
> system_filter_file_transport = sentfolder
>
> in transports of exim4.conf:
>
> sentfolder:
> debug_print = "T: Sentfolder for $local_part@$domain"
> headers_add = Status: RO
> headers_remove = x-dkim-signature
> headers_remove = x-spf-signature
> headers_remove = x-trusted-domain
> headers_remove = keywords
> headers_remove = x-priority
> headers_remove = x-status
> driver = appendfile
> delivery_date_add
> envelope_to_add
> return_path_add
> use_flock_lock = yes
> use_fcntl_lock = no
> use_lockfile = no
> group = mail
> mode = 0777
> mode_fail_narrower = false
> current_directory = /
>
>
>
> that should solve it.
>
> -Ursprungligt meddelande-
> Från: dovecot-boun...@dovecot.org För
> André Rodier
> Skickat: den 4 april 2021 12:48
> Till: dovecot@dovecot.org
> Kopia: us...@sogo.nu
> Ämne: Avoid duplicate emails in the sent folder
>
> Hello,
>
> I have a webmail program, that I cannot configure to not save sent
> emails in the Sent folder. My emails to the SOGo team don't seem to
> reach the mailing list anymore.
>
> So, I have no other choice to think the problem differently, and
> _maybe_ use Dovecot to fix this issue. Moreover, I think this could
> be useful for other email clients.
>
> First, I have tried to use Sieve, but no success so far:
>
> --
> imapsieve_mailbox4_name = *
> imapsieve_mailbox4_from = Sent
> imapsieve_mailbox4_causes = APPEND, COPY
> imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent-
> checks.sieve
> --
>
> And this sieve script:
>
> --
> Message-ID header.
> require [
> "variables",
> "environment",
> "duplicate",
> "imapsieve"
> ];
>
> […]
>
> if duplicate :seconds 10 {
>discard;
> }
> --
>
> Otherwise, can I configure dovecot in a certain way, for instance
> create a virtual Sent folder. This would not save the email when
> appended, but still returns the list of sent emails ?
>
> But this is not working. Any idea?
>
> Thanks for your help!
>
> André
>
>
Avoid duplicate emails in the sent folder
Hello,
I have a webmail program, that I cannot configure to not save sent
emails in the Sent folder. My emails to the SOGo team don't seem to
reach the mailing list anymore.
So, I have no other choice to think the problem differently, and
_maybe_ use Dovecot to fix this issue. Moreover, I think this could be
useful for other email clients.
First, I have tried to use Sieve, but no success so far:
--
imapsieve_mailbox4_name = *
imapsieve_mailbox4_from = Sent
imapsieve_mailbox4_causes = APPEND, COPY
imapsieve_mailbox4_after = file:/var/lib/dovecot/sieve/sent-
checks.sieve
--
And this sieve script:
--
Message-ID header.
require [
"variables",
"environment",
"duplicate",
"imapsieve"
];
[…]
if duplicate :seconds 10 {
discard;
}
--
Otherwise, can I configure dovecot in a certain way, for instance
create a virtual Sent folder. This would not save the email when
appended, but still returns the list of sent emails ?
But this is not working. Any idea?
Thanks for your help!
André
Re: How to use xapian with non-text attachments
Hello, The version packaged on Bullseye is slightly out of date, I have filled a bug report: https://bugs.debian.org/985654 Thanks to the maintainers for their hard work! André On Sun, 2021-03-21 at 10:51 +, André Rodier wrote: > Hello, > > I am developing a hosting platform on Debian Bullseye, with Dovecot > amongst other tools. > > I am trying to use the xapian full test search plugin, but I can see > the attachments are skipped: > > This is what I have in the logs when running the indexing in verbose > mode: > > --- > > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access > doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free > = > 66 MB > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query > doveadm(camille): Info: FTS Xapian: Query= uid:"44" > doveadm(camille): Info: FTS Xapian: Ngram(S) -> 63 items (total 0 KB) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_unset_build_key > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Message- > Id,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_build_more > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_check_access > doveadm(camille): Info: FTS Xapian: Memory stats : Used = 56 MB, Free > = > 66 MB > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_index_hdr > doveadm(camille): Info: FTS Xapian: fts_backend_xapian_query > doveadm(camille): Info: FTS Xapian: Query= uid:"44" > doveadm(camille): Info: FTS Xapian: Ngram(XMID) -> 4 items (total 0 > KB) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_unset_build_key > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=X- > Mailer,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'xmailer' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=MIME- > Version,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'mimeversion' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Type,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttype' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Authentication- > Results,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'authenticationresults' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=X-AV- > Checked,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'xavchecked' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Type,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttype' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part > (Header=(null),Type=text/plain,Disposition=(null)) > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_build_more > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_unset_build_key > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Type,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contenttype' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Description,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) > 'contentdescription' > doveadm(camille): Info: FTS Xapian: > fts_backend_xapian_update_set_build_key > doveadm(camille): Info: FTS Xapian: New part (Header=Content- > Disposition,Type=(null),Disposition=(null)) > doveadm(camille): Info: FTS Xapian: Unknown header (indexing) >
How to use xapian with non-text attachments
(camille): Info: FTS Xapian: Unset box 'INBOX' (c0d4e304584e5460dae3075d7e67) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_oldbox doveadm(camille): Info: FTS Xapian: Done indexing 'INBOX' (c0d4e304584e5460dae3075d7e67) (13 msgs in 261 ms, rate: 49.8) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release (unset_box) doveadm(camille): Info: FTS Xapian: Committed 'unset_box' in 17 ms doveadm(camille): Info: FTS Xapian: Box is empty doveadm(camille): Info: FTS Xapian: fts_backend_xapian_update_deinit (/home/users/camille/mails/indexes/xapian-indexes) doveadm(camille): Info: FTS Xapian: fts_backend_xapian_release (update_deinit) doveadm(camille): Info: FTS Xapian: Committed 'update_deinit' in 0 ms doveadm(camille): Info: FTS Xapian: Deinit /home/users/camille/mails/indexes/xapian-indexes) --- Especially thos line: doveadm(camille): Info: FTS Xapian: Skipping part of type 'text/csv' and disposition 'attachment; filename="file.csv"' I have seen this as a bug on fts-xapian: https://github.com/grosjo/fts-xapian/issues/68 Which then references a PR on Dovecot core itself: https://github.com/dovecot/core/pull/155 Any idea on how to use xapian with non-text attachments ? Thanks a lot for your help! André
Re: User's mail encryption with mail-crypt-plugin
I understand it is a tough questions. Another approach, then. Would it be possible to receive a hash of the password, in Dovecot post-login script? This might be used to decrypt the encryption key in the user directory. Thanks, André On Sun, 2020-04-26 at 11:38 +0100, André Rodier wrote: > Hello, > > I have a question about the "mail-crypt-plugin". > > If each user is having its own mail encryption key, is there a way to > encrypt the key with the user password, and to decrypt it when the > user > logs in? > > I want to, if this is possible, to prevent any user, even root, to > access another user's emails. > > Thanks, > André >
User's mail encryption with mail-crypt-plugin
Hello, I have a question about the "mail-crypt-plugin". If each user is having its own mail encryption key, is there a way to encrypt the key with the user password, and to decrypt it when the user logs in? I want to, if this is possible, to prevent any user, even root, to access another user's emails. Thanks, André
Re: managesieve user
On Sat, 2020-03-28 at 15:14 +, André Rodier wrote:
> Hello,
>
> I am updating dovecot, from Debian Stretch to Buster, and I have an
> issue with the managesieve service:
>
> > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth:
> > connect(login) in directory / failed: Permission denied
> > (euid=120() egid=128() missing
> > +x
> > perm: /, we're not in group 127, dir owned by 0:127 mode=0770)
> > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth:
> > connect(login) in directory / failed: Permission denied
> > (euid=120() egid=128() missing +x perm: /, we're
> > not in group 127, dir owned by 0:127 mode=0770)
>
> The two folders I found with these permissions are /run/dovecot/login
> and /run/dovecot/token-login.
> Both folders belongs to root:dovecot
>
> If I change the permissions of the following folders, like this:
>
> chmod 0775 /run/dovecot/login
> chmod 0775 /run/dovecot/token-login
>
> Then, dovecot managesieve is working, but this is just a test, I
> suppose the service should run with a specific user?
>
> Do you have any idea on what is wrong here?
>
> If I restart dovecot, I have an error message that states the
> permissions are wrong, and they are set again to 0770
>
> Thanks,
> André Rodier
>
Hi again,
I can fix it by using this:
> service managesieve-login {
> inet_listener sieve {
> port = 4190
> }
>
> # Run as an unprivileged user
> group = dovecot
> user = dovecot
> }
>
Is it the right setting in term of security?
Thanks again,
André
managesieve user
Hello, I am updating dovecot, from Debian Stretch to Buster, and I have an issue with the managesieve service: > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth: > connect(login) in directory / failed: Permission denied > (euid=120() egid=128() missing +x > perm: /, we're not in group 127, dir owned by 0:127 mode=0770) > Mar 28 15:05:06 mail dovecot[2637]: managesieve-login: Error: auth: > connect(login) in directory / failed: Permission denied > (euid=120() egid=128() missing +x perm: /, we're > not in group 127, dir owned by 0:127 mode=0770) The two folders I found with these permissions are /run/dovecot/login and /run/dovecot/token-login. Both folders belongs to root:dovecot If I change the permissions of the following folders, like this: chmod 0775 /run/dovecot/login chmod 0775 /run/dovecot/token-login Then, dovecot managesieve is working, but this is just a test, I suppose the service should run with a specific user? Do you have any idea on what is wrong here? If I restart dovecot, I have an error message that states the permissions are wrong, and they are set again to 0770 Thanks, André Rodier
Get protocol used in the post login script
Hello, I have a post-login script, in Dovecot, that send warnings to users using XMPP. However, I would like hthe warning to include the protocol used, i.e. IMAP or POP3. How can I do this, please? Thanks, André.
Virtual mailboxes: errors after renaming a folder
Hello, I have a dovecot server, running fine, on Debian Stretch. Virtual folders are set up, and works correctly. For instance, I have Unread, All and Conversations, and everything works fine. However, when I rename a folder, I have error messages from Dovecot. For instance, I had a folder INBOX/Bills, which I renamed INBOX/Invoices. After that, I have these error messages: > The reported error was “Failed to select mailbox: Backend mailbox INBOX/Bills' added by another session. Reopen the virtual mailbox. (for backend mailbox Search/All) (0.000 + 0.000 secs).”. How can I avoid these errors, for instance, how can I rebuild the indexes every night or so? Kind regards, André
Second private namespace
Hello, I have Dovecot working, with a private namespace only. I would like to know if the following is feasible. - Add a second private namespace, in another directory scheme, for instance /mnt/backup/homebox//Mails/... - This namespace would be read-only. Perhaps the index and flags should be stored in a different folder with r/w access (e.g. /var, or even /tmp, it's not important) Now, as a matter of fact, this folder would be mounted on demand, via systemd-automount and fuse and is likely to be remote. This might implies some flags or caching on Dovecot performance. Thanks for your insights, André
Re: Outlook renames Junk folder
On Wed, 2019-06-12 at 15:01 -0500, Cliff Hayes via dovecot wrote: > Hello, > > Some versions of Outlook will randomly rename the Junk folder to Junk Email. > > This of course causes errors because Dovecot can no longer store junk. > > I go in to Outlook for each user and add another Junk folder, so now > they have Junk and Junk Email. This fix action lasts a while until > Outlook decides to do it again. > > Is there a way to tell Dovecot that if the Junk folder doesn't exist, > try sending the email to Junk Email folder? > > Thanks in advance. Hello Cliff, Outlook is not a software really known to respect other standards than Microsoft Exchange... Hopefully, Dovecot can overcome this behaviour, with the mail alias plugin: https://wiki2.dovecot.org/Plugins/MailboxAlias Just create an alias 'Junk Email' as 'Junk'. Have a nice day! -- André Rodier HomeBox: https://github.com/progmaticltd/homebox
nginx configuration to pass x-originating-ip
Hello, There is a bug in SOGo, as it sends the original IP after successful login, and not before the login process. I traced the bug to the source code. https://sogo.nu/bugs/view.php?id=2979. Then, in my research, I found this old thread: https://forum.nginx.org/read.php?29,237299,237367#msg-237367 Can I use Nginx as an IMAP proxy to add the missing ID. I suspect this is something that can be achieved? I already have an Nginx for the SOGo front-end, so I would add another one for the IMAP proxy. With a little bit of luck, I should be able to pass the original IP between the two servers Does anyone has a working example? Thanks, André -- André Rodier HomeBox: https://github.com/progmaticltd/homebox
Re: Post login scripts environment
On Sun, 2019-04-07 at 19:03 +0300, Aki Tuomi wrote:
> > On 7 April 2019 18:55 Aki Tuomi via dovecot wrote:
> >
> >
> > > On 7 April 2019 18:45 André Rodier via dovecot
> > > wrote:
> > >
> > >
> > > On Sun, 2019-04-07 at 17:49 +0300, Aki Tuomi via dovecot wrote:
> > > > > On 7 April 2019 17:26 André Rodier via dovecot < dovecot@dovecot.org>
> > > > > wrote:
> > > > >
> > > > >
> > > > > Dear Dovecot users,
> > > > >
> > > > > I am running Dovecot 2.2.27 on Debian Stretch, no issue so far.
> > > > >
> > > > > I wonder if there is a way to pass the remote IP address, in an
> > > > > environment variable, in the post login script.
> > > > >
> > > > > My Post login scripts are working well, except that when the server is
> > > > > accessed through a webmail (Roundcube or SOGo), the remote IP address
> > > > > is systematically 127.0.0.1.
> > > > >
> > > > > The other question I have is, is it possible to pass the user agent of
> > > > > the email client used to access the server? I know this can be easily
> > > > > forged, but I would like to log it.
> > > > >
> > > > > Thanks for your insight.
> > > > >
> > > > > --
> > > > > André Rodier
> > > >
> > > > You can use IMAP ID command to pass e.g. x-originating-ip. See
> > > > https://github.com/dovecot/core/blob/master/src/imap-login/imap-login-cmd-id.c
> > > > ---
> > > > Aki Tuomi
> > >
> > > Thanks, Aki,
> > >
> > > I had a look on the version, I don't think this is implemented in
> > > 2.2.27, it it seems this file has been added in 2.3.
> > >
> > > I may have to use a more recent version of Dovecot, but I think this is
> > > exactly what I was looking for, for the IP address.
> > >
> > > Regarding the original user agent (e.g. Evolution, Thunderbird, etc.),
> > > I suppose I can use the same approach?
> > >
> > > Thanks again for your help.
> > >
> > > André
> > >
> >
> > This feature is supported since 1.2 alpha.
> >
> > Aki
>
> See https://wiki2.dovecot.org/Design/ParameterForwarding for more details on
> this feature. I forgot to link this in the original reply.
>
> Aki
Dear Aki et al,
Thank you, this is working perfectly, at least with a simple RoundCube
plugin.
For those who need the same as me, here a minimal example plug-in with
RoundCube:
==<
?php
class dovecot_ident extends rcube_plugin
{
function init()
{
$this->add_hook('storage_connect', [$this, 'add_ident']);
}
function add_ident($args)
{
$remoteIP = $_SERVER['REMOTE_ADDR'];
$identInfo = [ 'x-originating-ip' => $remoteIP ];
if ($args['ident']) {
$args['ident'] = array_merge($args['ident'], $identInfo);
} else {
$args['ident'] = $identInfo;
}
return $args;
}
}
?>
==
I am struggling to obtain answers from SOGo, but eventually I will get
there.
Maybe there is a way with imapproxy and an nginx setting ?
Kind regards,
André
--
André Rodier
HomeBox: https://github.com/progmaticltd/homebox
Re: Post login scripts environment
On Sun, 2019-04-07 at 19:03 +0300, Aki Tuomi wrote: > > On 7 April 2019 18:55 Aki Tuomi via dovecot wrote: > > > > > > > On 7 April 2019 18:45 André Rodier via dovecot > > > wrote: > > > > > > > > > On Sun, 2019-04-07 at 17:49 +0300, Aki Tuomi via dovecot wrote: > > > > > On 7 April 2019 17:26 André Rodier via dovecot < dovecot@dovecot.org> > > > > > wrote: > > > > > > > > > > > > > > > Dear Dovecot users, > > > > > > > > > > I am running Dovecot 2.2.27 on Debian Stretch, no issue so far. > > > > > > > > > > I wonder if there is a way to pass the remote IP address, in an > > > > > environment variable, in the post login script. > > > > > > > > > > My Post login scripts are working well, except that when the server is > > > > > accessed through a webmail (Roundcube or SOGo), the remote IP address > > > > > is systematically 127.0.0.1. > > > > > > > > > > The other question I have is, is it possible to pass the user agent of > > > > > the email client used to access the server? I know this can be easily > > > > > forged, but I would like to log it. > > > > > > > > > > Thanks for your insight. > > > > > > > > > > -- > > > > > André Rodier > > > > > > > > You can use IMAP ID command to pass e.g. x-originating-ip. See > > > > https://github.com/dovecot/core/blob/master/src/imap-login/imap-login-cmd-id.c > > > > --- > > > > Aki Tuomi > > > > > > Thanks, Aki, > > > > > > I had a look on the version, I don't think this is implemented in > > > 2.2.27, it it seems this file has been added in 2.3. > > > > > > I may have to use a more recent version of Dovecot, but I think this is > > > exactly what I was looking for, for the IP address. > > > > > > Regarding the original user agent (e.g. Evolution, Thunderbird, etc.), > > > I suppose I can use the same approach? > > > > > > Thanks again for your help. > > > > > > André > > > > > > > This feature is supported since 1.2 alpha. > > > > Aki > > See https://wiki2.dovecot.org/Design/ParameterForwarding for more details on > this feature. I forgot to link this in the original reply. > > Aki Thanks a lot, this is great, now I can implement it properly. I may post a link on this list, once implemented. Kind regards, André -- André Rodier HomeBox: https://github.com/progmaticltd/homebox
Re: Post login scripts environment
On Sun, 2019-04-07 at 17:49 +0300, Aki Tuomi via dovecot wrote: > > On 7 April 2019 17:26 André Rodier via dovecot < dovecot@dovecot.org> wrote: > > > > > > Dear Dovecot users, > > > > I am running Dovecot 2.2.27 on Debian Stretch, no issue so far. > > > > I wonder if there is a way to pass the remote IP address, in an > > environment variable, in the post login script. > > > > My Post login scripts are working well, except that when the server is > > accessed through a webmail (Roundcube or SOGo), the remote IP address > > is systematically 127.0.0.1. > > > > The other question I have is, is it possible to pass the user agent of > > the email client used to access the server? I know this can be easily > > forged, but I would like to log it. > > > > Thanks for your insight. > > > > -- > > André Rodier > > You can use IMAP ID command to pass e.g. x-originating-ip. See > https://github.com/dovecot/core/blob/master/src/imap-login/imap-login-cmd-id.c > --- > Aki Tuomi Thanks, Aki, I had a look on the version, I don't think this is implemented in 2.2.27, it it seems this file has been added in 2.3. I may have to use a more recent version of Dovecot, but I think this is exactly what I was looking for, for the IP address. Regarding the original user agent (e.g. Evolution, Thunderbird, etc.), I suppose I can use the same approach? Thanks again for your help. André -- André Rodier
Post login scripts environment
Dear Dovecot users, I am running Dovecot 2.2.27 on Debian Stretch, no issue so far. I wonder if there is a way to pass the remote IP address, in an environment variable, in the post login script. My Post login scripts are working well, except that when the server is accessed through a webmail (Roundcube or SOGo), the remote IP address is systematically 127.0.0.1. The other question I have is, is it possible to pass the user agent of the email client used to access the server? I know this can be easily forged, but I would like to log it. Thanks for your insight. -- André Rodier
Re: SMTPUTF8 support
On Sat, 2019-04-06 at 11:19 +0200, John Fawcett via dovecot wrote: > On 06/04/2019 10:07, André Rodier via dovecot wrote: > > On 05/04/2019 08:57, David Bürgin via dovecot wrote: > > > André, are you quite sure you have it working? > > > > > > In this thread someone from Open-Xchange stated that no, Dovecot doesn’t > > > have SMTPUTF8 support implemented, and the same response was given by > > > another Dovecot developer last September (it ‘is being considered’ was > > > the answer then, see > > > https://dovecot.org/pipermail/dovecot/2018-September/112887.html). > > > > > > I am using LMTP to deliver mail to Dovecot from Postfix, and delivery > > > fails with Postfix reporting: ‘SMTPUTF8 is required, but was not offered > > > by host mail.my.org[private/dovecot-lmtp]’. I doubt that this can work > > > without a change in Dovecot LMTP. > > > > > > Anyway, I am also interested in SMTPUTF8 support, so +1 from me! > > > > > > > > > > Hello David et al, > > > > I have it working with my set up, and I confirm it is not easy. > > > > If I limit the stack to OpenLDAP, Postfix and Dovecot to their > > simplest configuration, it is relatively easy, even with slightly > > outdated software in Debian Stretch. > > I think you are confusing acceptance of UTF8 in the localpart of the > email address (which if I remember correctly worked in the 2.2 branch) > with SMTPUTF8 support. As mentioned in the thread above Dovecot does not > as yet support SMTPUTF8. Support for SMTPUTF8 would mean among other > things changing the source code of Dovecot so that it announces the > SMTPUTF8 key word to the upstream MTA. > > John Hello John, You are probably right. I am not well aware of the low level details and the terminology. I confess I am mostly interested in the proper reception of the emails with accentuated user names, which is already sometimes difficult to achieve. Is there any link where I can learn the communication rules between Dovecot and Postfix and the SMTPUTF8 standard ? Kind regards, André -- André Rodier HomeBox: https://github.com/progmaticltd/homebox
Re: SMTPUTF8 support
On 05/04/2019 08:57, David Bürgin via dovecot wrote: André, are you quite sure you have it working? In this thread someone from Open-Xchange stated that no, Dovecot doesn’t have SMTPUTF8 support implemented, and the same response was given by another Dovecot developer last September (it ‘is being considered’ was the answer then, see https://dovecot.org/pipermail/dovecot/2018-September/112887.html). I am using LMTP to deliver mail to Dovecot from Postfix, and delivery fails with Postfix reporting: ‘SMTPUTF8 is required, but was not offered by host mail.my.org[private/dovecot-lmtp]’. I doubt that this can work without a change in Dovecot LMTP. Anyway, I am also interested in SMTPUTF8 support, so +1 from me! Hello David et al, I have it working with my set up, and I confirm it is not easy. If I limit the stack to OpenLDAP, Postfix and Dovecot to their simplest configuration, it is relatively easy, even with slightly outdated software in Debian Stretch. The trick I am using is to include the internationalised email address aside the main one, and to use Postfix LDAP lookup. For instance, I am using this LDAP schema: https://tools.ietf.org/html/draft-stroeder-mailboxrelatedobject-07 So, my LDAP lookup table for Postfix looks like this: # Basic LDAP settings for postfix server_host = ldap://ldap.rodier.me/ start_tls = yes search_base = dc=rodier,dc=me search_scope = sub # Allow advanced LDAP search with the recipient delimiter query_filter = (|(mail=%s)(intlMailAddr=%s)) result_attribute = mail # Force support for UTF8 version = 3 Therefore, postmap query works as expected: postmap -q andré@rodier.me ldap:/etc/postfix/ldap-aliases.cf an...@rodier.me That said, if only one of the milter reject your message, it is breaking the whole chain. For instance, the quota plugin for Dovecot, I had to use DUNNO when the user is unknown instead of reject. I have it working perfectly from Homebox to Homebox, so perhaps Postfix to Postfix, but I reckon I still have some bugs. Some providers are working, others not. Outlook does not seems to work , but I am on it. Yahoo does not let you type an accent in the email address, only Google allows you to. However, I made a test yesterday, and realised that the emails from GMail are rejected by one of the plugin, and I need to investigate which one. The error message is: Apr 5 10:16:39 portal postfix/smtpd[2566]: Anonymous TLS connection established from mail-lf1-f51.google.com[209.85.167.51]: TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits) Apr 5 10:16:39 portal policyd-spf[2574]: prepend Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=209.85.167.51; helo=mail-lf1-f51.google.com; envelope-from=andre.rod...@gmail.com; receiver= Apr 5 10:16:39 portal postfix/smtpd[2566]: EB38240467: client=mail-lf1-f51.google.com[209.85.167.51] Apr 5 10:16:40 portal postfix/cleanup[2576]: EB38240467: message-id= Apr 5 10:16:40 portal opendkim[15477]: EB38240467: mail-lf1-f51.google.com [209.85.167.51] not internal Apr 5 10:16:40 portal opendkim[15477]: EB38240467: not authenticated Apr 5 10:16:40 portal opendkim[15477]: EB38240467: DKIM verification successful Apr 5 10:16:40 portal opendkim[15477]: EB38240467: s=20161025 d=gmail.com SSL Apr 5 10:16:40 portal opendmarc[16548]: implicit authentication service: portal.homebox.space Apr 5 10:16:40 portal opendmarc[16548]: EB38240467: gmail.com pass Apr 5 10:16:40 portal postfix/cleanup[2576]: EB38240467: milter-reject: END-OF-MESSAGE from mail-lf1-f51.google.com[209.85.167.51]: 4.7.1 Try again later; from= to= proto=ESMTP helo= Apr 5 10:16:40 portal postfix/smtpd[2566]: disconnect from mail-lf1-f51.google.com[209.85.167.51] ehlo=2 starttls=1 mail=1 rcpt=1 data=0/1 quit=1 commands=6/7 So perhaps it is one of the milter. I will investigate today. Also, it seems the client is important. I have no problem with SOGo or Evolution. Roundcube and Thunderbird, in Debian Stretch, don't support this yet. With Debian Buster coming soon, I am sure this will work better. I will create a branch to test this. My preliminary investigations don't show major problems in the upgrade. At one point, I was using the Dovecot packages from backports, but I reverted this as there has been an incompatibility. Happy to answer more questions if you have. -- André Rodier
Re: SMTPUTF8 support
On 2019-04-05 08:57, David Bürgin via dovecot wrote: > André, are you quite sure you have it working? > > In this thread someone from Open-Xchange stated that no, Dovecot > doesn’t > have SMTPUTF8 support implemented, and the same response was given by > another Dovecot developer last September (it ‘is being considered’ was > the answer then, see > https://dovecot.org/pipermail/dovecot/2018-September/112887.html). > > I am using LMTP to deliver mail to Dovecot from Postfix, and delivery > fails with Postfix reporting: ‘SMTPUTF8 is required, but was not > offered > by host mail.my.org[private/dovecot-lmtp]’. I doubt that this can work > without a change in Dovecot LMTP. > > Anyway, I am also interested in SMTPUTF8 support, so +1 from me! Hello David, Well, it seems to work from one Homebox to another, for instance: Here the server logs: -- > Apr 05 19:20:46 portal postfix/qmgr[21470]: 6AC3240807: > from=, size=3066, nrcpt=1 (queue active) > Apr 05 19:20:46 portal clamsmtpd[14542]: 10004D: from=an...@rodier.me, > to=andré@homebox.space, status=CLEAN > Apr 05 19:20:46 portal postfix/smtp[30761]: 1FCA540467: > to=, relay=127.0.0.1[127.0.0.1]:34001, delay=1.4, > delays=0.73/0.02/0.07/0.59, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued > as 6AC3240807) > Apr 05 19:20:46 portal postfix/smtpd[30763]: disconnect from > localhost[127.0.0.1] ehlo=1 xforward=2 mail=1 rcpt=1 data=1 quit=1 > commands=7 > Apr 05 19:20:46 portal postfix/qmgr[21470]: 1FCA540467: removed > Apr 05 19:20:46 portal dovecot[12813]: lmtp(30766): Connect from local > Apr 05 19:20:46 portal dovecot[12813]: lmtp(andre): > WGTdAo6qp1wueAAAeNB+QA: sieve: > msgid=<7d53-5ca7aa80-1-51005a80@31476855>: stored mail into mailbox > 'INBOX' > Apr 05 19:20:46 portal postfix/lmtp[30765]: 6AC3240807: > to=, orig_to=, > relay=portal.homebox.space[private/dovecot-lmtp], delay=0.62, > delays=0.59/0.01/0.01/0.01, dsn=2.0.0, status=sent (250 2.0.0 > WGTdAo6qp1wueAAAeNB+QA Saved) > Apr 05 19:20:46 portal dovecot[12813]: lmtp(30766): Disconnect from > local: Successful quit > Apr 05 19:20:46 portal postfix/qmgr[21470]: 6AC3240807: removed -- And the message source: -- > Return-Path: > Delivered-To: andre@homebox.space > Received: from portal.homebox.space > by portal (Dovecot) with LMTP id WGTdAo6qp1wueAAAeNB+QA > for ; Fri, 05 Apr 2019 19:20:46 + > Received: from portal.homebox.space (localhost [127.0.0.1]) > by portal.homebox.space (Postfix) with ESMTP id 6AC3240807 > for ; Fri, 5 Apr 2019 19:20:45 + (UTC) > Authentication-Results: portal.homebox.space; > dkim=pass (4096-bit key; unprotected) header.d=rodier.me > header.i=@rodier.me header.b="bNaIhJcQ"; > dkim-atps=neutral > Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=92.19.253.42; > helo=osaka.rodier.me; envelope-from=an...@rodier.me; receiver= > Received: from osaka.rodier.me (host-92-19-253-42.static.as13285.net > [92.19.253.42]) > by portal.homebox.space (Postfix) with ESMTPS id 1FCA540467 > for ; Fri, 5 Apr 2019 19:20:44 + (UTC) > Received: from osaka.rodier.me (localhost [127.0.0.1]) > by osaka.rodier.me (Postfix) with ESMTP id 68658201DE > for ; Fri, 5 Apr 2019 20:20:43 +0100 (BST) > Received: from localhost (localhost [127.0.0.1]) > by osaka.rodier.me (Postfix) with ESMTPA id 32C1A200CC > for ; Fri, 5 Apr 2019 20:20:43 +0100 (BST) > DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=rodier.me; > s=main; > t=1554492043; bh=ojnzW/eF3u3/ihaSAhQfvUWEX7Blx4fCTTVCC9mfgZE=; > h=From:Date:To:Subject:From; > b=bNaIhJcQtD7/IbjI/VGUIWpOk1QyYioKsgsCva8sL040L7VDMmLgfbQ5TWFwO/wub >+8THBgOTWj/39FyoKQJYk0BahQ7Z4LkUrH20Jcq5yDEOC7q/xfwIwUV/sUaj6C3wLZ >VGUp8Fg2oVDD+ciGEnfa2OuEC/LajvmXoDqUHHtSwTFZvfkh1nRDqxcNd/FSyvYQe8 >frgBySFgzI+xTWVG7Oa07CQPG+HenqwrPyaZD0SNpHnrvH5ICdAriv2UMKgQnsWO80 >aSz4EDySdwMah17kfh4u06GlXd3duQCN8hyxBBDjcI879N/+CQM3Uz1NTbIDe82Lf7 >bSUJzvMczCdZFm5J84hvLd+j85pY7T9052JTcUANUQPnTc1Q3yrQFC3QuKD8Amb8VO >qHR0q0OP6SYqoQ0g/moMF2FhWaQg3pCRpIEOS73Ncnnf+fJGQJnPi8G8YcLVLZCDCx >RlSauA/YuF5tiIBHFLdjtlgVgwk38tT8ck+D8kS3BAB4nXBVlreCiK0vMK8HyoCpsC >LMJ0e5KhBzCDXOPvSLiohn2zcmYpa9DywmWFVTZNx3INzTH7nsa40AYGFl+k+G+pe+ >ej+MRDwmdOePfl3DGGEgpn0JMNiAmECe6gHA+/N1TTk5SbbiKLNh8KXzrZECWVrvUx >55kqY/LPuQEwxCnA/VZsQbDk= > Content-Type: multipart/alternative; > boundary="=_=-_OpenGroupware_org_NGMime-32083-1554492042.981625-0--" > From: =?utf-8?q?Andr=C3=A9_Rodier?= > X-Forward: 127.0.0.1 > Date: Fr
Re: SMTPUTF8 support
On 2019-04-05 02:44, sylvhem--- via dovecot wrote: Le 2019-04-04 03:28, André a écrit : I have it working, only with the user part of the email address. It works well, as long as the whole chain of software is correctly configured. I have not tested with a whole domain in UTF8 yet. More details and working examples here: https://github.com/progmaticltd/homebox Good luck. Homebox looks quite interesting, but I'm not sure I want to redo my entire e-mail stack for this. May I ask what did you do make Dovecot support UTF-8 characters in the user part of the e-mail address? Did you patch it? -- Sylvhem Sorry, I was sending the email quickly, and I did not point you in the exact direction. - Dovecot templates: https://github.com/progmaticltd/homebox/tree/master/install/playbooks/roles/dovecot/templates/conf.d - Postfix templates: https://github.com/progmaticltd/homebox/tree/master/install/playbooks/roles/postfix/templates For Postfix, you will be probably interested in main.cf and master.cf Give you direct access to the templates is the best I can do, there are too many parameters, and I don't remember all of them by heart. For instance, it can depends on your milters as well. Good luck! -- https://github.com/progmaticltd/homebox
Re: SMTPUTF8 support
I have it working, only with the user part of the email address. It works well, as long as the whole chain of software is correctly configured. I have not tested with a whole domain in UTF8 yet. More details and working examples here: https://github.com/progmaticltd/homebox Good luck. On 4 April 2019 07:03:10 BST, sylvhem--- via dovecot wrote: >Le 2019-04-04 01:36, Michael Slusarz a écrit : >>> On April 3, 2019 at 10:12 PM sylvhem--- via dovecot >>> wrote: >>> >>> I'm currently trying to set up SMTPUTF8 on my mail stack, but I >can't >>> find any information on Dovecot's RFC 6531 support. Has it been >>> implemented yet? >> >> No. >> >> michael >Thank you for the answer Michael. >Does anybody know if it's been worked on? > >-- >Sylvhem -- André
TFA authentication in dovecot, using XMPP and RFC 4226
Hello, I would like to implement some kind of two factors authentication, in Dovecot. I am thinking about using the post login script, to check for unusual behaviour, like say, a different country / IP address or an unusual hour. I already wrote a simple shell script that check these factors, but now, I have some options for the following, and I need to know your opinion if this is feasible or not. I want to use google authenticator Debian package (support the HMAC- Based One-time Password (HOTP) algorithm specified in RFC 4226 and the Time-based One-time Password (TOTP)) The challenge would be send via XMPP. This second part is fairly easy to do, I have all the packages on Debian, for instance sendxmpp. The first tests are promising. In case of success, the IP address is added to the list, let's say for one month... My back-end for authentication is OpenLDAP. My questions are: - Do you see any performance issues for other users or login processes, if I implement this? - I am planning to use a timeout, for instance one minute to confirm the connection. Does Dovecot have a timeout on its side, that would abort the connection before? Otherwise: - Is it possible to have multiple authentication back-ends in Dovecot? For instance LDAP and/or OTP? - I think to have seen some TFA options in Dovecot, but AFAICS, they are mandatory. Thanks for your insights, and this fabulous software. -- André Rodier HomeBox: https://github.com/progmaticltd/homebox
Re: panic when using dovecot master account
On 2019-01-20 12:38, Stephan Bosch wrote: Op 12/01/2019 om 20:31 schreef André Rodier via dovecot: On Sat, 2019-01-12 at 19:11 +, André Rodier via dovecot wrote: Happy new year, everyone! Aki, I have been able to reproduce the problem, and this time, with the packages from Debian stable. So you were right, this had nothing to do with master user, but virtual folders. I found what caused the crash: the line "auto = subscribe" for the virtual folders. Dovecot does not crash when I comment this line. Tracking internally as DOP-866. Regards, Stephan. Thanks, I am happy to have been useful. Kind regards, André
Re: panic when using dovecot master account
On Sat, 2019-01-12 at 19:11 +, André Rodier via dovecot wrote: > On 2018-11-09 07:40, André Rodier wrote: > > On 2018-11-09 05:25, Aki Tuomi wrote: > > > This seems to have nothing to do with master account or not. Does this > > > happen if you try to open the virtual mailbox again? > > > > > > Aki > > > > > > > On 09 November 2018 at 00:13 André Rodier wrote: > > > > > > > > > > > > Hello, > > > > > > > > I am running dovecot 2.2.34 (874deae), on Debian stable, from > > > > backports. > > > > > > > > I just tried the master account, and although everything worked in > > > > the > > > > email client, I had logs in the error logs: > > > > > > > > > > > > > imap(mirina): Panic: file mail-index-sync.c: line 413 > > > > > (mail_index_sync_begin_to2): assertion failed: (!index->syncing) > > > > > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Error: Raw > > > > > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9e412) [0x7f20ae813412] > > > > > -> /usr/lib/dovecot/libdovecot.so.0(+0x9e50d) [0x7f20ae81350d] -> > > > > > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f20ae7a2c51] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(+0xe0fe4) [0x7f20aeb88fe4] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x4f) > > > > > [0x7f20aeb890bf] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1c) > > > > > [0x7f20aeb8915c] -> > > > > > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x218) > > > > > [0x7f20ad51f308] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > > > > > [0x7f20aeaf02d4] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > > > > > [0x7f20aeaf0387] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > > > > > [0x7f20aeb6bbf1] -> > > > > > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > > > > > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > > > > > [0x7f20aeb50771] -> > > > > > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > > > > > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > > > > > [0x7f20aeaf0781] -> > > > > > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > > > > > [0x7f20ad5201a8] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > > > > > [0x7f20aeaf02d4] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > > > > > [0x7f20aeaf0387] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > > > > > [0x7f20aeb6bbf1] -> > > > > > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > > > > > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > > > > > [0x7f20aeb50771] -> > > > > > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > > > > > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > > > > > [0x7f20aeaf0781] -> > > > > > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > > > > > [0x7f20ad5201a8] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > > > > > [0x7f20aeaf02d4] -> > > > > > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > > > > > [0x7f20aeaf0387] -> dovecot/imap(cmd_select_full+0x16b) > > > > > [0x557de970cc3b] -> dovecot/imap(command_exec+0x5c) [0x557de971444c] > > > > > -> > > > > > dovecot/imap(+0x1a912) [0x557de9712912] > > > > > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Fatal: master: > > > > > service(imap): child 4289 killed with signal 6 (core dumps disabled) > > > > > > > > I attach my dovecot configuration too. > > > > > > > > Thanks for your help. > > > > > > > > Hello Aki, > > > > You are right, the same error happens, even
Re: panic when using dovecot master account
On 2018-11-09 07:40, André Rodier wrote: On 2018-11-09 05:25, Aki Tuomi wrote: This seems to have nothing to do with master account or not. Does this happen if you try to open the virtual mailbox again? Aki On 09 November 2018 at 00:13 André Rodier wrote: Hello, I am running dovecot 2.2.34 (874deae), on Debian stable, from backports. I just tried the master account, and although everything worked in the email client, I had logs in the error logs: > imap(mirina): Panic: file mail-index-sync.c: line 413 > (mail_index_sync_begin_to2): assertion failed: (!index->syncing) > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9e412) [0x7f20ae813412] > -> /usr/lib/dovecot/libdovecot.so.0(+0x9e50d) [0x7f20ae81350d] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f20ae7a2c51] -> > /usr/lib/dovecot/libdovecot-storage.so.0(+0xe0fe4) [0x7f20aeb88fe4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x4f) > [0x7f20aeb890bf] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1c) > [0x7f20aeb8915c] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x218) > [0x7f20ad51f308] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > [0x7f20aeb6bbf1] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > [0x7f20aeb50771] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > [0x7f20aeaf0781] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > [0x7f20ad5201a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > [0x7f20aeb6bbf1] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > [0x7f20aeb50771] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > [0x7f20aeaf0781] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > [0x7f20ad5201a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> dovecot/imap(cmd_select_full+0x16b) > [0x557de970cc3b] -> dovecot/imap(command_exec+0x5c) [0x557de971444c] -> > dovecot/imap(+0x1a912) [0x557de9712912] > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Fatal: master: > service(imap): child 4289 killed with signal 6 (core dumps disabled) I attach my dovecot configuration too. Thanks for your help. Hello Aki, You are right, the same error happens, even if I do not use the master password. The weird thing is that it happens only with one specific account. For this account, the virtual folders I have don't work, but they work for the other accounts. I do not have too much time to investigate now, but I will continue this weekend. Kind regards, André Happy new year, everyone! Aki, I have been able to reproduce the problem, and this time, with the packages from Debian stable. So you were right, this had nothing to do with master user, but virtual folders. I can now send the full stack trace, and doveconf. Package versions: ii dovecot-core 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - core files ii dovecot-dbg1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - debug symbols ii dovecot-imapd 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - ManageSieve server ii dovecot-pop3d 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - Sieve filters support ii dovecot-solr 1:2.2.27-3+deb9u2 amd64secure POP3/IMAP server - Solr support Kind
Re: Indexer worker small bug
On 2018-12-10 14:12, André via dovecot wrote: Hello Timo, I will try this evening. Kind regards, André. On 10 December 2018 07:56:26 GMT+00:00, Timo Sirainen wrote: On 10 Dec 2018, at 7.50, André Rodier wrote: On 2018-12-09 23:13, Timo Sirainen wrote: On 9 Dec 2018, at 16.44, André Rodier via dovecot wrote: Hello, I think I submitted this before, but I am not sure this has been addressed I am using AppArmor with Dovecot, without any issue. However, I think there is a bug in the indexer working, from what I can see, a missing trailing slash. See: Dec 09 14:35:53 portal2 kernel: audit: type=1400 audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap" info="Failed name lookup - disconnected path" error=-13 profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts" pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r" fsuid=1001 ouid=0 The indexer worker is trying to open the file "var/cache/nscd/hosts" instead of "/var/cache/nscd/hosts", which of course fails. Can someone double check the code of the indexer worker, or this has been fixed? Dovecot is definitely not trying to open that file itself. It has to be libc or some other library. I also can't think of anything special in indexer-worker compared to other Dovecot binaries that could cause this. What's your doveconf -n? You are probably right, I will continue to investigate on my side. My configuration is attached. Does it make a difference if you use "127.0.0.1" instead of "localhost" in fts_solr setting? Sorry for the top-posting. Same error, with 127.0.0.1. However, if I disable the nscd cache in nscd.conf, the error does not happen any more. I have downloaded the source code (glibc, if I understand correctly), but I have not yet found the cause. Kind regards, André
Re: Indexer worker small bug
Hello Timo, I will try this evening. Kind regards, André. On 10 December 2018 07:56:26 GMT+00:00, Timo Sirainen wrote: >On 10 Dec 2018, at 7.50, André Rodier wrote: >> >> On 2018-12-09 23:13, Timo Sirainen wrote: >>> On 9 Dec 2018, at 16.44, André Rodier via dovecot > wrote: >>>> Hello, >>>> I think I submitted this before, but I am not sure this has been >addressed >>>> I am using AppArmor with Dovecot, without any issue. >>>> However, I think there is a bug in the indexer working, from what I >can see, a missing trailing slash. See: >>>> >>>> Dec 09 14:35:53 portal2 kernel: audit: type=1400 >audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap" >info="Failed name lookup - disconnected path" error=-13 >profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts" >pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r" >fsuid=1001 ouid=0 >>>> >>>> The indexer worker is trying to open the file >"var/cache/nscd/hosts" instead of "/var/cache/nscd/hosts", which of >course fails. >>>> Can someone double check the code of the indexer worker, or this >has been fixed? >>> Dovecot is definitely not trying to open that file itself. It has to >>> be libc or some other library. I also can't think of anything >special >>> in indexer-worker compared to other Dovecot binaries that could >cause >>> this. What's your doveconf -n? >> >> You are probably right, I will continue to investigate on my side. >> My configuration is attached. > >Does it make a difference if you use "127.0.0.1" instead of "localhost" >in fts_solr setting? -- André
Re: Indexer worker small bug
On 2018-12-09 23:13, Timo Sirainen wrote:
On 9 Dec 2018, at 16.44, André Rodier via dovecot
wrote:
Hello,
I think I submitted this before, but I am not sure this has been
addressed
I am using AppArmor with Dovecot, without any issue.
However, I think there is a bug in the indexer working, from what I
can see, a missing trailing slash. See:
Dec 09 14:35:53 portal2 kernel: audit: type=1400
audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap"
info="Failed name lookup - disconnected path" error=-13
profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts"
pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r"
fsuid=1001 ouid=0
The indexer worker is trying to open the file "var/cache/nscd/hosts"
instead of "/var/cache/nscd/hosts", which of course fails.
Can someone double check the code of the indexer worker, or this has
been fixed?
Dovecot is definitely not trying to open that file itself. It has to
be libc or some other library. I also can't think of anything special
in indexer-worker compared to other Dovecot binaries that could cause
this. What's your doveconf -n?
You are probably right, I will continue to investigate on my side.
My configuration is attached.
--
https://github.com/progmaticltd/homebox# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.6
auth_debug = yes
auth_verbose = yes
auth_verbose_passwords = yes
lmtp_save_to_detail_mailbox = yes
mail_debug = yes
mail_plugins = " quota fts fts_solr"
mailbox_list_index = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext imapsieve
namespace inbox {
inbox = yes
location = maildir:~/mails/maildir:INDEX=~/mails/indexes/
mailbox Archives {
auto = subscribe
special_use = \Archive
}
mailbox Drafts {
auto = subscribe
special_use = \Drafts
}
mailbox Junk {
auto = subscribe
special_use = \Junk
}
mailbox Sent {
auto = subscribe
special_use = \Sent
}
mailbox Trash {
auto = subscribe
special_use = \Trash
}
prefix =
separator = /
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
fts = solr
fts_decoder = decode2text
fts_index_timeout = 60
fts_solr = url=http://localhost:8080/solr/
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_from = *
imapsieve_mailbox1_name = Junk
imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_name = *
imapsieve_mailbox3_after = file:/etc/dovecot/sieve/duplicate-checks.sieve
imapsieve_mailbox3_causes = APPEND
imapsieve_mailbox3_from = *
imapsieve_mailbox3_name = INBOX
imapsieve_mailbox4_after = file:/etc/dovecot/sieve/sent-checks.sieve
imapsieve_mailbox4_causes = APPEND
imapsieve_mailbox4_from = Sent
imapsieve_mailbox4_name = *
imapsieve_mailbox5_after = file:/etc/dovecot/sieve/tags-important.sieve
imapsieve_mailbox5_causes = APPEND, COPY
imapsieve_mailbox5_from = *
imapsieve_mailbox5_name = Important
quota = maildir:user-quota
quota_rule = *:storage=500M
recipient_delimiter = ~
sieve = file:~/mails/sieve;active=~/mails/sieve/active.sieve
sieve_before = file:/etc/dovecot/sieve/before-global.sieve
sieve_default_name = default
sieve_global = /etc/dovecot/sieve
sieve_global_extensions = +vnd.dovecot.pipe
sieve_pipe_bin_dir = /etc/dovecot/sieve
sieve_plugins = sieve_imapsieve sieve_extprograms
sieve_trace_addresses = yes
sieve_trace_debug = yes
sieve_trace_dir = ~/mails/sieve/logs/
sieve_trace_level = commands
sieve_user_log = ~/mails/sieve/logs/main.log
}
protocols = " imap lmtp sieve pop3"
recipient_delimiter = ~
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0666
}
}
service decode2text {
executable = script /usr/local/bin/decode2text
unix_listener decode2text {
mode = 0666
}
user = $default_internal_user
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service imap {
executable = imap
}
service lmtp {
unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
service managesieve {
process_limit = 1024
}
service pop3-login {
inet_listener pop3 {
port = 110
}
inet_listener po
Indexer worker small bug
Hello, I think I submitted this before, but I am not sure this has been addressed I am using AppArmor with Dovecot, without any issue. However, I think there is a bug in the indexer working, from what I can see, a missing trailing slash. See: Dec 09 14:35:53 portal2 kernel: audit: type=1400 audit(1544366153.379:3035): apparmor="DENIED" operation="file_mmap" info="Failed name lookup - disconnected path" error=-13 profile="/usr/lib/dovecot/indexer-worker" name="var/cache/nscd/hosts" pid=10540 comm="indexer-worker" requested_mask="r" denied_mask="r" fsuid=1001 ouid=0 The indexer worker is trying to open the file "var/cache/nscd/hosts" instead of "/var/cache/nscd/hosts", which of course fails. Can someone double check the code of the indexer worker, or this has been fixed? Thanks, André -- https://github.com/progmaticltd/homebox
Re: panic when using dovecot master account
On 2018-11-09 05:25, Aki Tuomi wrote: This seems to have nothing to do with master account or not. Does this happen if you try to open the virtual mailbox again? Aki On 09 November 2018 at 00:13 André Rodier wrote: Hello, I am running dovecot 2.2.34 (874deae), on Debian stable, from backports. I just tried the master account, and although everything worked in the email client, I had logs in the error logs: > imap(mirina): Panic: file mail-index-sync.c: line 413 > (mail_index_sync_begin_to2): assertion failed: (!index->syncing) > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Error: Raw > backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9e412) [0x7f20ae813412] > -> /usr/lib/dovecot/libdovecot.so.0(+0x9e50d) [0x7f20ae81350d] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f20ae7a2c51] -> > /usr/lib/dovecot/libdovecot-storage.so.0(+0xe0fe4) [0x7f20aeb88fe4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x4f) > [0x7f20aeb890bf] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1c) > [0x7f20aeb8915c] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x218) > [0x7f20ad51f308] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > [0x7f20aeb6bbf1] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > [0x7f20aeb50771] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > [0x7f20aeaf0781] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > [0x7f20ad5201a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> > /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31) > [0x7f20aeb6bbf1] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d) > [0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771) > [0x7f20aeb50771] -> > /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46] > -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31) > [0x7f20aeaf0781] -> > /usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8) > [0x7f20ad5201a8] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44) > [0x7f20aeaf02d4] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f20aeaf0387] -> dovecot/imap(cmd_select_full+0x16b) > [0x557de970cc3b] -> dovecot/imap(command_exec+0x5c) [0x557de971444c] -> > dovecot/imap(+0x1a912) [0x557de9712912] > Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Fatal: master: > service(imap): child 4289 killed with signal 6 (core dumps disabled) I attach my dovecot configuration too. Thanks for your help. Hello Aki, You are right, the same error happens, even if I do not use the master password. The weird thing is that it happens only with one specific account. For this account, the virtual folders I have don't work, but they work for the other accounts. I do not have too much time to investigate now, but I will continue this weekend. Kind regards, André -- https://github.com/progmaticltd/homebox
panic when using dovecot master account
Hello,
I am running dovecot 2.2.34 (874deae), on Debian stable, from backports.
I just tried the master account, and although everything worked in the
email client, I had logs in the error logs:
imap(mirina): Panic: file mail-index-sync.c: line 413
(mail_index_sync_begin_to2): assertion failed: (!index->syncing)
Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Error: Raw
backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x9e412) [0x7f20ae813412]
-> /usr/lib/dovecot/libdovecot.so.0(+0x9e50d) [0x7f20ae81350d] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f20ae7a2c51] ->
/usr/lib/dovecot/libdovecot-storage.so.0(+0xe0fe4) [0x7f20aeb88fe4] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin_to+0x4f)
[0x7f20aeb890bf] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_begin+0x1c)
[0x7f20aeb8915c] ->
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x218)
[0x7f20ad51f308] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44)
[0x7f20aeaf02d4] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
[0x7f20aeaf0387] ->
/usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31)
[0x7f20aeb6bbf1] ->
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d)
[0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771)
[0x7f20aeb50771] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46]
-> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31)
[0x7f20aeaf0781] ->
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8)
[0x7f20ad5201a8] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44)
[0x7f20aeaf02d4] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
[0x7f20aeaf0387] ->
/usr/lib/dovecot/libdovecot-storage.so.0(index_storage_get_status+0x31)
[0x7f20aeb6bbf1] ->
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(+0x936d)
[0x7f20ad51c36d] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa8771)
[0x7f20aeb50771] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xfd46) [0x7f20adb8bd46]
-> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_status+0x31)
[0x7f20aeaf0781] ->
/usr/lib/dovecot/modules/lib20_virtual_plugin.so(virtual_storage_sync_init+0x10b8)
[0x7f20ad5201a8] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x44)
[0x7f20aeaf02d4] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37)
[0x7f20aeaf0387] -> dovecot/imap(cmd_select_full+0x16b)
[0x557de970cc3b] -> dovecot/imap(command_exec+0x5c) [0x557de971444c] ->
dovecot/imap(+0x1a912) [0x557de9712912]
Nov 08 22:06:24 osaka dovecot[1450]: imap(mirina): Fatal: master:
service(imap): child 4289 killed with signal 6 (core dumps disabled)
I attach my dovecot configuration too.
Thanks for your help.
# 2.2.34 (874deae): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.22 (22940fb7)
# OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.5
# Hostname: osaka.office.pmc
# NOTE: Send doveconf -n output instead when asking for help.
auth_anonymous_username = anonymous
auth_cache_negative_ttl = 1 hours
auth_cache_size = 0
auth_cache_ttl = 1 hours
auth_cache_verify_password_with_worker = no
auth_debug = no
auth_debug_passwords = no
auth_default_realm =
auth_failure_delay = 2 secs
auth_gssapi_hostname =
auth_krb5_keytab =
auth_master_user_separator = /
auth_mechanisms = plain
auth_policy_check_after_auth = yes
auth_policy_check_before_auth = yes
auth_policy_hash_mech = sha256
auth_policy_hash_nonce =
auth_policy_hash_truncate = 12
auth_policy_reject_on_fail = no
auth_policy_report_after_auth = yes
auth_policy_request_attributes = login=%{requested_username}
pwhash=%{hashed_password} remote=%{rip} device_id=%{client_id} protocol=%s
auth_policy_server_api_header =
auth_policy_server_timeout_msecs = 2000
auth_policy_server_url =
auth_proxy_self =
auth_realms =
auth_socket_path = auth-userdb
auth_ssl_require_client_cert = no
auth_ssl_username_from_cert = no
auth_stats = no
auth_use_winbind = no
auth_username_chars =
auth_username_format = %Lu
auth_username_translation =
auth_verbose = no
auth_verbose_passwords = no
auth_winbind_helper_path = /usr/bin/ntlm_auth
auth_worker_max_count = 30
base_dir = /var/run/dovecot
config_cache_size = 1 M
debug_log_path =
default_client_limit = 1000
default_idle_kill = 1 mins
default_internal_user = dovecot
default_login_user = dovenull
default_process_limit = 100
default_vsz_limit = 256 M
deliver_log_format = msgid=%m: %$
dict_db_config =
director_consistent_hashing = no
director_doveadm_port = 0
director_flush_socket =
director_mail_servers =
director_max_parallel_kicks = 100
director_max_parallel_moves = 100
director_output_buffer_size = 10 M
director_ping_idle_timeout = 30 secs
director_ping_max_timeout = 1 mins
director_servers =
director_user_expire = 15 mins
director_user_kick_delay = 2 secs
director_username_hash = %u
disable_plaintext_auth = yes
dotlock_use_excl = yes
doveadm_allowed_commands =
doveadm_a
Re: Disable logging for localhost
On 2018-10-22 12:16, Shawn Heisey wrote: On 10/21/2018 8:23 AM, André Rodier wrote: I am using Zabbix to monitor my Dovecot server, and my logs are filled with lines like this: Oct 21 15:04:46 osaka dovecot[1256]: pop3-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, secured, session= Oct 21 15:05:29 osaka dovecot[1256]: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, secured, session= How can I disable logging these actions from 127.0.0.1? Presumably you are concerned not so much with having these lines in the logfile, as you are with your monitoring software seeing them and generating alarms because they are reported to syslog as errors. If I'm wrong about that, then the rest of this message probably won't apply. Monitoring software that reports errors in logfiles should have the ability to specify messages to ignore. Often this is done with regex. Since you mentioned Zabbix, I'm betting it's that software that generates the reports you would like to stop. It looks like it does have the ability to use a regex to filter logs. https://www.zabbix.com/documentation/4.0/manual/config/items/itemtypes/log_items This regex ought to do the trick and not filter out other error messages (like login failures) that occur on localhost. Hopefully it gets preserved on one line, not mangled by my mail client: dovecot.*(imap|pop3)-login: Aborted .*rip=127\.0\.0\.\d+, lip=127\.0\.0\.\d+, The "\d+" notation generally only works if the software is using PCRE - Perl Compatible Regular Expressions. It does look like Zabbix is using PCRE, so if it's Zabbix we're talking about, that regex should be OK. Possible replacements for \d+ if it doesn't work: [0-9]+ [0-9]* Thanks, Shawn These lines are in the mail.log. I am not talking about Zabbix. Zabbix generate a connection every second, on both Postfix and Dovecot. I want these two services to not log connections when they are coming from 127.0.0.1, if possible. Both journalctl (Systemd) and log files are spammed with these lines, and I would like to avoid dovecot logging them. I should have made this clearer. Kind regards, André -- https://github.com/progmaticltd/homebox
Re: Calendar function ?
On 2018-10-21 20:16, Patrick Ben Koetter wrote: * André Rodier : On 2018-10-21 16:33, mj wrote: > On 10/21/2018 01:22 PM, Maurizio Caloro wrote: > > Please this are a complet Grouware solutions, are possible to use > > from this only the Calendar Synchronization and > > > > Date, Appontment functionality ? > > > > If that is hat you need, perhas you should checkout sogo: > > https://sogo.nu/ > > We have been running it for years, with the same backend-components > you are using: postfix and dovecot. (and active directory) > > MJ Hello Maurizio, I am interested to add Sogo to the mail server I have, and your feedback would be very interesting. I use postfix and dovecot as well, but I have a few questions: - Do I have to implement shared folders in order to share calendars? No, you don't. You control sharing calendars either via the SOGo admin interface or by yourself, when you allow access to your calendar. - Are the calendars and address books are in fact, stored in IMAP folders. No, they aren't. SOGo stores them in a database, e.g. a PostgreSQL server. You access the calendar either (natively) via the SOGo web interface or with a client that speaks either CalDAV or ActiveSync. In both cases the client connects to a http(s)-server that proxies connections to the SOGo sogod server. This server takes care of ACLs and logic and it also accesses the data backend, i.e. the SQL database. - If yes, are these folders hidden? There aren't any. p@rick Thank you, Patrick. This is very useful, I will add Sogo to Homebox, then. Kind regards, André -- https://github.com/progmaticltd/homebox
Re: AW: Calendar function ?
On 2018-10-21 16:33, mj wrote: Hi, On 10/21/2018 01:22 PM, Maurizio Caloro wrote: Please this are a complet Grouware solutions, are possible to use from this only the Calendar Synchronization and Date, Appontment functionality ? If that is hat you need, perhas you should checkout sogo: https://sogo.nu/ We have been running it for years, with the same backend-components you are using: postfix and dovecot. (and active directory) MJ Hello Maurizio, I am interested to add Sogo to the mail server I have, and your feedback would be very interesting. I use postfix and dovecot as well, but I have a few questions: - Do I have to implement shared folders in order to share calendars? - Are the calendars and address books are in fact, stored in IMAP folders. - If yes, are these folders hidden? Thanks for your insights Kind regards, André -- https://github.com/progmaticltd/homebox
Disable logging for localhost
Hello Everyone, I am using Zabbix to monitor my Dovecot server, and my logs are filled with lines like this: Oct 21 15:04:46 osaka dovecot[1256]: pop3-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, secured, session= Oct 21 15:05:29 osaka dovecot[1256]: imap-login: Aborted login (no auth attempts in 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, secured, session= How can I disable logging these actions from 127.0.0.1? Thanks, André -- https://github.com/progmaticltd/homebox
Re: Autofile Junk for all users
On 2018-05-23 15:11, @lbutlr wrote:
How would I get the rule
if header :contains "X-spam-flag" "YES" {
fileinto "Junk";
}
To apply to all users mail at delivery time?
Use the default scripts, works well for me.
https://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Visible_Default_Script
The Dovecot wiki is really helpful, have a look on it ;-)
--
André
Re: XMPP notifications
On 2018-05-23 12:42, Stephan Bosch wrote: Op 23-5-2018 om 9:54 schreef André Rodier: Hello all, What is the best way, using sieve, to send XMPP notifications? I am using Debian Stretch, with Dovecot core / sieve packages version 2.2.27-3+deb9u2. Should I use the enotify extension, or a script with extprograms extension? Thanks for your answers. Pigeonhole's enotify extension sadly still doesn't support doing that. You'll need to use extprograms with some external command line xmpp client. Regards, Stephan. Thank you, Stephan. Any example script somewhere? Thanks, André
XMPP notifications
Hello all, What is the best way, using sieve, to send XMPP notifications? I am using Debian Stretch, with Dovecot core / sieve packages version 2.2.27-3+deb9u2. Should I use the enotify extension, or a script with extprograms extension? Thanks for your answers. -- André Rodier
Re: sieve trace directory: error
Thank you for your support, Stephan. I will continue to send logs like that when relevant. Kind regards, André. On 14 May 2018 22:11:30 BST, Stephan Bosch wrote: > > >Op 31/03/2018 om 10:50 schreef André Rodier: >> On 31/03/18 09:29, Stephan Bosch wrote: >>> Op 3/31/2018 om 9:27 AM schreef André Rodier: >>>> Hello, >>>> >>>> I am using a custom directory for sieve logs and trace and it >apparently >>>> generate an error, in some corner case. >>>> >>>> - I have an IMAP global sieve script, that is run when appending >emails >>>> into the "Archives" folder. >>>> - I have Thunderbird configured to archive emails by year >>>> >>>> When archiving an email, for instance in "Archives/2018", I often >have >>>> an error message like this: >>>> >>>> Mar 31 08:03:02 homebox dovecot[14078]: imap(andre): Error: sieve: >>>> trace: >>>> >creat(/home/users/andre/mails/sieve/logs//andre.Archives/2018.3.20180331-080302.14235.5.trace) >>>> failed: No such file or directory >>>> >>>> I have this configuration (amongst) for sieve logs: >>>>sieve_user_log = ~/mails/sieve/logs/ >>>>sieve_trace_dir = ~/mails/sieve/logs/ >>>>sieve_trace_level = commands >>>>sieve_trace_debug = yes >>>>sieve_trace_addresses = yes >>>> >>>> Did I a misconfigured something? >>> This should fix the trace directory problem: >>> >https://github.com/stephanbosch/pigeonhole-core/commit/ee0d15407d231198519b4f87a0fc533b941765a2 >>> I still need to create some CI tests for this and it needs some >internal >>> review before it will be merged in Pigeonhole core. >>> >>> Also, I notice that your sieve_user_log setting points to a >directory. >>> That needs to be a file, otherwise it will not work. >>> >>> Regards, >>> >>> Stephan. >>> >> Thanks for your help and your advice, Stephen. >> >> I suppose it will take some time before being included into the >Debian >> main repository. >> >> I have amended the sieve_user_log setting. > >Currently scheduled for 2.3.3, which will take some time. > >Regards, > >Stephan. -- André
Dovecot fts script with solr
Hello again, I have created a parser script, a little bit more advanced than the one provided with Dovecot. The main feature is probably to index documents inside zip/rar/tgz archives... I am using Ansible, swaks and doveadm to run automatic tests for each supported content. For specific reasons, I am not yet able to add Apache Tika to the distribution. However, I already made some tests with it. For now, I want to talk about the indexing script. I also have noticed a few weird behaviours. I will mention them at the end, albeit I am not 100% sure where they are coming from. I realised last week that using QEMU snapshots was not working as expected, so I am now more careful with this feature. For the developers or users who would be interested and the Dovecot team members to understand my questions, here how the tests are working: To run my tests, I have a set of files in various formats, with a UUID inside. They are office files, text files, or even archives with a text file inside... The first test I am running is the script alone. I check that the script can convert the file to text, and then I use grep to check the UUID is present. This works *perfectly* for all the content, except ppt, but it's minor. The second test is full: - I use swaks to send the email with an attachment and the appropriate mime type. - I then refresh the index using doveadm rescan. - I check that fts search returns a line, with doveadm fts search. - I then expunge the mailbox to be sure that the next test is valid. For the second test, it works almost all the time, except in the following situations: - When the attachment is an email (mime type message/rfc822) - RTF (could be a bug in my script) - Text file in UTF16 (Even if this file is converted to UTF8) *Questions:* 1 - Is there any limitation or special case for the mime message/rfc822 2 - Is the mime type received coming from the email headers? 3 - When the script is called without arguments, what is the purpose of the extension at the end of each supported mime types? 4 - Can I return a wildcard in the supported mime types, for instance "text/* *" ? 5 - I would like to handle attachments of types application/octet-stream. I have added "application/octet-stream *", but I am not sure if dovecot will pass the attachments with these mime type or not. *Notes:* 1 - I used netcat to monitor the solr server. I realise that sometimes, the data sent to the solr server only contains the headers of the email, not the text returned by the parser. Especially with rfc822 messages. I will do more tests. 2 - I just finished to write the script, it's not yet refactored, but at list it is well documented. I will do a full security audit later. I am actually testing an associated AppArmor profile. 3 - I will do more intensive test on the script on bigger mail boxes with more attachments. 4 - I may rewrite the script in Python 5 - Suggestions welcome. I initially attached the current version of the script, but the email is probably pending for review...In this case, the last development version is on Github: https://github.com/progmaticltd/homebox/blob/dev/install/playbooks/roles/dovecot/files/fts/decode2text The configuration of supported mime types is a simple file, accessible on github as well: https://github.com/progmaticltd/homebox/blob/dev/install/playbooks/roles/dovecot/templates/fts/mime-supported.conf Thanks for your advices or suggestions.
Re: dovecot fts hangs on search
Hello Teemu, I am working on this today. The errors I have might be an artefact of the virtual machine I am using (KVM), especially the multiple snapshots. I will send a confirmation when I am sure. However, I had some difficulties understanding the principle of the original script, and how it runs. Perhaps the documentation can be improved a little bit. Kind regards, André On 04/05/18 07:11, Teemu Huovila wrote: Hello Could you plese 1. send the full output of doveconf -n 2. Check the SOLR logs for any errors 3. Describe your dovecot architecture, ie. if you are running a single backend or a more complex configuration. 4. Provide a backtrace of the core dump using the instructions on https://dovecot.org/bugreport.html br, Teemu On 04.05.2018 01:13, André Rodier wrote: On 02/05/18 22:17, André Rodier wrote: On 02/05/18 11:45, André Rodier wrote: On 2018-05-01 21:29, André Rodier wrote: On 2018-05-01 07:22, André Rodier wrote: Hello, I am trying to use Doevecot fts, with solr the script provided. To rebuild the index, I use the command: doveadm -D index -u mirina 'inbox' To rescan, I use: doveadm -D fts rescan -u mirina But when I do a search, with doveadm, the program hangs: doveadm -D search -u mirina text Madagascar Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this m$ ssage) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore thi$ message) doveadm(mirina): Debug: auth USER input: mirina home=/home/users/mirina uid=1002 gid=1001 mail=mirina@homebox.space doveadm(mirina): Debug: Added userdb setting: mail=mirina@homebox.space doveadm(mirina): Debug: Effective uid=1002, gid=1001, home=/home/users/mirina doveadm(mirina): Debug: Quota root: name=User quota backend=maildir args= doveadm(mirina): Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 doveadm(mirina): Debug: Quota grace: root=User quota bytes=214748364 (10%) doveadm(mirina): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/mails/maildir:INDEX=~/mails/indexes/ doveadm(mirina): Debug: maildir++: root=/home/users/mirina/mails/maildir, index=/home/users/mirina/mails/indexes, indexpvt=, control=, inbox=/home/users/mirina/mails/maildir, alt= doveadm(mirina): Debug: quota: quota_over_flag check: STORAGE ret=1 value=134 limit=2097152 doveadm(mirina): Debug: quota: quota_over_flag check: MESSAGE ret=0 value=3 limit=0 doveadm(mirina): Debug: quota: quota_over_flag=0((null)) vs currently overquota=0 doveadm(mirina): Debug: Namespace : Using permissions from /home/users/mirina/mails/maildir: mode=0700 gid=default doveadm(mirina): Debug: http-client: host localhost: Host created doveadm(mirina): Debug: http-client: host localhost: DNS lookup successful; got 2 IPs doveadm(mirina): Debug: http-client: peer [::1]:8080: Peer created doveadm(mirina): Debug: http-client: queue http://localhost:8080: Setting up connection to [::1]:8080 (1 requests pending) doveadm(mirina): Debug: http-client: peer [::1]:8080: Linked queue http://localhost:8080 (1 queues linked) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Started new connection to [::1]:8080 doveadm(mirina): Debug: http-client: request [Req1: GET http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:838a1701af1ae65a3e1232c62018+%2Buser:mi$ ina]: Submitted doveadm(mirina): Debug: http-client: Waiting for 1 request
Re: dovecot fts hangs on search
On 02/05/18 22:17, André Rodier wrote: On 02/05/18 11:45, André Rodier wrote: On 2018-05-01 21:29, André Rodier wrote: On 2018-05-01 07:22, André Rodier wrote: Hello, I am trying to use Doevecot fts, with solr the script provided. To rebuild the index, I use the command: doveadm -D index -u mirina 'inbox' To rescan, I use: doveadm -D fts rescan -u mirina But when I do a search, with doveadm, the program hangs: doveadm -D search -u mirina text Madagascar Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this m$ ssage) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore thi$ message) doveadm(mirina): Debug: auth USER input: mirina home=/home/users/mirina uid=1002 gid=1001 mail=mirina@homebox.space doveadm(mirina): Debug: Added userdb setting: mail=mirina@homebox.space doveadm(mirina): Debug: Effective uid=1002, gid=1001, home=/home/users/mirina doveadm(mirina): Debug: Quota root: name=User quota backend=maildir args= doveadm(mirina): Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 doveadm(mirina): Debug: Quota grace: root=User quota bytes=214748364 (10%) doveadm(mirina): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/mails/maildir:INDEX=~/mails/indexes/ doveadm(mirina): Debug: maildir++: root=/home/users/mirina/mails/maildir, index=/home/users/mirina/mails/indexes, indexpvt=, control=, inbox=/home/users/mirina/mails/maildir, alt= doveadm(mirina): Debug: quota: quota_over_flag check: STORAGE ret=1 value=134 limit=2097152 doveadm(mirina): Debug: quota: quota_over_flag check: MESSAGE ret=0 value=3 limit=0 doveadm(mirina): Debug: quota: quota_over_flag=0((null)) vs currently overquota=0 doveadm(mirina): Debug: Namespace : Using permissions from /home/users/mirina/mails/maildir: mode=0700 gid=default doveadm(mirina): Debug: http-client: host localhost: Host created doveadm(mirina): Debug: http-client: host localhost: DNS lookup successful; got 2 IPs doveadm(mirina): Debug: http-client: peer [::1]:8080: Peer created doveadm(mirina): Debug: http-client: queue http://localhost:8080: Setting up connection to [::1]:8080 (1 requests pending) doveadm(mirina): Debug: http-client: peer [::1]:8080: Linked queue http://localhost:8080 (1 queues linked) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Started new connection to [::1]:8080 doveadm(mirina): Debug: http-client: request [Req1: GET http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:838a1701af1ae65a3e1232c62018+%2Buser:mi$ ina]: Submitted doveadm(mirina): Debug: http-client: Waiting for 1 requests to finish doveadm(mirina): Debug: http-client: peer [::1]:8080: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) doveadm(mirina): Debug: http-client: peer [::1]:8080: Making new connection 1 of 1 doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: HTTP connection created (1 parallel connections exist) doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Connected doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Ready for requests doveadm(mirina): Debug: http-client: peer [::1]:8080: Successfully connected (connections=1) doveadm(mirina): Debug: http-client: peer [::1]:8080: Using 1 idle connections to handle 1 requests (1 total connections ready) doveadm(mirina): Debug: http-client: queue http://loca
Re: dovecot fts hangs on search
On 02/05/18 11:45, André Rodier wrote: On 2018-05-01 21:29, André Rodier wrote: On 2018-05-01 07:22, André Rodier wrote: Hello, I am trying to use Doevecot fts, with solr the script provided. To rebuild the index, I use the command: doveadm -D index -u mirina 'inbox' To rescan, I use: doveadm -D fts rescan -u mirina But when I do a search, with doveadm, the program hangs: doveadm -D search -u mirina text Madagascar Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this m$ ssage) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore thi$ message) doveadm(mirina): Debug: auth USER input: mirina home=/home/users/mirina uid=1002 gid=1001 mail=mirina@homebox.space doveadm(mirina): Debug: Added userdb setting: mail=mirina@homebox.space doveadm(mirina): Debug: Effective uid=1002, gid=1001, home=/home/users/mirina doveadm(mirina): Debug: Quota root: name=User quota backend=maildir args= doveadm(mirina): Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 doveadm(mirina): Debug: Quota grace: root=User quota bytes=214748364 (10%) doveadm(mirina): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/mails/maildir:INDEX=~/mails/indexes/ doveadm(mirina): Debug: maildir++: root=/home/users/mirina/mails/maildir, index=/home/users/mirina/mails/indexes, indexpvt=, control=, inbox=/home/users/mirina/mails/maildir, alt= doveadm(mirina): Debug: quota: quota_over_flag check: STORAGE ret=1 value=134 limit=2097152 doveadm(mirina): Debug: quota: quota_over_flag check: MESSAGE ret=0 value=3 limit=0 doveadm(mirina): Debug: quota: quota_over_flag=0((null)) vs currently overquota=0 doveadm(mirina): Debug: Namespace : Using permissions from /home/users/mirina/mails/maildir: mode=0700 gid=default doveadm(mirina): Debug: http-client: host localhost: Host created doveadm(mirina): Debug: http-client: host localhost: DNS lookup successful; got 2 IPs doveadm(mirina): Debug: http-client: peer [::1]:8080: Peer created doveadm(mirina): Debug: http-client: queue http://localhost:8080: Setting up connection to [::1]:8080 (1 requests pending) doveadm(mirina): Debug: http-client: peer [::1]:8080: Linked queue http://localhost:8080 (1 queues linked) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Started new connection to [::1]:8080 doveadm(mirina): Debug: http-client: request [Req1: GET http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:838a1701af1ae65a3e1232c62018+%2Buser:mi$ ina]: Submitted doveadm(mirina): Debug: http-client: Waiting for 1 requests to finish doveadm(mirina): Debug: http-client: peer [::1]:8080: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) doveadm(mirina): Debug: http-client: peer [::1]:8080: Making new connection 1 of 1 doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: HTTP connection created (1 parallel connections exist) doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Connected doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Ready for requests doveadm(mirina): Debug: http-client: peer [::1]:8080: Successfully connected (connections=1) doveadm(mirina): Debug: http-client: peer [::1]:8080: Using 1 idle connections to handle 1 requests (1 total connections ready) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Connection to peer [::1]:8080
Re: dovecot fts hangs on search
On 2018-05-01 21:29, André Rodier wrote: On 2018-05-01 07:22, André Rodier wrote: Hello, I am trying to use Doevecot fts, with solr the script provided. To rebuild the index, I use the command: doveadm -D index -u mirina 'inbox' To rescan, I use: doveadm -D fts rescan -u mirina But when I do a search, with doveadm, the program hangs: doveadm -D search -u mirina text Madagascar Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this m$ ssage) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore thi$ message) doveadm(mirina): Debug: auth USER input: mirina home=/home/users/mirina uid=1002 gid=1001 mail=mirina@homebox.space doveadm(mirina): Debug: Added userdb setting: mail=mirina@homebox.space doveadm(mirina): Debug: Effective uid=1002, gid=1001, home=/home/users/mirina doveadm(mirina): Debug: Quota root: name=User quota backend=maildir args= doveadm(mirina): Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 doveadm(mirina): Debug: Quota grace: root=User quota bytes=214748364 (10%) doveadm(mirina): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/mails/maildir:INDEX=~/mails/indexes/ doveadm(mirina): Debug: maildir++: root=/home/users/mirina/mails/maildir, index=/home/users/mirina/mails/indexes, indexpvt=, control=, inbox=/home/users/mirina/mails/maildir, alt= doveadm(mirina): Debug: quota: quota_over_flag check: STORAGE ret=1 value=134 limit=2097152 doveadm(mirina): Debug: quota: quota_over_flag check: MESSAGE ret=0 value=3 limit=0 doveadm(mirina): Debug: quota: quota_over_flag=0((null)) vs currently overquota=0 doveadm(mirina): Debug: Namespace : Using permissions from /home/users/mirina/mails/maildir: mode=0700 gid=default doveadm(mirina): Debug: http-client: host localhost: Host created doveadm(mirina): Debug: http-client: host localhost: DNS lookup successful; got 2 IPs doveadm(mirina): Debug: http-client: peer [::1]:8080: Peer created doveadm(mirina): Debug: http-client: queue http://localhost:8080: Setting up connection to [::1]:8080 (1 requests pending) doveadm(mirina): Debug: http-client: peer [::1]:8080: Linked queue http://localhost:8080 (1 queues linked) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Started new connection to [::1]:8080 doveadm(mirina): Debug: http-client: request [Req1: GET http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:838a1701af1ae65a3e1232c62018+%2Buser:mi$ ina]: Submitted doveadm(mirina): Debug: http-client: Waiting for 1 requests to finish doveadm(mirina): Debug: http-client: peer [::1]:8080: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) doveadm(mirina): Debug: http-client: peer [::1]:8080: Making new connection 1 of 1 doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: HTTP connection created (1 parallel connections exist) doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Connected doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Ready for requests doveadm(mirina): Debug: http-client: peer [::1]:8080: Successfully connected (connections=1) doveadm(mirina): Debug: http-client: peer [::1]:8080: Using 1 idle connections to handle 1 requests (1 total connections ready) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Connection to peer [::1]:8080 claimed request [Req1: GET http://localho
Re: dovecot fts hangs on search
On 2018-05-01 07:22, André Rodier wrote: Hello, I am trying to use Doevecot fts, with solr the script provided. To rebuild the index, I use the command: doveadm -D index -u mirina 'inbox' To rescan, I use: doveadm -D fts rescan -u mirina But when I do a search, with doveadm, the program hangs: doveadm -D search -u mirina text Madagascar Debug: Loading modules from directory: /usr/lib/dovecot/modules Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_lookup_dict_iterate_visible_next (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this m$ ssage) Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore thi$ message) doveadm(mirina): Debug: auth USER input: mirina home=/home/users/mirina uid=1002 gid=1001 mail=mirina@homebox.space doveadm(mirina): Debug: Added userdb setting: mail=mirina@homebox.space doveadm(mirina): Debug: Effective uid=1002, gid=1001, home=/home/users/mirina doveadm(mirina): Debug: Quota root: name=User quota backend=maildir args= doveadm(mirina): Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 doveadm(mirina): Debug: Quota grace: root=User quota bytes=214748364 (10%) doveadm(mirina): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/mails/maildir:INDEX=~/mails/indexes/ doveadm(mirina): Debug: maildir++: root=/home/users/mirina/mails/maildir, index=/home/users/mirina/mails/indexes, indexpvt=, control=, inbox=/home/users/mirina/mails/maildir, alt= doveadm(mirina): Debug: quota: quota_over_flag check: STORAGE ret=1 value=134 limit=2097152 doveadm(mirina): Debug: quota: quota_over_flag check: MESSAGE ret=0 value=3 limit=0 doveadm(mirina): Debug: quota: quota_over_flag=0((null)) vs currently overquota=0 doveadm(mirina): Debug: Namespace : Using permissions from /home/users/mirina/mails/maildir: mode=0700 gid=default doveadm(mirina): Debug: http-client: host localhost: Host created doveadm(mirina): Debug: http-client: host localhost: DNS lookup successful; got 2 IPs doveadm(mirina): Debug: http-client: peer [::1]:8080: Peer created doveadm(mirina): Debug: http-client: queue http://localhost:8080: Setting up connection to [::1]:8080 (1 requests pending) doveadm(mirina): Debug: http-client: peer [::1]:8080: Linked queue http://localhost:8080 (1 queues linked) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Started new connection to [::1]:8080 doveadm(mirina): Debug: http-client: request [Req1: GET http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:838a1701af1ae65a3e1232c62018+%2Buser:mi$ ina]: Submitted doveadm(mirina): Debug: http-client: Waiting for 1 requests to finish doveadm(mirina): Debug: http-client: peer [::1]:8080: Creating 1 new connections to handle requests (already 0 usable, connecting to 0, closing 0) doveadm(mirina): Debug: http-client: peer [::1]:8080: Making new connection 1 of 1 doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: HTTP connection created (1 parallel connections exist) doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Connected doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Ready for requests doveadm(mirina): Debug: http-client: peer [::1]:8080: Successfully connected (connections=1) doveadm(mirina): Debug: http-client: peer [::1]:8080: Using 1 idle connections to handle 1 requests (1 total connections ready) doveadm(mirina): Debug: http-client: queue http://localhost:8080: Connection to peer [::1]:8080 claimed request [Req1: GET http://localhost:8080/solr/select?fl=uid,score&rows=1&
dovecot fts hangs on search
agascar+OR+bod$
:Madagascar)&fq=%2Bbox:838a1701af1ae65a3e1232c62018+%2Buser:mirina]
doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Claimed request [Req1: GET
http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:838a1701af1ae6$
a3e1232c62018+%2Buser:mirina]
[...]
doveadm(mirina): Debug: http-client: request [Req5: GET
http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:6dd14d3aae1ae65a3e1232c62018+%2Buser:mirina]:
Sent header
doveadm(mirina): Debug: http-client: peer [::1]:8080: No more requests to
service for this peer (1 connections exist)
doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Got 200 response for request [Req5:
GET
http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:6dd14d3aae1ae65a3e1232c62018+%2Buser:mirina]
(took 2 ms + 0 ms in queue)
doveadm(mirina): Debug: http-client: conn [::1]:8080 [0]: Response payload
stream destroyed (0 ms after initial response)
doveadm(mirina): Debug: http-client: request [Req5: GET
http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:6dd14d3aae1ae65a3e1232c62018+%2Buser:mirina]:
Finished
doveadm(mirina): Debug: http-client: queue http://localhost:8080: Dropping request [Req5: GET
http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:6dd14d3aae1ae65a3e1232c62018+%2Buser:mirina]
doveadm(mirina): Debug: http-client: host localhost: Host is idle (timeout =
1799974 msecs)
doveadm(mirina): Debug: http-client: request [Req5: GET
http://localhost:8080/solr/select?fl=uid,score&rows=1&sort=uid+asc&q=%7b!lucene+q.op%3dAND%7d(hdr:Madagascar+OR+body:Madagascar)&fq=%2Bbox:6dd14d3aae1ae65a3e1232c62018+%2Buser:mirina]:
Free (requests left=1)
doveadm(mirina): Debug: http-client: All requests finished
When I do the search with an imap client, I have this error after a long
timeout:
May 01 07:08:58 homebox dovecot[5683]: imap(mirina): Panic: file imap-client.c: line 854
(client_check_command_hangs): assertion failed: (client->io != NULL ||
(client->output_cmd_lock != NULL && client->output_cmd_lock != client->input_lock))
May 01 07:08:58 homebox dovecot[5683]: imap(mirina): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0x95e92) [0x7f216b6f6e92] -> /usr/lib/dovecot/libdovecot.so.0(+0x95f8d)
[0x7f216b6f6f8d] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f216b68ca91] ->
dovecot/imap(client_continue_pending_input+0x1fc) [0x55f44a7eb15c] -> dovecot/imap(client_input+0x120)
[0x55f44a7eb460] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x52) [0x7f216b70b9f2] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109) [0x7f216b70d029] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x3c) [0x7f216b70ba8c] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f216b70bc38] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f216b692fd3] -> dovecot/imap(main+0x328)
[0x55f44a7dde68] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f216b2e22e1] ->
dovecot/imap(_start+0x2a) [0x55f44a7ddfea]
May 01 07:08:58 homebox dovecot[5683]: imap(mirina): Fatal: master:
service(imap): child 6149 killed with signal 6 (core dumps disabled)
May 01 07:08:58 homebox in.imapproxyd[20305]: Raw_Proxy(): IMAP server
unexpectedly closed the connection on sd 11
My 10-master.conf:
service decode2text {
executable = script /usr/local/bin/decode2text.sh
user = dovecot
unix_listener decode2text {
mode = 0666
}
}
And my 90-plugins.cof:
# Add dovecot fts config
fts = solr
fts_autoindex = no
fts_solr = url=http://localhost:8080/solr/ debug
fts_decoder = decode2text
Am I doing anything wrong?
Thanks for your help.
André
Panic when using doveadm fts lookup
Hello, I am actually testing dovecot fts search, using doveadm fts lookup. I run this command: doveadm fts lookup -u andre subject "Test" And this generate this error: doveadm(andre): Panic: file mail-storage.c: line 1694 (mailbox_get_open_status): assertion failed: (box->opened) doveadm(andre): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x95e92) [0x7fa7ac9e8e92] -> /usr/lib/dovecot/libdovecot.so.0(+0x95f19) [0x7fa7ac9e8f19] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fa7ac97ea91] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_get_open_status+0x68) [0x7fa7accb5f68] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x494d) [0x7fa7ab90394d] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_backend_lookup+0x4d) [0x7fa7abf6861d] -> /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so(+0x2821) [0x7fa7aabff821] -> doveadm(+0x28fcd) [0x5617545e8fcd] -> doveadm(+0x29ba8) [0x5617545e9ba8] -> doveadm(doveadm_cmd_ver2_to_mail_cmd_wrapper+0x21c) [0x5617545ea9ec] -> doveadm(doveadm_cmd_run_ver2+0x555) [0x5617545f9d25] -> doveadm(doveadm_cmd_try_run_ver2+0x37) [0x5617545f9d77] -> doveadm(main+0x1da) [0x5617545d9afa] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7fa7ac5d42e1] -> doveadm(_start+0x2a) [0x5617545d9eda] Do I have to do anything special to test the dovecot fts index with doveadm lookup? I have not found the documentation on the dovecot web site. Thanks for your advices. -- André Rodier. https://github.com/progmaticltd/homebox
Re: dovecot sieve duplicates detection
On 25/04/18 20:20, Stephan Bosch wrote: Op 23/04/2018 om 22:03 schreef André Rodier: On 23/04/18 14:18, Stephan Bosch wrote: Op 11-4-2018 om 23:58 schreef André Rodier: Hello, I have tested the sieve duplicate script with success so far, but I have a question. Sieve duplicate script? You mean the Sieve duplicate extension (RFC 7352)? I would like to know if the "duplicate" sieve flag in Dovecot is global to all folders, or specific to one folder only. It uses the lda-dupes file in the user's home directory. So, it is not normally related to folders, although the identifier used for duplicate matching could be composed of the mailbox name if you want. For instance, if I copy an email from one folder to another, and I have a discard action on duplicate email, is this action will be applied (in this case, discard) or not. Are you talking about IMAPSieve now? I am not sure "duplicate" is currently even allowed in that context. If the duplicate is global to all folders, is there a way to restrict the search in one folder only. You can set the :uniqueid parameter accordingly. Regards, Stephan. Thank you, Stephan. Yes, I meant the Sieve duplicate extension. I am using a program to import email (mbsync), which use the IMAP append function. Sometimes, the import fail and I have to restart the program. Unfortunately, the same emails are imported again. I found a fix by using a dovecot IMAP sieve script executed on the APPEND action (https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). I wrote a custom sieve script that "discard" the ones that are detected as "duplicate". It worked very well and the emails were not any more imported twice. However, there was a huge side effect: archiving an email with Thunderbird is not working any more, and even lost! I have been able to understand the error as this: 1. When archiving an email with Thunderbird, it is first copied (APPEND) into the archive folder, but the original folder is not expunged. 2. The sieve script detect the email as duplicate, and discard it. 3. When the original folder is expunged, the source email is lost... My conclusion was the duplicate detection function is global to all folders. If I could restrict the detection of duplicates in the current folder only, this would let me run the import program again without error. Specify the ID used for duplicate checking explicitly using the :uniqueid argument (https://tools.ietf.org/html/rfc7352#section-3.1). Using the variables extenion, compose the uniqueid from the message-id and the mailbox name. Regards, Stephan. Thank you, I will try this. André
Re: dovecot sieve duplicates detection
On 23/04/18 14:18, Stephan Bosch wrote: Op 11-4-2018 om 23:58 schreef André Rodier: Hello, I have tested the sieve duplicate script with success so far, but I have a question. Sieve duplicate script? You mean the Sieve duplicate extension (RFC 7352)? I would like to know if the "duplicate" sieve flag in Dovecot is global to all folders, or specific to one folder only. It uses the lda-dupes file in the user's home directory. So, it is not normally related to folders, although the identifier used for duplicate matching could be composed of the mailbox name if you want. For instance, if I copy an email from one folder to another, and I have a discard action on duplicate email, is this action will be applied (in this case, discard) or not. Are you talking about IMAPSieve now? I am not sure "duplicate" is currently even allowed in that context. If the duplicate is global to all folders, is there a way to restrict the search in one folder only. You can set the :uniqueid parameter accordingly. Regards, Stephan. Thank you, Stephan. Yes, I meant the Sieve duplicate extension. I am using a program to import email (mbsync), which use the IMAP append function. Sometimes, the import fail and I have to restart the program. Unfortunately, the same emails are imported again. I found a fix by using a dovecot IMAP sieve script executed on the APPEND action (https://wiki.dovecot.org/Pigeonhole/Sieve/Plugins/IMAPSieve). I wrote a custom sieve script that "discard" the ones that are detected as "duplicate". It worked very well and the emails were not any more imported twice. However, there was a huge side effect: archiving an email with Thunderbird is not working any more, and even lost! I have been able to understand the error as this: 1. When archiving an email with Thunderbird, it is first copied (APPEND) into the archive folder, but the original folder is not expunged. 2. The sieve script detect the email as duplicate, and discard it. 3. When the original folder is expunged, the source email is lost... My conclusion was the duplicate detection function is global to all folders. If I could restrict the detection of duplicates in the current folder only, this would let me run the import program again without error. Kind regards, André.
dovecot sieve duplicates detection
Hello, I have tested the sieve duplicate script with success so far, but I have a question. I would like to know if the "duplicate" sieve flag in Dovecot is global to all folders, or specific to one folder only. For instance, if I copy an email from one folder to another, and I have a discard action on duplicate email, is this action will be applied (in this case, discard) or not. If the duplicate is global to all folders, is there a way to restrict the search in one folder only. Thanks for your help. André
Re: {Spam?} domain email autoconfiguration
On 01/04/18 05:02, DurgaPrasad - DatasoftComnet wrote:
> Hi,
> Actually I am also interested in learning.
>
> Regards
> Durga Prasad
> +919849111010
>
> -Original Message-
> From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf Of David Mehler
> Sent: 01 April 2018 06:10
> To: dovecot
> Subject: {Spam?} domain email autoconfiguration
>
> Hello,
>
> If anyone has autoconfiguration going with their email domain please email me
> privately. I'd like to ask you some questions about your setup. What do you
> use?
>
> Thanks.
> Dave.
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>
Hello,
The official option is RFC6186:
https://tools.ietf.org/html/rfc6186
There is also Autoconfig for Mozilla Thunderbird, and Autodiscover, for
Microsoft Outlook.
Thunderbird will return the parameters at an address like
http://autoconfig.example.com/mail/config-v1.1.xml?emailaddress
Outlook will send a POST query to a URL like this:
https://autodiscover.example.com//autodiscover/autodiscover.xml
They both return XML with the parameters, and a special subdomain.
I have implemented both for a custom mail server project, you can find
the link below:
https://github.com/progmaticltd/homebox/tree/master/install/playbooks/roles
See the roles with a matching name.
I might implement the first one later, it is just a matter of DNS records...
André Rodier
--
https://github.com/progmaticltd/homebox
Re: sieve trace directory: error
On 31/03/18 09:29, Stephan Bosch wrote: > Op 3/31/2018 om 9:27 AM schreef André Rodier: >> Hello, >> >> I am using a custom directory for sieve logs and trace and it apparently >> generate an error, in some corner case. >> >> - I have an IMAP global sieve script, that is run when appending emails >> into the "Archives" folder. >> - I have Thunderbird configured to archive emails by year >> >> When archiving an email, for instance in "Archives/2018", I often have >> an error message like this: >> >> Mar 31 08:03:02 homebox dovecot[14078]: imap(andre): Error: sieve: >> trace: >> creat(/home/users/andre/mails/sieve/logs//andre.Archives/2018.3.20180331-080302.14235.5.trace) >> failed: No such file or directory >> >> I have this configuration (amongst) for sieve logs: >> sieve_user_log = ~/mails/sieve/logs/ >> sieve_trace_dir = ~/mails/sieve/logs/ >> sieve_trace_level = commands >> sieve_trace_debug = yes >> sieve_trace_addresses = yes >> >> Did I a misconfigured something? > > This should fix the trace directory problem: > https://github.com/stephanbosch/pigeonhole-core/commit/ee0d15407d231198519b4f87a0fc533b941765a2 > I still need to create some CI tests for this and it needs some internal > review before it will be merged in Pigeonhole core. > > Also, I notice that your sieve_user_log setting points to a directory. > That needs to be a file, otherwise it will not work. > > Regards, > > Stephan. > Thanks for your help and your advice, Stephen. I suppose it will take some time before being included into the Debian main repository. I have amended the sieve_user_log setting. Kind regards, André -- https://github.com/progmaticltd/homebox
sieve trace directory: error
Hello, I am using a custom directory for sieve logs and trace and it apparently generate an error, in some corner case. - I have an IMAP global sieve script, that is run when appending emails into the "Archives" folder. - I have Thunderbird configured to archive emails by year When archiving an email, for instance in "Archives/2018", I often have an error message like this: Mar 31 08:03:02 homebox dovecot[14078]: imap(andre): Error: sieve: trace: creat(/home/users/andre/mails/sieve/logs//andre.Archives/2018.3.20180331-080302.14235.5.trace) failed: No such file or directory I have this configuration (amongst) for sieve logs: sieve_user_log = ~/mails/sieve/logs/ sieve_trace_dir = ~/mails/sieve/logs/ sieve_trace_level = commands sieve_trace_debug = yes sieve_trace_addresses = yes Did I a misconfigured something? Thanks for your help, André -- https://github.com/progmaticltd/homebox
Re: recipient delimiter and sieve filters
On 24/03/18 10:24, Stephan Bosch wrote:
> Op 3/24/2018 om 9:16 AM schreef André Rodier:
>> Dear all,
>>
>> I have found a way to automatically copy sent emails in the "Sent"
>> folder, but I am not sure it is the simplest and more reliable way on
>> the long term.
>>
>> I am open to suggestions if I miss a feature in Dovecot - or Postfix,
>> that allows me to do this. I vaguely remember an SMTP extension that do
>> that, but my memory could be wrong.
>
> I think you mean:
>
> https://tools.ietf.org/html/rfc4468
>
> Which was recently added to Dovecot as a feature:
>
> https://wiki.dovecot.org/Submission
>
> However, clients will not support this at this time, so you will not be
> helped there.
>
>> First, I use "~" as a recipient delimiter. Then, I set up a postfix
>> senders_bcc_map that add the ~Sent part:
>>
>> andre@homebox.space → bcc: andre~Sent@homebox.space
>>
>> It worked, except that the emails was marked as new.
>>
>> I tried first to add a global sieve filter like this:
>> --
>> # Sieve script executed before for user
>> require ["fileinto","imap4flags"];
>>
>> # Move automatically copied emails to the sent folder
>> # And mark them as read
>> if header :contains "Delivered-To" "~Sent"
>> {
>> setflag "\\Seen";
>> fileinto "Sent";
>> }
>> --
>> But it did not work, so maybe there is a trick I am not aware of.
>
> The Delivered-To header is currently only added when there is a single
> recipient in the LMTP session. It would not surprise me when the ~Sent
> Bcc is batched in the same transaction by Postfix, which means that this
> header is never added.
>
> You'd better use the envelope extension:
>
> https://tools.ietf.org/html/rfc5228#section-5.4
>
>
> Regards,
>
> Stephan.
>
Thank you, Stephan.
Yes, the link was what I had in mind.
Kind regards,
André
recipient delimiter and sieve filters
Dear all,
I have found a way to automatically copy sent emails in the "Sent"
folder, but I am not sure it is the simplest and more reliable way on
the long term.
I am open to suggestions if I miss a feature in Dovecot - or Postfix,
that allows me to do this. I vaguely remember an SMTP extension that do
that, but my memory could be wrong.
First, I use "~" as a recipient delimiter. Then, I set up a postfix
senders_bcc_map that add the ~Sent part:
andre@homebox.space → bcc: andre~Sent@homebox.space
It worked, except that the emails was marked as new.
I tried first to add a global sieve filter like this:
--
# Sieve script executed before for user
require ["fileinto","imap4flags"];
# Move automatically copied emails to the sent folder
# And mark them as read
if header :contains "Delivered-To" "~Sent"
{
setflag "\\Seen";
fileinto "Sent";
}
--
But it did not work, so maybe there is a trick I am not aware of.
If I use the "Received" header and a regex, it works
--
require ["fileinto","imap4flags","regex"];
if header :regex "Received" "for <[a-z]+~Sent@homebox.space>;"
{
setflag "\\Seen";
fileinto "Sent";
}
--
This is working perfectly, but there is limitations, IMHO.
- If the recipient delimiter is not compatible with a regex (e.g. "+"),
then the script will probably fail.
- Parsing every new email using a regular expression might not be the
best option in term of system load.
So, I finally use this:
--
# Move automatically copied emails to the sent folder
# And mark them as read
require ["fileinto","imap4flags"];
if header :contains "Received" "~Sent"
{
setflag "\\Seen";
fileinto "Sent";
}
------
My question is why the first one does not work? Does the email has been
moved to the Sent folder, and dovecot sieve plugin is not able to find
the email any more?
Kind regards,
André
--
https://github.com/progmaticltd/homebox
Re: Accentuated email addresses stored in LDAP
On 19/03/18 06:30, Aki Tuomi wrote: > > > On 19.03.2018 07:58, Alexander Dalloz wrote: >> Am 18.03.2018 um 23:25 schrieb André Rodier: >>> Every other alias works, but not those with accents. They are correctly >>> stored in the LDAP database, using base64. >> >> Why do you even think a character like 'é' would be legal in the local >> part of an email address? >> >> Alexander >> > Hi! > > This is slightly more complicated than that. UTF-8 is legal part of > local part of an email address, if SMTPUTF8 is used (as per RFC 6532). > Dovecot does not yet fully support UTF-8 in certain parts, particularly > we do not yet support RFC 6855 (IMAP Support for UTF-8), but we are > working on getting that to work, after which you can log in with UTF-8 > usernames and passwords reliably. > > Aki > Thank you, Aki, Does it means Dovecot LMTP supports querying the LDAP server using UTF-8 characters? In this case, the problem might not be Dovecot, but OpenLDAP. I had to use base64 encoding to store email addresses, because they are rejected otherwise, with an error message of illegal characters. I also noticed that some of the fields, like 'dn' and 'cn' where automatically encoded in base64 when stored when including accentuated chars. Regarding logging in with international characters, I have not tested it yet, but I am happy to test and to provide feedback if you are interested. Thanks again for this excellent software, I have been using it for more than a decade, now... Kind regards, André
Accentuated email addresses stored in LDAP
Hello everyone, I have successfully set up Dovecot with an LDAP backend, and everything works like a charm, except one small use case. I know it is possible, so if anyone knows a solution, thank you! Basically, I just want to send an email to one of my email address, which contains an accent, i.e. andré@homebox.space Every other alias works, but not those with accents. They are correctly stored in the LDAP database, using base64. I suspect I would need to use base64 again before the lookup, but I don't know how to do and if it is even possible. My filter is set up as this: filter=(&(objectClass=posixAccount)(|(uid=%n)(mail=%u)(mailLocalAddress=%u))) It works perfectly for any email without accent. Thanks for your help. > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: Loading modules from > directory: /usr/lib/dovecot/modules/auth > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: Loading modules from > directory: /usr/lib/dovecot/modules/auth > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: Module loaded: > /usr/lib/dovecot/modules/auth/libauthdb_ldap.so > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: Read auth token secret > from /var/run/dovecot/auth-token-secret.dat > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: LDAP initialization took > 7 msecs > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: auth client connected > (pid=0) > > Mar 18 22:14:28 homebox dovecot[24636]: quota-status(andré@homebox.space): > Debug: Loading modules from directory: /usr/lib/dovecot/modules > > Mar 18 22:14:28 homebox dovecot[24636]: > quota-status(andré@homebox.space): Debug: Module loaded: > /usr/lib/dovecot/modules/lib10_quota_plugin.so > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: master in: USER1 >andré@homebox.spaceservice=quota-status > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: > ldap(andré@homebox.space): user search: base=dc=homebox,dc=space > scope=subtree > filter=(&(objectClass=posixAccount)(|(uid=andré)(mailLocalAddress=andré@homebox.space))) > fields=homeDirectory,uidNumber,gidNumber,mail > > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: > ldap(andré@homebox.space): no fields returned by the server > > Mar 18 22:14:28 homebox dovecot[24636]: auth: ldap(andré@homebox.space): > unknown user > > > Mar 18 22:14:28 homebox dovecot[24636]: auth: Debug: userdb out: NOTFOUND >1 > > Mar 18 22:14:28 homebox dovecot[24636]: quota-status(andré@homebox.space): > Debug: auth USER input: > > > Mar 18 22:14:28 homebox postfix/smtpd[24734]: NOQUEUE: reject: RCPT from > smtpfr.rodier.me[164.132.53.161]: 554 5.7.1 : Recipient > address rejected: Unknown user; from= > to= pro > to=ESMTP helo= > > > Mar 18 22:14:28 homebox postfix/smtpd[24734]: disconnect from > smtpfr.rodier.me[164.132.53.161] ehlo=1 mail=1 rcpt=0/1 data=0/1 rset=1 > quit=1 commands=4/6 >
AppArmor profile for Dovecot on Debian Stretch
Hello all, Is there any guide to help in the configuration of Dovecot for AppArmor on Debian / Ubuntu ? Or maybe does any of you already have something that works? I am actually adding AppArmor on an email server project, and I had some trouble with the versions from Debian. I would like to avoid - if possible - the long try and error process for each Dovecot executable. I am using IMAP, LMTP, Sieve and ManageSieve, with LDAP users. The full project reference is here: https://github.com/progmaticltd/homebox ii dovecot-core 1:2.2.27-3+deb9u2 amd64 ii dovecot-imapd 1:2.2.27-3+deb9u2 amd64 ii dovecot-ldap 1:2.2.27-3+deb9u2 amd64 ii dovecot-lmtpd 1:2.2.27-3+deb9u2 amd64 ii dovecot-managesieved 1:2.2.27-3+deb9u2 amd64 ii dovecot-pop3d 1:2.2.27-3+deb9u2 amd64 ii dovecot-sieve 1:2.2.27-3+deb9u2 amd64 Thanks for your advices André
Re: hosting emails at home
Hello Bruce et al, One thing I did not specify. I have very strict idea in the way I am installing a package on a server I put online. Ideally, I want to have some install process I setup once and I don't have to worry about anything, especially security. When I have to install a software, let's say Roundcube, I prefer to use the native version that comes with Debian server. Of course, it is a little bit outdated, but I know there is a security team behind that publishes security patches. I know these security patches will be applied, while I sleep or while I am in holidays, if I install and configure correctly unattended upgrades packages. If really I want more control, I know there is packages on Debian that will send me an email when updates are available, and I can install them from anywhere using SSH. In no case, I would be comfortable installing, on a Live server, Roundcube from the git repository, which is done fir this project. It is far too easy to forget and leave it for months with security issues opened for a while. Yes, I know there is cron scripts I can use to update the repository. But even in this case, who is guarantee me that nothing will break on my server? Once again, there is a Debian team that do a fabulous integration work, and I don't want to break my mail server just to have the latest version of Roundcube or Owncloud. By staying inside the Debian ecosystem, I am also sure that some third party applications or repositories will stay nicely integrated with the current state of my server. For instance, I know that I should be able to add syncthing (https://apt.syncthing.net/) repository as part of the deployment process, without worrying too much about conflict from files overwritten by these kind of manipulations. I don't say one opinion is better that the other, it is just the way I prefer to work - and as a matter - to live. Kind regards, André On 10/12/17 19:46, André Rodier wrote: > Thank you, > > I remember to had a look at this project, and I found it huge. > > I started mine because I want LDAP authentication. > > I also wanted less features / programs, less obtrusive, and better > attention to small details, like automatic DKIM generation and DNS updates. > > I hope not to end up with something as huge. > > André > > On 10/12/17 19:19, br...@secryption.com wrote: >> Check out https://github.com/sovereign/sovereign/blob/master/README.md >> >> >> Might have some of what you are looking for already done. >> >> >> Bruce >> >> >> On Dec 10, 2017 2:06 PM, André Rodier wrote: >> >> Hello everyone, >> >> I have been using Postfix and Dovecot for my personal emails for years. >> After being tired of reinstalling my personal mail server many times, I >> am currently writing some Ansible scripts to do it automatically. >> >> I obviously checked the other projects, and did not found anything close >> to what I am looking for, so I am implementing it now. >> >> The final goal is to have a box that once online, would setup itself, by >> creating the certificates, the DKIM keys and update the appropriate DNS >> records. >> >> This is so far what I have achieved: >> - Automatic generation of certificates using LetsEncrypt >> - Automatic update of the domain entries: imap, smtp, webmail, etc. >> - Automatic generation of a DKIM keys >> - Automatic update of specific records (MX, SPF, DKIM, etc.) >> - LDAP server for user accounts, with or without system login. >> - Installation of Postfix, Dovecot and Roundcube >> >> Sending DKIM signed emails is working, and the IMAP server is configured >> as well, although basic. >> >> The postfix and dovecot configuration are not yet entirely finished. I >> am planing to add an anti spam system, and sieve, amongst other things. >> >> Although in development during my spare time, the system is normally >> robust and you should be able to run it multiple times without errors. >> >> If anyone is interested to use it, to have a look, or to take part, it >> is here: https://github.com/progmaticltd/homebox >> >> Kind regards, >> André Rodier. >>
Re: hosting emails at home
Hello Mike, Yes, I am using a static IP address, but in theory, you could use a dynamic one. Nothing technically would prevent email exchanges between two boxes, as long as the SPF records are up to date and the DKIM is properly setup. Unfortunately, some ISPs are simply blacklisting full range of private IP addresses just because they are not officials / commercials. Kind regards, André On 10/12/17 19:24, Mike wrote: > > Nice. > > Are you using a static IP in this setup? > > It doesn't seem like it, but wanted to be sure. > > Mike. > > Quoting André Rodier : > >> Hello everyone, >> >> I have been using Postfix and Dovecot for my personal emails for years. >> After being tired of reinstalling my personal mail server many times, I >> am currently writing some Ansible scripts to do it automatically. >> >> I obviously checked the other projects, and did not found anything close >> to what I am looking for, so I am implementing it now. >> >> The final goal is to have a box that once online, would setup itself, by >> creating the certificates, the DKIM keys and update the appropriate DNS >> records. >> >> This is so far what I have achieved: >> - Automatic generation of certificates using LetsEncrypt >> - Automatic update of the domain entries: imap, smtp, webmail, etc. >> - Automatic generation of a DKIM keys >> - Automatic update of specific records (MX, SPF, DKIM, etc.) >> - LDAP server for user accounts, with or without system login. >> - Installation of Postfix, Dovecot and Roundcube >> >> Sending DKIM signed emails is working, and the IMAP server is configured >> as well, although basic. >> >> The postfix and dovecot configuration are not yet entirely finished. I >> am planing to add an anti spam system, and sieve, amongst other things. >> >> Although in development during my spare time, the system is normally >> robust and you should be able to run it multiple times without errors. >> >> If anyone is interested to use it, to have a look, or to take part, it >> is here: https://github.com/progmaticltd/homebox >> >> Kind regards, >> André Rodier. > >
Re: hosting emails at home
Thank you, I remember to had a look at this project, and I found it huge. I started mine because I want LDAP authentication. I also wanted less features / programs, less obtrusive, and better attention to small details, like automatic DKIM generation and DNS updates. I hope not to end up with something as huge. André On 10/12/17 19:19, br...@secryption.com wrote: > Check out https://github.com/sovereign/sovereign/blob/master/README.md > > > Might have some of what you are looking for already done. > > > Bruce > > > On Dec 10, 2017 2:06 PM, André Rodier wrote: > > Hello everyone, > > I have been using Postfix and Dovecot for my personal emails for years. > After being tired of reinstalling my personal mail server many times, I > am currently writing some Ansible scripts to do it automatically. > > I obviously checked the other projects, and did not found anything close > to what I am looking for, so I am implementing it now. > > The final goal is to have a box that once online, would setup itself, by > creating the certificates, the DKIM keys and update the appropriate DNS > records. > > This is so far what I have achieved: > - Automatic generation of certificates using LetsEncrypt > - Automatic update of the domain entries: imap, smtp, webmail, etc. > - Automatic generation of a DKIM keys > - Automatic update of specific records (MX, SPF, DKIM, etc.) > - LDAP server for user accounts, with or without system login. > - Installation of Postfix, Dovecot and Roundcube > > Sending DKIM signed emails is working, and the IMAP server is configured > as well, although basic. > > The postfix and dovecot configuration are not yet entirely finished. I > am planing to add an anti spam system, and sieve, amongst other things. > > Although in development during my spare time, the system is normally > robust and you should be able to run it multiple times without errors. > > If anyone is interested to use it, to have a look, or to take part, it > is here: https://github.com/progmaticltd/homebox > > Kind regards, > André Rodier. >
hosting emails at home
Hello everyone, I have been using Postfix and Dovecot for my personal emails for years. After being tired of reinstalling my personal mail server many times, I am currently writing some Ansible scripts to do it automatically. I obviously checked the other projects, and did not found anything close to what I am looking for, so I am implementing it now. The final goal is to have a box that once online, would setup itself, by creating the certificates, the DKIM keys and update the appropriate DNS records. This is so far what I have achieved: - Automatic generation of certificates using LetsEncrypt - Automatic update of the domain entries: imap, smtp, webmail, etc. - Automatic generation of a DKIM keys - Automatic update of specific records (MX, SPF, DKIM, etc.) - LDAP server for user accounts, with or without system login. - Installation of Postfix, Dovecot and Roundcube Sending DKIM signed emails is working, and the IMAP server is configured as well, although basic. The postfix and dovecot configuration are not yet entirely finished. I am planing to add an anti spam system, and sieve, amongst other things. Although in development during my spare time, the system is normally robust and you should be able to run it multiple times without errors. If anyone is interested to use it, to have a look, or to take part, it is here: https://github.com/progmaticltd/homebox Kind regards, André Rodier.
Re: http://xi.rename-it.nl 2.2.16-1~auto+10/11/12/13: segfault /var/lib/dovecot/auth OR (db_ldap_connect_delayed):
Hi, I mailed Timo S. about this problem this morning. He fixed it minutes later. Thanks again! :-) http://hg.dovecot.org/dovecot-2.2/rev/870cb73e5960 André Am 17.03.2015 um 15:13 schrieb Denny Fuchs: hi, I testing around with Kolab with LDAP and have some strange problems, with the auto generated packages for Wheezy. On my first test VM, I have 2:2.2.16~rc1-1~auto+4 installed, and everything works as expected. On a new VM I have 2.2.16-1~auto+13_amd64.deb packages and if I include LDAP backend, I get a segmentation fault on /usr/lib/dovecot/auth: root@kolab:/etc/dovecot/conf.d# /usr/lib/dovecot/auth Segmentation fault with auto+12 and auto+13 If I downgrade to auto+11 or auto+10 I get: Mar 17 14:42:59 kolab dovecot: auth: Panic: file db-ldap.c: line 1228 (db_ldap_connect_delayed): assertion failed: (conn->to == NULL) Mar 17 14:42:59 kolab dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7b5ef) [0x7f57d815c5ef] -> /usr/lib/dovecot/libdovecot.so.0(+0x7b64e) [0x7f57d815c64e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f57d810b0d5] -> /usr/lib/dovecot/modules/auth/libauthdb_ldap.so(+0x5455) [0x7f57d70e2455] -> dovecot/auth(userdb_init+0x1a) [0x42765a] -> dovecot/auth(auths_init+0xc9) [0x40d979] -> dovecot/auth(main+0x2b5) [0x40cc85] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f57d7523ead] -> dovecot/auth() [0x40cee9] Mar 17 14:42:59 kolab dovecot: auth: Fatal: master: service(auth): child 1161 killed with signal 6 (core dumps disabled) Mar 17 14:42:59 kolab dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs Mar 17 14:42:59 kolab dovecot: pop3-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=::1, lip=::1, secured, session=<1QMUJXwRTQAB> ii dovecot-sieve 2:2.2.16~rc1-1~auto+4 amd64 secure POP3/IMAP server - Sieve filters support root@kolab:/# ldd /usr/lib/dovecot/modules/auth/libauthdb_ldap.so linux-vdso.so.1 => (0x7ffc4a1c6000) libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x7effe2d1a000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x7effe2b12000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7effe2786000) liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x7effe2577000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x7effe2361000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x7effe2145000) libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x7effe1e85000) libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x7effe1c06000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x7effe19e9000) /lib64/ld-linux-x86-64.so.2 (0x7effe318) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7effe17e5000) libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x7effe15d3000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7effe13bc000) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x7effe11aa000) libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x7effe0fa6000) Then I copied from my first VM the libauthdb_ldap.so file on the new test VM to /usr/lib/dovecot/modules/auth/libauthdb_ldap.so (remember: ~rc1-1~auto+4) and the result: root@kolab:/etc/dovecot/conf.d# /usr/lib/dovecot/auth Error: LDAP: binding failed (dn cn=Directory Manager): Invalid credentials and, it works again ... linux-vdso.so.1 => (0x7fffa00c6000) libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x7f94b7ec3000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x7f94b7cbb000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f94b792f000) liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x7f94b772) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x7f94b750a000) libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x7f94b72ee000) libgnutls.so.26 => /usr/lib/x86_64-linux-gnu/libgnutls.so.26 (0x7f94b702e000) libgcrypt.so.11 => /lib/x86_64-linux-gnu/libgcrypt.so.11 (0x7f94b6daf000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f94b6b92000) /lib64/ld-linux-x86-64.so.2 (0x7f94b8328000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x7f94b698e000) libtasn1.so.3 => /usr/lib/x86_64-linux-gnu/libtasn1.so.3 (0x7f94b677c000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7f94b6565000) libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0
Re: script-login help
Hi, I just want to remind you of this plugin: http://wiki2.dovecot.org/Plugins/LastLogin - maybe an option? André Am 14.02.2015 um 00:49 schrieb Jorge Bastos: Hi, I'm trying to execute a bash script to update the last login, IP, and protocol on the user's record, but I'm not being able 'cause I think this is only for 2.2.24+ http://dovecot.org/pipermail/dovecot/2014-January/094610.html I'm on 2.2.23, any change/example to make it work here? Thanks in advanced, Jorge Bastos smime.p7s Description: S/MIME Cryptographic Signature
Re: examples of shared shared-mailboxes.db
Hi Kevin,
You don't put anything into it by yourself.
This file only helps Dovecot to lookup shares to prevent iterating
through every users mailbox.
When you create a share, Dovecot places a plain-text entry inside this
file. When a user logs in, Dovecot asks its acl "database" in whose mail
directory it should look for a share.
An entry could look like this:
shared/shared-boxes/user/shared-for-u...@domain.tld/shared...@domain.tld
You don't have to put a .db ending to that file. You could name it
shares.txt, too. Important: "file:".
André
Am 13.02.2015 um 08:47 schrieb Kevin Laurie:
Dear Andre,
Thanks for yr reply.
What do i include in the file shared-mailboxes.db ?
As in the input. Is it really plain text ?
So for example i write into the file shared-mailbox.db :-
sha...@domain.net
shar...@domain.net etc...
Is it like that or what kind of syntax needs to be used?
On Fri, Feb 13, 2015 at 12:45 PM, André Peters
wrote:
Hi,
that looks correct. I am not sure if this was necessary, but I "touched"
the file before starting Dovecot...
You also need to have the correct permissions set.
André
Am 13.02.2015 um 04:59 schrieb Kevin Laurie:
Hello,
I need help. i am trying to configure shared in boxes.
I need to create a dictionary for shared inbox.
how does 1 write shared-mailboxes.db dictionary?
any example for the syntax/config?
thanks
kevin
Below is part of the config i am trying to achieve:-
plugin {
acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
}
smime.p7s
Description: S/MIME Cryptographic Signature
Re: examples of shared shared-mailboxes.db
Hi,
that looks correct. I am not sure if this was necessary, but I "touched"
the file before starting Dovecot...
You also need to have the correct permissions set.
André
Am 13.02.2015 um 04:59 schrieb Kevin Laurie:
Hello,
I need help. i am trying to configure shared in boxes.
I need to create a dictionary for shared inbox.
how does 1 write shared-mailboxes.db dictionary?
any example for the syntax/config?
thanks
kevin
Below is part of the config i am trying to achieve:-
plugin {
acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
}
smime.p7s
Description: S/MIME Cryptographic Signature
Re: Controlling inactivity timeout for IMAP
I was following this in hope someone would answer.
As a workaround I recommend to set up a POP3 connection with a low
polling interval (besides using a TCP Proxy...).
André
Am 07.02.2015 um 07:15 schrieb Joseph Tam:
I have a problem with a user who uses a wireless carrier that keeps
changing his IP as he travels throughout the city. From the perspective
of our dovecot IMAP server, the user keeps logging in from another IP,
and after a short while, hits up against the mail_max_userip_connections
limit. It takes 30 minutes before those orphaned connections times out.
Is there any way to decrease the IMAP idle timeout other than to
recompile dovecot with a new value?
imap-common.h:#define CLIENT_IDLE_TIMEOUT_MSECS (60*30*1000)
For example, will this work?
service imap {
idle_kill = 600
}
Joseph Tam
smime.p7s
Description: S/MIME Cryptographic Signature
Block folder names in IMAP
Hi, I just merged to Maildir "/" as separator with a fs layout. So I am afraid of running into problems when someone creates a folder named "new", "cur", etc. ... I know I could work this around, but isn't there a way to block folders by their name from being created? Thank you! -- André Peters smime.p7s Description: S/MIME Cryptographic Signature
AW: Resubmission after N days
Hi, Sounds to me like it can be implemented using sieve. Something like "if is in inbox & is read & is older than n days; then mark as unread". But this means you need to redeliever this mail somehow. Maybe like this: http://serverfault.com/questions/418681/run-sieve-on-maildir (http://serverfault.com/questions/418681/run-sieve-on-maildir) Regards André Am Mi., Febr. 4, 2015 07:02 schrieb Thomas Güttler : Hi, I would like to implement a 43Folder system[1] with dovecot and a mail user agent. Use case: - I have a new mail in my inbox. I read it and see that I can't handle it now. I want to handle this mail in 5 days. - Now I want to have some sort of resubmission: the mail should be moved to a different location for these 5 days. - After 5 days the mail should be moved to my inbox again. How can this be done with Dovecot/IMAP? I have programming experience, but I am new to Dovecot/IMAP. I would like solve this with Dovecot only (without a database or web server). Does the IMAP protocol over methods which could help? [1] 43Folder https://en.wikipedia.org/wiki/Tickler_file (https://en.wikipedia.org/wiki/Tickler_file) -- http://www.thomas-guettler.de/ (http://www.thomas-guettler.de/)
Important emails
Hi, Sorry if I am not posting on the right mailing list, but I need email and especially IMAP experts opinion. I have developed a small perl script for Amavis that tries to recognise business emails from bulk & mailing lists, in order to promote 'important' emails, ala google. This is a feature I found very useful in corporate environments, where a lot of users are receiving advertisements, but not consider it as spam. (For info, the code is here: https://github.com/arodier/emclass) My script add an email header (X-Email-Type), and dovecot uses a virtual folder called 'Important' to present them. I know there is IMAP labels, and they are server side stored. I have successfully played with Thunderbird, but IMHO, it's not very reliable. The name of the label is the responsibility of the email client. As a result, if one client decided that $Label1 is 'Important', and $Label2 is 'Business', other clients should be configured exactly the same. I wonder if there is some norm - or convention - to attach labels to email headers, or if such a header exists, for instance a header called e.g. 'X-Tags', 'X-Keywords' or 'X-Labels'. So, once stored, an email could contains - for instance - a header like this: X-Tags: Travel,Advertisement,France. Then, it would be easy to have server side search for labels. Thanks for your feedback. Kind regards, André.
Re: [Dovecot] Dovecot Solr Panic
Thanks Robert, I was not aware of this distribution. ATB, André On 29/04/13 10:21, Robert Schetterer wrote: Am 29.04.2013 11:08, schrieb Andre Rodier: Sorry, I checked my dovecot version, and it has not been updated by Debian yet. I will check if I can find a method to update without breaking dependancies. André perhaps look at http://wiki.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages leads to rep http://xi.rename-it.nl/debian/ Best Regards MfG Robert Schetterer
Re: [Dovecot] Problem when indexing maildir with a large number of folders
Hello Allessio, I am using readpst -r from libpst (http://www.five-ten-sg.com/libpst/) André On 27 March 2013 13:25, Alessio Cecchi wrote: > Il 27/03/2013 00:20, Andre Rodier ha scritto: > >> Hello everybody, >> >> As part of a migration from exchange 2003 to dovecot, I have converted a >> lot of PST files for a user. I am importing his PST files in a maildir >> folder called >> 'Archives'. Actually, I reached about 770 'Archives' folders, and the >> total length of their names is about 59k characters. Thanks to wc! >> >> I am using Maildir++ > > > Hi, > > what kind of method are you using for converting PST into Maildir? > > Thanks > -- > Alessio Cecchi is: > @ ILS -> http://www.linux.it/~alessice/ > on LinkedIn -> http://www.linkedin.com/in/alessice > Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ > @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it -- André Rodier
Re: [Dovecot] Zlib plugin: Compress emails for one folder only
Thanks Timo, I will see. If I have some time, I may have a look at the source code. Kind regards, André. On 20 March 2013 17:58, Timo Sirainen wrote: > On 16.3.2013, at 14.43, Andre Rodier wrote: > >> Hello everyone, >> >> Is it possible to configure the zlib plugin to compress the emails in one >> folder only? >> >> I would like to compress the emails when they are stored in my 'Archives' >> folders and sub folders. > > Nope. With hardcoding it would be a simple one line of code addition. > > Some day in future there will be namespace-specific settings. Then you could > create Archive namespace and enable zlib compression only for that namespace. > -- André Rodier
Re: [Dovecot] Panic when indexing virtual folder with solr
Thank you, I will try this soon ! Kind regards, André Rodier. On 14 March 2013 13:07, tmikaeld wrote: > This happened to me too, i just downloaded the source for Dovecot 2.1.15 and > used the Solr XML scheme from there instead of the one supplied with Ubuntu. > > > > -- > View this message in context: > http://dovecot.2317879.n4.nabble.com/Panic-when-indexing-virtual-folder-with-solr-tp40764p40779.html > Sent from the Dovecot mailing list archive at Nabble.com. -- André Rodier
Re: [Dovecot] virtual folder - crash while searching
Hello everyone, I had the same errors on my server, and I fixed it by increasing the header size buffer of my server to 65535, for instance. For jetty, the option is named headerBufferSize. See: 3 2 false 8443 65536 5000 5000 On 3 August 2012 15:14, Timo Sirainen wrote: > On 1.8.2012, at 23.23, Matthias Rieber wrote: > >> alright. Some resources say that "413 FULL head" means request too large. >> Maybe that's caused by the pretty big folder virtual.all which contains more >> than 36 mails. > Probably it means that the virtual folder consists of too many real folders. > Dovecot's Solr query adds each real folder's GUID to the query. I guess there > should be some limit and it would have to send more than one query and merge > the results. > No need to do this. -- André Rodier
Re: [Dovecot] panic in dovecot when using fts with solr
Ok,
I found the issue.
The solr schema xml file was wrong, and I did not check the error logs
correctly. So, the error was in solr:
04-Mar-2013 15:15:30 org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: ERROR:unknown field 'uid'
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:289)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:60)
at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:139)
at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:69)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:54)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1317)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:843)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
04-Mar-2013 15:15:30 org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update params={} status=400 QTime=70
However, IMHO, I don't think that dovecot handle the errors correctly.
It should return an error message.
Kind regards,
André Rodier
On 4 March 2013 10:01, André Rodier wrote:
> This is my doveconf output. Please, note that since the panic, I have
> switched to fts with squat, but it is very limited for large
> mailboxes.
>
> Any idea how to fix this?
>
> # 2.1.7: /etc/dovecot/dovecot.conf
> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.0
> auth_default_realm = indiefield.co.uk
> auth_master_user_separator = :
> auth_mechanisms = plain login
> auth_username_chars =
> abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890 .-_@
> auth_username_translation = "- _ "
> auth_verbose = yes
> disable_plaintext_auth = no
> dotlock_use_excl = no
> hostname = smtp.indiefield.org.uk
> lda_mailbox_autocreate = yes
> lda_mailbox_autosubscribe = yes
> lock_method = dotlock
> mail_access_groups = vmail
> mail_fsync = always
> mail_gid = vmail
> mail_location = maildir:~/Mails/Maildir/:INDEX=~/Mails/Indexes/
> mail_plugins = autocreate quota acl virtual listescape fts fts_squat
> mail_privileged_group = vmail
> mail_uid = vmail
> maildir_copy_with_hardlinks = no
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope
> encoded-character vacation subaddress comparator-i;ascii-numeric
> relational regex imap4flags copy include variables body enotify
> environment mailbox date ihave
> mmap_disable = yes
> namespace {
> hidden = no
> inbox = no
> list = yes
> location =
> maildir:/var/vmail/%d/Public/Mails/Maildir/:INDEX=~/Mails/PublicIndexes/
> prefix = Public/
> separator = /
> subscriptions = yes
> type = public
> }
> namespace {
> list = children
> location = maildir:%%h/Mails/Maildir/:INDEX=~/Mails/SharedIndexes/%%u
> prefix = Shared/%%d/%%n/
> separator = /
> subscriptions = no
> type = shared
> }
> namespace inbox {
> hidden = no
> inbox = yes
> list = yes
> location = maildir:~/Mails/Maildir/:INDEX=~/Mails/Indexes/
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
>
Re: [Dovecot] panic in dovecot when using fts with solr
This is my doveconf output. Please, note that since the panic, I have
switched to fts with squat, but it is very limited for large
mailboxes.
Any idea how to fix this?
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.0
auth_default_realm = indiefield.co.uk
auth_master_user_separator = :
auth_mechanisms = plain login
auth_username_chars =
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890 .-_@
auth_username_translation = "- _ "
auth_verbose = yes
disable_plaintext_auth = no
dotlock_use_excl = no
hostname = smtp.indiefield.org.uk
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
lock_method = dotlock
mail_access_groups = vmail
mail_fsync = always
mail_gid = vmail
mail_location = maildir:~/Mails/Maildir/:INDEX=~/Mails/Indexes/
mail_plugins = autocreate quota acl virtual listescape fts fts_squat
mail_privileged_group = vmail
mail_uid = vmail
maildir_copy_with_hardlinks = no
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
mmap_disable = yes
namespace {
hidden = no
inbox = no
list = yes
location =
maildir:/var/vmail/%d/Public/Mails/Maildir/:INDEX=~/Mails/PublicIndexes/
prefix = Public/
separator = /
subscriptions = yes
type = public
}
namespace {
list = children
location = maildir:%%h/Mails/Maildir/:INDEX=~/Mails/SharedIndexes/%%u
prefix = Shared/%%d/%%n/
separator = /
subscriptions = no
type = shared
}
namespace inbox {
hidden = no
inbox = yes
list = yes
location = maildir:~/Mails/Maildir/:INDEX=~/Mails/Indexes/
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
separator = /
subscriptions = yes
type = private
}
passdb {
args = /etc/dovecot/dovecot-ldap.conf.ext
driver = ldap
}
plugin {
acl = vfile:/etc/dovecot/global-acls:cache_secs=300
acl_anyone = allow
acl_shared_dict = file:/var/vmail/indiefield.co.uk/shared-mailboxes
autocreate = Trash
autocreate2 = Drafts
autocreate3 = Sent
autocreate4 = Junk
autocreate5 = Archives
autosubscribe = Trash
autosubscribe2 = Drafts
autosubscribe3 = Sent
autosubscribe4 = Junk
autosubscribe5 = Archives
fts = squat
fts_squat = partial=4 full=10
listescape_char = %%
quota = maildir:User quota
quota_warning = storage=95%% quota-warning 95 %u
sieve = ~/Mails/.dovecot.sieve
sieve_after = %h/Mails/Sieve/After/
sieve_before = /etc/dovecot/sieve-default/before/
sieve_dir = ~/Mails/Sieve
sieve_max_script_size = 4M
}
protocols = " imap lmtp sieve pop3"
service auth-worker {
user = vmail
}
service auth {
unix_listener /var/spool/postfix/private/auth {
group = vmail
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0666
user = postfix
}
user = vmail
}
service dict {
unix_listener dict {
group = vmail
mode = 0600
user = vmail
}
}
service director {
fifo_listener login/proxy-notify {
group = postfix
mode = 0666
user = postfix
}
unix_listener director-userdb {
group = postfix
mode = 0600
user = postfix
}
unix_listener login/director {
group = postfix
mode = 0666
user = postfix
}
}
service lmtp {
group = vmail
inet_listener lmtp {
port = 24
}
user = vmail
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
}
service quota-warning {
executable = script /etc/dovecot/scripts/quota-warning.sh
unix_listener quota-warning {
user = vmail
}
user = dovecot
}
ssl_cert =
[Dovecot] userdb errors after upgrading to 2.1
Hello everybody, I am running debian wheezy for development and test, and I recently upgrade to dovecot 2.1.7 I am using LDAP lookups, and virtual users with the same UID/GID. Everything was working fine before, but now, I have this error when I try to send an email to a local account: Jun 26 15:46:52 lapetus dovecot: lmtp(24518): Error: user user.t...@indienet.com: Auth USER lookup failed Jun 26 15:46:52 lapetus dovecot: auth: Error: userdb(user.t...@indienet.com,127.0.0.1): client doesn't have lookup permissions for this user: userdb reply doesn't contain uid (change userdb socket However, even if I set the permissions to 0666, I still have the same error. Can you point me in the right direction to fix this, please? Kind regards, André Rodier
Re: [Dovecot] How to grant a kerberos ticket after successful imap authentication from dovecot
On 31/03/2011 15:37, Timo Sirainen wrote: On 31.3.2011, at 17.32, André Rodier wrote: Thanks, Timo. So, other questions: * Can I use a post login script to try to initialise the kerberos ticket ? With v1.x yes, with v2.x no (because in v2.x it's again in a separate process to allow support for multiple clients per process). * Can I write a dovecot plugin in C/C++ to do that, and in this case ? Yes. * If I use a plugin or a script, do I have access to the username / password ? Username yes, password no. I guess you could modify Dovecot code so PAM code saves the password and passes it to mail process. * If I use a plugin, where can I found a skeleton ? v1.x or v2.x? v1.x is really simple, v2.x needs more work. How are mails delivered then anyway? Doesn't that process also need some kerberos ticket? Hello Timo, You were right. Since I have switched to MFSv4/Kerberos, I started by testing the mail access before the delivery. Testing just now the dovecot deliver script fail as well... I probably have to use another method to obtain the ticket. I can also try to use a virtual user for the whole mail storage... If I found a solution, I'll post it on this list. I use dovecot 1.2, included by default on Debian squeeze. Kind regards. André Rodier.
