Re: Dovecot HSM
you can also create an archive namespace, and put that on your cheap storage. On Oct 30, 2019 22:09, Júlio Covolato via dovecot wrote: Hi. I'm looking for a tutorial/how-to for a HSM (Hierarchical Storage Management). keeping old messages for a user in a cheap storage and recent messages in a faster one. I see on dovecot2 wiki an alternative for hsm as "Alternate storage", but I don't now if it's a good solution for me. The expected result is a faster imap/pop access for new messages on a "heavy mail users" mailbox, and obviously low cost! Thanks in advance. -- -- _Engº Julio Cesar Covolato 0v0 /(_)\ F: +55 11 99175-9260 ^ ^ PSI INTERNET --
Re: Dovecot and MySQL aborted connections.
> We also spotted these sql connections getting aborted, upon upgrading MySQL > from 5.6 to 5.7. (Going back to 5.6 we don't see them!) I have a mailserver running MySQL 5.6.44 which is not very busy that logs these warnings. Another busy one running MySQL 5.6.45 does not log any warnings. MySQL interactive-/wait_timeout is 28800 (8 hours), which is more than enough time to wait for a query. Connections are closed way earlier. dovecot and postfix use the same database and mysql user, so I created a new user for dovecot. syslog shows a few communication packet errors but these are all from postfix, none from dovecot so far. It seems connections are closed after 60s of inactivity (vmail is postfix, vmail2 is dovecot): mysql> select * from INFORMATION_SCHEMA.PROCESSLIST where USER like 'vmail%'; +-++-+---+-+--+---+--+-+---+---+--+ | ID | USER | HOST| DB| COMMAND | TIME | STATE | INFO | TIME_MS | ROWS_SENT | ROWS_EXAMINED | TID | +-++-+---+-+--+---+--+-+---+---+--+ | 147 | vmail | localhost | vmail | Sleep |1 | | NULL | 566 | 0 | 0 | 6664 | | 148 | vmail | localhost | vmail | Sleep |1 | | NULL | 564 | 0 | 0 | 6069 | | 149 | vmail | localhost | vmail | Sleep |1 | | NULL | 566 | 0 | 0 | 6037 | | 151 | vmail2 | 127.0.0.2:35058 | vmail | Sleep | 51 | | NULL | 50872 | 1 | 1 | 6072 | | 152 | vmail2 | 127.0.0.2:35060 | vmail | Sleep | 60 | | NULL | 59324 | 1 | 1 | 6025 | | 150 | vmail | localhost | vmail | Sleep |1 | | NULL | 565 | 0 | 0 | 6071 | +-++-+---+-+--+---+--+-+---+---+--+ 6 rows in set (0.00 sec) mysql> select * from INFORMATION_SCHEMA.PROCESSLIST where USER like 'vmail%'; +-++-+---+-+--+---+--+-+---+---+--+ | ID | USER | HOST| DB| COMMAND | TIME | STATE | INFO | TIME_MS | ROWS_SENT | ROWS_EXAMINED | TID | +-++-+---+-+--+---+--+-+---+---+--+ | 147 | vmail | localhost | vmail | Sleep |2 | | NULL | 1775 | 0 | 0 | 6664 | | 148 | vmail | localhost | vmail | Sleep |2 | | NULL | 1772 | 0 | 0 | 6069 | | 149 | vmail | localhost | vmail | Sleep |2 | | NULL | 1774 | 0 | 0 | 6037 | | 151 | vmail2 | 127.0.0.2:35058 | vmail | Sleep | 52 | | NULL | 52080 | 1 | 1 | 6072 | | 150 | vmail | localhost | vmail | Sleep |2 | | NULL | 1773 | 0 | 0 | 6071 | +-++-+---+-+--+---+--+-+---+---+--+ 5 rows in set (0.00 sec) Only vmail (postfix) has dropped connections (you need performance schema enabled): mysql> SELECT ess.user, ess.host, (a.total_connections - a.current_connections) - ess.count_star as not_closed, ((a.total_connections - a.current_connections) - ess.count_star) * 100 / (a.total_connections - a.current_connections) as pct_not_closed FROM performance_schema.events_statements_summary_by_account_by_event_name ess JOIN performance_schema.accounts a on (ess.user = a.user and ess.host = a.host) WHERE ess.event_name = 'statement/com/quit' AND (a.total_connections - a.current_connections) > ess.count_star; +---+---+++ | user | host | not_closed | pct_not_closed | +---+---+++ | vmail | localhost | 11 |15.4930 | +---+---+++ 1 row in set (0.02 sec) My setup is different concerning quotas, they are not stored in MySQL. So if you don't use postfix with the same user/database the source for your warnings might be quota. For me it seems to be related to postfix. Don't know if this helps, there is a new option in 5.7 https://dev.mysql.com/doc/refman/5.7/en/x-plugin-options-system-variables.html#sysvar_mysqlx_idle_worker_thread_timeout Gerald > Turning on mysql general query logging we can see it is Dovecot's mysql > connections that inquire about or update quota usage in particular: > > > *** /logs//mysql.log *** > 2019-10-30T10:52:22.624690-07:00 2 Connect dovecot@localhost on > npomail using Socket > > 2019-10-30T10:52:40.019780-07:00 2 Query SELECT bytes FROM > quota2 WHERE username = 'a@bla' > 2019-10-30T10:52:40.020948-07:00 2 Query SELECT mes
Dovecot HSM
Hi. I'm looking for a tutorial/how-to for a HSM (Hierarchical /Storage/ Management). keeping old messages for a user in a cheap storage and recent messages in a faster one. I see on dovecot2 wiki an alternative for hsm as "Alternate storage", but I don't now if it's a good solution for me. The expected result is a faster imap/pop access for new messages on a "heavy mail users" mailbox, and obviously low cost! Thanks in advance. -- -- _Engº Julio Cesar Covolato 0v0 /(_)\ F: +55 11 99175-9260 ^ ^ PSI INTERNET --
Dovecot and MySQL aborted connections.
We also spotted these sql connections getting aborted, upon upgrading MySQL from 5.6 to 5.7. (Going back to 5.6 we don't see them!) Turning on mysql general query logging we can see it is Dovecot's mysql connections that inquire about or update quota usage in particular: *** /logs//mysql.log *** 2019-10-30T10:52:22.624690-07:00 2 Connect dovecot@localhost on npomail using Socket 2019-10-30T10:52:40.019780-07:00 2 Query SELECT bytes FROM quota2 WHERE username = 'a@bla' 2019-10-30T10:52:40.020948-07:00 2 Query SELECT messages FROM quota2 WHERE username = 'a@bla' 2019-10-30T10:53:40.113374-07:00 2 [Note] Aborted connection 2 to db: 'npomail' user: 'dovecot' host: 'localhost' (Got an error reading communication packets) Increasing mysqld's net_read_timeout and net_write_timeout values to 3600 does not stop the aborted connections. It seems asymptomatic. If the connection to mysql is still there to do quota operations, Dovecot seems to use it. If it's been aborted by the server, Dovecot seems to just (re)connect. . . But I'd love to help find the root or file a bug report or learn what it is we've configured wrong. . . Benjamin
Re: encrypt incoming emails with public gpg key before they are stored to maildir
On 30/10/2019 15:29, Fourhundred Thecat via dovecot wrote: Hello, I have asked on the postfix mailing list for a solution, how to encrypt incoming emails with public gpg key My original idea was to use a smtpd-milter, which would encrypt all incoming plaintext messages of given user, using the users public gpg key. This way, it would look as if the original sender has sent the message encrypted. Somebody suggested this might be better done in Dovecot, as it is Dovecot who stores the emails in Maildir. Does anybody have any suggestions how to implement this scheme in Dovecot ? Something like this? https://perot.me/encrypt-specific-incoming-emails-using-dovecot-and-sieve Regards, Stephan. I am including the original email below: when new email arrives, and it is not already encrypted, I would like to run it through a filter, which would encrypt the message with my public gpg key, as if the original sender has sent the email encrypted. Why do I want to do this ? Why not ask the sender to send encrypted messages to start with ? Lets say my bank sends me emails. I cannot force my bank to use gpg encryption. I am happy they use email at all, instead of paper mail. My email server is untrusted. It can be hacked into and emails stolen. Full disk encryption will not help, because the disk must be decrypted during runtime. With my scheme, all emails would be stored encrypted on my server, and decryption key does not exist on the server (emails are decrypted on my local client) What would be the best way to implement this ? Can such filter work, without ever storing plaintext email on disk ? thanks,
encrypt incoming emails with public gpg key before they are stored to maildir
Hello, I have asked on the postfix mailing list for a solution, how to encrypt incoming emails with public gpg key My original idea was to use a smtpd-milter, which would encrypt all incoming plaintext messages of given user, using the users public gpg key. This way, it would look as if the original sender has sent the message encrypted. Somebody suggested this might be better done in Dovecot, as it is Dovecot who stores the emails in Maildir. Does anybody have any suggestions how to implement this scheme in Dovecot ? I am including the original email below: > when new email arrives, and it is not already encrypted, I would like to > run it through a filter, which would encrypt the message with my public > gpg key, as if the original sender has sent the email encrypted. > > Why do I want to do this ? Why not ask the sender to send encrypted > messages to start with ? > > Lets say my bank sends me emails. I cannot force my bank to use gpg > encryption. I am happy they use email at all, instead of paper mail. > > My email server is untrusted. It can be hacked into and emails stolen. > Full disk encryption will not help, because the disk must be decrypted > during runtime. > > With my scheme, all emails would be stored encrypted on my server, and > decryption key does not exist on the server (emails are decrypted on my > local client) > > What would be the best way to implement this ? > > Can such filter work, without ever storing plaintext email on disk ? thanks,
