Re: Random behavior with sieve extprograms
> "Thomas" == Thomas Sommer writes:
Thomas> I have a random behavior with dovecot and sieve extprograms.
Thomas> Here is my sieve file:
Thomas> require ["fileinto", "vnd.dovecot.pipe", "copy", "imap4flags"];
Thomas> # rule:[DABS]
Thomas> if header :contains "X-Original-To" "d...@mydomain.ch"
Thomas> {
Thomas> pipe "sieve-dabs-execute.sh";
Thomas> setflag "\\Seen";
Thomas> fileinto "acme.DABS";
Thomas> stop;
Thomas> }
Can you post the code of this script? Are you trapping all exceptions
in that script and making sure you only return errors when there
really is an error?
Thomas> Emails matching the condition are processed by a laravel (php) artisan
Thomas> command. See service sieve-pipe-script below.
Thomas> The exit code of this php command is 0.
You are calling the php command from a shell script, so there's
multiple places things could go wrong. Why not just pipe directly to
the php script (which wasn't included unless I'm totally blind and
dumb tonight... :-) instead?
It honestly sounds like a timing issue, maybe just putting a sleep
into your shell script at the end would be good? Or maybe run with
the -vx switches so you log all the commands and their results?
Thomas> I randomly get the following in my postfix logs:
Thomas> Sieve thinks that the command failed, but the email was always
processed
Thomas> correctly. In that case I get a copy in my Inbox.
Thomas> I'm wondering what could be the cause for this random behavior.
Thomas> My guess is that approximately 70% are processed correctly, 30% is as
Thomas> below.
Thomas> May 31 13:50:38 star dovecot[99425]:
Thomas> lda(user)<99425>: sieve:
Thomas> msgid=<62961d1c.5y4hr0vqi97jfnyb%dabs.zsmsv...@example.com>: fileinto
Thomas> action: stored mail into mailbox 'acme.DABS'
Thomas> May 31 13:50:39 star dovecot[99425]:
Thomas> lda(user)<99425>: sieve:
Thomas> msgid=<62961d1c.5y4hr0vqi97jfnyb%dabs.zsmsv...@example.com>: stored
mail
Thomas> into mailbox 'INBOX'
Thomas> May 31 13:50:39 star dovecot[99425]:
Thomas> lda(user)<99425>: sieve: Execution of script
Thomas> /home/user/sieve/.dovecot.sieve failed, but implicit keep was
successful
Thomas> (user logfile /home/user/sieve/.dovecot.sieve.log may reveal additional
Thomas> details)
Thomas> .dovecot.sieve.log:
Thomas> sieve: info: started log at May 31 13:50:39.
Thomas> error: failed to pipe message to program `sieve-dabs-execute.sh': refer
Thomas> to server log for more information. [2022-05-31 13:50:39].
Thomas> It's weird. "failed to pipe message to program" is simply not true. The
Thomas> command was processed correctly.
Thomas> Any ideas where to look for clues or how to debug this?
Thomas> Regards
Thomas> Thomas
Thomas> config:
Thomas> # 2.3.14 (cee3cbc0d): /etc/dovecot/dovecot.conf
Thomas> # Pigeonhole version 0.5.14 (1b5c82b2)
Thomas> # OS: Linux 5.17.5-x86_64-linode154 x86_64 Ubuntu 20.04.4 LTS
Thomas> auth_mechanisms = plain login
Thomas> auth_username_format = %n
Thomas> auth_verbose = yes
Thomas> mail_location = maildir:~/Maildir
Thomas> mail_plugins = " quota"
Thomas> managesieve_notify_capability = mailto
Thomas> managesieve_sieve_capability = fileinto reject envelope
Thomas> encoded-character vacation subaddress comparator-i;ascii-numeric
Thomas> relational regex imap4flags copy include variables body enotify
Thomas> environment mailbox date index ihave duplicate mime foreverypart
Thomas> extracttext vnd.dovecot.pipe vnd.dovecot.execute
Thomas> namespace inbox {
Thomas>inbox = yes
Thomas>location =
Thomas>mailbox Drafts {
Thomas> special_use = \Drafts
Thomas>}
Thomas>mailbox Junk {
Thomas> special_use = \Junk
Thomas>}
Thomas>mailbox Sent {
Thomas> special_use = \Sent
Thomas>}
Thomas>mailbox "Sent Messages" {
Thomas> special_use = \Sent
Thomas>}
Thomas>mailbox Trash {
Thomas> special_use = \Trash
Thomas>}
Thomas>prefix =
Thomas> }
Thomas> passdb {
Thomas>driver = pam
Thomas> }
Thomas> plugin {
Thomas>quota = fs:User quota
Thomas>quota_grace = 1%%
Thomas>quota_status_nouser = DUNNO
Thomas>quota_status_overquota = 552 5.2.2 Mailbox is full
Thomas>quota_status_success = DUNNO
Thomas>sieve = file:~/sieve;active=~/sieve/.dovecot.sieve
Thomas>sieve_execute_socket_dir =
Thomas>sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute
Thomas>sieve_pipe_exec_timeout = 30s
Thomas>sieve_pipe_socket_dir =
Thomas>sieve_plugins = sieve_extprograms
Thomas>sieve_redirect_envelope_from = recipient
Thomas>sieve_trace_debug = no
Thomas>sieve_trace_dir = ~/sieve/trace
Thomas>sieve_trace_level = matching
Thomas> }
Thomas> protocols = imap sieve
Thomas> service auth {
Thomas>unix_listener /var/spool/postfix/private/dovecot-auth {
Thomas> group = postfix
Thomas> mode = 0660
Thomas> user = postfix
Thomas>}
Thomas> }
Thomas> service quota-status {
Thomas>client_limit = 1
Random behavior with sieve extprograms
Hi
I have a random behavior with dovecot and sieve extprograms.
Here is my sieve file:
require ["fileinto", "vnd.dovecot.pipe", "copy", "imap4flags"];
# rule:[DABS]
if header :contains "X-Original-To" "d...@mydomain.ch"
{
pipe "sieve-dabs-execute.sh";
setflag "\\Seen";
fileinto "acme.DABS";
stop;
}
Emails matching the condition are processed by a laravel (php) artisan
command. See service sieve-pipe-script below.
The exit code of this php command is 0.
I randomly get the following in my postfix logs:
Sieve thinks that the command failed, but the email was always processed
correctly. In that case I get a copy in my Inbox.
I'm wondering what could be the cause for this random behavior.
My guess is that approximately 70% are processed correctly, 30% is as
below.
May 31 13:50:38 star dovecot[99425]:
lda(user)<99425>: sieve:
msgid=<62961d1c.5y4hr0vqi97jfnyb%dabs.zsmsv...@example.com>: fileinto
action: stored mail into mailbox 'acme.DABS'
May 31 13:50:39 star dovecot[99425]:
lda(user)<99425>: sieve:
msgid=<62961d1c.5y4hr0vqi97jfnyb%dabs.zsmsv...@example.com>: stored mail
into mailbox 'INBOX'
May 31 13:50:39 star dovecot[99425]:
lda(user)<99425>: sieve: Execution of script
/home/user/sieve/.dovecot.sieve failed, but implicit keep was successful
(user logfile /home/user/sieve/.dovecot.sieve.log may reveal additional
details)
.dovecot.sieve.log:
sieve: info: started log at May 31 13:50:39.
error: failed to pipe message to program `sieve-dabs-execute.sh': refer
to server log for more information. [2022-05-31 13:50:39].
It's weird. "failed to pipe message to program" is simply not true. The
command was processed correctly.
Any ideas where to look for clues or how to debug this?
Regards
Thomas
config:
# 2.3.14 (cee3cbc0d): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.14 (1b5c82b2)
# OS: Linux 5.17.5-x86_64-linode154 x86_64 Ubuntu 20.04.4 LTS
auth_mechanisms = plain login
auth_username_format = %n
auth_verbose = yes
mail_location = maildir:~/Maildir
mail_plugins = " quota"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext vnd.dovecot.pipe vnd.dovecot.execute
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox "Sent Messages" {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
driver = pam
}
plugin {
quota = fs:User quota
quota_grace = 1%%
quota_status_nouser = DUNNO
quota_status_overquota = 552 5.2.2 Mailbox is full
quota_status_success = DUNNO
sieve = file:~/sieve;active=~/sieve/.dovecot.sieve
sieve_execute_socket_dir =
sieve_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute
sieve_pipe_exec_timeout = 30s
sieve_pipe_socket_dir =
sieve_plugins = sieve_extprograms
sieve_redirect_envelope_from = recipient
sieve_trace_debug = no
sieve_trace_dir = ~/sieve/trace
sieve_trace_level = matching
}
protocols = imap sieve
service auth {
unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0660
user = postfix
}
}
service quota-status {
client_limit = 1
executable = /usr/lib/dovecot/quota-status -p postfix
inet_listener {
address = 127.0.0.1
port = 8881
}
}
service sieve-pipe-script {
executable = script /usr/bin/php /srv/www/mydomain/status/artisan
dabs:processEmail
unix_listener sieve-dabs-execute.sh {
mode = 0660
user = user
}
user = www-data
vsz_limit = 512 M
}
ssl = required
ssl_cert = rejection_reason = Your message to <%t> was automatically
rejected:%n%r
}
protocol imap {
mail_max_userip_connections = 20
mail_plugins = " quota mail_log notify imap_quota"
}
doveadm-deduplicate deletes non-duplicates
Hi, I've been trying to use `doveadm deduplicate` to deduplicate mailboxes. According to doveadm-deduplicate(1), "deduplication will be done by message GUIDs". However, deduplication deletes messages with distinct message GUIDs, i.e., it deletes messages that are not duplicates. Is this a case of user error, do I have some form of corruption going on, or am I running into a bug? In case it helps, I'm including: 1) the list of GUIDs for messages in my INBOX before a deduplication run (as documented in doveadm-deduplicate(1)), 2) the output of `doveadm -D deduplicate -u rak mailbox INBOX`, 3) the list of GUIDs after deduplication, 4) a diff of (1) and (3), 5) the output of doveconf -n. Thanks, Ryan $ doveadm -f table fetch -u rak 'guid uid' mailbox INBOX | sort 0b3bee1414118f6282db226807b07100 0b4b681c18168f622094226807b07104 0b83ac1d79cf2962eb3a0100226807b06153 0b97791e83108f6282db226807b07095 1614451513.M180742P77875.hades.rak.ac,S=3516,W=3600 22 1614451513.M180779P77875.hades.rak.ac,S=5252,W=5370 52 1614452870.M315137P88362.hades.rak.ac,S=5516,W=5623 68 1614452870.M315152P88362.hades.rak.ac,S=5977,W=6085 69 23a5a0179e108f6282db226807b07097 33318e0686108f6282db226807b07096 3351581c9c0e8f624cd5226807b07091 3359032360108f6282db226807b07093 3b3927352be48c62b65b226807b07072 3ba2252814118f62a2540100226807b07101 4154be0c2fc77761c255226807b04050 5b3a7327d73b866299cd226807b07013 638e073255235d62a5280100226807b06660 63b6422e14118f6282db226807b07102 8b9b942909118f6282db226807b07099 8bc95639a6168f62ad4e226807b07105 a316ee28980d8f627c46226807b07090 ab802c1e37ac3d6214680100226807b06343 b3d12f21cd108f6282db226807b07098 bb89431b6e728d623092226807b07076 c386310f062adb61d198226807b05306 cb56992484478d62f3090100226807b07074 cb79bf3503128f6275220100226807b07103 eb2818367d108f62484b0100a558518d7094 eb709020e70f8f62292e226807b07092 f19a8c06409a7d61271c226807b04128 f338ad1ff65e8562ff390100226807b07007 f9cd03363c457c613a0e226807b04107 guiduid $ doveadm -D deduplicate -u rak mailbox INBOX Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_flatcurve_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) May 31 12:50:35 Debug: Loading modules from directory: /usr/local/lib/dovecot May 31 12:50:35 Debug: Module loaded: /usr/local/lib/dovecot/lib15_notify_plugin.so May 31 12:50:35 Debug: Module loaded: /usr/local/lib/dovecot/lib20_replication_plugin.so May 31 12:50:35 Debug: Module loaded: /usr/local/lib/dovecot/lib20_virtual_plugin.so May 31 12:50:35 Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm May 31 12:50:35 Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) May 31 12:50:35 Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) May 31 12:50:35 Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) May 31 12:50:35 Debug: Skipping module doveadm_fts_flatcurve_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) May 31 12:50:35 Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: Cannot load specified object (this is usually intentional, so just ignore this message) May 31 12:50:35
iPhone does not show 'read marks'.
The iPhone mail app does not show the read marks.The OS/X mail sets and shows the read marks correctly.Is there a way to fix or at least debug this behaviour ?
