[Dovecot] FTS Plugin design
Hi all, Currently I am developing some changes on the solr plugin. I want this plugin indexing also the attachment's content. I have already started to look on plugin's source but I am having some problems understanding how it works. I didn't understood yet what is the plugin's design and how the plugins are called from the core system and I was wondering if anyone could help me with that. Sorry if this doubts sound stupid but I am newcomer on Dovecot. Regards, Rui Carneiro
Re: [Dovecot] Dovecot+Sieve
Hi All!!!
I have successfully configure my mail server. Dovecot+Postfix+Mysql.
It is work.
In the mail.log I see:
postfix/pipe[19520]: DFCCC640048: to=u...@domain.ru, relay=dovecot,
delay=0.19, delays=0.06/0.07/0/0.07, dsn=2.0.0, status=sent (delivered
via dovecot service)
But the sieve plugin isn't work. :'(
My dovecot.conf:
plugin {
sieve = /home/vmail/domain.ru/u...@domain.ru/dovecot.sieve
}
/In the dovecot.log I see:/
Error:
sieve_execute_bytecode(/home/vmail/domain.ru/u...@domain.ru/dovecot.sievec)
failed
/In the dovecot-info.log I see:/
Info: cmusieve: Using sieve path:
/home/vmail/domain.ru/u...@domain.ru/dovecot.sieve
deliver(u...@domain.ru):
Info: cmusieve: Executing script
/home/vmail/domain.ru/u...@domain.ru/dovecot.sievec
Please Help Me... :'(
Re: [Dovecot] Issue with converting users from cyrus user.domain.com
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, Apr 12, 2009 at 09:47:22AM -0600, Preston Lord wrote: [...] dovecot --version 1.0.15 Hmmm. The newest avaliable packaged version to etch seems to be 1.0.15, that's from backports [1], it seems. The standard etch package is 1.0.rc15 [2] Even Lenny (stable by now) has 1.0.15 as default [3]. If you go to squeeze (aka testing), you'd get 1.1.13 [4]. So even upgrading to lenny (which I would recommend, if there are no reasons holding you back) would't bring you a newer Dovecot version. And upgrading to testing... well, it depends very much on what you are doing. So if you want a newer version on etch or lenny, you'll have to compile it for yourself. With Debian, you have two options: just downloading the original sources and compiling (whicshould be fairly easy with dovecot!) or letting APT do it for you. The advantage of the second approach would be that the package comes already configured in a way that it fits the rest of the system, that the package database knows about the installed package (and would be willing to upgrade it with a later version) and that APT can do for you the legwork of geetting whatever packages needed to actually do the build (the so-called build dependencies). See [5] for some instructions on how to do that. [1] http://packages.debian.org/etch-backports/dovecot-imapd [2] http://packages.debian.org/etch/dovecot-imapd [3] http://packages.debian.org/lenny/dovecot-imapd [4] http://packages.debian.org/squeeze/dovecot-imapd [5] http://www.debian.org/doc/manuals/apt-howto/ch-sourcehandling.en.html Regards - -- tomás -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFJ40F1Bcgs9XrR2kYRAorWAJ0fPNl4aIuZ2REEu8Bj2EX8cSE/BwCfVnz/ VqKHR2PvpaC8+9GFf8lLNH0= =p7HC -END PGP SIGNATURE-
[Dovecot] How can I skip EOH in headers?
Hello.
My callback for get_stream is called from mail_get_stream in
mail_storage_copy. Further, mbox_save_init is called and it called
mbox_save_get_input_stream.
backtrace (A) for my callback:
#0 i_stream_header_filter_read (stream=0x8147e00) at istream-header-
filter.c:314
#1 0x080cdbaa in parse_header (mstream=0x8147e00) at istream-header-
filter.c:356
#2 0x080cdc0c in i_stream_header_filter_seek (stream=0x8147e00,
v_offset=0, mark=false) at istream-header-filter.c:370
#3 0x080dea5b in i_stream_seek (stream=0x8147e28, v_offset=0) at
istream.c:155
#4 0x080dfeae in i_stream_create_concat (input=0x8115528) at istream-
concat.c:260
#5 0x2831e03d in rarules_get_stream (mail=0x8141860, hdr_size=0x0,
body_size=0x0, stream_r=0xbfbfe920)
at /home/pitman/work/rarules/src/rarules-plugin.c:181
#6 0x0809d830 in mail_get_stream (mail=0x8141860, hdr_size=0x0,
body_size=0x0, stream_r=0xbfbfe920) at mail.c:149
#7 0x0809d936 in mail_storage_copy (t=0x811b980, mail=0x8141860,
flags=0, keywords=0x0, dest_mail=0x0) at mail-copy.c:17
#8 0x080a0632 in mailbox_copy (t=0x811b980, mail=0x8141860, flags=0,
keywords=0x0, dest_mail=0x0) at mail-storage.c:763
#9 0x0805960d in deliver_save (namespaces=0x8121c68,
storage_r=0xbfbfeb94, mailbox=0x811a360 Draft, mail=0x8141860,
flags=0, keywords=0x0) at deliver.c:234
...
backtrace (B) for mbox_save_init:
#0 mbox_save_get_input_stream (ctx=0x811bb00, input=0x814c728) at
mbox-save.c:384
#1 0x08078165 in mbox_save_init (_t=0x811b980, flags=0, keywords=0x0,
received_date=1239634649, timezone_offset=0,
from_envelope=0x80f292b MAILER-DAEMON, input=0x814c728,
dest_mail=0x8150060, ctx_r=0xbfbfe924) at mbox-save.c:512
#2 0x080a0560 in mailbox_save_init (t=0x811b980, flags=0,
keywords=0x0, received_date=-1, timezone_offset=0,
from_envelope=0x80f292b MAILER-DAEMON, input=0x814c728,
dest_mail=0x0, ctx_r=0xbfbfe924) at mail-storage.c:728
#3 0x0809d9c4 in mail_storage_copy (t=0x811b980, mail=0x8141860,
flags=0, keywords=0x0, dest_mail=0x0) at mail-copy.c:28
#4 0x080a0632 in mailbox_copy (t=0x811b980, mail=0x8141860, flags=0,
keywords=0x0, dest_mail=0x0) at mail-storage.c:763
#5 0x0805960d in deliver_save (namespaces=0x8121c68,
storage_r=0xbfbfeb94, mailbox=0x80f2af9 INBOX, mail=0x8141860,
flags=0, keywords=0x0) at deliver.c:234
...
The function mbox_save_get_input_stream calls
i_stream_create_header_filter and i_stream_create_concat.
The function read_header (istream-header-filter.c) finds EOH for call
of i_stream_seek (backtrace B), but my calls of
i_stream_create_header_filter and i_stream_create_concat doesn't find
EOH (backtrace A). Why? I don't understand.
I want to call rarules_header_callback for disable eoh so as to add my
headers.
Thanks
P.S. Sorry for my bad english :-(
It's mbox_save_get_input_stream:
mbox_save_get_input_stream(struct mbox_save_context *ctx, struct
istream *input)
{
struct istream *filter, *ret, *cache_input, *streams[3];
/* filter out unwanted headers and keep track of headers' MD5
sum */
filter = i_stream_create_header_filter(input,
HEADER_FILTER_EXCLUDE |
HEADER_FILTER_NO_CR |
HEADER_FILTER_ADD_MISSING_EOH,
mbox_save_drop_headers,
mbox_save_drop_headers_count,
save_header_callback,
ctx);
if ((ctx-mbox-storage-storage.flags
MAIL_STORAGE_FLAG_KEEP_HEADER_MD5) != 0) {
/* we're using MD5 sums to generate POP3 UIDLs.
clients don't like it much if there are duplicates,
so make sure that there can't be any by appending
our own X-Delivery-ID header. */
const char *hdr;
T_BEGIN {
mbox_save_x_delivery_id(ctx);
} T_END;
hdr = ctx-x_delivery_id_header;
streams[0] = i_stream_create_from_data(hdr,
strlen(hdr));
streams[1] = filter;
streams[2] = NULL;
ret = i_stream_create_concat(streams);
i_stream_unref(filter);
filter = ret;
}
It's my callback for get_stream:
---
static int rarules_get_stream(struct mail *mail, struct message_size
*hdr_size,
struct message_size *body_size,
struct istream **stream_r)
{
struct rarules_mail *m = RA_MAIL_CONTEXT(mail);
struct istream *stream = NULL;
unsigned int deleted = 0;
unsigned int added = 0;
int ret = 0;
DPRINT(func is called);
/* standard get_stream for mail storage format */
Re: [Dovecot] help with ldap config
Thank you. Ricardo Kleemann wrote: user_filter = ((objectClass=posixAccount)(uid=%u,ou=%d)) Do you mean '((objectClass=posixAccount)(uid=%u)(ou=%d))'? or '((objectClass=posixAccount)(uid=%u))'? Or you can just use '(objectClass=posixAccount)' for test. This seems to have worked (using only (objectClass=posixAccount)). Now I have another question... I'm currently using courier-imap, Maildir files. Do I need to run conversion, or can I simply let dovecot run and create its own indexing without attempting to convert anything from courier? Ricardo
[Dovecot] Suggestion: deliver should honor mail_log_prefix
I can set the log prefix for imap/pop3 processes using mail_log_prefix but the log prefix for deliver is hard-coded. (I'm using 1.1.13.) Deliver should either honor mail_log_prefix or have its own setting (deliver_log_prefix?). What do you think?
[Dovecot] Handling ENFILE
My mail server consumed all of its configured file table slots and started returning ENFILE (Too many open files in system) for many operations. This wreaked havoc with dovecot-1.1.13. Here are some areas where dovecot should detect and more gracefully handle ENFILE error returns: 1. Deliver should report an error more helpful than Unknown internal error when fdatasync_path fails: deliver(user): Apr 10 14:53:20 Error: fdatasync_path(/Volumes/MailData/ user/new) failed: Too many open files in system deliver(user): Apr 10 14:53:20 Info: msgid=...: save failed to INBOX: BUG: Unknown internal error 2. Index files should not be fscked when they could not be opened due to ENFILE: Error: IMAP(user): open() failed with index file /Volumes/MailData/ user/.Deleted Messages/dovecot.index: Too many open files in system Warning: IMAP(user): fscking index file /Volumes/MailData/ user/.Deleted Messages/dovecot.index 3. Dovecot should not crash when mail transaction logs become corrupt, or appear to have become corrupt due to ENFILE. Here is a sequence of error messages from the log but I can't tell whether the IMAP ones up through the panic in the middle are all from the same process, or from different processes. deliver(user): Apr 10 14:55:32 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:55:54 Error: open() failed with index file / Volumes/MailData/user/dovecot.index: Too many open files in system deliver(user): Apr 10 14:55:54 Error: open() failed with file /Volumes/ MailData/user/dovecot.index.log.2: Too many open files in system deliver(user): Apr 10 14:55:57 Error: open() failed with index cache file /Volumes/MailData/user/dovecot.index.cache: Too many open files in system dovecot: Apr 10 14:56:08 Warning: IMAP(user): fscking index file / Volumes/MailData/user/dovecot.index dovecot: Apr 10 14:56:16 Error: IMAP(user): Fixed index file /Volumes/ MailData/user/dovecot.index: log_file_seq 15 - 2 deliver(user): Apr 10 14:56:19 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:56:20 Error: Corrupted transaction log file / Volumes/MailData/user/dovecot.index.log seq 2: record size too small (type=0x0, offset=23088, size=0) (sync_offset=36848) deliver(user): Apr 10 14:56:20 Warning: fscking index file (in-memory index) deliver(user): Apr 10 14:56:21 Info: msgid=...: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2009-04-10 14:56:20] dovecot: Apr 10 14:56:40 Panic: IMAP(user): file mail-transaction-log- view.c: line 103 (mail_transaction_log_view_set): assertion failed: (min_file_seq = max_file_seq) dovecot: Apr 10 14:56:40 Error: IMAP(user): Raw backtrace: [see below for better backtrace] deliver(user): Apr 10 14:56:45 Error: Transaction log file /Volumes/ MailData/user/dovecot.index.log: marked corrupted deliver(user): Apr 10 14:57:04 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:57:23 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:57:49 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:58:21 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:58:41 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:59:09 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:59:28 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 14:59:43 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 15:00:57 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 15:03:01 Info: msgid=...: saved mail to INBOX deliver(user): Apr 10 15:03:27 Info: msgid=...: saved mail to INBOX dovecot: Apr 10 15:21:41 Error: IMAP(user): Corrupted transaction log file /Volumes/MailData/user/dovecot.index.log seq 2: Invalid transaction log size (36848 vs 23088): /Volumes/MailData/user/ dovecot.index.log (sync_offset=36848) dovecot: Apr 10 15:21:42 Info: imap-login: Login: user=user, method=CRAM-MD5, rip=..., lip=..., TLS The backtrace for the assertion failure was: 0 libSystem.B.dylib 0x7fff8007c102 __kill + 10 1 libSystem.B.dylib 0x7fff800f7ffc abort + 83 2 imap 0x00010006a22e default_fatal_finish + 70 3 imap 0x00010006a27c i_syslog_fatal_handler + 0 4 imap0x0001000699db i_info + 0 5 imap 0x000100058542 mail_transaction_log_view_set + 104 6 imap 0x000100053b4c view_sync_set_log_view_range + 125 7 imap 0x000100053fc5 mail_index_view_sync_begin + 378 8 imap 0x00010003c26e index_mailbox_sync_init + 129 9 imap 0x000100018457 maildir_storage_sync_init + 229 10 imap 0x0001c6bb imap_sync_init + 132 11 imap 0x0001c90d cmd_sync_delayed + 477 12 imap
Re: [Dovecot] Handling ENFILE
On Mon, 2009-04-13 at 14:29 -0500, Mike Abbott wrote: My mail server consumed all of its configured file table slots and started returning ENFILE (Too many open files in system) for many operations. This wreaked havoc with dovecot-1.1.13. Here are some areas where dovecot should detect and more gracefully handle ENFILE error returns: I also just tried making nfs_safe_open() fail randomly and reproducing these. 1. Deliver should report an error more helpful than Unknown internal error when fdatasync_path fails: deliver(user): Apr 10 14:53:20 Error: fdatasync_path(/Volumes/MailData/ user/new) failed: Too many open files in system deliver(user): Apr 10 14:53:20 Info: msgid=...: save failed to INBOX: BUG: Unknown internal error http://hg.dovecot.org/dovecot-1.1/rev/576e77047959 2. Index files should not be fscked when they could not be opened due to ENFILE: .. 3. Dovecot should not crash when mail transaction logs become corrupt, or appear to have become corrupt due to ENFILE. I made nfs_safe_open() return failures randomly and found all kinds of bugs with it. I committed several fixes to hg, but I still get some weird errors once in a while, I'll try to debug more later.. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] auth-master: Permission denied [sigh]
My latest test:
spam:dovecot = user: spam
user1:dovecot = user: user1
root:dovecot = binary: /usr/local/libexec/deliver
root:dovecot 777 = dir: /var/run/dovecot/
Still getting:
deliver(user1): Can't connect to auth server at \
/var/run/dovecot/auth-master: Permission denied
What's the key to this problem?
If I set spam, user1, deliver and /var/run/dovecot/ to the same group, and
give read/write permission in that directory to that group, why can't they
all use auth-master?
1) User 'spam:dovecot' runs Smapassassin
2) Hands off to deliver (root:dovecot)
3) Deliver assumes 'user1:dovecot' identity
4) Can't access auth-master in 'root:dovecot' directory (777)
So it's 'auth-master' that is (a) not available to 'user1' AND (b) not
available to group 'dovecot'. Huh? Why not?
I'm obviously missing info about the temporary 'auth-master'.
Can anyone please give me a hand? I'd really appreciate it.
Thank you.
James
Thank you! Even setting the /var/run/dovecot tree to all chmod 777s
doesn't help. I'm probably mis-remembering the ownership of auth-master,
in my original note. I haven't seen it since I left my notes at work.
With regard to this maillog entry:
postfix/pipe[29452]: 60990FA01BA: to=recipi...@example-receive.com, \
relay=spamassassin, delay=6, delays=0.33/0.01/0/5.7, dsn=4.3.0, \
status=deferred (temporary failure)
It IS a (temporary failure), because soon after I revert to the simple:
mailbox_command = /usr/local/libexec/dovecot/deliver
the message arrives to the recipient user's mailbox.
It's the spamassassin = deliver handoff and user SWITCH that seems to be
problematic.
But then, my brain is all garbled, at this point, so I can't really trust
any of my logic. I'll check back in, tomorrow.
Thanks, again.
James
Hi,
I was having problems with permissions on auth-master too. I solve them
creating manually the folder /var/run/dovecot with correct permissions
but
i
see you already did that :\
On Sun, Apr 12, 2009 at 5:27 PM, James Butler
jbut...@thebestdefense.comwrote:
I've been messing with this for too long, now, and I'm blind to
whatever's
wrong. Or I'm simply being dense. Either way, I need help with a common
issue.
I'm trying to get Postfix+Spamassassin+Dovecot going on Fedora 10.
(I'll
get back to the global Sieve thingy soon, but I need to get this going,
first.)
When using the simple:
mailbox_command = /usr/local/libexec/dovecot/deliver
everything is cool, except there's no Spamassassin involvement,
obviously.
The problem shows itself when the Spamassassin user hands off to the
recipient user and Deliver + the recipient user tries to access
/var/run/dovecot/auth-master.
Thank you for any insight you can provide.
/var/run/dovecot: 755 root:dovecot
/var/run/dovecot/login: 750 root:dovecot
/var/run/dovecot/auth-master: 750 root:dovecot
(I think. auth-master is a temporary file? Comes and goes.)
From /etc/postfix/main.cf
mailbox_transport = spamassassin
From /etc/postfix/master.cf:
spamassassin unix - n n - - pipe
user=spam argv=/usr/bin/spamc -f -e /usr/libexec/dovecot/deliver
-f ${sender} -d ${user} -m ${extension}
Here's my 'socket listen' section from /usr/local/etc/dovecot.conf:
socket listen {
master {
path = /var/run/dovecot/auth-master
mode = 0666
#user =
group = dovecot
}
client {
path = /var/run/dovecot/auth-client
mode = 0666
#user =
group = dovecot
}
}
From /var/log/maillog:
Postfix receives the message:
postfix/smtpd[29447]: connect from \
IP-ADD-RE-SS.ptr.example-send.com[IP.ADD.RE.SS]
postfix/smtpd[29447]: 60990FA01BA: \
client=IP-ADD-RE-SS.ptr.example-send.com[IP.ADD.RE.SS]
postfix/cleanup[29451]: 60990FA01BA: \
message-id=49e20bf2.4090...@example-send.com
postfix/qmgr[29441]: 60990FA01BA: from=sen...@example-send.com, \
size=812, nrcpt=1 (queue active)
postfix/smtpd[29447]: disconnect from \
IP-ADD-RE-SS.ptr.example-send.com[IP.ADD.RE.SS]
Spamassassin processes the message as user 'spam':
spamd[4121]: spamd: processing message\
49e20bf2.4090...@example-send.com for spam:653
spamd[4121]: spamd: clean message (3.0/5.0) for spam:653 in 5.2
seconds,\
793 bytes.
spamd[4121]: spamd: result: . 2 - RDNS_DYNAMIC,TVD_SPACE_RATIO \
scantime=5.2,size=793,user=spam,uid=653,required_score=5.0, \
rhost=localhost.localdomain,raddr=127.0.0.1,rport=42493, \
mid=49e20bf2.4090...@example-send.com,autolearn=no
Spamassassin pipes result to Deliver which runs as recipient user.
Deliver as recipient user doesn't have permission to auth:
deliver(recipient): Can't connect to auth server at \
/var/run/dovecot/auth-master: Permission denied
postfix/pipe[29452]: 60990FA01BA: to=recipi...@example-receive.com, \
relay=spamassassin, delay=6, delays=0.33/0.01/0/5.7, dsn=4.3.0, \
status=deferred (temporary failure)
1) I must use the 'user=' arg for spamc
2) Can't use 'user=${user}' or $user:
fatal: get_service_attr: unknown username: ${user}
3) Must
Re: [Dovecot] auth-master: Permission denied [sigh]
Unless they've changed something, doesn't the S.A method as you are using generate backscatter, I'd be using something more efficient like MailScanner or amavisd-new either will take you no more than 15 minutes to setup and avoid the past several days of your problems, pf and dovecot work well together. On Tue, 2009-04-14 at 08:48, James Butler wrote: My latest test: spam:dovecot = user: spam user1:dovecot = user: user1 root:dovecot = binary: /usr/local/libexec/deliver root:dovecot 777 = dir: /var/run/dovecot/ Still getting: deliver(user1): Can't connect to auth server at \ /var/run/dovecot/auth-master: Permission denied What's the key to this problem? If I set spam, user1, deliver and /var/run/dovecot/ to the same group, and give read/write permission in that directory to that group, why can't they all use auth-master? 1) User 'spam:dovecot' runs Smapassassin 2) Hands off to deliver (root:dovecot) 3) Deliver assumes 'user1:dovecot' identity 4) Can't access auth-master in 'root:dovecot' directory (777) So it's 'auth-master' that is (a) not available to 'user1' AND (b) not available to group 'dovecot'. Huh? Why not?
Re: [Dovecot] auth-master: Permission denied [sigh]
On Mon, 2009-04-13 at 15:48 -0700, James Butler wrote: 1) User 'spam:dovecot' runs Smapassassin 2) Hands off to deliver (root:dovecot) Have you set up some kind of setuid-root deliver, or why is it running as root:dovecot here instead of spam:dovecot? 3) Deliver assumes 'user1:dovecot' identity 4) Can't access auth-master in 'root:dovecot' directory (777) 4) happens before 3). So it's 'auth-master' that is (a) not available to 'user1' AND (b) not available to group 'dovecot'. Huh? Why not? My guess is that deliver isn't really started with dovecot group permission. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] why not install utils (idxview logview ...) with dovecot in bin/sbin dir?
On Thu, 2009-04-09 at 10:37 +0800, Xueron Nee wrote: Hi, I just found these useful tools :) they were installed in libexec dir :) If they were installed to bin/ or sbin/, they would have to be renamed so people would know they're part of Dovecot. I don't think it's really worth the effort. What do you use them for anyway? For me they're mainly for debugging what went wrong. But I am considering on moving dovecot-lda (ex. deliver) to bin/. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] why not install utils (idxview logview ...) with dovecot in bin/sbin dir?
On Tue, 2009-04-14 at 10:19, Timo Sirainen wrote: But I am considering on moving dovecot-lda (ex. deliver) to bin/. Move? Maybe a soft symlink be better?
Re: [Dovecot] why not install utils (idxview logview ...) with dovecot in bin/sbin dir?
On Tue, 2009-04-14 at 10:23 +1000, Noel Butler wrote: On Tue, 2009-04-14 at 10:19, Timo Sirainen wrote: But I am considering on moving dovecot-lda (ex. deliver) to bin/. Move? Maybe a soft symlink be better? For backwards compatibility or why? signature.asc Description: This is a digitally signed message part
Re: [Dovecot] why not install utils (idxview logview ...) with dovecot in bin/sbin dir?
On Tue, 2009-04-14 at 10:26, Timo Sirainen wrote: On Tue, 2009-04-14 at 10:23 +1000, Noel Butler wrote: On Tue, 2009-04-14 at 10:19, Timo Sirainen wrote: But I am considering on moving dovecot-lda (ex. deliver) to bin/. Move? Maybe a soft symlink be better? For backwards compatibility or why? Yes, for backwards compatibility.
