Re: [Dovecot] fts-solr indexer-worker connects to wrong solr host dovecot-2.2.4
Here’s a more complex fix that reuses the HTTP connections for the same hosts:
http://hg.dovecot.org/dovecot-2.2/rev/26355654c314
On 3.10.2013, at 19.27, Richard Platel wrote:
> Did some more digging.
>
> The problem is that the fts-solr plugin has a global solr_conn pointer, that
> persists between users. I think this patch fixes the problem:
>
> --- a/dovecot/fts_solr_plugin/fts-solr-plugin.c
> +++ b/dovecot/fts_solr_plugin/fts-solr-plugin.c
> @@ -50,6 +50,13 @@ static void fts_solr_mail_user_create(struct mail_user
> *user, const char *env)
> {
>struct fts_solr_user *fuser;
>
> + /** solr URL may be different per-user **/
> + if (solr_conn != NULL) {
> + solr_connection_deinit(solr_conn);
> + solr_conn = NULL;
> + }
> + /**/
> +
>fuser = p_new(user->pool, struct fts_solr_user, 1);
>if (fts_solr_plugin_init_settings(user, &fuser->set, env) < 0) {
>/* invalid settings, disabling */
>
>
> On 2013-10-02, at 3:28 PM, Richard Platel wrote:
>
>> I've confirmed that this problem still exists in 2.2.5
>>
>> It seems that indexer-worker only init's plugins at startup, so the fts_solr
>> plugin is holding the url= parameter from the first user.
>>
>> The problem doesn't happen if the indexer-worker process is idle-killed
>> between users. A new process starts up with the new user's userdb settings.
>>
>> I thought I could work around this problem by adjusting indexer-worker's
>> settings:
>>
>> service indexer-worker {
>> service_count = 1
>> idle_kill = 1
>> }
>>
>> but these changes don't seem to have any effect, the indexer-worker process
>> still hangs around idling after indexing a user, and isn't idle-killed for
>> upwards of a minute.
>>
>> Any help?
>>
>>
>> On 2013-09-27, at 11:46 AM, Richard Platel wrote:
>>
>>> Hello.
>>> We're setting up fts solr and want to have the solr server host be set
>>> per-user via UserDB.
>>>
>>> It looks like if a user connects and fts indexes mail, and then another
>>> user connects and indexes mail, indexer-worker is connecting to the first
>>> user's fts host:
>>>
>>> User1, ham...@rp-auth-test.com connects, does a SEARCH for the first time,
>>> indexer-worker gets UserDB settings and correctly indexes mail on ftsvs01:
>>>
>>> [...]
>>> auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): lookup
>>> shared/userdb/ham...@rp-auth-test.com
>>> auth-worker(2195): Debug: dict(ham...@rp-auth-test.com): result:
>>> {"uid":"8","fts":"solr","quota_rule4":"Spam:ignore","_session":"talk15_590ec6d100042","quota_rule3":"Trash:ignore","quota_rule2":"*:messages=2684354","quota_rule":"*:storage=5242880k","mail":"maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/","fts_solr":"debug
>>> url=http://ftsvs01:8080/solr/","gid":"8"}
>>> auth: Debug: userdb out: USER 1 ham...@rp-auth-test.com uid=8
>>> fts=solrquota_rule4=Spam:ignore _session=talk15_590ec6d100042
>>> quota_rule3=Trash:ignorequota_rule2=*:messages=2684354
>>> quota_rule=*:storage=5242880k
>>> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
>>> fts_solr=debug url=http://ftsvs01:8080/solr/gid=8
>>> indexer-worker: Debug: auth input: ham...@rp-auth-test.com uid=8 fts=solr
>>> quota_rule4=Spam:ignore _session=talk15_590ec6d100042
>>> quota_rule3=Trash:ignore quota_rule2=*:messages=2684354
>>> quota_rule=*:storage=5242880k
>>> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
>>> fts_solr=debug url=http://ftsvs01:8080/solr/ gid=8
>>> indexer-worker: Debug: Added userdb setting:
>>> plugin/_session=talk15_590ec6d100042
>>> indexer-worker: Debug: Added userdb setting: plugin/fts=solr
>>> indexer-worker: Debug: Added userdb setting: plugin/fts_solr=debug
>>> url=http://ftsvs01:8080/solr/
>>> indexer-worker: Debug: Added userdb setting:
>>> mail=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ha
>>> m...@rp-auth-test.com/
>>> indexer-worker: Debug: Added userdb setting:
>>> plugin/quota_rule=*:storage=5242880k
>>> indexer-worker: Debug: Added userdb setting:
>>> plugin/quota_rule2=*:messages=2684354
>>> indexer-worker: Debug: Added userdb setting: plugin/quota_rule3=Trash:ignore
>>> indexer-worker: Debug: Added userdb setting: plugin/quota_rule4=Spam:ignore
>>> indexer-worker(ham...@rp-auth-test.com): Debug: Effective uid=8, gid=8,
>>> home=
>>> indexer-worker(ham...@rp-auth-test.com): Debug: Namespace inbox:
>>> type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions
>>> =yes
>>> location=maildir:/mail/mailstore01/215/573/ham...@rp-auth-test.com/:INDEX=/mail/index01/215/573/ham...@rp-auth-test.com/
>>> indexer-worker(ham...@rp-auth-test.com): Debug: maildir++:
>>> root=/mail/mailstore01/215/573/h
Re: [Dovecot] Dovecot proxy hooks
On 14.10.2013, at 21.22, d...@getodata.ro wrote: > I am interested in the possibility of using Dovecot IMAP/POP proxying > capabilities to analyze emails that are passing through and possibly modify > content on the fly. This subject has been discussed here [1] before. > I have tried the mail-filter plugin [2], but the hooks it uses are only > called in a non-proxy setup. > > Is there a practical way of doing this, or plans to add such a feature? > > Links: > [1]: http://dovecot.org/list/dovecot/2006-February/011704.html > [2]: http://www.dovecot.org/patches/2.2/mail-filter.tar.gz You can’t use the simple proxying feature for this. If you modify the mail content, it would require modifying quite a lot of different command outputs and there’s no way a proxy could do it without more or less reimplementing half of the IMAP server functionality. But what you could do is to use the imapc backend and the mail-filter.
Re: [Dovecot] Plugin issue with update from 2.0.19 to 2.1.17
On 15.10.2013, at 23.33, Reinaldo Matukuma wrote: > Hello. Probably only Timo can help-me with this. > > I have a self-made plugin based on the zlib plugin that i use to cryptograph > the messages at inbox. > > As a side-effect of the cryptography, my plugin changes the size of the > message, but until 2.0.19 this works well with dovecot index and the W/S > flags. > > But now, i'm going to upgrade to 2.1.17 and now i have these messages on log > at my test ambiance: > > Oct 15 20:19:25 test dovecot: imap(reina...@exemplo.com.br): Error: Cached > message size smaller than expected (367 < 529) > Oct 15 20:19:25 test dovecot: imap(reina...@exemplo.com.br): Error: Maildir > filename has wrong S value, renamed the file from > /storage/test/messages/exemplo.com.br/reinaldo/Maildir/.Sent/cur/1381879158.M634385P5208.test,S=367,W=378:2,S > to > /storage/test/messages/exemplo.com.br/reinaldo/Maildir/.Sent/cur/1381879158.M634385P5208.test,S=529:2,S > Oct 15 20:19:25 test dovecot: imap(reina...@exemplo.com.br): Error: Corrupted > index cache file > /storage/test/messages/exemplo.com.br/reinaldo/Maildir/.Sent/dovecot.index.cache: > Broken physical size for mail UID 6 Is it intended that when decrypting messages you’ll get exactly the original message back? Or are you also modifying the message? Assuming there is no modification, you could take a look at how http://dovecot.org/patches/2.2/mail-filter.tar.gz does that without problems. Message modifications should also be possible with that code with small modifications. I’m not sure if that code works for v2.1.
Re: [Dovecot] unusual dsync lines
On 20.10.2013, at 20.01, Anand Kumria wrote: > Hi, > > $ doveadm sync -1 -r raw.log -R 'doveadm -o imapc_user=foo -o > imapc_password=bar -o mail=imapc: dsync-server' > > I couldn't get that line to work, I get errors like: > > doveadm(root): Fatal: Error reading configuration: Invalid -o parameter > imapc:: Missing ‘=' Not sure about the above error, but > dsync-local(root): Error: read(remote) failed: EOF (version not received) > dsync-local(root): Panic: file iostream.c: line 37 (io_stream_unref): > assertion failed: (stream->refcount > 0) > *** glibc detected *** doveadm: corrupted double-linked list: > 0x02312620 *** this is fixed now: http://hg.dovecot.org/dovecot-2.2/rev/d66b4b1b343a
Re: [Dovecot] backup maildir mailbox bugs
On 20.10.2013, at 19.24, Anand Kumria wrote: > Using dovecot v2.2.5.5, I get the following: > > $ doveadm -v backup -R -u u...@example.com maildir:/home/rsync/ > example.com/user/Maildir/ > > [...] > > dsync(u...@example.com): Panic: file dsync-mailbox-export.c: line 228 > (export_save_change_get): assertion failed: (change->type == > DSYNC_MAIL_CHANGE_TYPE_FLAG_CHANGE) That’s definitely a bug, but I’m not sure how to reproduce it. Can you create such a test maildir where this happens that you could send to me? For example you could change all the mail contents to just use “x” letters. Here’s a script that does it: http://dovecot.org/tools/maildir-anonymize.pl Most likely this is related to your specific dovecot.index* files, and deleting them would fix the problem. I’d still like to fix the real bug though.
Re: [Dovecot] Strange output from LIST command
> Od: Timo Sirainen > Komu: azurIt > Dátum: 26.10.2013 17:01 > Predmet: Re: [Dovecot] Strange output from LIST command > > CC: "dovecot@dovecot.org List" >On 14.10.2013, at 21.28, azurIt wrote: > >> i'm using Dovecot 2.1.7 (Debian Wheezy) and output from LIST command looks >> strange: >> >> C: 4 LIST () "" (INBOX INBOX.Karantena INBOX.Spam) RETURN (STATUS (UNSEEN)) >> S: * LIST () "." "INBOX" >> S: * LIST () "." "INBOX.Karantena" >> S: * STATUS "INBOX.Karantena" (UNSEEN 0) >> S: * LIST () "." "INBOX.Spam" >> S: * STATUS "INBOX.Spam" (UNSEEN 0) >> S: 4 OK List completed. >> >> The UNSEEN information for INBOX is completely missing. It is correct >> behavior? If not, is this a known bug in 2.1.7? Thank you. > >Fixed: http://hg.dovecot.org/dovecot-2.1/rev/d16e212531ec > >It was also already working for v2.2. Thank you. Which version will include this fix and approximately when it will be released? Thanks for info. azur
Re: [Dovecot] Login into other user's account // master user for non-master users // chroot to users.
On 18.10.2013, at 11.31, Steffen Kaiser wrote: > Now, I came into thinking that it would be good in such case, if userB could > authentificate as, say "sales*userB" - much like a master user - and ends in > "sales"'s home, but with access permissions of "userB", well, like a chroot. > > > Would it be an interesting feature to add to Dovecot's core? > > > If I simulate "sales*userB" with password of userB and let the userdb return > the home of sales, userB would gain "owner" privilegues of sales implicitly. > So there seems to exist no workaround. I think that’s already possible. If master user=sales and userB = login user, the ACLs would work the way you want. The problem is how to have different passwords for the sales master user for userA and userB. But that could be done by e.g. a checkpassword script.
Re: [Dovecot] proxy, userdb and passdb
On 22.10.2013, at 13.13, Jogi Hofmüller wrote:
> Hi Steffen,
>
> Am 2013-10-22 10:05, schrieb Steffen Kaiser:
>
>> see http://wiki2.dovecot.org/PasswordDatabase/ExtraFields
>
> Did, thanks. The errors I mentioned in my previous post are gone. Still,
> proxying does not work as expected. Instead I get strange warnings:
>
> Oct 22 12:06:51 server dovecot: auth-worker(PID): Warning: userdb passwd:
> Move templates args to override_fields setting
>
> This is the proxy-userdb file's content (I removed the UID and IP address):
>
> user:::proxy=y host=IP-ADDRESS starttls=y nopassword=y
> passdb {
> args = session=yes
> driver = pam
> }
> userdb {
> args = /etc/dovecot/proxy-userdb
> driver = passwd
> }
1) Use passwd-file, not passwd
2) userdb has no effect on proxying, it must be passdb.
If you really want to keep using PAM, you need to use Dovecot v2.2 with an
additional passdb configuring the proxying for the users.
http://wiki2.dovecot.org/PasswordDatabase#Passdb_settings explains more. It
can’t be done with v2.1.
Re: [Dovecot] LMTP, TLS/SSL, authentication, proxy
On 22.10.2013, at 14.37, Jogi Hofmüller wrote:
> Several questions packed into one email ;)
>
> Can dovecot use TLS/SSL on LTMP inet socket?
Probably. Try adding ssl=yes to the inet_listener {}.
> Can I configure dovecot to only let an authenticated user deliver mail via
> LMTP?
Currently LMTP server doesn’t support AUTH.
> Can I tell dovecot to use a user/password for proxying LMTP connections?
Yes, the same way as for IMAP/POP3.
Re: [Dovecot] failed: Message has been copied too many times
On 24.10.2013, at 13.41, Angel L. Mateo wrote: >> amateo_adm@myotis51:~$ sudo doveadm search -u vlo mailbox >> BORRADOS.INBOX.MNCS >> doveadm(vlo): Error: Syncing mailbox BORRADOS.INBOX.MNCS failed: Message >> has been copied too many times (59306 + -1) Looks like the index is corrupted. It should never have gotten that high. Have you tried doveadm force-resync -u vlo INBOX? > One more thing, when I said that the index has grown too much is > because index directory for this mailbox folder is about 850MB, and the > mailbox originally had about 3000 messages. This is the list of files in the > index directory: > > root@myotis51:/mail/indexes/vl/vlo/expunged/mailboxes/INBOX/MNCS# ls -lh > total 853M > -rw--- 1 vmail vmail 127M oct 24 11:16 dovecot.index > -rw--- 1 vmail vmail 127M oct 24 11:02 dovecot.index.backup > -rw--- 1 vmail vmail 186M oct 24 12:14 dovecot.index.cache > -rw--- 1 vmail vmail 180M oct 24 12:36 dovecot.index.log > -rw--- 1 vmail vmail 138M oct 24 11:02 dovecot.index.log.2 These files are way too large for a mailbox with just 3000 mails.
Re: [Dovecot] Problems with userdb lookup
On 24.10.2013, at 16.13, Mario Arnold wrote: > #6 0xb7751886 in p_strndup (pool=0xb7796544 , > str=str@entry=0x0, max_chars=max_chars@entry=4294967295) at strfuncs.c:74 >mem = >len = >__FUNCTION__ = "p_strndup" > #7 0xb7751d7f in t_strndup (str=0x0, max_chars=4294967295) at strfuncs.c:236 > No locals. > #8 0x08057003 in auth_request_append_password (str=str@entry=0x97e68a8, > request=0x97f8da8, request=0x97f8da8) at auth-request.c:1799 >p = >log_type = >max_len = 4294967295 2 fixes related to this: http://hg.dovecot.org/dovecot-2.2/rev/8a8e63a351f5 http://hg.dovecot.org/dovecot-2.2/rev/1d222bd0a49c
Re: [Dovecot] Strange output from LIST command
On 14.10.2013, at 21.28, azurIt wrote: > i'm using Dovecot 2.1.7 (Debian Wheezy) and output from LIST command looks > strange: > > C: 4 LIST () "" (INBOX INBOX.Karantena INBOX.Spam) RETURN (STATUS (UNSEEN)) > S: * LIST () "." "INBOX" > S: * LIST () "." "INBOX.Karantena" > S: * STATUS "INBOX.Karantena" (UNSEEN 0) > S: * LIST () "." "INBOX.Spam" > S: * STATUS "INBOX.Spam" (UNSEEN 0) > S: 4 OK List completed. > > The UNSEEN information for INBOX is completely missing. It is correct > behavior? If not, is this a known bug in 2.1.7? Thank you. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/d16e212531ec It was also already working for v2.2.
Re: [Dovecot] Empty Mails from MAILER-DAEMON with Dovecot-Antispam and DSPAM
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 24-10-13 09:59, Anna Christina Naß wrote:
> Hallo,
>
> I've installed the dovecot-Antispam extension to my dovecot IMAP
> installation in conjunction with DSPAM.
>
> But when moving Mails from e.g. INBOX to Spam or vice versa, empty
> mails from MAILER-DAEMON appear after the original mail has been
> moved. This happens using Thunderbird 17esr, K-9 Mail on Android
> and Apple Mail (Mountain Lion). (Perhaps also when using a Webmail
> IMAP client)
>
> The source from one of these empty mails looks like this:
>
> snip Return-Path: X-Original-To: acn
> Delivered-To: a...@mydomain.name Received: from localhost
> (localhost.localdomain [127.0.0.1]) by (Postfix)
> with SMTP id CB81B140011 for ; Thu, 24 Oct 2013 09:23:33 +0200
> (CEST) X-DSPAM-Reclassified: Spam Message-Id:
> <20131024072333.CB81B140011@> Date: Thu, 24 Oct 2013
> 09:23:33 +0200 (CEST) From: MAILER-DAEMON /snip
>
> Do you have any idea why these mails appear and how I can prevent
> it?
>
> # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-686-pae i686
> Debian 7.2 ext4 auth_cache_size = 1 M auth_mechanisms = plain
> login auth_worker_max_count = 5 listen = *, [::] log_timestamp =
> "%Y-%m-%d %H:%M:%S " mail_location = maildir:/var/vmail/%u/Maildir
> mail_privileged_group = mail mailbox_idle_check_interval = 15 secs
> managesieve_notify_capability = mailto managesieve_sieve_capability
> = fileinto reject envelope encoded-character vacation subaddress
> comparator-i;ascii-numeric relational regex imap4flags copy include
> variables body enotify environment mailbox date ihave imapflags
> notify passdb { args = /etc/dovecot/ldap-passdb.conf.ext driver =
> ldap } plugin { antispam_allow_append_to_spam = no antispam_backend
> = dspam antispam_dspam_args =
> --user;%Lu;--deliver=spam,innocent;--source=error
You're telling DSPAM to re-deliver e-mail after retraining. You don't
want that, as you already received the e-mail. You only want DSPAM to
re-learn the message, so try something like:
antispam_dspam_args = --user;%Lu;--deliver=;--source=error
> antispam_dspam_binary = /usr/bin/dspamc antispam_dspam_notspam =
> --class=innocent antispam_dspam_spam = --class=spam
> antispam_signature = X-DSPAM-Signature antispam_signature_missing =
> move antispam_spam = Spam;Junk antispam_trash = Trash;Deleted
> Items;Deleted Messages
Kind regards,
Tom
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJSa8b2AAoJEJPfMZ19VO/1J9EP/08yHDm8Okxe65TA83eQMHp4
SHj3EvJH115BtRjSEJMME3CQ7Vd5G3Ah1l3b6QMWhkEM2zRiFXD10WfrMplZvO/5
snS70CFHKataNyaTuBG7XxK5Gv56DmCIH2j1k/YjSXnUsl8G9Xpbh3lLoPZDALoC
9BThSrTCxMc2hN3W+VGBvhxh1Nk2t8hak56QvzSebKDkazLX7MHcexZWZuBqh+v5
saicGoSak3FSNhowTul0JI9EzFtU8VKlLOFOqy/31/4NynszJQs+aWrm1xkxkZV4
l+nLiRvQUC+TUO0AfvlUwxFCs8cT39IAhPU9lqWNoR8SAM31aqOJRBeO28HMx3Ur
KOlefJnm77fQk6C+upqr0pKu+EgqvxQQBHjacZOKnzUptlwxzS2he38OswiWBk8b
oSqPXpGvZ9jxxmfXQAg0M/a1ztIz6/8vAxNAZHKsuKciXFIn6iZyswn7A1fm2W06
KHHu4ocQ3mGmJ2n5AdnYQP+9axuPBRjCKL2gT2K5hP7uHuJr9vXOUaWpAfAmTPVa
ATyXa6rnmtxOwKZt0azLFQhMzLu6kjEgTTcus3PLJFURECBoatrGMaWOCasLBu6w
4+g63ProScxZ1+P+MDUifxIBdiVD2M5qFc2zhnExt254hE/5YOYcgDq5f8x/XL6m
gy34lWdAQtg8aKoCD07c
=hw6D
-END PGP SIGNATURE-
