[Dovecot] 回复: Re: mysql auth failover failing
it is so easy to use .
if you using dovecot2.0 ,in /etc/dovecot/dovecot-sql.conf.ext like this :
connect =host=localhost dbname=extmail user=root password=
user_query=select concat('*:bytes=',trim(TRAILING 'S' FROM quota)) as
quota_rule,concat('/home/data/domains/',maildir) as mail
,concat('/home/data/domains/',homedir) as home,uidnumber as uid,gidnumber as
gid from mailbox where username='%u'
password_query= select username as user ,password,uidnumber as
userdb_uid,gidnumber as userdb_gid,concat('*:bytes=',trim(TRAILING 'S' FROM
quota)) as userdb_quota_rule from mailbox where username='%u'
my profile include quota, see this http://wiki2.dovecot.org/AuthDatabase/SQL
fy
From: Noel Butler
Date: 2012-07-29 11:45
To: dovecot
Subject: Re: [Dovecot] mysql auth failover failing
On Sat, 2012-07-28 at 19:53 +0300, Timo Sirainen wrote:
> It's in my TODO, but I don't know when I'll get around to implementing it. So
> many things to do right now..
>
Thanks
[Dovecot] Enable ACL for master user
Hi Team,
I have dovecot setup and I have enabled master User login in dovecot.conf file
as below :-
auth_master_user_separator = *
passdb {
driver = passwd-file
args = /etc/dovecot/passwd.masterusers
master = yes
pass = yes
}
passdb {
driver = shadow
}
userdb {
driver = passwd
}
and craeted file. I can login to the user but what I see after login is : -
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT
LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED
I18NLEVEL=1 STARTTLS AUTH=PLAIN] Dovecot ready.
a login user*masteruser password
a OK Logged in.
a list "" *
* LIST (\HasNoChildren) "." "Trash"
* LIST (\Unmarked) "." "INBOX"
a OK List completed.
However when I login as a normal user I see this : -
Connected to 0.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT
LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED
I18NLEVEL=1 STARTTLS AUTH=PLAIN] Dovecot ready.
a login user userpassword
a OK Logged in.
a list "" *
* LIST (\HasNoChildren) "." "Drafts"
* LIST (\HasNoChildren) "." "Trash"
* LIST (\HasNoChildren) "." "Sent items"
* LIST (\HasNoChildren) "." "INBOX"
a OK List completed.
I think its because of ACL for user and it is using masterusers ACL.
I looked at http://wiki2.dovecot.org/PostLoginScripting, however do we have
sample
login script so that I can use that script with modifications.
Sorry I am new to dovecot.
Thanks and Regards.
[Dovecot] Dovecot Director behaviour when two or more fail
Hi, On the Dovecot wiki here http://wiki2.dovecot.org/Director it says the following: "The directors are going to connect to each others in a ring. For example if you have servers called A, B and C, director will create connections A->B, B->C and C->A" I am curious what happens in a scenario when two or more directors fail and potentially break a ring in half, do the active directors detect this and reform a new ring? Dominic
Re: [Dovecot] mysql auth failover failing
On Sat, 2012-07-28 at 19:53 +0300, Timo Sirainen wrote: > It's in my TODO, but I don't know when I'll get around to implementing it. So > many things to do right now.. > Thanks signature.asc Description: This is a digitally signed message part
[Dovecot] Problem with preserving MIME format with Dovecot IMAP and Outlook
Hello,
I have installed Dovecot 2.0.19 on Ubuntu 12.04 LTS. I have set up a very
basic IMAP configuration. I have two different laptops running Outlook (one
MacOS, one Windows 7) and I have configured both to connect to the Dovecot
service. My problem is, emails in MIME format that I synchronise from one
Outlook client are not being correctly decoded as MIME messages on the other
client. Instead, the message is rendered in plaintext, starting with the
lines: This message is in MIME format. Since your mail reader does not
understandthis format, some or all of this message may not be legible.
--B_3426360453_22814Content-type: multipart/alternative;
boundary="B_3426360453_4099"
--B_3426360453_4099Content-type: text/plain;
charset="ISO-8859-1"Content-transfer-encoding: quoted-printable
Can anyone help me understand why this might be happening?
The output from dovecot -n is:
# 2.0.19: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.0-26-generic-pae i686 Ubuntu 12.04 LTS
auth_debug = yes
auth_verbose = yes
disable_plaintext_auth = no
mail_location = maildir:~/Maildir
passdb {
driver = pam
}
protocols = " imap"
ssl_cert =
Re: [Dovecot] lmtp log message
On 29.7.2012, at 2.50, Daniel L. Miller wrote: > Jul 28 16:21:05 bubba dovecot: lmtp(26702): Disconnect from local: Client > quit (in reset) > > Everything looks clean - except that lmtp disconnect message. It may not be > an error - or a warning - but it looks like one. If I need to change my > configuration in either Postfix or Dovecot to "fix" it - please tell me so. > Otherwise, if the message just read "Disconnect from local" to match with the > initial connect it would be fine. But as it is - it leaves me feeling I've > got something setup wrong. It simply says that the LMTP client sent a QUIT command. Nothing wrong. Maybe it would be less bad looking without the "(in reset)" text?
Re: [Dovecot] lmtp log message
On 7/28/2012 12:01 PM, Timo Sirainen wrote: On 28.7.2012, at 21.50, Bård Johannessen wrote: On 2012-07-28 18:36, Timo Sirainen wrote: On 27.7.2012, at 9.01, Daniel L. Miller wrote: Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? It's normal. I'm not even sure it's noteworthy enough to justify logging it; if the message is going to be exactly the same in all (non error) cases, what's the point of logging it? Since LMTP is supposed to be accessible only internally from your network, maybe there isn't much of a point.. Maybe if DATA command is used it wouldn't be logged, but otherwise a QUIT would log "Client quit without delivering mail". Would be useful to get some opinions from more people though. To clarify - I see this message after EVERY message reception. My last log section is: Jul 28 16:21:03 bubba assp/smtpd[27486]: connect from localhost[127.0.0.1] Jul 28 16:21:04 bubba assp/smtpd[27486]: EBBE1404015C: client=dovecot.org[193.210.130.67] Jul 28 16:21:05 bubba postfix/cleanup[27490]: EBBE1404015C: message-id=<501473c0.2080...@starbridge.org> Jul 28 16:21:05 bubba postfix/qmgr[20352]: EBBE1404015C: from=, size=6944, nrcpt=1 (queue active) Jul 28 16:21:05 bubba dovecot: lmtp(26702): Connect from local Jul 28 16:21:05 bubba dovecot: lmtp(26702, dmil...@amfes.com): 16WjK+FzFFBOaAAA4TDHRA: sieve: msgid=<501473c0.2080...@starbridge.org>: stored mail into mailbox 'Lists/Dovecot-Users' Jul 28 16:21:05 bubba postfix/lmtp[27536]: EBBE1404015C: to=, relay=mail.amfes.com[private/dovecot-lmtp], delay=1.2, delays=0.99/0.01/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 16WjK+FzFFBOaAAA4TDHRA Saved) Jul 28 16:21:05 bubba assp/smtpd[27486]: disconnect from dovecot.org[193.210.130.67] Jul 28 16:21:05 bubba dovecot: lmtp(26702): Disconnect from local: Client quit (in reset) Jul 28 16:21:05 bubba postfix/qmgr[20352]: EBBE1404015C: removed Everything looks clean - except that lmtp disconnect message. It may not be an error - or a warning - but it looks like one. If I need to change my configuration in either Postfix or Dovecot to "fix" it - please tell me so. Otherwise, if the message just read "Disconnect from local" to match with the initial connect it would be fine. But as it is - it leaves me feeling I've got something setup wrong. And indeed - do I? In Postfix, I've simply set my transport to use the lmtp via unix socket. I have NOT set a destination_recipient_limit - as I didn't think I needed to do so, unlike when use the lda via pipe. I had lmtp timeouts, so I set process_min_avail=5. -- Daniel
Re: [Dovecot] fts solr : out of memory
Le 28/07/2012 18:34, Timo Sirainen a écrit : > On 28.7.2012, at 19.09, Matthew Powell wrote: > >> This looks very much like an issue I've been having with fts-lucene. >> Indexing works (or at least completes without errors) in 2.1.7, but runs out >> of memory in 2.1.8. >> >> doveadm -v index -u username '*' crashes with: >> >> doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory > .. >> I've posted an example message that triggers the crash here: >> https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test >> case, but I've encountered the same behaviour with a real message. > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/cbd2b321a68f > > Thanks Timo, it seems ok now with memory problem, but an other error shows up: doveadm -v index -u c...@spamguard.fr INBOX doveadm(c...@spamguard.fr): Info: INBOX: Caching mails seq=1..59363 21500/59363doveadm(c...@spamguard.fr): Error: fts_solr: Invalid XML input at line 1: mismatched tag doveadm(c...@spamguard.fr): Panic: file solr-connection.c: line 545 (solr_connection_post_more): assertion failed: (maxfd >= 0) doveadm(c...@spamguard.fr): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43d9a) [0x7f0fbaa87d9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f0fbaa87e5a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0fbaa5f62a] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(solr_connection_post_more+0x249) [0x7f0fb8bef899] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x4a87) [0x7f0fb8beca87] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x339) [0x7f0fb9210dd9] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc570) [0x7f0fb9215570] -> doveadm() [0x412ce1] -> doveadm() [0x40fbd2] -> doveadm() [0x4105db] -> doveadm(doveadm_mail_try_run+0x161) [0x410a41] -> doveadm(main+0x375) [0x40f7e5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f0fba6dbead] -> doveadm() [0x40f9d1] Aborted It doesnt seems happened too often, but I've some mailboxes that cant be indexed
Re: [Dovecot] lmtp log message
On 28.7.2012, at 21.50, Bård Johannessen wrote: > On 2012-07-28 18:36, Timo Sirainen wrote: >> On 27.7.2012, at 9.01, Daniel L. Miller wrote: >>> Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client >>> quit (in reset) >>> in my log for every message. Is this a normal sign-off - or does this >>> indicate a misconfiguration? >> >> It's normal. > > I'm not even sure it's noteworthy enough to justify logging it; if the > message is going to be exactly the same in all (non error) cases, what's the > point of logging it? Since LMTP is supposed to be accessible only internally from your network, maybe there isn't much of a point.. Maybe if DATA command is used it wouldn't be logged, but otherwise a QUIT would log "Client quit without delivering mail". Would be useful to get some opinions from more people though.
Re: [Dovecot] lmtp log message
On 2012-07-28 18:36, Timo Sirainen wrote: On 27.7.2012, at 9.01, Daniel L. Miller wrote: Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? It's normal. I'm not even sure it's noteworthy enough to justify logging it; if the message is going to be exactly the same in all (non error) cases, what's the point of logging it? -- Med vennlig hilsen Bård Johannessen
Re: [Dovecot] expire-tool stops on first box with nothing to expire in it
On 19.7.2012, at 4.24, Joseph Tam wrote: > I previously observed that the "date.saved" field (tested by savedbefore) > seems to be clustered around particular timestamps. > > Dumping out this field using > > doveadm -ftab fetch -A \ > "mailbox date.saved" \ > \( mailbox Trash OR mailbox Junk OR mailbox "Deleted Items" \) > > it seems the timestamps do not actually correspond to when the client > moves messages to their trash folder, but rather, to the time I query > for it i.e. a query instantiates this value. This timestamp is then > retained. > > In practice, if this query is done on a periodic interval (e.g. daily), > the date.saved will be roughly correct, and is usable as an age test > for expungement. > > Is there a way to force this value to be cached when the client actually > saves or moves a message? Done for save: http://hg.dovecot.org/dovecot-2.1/rev/c598f76d Fixed copy: http://hg.dovecot.org/dovecot-2.1/rev/b13b8267999d It still doesn't add it to cache if it's not already cached in both the source and destination mailbox, but usually it should be after initial doveadm run.
Re: [Dovecot] [PATCH] Pop3/imap ordering in courier-dovecot-migrate.pl
On 18.7.2012, at 17.48, Pavel Stano wrote: > current version of courier migration script courier-dovecot-migrate.pl > preserve order of POP3 UIDLs, but it keep IMAP UIDs not in order which cause > problems with clients using IMAP along with POP3 in some cases. > > Attached patch use MAIL_FETCH_POP3_ORDER feature added in version 2.0.13 to > keep POP3 and IMAP UID[L]s in order. Applied. I had wanted to add that feature for a while now but never had time myself.
Re: [Dovecot] Disable W= in filenames
On 19.7.2012, at 21.18, Daniel Pieper wrote:
> I implemented a plugin for dovecot and now i got a error that maildir
> filename has wrong W value. When i rename the filename manually and delete
> the W value, all is ok.
Maybe your plugin should fix also the value? It sounds like you would also run
into trouble with dovecot.index.cache file. See zlib plugin as an example how
to fix the W value ("virtual size"). I think you can also disable it that way
by setting it to (uoff_t)-1.
Re: [Dovecot] Dsync replication
On 18.7.2012, at 19.24, l...@airstreamcomm.net wrote: >> Anyway, in my small installation I'm using it in active-active mode and it >> works well enough. I've even configured my clients intentionally so that >> they use different servers. > > Does dsync replication only work between two hosts? In my scenario I would > have two sites with X number of nodes at each with an NFS backend for each > site. For this example lets say I have site A with two nodes that mount one > NFS share, and site B with two nodes that mount one NFS share. Is it > possible to implement dsync replication between these two clusters of nodes? You can have as many hosts as you want, but you most likely don't want to use active-active setup via NFS or you'll run into NFS caching (= corruption) problems.
Re: [Dovecot] maildir_copy_with_hardlinks on v.2.0.19
On 23.7.2012, at 22.12, mailing list subscriber wrote:
> As requested, here is my update. As you can see I am running now the
> latest release however emails delivered through lmtp gets split into
> different files instead of expected hardlinked files.
..
> userdb {
> driver = passwd
> }
Looks like you're using system users. Each mail then needs to be written using
different permissions, so hard linking can't work.
Re: [Dovecot] get recipient entry while in the zlib plugin context
On 25.7.2012, at 22.31, Tofig Suleymanov wrote: > is it possible to get a recipient of an email while being in the zlib plugin > context ? Maybe. Depends on where exactly you want to get it. The recipient of the email would anyway be the struct mail_user->username however you can get it.
Re: [Dovecot] fts-lucene vs SEARCH HEADER
On 25.7.2012, at 5.39, Matthew Powell wrote: > It looks as though the header names are indexed in lower case. But Dovecot is > passing the header name to the backend in upper case when SEARCH HEADER is > used. > > I'm not sure whether this is the *right* fix, but lower-casing the header > name in lucene-wrapper.cc seems to fix the problem: Committed with another change: http://hg.dovecot.org/dovecot-2.1/rev/77f2510bb009
Re: [Dovecot] SSL Warnings in Debug Logs
On 24.7.2012, at 21.27, Asai wrote: > Greetings, > > In doing some debugging of authentication issues, I'm wondering if these SSL > warnings are anything to be investigating? > > Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x10, ret=1: > before/accept initialization [192.168.70.101] > Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, > ret=1: before/accept initialization [192.168.70.101] They should be debug messages, not warnings. Fixed in hg.
Re: [Dovecot] mysql auth failover failing
It's in my TODO, but I don't know when I'll get around to implementing it. So many things to do right now.. On 25.7.2012, at 14.47, Noel Butler wrote: > As per this discussion almost a year ago, was there any attempt to > introduce failover mode planned Timo? > > > On Mon, 2011-09-12 at 13:26 -0700, Paul B. Henson wrote: > > >> >> Perhaps it could be an option, either load balancing between all >> available servers, or only using later listed servers when the earlier >> listed ones are failing. For my purposes, either way is fine, as long as >> authentications don't fail :). The other contributor to this thread, who >> has a local mysql replica listed first and the central master listed >> second probably wouldn't want the load balanced between them. > >
Re: [Dovecot] Re move .INBOX from folders.
On 26.7.2012, at 22.38, ssukh wrote: > > Hello all, i am just new to dovecot and trying to figure out how to remove > .INBOX from the folders. > > Now all the folders are created like > .INBOX.spam > .INBOX.temp > > But i would like to have > .spam > .temp Either: a) Remove INBOX. namespace prefix from your client's configuration. or b) Set prefix=INBOX. to Dovecot's namespace configuration
Re: [Dovecot] Problem with running two dovecot instances on the same server
On 26.7.2012, at 5.27, Alexandr Sabitov wrote: > Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list > Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for > director_mail_servers setting > Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): > command startup failed, throttling for 60 secs Director process gets started for the backend installation where it of course isn't configured. My guess: doveadm (from some cronjob?) is attempting to connect to /var/run/dovecot/director-admin instead of /var/run/dovecot-proxy/director-admin.
Re: [Dovecot] Auto logging-out client
On 26.7.2012, at 16.58, Nicolás wrote: > I'm writing because of an issue I'm having with my Postfix-Dovecot > installation. Everything's working fine but when an IMAP client is idle for > some time, Dovecot automatically logs it out. > > Jul 26 14:01:16 mail dovecot: imap(nico...@devels.es): Disconnected: Logged > out bytes=1271/7439 IMAP client logs out. Dovecot doesn't automatically do anything. There's no way (even theoretically) to prevent this from Dovecot's side. > This entails errors when trying to send e-mails after this auto logout, such > as: > > host devels.es[91.215.158.237] said: >550-Please turn on SMTP Authentication in your mail client, or login to the >550-IMAP/POP3 server before sending your message. Your server supports pop3/imap-before-smtp kludge. I'd remove that entirely and require SMTP AUTH always.
Re: [Dovecot] Slightly OT - Winbind and remote authentication.
On 27.7.2012, at 20.01, Simon Brereton wrote: > I was doing some research on setting up a samba server for my internal > network (as an alternative to NAS). The setup looks easy enough, but > then I got to thinking - I already have an internet host with an mysql > DB that has usernames and passwords and it would be simpler all round > to reuse that as the authentication server. That server is also > running Dovecot, so I thought it might be simpler still to set up > samba to authenticate using IMAP (the mysql server is not accessible > on port 3306 anyway). A quick google resulted in a suggestion from > Robert (who by now must be on every single list I'm on or rather > vice-versa) about using winbind. I went off to the link he posted - > http://wiki.dovecot.org/Authentication/Mechanisms/Winbind?highlight=%28winbind%29 > and the problem is that seems to indicate bind and dovecot are on the > same server. That link is about Dovecot authenticating via Samba, not vice versa. > So, this not being the samba list notwithstanding, is it possible to > use a remote dovecot host as an authentication back-end on a local > samba installation? No idea, but I think at least some Samba auth mechanisms probably want to have access to the plaintext password.
Re: [Dovecot] lmtp log message
On 27.7.2012, at 9.01, Daniel L. Miller wrote: > I just tried switching to lmtp from lda. Messages are being delivered, but I > now see messages similar to: > > Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client > quit (in reset) > > in my log for every message. Is this a normal sign-off - or does this > indicate a misconfiguration? It's normal.
Re: [Dovecot] fts solr : out of memory
On 28.7.2012, at 19.09, Matthew Powell wrote: > This looks very much like an issue I've been having with fts-lucene. Indexing > works (or at least completes without errors) in 2.1.7, but runs out of memory > in 2.1.8. > > doveadm -v index -u username '*' crashes with: > > doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory .. > I've posted an example message that triggers the crash here: > https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test > case, but I've encountered the same behaviour with a real message. Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/cbd2b321a68f
Re: [Dovecot] fts solr : out of memory
Le 28/07/2012 18:09, Matthew Powell a écrit :
> On 2012-07-28, at 11.21, to...@starbridge.org wrote:
>
>> Fatal: master: service(indexer-worker): child 9582 returned error 83
>> (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may
>> need to increase it))
> This looks very much like an issue I've been having with fts-lucene. Indexing
> works (or at least completes without errors) in 2.1.7, but runs out of memory
> in 2.1.8.
>
> doveadm -v index -u username '*' crashes with:
>
> doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory
> doveadm(username): Error: Raw backtrace:
> /usr/lib/dovecot/libdovecot.so.0(+0x41d31) [0xb7699d31] ->
> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x47) [0xb7699e47] ->
> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb766c958] ->
> /usr/lib/dovecot/libdovecot.so.0(+0x5420e) [0xb76ac20e] ->
> /usr/lib/dovecot/libdovecot.so.0(+0x3e358) [0xb7696358] ->
> /usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x72) [0xb76967f2]
> -> /usr/lib/dovecot/libdovecot.so.0(buffer_append_space_unsafe+0x2b)
> [0xb769691b] -> /usr/lib/dovecot/libdovecot.so.0(charset_to_utf8+0xe8)
> [0xb768c3f8] ->
> /usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x3dd)
> [0xb767fa4d] ->
> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x11c)
> [0xb732881c] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xb256)
> [0xb732e256] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0xf)
> [0xb774754f] -> doveadm() [0x805736e] -> doveadm() [0x805422d] -> doveadm()
> [0x8054aad] -> doveadm(doveadm_mail_try_run+0x12e) [0x8054fae] ->
> doveadm(main+0x3e4) [0x805bf24] ->
> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb7527ca6] -> doveadm()
> [0x8053c11]
>
> The problem occurs repeatably with specific messages. It looks as though the
> crash happens in lib-charset/charset-iconv.c and the problem was introduced
> by this change: http://hg.dovecot.org/dovecot-2.1/rev/0fde692cb565. If I
> remove that one change from 2.1.8, I can index the affected messages again.
>
> I've posted an example message that triggers the crash here:
> https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test
> case, but I've encountered the same behaviour with a real message.
>
Hi Matthew,
thanks for your answer.
most of our email contains French accents, so may be it's related.
I've forgot to post the complete log message for this error earlier:
Jul 27 19:01:00 mx2 dovecot: indexer-worker(c...@crxx.fr): Fatal:
pool_system_realloc(268435456): Out of memory
Jul 27 19:01:00 mx2 dovecot: indexer-worker(c...@crxx.fr): Error: Raw
backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43e4a) [0x7fa49c7e4e4a]
-> /usr/lib/dovecot/libdovecot.so.0(+0x43e8e) [0x7fa49c7e4e8e] ->
/usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa49c7bc784] ->
/usr/lib/dovecot/libdovecot.so.0(+0x542f0) [0x7fa49c7f52f0] ->
/usr/lib/dovecot/libdovecot.so.0(+0x40fe5) [0x7fa49c7e1fe5] ->
/usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x6b)
[0x7fa49c7e297b] ->
/usr/lib/dovecot/libdovecot.so.0(charset_to_utf8+0xe7) [0x7fa49c7d9867]
->
/usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x3b8)
[0x7fa49c7ce9d8]
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x194)
[0x7fa49af6dc34] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc570) [0x7fa49af72570]
-> dovecot/indexer-worker() [0x402544] ->
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7fa49c7f1b96]
-> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7)
[0x7fa49c7f2827] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28)
[0x7fa49c7f15a8] ->
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)
[0x7fa49c7de2c3] -> dovecot/indexer-worker(main+0xe2) [0x402002] ->
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fa49c438ead]
-> dovecot/indexer-worker() [0x4020a1]
Jul 27 19:01:00 mx2 dovecot: indexer: Error: Indexer worker
disconnected, discarding 1 requests for c...@crxx.fr
Jul 27 19:01:00 mx2 dovecot: imap(c...@crxx.fr): Error: indexer failed to
index mailbox INBOX
Jul 27 19:01:00 mx2 dovecot: indexer-worker(cxx@crxx): Fatal: master:
service(indexer-worker): child 9582 returned error 83 (Out of memory
(service indexer-worker { vsz_limit=256 MB }, you may need to increase it))
Hope this helps !
Re: [Dovecot] fts solr : out of memory
On 2012-07-28, at 11.21, to...@starbridge.org wrote:
> Fatal: master: service(indexer-worker): child 9582 returned error 83
> (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may
> need to increase it))
This looks very much like an issue I've been having with fts-lucene. Indexing
works (or at least completes without errors) in 2.1.7, but runs out of memory
in 2.1.8.
doveadm -v index -u username '*' crashes with:
doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory
doveadm(username): Error: Raw backtrace:
/usr/lib/dovecot/libdovecot.so.0(+0x41d31) [0xb7699d31] ->
/usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x47) [0xb7699e47] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb766c958] ->
/usr/lib/dovecot/libdovecot.so.0(+0x5420e) [0xb76ac20e] ->
/usr/lib/dovecot/libdovecot.so.0(+0x3e358) [0xb7696358] ->
/usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x72) [0xb76967f2] ->
/usr/lib/dovecot/libdovecot.so.0(buffer_append_space_unsafe+0x2b) [0xb769691b]
-> /usr/lib/dovecot/libdovecot.so.0(charset_to_utf8+0xe8) [0xb768c3f8] ->
/usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x3dd)
[0xb767fa4d] ->
/usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x11c) [0xb732881c]
-> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xb256) [0xb732e256] ->
/usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0xf) [0xb774754f] ->
doveadm() [0x805736e] -> doveadm() [0x805422d] -> doveadm() [0x8054aad] ->
doveadm(doveadm_mail_try_run+0x12e) [0x8054fae] -> doveadm(main+0x3e4)
[0x805bf24] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb7527ca6] ->
doveadm() [0x8053c11]
The problem occurs repeatably with specific messages. It looks as though the
crash happens in lib-charset/charset-iconv.c and the problem was introduced by
this change: http://hg.dovecot.org/dovecot-2.1/rev/0fde692cb565. If I remove
that one change from 2.1.8, I can index the affected messages again.
I've posted an example message that triggers the crash here:
https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test case,
but I've encountered the same behaviour with a real message.
--
Matthew Powell matt...@atom.net
[Dovecot] fts solr : out of memory
Hi Timo,
since I've upgraded to 2.1.8, solr indexer doesnt work correctly.
I see a lot of these errors in logs:
Fatal: master: service(indexer-worker): child 9582 returned error 83
(Out of memory (service indexer-worker { vsz_limit=256 MB }, you may
need to increase it))
I've tried to set vsz_limit to a lot more, (around 4096 MB) but same
problem occurs.
thank for your help !
Regards
Tonio
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
doveconf -n
# 2.1.8: /etc/dovecot/dovecot.conf
# OS: Linux 3.2.13--std-ipv6-64 x86_64 Debian wheezy/sid xfs
auth_cache_negative_ttl = 0
auth_cache_size = 10 M
auth_mechanisms = plain login
dict {
quota = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf
}
hostname = maildir2.spamguard.fr
log_timestamp = %Y-%m-%d %H:%M:%S
mail_gid = vmail
mail_location = sdbox:/home/virtual/%u
mail_plugins = quota fts fts_solr acl zlib
mail_privileged_group = vmail
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = comparator-i;octet
comparator-i;ascii-casemap fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex
imap4flags copy include variables body enotify environment mailbox date
spamtest spamtestplus virustest
namespace {
inbox = yes
location =
prefix =
separator = /
type = private
}
namespace {
alias_for =
hidden = yes
inbox = no
list = no
location =
prefix = INBOX/
separator = /
type = private
}
namespace {
list = children
location = sdbox:/home/virtual/%%u:INDEX=/home/virtual/%u/shared/%%u
prefix = shared/%%u/
separator = /
subscriptions = no
type = shared
}
passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
plugin {
acl = vfile
acl_shared_dict = file:/home/virtual/shared-%d-mailboxes.db
autocreate = Spam
autocreate2 = SpamFalse
autocreate3 = SpamToLearn
autocreate4 = Sent
autosubscribe = Spam
autosubscribe2 = SpamFalse
autosubscribe3 = SpamToLearn
autosubscribe4 = Sent
fts = solr
fts_solr = url=http://localhost:8081/solr/
mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename save mailbox_create
mail_log_fields = uid box msgid size
quota = dict:User quota::proxy::quota
quota2 = dict:User quota::file:/home/virtual/%u/dovecot-quota
quota_warning = storage=85%% quota-warning 85 %u
sieve = ~/dovecot.sieve
sieve_before = /home/virtual/sieve/
sieve_dir = ~/sieve
zlib_save = bz2
zlib_save_level = 9
}
protocols = imap pop3 sieve lmtp
service auth {
unix_listener /var/spool/postfix-in2/private/auth {
group = mail
mode = 0660
user = postfix
}
unix_listener /var/spool/postfix-maildir2/private/auth {
group = mail
mode = 0660
user = postfix
}
unix_listener /var/spool/postfix-msa/private/auth {
group = mail
mode = 0660
user = postfix
}
unix_listener /var/spool/postfix-smtp06/private/auth {
group = mail
mode = 0660
user = postfix
}
unix_listener /var/spool/postfix/private/auth {
group = mail
mode = 0660
user = postfix
}
unix_listener auth-in2-master {
mode = 0660
user = vmail
}
unix_listener auth-maildir2-master {
mode = 0660
user = vmail
}
unix_listener auth-master {
mode = 0660
user = vmail
}
unix_listener auth-msa-master {
mode = 0660
user = vmail
}
unix_listener auth-out-master {
mode = 0660
user = vmail
}
unix_listener auth-smtp06-master {
mode = 0660
user = vmail
}
unix_listener auth-userdb {
mode = 0660
user = vmail
}
}
service dict {
unix_listener dict {
mode = 0600
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
}
process_limit = 256
process_min_avail = 4
service_count = 0
vsz_limit = 256 M
}
service lmtp {
inet_listener lmtp {
address = 46.105.108.210
port = 24
}
user = vmail
vsz_limit = 368 M
}
service managesieve-login {
inet_listener sieve {
port = 4190
}
process_limit = 256
vsz_limit = 64 M
}
service pop3-login {
inet_listener pop3 {
port = 110
}
inet_listener pop3s {
port = 995
}
process_limit = 256
process_min_avail = 4
service_count = 0
vsz_limit = 256 M
}
service quota-warning {
executable = script /usr/bin/quota-warning.sh
unix_listener quota-warning {
mode = 0600
user = vmail
}
user = vmail
}
ssl = required
ssl_ca =
Re: [Dovecot] Problem overriding default quota limit
Ok, I'll do it next monday. Thank you very much!!! A las Sábado, 28-07-2012 en 4:34 Daniel Parthey escribió: Hello Raul, Raul Jareño Morago wrote: > We get user's quota limit from LDAP. It works fine when user has > quota attribute in LDAP (this is when it's different from default > quota). But when user doesn't have quota limit attribute Dovecot > doesn't take default value defined in quota plugin and assign unlimit > quota to user. Your version 2.1.1 is affected by a bug which has been fixed in newer Dovecot Releases: http://hg.dovecot.org/dovecot-2.1/rev/4bbc12a87a29 Also see http://dovecot.org/doc/NEWS-2.1 v2.1.7 2012-05-29 Timo Sirainen * LDAP: Compatibility fix for v2.0: ldap: If attributes contain ldapAttr=key=template%$ and ldapAttr doesn't exist, skip the key instead of using "template" value with empty %$ part for the key. Please update to latest stable release 2.1.8 which contains a lot of important fixes. Regards Daniel -- https://plus.google.com/103021802792276734820
