Re: [Dovecot] dovecot & OCFS2 Cluster
On 16/05/2011 13:46, Timo Sirainen wrote: > On Mon, 2011-05-16 at 14:08 +0200, Osvaldo Alvarez Pozo wrote: > >> But if I erase mail from Mozilla Thunderbird then I find no mail with >> ref count=0 and doveadm purge does nothing. > > doveadm purge only purges mails that have been expunged. So I guess your > TB has only marked mails \Deleted, but not actually expunged anything. Yes, Thunderbird's default is to only mark as deleted (true of many clients I think?). There are some options to encourage it to expunge "more frequently". Also (and you need EVERY user to change this..) you can set your delete action to (say) move the mails to a Trash folder. I think this (mark, then expunge) behaviour is very alien to most email users, it's just not how any other MUA actually behaves (at best they all hide the imap behaviour). There are some features in dovecot to help hide this, but I think it would actually be quite helpful to have a Dovecot feature which literally intercepts an MUA marking messages deleted and whips them into the Trash folder..? Anyway, just a wierdness in IMAP which I think really hasn't worked out as the model of how the rest of the world decided to work. I personally desire this behaviour to go away since it's confusing to 99% of non technical users... Cheers Ed W
Re: [Dovecot] dovecot & OCFS2 Cluster
Yeah, looks right. And once you've migrated everyone, change mail_location globally and remove them from userdb. On Fri, 2011-05-13 at 08:46 +0200, Osvaldo Alvarez Pozo wrote: > Ok I want to do migration without downtime > > Yes i have red that the separator depends on mailbox format and that > for maildir it is '.' > > So in my dovecot.conf I have no namespace definition and my mail-location is: > > mail_location = maildir:/var/vmail/%d/%u/Maildir > > But when I execute the dsync command: > > /usr/local/dovecot/bin/dsync -v -f -u exam...@example.org mirror > mdbox:/var1/vmail/example.org/example/mdbox > > i get > > dsync(root): Fatal: Mail locations must use the same virtual mailbox > hierarchy separator (specify separator for the default namespace) > > > So i have to add to my dovecot.conf > namespace { > inbox = yes > separator = . > subscriptions = yes > } > > And after finishig dsync for all mailboxes to a new location > I change separator to: > > separator = / > > as a final step I have to return the new mdbox mail location for > each user using userdb > > and my dsync command would change: > > /usr/local/dovecot/bin/dsync -v -f -u exam...@example.org mirror > maildir:/var/vmail/example.org/example/Maildir > > Am I rigth? > > Thanks you all > > > 2011/5/13 Timo Sirainen : > > On 13.5.2011, at 1.00, Osvaldo Alvarez Pozo wrote: > > > >> Actually I have no namespace definition. If i add namespace definition > >> for start migrating, waht about my clients? will they be able to find > >> there mail & mailboxes while I am migrating from maildir to mdbox? > > > > Your clients see the default namespace if it's not explicitly specified. > > That namespace has prefix="" and the separator depends on what mailbox > > format you're using. With Maildir that would be "." character. If you keep > > that there shouldn't be even any potential problems. > > > > > > >
Re: [Dovecot] dovecot & OCFS2 Cluster
On Mon, 2011-05-16 at 14:08 +0200, Osvaldo Alvarez Pozo wrote: > But if I erase mail from Mozilla Thunderbird then I find no mail with > ref count=0 and doveadm purge does nothing. doveadm purge only purges mails that have been expunged. So I guess your TB has only marked mails \Deleted, but not actually expunged anything.
Re: [Dovecot] dovecot & OCFS2 Cluster
Hi I have done some test. If I use a webmail as roundcube to read mail mails, when I erase a mail from Inbox and after that from trash.the command. /usr/local/dovecot/bin/doveadm dump /var/vmail1/example.org/user/mdbox/storage/|grep 'ref.*\b0\b' show mail with ref count=0 - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () - ext 3 ref : 0 () But if I erase mail from Mozilla Thunderbird then I find no mail with ref count=0 and doveadm purge does nothing. It is strange I thought thunderbird was a good mail client. My thunderbird is up to date. Thanks 2011/5/16 Pascal Volk : > On 05/16/2011 12:37 AM Osvaldo Alvarez Pozo wrote: >> Hi >> thanks but I do not have doveadm-purge. I have doveadm and I can run >> it with -purge option and that's wath I did with no result. >> > > Please stop Top-posting. > > OK, if man 1 doveadm-purge didn't work for you, run doveadm purge help. > > You may check if there are some deleted mails in the mdbox: > doveadm dump /path/2/mdbox/storage | egrep 'ext 3 ref\s+\:\s+0' -c > If the printed count is greater then 0, there are deleted mails in the > mdbox. Then you can delete them permanently with doveadm purge … > > > Regards, > Pascal > -- > The trapper recommends today: fabaceae.1113...@localdomain.org >
Re: [Dovecot] dovecot & OCFS2 Cluster
On 05/16/2011 12:37 AM Osvaldo Alvarez Pozo wrote: > Hi > thanks but I do not have doveadm-purge. I have doveadm and I can run > it with -purge option and that's wath I did with no result. > Please stop Top-posting. OK, if man 1 doveadm-purge didn't work for you, run doveadm purge help. You may check if there are some deleted mails in the mdbox: doveadm dump /path/2/mdbox/storage | egrep 'ext 3 ref\s+\:\s+0' -c If the printed count is greater then 0, there are deleted mails in the mdbox. Then you can delete them permanently with doveadm purge … Regards, Pascal -- The trapper recommends today: fabaceae.1113...@localdomain.org
Re: [Dovecot] dovecot & OCFS2 Cluster
Hi thanks but I do not have doveadm-purge. I have doveadm and I can run it with -purge option and that's wath I did with no result. Thanks 2011/5/16 Pascal Volk : > On 05/16/2011 12:29 AM Osvaldo Alvarez Pozo wrote: >> Hi >> I am testing mdbox. >> … >> How can I erase deleted mails? >> > > See doveadm-purge(1) > > > Regards, > Pascal > -- > The trapper recommends today: fabaceae.1113...@localdomain.org >
Re: [Dovecot] dovecot & OCFS2 Cluster
On 05/16/2011 12:29 AM Osvaldo Alvarez Pozo wrote: > Hi > I am testing mdbox. > … > How can I erase deleted mails? > See doveadm-purge(1) Regards, Pascal -- The trapper recommends today: fabaceae.1113...@localdomain.org
Re: [Dovecot] dovecot & OCFS2 Cluster
Hi I am testing mdbox. I noticed that after deleting mail from inbox and trash the mail are always in the mdbox I mean it size has not changed. I executed: /usr/local/dovecot/bin/doveadm -Dv purge -u u...@example.org but nothing changed the filess size have not changed I did /usr/local/dovecot/bin/doveadm search all -u u...@example.org 4673461242fdcb4d8d094ccdb073 1 4673461242fdcb4d8d094ccdb073 2 4673461242fdcb4d8d094ccdb073 3 4673461242fdcb4d8d094ccdb073 4 4673461242fdcb4d8d094ccdb073 5 52dfc91bb54cd04df30b4ccdb073 1 52dfc91bb54cd04df30b4ccdb073 2 52dfc91bb54cd04df30b4ccdb073 3 52dfc91bb54cd04df30b4ccdb073 4 52dfc91bb54cd04df30b4ccdb073 5 52dfc91bb54cd04df30b4ccdb073 6 52dfc91bb54cd04df30b4ccdb073 7 52dfc91bb54cd04df30b4ccdb073 8 52dfc91bb54cd04df30b4ccdb073 9 52dfc91bb54cd04df30b4ccdb073 10 52dfc91bb54cd04df30b4ccdb073 11 52dfc91bb54cd04df30b4ccdb073 12 52dfc91bb54cd04df30b4ccdb073 13 So all files are there rigth? How can I erase deleted mails? Thanks 2011/5/13 Timo Sirainen : > On 13.5.2011, at 1.00, Osvaldo Alvarez Pozo wrote: > >> Actually I have no namespace definition. If i add namespace definition >> for start migrating, waht about my clients? will they be able to find >> there mail & mailboxes while I am migrating from maildir to mdbox? > > Your clients see the default namespace if it's not explicitly specified. That > namespace has prefix="" and the separator depends on what mailbox format > you're using. With Maildir that would be "." character. If you keep that > there shouldn't be even any potential problems. > > >
Re: [Dovecot] dovecot & OCFS2 Cluster
Am 13.05.2011 08:46, schrieb Osvaldo Alvarez Pozo: > Ok I want to do migration without downtime > > Yes i have red that the separator depends on mailbox format and that > for maildir it is '.' > > So in my dovecot.conf I have no namespace definition and my mail-location is: > > mail_location = maildir:/var/vmail/%d/%u/Maildir > > But when I execute the dsync command: > > /usr/local/dovecot/bin/dsync -v -f -u exam...@example.org mirror > mdbox:/var1/vmail/example.org/example/mdbox > > i get > > dsync(root): Fatal: Mail locations must use the same virtual mailbox > hierarchy separator (specify separator for the default namespace) > > > So i have to add to my dovecot.conf > namespace { > inbox = yes > separator = . > subscriptions = yes > } > > And after finishig dsync for all mailboxes to a new location > I change separator to: > > separator = / > > as a final step I have to return the new mdbox mail location for > each user using userdb > > and my dsync command would change: > > /usr/local/dovecot/bin/dsync -v -f -u exam...@example.org mirror > maildir:/var/vmail/example.org/example/Maildir > > Am I rigth? > > Thanks you all that doesnt look bad, but i am not sure wait for others comments ( like Timo ) > > > 2011/5/13 Timo Sirainen : >> On 13.5.2011, at 1.00, Osvaldo Alvarez Pozo wrote: >> >>> Actually I have no namespace definition. If i add namespace definition >>> for start migrating, waht about my clients? will they be able to find >>> there mail & mailboxes while I am migrating from maildir to mdbox? >> >> Your clients see the default namespace if it's not explicitly specified. >> That namespace has prefix="" and the separator depends on what mailbox >> format you're using. With Maildir that would be "." character. If you keep >> that there shouldn't be even any potential problems. >> >> >> however , i am thinking about migrate to mdbox too, on ocfs2, looks like the store is getting slow with thousends of maildirs, there may other reason for that too, depending hardware ( raid etc ), but current i only see the migrate option to speed up, so i am interested in the results of your migration ,please report -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria
Re: [Dovecot] dovecot & OCFS2 Cluster
Ok I want to do migration without downtime Yes i have red that the separator depends on mailbox format and that for maildir it is '.' So in my dovecot.conf I have no namespace definition and my mail-location is: mail_location = maildir:/var/vmail/%d/%u/Maildir But when I execute the dsync command: /usr/local/dovecot/bin/dsync -v -f -u exam...@example.org mirror mdbox:/var1/vmail/example.org/example/mdbox i get dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) So i have to add to my dovecot.conf namespace { inbox = yes separator = . subscriptions = yes } And after finishig dsync for all mailboxes to a new location I change separator to: separator = / as a final step I have to return the new mdbox mail location for each user using userdb and my dsync command would change: /usr/local/dovecot/bin/dsync -v -f -u exam...@example.org mirror maildir:/var/vmail/example.org/example/Maildir Am I rigth? Thanks you all 2011/5/13 Timo Sirainen : > On 13.5.2011, at 1.00, Osvaldo Alvarez Pozo wrote: > >> Actually I have no namespace definition. If i add namespace definition >> for start migrating, waht about my clients? will they be able to find >> there mail & mailboxes while I am migrating from maildir to mdbox? > > Your clients see the default namespace if it's not explicitly specified. That > namespace has prefix="" and the separator depends on what mailbox format > you're using. With Maildir that would be "." character. If you keep that > there shouldn't be even any potential problems. > > >
Re: [Dovecot] dovecot & OCFS2 Cluster
On 13.5.2011, at 1.00, Osvaldo Alvarez Pozo wrote: > Actually I have no namespace definition. If i add namespace definition > for start migrating, waht about my clients? will they be able to find > there mail & mailboxes while I am migrating from maildir to mdbox? Your clients see the default namespace if it's not explicitly specified. That namespace has prefix="" and the separator depends on what mailbox format you're using. With Maildir that would be "." character. If you keep that there shouldn't be even any potential problems.
Re: [Dovecot] dovecot & OCFS2 Cluster
On 5/12/2011 6:00 PM, Osvaldo Alvarez Pozo wrote: Hi all I migrated from dovecot 1.2 to dovecot 2.0 Next step" migrate from maildir to mdbox" I have being reading about migrating to mdbox as Timo pointed me out. But I have douths with namespace. I tried to use dsync and I got a error: dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) Please provide the exact command line that resulted in this error. I think It could be solved adding: namespace { inbox = yes separator = / subscriptions = yes } Actually I have no namespace definition. If i add namespace definition for start migrating, waht about my clients? will they be able to find there mail& mailboxes while I am migrating from maildir to mdbox? Is this http://wiki2.dovecot.org/Tools/Dsync the documentation you read? -- Stan
Re: [Dovecot] dovecot & OCFS2 Cluster
Hi all I migrated from dovecot 1.2 to dovecot 2.0 Next step" migrate from maildir to mdbox" I have being reading about migrating to mdbox as Timo pointed me out. But I have douths with namespace. I tried to use dsync and I got a error: dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) I think It could be solved adding: namespace { inbox = yes separator = / subscriptions = yes } Actually I have no namespace definition. If i add namespace definition for start migrating, waht about my clients? will they be able to find there mail & mailboxes while I am migrating from maildir to mdbox? Thanks. 2011/5/9 Timo Sirainen : > On Mon, 2011-04-25 at 19:02 +0200, Osvaldo Alvarez Pozo wrote: >> hi all >> >> We have an ocfs2 cluster compose of 4 Debian lenny serveurs wich have >> access to an ISCSI LUN we have create a partition on this Lun and >> formated this partition as OCFS2. >> 2 serveurs does mail delivery (SMTP) and the two other are pop/imap >> servers. The smtp servers use dovecot LDA to deliver to mailboxes. La >> version de dovecot is 1.2.13, ldirectord balance conexion to the >> servers(pop/pops/imap/imaps to the two imap/pop serveurs) (smtp to the >> two smtp servers). > > With v2.0 I'd suggest using http://wiki2.dovecot.org/Director > >> mail_location = maildir:/var/vmail/%d/%u/Maildir > > Cluster filesystems don't seem to like Maildir much. With v2.0 you could > use mdbox. > >> We are having severals problems from "cpu stuck for 61 seconds" to >> "too many connexion". > > I don't know what either of those errors mean. In any case you should > switch to high-performance login processes: > http://wiki.dovecot.org/LoginProcess > > http://wiki.dovecot.org/PerformanceTuning has more stuff > > >
Re: [Dovecot] dovecot & OCFS2 Cluster
On Mon, 2011-04-25 at 19:02 +0200, Osvaldo Alvarez Pozo wrote: > hi all > > We have an ocfs2 cluster compose of 4 Debian lenny serveurs wich have > access to an ISCSI LUN we have create a partition on this Lun and > formated this partition as OCFS2. > 2 serveurs does mail delivery (SMTP) and the two other are pop/imap > servers. The smtp servers use dovecot LDA to deliver to mailboxes. La > version de dovecot is 1.2.13, ldirectord balance conexion to the > servers(pop/pops/imap/imaps to the two imap/pop serveurs) (smtp to the > two smtp servers). With v2.0 I'd suggest using http://wiki2.dovecot.org/Director > mail_location = maildir:/var/vmail/%d/%u/Maildir Cluster filesystems don't seem to like Maildir much. With v2.0 you could use mdbox. > We are having severals problems from "cpu stuck for 61 seconds" to > "too many connexion". I don't know what either of those errors mean. In any case you should switch to high-performance login processes: http://wiki.dovecot.org/LoginProcess http://wiki.dovecot.org/PerformanceTuning has more stuff
Re: [Dovecot] dovecot & OCFS2 Cluster
Am 27.04.2011 00:50, schrieb Joseph Tam: > Robert Schetterer writes: > >>> Anyway, why use lmtp over lda ? >> >> i.e avoid backscatter by overquota > > A clarification on this: backscatter (or rather, ambiguous NDR notices > that claim many users are over quota rather than just one) is a result > of a local mailer that support multiple recipient delivery in one > invocation, but can only return one result code to the MTA. > > Since Dovecot's deliver-lda does not support multiple recipient delivery, > this is not a problem. > > LMTP can support multiple deliveries of the same message per invocation, > but replies with individual SMTP codes to avoid the above problem. > It can take advantage of this to efficiently implement single instance > storage if it is enabled. in my understanding lmtp acts more like a mta so overquota "reject" can be send during smtp income stage i.e with postfix, to the sender client with lda this isnt possible cause the the mta does give the whole mail to lda which and mta send "recieved" to the sender client ( so the bounce must be send later by binary sendmail etc ) whatever this is the behavior i ve tested in my setups, may Timo correct me if i fail here > >> performance should better too > > Apart from multiple deliveries or SIS, I'm not sure this is true, or > significant if it were. yes this should be meassured, youre right i dont know too if its really performance relevant, but multiple deliver is nice to have anyway, why should not use it > > Joseph Tam however i see no reason to use lda longer in my setups as lmtp exits youre right the backscatter case is really rare anyway -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria
Re: [Dovecot] dovecot & OCFS2 Cluster
Hi all I decided to upgrade to 2.0 i compiled dovecot with ssl and mysql, but i have problems with lmtp configuration. this is the error. pr 27 01:04:36 debian dovecot: auth: Fatal: net_connect_unix(auth-worker) in directory /usr/local/dovecot/var/run/dovecot failed: Permission denied (euid=65534(nobody) egid=65534(nogroup) missing +r perm: auth-worker, euid is not dir owner) Apr 27 01:04:36 debian dovecot: lmtp(25961): Error: userdb lookup(u...@exemple.org): Disconnected unexpectedly Apr 27 01:04:36 debian dovecot: master: Error: service(auth): command startup failed, throttling dovecot.conf auth_mechanisms = plain login disable_plaintext_auth = no first_valid_uid = 5000 last_valid_uid = 5000 mail_access_groups = mail mail_location = maildir:/var/vmail/%d/%u/Maildir ssl = no passdb { args = /etc/dovecot/sql.conf driver = sql } plugin { quota = maildir:User quota } protocols = imap pop3 lmtp service auth { unix_listener auth-userdb { group = mail mode = 0660 user = vmail } unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } user = nobody } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } userdb { driver = prefetch } userdb { args = /etc/dovecot/sql.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } !include conf.d/*.conf 20-lmtp.conf protocol lmtp { # Space separated list of plugins to load (default is global mail_plugins). #mail_plugins = $mail_plugins postmaster_address = ad...@exemple.org } I think is a permisson problem, in the conf i set group = mail mode = 0660 user = vmail i should not have problems, rigth? Thanks you all 2011/4/27 Joseph Tam : > Robert Schetterer writes: > >>> Anyway, why use lmtp over lda ? >> >> i.e avoid backscatter by overquota > > A clarification on this: backscatter (or rather, ambiguous NDR notices > that claim many users are over quota rather than just one) is a result > of a local mailer that support multiple recipient delivery in one > invocation, but can only return one result code to the MTA. > > Since Dovecot's deliver-lda does not support multiple recipient delivery, > this is not a problem. > > LMTP can support multiple deliveries of the same message per invocation, > but replies with individual SMTP codes to avoid the above problem. > It can take advantage of this to efficiently implement single instance > storage if it is enabled. > >> performance should better too > > Apart from multiple deliveries or SIS, I'm not sure this is true, or > significant if it were. > > Joseph Tam >
Re: [Dovecot] dovecot & OCFS2 Cluster
Robert Schetterer writes: Anyway, why use lmtp over lda ? i.e avoid backscatter by overquota A clarification on this: backscatter (or rather, ambiguous NDR notices that claim many users are over quota rather than just one) is a result of a local mailer that support multiple recipient delivery in one invocation, but can only return one result code to the MTA. Since Dovecot's deliver-lda does not support multiple recipient delivery, this is not a problem. LMTP can support multiple deliveries of the same message per invocation, but replies with individual SMTP codes to avoid the above problem. It can take advantage of this to efficiently implement single instance storage if it is enabled. performance should better too Apart from multiple deliveries or SIS, I'm not sure this is true, or significant if it were. Joseph Tam
Re: [Dovecot] dovecot & OCFS2 Cluster
Am 25.04.2011 19:02, schrieb Osvaldo Alvarez Pozo: > hi all > > We have an ocfs2 cluster compose of 4 Debian lenny serveurs wich have > access to an ISCSI LUN we have create a partition on this Lun and > formated this partition as OCFS2. > 2 serveurs does mail delivery (SMTP) and the two other are pop/imap > servers. The smtp servers use dovecot LDA to deliver to mailboxes. La > version de dovecot is 1.2.13, ldirectord balance conexion to the > servers(pop/pops/imap/imaps to the two imap/pop serveurs) (smtp to the > two smtp servers). > > > this is a part of my dovecot.conf > > mail_location = maildir:/var/vmail/%d/%u/Maildir > mail_access_groups = mail > first_valid_uid = 5000 > last_valid_uid = 5000 > mail_max_userip_connections = 100 > maildir_copy_with_hardlinks = yes > disable_plaintext_auth = no > dotlock_use_excl=yes > mmap_disable = yes > protocols = imap imaps pop3 pop3s managesieve > protocol imap { > mail_plugins = quota imap_quota > imap_client_workarounds = outlook-idle delay-newmail > } > protocol pop3 { > mail_plugins = quota > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv > } > protocol lda { > mail_plugins = quota sieve > log_path = /var/log/dovecot-deliver.log > info_log_path = /var/log/dovecot-deliver.log > } > protocol managesieve { > managesieve_implementation_string = dovecot > } > auth default { > mechanisms = plain login > passdb sql { > args = /etc/dovecot/sql.conf > } > userdb prefetch { > } > userdb sql { > args = /etc/dovecot/sql.conf > } > user = nobody > socket listen { > master { > path = /var/run/dovecot/auth-master > mode = 0660 > user = vmail > group = mail > } > client { > path = /var/spool/postfix/private/auth > mode = 0660 > user = postfix > group = mail > } > > We are having severals problems from "cpu stuck for 61 seconds" to > "too many connexion". > > I wonder if there is a configuation type of dovecot when using ocfs2. > In tje dovecot site they give advices about how to configure dovecot > when using NFS, but not about OCFS2. > Somme people say that lda shuld not be used. > Any help please? > > Thanks > nebano first upgrade your dovecot and i.e postfix to latest versions use lmtp instead of lda read http://wiki1.dovecot.org/MailLocation/SharedDisk look archives here i use 5000 mailboxes on maildir with drbd ocfs2 ,no problem so far there is an option mail if you format with ocfs2 look bottlenecks with your ISCSI as the nature of the beast there may be several reasons for your problem but there is no known reason or bug to me why you shouldnt use ocfs2 you need to make more analysis -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria