Re: Dovecot Proxy
Hi Thoralf, I’d say first of all you should read the current docs for 2.x not the archived stuff. —> https://wiki2.dovecot.org/ - (It’s even mentioned in bold in the header) Then to front multiple backends perhaps you want to take a look at Dovecot Director. —> https://wiki2.dovecot.org/Director About SMTP I’m not sure why you would want to rely on Dovecot for that. I only do Postfix with Dovecot as auth backend so they can share passdb access. When you have 465 set up it is no big deal to also enable 587 in Postfixs master.cf. If you want to keep Dovecot for Submission you can check the latest docs for Dovecot submission service: https://doc.dovecot.org/admin_manual/submission_server/. It has a relay server option with port. Also settings for STARTTLS etcpp can be found there. Mahlzeit! Philon > On 31 Aug 2020, at 11:33, Thoralf Rickert-Wendt wrote: > > Hello everyone, > > it's my first post here on this mailing list and I hope, I make it right. > > I posted a question on > https://serverfault.com/questions/1031441/dovecot-as-proxy-with-submission > and nobody was able to answer it. So I decided to push that question here > (I'm talking about any new dovecot version and I've tested it with 2.3.4.1 > (f79e8e7e4)). > > I try to run a dovecot proxy in front of a big number of mail servers > (serving SMTP-in, submission, IMAP, POP3, Sieve). I need that proxy, because > I run out of IPv4 addresses. Of course I use IPv6 too, but many customers > still have problems with there providers and they really don't want to share > their mails on a "shared-mailserver". I planed to use Dovecot for IMAPS, > POP3S, SMTP-submission(465) and postfix for the rest. If I find a solution > for sieve, I would try that too, but that is very optional. > > With the documentation https://wiki1.dovecot.org/HowTo/ImapProxy (which is > really old and should be updated) and some other ascii docs (from an Apple > mirror somewhere deep in the web) I was able to build a IMAP/POP3 proxy that > forwards requests from outside to a specific backend using SSL (993,995). > That works - I think.You can find the config on the serverfault page. > > In general - all known domains in backend are using SSL and the passdb > forwards all requests to the backend via SSL. So - I understand:|| > > |password_query = > SELECT > NULL AS password, > NULL AS destuser, > host, > 'Y' AS nologin, > 'Y' AS nodelay, > 'Y' AS nopassword, > 'Y' AS proxy, > 'any-cert' AS `ssl` > FROM > proxy_domain > WHERE > domain = '%d' | > > But that is only 50% of the show. The rest ist submission (and maybe sieve). > Practically the submission implementation in dovecot works too. But because > dovecot by default only opens port 587 (starttls), my passdb setting has a > problem. > > When I try to use that port Dovecot tries to use SSL on the backend/587 too - > but that is wrong (it should either use 465 or should try to use starttls). > > So, I have the following options. > > - find a way to configure dovecot-proxy to listen on 465 with SSL for > submission service and hope that it uses the same port > - but I didn't find any documentation for that and need help > > - find a way to configure dovecot-proxy/passdb to return starttls=y when > dovecot-submission is used (use a different passdb) > - but I didn't find any documentation for that and I'm not sure, if this > worls on service/protocol level > > - find a way to configure the passdb answer based on the used port/protocol. > But I only know the parameter %u, %d and %p. > - so it would be nice to find a way to also select the protocol (if already > developed) > > - find a way to make a patch in dovecot (which isn't easy for me, because I > don't really know the code) > > Has somebody an idea, how I can configure the dovecot-proxy in that way. > > bye > Thoralf > >
Re: Strategy for fts and Replication
Hi Francis, My Solr instance is on 1GB but using less than 512MB. You might need to adjust Java VM memory usage but it's possible. I have only my own email but also 10-15 years history and search results including headers and body are instant. Things are on SSD but still I think the search storage fits into memory. Philon Am 04.02.2020 11:46, schrieb Francis Augusto Medeiros-Logeay: Hi Philon, Thanks a lot for your thoughts! Can I ask you if using Solr improved things for you? I have a mailbox with 15 years of e-mail and searching things take a long time. On 04.02.2020 09:39, Philon wrote: Hi Francis, next to fts-solr there was fts-lucene. But that Lucene there seems heavily outdated why the Dovecot docs also suggest using Solr. Elasticsearch probably is similar to Solr but the later is maintained by Dovecot team. I started with downloading the Solr binary distribution to Debian with JRE preinstalled and things were running like after 10 min. Yes it’s a bit more complicated to find the schema and edit things like header size (in tips section). It’s running quite nicely since then and has zero maintenance. I will try again - I kept getting some weird errors, so I don't know if that's why I wasn't seing much of improvement. As FTS indexes are separate in external Solr instance I’d guess that it won’t interfere with dsync. What I don’t know is if dsync’ing would trigger indexing. This brings me to wonder how one could actually replicate the Solr instance!? Good question. But what I thought about doing was to install FTS on my backup instance, and if things go fine, then I install an FTS instance on my production server - that is, if one doesn't interfere with the other. I will give Solr another shot - my worries are mostly if Solr is supported on ARM (my prod instance is running on ARM) - I know Elasticsearch has an ARM build. Ii thought about the Xapian engine, but since it requires dovecot 2.3, I will have to wait. Best, Francis Philon On 31 Jan 2020, at 17:24, Francis Augusto Medeiros-Logeay wrote: Hi there, I got successfully to replicate my mail server to another dovecot install using dsync, mainly for redundancy, and it works great. I want to try to install fts, as some of the mailboxes have tens of thousands of messages, and it takes minutes to get some results when searching via IMAP on a Roundcube interface. I want to experiment with fts-solr first, and firstly on my redundant server, ie., not on my main dovecot install. Is it ok to do this? I ask because I am afraid of how this whole reindexing on the redundant install will affect the production server. Also, any tips on something else than fts-solr? I tried it once, but it was so hard to get it right, so many configurations, java, etc., that I'd rather try something else. I also could try fts-elastic or something like that, but, again, having to maintain an elasticsearch install might use more resources than I think is worth. Any thoughts on that? Best, -- Francis
Re: Strategy for fts and Replication
Hi Francis, next to fts-solr there was fts-lucene. But that Lucene there seems heavily outdated why the Dovecot docs also suggest using Solr. Elasticsearch probably is similar to Solr but the later is maintained by Dovecot team. I started with downloading the Solr binary distribution to Debian with JRE preinstalled and things were running like after 10 min. Yes it’s a bit more complicated to find the schema and edit things like header size (in tips section). It’s running quite nicely since then and has zero maintenance. As FTS indexes are separate in external Solr instance I’d guess that it won’t interfere with dsync. What I don’t know is if dsync’ing would trigger indexing. This brings me to wonder how one could actually replicate the Solr instance!? Philon > On 31 Jan 2020, at 17:24, Francis Augusto Medeiros-Logeay > wrote: > > Hi there, > > I got successfully to replicate my mail server to another dovecot install > using dsync, mainly for redundancy, and it works great. > > I want to try to install fts, as some of the mailboxes have tens of thousands > of messages, and it takes minutes to get some results when searching via IMAP > on a Roundcube interface. > > I want to experiment with fts-solr first, and firstly on my redundant server, > ie., not on my main dovecot install. Is it ok to do this? I ask because I am > afraid of how this whole reindexing on the redundant install will affect the > production server. > > Also, any tips on something else than fts-solr? I tried it once, but it was > so hard to get it right, so many configurations, java, etc., that I'd rather > try something else. I also could try fts-elastic or something like that, but, > again, having to maintain an elasticsearch install might use more resources > than I think is worth. Any thoughts on that? > > Best, > > -- > Francis >
Re: Multi-dbox storage space
> > On 03 Feb 2019, at 22:07, MRob wrote: > > Hi, I originally picked mdbox because I had the impression from reading about > it on the mailing list that it was more performant and that it would conserve > disc space. > > But lately i found mailboxes have nearly double the storage as reported. I > mean, IMAP QUOTA reports around 900MB for one account that has over 2GB when > I inspect the mail location of the user using the linux command `du` > > Another account has around 750MB according to IMAP but using du I find 1.5GB. > This is for message files and index files combined. > > Is this normal? I tried to read the MailboxFormat/dbox wiki page and it does > not directly address this topic but it does reference "doveadm purge". Do I > need to use "doveadm purge"? From https://wiki2.dovecot.org/MailboxFormat/dbox: > Expunging a message only decreases the message's refcount. The space is later > freed in "purge" step. Mails are not directly removed from disk. You should probably setup the nightly cronjob to clean your mailbox. Check the mailbox size once after running doveadm purge. Philon
Re: My sub-folder with Outlook work-around to date
Hi David, I somehow remember that there is a way to get the mail_location from per-user-settings. I had this once when migrating from maildir to mdbox. You might want to use Google and the mailing list archives for the details but in summary it went like this: - set Dovecot to grab the mail_location from user-settings (I think we used MySQL db field - or check docs for auto:) - have login script which converts maildir from setup A to B and update the user setting - kick the user once again to apply the setting (I’m not sure about this) This then automates the conversion process to happen on next user login. Be warned though that today with all the idle phones, probably many users will re-login after you restart the server. Perhaps the scripting could handle a timer which randomizes this a little. Philon > Am 07.10.2017 um 02:22 schrieb David.M.Clark : > > Hi All, > > Thanks to MJ for the suggestion as my only hope, and I have tested this with > Outlook 2016 and Thunderbird today - seamless creation of folders. > > Now the mammoth task of converting customers with masses of mbox folders to > MailDir format using the on-line perl script: mb2md-3.20.pl > > Works nicely and will be what gets us out of trouble. My only sadness is that > Outlook then has to be 're-subscribed' to the folders again which makes > sense or maybe I didn't give it long enough - its IMAP folder syncs are > extremely slow compared to Thunderbird which I also tested in parallel. > Thunderbird didn't need any re-sync, it just found the new subscription setup > and was ready to go. But in all fairness, Outlook is still really only an > Exchange compatible product for the most part. > > Would be great to run both mbox and MailDir at the same time so we can do the > slow user-by-user process rather than having them all down at some point > completely - might look at running Dovecot on a different port maybe with a > different ".conf" but thinking this may not work anyway or cause issues > back to my server tests to see :-) > > I use additional ports to 143 for external IMAP restrictions anyway so might > be able to do this method somehow. > > Anyway, for those interested in where I got to in my testing so far... > > 1. Download perl script "mb2md-3.20.pl" from: > > http://batleth.sapienti-sat.org/projects/mb2md > > 2. Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to MailDir: > > #mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u > mail_location = maildir:~/Maildir > > 3. Restarted Dovecot > > 4. su as user and in their home (/u/home/user on my setup): > > /u/packages/mb2md-3.20.pl -m > > then: > > mv mail mail_old > > then: > > /u/packages/mb2md-3.20.pl -s mail_old -R > > then: > > Login with Outlook and subscribe to IMAP folders again. > > Not a big fan of gazillions of little eml based text mail files in > directories but hey, if it fixes our issues, maybe I need to stop > wambulancing about it all and do as I am told :-) > > Hope this helps - will keep this e-mail updated on findings if anything else > comes to light.
Re: Full text search indexing behavior and testing?
Hi Mart,
I think it’s a bit more… you might want to refer to
https://wiki.dovecot.org/Plugins/FTS <https://wiki.dovecot.org/Plugins/FTS>
You only mentioned the function to automatically start indexing. Nothing about
were this should actually happen. We are using Solr…
mail_plugins = $mail_plugins fts fts_solr
and:
plugin {
fts = solr
fts_solr = url=http://solr.example.org:8983/solr/
}
The autoindex does nothing more than automatically sending incoming mails to
the FTS index system. Solr is external while Lucene is library based and saved
inside the mailbox hierarchy. There even is an implementation for Elasticsearch.
For all options… you might want to observe the logs once it has been enabled.
Philon
> Am 03.07.2017 um 13:34 schrieb Mart Pirita :
>
> Hi,
>
> I'm confused about FTS. https://wiki.dovecot.org/Tools/Doveadm/Index say's:
>
> "Messages can also be added automatically to full text search index using:
>
> plugin {
> ...
> fts_autoindex = yes
> }"
>
>
> So I added "fts_autoindex = yes" into dovecot.conf (I'm using single
> config file). That's it? How to test FTS? Or I still must use some
> external indexing server?
>
>
> --
> Mart
Re: How to add Ubuntu 16.04.2 server to join and authenticate with domain controller windows2012r2
Let me make a guess and argument he wanted to authenticate Dovecot against AD... Then there is this article in the wiki: http://wiki2.dovecot.org/HowTo/ActiveDirectoryNtlm Should do as requested clean document...!? Philon Am 23.02.2017 08:48, schrieb Steffen Kaiser: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 22 Feb 2017, Basdove wrote: 1. I have Ubuntu 16.0.4.2 server edition 2. I have windows server 2012R2 domain controller (Active directory installed) 3. I want to join and Authenticate Ubuntu with windows server 2012 domain controller 4. Please provide the clean document http://lmgtfy.com/?q=ubuntu+join+windows+ad How is this question related to Dovecot? - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEVAwUBWK6TxHz1H7kL/d9rAQKdVAf/cEtO1icmE/fdB05SPloYKOy9T5otMJxN nn1dCmJoop4U9OhM9YOMZJYePnwIgCGqF7cRTih8u5pPmQIpFeNAKmIfrLHmk2Jl GuGdvbq3GHyH3kpfUuLsjhJ3p+x4hI9/xDSVmEc16C9606LLzrf+/JK89JHlHxh9 4tzvDUrYU8CYQGebi4RVXIRpnb128xfubFNY6+Tbumg+GoeeubUfOrSgTbfj5IG+ CZ9LHzKqXQcyE5AkOFzv5p2NQCViNskDKtzhEqmKVNVvLAzkR/fC/d2qpLPvAVp4 dkfNNn6PWa5rbpvYViTyH720Q0OsIawmznZf3IIcn9fF7R8J7Zex2w== =v/cc -END PGP SIGNATURE-
Re: autoexpunge clarification
Hi Michael you are correct I guess based on your assumptions. autoexpunge kicks in on user interaction either by email delivery or a user checking in. If both does not occour then no cleanup will happen. Question would be if there is that much to cleanup in this case. A user not logging in for long time and not receiving any emails… seems inactive to me. Ok, he might have put all his mails to the trash and this was not emptied before he went for his sabatical. But the use case seems to be in the n<10 range!? A monthly cronjob could be sufficent in this case? I’m quite happy with the expire plugin and SQL timestamping. Using this the cronjob only checks through the listed folders and not all users. Slim approach! And for me a cleanup once per day is at least currently more than enough. Philon > Am 31.08.2016 um 17:44 schrieb Michael Fox : > > Thanks Philon, > > I did read the extra bullets, as indicated in my email below. But your "When > the user quits and thus closes his mailbox/connection" is more clear than > "after the client is already disconnected", since the latter is really > anytime, rather than at the time they quit. > > I can guess that the bulletin about LMTP similarly means at the end of each > time LMTP delivers mail to the mailbox. > > Assuming that is true, then the problem I see with autoexpunge is that it > doesn't address the case of a user that has not logged in nor received mail > in that mailbox for the specified time. Those messages would apparently stay > forever. Correct? > > And, if that's true, then the cron job seems like the only way to expunge all > old messages. Correct? > > Thanks, > Michael
Re: autoexpunge clarification
Hi Micheal, the article is fine if you continue reading it to the next bullet points about IMAP, POP3 and LMTP. In short words… When the user quits and thus closes his mailbox/connection, Dovecot quickly looks through the folders to clean up mails which are older then configured days. In the past there was a cron job which could do this every n hours, days, … but this setting does this automatically. Still I prefer using cron which gives me more control over when this lookup happens. User deinitialization is simply developer „slang" for user closes connection/quits his program. Philon > Am 30.08.2016 um 14:41 schrieb Michael Fox : > > I'm trying to understand autoexpunge, but the documentation is just not > clear. Hopefully, someone can clear up a few questions. > > > > http://wiki.dovecot.org/MailboxSettings says the following: > > > > autoexpunge=: (v2.2.20+) Automatically at user deinitialization > expunge all mails in this mailbox whose saved-timestamp is older than > (e.g. autoexpunge=30d). This removes the need for expire plugin > <http://wiki.dovecot.org/Plugins/Expire> if you don't care that the > expunging may not always happen in time. > > > > What does "at user deinitialization" mean? > > > > What does "if you don't care that the expunging may not always happen in > time" mean? > > > > I read the sub-bullets but they just aren't clear. When exactly does > autoexpunge occur? > > > > Thanks, > > Michael > >
Re: Last Login Plugin with mysql
Hi Adrian it seems likely that what’s documented on the page you refer to just works… in the configured >>dictionary<< <— follow the white rabbit eh blue link So I’d go replace: last_login_dict = redis:host=127.0.0.1:port=6379 with this: last_login_dict = sql:my_sql_config_file.conf Please test it out as I myself haven’t used the plugin yet. Philon > Am 30.08.2016 um 13:59 schrieb Adrian M : > > Hello, > is it possible to use Last Login Plugin with mysql instead of redis ? > http://wiki2.dovecot.org/Plugins/LastLogin > > .-- > Best regards, > Adrian Minta
Re: Replication between two ISPCOnfig servers
Hi Otto if Dovecot users are setup through SQL you might want to check the iterate query. So that things like for example doveadm quota get -A work. So look into your SQL config, there you must setup the sql query. Then doveadm sync -A should do what you expect. Philon 2016-02-22 11:44 GMT+01:00 Otto Saayman : > Hello > > I have set up two servers, one master and one slave, using ISPConfig. I > have been using rsync to synchronise mails between the two machines, but > this has some unstable results. > > I had a look at setting up replication: > http://wiki.dovecot.org/Replication > > When I run the command, doveadm user '*', only two users are returned. So > when I run this command: doveadm replicator replicate \* I get: 2 users > updated > > I run this command: > doveadm sync -u [e-mail address] tcp:[2nd/ slave dovecot server IP > address] > > Here I can input any valid e-mail address in the ISPConfig MySQL database > and it works. > > How do I tell the dovecote server to replicate all the users configured in > the MySQL database it uses when it handles logins via POP/ IMAP, all of > which is working fine? > > Is there a command I need to run at a set interval via cron to synchronise > the two machines' mailboxes, which are identical in every way except the > mails the users' mailboxes contain? > > OR > > Is there a way that I can instruct the two machines to keep each other's > user mailbox contents "up to date"? > > The main idea is to have a hot backup of my "main" dovecot server and also > to enable a lower mail priority for the second server so that mails are > still received when the main server becomes busy, or is taken offline for > some reason. > > Any help, or link to a HOWTO would be appreciated. > > -- > > Best Regards > > Otto Saayman >
Re: * Re: procmail as a content_filter with dovecot
Vicki Your point here is that you are using Mac OS X. This as you have already learned is overriding your changes with what the guys in Cupertino think is good for you. So I’d rather move on to sieve then fighting my way against this not-so-servery OS. You will get killed on the next OS upgrade I tell you! Sieve is not bad! And OSX is not a server OS as long as you not switch everything over to macports and leave the Server GUI app behind... Make your decision :) > Am 10.11.2015 um 21:01 schrieb Vicki Brown : > >> dovecot supports sieve, so why the need for procmail ? > > Because I already HAVE procmail recipes and know proemial. > > The point is to make what I have, work. > > >> On Nov 9, 2015, at 22:49, Benny Pedersen wrote: >> >> On November 10, 2015 6:36:00 AM Vicki Brown wrote: >> >>> Can anyone help? >> >> dovecot supports sieve, so why the need for procmail ? >> >> but if mta is configured to use procmail then next step is just configure >> procmailrc to use mda dovecot-lda >> > > -- Vicki > cfcl.com/vlb > twitter.com/vlb
Re: procmail as a content_filter with dovecot
Like Benny said: dovecot has sieve, even on OSX: LMGIFY :) : https://support.apple.com/kb/PH8750 is a bit dated as for Snow Leopard… but here I find something more recent: http://www.markreinmuth.com/2014/07/server-side-mail-rules-in-os-x-server/ I can only recommend to install Roundcube and the managesieve plugin. That will give you a nice webui for editing your filters. Philon > Am 10.11.2015 um 07:49 schrieb Benny Pedersen : > > On November 10, 2015 6:36:00 AM Vicki Brown wrote: > >> Can anyone help? > > dovecot supports sieve, so why the need for procmail ? > > but if mta is configured to use procmail then next step is just configure > procmailrc to use mda dovecot-lda
Re: Problems Converting from Cyrus to Dovecot (cyrus2dovecot)
Hi there, I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. Philon > Am 04.11.2015 um 20:47 schrieb Forrest : > > I have been attempting to use the cyrus2dovecot script, to no avail. > > I have many years of content that I want to convert from Cyrus to Dovecot; > with the above not working, what are other options out there? Another idea I > had is simply set up another IMAP server (using Dovecot) and drag-and-drop > and just wait, which I may end up doing. > > In the above, I copied over my entire /var/imap and /var/spool/imap to > another system; there is only one account (mine), so calling the script was > fairly easy; it just doesn't work. > > > inboxes=the "myaccount" that was copied over > > /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ > --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen\ > --cyrus-sub /home/varimap/user/%h/%u.sub \ > --dovecot-inbox /home/myaccount/dovecot/Maildir \ > myaccount > > > > the log output complains of: > >cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. > > and correctly complains about squat indices, as that's not a file it would > handle. There is no output into the Maildir, however. > > All directory paths are correct. > > > Thanks.
Re: updating and wsitching repo to yum.dovecot.fi - Unknown protocol: sieve
Hi Götz, for my debian repository at least I find these packages: dovecot-ee-sieve - sieve filters support for Dovecot dovecot-ee-managesieved - secure ManageSieve server for Dovecot Could be that your third party yum package had these included in the default build. Grüße aus Tübingen Philon > Am 30.10.2015 um 15:56 schrieb Götz Reinicke - IT Koordinator > : > > Am 30.10.15 um 15:44 schrieb Teemu Huovila: >> >> >> On 30.10.2015 15:35, Götz Reinicke - IT Koordinator wrote: >>> Am 30.10.15 um 11:49 schrieb Teemu Huovila: >>>> >>>> >>>> On 30.10.2015 12:18, Götz Reinicke - IT Koordinator wrote: >>>>> Hi, >>>>> >>>>> winter is coming and so I start to clean up some left overs of the year. >>>>> >>>>> One thing is to use the yum.dovecot.fi repository. >>>>> >>>>> After installing the current availabel dovecot and dovecot-ee-pigeonhole >>>>> package and restarting dovecot I do get the error: >>>>> >>>>> >>>>> doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf: >>>>> protocols: Unknown protocol: sieve >>>> Could you please reply with the output of doveconf -n >>> >>> >>> my guess: in the currently used rpms the "managesieve" libs are >>> included; for the official dovecot repo I do have to install the >>> dovecot-ee-managesieve.rpm too... >> Yes, if you have "protocols = sieve .." then you need the managesieve >> package too. Also, to use sieve filtering, you need to load the sieve plugin >> in mail_plugins for lmtp or lda. Please refer to >> http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration >> http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration > > Hi, > > currently sieve filtering is working (sieve plugin in mail_plugins for > lmtp is enabled) with the old repo packages; do I have to install an > other something-plugin.rpm from the dovecot repo? > > Thanks & Regards . Götz > > -- > Götz Reinicke > IT-Koordinator > > Tel. +49 7141 969 82420 > E-Mail goetz.reini...@filmakademie.de <mailto:goetz.reini...@filmakademie.de> > > Filmakademie Baden-Württemberg GmbH > Akademiehof 10 > 71638 Ludwigsburg > www.filmakademie.de <http://www.filmakademie.de/> > > Eintragung Amtsgericht Stuttgart HRB 205016 > > Vorsitzender des Aufsichtsrats: Jürgen Walter MdL > Staatssekretär im Ministerium für Wissenschaft, > Forschung und Kunst Baden-Württemberg > > Geschäftsführer: Prof. Thomas Schadt
Re: Dovecot don't erase mails from storage.
Dear Carlos,
it looks like you are using mdbox. Those files only get a marker for deletion
of mails. The actual deletion needs to be done by purging the mailboxes…
See Timos writing: http://www.dovecot.org/list/dovecot/2010-July/050778.html
And the wiki here: http://wiki2.dovecot.org/Tools/Doveadm/Purge
This will rewrite the mdbox file without the deleted emails. Thus only then the
emails are actually deleted.
Philon
> Am 05.10.2015 um 21:22 schrieb Carlos Baptista :
>
> Hi,
>
> I have a dovecot installation with a strange problem: all the mails stays
> in storage after deleted from users mailbox. If user logon after the mails
> are deleted, they don't receive them any more, but the m.## files aren’t
> deleted. This happens with pop3 (default for this installation) and/or imap.
>
> The dovecot is installed along postfix with mysql and multi-domains support
>
> Here's the dovecot conf
>
>
>
> dovecot -n
> # 2.1.7: /etc/dovecot/dovecot.conf
> # OS: Linux 2.6.32-042stab111.11 x86_64 Debian 7.9
> auth_mechanisms = plain login
> first_valid_uid = 111
> last_valid_uid = 111
> lda_mailbox_autocreate = yes
> mail_debug = yes
> mail_gid = vmail
> mail_plugins = " quota"
> mail_privileged_group = vmail
> mail_uid = vmail
> namespace inbox {
> inbox = yes
> location =
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special_use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Trash {
> special_use = \Trash
> }
> prefix =
> }
> passdb {
> args = /etc/dovecot/dovecot-sql.conf
> driver = sql
> }
> plugin {
> mail_log_events = delete undelete expunge copy mailbox_delete
> mailbox_rename
> mail_log_fields = uid box msgid size
> quota = dict:User quota::file:%h/mail/dovecot-quota
> quota_rule = *:storage=500MB
> }
> postmaster_address = sysad...@inside.pt
> protocols = " imap pop3"
> service auth {
> unix_listener /var/spool/postfix/private/auth {
> group = postfix
> mode = 0660
> user = postfix
> }
> }
> service lmtp {
> unix_listener /var/spool/postfix/private/dovecot-lmtp {
> group = postfix
> mode = 0660
> user = postfix
> }
> }
> ssl_cert = ssl_key = userdb {
> driver = prefetch
> }
> userdb {
> args = /etc/dovecot/dovecot-sql.conf
> driver = sql
> }
> protocol lmtp {
> mail_plugins = " quota sieve"
> }
> protocol lda {
> mail_plugins = " quota"
> }
> protocol imap {
> mail_plugins = " quota imap_quota"
> }
>
> ###
>
> Here is an example of log file with debug
>
> Oct 02 17:50:38 auth: Debug: auth client connected (pid=11621)
> Oct 02 17:50:38 auth: Debug: client in: AUTH1 PLAIN
> service=pop3secured session=Xiif9iEhmgBV80+u
> lip=xxx.xxx.xxx.xxx rip=xxx.xxx.xxx.xxx lport=995
> rport=53914 resp=
> Oct 02 17:50:38 auth-worker(11561): Debug:
> sql(carla.franj...@xxx.xx,xxx.xxx.xxx.xxx):
> query: /* dovecot-sql.conf password_query */ SELECT
> '/var/vmail/xx.xx/carla.franjoso' AS userdb_home, 'mdbox:~/mail' AS
> userdb_mail, 111 AS userdb_uid, 114 AS userdb_gid, CONCAT('*:bytes=',
> CAST(m.quota AS CHAR)) AS userdb_quota_rule, CONCAT(m.local_part, '@',
> d.name) AS user, m.password AS password FROM mailboxes AS m LEFT JOIN
> domains AS d ON m.domain_id = d.id WHERE m.local_part = 'carla.franjoso'
> AND d.name = 'xxx.xx' AND m.is_active AND d.is_active
> Oct 02 17:50:38 auth: Debug: client out: OK 1
> user=carla.franj...@xx.xx
> Oct 02 17:50:38 auth: Debug: master in: REQUEST 3537633281 11621 1
> 41ef179d4a37a603ce38c43fab768f78
> Oct 02 17:50:38 auth: Debug:
> prefetch(carla.franj...@xx.xx,xxx.xxx.xxx.xxx,):
> success
> Oct 02 17:50:38 auth: Debug: master out: USER 3537633281
> carla.franj...@x.xx home=/var/vmail//carla.franjoso
> mail=mdbox:~/mail uid=1
> 11 gid=114 quota_rule=*:bytes=104857600
> Oct 02 17:50:38 pop3: Debug: Loading modules from directory:
> /usr/lib/dovecot/modules
> Oct 02 17:50:38 pop3: Debug: Module loaded:
> /usr/lib/dovecot/modules/lib10_quota_plugin.so
> Oct 02 17:50:38 pop3: Debug: Added userdb setting: mail=mdbox:~/mail
> Oct 02 17:50:38 pop3: Debug: Added userdb setting:
> plugin/quota_rule=*:bytes=104857600
> Oct 02 17:50:38 pop3(carla.franj...@x.xx): Debug: Effective uid=111,
> gid=114, home=/var/vmail/.xx/carla.franjoso
> Oct 02 17:50:38 pop3(carla.franj...@x.xx): Debug: Quota roo
Re: [IMAP] Nothing in INBOX
Hi Yvon, perhaps you can take a look into your logs at /var/log/mail.log (and dovecot.log as you configured). There Postfix should pickup the incoming email and deliver it to Dovecot. From this communication the problem should be pretty obvious. Philon > Am 30.09.2015 um 15:14 schrieb Yvon Thoraval : > > Hey all, > > I've just installed Postfix + Dovecot on Xubunto 14.04 LTS. > Following this page : > Setup a mail server with Postfix and Dovecot on Ubuntu / Debian > <http://www.binarytides.com/install-postfix-dovecot-debian/> > > although I'm able to send emails to GMail and other kind of domain, nothing > appears on Thunderbird INBOX, nor using openssl : > > > $ openssl s_client -connect imap..:993 > CONNECTED(0003) > > > >Start Time: 1443543797 >Timeout : 300 (sec) >Verify return code: 18 (self signed certificate) > --- > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE > AUTH=PLAIN AUTH=LOGIN] Dovecot (Ubuntu) ready. > A1 LOGIN @. > A1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE > SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN > CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in > A2 LIST "" "*" > * LIST (\HasNoChildren) "." GMail > * LIST (\HasNoChildren \Drafts) "." Drafts > * LIST (\HasNoChildren) "." j-kiosk > * LIST (\HasNoChildren \Trash) "." Trash > * LIST (\HasNoChildren) "." Pourriel > * LIST (\HasNoChildren) "." "Linux Mint" > * LIST (\HasNoChildren) "." free > * LIST (\HasNoChildren \Sent) "." Sent > * LIST (\HasNoChildren) "." INBOX > A2 OK List completed. > A3 EXAMINE INBOX > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) > * OK [PERMANENTFLAGS ()] Read-only mailbox. > * 0 EXISTS > * 0 RECENT > * OK [UIDVALIDITY 1443454274] UIDs valid > * OK [UIDNEXT 1] Predicted next UID > * OK [NOMODSEQ] No permanent modsequences > A3 OK [READ-ONLY] Examine completed (0.000 secs). > > > > > However, using mail, i can read the emails : > > > $ mail > > Heirloom mailx version 12.5 6/20/10. Type ? for help. > > "/var/mail/desktop": 20 messages 9 new 18 unread > > U 1 Desktop User Sun Sep 27 13:01 19/621 ceci est le sujet > > > > N 20 Yvon Thoraval Tue Sep 29 18:02 25/1042 another mail with > Maildir... > > ? > > > > > my dovecot setup uses vhosts : > > > $ ls -al /var/mail/vhosts/// > > total 76K > > drwxr-sr-x 14 vmail vmail 4,0K sept. 29 18:01 ./ > > drwxr-sr-x 4 vmail vmail 4,0K sept. 27 10:31 ../ > > drwxr-sr-x 2 vmail vmail 4,0K sept. 28 17:31 cur/ > > -rw-r--r-- 1 vmail vmail 272 sept. 28 17:31 dovecot.index.log > > -rw-r--r-- 1 vmail vmail 168 sept. 29 16:16 dovecot.mailbox.log > > -rw-r--r-- 1 vmail vmail 51 sept. 28 17:31 dovecot-uidlist > > -rw-r--r-- 1 vmail vmail8 sept. 29 16:17 dovecot-uidvalidity > > -r--r--r-- 1 vmail vmail0 sept. 28 17:31 dovecot-uidvalidity.56095d49 > > drwxr-sr-x 5 vmail vmail 4,0K sept. 29 15:09 .Drafts/ > > drwxr-sr-x 5 vmail vmail 4,0K sept. 28 18:40 .free/ > > drwxr-sr-x 5 vmail vmail 4,0K sept. 28 18:41 .GMail/ > > drwxr-sr-x 5 vmail vmail 4,0K sept. 29 16:17 .j-kiosk/ > > drwxr-sr-x 5 vmail vmail 4,0K sept. 29 16:17 .Linux Mint/ > > drwxr-sr-x 2 vmail vmail 4,0K sept. 29 18:01 Maildir/ > > -rw-r--r-- 1 vmail vmail0 sept. 28 17:31 maildirfolder > > drwxr-sr-x 2 vmail vmail 4,0K sept. 28 17:31 new/ > > drwxr-sr-x 5 vm
Re: Dovecot auth username mapping
Hi Laz,
I’m just wondering… why are you using LDAP and/or PAM to access the MySQL
server? If also the password is stored in the db you could use MySQL directly?
Because then you could use password_query and user_query to actually split the
provided email address into name and domain parts. Then you can lookup each
individually or adjust as needed...
I have something like this:
user_query = SELECT CONCAT('/var/mail/virtual/', SUBSTRING(`mail_addr`,
LOCATE('@', `mail_addr`) +1 ), '/', \
SUBSTRING(`mail_addr`, 1, LOCATE('@', `mail_addr`) -1) ) AS 'home', '1000' AS
'uid', \
'8' AS 'gid', CONCAT('*:bytes=', `quota`, 'M') AS 'quota_rule' FROM
`mail_users` \
WHERE `mail_addr` = '%u' AND `status` = 'ok' AND `mail_type` LIKE '%%_mail%%‘
With an SQL statement you could even use sub-selects and whatnot to do
complicated things. Perhaps you could do something similar with the LDAP string
but I never used LDAP that much…
Philon
> Am 02.07.2015 um 02:27 schrieb Laz C. Peterson :
>
> It’s actually unbelievable how much slower LDAP auth is than PAM. Does
> anyone have any suggestions how I can improve Dovecot LDAP auth? I have
> tried caching authentications and that doesn’t help either.
>
> ~ Laz Peterson
> Paravis, LLC
> Ph: 951.319.3240 x201
>
>> On Jul 1, 2015, at 4:41 PM, Laz C. Peterson wrote:
>>
>> Thank you for the response Axel. I will look into that.
>>
>> I did attempt to switch the PAM/Kerberos authentication to Dovecot LDAP
>> authentication, but now performance is unbelievably slow. For example, with
>> PAM/Kerberos, a user can log into webmail and have all of their
>> emails/folders showing almost immediately. When using Dovecot LDAP, it
>> takes literally 8-10 seconds to see the same thing.
>>
>> I was hoping that was a possible replacement for this, but my goodness it
>> was so incredibly slow! This would definitely be an option though, as it
>> does serve the purpose. I just can’t figure out how to fix the performance
>> issue. Any thoughts to this?
>>
>> ~ Laz Peterson
>> Paravis, LLC
>> Ph: 951.319.3240 x201
>>
>>> On Jul 1, 2015, at 3:24 PM, Axel Luttgens wrote:
>>>
>>>
>>>> Le 1 juil. 2015 à 04:38, Laz C. Peterson
>>>
>>>> a écrit :
>>>>
>>>> I have an interesting case here …
>>>>
>>>> Virtual mailboxes, domain/username/aliases stored in MySQL, authentication
>>>> done using PAM. PAM authenticates through Kerberos, which are internal
>>>> realms and not the email domains — for example, my username would be
>>>> laz@PARAVIS.LOCAL <mailto:laz@PARAVIS.LOCAL> and my email address would be
>>>> l...@paravis.net <mailto:l...@paravis.net>.
>>>>
>>>> All of this works just fine. But what I want to do is allow the users to
>>>> log in using their email address and not their full Kerberos name. It is
>>>> becoming laborious to help the users understand the difference between
>>>> their username@LOCAL.REALM and username@email.address
>>>> <mailto:username@email.address> and why we have to have two separate
>>>> identities that mean the same thing.
>>>>
>>>> I have the SQL statements to convert either the Kerberos login or the
>>>> email address to the actual Kerberos login (so they may use either). But
>>>> I cannot seem to figure out how to get Dovecot to acknowledge this as the
>>>> mapped username.
>>>>
>>>> I’m sure there has to be a way. Any help will be greatly appreciated.
>>>> Thank you!
>>>
>>> Hello Laz,
>>>
>>> I fear you’ll have to resort to CheckPassword
>>> (http://wiki2.dovecot.org/AuthDatabase/CheckPassword) or something similar.
>>>
>>> Indeed, your MySql database may contain everything needed to convert email
>>> addresses to kerb login (and vice-versa), but Dovecot’s PAM interface
>>> understandably just knows about a (login, password) pair, where the login
>>> is the one provided by the user wanting to log in.
>>>
>>> That said, I hope to be wrong,
>>> Axel
Re: zlib, mdbox and spam filtering scripts
> Let the MTA feed SA. > http://gtmp.org/doku.php?id=publications:sa-postfix-en Hi Oscar, Hi dovecot-list, I read through the docs, quite interesting read. But I’m wondering, mails are in this case only kept temporary? So re-reading spam and ham is not possible…!? It seems useful still, as it’s direct and not run via a daily cron script. I also looked into Sven’s advice (Danke!) about doveadm fetch. I just came across this script here: https://git.mnt-tech.fr/admintools.git/raw/master/blacklist.sh. It does much more than just fetch, but for me it was a good reference for retrival of mails using doveadm. So thanks for the hints and merry X-mas! Philon
zlib, mdbox and spam filtering scripts
Hi all, I am quite happily running a dovecot setup with maildirs. This was once running courier but since the migration I have never ever looked back. Now, with growing userbase I’m looking into using zlib compression and mdbox instead of maildir. This seems to go quite well using dsync on a testbed server. So let’s go production! Before final decision I came across this: - I’m using a spam learning address, where users forward their mails to train SpamAssassin. - there is a cron script on the server, mime-stripping the attachments and feeding them into SA - works quite well even for the still numerous POP users So well… compression ok, I could add a gunzip into the script and go from there. But mdbox seems only accessible via dovecot. I don’t think that accessing via IMAP makes sense for a local script. So I would block myself out when using mdbox. How are your spam/ham training routines? How do you feed SA? thank you! Philon
Re: Dbox and Exim
Hi Chuck, I’m really curious as to if I really understand you correctly. Neither Exim nor Postfix do need to support any mailbox format. They both should hand incoming mail to either LDA or LMTP. You can use sieve and managesieve for filtering and there even is a Thunderbird plugin for managing the filters. So all in all it should be pretty convenient. I don't see why dovecot should not be able to handle multiple GBs of mail, even on a local network. Philon You might want to check here: http://wiki2.dovecot.org/LDA/Exim http://wiki2.dovecot.org/LMTP/Exim 2014-10-09 14:20 GMT+02:00 C Peters : > The Dovecot wiki [1] doesn't list either Exim, Postfix or anything > else as supporting the dbox. I did some searching in the Exim > archives and have found only one message mentioning dbox, sdbox or > mdbox. [2] I think an RFC would go a long way towards getting the > format supported. Has Timo, or anyone else, considered submitting an > RFC to the IETF? > > My question was how are we going to use it if Exim doesn't support it. > It looks like we can filter it and convert it with dovecot-lda. [3] > I have generally avoided using Procmail for a few years because of > various problems and it's a dead software project that hasn't been > developed in what 15 to 20 years. I am thinking about trying it with > Exim's built in Maildrop filtering support. Has anybody tried that? > What else are people using to enable dbox? > > > The more specific question on the problem I am trying to solve. I > have been using mbox for years, and I can't even get Dovecot, IMAP and > Thunderbird working on the LAN because it can't handle the GB's of old > mail. Mutt works on the local machine, but I want something that > works while I am using something other than ssh and mutt. For example > I want to try Mailpile and K9. I would like to try dbox, but given > it's lack of support in Exim or Postfix, I'll probably convert to > maildir. > > Thanks, > Chuck > > > 1. http://wiki2.dovecot.org/MailboxFormat > 1. https://lists.exim.org/lurker/message/20110220.014144.87010abd.en.html > Thread view: > https://lists.exim.org/lurker/thread/20110216.223226.015539ba.en.html#i20110216.223226.015539ba > 2. http://ramblings.narrabilis.com/mdboxprocmail >
