Re: [Dovecot] Doveadm director flush/remove
On 15.7.2010, at 23.58, Timo Sirainen wrote: > On 15.7.2010, at 22.00, Brad Davidson wrote: > >> It times out after a minute: >> >> [r...@cc-popmap7 ~]# time doveadm director map >> doveadm(root): Error: User listing returned failure >> doveadm(root): Error: user listing failed >> >> real1m0.028s >> user0m0.088s >> sys 0m0.072s >> >> Jul 15 13:46:24 cc-popmap7 dovecot: auth: Error: auth worker: Aborted >> request: Lookup timed out >> Jul 15 13:53:25 cc-popmap7 dovecot: auth: Error: getpwent() failed: No such >> file or directory > > I suppose it shouldn't abort if the query keep sending data.. I'll see if I > can get this fixed. Also see if http://hg.dovecot.org/dovecot-2.0/rev/d13c1043096e fixes this or if there are other timeouts?
Re: [Dovecot] Doveadm director flush/remove
On 15.7.2010, at 23.58, Timo Sirainen wrote: >> userdb passwd. Our LDAP directory might not be optimally configured. The >> group that administers it only really cares about binds, iteration can be >> rather slow: >> >> [r...@cc-popmap7 ~]# time getent passwd | wc -l >> 51552 >> >> real8m0.120s >> user0m2.507s >> sys 0m1.093s >> >> That comes out to just over 100 entries a second. > > Maybe there could be a parameter to get the user list from a file (one > username per line) instead of userdb. Added -f parameter for this.
Re: [Dovecot] Doveadm director flush/remove
On 15.7.2010, at 22.00, Brad Davidson wrote: > It times out after a minute: > > [r...@cc-popmap7 ~]# time doveadm director map > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > real1m0.028s > user0m0.088s > sys 0m0.072s > > Jul 15 13:46:24 cc-popmap7 dovecot: auth: Error: auth worker: Aborted > request: Lookup timed out > Jul 15 13:53:25 cc-popmap7 dovecot: auth: Error: getpwent() failed: No such > file or directory I suppose it shouldn't abort if the query keep sending data.. I'll see if I can get this fixed. > userdb passwd. Our LDAP directory might not be optimally configured. The > group that administers it only really cares about binds, iteration can be > rather slow: > > [r...@cc-popmap7 ~]# time getent passwd | wc -l > 51552 > > real8m0.120s > user0m2.507s > sys 0m1.093s > > That comes out to just over 100 entries a second. Maybe there could be a parameter to get the user list from a file (one username per line) instead of userdb.
Re: [Dovecot] Dovecot sieve support
On 15.7.2010, at 23.00, Daniel L. Miller wrote: > Is sieve going to be merged into the main server source, No. > or at this time is it going to remain an external patch? No. It no longer requires patching.
[Dovecot] Dovecot sieve support
Is sieve going to be merged into the main server source, or at this time is it going to remain an external patch? -- Daniel
Re: [Dovecot] Doveadm director flush/remove
Timo, > -Original Message- > From: Timo Sirainen [mailto:t...@iki.fi] > > See what it says in logs. It times out after a minute: [r...@cc-popmap7 ~]# time doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed real1m0.028s user0m0.088s sys 0m0.072s Jul 15 13:46:24 cc-popmap7 dovecot: auth: Error: auth worker: Aborted request: Lookup timed out Jul 15 13:53:25 cc-popmap7 dovecot: auth: Error: getpwent() failed: No such file or directory > Are you using userdb passwd or userdb ldap? With userdb ldap you need to > configure iterate_attrs and iterate_filter in your LDAP config. With > passwd I think it should work directly.. userdb passwd. Our LDAP directory might not be optimally configured. The group that administers it only really cares about binds, iteration can be rather slow: [r...@cc-popmap7 ~]# time getent passwd | wc -l 51552 real8m0.120s user0m2.507s sys 0m1.093s That comes out to just over 100 entries a second. -Brad
Re: [Dovecot] Doveadm director flush/remove
On Thu, 2010-07-15 at 21:37 +0100, Timo Sirainen wrote: > On Thu, 2010-07-15 at 13:15 -0700, Brad Davidson wrote: > > [r...@cc-popmap7 ~]# doveadm director map > > doveadm(root): Error: User listing returned failure > > See what it says in logs. Oh, probably that you don't have a userdb at all. > > Our environment might be a little weird. We're using LDAP accounts via > > pam_ldap on the backend servers, so they are essentially local accounts > > (not virtual). I'm using passthrough auth with NOPASSWORD in the director > > proxy query. The accounts are also available on the directors, but since > > there's about 45k of them it would take quite a while to iterate and test > > hashes for all of them, if that's what it's trying to do. I doubt it's that slow. I think the speed only depends on how fast it can iterate through all the users in userdb. 45k accounts probably won't take that long.. I'm more concerned about setups with 100 times that :)
Re: [Dovecot] Doveadm director flush/remove
On Thu, 2010-07-15 at 13:15 -0700, Brad Davidson wrote: > [r...@cc-popmap7 ~]# doveadm director map > doveadm(root): Error: User listing returned failure See what it says in logs. > Our environment might be a little weird. We're using LDAP accounts via > pam_ldap on the backend servers, so they are essentially local accounts (not > virtual). I'm using passthrough auth with NOPASSWORD in the director proxy > query. The accounts are also available on the directors, but since there's > about 45k of them it would take quite a while to iterate and test hashes for > all of them, if that's what it's trying to do. Are you using userdb passwd or userdb ldap? With userdb ldap you need to configure iterate_attrs and iterate_filter in your LDAP config. With passwd I think it should work directly..
Re: [Dovecot] Dinamic 'mail_location' for each user
On Thu, 2010-07-15 at 17:12 -0300, Thiago Henrique wrote: > Em Qui, 2010-07-15 às 20:45 +0100, Timo Sirainen escreveu: > > > > You mean your userdb is returning a "mail" field that contains those % > > variables? No, that won't work. > > It is important for me to spread the mailboxes inside the partition. Use > the 'H' modifier for it would be perfect. But you can do that yourself when storing the path to database. Build a script that hashes the username path in a way you want, and store that. > A patch for this would be accepted? I'm still not sure if you mean what I thought you mean. If you mean that % variables would be expanded when they're coming out of userdb then no, it would not be accepted. It would break people's setup who actually have % characters in paths.
Re: [Dovecot] Ok, I've given up
> Phil Howard : > And I fully accept that as a sufficient reason to make a choice. I > made the choice of Dovecot, having zero experience with it, because of > my experience with Courier. Sadly, making such a choice with zero > experience and zero knowledge of either options is the really hard one > to make. Ah, wrt. to IMAP servers I at least have the experience of three different servers: UoW imapd, Cyrus imapd and now dovecot. Based on that selection: dovecot is by _far_ the simplest one to get started with, and by far the most stable and scalable. The second time a cyrus upgrade trashed my read marks I started looking for a replacement, and a quick google search found me dovecot. I took some time figuring out what method I should use to transfer the mailboxes, and eventually settled on imapsync. I set up dovecot on a different machine than the one cyrus was running on, used imapsync to pull the mailboxes over, and then shut down cyrus on the first machine, installed dovecot, rsync'd the mailboxes back, and was up and running with the new dovecot. As a bonus I could go back to use procmail for mail filtering. It's ugly... but I know it...
Re: [Dovecot] Doveadm director flush/remove
Timo, > -Original Message- > From: Timo Sirainen [mailto:t...@iki.fi] > > > I suppose it could get a list of all users and then list all users whose > hash matches what director has.. Hmm. I guess that would be usable too, > yes. :) > > See if this works: http://hg.dovecot.org/dovecot-2.0/rev/4138737f41e6 I get: [r...@cc-popmap7 ~]# doveadm director map doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed Our environment might be a little weird. We're using LDAP accounts via pam_ldap on the backend servers, so they are essentially local accounts (not virtual). I'm using passthrough auth with NOPASSWORD in the director proxy query. The accounts are also available on the directors, but since there's about 45k of them it would take quite a while to iterate and test hashes for all of them, if that's what it's trying to do. Sounds like this might just not work in our environment due to the size of our account base. I do appreciate very much that you added the feature though! -Brad
Re: [Dovecot] Dinamic 'mail_location' for each user
Em Qui, 2010-07-15 às 20:45 +0100, Timo Sirainen escreveu: > > You mean your userdb is returning a "mail" field that contains those % > variables? No, that won't work. It is important for me to spread the mailboxes inside the partition. Use the 'H' modifier for it would be perfect. A patch for this would be accepted? Thank you. -- []'s Thiago Henrique Network Administration Digirati Networks K8 Networks
Re: [Dovecot] How to Upgrade to dovecot 2.0rc2 from dovecot 1.x
On Thu, 2010-07-15 at 15:02 -0500, Wilberth Perez wrote: > Exists some instructions for /upgrade dovecot/ v1./2 to dovecot v2.0rc2 ? http://wiki2.dovecot.org/Upgrading/2.0 If you find that something is missing, let us know.
[Dovecot] How to Upgrade to dovecot 2.0rc2 from dovecot 1.x
Hi: Exists some instructions for /upgrade dovecot/ v1./2 to dovecot v2.0rc2 ? Cheers. /
Re: [Dovecot] IMAP super user
On Thu, 2010-07-15 at 16:24 -0300, Thiago Henrique wrote: > I plan to migrate from Cyrus to Dovecot. But I need a admin user > which is able to obtain information about the quotas of common users. Or did you mean to get other peoples' quota via IMAP protocol? There's kind of a support for this, but not really. If user is marked "admin", then GETQUOTAROOT lists all the users' quotas who have shared mailboxes visible to the the user. But there's currently no way to set a user to become admin without modifying sources.
Re: [Dovecot] Dinamic 'mail_location' for each user
On Thu, 2010-07-15 at 11:29 -0300, Thiago Henrique wrote: > mdbox:/var/spool/imap/partition1/%256HRu/%u:INDEX=/var/lib/imap/user/% > 256HRu/%u > > Can Dovecot to interpret 'mail_location' load from database ? You mean your userdb is returning a "mail" field that contains those % variables? No, that won't work. If you're already returning a partial location from userdb, why not just store and return a full path to it permanently?
Re: [Dovecot] IMAP super user
On Thu, 2010-07-15 at 16:24 -0300, Thiago Henrique wrote: > Is there any directive in Dovecot similar to 'imap_admins' of Cyrus? You could create a master user/password that can log in as other users: http://wiki.dovecot.org/Authentication/MasterUsers > I plan to migrate from Cyrus to Dovecot. But I need a admin user > which is able to obtain information about the quotas of common users. With v1.x the only good way is to log in as other users. Or possibly use http://wiki.dovecot.org/PreAuth With v2.0 you can do this really easily: doveadm quota get -A Another possibility is to use dict quota with SQL backend and just do a SELECT from table.
Re: [Dovecot] IMAP super user
Thiago Henrique (Do 15 Jul 2010 21:24:28 CEST):
> Hello,
>
> Is there any directive in Dovecot similar to 'imap_admins' of Cyrus?
>
> I plan to migrate from Cyrus to Dovecot. But I need a admin user
> which is able to obtain information about the quotas of common users.
Probably you're looking for things related to auth_master or master.
passdb passwd-file {
master = yes
args = /etc/vmail/master-users
}
Or search the Wiki for AuthDatabase.PasswdFile
Best regards from Dresden/Germany
Viele Grüße aus Dresden
Heiko Schlittermann
--
SCHLITTERMANN.de internet & unix support -
Heiko Schlittermann HS12-RIPE -
gnupg encrypted messages are welcome - key ID: 48D0359B ---
gnupg fingerprint: 3061 CFBF 2D88 F034 E8D2 7E92 EE4E AC98 48D0 359B -
signature.asc
Description: Digital signature
[Dovecot] IMAP super user
Hello, Is there any directive in Dovecot similar to 'imap_admins' of Cyrus? I plan to migrate from Cyrus to Dovecot. But I need a admin user which is able to obtain information about the quotas of common users. Thank you. -- []'s Thiago Henrique Network Administration Digirati Networks K8 Networks
Re: [Dovecot] Temporary files
On Thu, 2010-07-15 at 08:54 -0700, Daniel L. Miller wrote: > "/var/spool contains data which is awaiting some kind of later > processing. Data in /var/spool represents work to be done in the future > (by a program, user, or administrator); often data is deleted after it > has been processed." I don't think this is what the files are. They are created and they're immediately unlinked before anything is even written to them. The files are treated kind of like being virtual memory, but since they're stored in filesystem rather than kept in memory, the kernel knows that under memory pressure it's better to write them to disk than move memory pages to swap. So there is no "awaiting" or "work to be done in future". And data isn't deleted after it has been processed, it's deleted even before processing begins.. BTW. I've already committed the mail_temp_dir with a default to /tmp. I forgot also that nowadays if writing to the temp file fails, the mail delivery won't fail but instead everything is moved into memory. So a default /tmp isn't that bad anymore.
Re: [Dovecot] Temporary files
On 7/15/2010 1:27 AM, Thomas Leuxner wrote: On Wed, Jul 14, 2010 at 11:53:48PM -0500, Stan Hoeppner wrote: Don't rely on an existing temp directory. Create /var/spool/dovecot/ and /var/spool/dovecot/tmp/ with the installation script and assign the appropriate permissions. This solves the current problem and gives you a dedicated dovecot spool directory to use in the future for other functions/features. +1 +2. Right from the FHS section - "/var/spool contains data which is awaiting some kind of later processing. Data in /var/spool represents work to be done in the future (by a program, user, or administrator); often data is deleted after it has been processed." Having a configuration parameter as well is always good - but make /var/spool/dovecot the default. -- Daniel
[Dovecot] Dinamic 'mail_location' for each user
Hello, I have a lot of domains in a mail server, so I make a balanced allocation of mailboxes in the storage disks. That improves performance of access to records. Therefore, I want to overwrite 'mail_location' directive for each user. But I also want to use the modifier 'H' to spray the mailboxes of the same partition in a hash. Then, each user has your own 'mail_location' that I would like load from database. Like this: mdbox:/var/spool/imap/partition1/%256HRu/%u:INDEX=/var/lib/imap/user/% 256HRu/%u partition1 is in range [partition1 .. partiton7] In my tests the '%256HRu/%u' kind of 'mail_location' is not being interpreted by Dovecot. Can Dovecot to interpret 'mail_location' load from database ? Thank you. -- []'s Thiago Henrique Network Administration Digirati Networks K8 Networks
Re: [Dovecot] Doveadm director flush/remove
On Wed, 2010-07-14 at 23:36 +0100, Timo Sirainen wrote: > > While I'm making a wishlist... 'doveadm director status ' to show > > list of users mapped to a host? Or maybe just 'doveadm director status -v' > > to show list of users instead of just user count. > > I suppose it could get a list of all users and then list all users whose hash > matches what director has.. Hmm. I guess that would be usable too, yes. :) See if this works: http://hg.dovecot.org/dovecot-2.0/rev/4138737f41e6
Re: [Dovecot] Temporary files
On Thu, 2010-07-15 at 14:52 +0200, Edgar Fuß wrote: > > The file size isn't known beforehand. > I thought that because it's known to be >128k, it's known. Dovecot reads 128 kB to memory first, then thinks this is beginning to be too much and starts writing to disk instead.
Re: [Dovecot] Ok, I've given up
On Wed, Jul 14, 2010 at 17:49, Steinar Bang wrote: >> Phil Howard : > >> I abandoned sendmail many years ago and haven't looked back. I tried >> qmail and postfix, and was a lot happier with postfix. I overlooked >> exim at the time, but from what little I've seen and heard, it should >> be up there with postfix, making for a tough choice if you didn't have >> anything to bias your choice (like having used one of them already for >> a few years). > > FWIW I switched from sendmail to exim when switching my home server from > SuSE to debian back in 2001. And I've really never wanted to switch, > neither back to sendmail, nor to anything else. > > Main reason: I understand the configuration files' logic. > > I have no experience of MTAs other than sendmail or exim, though... And I fully accept that as a sufficient reason to make a choice. I made the choice of Dovecot, having zero experience with it, because of my experience with Courier. Sadly, making such a choice with zero experience and zero knowledge of either options is the really hard one to make. -- sHiFt HaPpEnS!
Re: [Dovecot] Temporary files
> The file size isn't known beforehand. I thought that because it's known to be >128k, it's known.
Re: [Dovecot] Temporary files
On 15.7.2010, at 10.02, William Blunn wrote:
> If know you are delivering to Maildir, and you know the folder you are
> delivering to,
But I don't. Sieve could do just about anything to the message.
> then surely you have a temporary directory in the "tmp" out of Maildir's
> {new,cur,tmp} ? And also the potential benefit of being able to rename the
> resulting file into the "new" directory.
This is a very Maildir-specific optimization that doesn't work with anything
else.
> I would like to have the temporary directory be configurable (but with a sane
> default like /tmp or perhaps /tmp/dovecot).
Sure.
Re: [Dovecot] Temporary files
On 15.7.2010, at 9.16, Christian Rohmann wrote: Any thoughts? >>> Could the location be made dependant on the file size? >> >> The file size isn't known beforehand. > > But it could be known if the SIZE extention of LMTP is used right? It wouldn't be known with deliver. And I wasn't planning on implementing SIZE extension for LMTP anytime soon.. > Talking about LMTP ... if one runs lets say postfix on the deliver > machine that is already maintaining a mailqueue and one uses the LMTP > server of dovecot 2.0 to get the messages delivered, why would you need > to write the mail to disk a second time (postfix once, then deliver > again) of the message size is within reasonable limits (lets say below > 100MB). So would you make the size when messages are spooled to disk > (currently 128k as I understand) configurable? Temporary files would typically stay in memory too and get deleted before written to disk. But I guess I could make this configurable.
Re: [Dovecot] Temporary files
On Wed, Jul 14, 2010 at 09:34:52PM +0100, Timo Sirainen wrote: [...] > So why was the move made in the first place? Because a) some people had > small (maybe ramfs) /tmp and Dovecot was eating it all up and b) to On Solaris, this is the situation, though it's not called ramfs but: l...@hydra:/tmp$ uname -a SunOS hydra 5.10 Generic_141444-09 sun4v sparc SUNW,Sun-Fire-T200 l...@hydra:/tmp$ df -h . Filesystem size used avail capacity Mounted on swap16G 0K16G 0%/tmp l...@hydra:/tmp$ swap -l swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 256,1 16 33554416 33554416 Yes, this machine has some RAM and enough swap zvol as well, but still there can be machines with less resources too, and in Solaris /tmp is not a "normal" filesystem anyway ... /var/tmp is available though, and it's a "normal" filesystem ...
Re: [Dovecot] Temporary files
1.
If know you are delivering to Maildir, and you know the folder you are
delivering to, then surely you have a temporary directory in the "tmp"
out of Maildir's {new,cur,tmp} ? And also the potential benefit of being
able to rename the resulting file into the "new" directory.
2.
I would like to have the temporary directory be configurable (but with a
sane default like /tmp or perhaps /tmp/dovecot).
In an installation where Dovecot is being used heavily, people might
want to assign specific resources for Dovecot operation. So someone
might decide that Dovecot should have a specific device (hard disk /
solid state disk / whatever) for its temporary files, other than the
default /tmp.
Bill
Re: [Dovecot] Temporary files
On Wed, Jul 14, 2010 at 11:53:48PM -0500, Stan Hoeppner wrote: > > Don't rely on an existing temp directory. Create /var/spool/dovecot/ and > /var/spool/dovecot/tmp/ with the installation script and assign the > appropriate permissions. This solves the current problem and gives you a > dedicated dovecot spool directory to use in the future for other > functions/features. +1
Re: [Dovecot] Temporary files
Hey all, On 07/14/2010 11:02 PM, Timo Sirainen wrote: > On Wed, 2010-07-14 at 22:55 +0200, Edgar Fuß wrote: >>> Any thoughts? >> Could the location be made dependant on the file size? > > The file size isn't known beforehand. But it could be known if the SIZE extention of LMTP is used right? Talking about LMTP ... if one runs lets say postfix on the deliver machine that is already maintaining a mailqueue and one uses the LMTP server of dovecot 2.0 to get the messages delivered, why would you need to write the mail to disk a second time (postfix once, then deliver again) of the message size is within reasonable limits (lets say below 100MB). So would you make the size when messages are spooled to disk (currently 128k as I understand) configurable? Regards, Christian
