Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 06/26/2012 07:11 AM, Wojciech Puchar wrote: >> The only draw back I see with maildir is one file per message. This > > it is mostly adventage. Agreed. > >> makes it resilient to corruption that mbox sees (if a message gets >> corrupted, you erase one message and that corruption won't propagate >> even if you leave it in place). In many setups this also leads to MUCH >> faster system. On unix systems this doesn't just waste disk space, it > > even with 32kB block/4kB fragment filesystem under FreeBSD which is my > common setup, it isn't that a problem. > i just checked one of my users folder - 2.3GB in 8500 files. the > average is 270 kilobytes per mail. > > checked few others and it looks similar. > > > dovecot's own storage system can do something in between - packing > smallest messages by a few in one file. > >> could lead to inode (or whatever your *nix of choice calls it) >> depletion. > you decide how much inode you need while creating filesystem on every > unix system, except filesystems where it is allocated on demand. Yes, as I noted, I haven't seen this. But it could be an annoyance depending on how things were created and when. I don't believe all file systems can do allocation on demand. I don't know. > > as of latter discussion about what microsoft recommends with linux > (being of course expert of everything) - i would keep silent. > The only reason I know what they recommend is it came up on several sites that described how to setup the service principals. I read something recently on Samba lists that explains why this may be their recommendation. The funny thing is, it really isn't any different than on their systems unless they think that because it is their system the keytab is some how miraculously going to stay more secure than it would on other systems. Sorry if I seemed like I was claiming to be some super expert. I just had a lot of help to pull things together. If he was struggling to find things, I would like to help. Trever -- "Fairy tales are more than true; not because they tell us that dragons exist, but because they tell us that dragons can be beaten." -- G.K. Chesterton
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
> b) Make sure your local samba setup is joined to the domain. Make sure > it writes an appropriate krb5.keytab (/etc/krb5.keytab in my setup) as > part of its password management, etc. > > net ads keytab add smtp/mail_server_fqdn > net ads keytab add imap/mail_server_fqdn > > > You may have to edit the sam.ldb on your S4 server as many times S3 > doesn't create the principals ( /usr/local/samba/bin/ldbedit -H > /usr/local/samba/private/sam.ldb sAMAccountName=mailserverhostname$ > should do the trick and add userPrincipalName so that it has > imap/MAILSERVER_FQDN and smtp/MAILSERVER_FQDN, each being its own > userPrincipalName, this should give the machine account 3 > userPrincipalName lines) Sorry to anyone who was following what I wrote. I made a mistake. This should NOT be userPrincipalName, it should be servicePrincipalName. (There should already be 1 or 2 such lines that says HOST/host or HOST/host.fqdn) Trever
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 06/25/2012 08:37 AM, Trever L. Adams wrote: On 06/25/2012 01:20 AM, Kaya Saman wrote: Now what I would like to know is, which is better for "virtual hosting" Maildir or mbox? Basically my requirement is that I would like to separate users via either individual folders and then put each user's mbox or Maildir in the created directory, or simply name each mbox or Maildir according to the user name. First up is this possible? Secondly, how would I go about doing it? Sorry, I missed this at first. It is quite simple. I don't store it in passwd or any other place, since you are doing vmail, you might find this easiest: in /etc/dovecot/conf.d/10-mail.conf (where mail_location is): mail_home = /home/vmail/%Ld/%Ln mail_location = maildir:~/Maildir in /etc/dovecot/conf.d/10-mail.conf (after paragraph “# System user and group used to access mails...”): mail_uid=vmail mail_gid=vmail Obviously, vmail may not be your user for vmail. Also, some of my notes may no longer be accurate for location, just find where it exists and edit. I hope this helps. Of course, this is a Maildir setup. mbox is probably very similar, but I have had too many mbox style mail queues go south losing all of the mail (or more than one would like), so I do Maildir, even though it isn't necessarily the best use of disk space. Trever -- "I do not fear computers. I fear the lack of them." -- Isaac Asimov Hi, I'm just responding as the OP to say that the above was what I was looking for! Thanks Trever :-) Everything is setup and working fine now. Though responding quite late and of course having read through the latest messages within the thread I don't feel that my users will notice any difference between mbox or mdbox and Maildir format, speedwise. The reasoning behind this is that my end users unfortunately are all using M$ Outlook which is absolute garbage! FULL STOP! Comparing the IMAP capability speeds between Thunderbird and Outlook linking to my server yielded that I was able to get around 150Mbps transfer rate using T-Bird while Outlook only managed a few 100's of kbps. I think it's because 2010 relies heavily on PST's (whatever they are) and the fact it is ultimately M$ also so it's basically built by nincompoops to be sold at hideous prices and even higher tech-support prices. In all fairness to Outlook I did manage to get a pathetic ~2Mvbps tops of transfer.. :-S Luckily I'm the only one using T-Bird or Alpine so am fine :-) Can't send any mail though as need to go through Exchange - there's no winning in the corporate world :-( Regards, Kaya
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
The only draw back I see with maildir is one file per message. This it is mostly adventage. makes it resilient to corruption that mbox sees (if a message gets corrupted, you erase one message and that corruption won't propagate even if you leave it in place). In many setups this also leads to MUCH faster system. On unix systems this doesn't just waste disk space, it even with 32kB block/4kB fragment filesystem under FreeBSD which is my common setup, it isn't that a problem. i just checked one of my users folder - 2.3GB in 8500 files. the average is 270 kilobytes per mail. checked few others and it looks similar. dovecot's own storage system can do something in between - packing smallest messages by a few in one file. could lead to inode (or whatever your *nix of choice calls it) depletion. you decide how much inode you need while creating filesystem on every unix system, except filesystems where it is allocated on demand. What you will gain is clear separation of mails. You may in any case use widely available standard unix tools to move, delete, search, whatever with this files, and dovecot would rebuild it's indexes then. The other major gain are backups. With one file per mail differential/incremental backups will work fine. As everyone do backups this is important, unless you have so cheap and quick backup system that you can just do full backup most cases. Tapes, while certainly fast, are unfortunately not a cheap solution anymore. I don't mean drive, but cartridges. The disadventages are more I/O when multiple files are processed but it is not a common case. Dovecot makes great job in indexing. The other may be (with linux) slow operation on huge directories. I wasn't using linux for 6 years and that's only what i am told from others. Possibly it is already improved in linux. In FreeBSD there is compile time option UFS_DIRHASH for kernel that make even million file directories work quick. as of latter discussion about what microsoft recommends with linux (being of course expert of everything) - i would keep silent.
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 06/25/2012 01:52 AM, Kaya Saman wrote: > On Mon, Jun 25, 2012 at 8:37 AM, Trever L. Adams > wrote: >> On 06/25/2012 01:20 AM, Kaya Saman wrote: >> >> Now what I would like to know is, which is better for "virtual >> hosting" Maildir or mbox? >> >> >> Basically my requirement is that I would like to separate users via >> either individual folders and then put each user's mbox or Maildir in >> the created directory, or simply name each mbox or Maildir according >> to the user name. >> >> >> First up is this possible? >> >> >> Secondly, how would I go about doing it? >> >> Sorry, I missed this at first. It is quite simple. I don't store it in >> passwd or any other place, since you are doing vmail, you might find this >> easiest: >> >> in /etc/dovecot/conf.d/10-mail.conf (where mail_location is): >> >> mail_home = /home/vmail/%Ld/%Ln >> mail_location = maildir:~/Maildir >> >> in /etc/dovecot/conf.d/10-mail.conf (after paragraph “# System user and >> group used to access mails...”): >> >> mail_uid=vmail >> mail_gid=vmail >> >> Obviously, vmail may not be your user for vmail. Also, some of my notes may >> no longer be accurate for location, just find where it exists and edit. >> >> I hope this helps. Of course, this is a Maildir setup. mbox is probably very >> similar, but I have had too many mbox style mail queues go south losing all >> of the mail (or more than one would like), so I do Maildir, even though it >> isn't necessarily the best use of disk space. >> >> Trever >> -- >> "I do not fear computers. I fear the lack of them." -- Isaac Asimov > Thanks for the responses! > > Sorry if I reply to every single one in this email however, I am using > Gmail's awful Web UI so I don't really have much control over what I'm > doing.. > > > To start with the reason I'm not using LDAP is because I couldn't find > enough information on how to set it up! I did post here a couple of > times but got no responses.. so I figured it was something that > people either didn't know or found trivial. Sorry, I missed these. I have been busy. Tim and others were very kind and helped me figure things out. I do not have any web sites where I can post things, so I will give an overview here and if you need help, either follow up privately or here. > > I'll take a look at the above config for Maildir format as briefly > playing around with mbox it seems that folders on the / root (parent) > IMAP directory are stored separately. It may be better if everything > got stored under the Maildir heading I've previously **only** ever > worked with Maildir but I was told that there are some benefits to > mbox which is why I decided to try to use it here! > > Regards, > > > Kaya > The only draw back I see with maildir is one file per message. This makes it resilient to corruption that mbox sees (if a message gets corrupted, you erase one message and that corruption won't propagate even if you leave it in place). In many setups this also leads to MUCH faster system. On unix systems this doesn't just waste disk space, it could lead to inode (or whatever your *nix of choice calls it) depletion. I haven't yet seen this in my setups. I archive most of my personal mail, so I have at least some of my mail going back to 1998, I think. Kerberos Directions: Microsoft suggests on linux systems that you create an account (separate from the HOST account) and add the appropriate user principal names. There are pros and cons to this, their way is a and doing it as part of the host account is b. a) On S4 dc (replace MAILSERVER_HOST and MAILSERVER_FQDN with host and host.example.org, in lowercase as I use the caps only to help me see what I need to change, respectively as fits your domain - i.e. not example.org and create some long random password and put it wherever you see $RANDOMPASSWORD) : /usr/local/samba/sbin/samba-tool newuser mail-MAILSERVER_HOST /usr/local/samba/sbin/samba-tool spn add imap/MAILSERVER_FQDN mail-MAILSERVER_HOST /usr/local/samba/sbin/samba-tool spn add smtp/MAILSERVER_FQDN mail-MAILSERVER_HOST /root/samba-master/source4/scripting/bin/ktpass.sh --out /tmp/mail.keytab --princ smtp/MAILSERVER_FQDN --path-to-ldbsearch /usr/local/samba/bin/ --pass $RANDOMPASSWORD /root/samba-master/source4/scripting/bin/ktpass.sh --out /tmp/mail.keytab --princ imap/MAILSERVER_FQDN --path-to-ldbsearch /usr/local/samba/bin/ --pass $RANDOMPASSWORD /root/samba-master/source4/scripting/bin/ktpass.sh --out /tmp/mail.keytab --princ mail-MAILSERVER_HOST --path-to-ldbsearch /usr/local/samba/bin/ --pass $RANDOMPASSWORD Move the mail.keytab to dovecot's main configuration directory on dovecot server (/etc/dovecot here). Then do the appropriate version of: chmod 640 /etc/dovecot/mail.keytab chown dovecot.dovenull /etc/dovecot/mail.keytab b) Make sure your local samba setup is joined to the domain. Make sure it writes an appropriate krb5.keytab (/etc/krb5.keytab in my setup) as part of its password management, etc. net ads keytab add smt
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 25 Jun 2012, at 10:44, Charles Marcus wrote: > On 2012-06-25 3:58 AM, J E Lyon wrote: >> I've not come up with any significant advantages of mbox that count >> for much in my experiences and installations . . Would be interested >> to hear of suggested advantages that I might have overlooked > > One major advantage of mbox (and now mdbox) over maildir is the time it takes > to back things up for larger mail stores. > > It takes much less time to compare a single mbox file that contains 20,000 > messages (and rsync only the changed bits) than it does to compare > read/compare 20,000 individual files (maildir)... > > I too like maildir, but am seriously considering implementing a solution > where older mail is automatically archived to slower/cheaper SATA III based > storage using mdbox format. Very interesting. I use "rdiff-backup" as a cronjob in the wee hours (, so the fact it takes a while to work out what to incrementally save, doesn't matter to me -- but the point is that I know it's an issue and have addressed it that way. I too have contemplated some automated archiving of older mail . . it's getting closer to needing to be addressed at some point soon I think. J.
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On Mon, 25 Jun 2012 08:52:51 +0100 Kaya Saman wrote: > To start with the reason I'm not using LDAP is because I couldn't find > enough information on how to set it up! I did post here a couple of > times but got no responses.. so I figured it was something that > people either didn't know or found trivial. For my own use I've switched to the LDAP as provider of user information and credentials (for Dovecot/Postfix/ejabberd/anything I can get to talk to the LDAP). It's not that hard to figure out, but getting used to LDAP itself can take a little bit of time. In my case I'm using the LDAP just for checking if a user is present on the system and for authentication purposes (for the mail server). Haven't tried using quota etc with LDAP. Anything in particular you're having problems coping with? :) -- Branko Majic Jabber: bra...@majic.rs Please use only Free formats when sending attachments to me. Бранко Мајић Џабер: bra...@majic.rs Молим вас да додатке шаљете искључиво у слободним форматима.
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 2012-06-25 3:58 AM, J E Lyon wrote: I've not come up with any significant advantages of mbox that count for much in my experiences and installations . . Would be interested to hear of suggested advantages that I might have overlooked One major advantage of mbox (and now mdbox) over maildir is the time it takes to back things up for larger mail stores. It takes much less time to compare a single mbox file that contains 20,000 messages (and rsync only the changed bits) than it does to compare read/compare 20,000 individual files (maildir)... I too like maildir, but am seriously considering implementing a solution where older mail is automatically archived to slower/cheaper SATA III based storage using mdbox format. -- Best regards, Charles
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On Mon, Jun 25, 2012 at 10:31 AM, Charles Marcus wrote: > On 2012-06-25 3:20 AM, Kaya Saman wrote: >> >> # cat dovecot.conf >> # v1.2+: >> auth_use_winbind = yes > > > Please always only provide output of doveconf -n, not copy/pastes from the > config files. > > This proves (to yourself and everyone else) that you are using the config > that dovecot is actually using - it this shows you mistakes like typos, > certain deprecated/invalid settings, and even if you are editing the wrong > config file(s). > > -- > > Best regards, > > Charles Thanks for the tip! I didn't know of the dovecot -n command so thanks for pointing that out to me.. It's strange as I've been fiddling around with mail servers for some time in test labs at home but I still feel like I'm on the outside looking in; oh well at least this design at work is much better even though it took forever to get the PAM potion for AD sorted out. Regards, Kaya
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 2012-06-25 3:20 AM, Kaya Saman wrote: # cat dovecot.conf # v1.2+: auth_use_winbind = yes Please always only provide output of doveconf -n, not copy/pastes from the config files. This proves (to yourself and everyone else) that you are using the config that dovecot is actually using - it this shows you mistakes like typos, certain deprecated/invalid settings, and even if you are editing the wrong config file(s). -- Best regards, Charles
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 25 Jun 2012, at 08:52, Kaya Saman wrote: > I've previously **only** ever worked with Maildir but I was told that there > are some benefits to mbox which is why I decided to try to use it here! I used mbox before Dovecot, but once I found Maildir, I never looked back. I've not come up with any significant advantages of mbox that count for much in my experiences and installations . . Would be interested to hear of suggested advantages that I might have overlooked or know of reasons why they're not an issue . . not sure how much the list wants to hear, but feel free to email me direct if you want. J.
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On Mon, Jun 25, 2012 at 8:37 AM, Trever L. Adams wrote: > On 06/25/2012 01:20 AM, Kaya Saman wrote: > > Now what I would like to know is, which is better for "virtual > hosting" Maildir or mbox? > > > Basically my requirement is that I would like to separate users via > either individual folders and then put each user's mbox or Maildir in > the created directory, or simply name each mbox or Maildir according > to the user name. > > > First up is this possible? > > > Secondly, how would I go about doing it? > > Sorry, I missed this at first. It is quite simple. I don't store it in > passwd or any other place, since you are doing vmail, you might find this > easiest: > > in /etc/dovecot/conf.d/10-mail.conf (where mail_location is): > > mail_home = /home/vmail/%Ld/%Ln > mail_location = maildir:~/Maildir > > in /etc/dovecot/conf.d/10-mail.conf (after paragraph “# System user and > group used to access mails...”): > > mail_uid=vmail > mail_gid=vmail > > Obviously, vmail may not be your user for vmail. Also, some of my notes may > no longer be accurate for location, just find where it exists and edit. > > I hope this helps. Of course, this is a Maildir setup. mbox is probably very > similar, but I have had too many mbox style mail queues go south losing all > of the mail (or more than one would like), so I do Maildir, even though it > isn't necessarily the best use of disk space. > > Trever > -- > "I do not fear computers. I fear the lack of them." -- Isaac Asimov Thanks for the responses! Sorry if I reply to every single one in this email however, I am using Gmail's awful Web UI so I don't really have much control over what I'm doing.. To start with the reason I'm not using LDAP is because I couldn't find enough information on how to set it up! I did post here a couple of times but got no responses.. so I figured it was something that people either didn't know or found trivial. I'll take a look at the above config for Maildir format as briefly playing around with mbox it seems that folders on the / root (parent) IMAP directory are stored separately. It may be better if everything got stored under the Maildir heading I've previously **only** ever worked with Maildir but I was told that there are some benefits to mbox which is why I decided to try to use it here! Regards, Kaya
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 06/25/2012 01:20 AM, Kaya Saman wrote: > Now what I would like to know is, which is better for "virtual > hosting" Maildir or mbox? > > > Basically my requirement is that I would like to separate users via > either individual folders and then put each user's mbox or Maildir in > the created directory, or simply name each mbox or Maildir according > to the user name. > > > First up is this possible? > > > Secondly, how would I go about doing it? > Sorry, I missed this at first. It is quite simple. I don't store it in passwd or any other place, since you are doing vmail, you might find this easiest: in /etc/dovecot/conf.d/10-mail.conf (where mail_location is): mail_home = /home/vmail/%Ld/%Ln mail_location = maildir:~/Maildir in /etc/dovecot/conf.d/10-mail.conf (after paragraph “# System user and group used to access mails...”): mail_uid=vmail mail_gid=vmail Obviously, vmail may not be your user for vmail. Also, some of my notes may no longer be accurate for location, just find where it exists and edit. I hope this helps. Of course, this is a Maildir setup. mbox is probably very similar, but I have had too many mbox style mail queues go south losing all of the mail (or more than one would like), so I do Maildir, even though it isn't necessarily the best use of disk space. Trever -- "I do not fear computers. I fear the lack of them." -- Isaac Asimov
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 25 Jun 2012, at 08:20, Kaya Saman wrote: > Now what I would like to know is, which is better for "virtual > hosting" Maildir or mbox? I always use Maildir in preference to mbox . . it's just such a lovely solution, imho :) (Mind you, I'm on a *nix server, so filesystem behaviour may be a consideration for non-*nix hosts.) > Basically my requirement is that I would like to separate users via > either individual folders and then put each user's mbox or Maildir in > the created directory, or simply name each mbox or Maildir according > to the user name. > > First up is this possible? > > Secondly, how would I go about doing it? In haste, I haven't been able to check your email thoroughly, but I have used passwd with an extra field appended to each user, to identify the mailbox location. If you're authenticating against ActiveDirectory, then I guess that means generating a passwd-file style 'database' from the users in ActiveDirectory and I have no idea if that's trivial. Sorry if my rushed thoughts are too sketchy to be of use, but thought I'd share my experience in case it offers any pointers. J.
Re: [Dovecot] Virtual users - what is better Maildir or mbox?
On 06/25/2012 01:20 AM, Kaya Saman wrote: > Hi, > > I'm trying to configure a server to use SAMBA and Winbind to > authenticate to Active Directory I managed to get this portion up > and running even the Dovecot portion. > > For reference something similar to this guide: > > http://www.whitneytechnologies.com/?p=119 > > > However PAM is slightly different: > > > # cat /etc/pam.d/dovecot > authsufficient pam_krb5.so no_user_check validate > account sufficient pam_permit.so > > > This is what my dovecot.conf file looks like: If you are using Samba 4 (possibly recent versions of S3), any reason you are not doing krb5 and ldap (for account verification, not authentication) on dovecot instead of through pam? It is a bit harder to setup, but no text passwords. I still do pam_krb5 for devices that cannot do kerberos. I am using Maildir, but my setup is currently largely idle. Trever -- "Marxist Law of Distribution of Wealth: Shortages will be divided equally among the peasants." -- Unknown
[Dovecot] Virtual users - what is better Maildir or mbox?
Hi,
I'm trying to configure a server to use SAMBA and Winbind to
authenticate to Active Directory I managed to get this portion up
and running even the Dovecot portion.
For reference something similar to this guide:
http://www.whitneytechnologies.com/?p=119
However PAM is slightly different:
# cat /etc/pam.d/dovecot
authsufficient pam_krb5.so no_user_check validate
account sufficient pam_permit.so
This is what my dovecot.conf file looks like:
# cat dovecot.conf
# v1.2+:
auth_use_winbind = yes
auth_winbind_helper_path = /usr/local/bin/ntlm_auth
protocols = imap
# It's nice to have separate log files for Dovecot. You could do this
# by changing syslog configuration also, but this is easier.
log_path = /var/log/dovecot.log
info_log_path = /var/log/dovecot-info.log
# Disable SSL for now.
ssl = no
disable_plaintext_auth = no
# We're using Maildir format
#mail_location = maildir:~/Maildir
mail_location = mbox:/mail:INBOX=/mail/%u
# Authentication configuration:
auth_verbose = yes
auth_debug = yes
auth_username_format = %n
auth_mechanisms = plain ntlm login
userdb {
driver = static
# args = uid=501 gid=1001 home=/mail/%u
args = home=/mail/%u
driver = static
}
passdb {
driver = pam
args = failure_show_msg=yes
}
Now what I would like to know is, which is better for "virtual
hosting" Maildir or mbox?
Basically my requirement is that I would like to separate users via
either individual folders and then put each user's mbox or Maildir in
the created directory, or simply name each mbox or Maildir according
to the user name.
First up is this possible?
Secondly, how would I go about doing it?
My users are not allowed to login to the system outside of IMAP as
it's a Mail only server.
Currently I've been looking at many links:
http://satish-linuxbug.blogspot.co.uk/2008/08/freebsd-with-active-directory-single.html
http://joseph.randomnetworks.com/2005/11/08/freebsd-users-and-groups-with-samba-winbind-and-active-directory/
http://wiki2.dovecot.org/HowTo/ActiveDirectoryNtlm
http://wiki2.dovecot.org/Authentication/Mechanisms/Winbind
http://wiki2.dovecot.org/TestInstallation
http://wiki2.dovecot.org/VirtualUsers
http://www.linuxmail.info/active-directory-dovecot-pam-authentication/
http://wiki2.dovecot.org/HowTo/SimpleVirtualInstall
I'm running dovecot version 2.1.7 on FreeBSD 8.2 x64 RELEASE.
The system is not being used as an MTA server meaning that only IMAP
transfers are being done using MS Outlook then filtered by
Thunderbird.
Regards,
Kaya
