Re: doveadm backup to local drive
> On Feb 15, 2016, at 11:43 AM, Terry Barnum <terry...@gmail.com> wrote:
>
> I'm trying to get doveadm to backup all users to a local drive without any
> luck.
>
> $ sudo doveadm backup -A /Volumes/mail_bak/mailstore_backup/
> Error: User listing returned failure
> doveadm: Error: Failed to iterate through some users
>
> The doveadm-backup man page has the instruction to make sure the iterate
> query matches the database layout but I'm not sure what that means.
>
> If the user_query in dovecot-sql.conf.ext looks like this, what should the
> iterate_query look like?
>
> user_query = \
> SELECT '/opt/local/virtual/%d/%n/mail' as home, \
> 'maildir:/opt/local/virtual/%d/%n/mail' as mail, 501 as uid, \
> 501 as gid, concat("*:storage=", quota) AS quota \
> FROM mailbox WHERE username = "%u" AND active = "1"
>
> # Query to get a list of all usernames.
> iterate_query = SELECT username AS user FROM users
I didn't see my original email show up on the mailing list so I'm resending. To
recap, is it possible to backup all users to a local drive with doveadm?
Thanks,
-Terry
Terry Barnum
digital OutPost
Carlsbad, CA
http://www.dop.com
800/464-6434
doveadm backup to local drive
I'm trying to get doveadm to backup all users to a local drive without any luck.
$ sudo doveadm backup -A /Volumes/mail_bak/mailstore_backup/
Error: User listing returned failure
doveadm: Error: Failed to iterate through some users
The doveadm-backup man page has the instruction to make sure the iterate query
matches the database layout but I'm not sure what that means.
If the user_query in dovecot-sql.conf.ext looks like this, what should the
iterate_query look like?
user_query = \
SELECT '/opt/local/virtual/%d/%n/mail' as home, \
'maildir:/opt/local/virtual/%d/%n/mail' as mail, 501 as uid, \
501 as gid, concat("*:storage=", quota) AS quota \
FROM mailbox WHERE username = "%u" AND active = "1"
# Query to get a list of all usernames.
iterate_query = SELECT username AS user FROM users
Thanks,
-Terry
Terry Barnum
digital OutPost
Carlsbad, CA
http://www.dop.com
800/464-6434
restrict map-login by geoip?
Is there a way to restrict my user logins from a set of IPs? For example, all my users are in the US so there shouldn't be any logins from other countries. Can I tell dovecot to restrict logins to a CIDR list of US IPs? Can someone point me to docs on how to set this up? I've searched but haven't found how to accomplish this. Thanks, -Terry Terry Barnum digital OutPost http://www.dop.com
Re: restrict map-login by geoip?
Thanks Benny. I should've said I saw AllowNets but in researching it looked like it expected a smaller comma separated list, not hundreds of IP blocks. Is that what you are using to accomplish this? Thanks, -Terry iPhone says Hello World! > On Sep 16, 2015, at 6:31 PM, Benny Pedersen <m...@junc.eu> wrote: > > Terry Barnum skrev den 2015-09-17 02:32: > >> I've searched but haven't found how to accomplish this. > > http://wiki2.dovecot.org/Authentication/RestrictAccess > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets > > took me 3 sec :=) >
Re: Transition from one server to another.
On Jul 14, 2014, at 1:22 PM, Anders Wegge Keller we...@wegge.dk wrote: A frind of mine and I are running a dedicated server, that among other things host mail for ourselves and friends and families. All in all about 15 different domains with 35-40 users. The machine in question is old, so we are doing a slow transition from the old server to the new one. So far, we've managed to move web hosts seamlessly. Due to the technical capabilities at some of the user base, it would be nice to get to a setup, where we can move individual users imaps from the old server to the new one, as we get the time to visit them. I have an idea how such a transition could go: 1. Upgrade the old dovecot 1.2.15 to 2.1.whateveritis from debian squeeze backports. 2. Set dsync up to replicate mails from the old server to the new server. I know that 2.2 is recommended, but with a limited amount of user, I'm willing to take a performance hit. 3. Migrate my parents c to use the new server. 4. When all users have been moved on to using the new server, upgrade MX records for the domain to point at the new server. 5. When all MX records are updated, decalre success. Is this feasible, and what would the risks be. For instance, during step 4, mails are bound to arrive at bothe the old and new server for some time. Will this cause problems? Is there a simpler solution to the problem? -- /Wegge Leder efter redundant peering af dk.*,linux.debian.* I did this several years ago using imapsync to sync the accounts from an old server to a new server. Once all the accounts have been sync'd and you're ready to cut over to the new server, run imapsync one last time to catch anything new that's arrived. It will go very quickly. Disable postfix/dovecot on the old server and change your DNS to point to the new server. Or better yet, if the old and new servers are on the same network, just swap the names and IPs of the new and old servers. -Terry Terry Barnum digital OutPost http://www.dop.com
Re: [Dovecot] undoing a zlib test
On Dec 9, 2013, at 11:41 PM, Robert Schetterer r...@sys4.de wrote: Am 10.12.2013 01:08, schrieb Terry Barnum: I'm trying to undo a dovecot zlib test and uncompress 6 emails in my Junk mailbox. There are about 25 total in different mailboxes but I thought I'd first start with those in my Junk. I ran: # dsync -v -u te...@dop.com -m Junk -o plugin/zlib_save= backup maildir:/path/to/temp/folder which returned without error and backed up my Junk mailbox but didn't uncompress the 6 emails. Am I calling dsync incorrectly to uncompress? zlib is currently not loaded as a plugin. as far i know, dove/dsync always looks at the conf, so to notice compressed mail the zlib plugin might have to be enabled ever , -o plugin/zlib_save= should be fine to decompress, please retest with zlib plugin enabled, testmails to decompress should be compressed by dovecot itself and not from elsewhere, that should work as my tests with 2.1.x, if it fails again it might be broken with 2.2.9 Thank you Robert, you were correct. I temporarily enabled zlib and ran dsync with verbose output. I think I could've also just added -o mail_plugins=zlib but I didn't think of it until later. # dsync -v -u te...@dop.com -m MyMailbox -o plugin/zlib_save= backup maildir:/path/to/temp/folder dsync(te...@dop.com): Error: Cached message size smaller than expected (4236 8192) dsync(te...@dop.com): Error: Maildir filename has wrong S value, renamed the file from .../terry/.MyMailbox/cur/1386272980.M20857P20282.mailbox.dop.com,S=4236:2,S to .../terry/.MyMailbox/cur/1386272980.M20857P20282.mailbox.dop.com,S=4236:2,S dsync(te...@dop.com): Error: Corrupted index cache file .../terry/.MyMailbox/dovecot.index.cache: Broken physical size for mail UID 37695 These 3 lines are repeated again. Why does dsync rename the file to the same name? I was able to import the now uncompressed email back into dovecot: # doveadm -v import -u te...@dop.com maildir:/path/to/temp/folder mailbox MyMailbox SENTON 2013-12-05 doveadm(te...@dop.com): Error: Cached message size smaller than expected (4236 8192) doveadm(te...@dop.com): Error: Maildir filename has wrong S value, renamed the file from /path/to/temp/folder/.MyMailbox/cur/1386272980.M20857P20282.mailbox.dop.com,S=4236:2,S to /path/to/temp/folder/.MyMailbox/cur/1386272980.M20857P20282.mailbox.dop.com,S=10267:2,S doveadm(te...@dop.com): Error: Corrupted index cache file /path/to/temp/folder/.MyMailbox/dovecot.index.cache: Broken physical size for mail UID 37695 doveadm(te...@dop.com): Error: Cached message size smaller than expected (4236 10267) doveadm(te...@dop.com): Error: Corrupted index cache file /path/to/temp/folder/.MyMailbox/dovecot.index.cache: Broken physical size for mail UID 37695 Hopefully this might help someone who ends up in the same situation. -Terry Terry Barnum digital OutPost http://www.dop.com
[Dovecot] undoing a zlib test
I'm trying to undo a dovecot zlib test and uncompress 6 emails in my Junk
mailbox. There are about 25 total in different mailboxes but I thought I'd
first start with those in my Junk. I ran:
# dsync -v -u te...@dop.com -m Junk -o plugin/zlib_save= backup
maildir:/path/to/temp/folder
which returned without error and backed up my Junk mailbox but didn't
uncompress the 6 emails. Am I calling dsync incorrectly to uncompress? zlib is
currently not loaded as a plugin.
Once I get them uncompressed is doveadm import the correct way to get them back
into my Junk mailbox? Something like this?
# doveadm -v import -u te...@dop.com maildir:/path/to/temp/folder mailbox
Junk SENTON 2013-12-06
Thanks,
-Terry
# 2.2.9: /opt/local/etc/dovecot/dovecot.conf
# OS: Darwin 11.4.2 x86_64 hfs
auth_verbose = yes
base_dir = /opt/local/var/run/dovecot/
default_internal_user = _dovecot
default_login_user = _dovenull
first_valid_uid = 102
last_valid_uid = 105
listen = *
mail_gid = 102
mail_location = maildir:/Volumes/mail/vmail/%d/%u
mail_uid = 102
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = /opt/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
plugin {
antispam_backend = dspam
antispam_debug_target = syslog
antispam_dspam_args = --user;%u;--deliver=;--source=error;--signature=%%s
antispam_dspam_binary = /opt/local/bin/dspamc
antispam_dspam_notspam = --class=innocent
antispam_dspam_result_header = X-DSPAM-Result
antispam_dspam_spam = --class=spam
antispam_signature = X-DSPAM-Signature
antispam_signature_missing = move
antispam_spam_pattern_ignorecase = junk;spam;junk *
antispam_trash_pattern_ignorecase = trash;deleted *
mail_log_events = flag_change append delete undelete expunge copy
sieve = ~/dovecot.sieve
sieve_before = /Volumes/mail/vmail/%d/global/global.sieve
sieve_dir = ~/sieve
}
protocols = imap pop3 lmtp sieve
sendmail_path = /opt/local/sbin/sendmail
service auth-worker {
user = $default_internal_user
}
service auth {
unix_listener /opt/local/var/spool/postfix/private/auth {
group = _postfix
mode = 0660
user = _postfix
}
unix_listener auth-userdb {
mode = 0660
user = _vmail
}
}
ssl_cert = /opt/local/etc/ssl/certs/dovecot.cert
ssl_key = /opt/local/etc/ssl/private/dovecot.key
userdb {
args = /opt/local/etc/dovecot/dovecot-sql.conf.ext
driver = sql
}
protocol lmtp {
mail_plugins = sieve
}
protocol lda {
mail_plugins = sieve
postmaster_address = postmas...@dop.com
}
protocol imap {
mail_max_userip_connections = 50
mail_plugins = antispam
}
Terry Barnum
digital OutPost
http://www.dop.com
Re: [Dovecot] zlib config questions
On Dec 6, 2013, at 8:23 AM, Robert Schetterer r...@sys4.de wrote: Am 06.12.2013 17:01, schrieb Charles Marcus: On 2013-12-06 10:55 AM, Richard Platel rpla...@tucows.com wrote: One gotcha if you're using maildir is, if you're planning to compress old mail, make sure that the S= and W= sizes in the filenames are correct, And how do you make sure of this? Is there a script? Or do you just pray? this is only if you manual convert the whole store i.e with dsync to get more free space , if you simple enable the plugin you have not be afraid cause only new mail will may get compressed but however zlib and maildir had bugs in some dove releases or version updates, see list archive how people fixed it Thank you Richard, Charles and Robert for your insight and suggestions. I'm curious how many people here use the zlib plugin. Is it common or is drive space cheap enough that it's not worth it to run the risk of potential problems? I am using Maildir and did read the threads from last year of problems with the manually compressed mail S and W values being incorrect as well as other filename problems. Have these been fixed in dovecot = v2.2.5? Once you enable zlib, does this mean you must use it from that point on? Once email is compressed, if you disable the plugin does dovecot lose the ability to access those compressed emails? Thanks, -Terry Terry Barnum digital OutPost http://www.dop.com
[Dovecot] zlib config questions
After nearly running out of space I swapped in larger disks and then saw the
recent threads about zlib compression. Unfortunately I'm still confused after
reading http://wiki2.dovecot.org/Plugins/Zlib.
In order to compress new email being stored do I only need to change
10-mail.conf to this:
mail_plugins = $mail_plugins zlib
and 90-plugin.conf to this:
plugin {
zlib_save_level = 6
zlib_save = gz
...
Or do I need to instead (or also?) add it to the list of plugins in
20-imap.conf?
A simple 'sudo doveadm reload' to enable?
Once enabled, is there an indication in the logs that compression is taking
place?
Do new mail files have a suffix like .Z to indicate they've been compressed?
Any gotchas to be aware of?
I'm running macports dovecot 2.2.5.
Thanks for any help.
-Terry
Terry Barnum
digital OutPost
http://www.dop.com
[Dovecot] dovecot2, antispam and sieve versions
I'm currently running dovecot2 v2.2.5 with dovecot2-antispam 0.0-51 and dovecot2-sieve 0.4.1, all via macports. If one wanted to downgrade to dovecot v2.2.0, would these versions of antispam and sieve still be okay to use? The reason for the question is I'm trying to diagnose a mailserver problem where after 2-6 days the machine goes deaf. Absolutely nothing of interest in any logfile that I can see except for an imap-login process crash which got me thinking about a downgrade test. Thanks, -Terry Terry Barnum digital OutPost http://www.dop.com
Re: [Dovecot] mysql log entry
On Apr 16, 2010, at 4:09 AM, Timo Sirainen wrote: On Tue, 2010-04-13 at 16:24 -0700, Terry Barnum wrote: I'm sure this is a really newbie question, but I notice in /var/log/mail.log that there are between 3 and 15 of the following log entries every day for the last five days, which is far back as my logs go--what do they mean? Apr xx hh:mm:ss mail dovecot[109]: auth-worker(default): mysql: Connected to localhost (postfix) Is it dovecot connecting to mysql for password or user info? If so, wouldn't it be more frequent? I've got 20 users with some connecting from outside many times per day. Dovecot uses persistent connections to MySQL. Above probably means that a new auth-worker process was created and it connected to MySQL. Auth-worker processes also die away automatically after idling for a while. If your server had more traffic, the worker processes wouldn't die and so new worker processes wouldn't be created either, and you wouldn't get those messages. Thank you very much Timo. It's what I hoped was going on but as a new dovecot user it's quite reassuring to hear it from the actual, real life auth(oritative)-worker! -Terry
[Dovecot] mysql log entry
I'm sure this is a really newbie question, but I notice in /var/log/mail.log that there are between 3 and 15 of the following log entries every day for the last five days, which is far back as my logs go--what do they mean? Apr xx hh:mm:ss mail dovecot[109]: auth-worker(default): mysql: Connected to localhost (postfix) Is it dovecot connecting to mysql for password or user info? If so, wouldn't it be more frequent? I've got 20 users with some connecting from outside many times per day. Thanks, -Terry
Re: [Dovecot] dovecot user
-Terry Sent from my iPhone On Mar 25, 2010, at 9:50 PM, Bradley Giesbrecht bradley.giesbre...@gmail.com wrote: On Mar 25, 2010, at 9:48 PM, Stan Hoeppner wrote: Timo Sirainen put forth on 3/25/2010 1:30 PM: I think the next v2.0 release (rc1?) will include the new changed default_login_user. I'm still not completely sure what it is though. Two more ideas: dovenest (by Pascal Volk) - although reminds me a bit too much of lovenest :) dovehole - you go inside dovecot via a hole, right? That is downright pornographic. dovehole - lovehole? dovenest isn't totally horrible (close), but dovehole is ... just not right at all. dovetail
Re: [Dovecot] dovecot user
On Mar 25, 2010, at 9:50 PM, Bradley Giesbrecht bradley.giesbre...@gmail.com wrote: On Mar 25, 2010, at 9:48 PM, Stan Hoeppner wrote: Timo Sirainen put forth on 3/25/2010 1:30 PM: I think the next v2.0 release (rc1?) will include the new changed default_login_user. I'm still not completely sure what it is though. Two more ideas: dovenest (by Pascal Volk) - although reminds me a bit too much of lovenest :) dovehole - you go inside dovecot via a hole, right? That is downright pornographic. dovehole - lovehole? dovenest isn't totally horrible (close), but dovehole is ... just not right at all. dovetail Nice!!
Re: [Dovecot] Removing Duplicates
On Mar 14, 2010, at 4:21 AM, Sabahattin Gucukoglu wrote: Hi all, I am starting fresh with a local repository of mails, which almost certainly have duplicates in them. I am going to use maildirs, and ensure all mails are input with CRLFs. The question is: does anybody know how I can find and remove duplicates, either while injecting mail with IMAP, or afterward? I can use tools to find duplicate Message-IDs, but don't know of a way to remove duplicates in mailboxes that are already imported as opposed to incoming mail. Perhaps there is a way to use the IMAP protocol for this? http://freshmeat.net/projects/imapsync It will skip duplicates during transfer. -Terry
Re: [Dovecot] Limit login attempts per connection?
On Fri, 05.03.2010 at 09:44:35 +, Ed W li...@wildgooses.com wrote: I would be all in favour of a setting like this because it's easier to configure than fail2ban... There's also denyhosts. http://denyhosts.sourceforge.net/ -Terry
Re: [Dovecot] any limitations running on a Mac?
On Feb 27, 2010, at 2:08 AM, Stan Hoeppner wrote: Terry Barnum put forth on 2/26/2010 4:20 PM: The 500MB+, 5k+ message problem is with another mailserver not running postfix/dovecot. It's the reason I've started investigating postfix with your software and so far Im very happy. Hi Terry, If it is remotely a possibility, I'd recommend going with Linux instead of OSX. Going Linux gets you access to the XFS filesystem which gives the best overall performance for maildirs. It seems very likely that your current production server is suffering at 5K files and 500MB mailboxen due to a deficient filesystem, among other things. I'm partial to Debian Stable (Lenny) myself. I use mbox with XFS due to the fact I archive all my list mail and thus generate some pretty large mbox files, growing every day. If I decide to switch to or add maildirs later, I've already got the filesystem that performs best for both worlds. If you must go with OSX, avoid HFS+ and go with ZFS. ZFS performance is likely inferior to XFS for maildirs, but it would still be much better than HFS+. Thank you Stan. Unfortunately I read that Apple ran into licensing issues with Sun/Oracle over ZFS and removed it from Snow Leopard before it was released last year. -Terry
[Dovecot] any limitations running on a Mac?
I have postfix/dovecot/mysql installed using MacPorts on a quad-core 2.8GHz MacPro running Snow Leopard (10.6.2). I moved the base mail directory to a pair of 10k RPM Raptors that are mirrored (/Volumes/email/) and everything seems to be working fine on an unused domain with very little traffic. I used imapsync to pull everything from the current mailserver to this test server to play with. Our current mailserver's IMAP performance *really* suffers when an IMAP folder exceeds ~500MB or ~5k messages. I've become very tired of being the mailbox police trying to get my 20 users to delete email or divide into smaller mailboxes. Are there folks on the list running postfix/dovecot on similar Mac hardware that can share their experiences? Specifically, are there any limitations (file descriptors, other?) that can impact performance I should be aware of? How does dovecot on the Mac deal with 500GB maildirs? Is there a recommended tool for loading and testing the server? Postal? I have 20 IMAP users, with a total of ~37GB of mail currently. Thanks for any help and insight. -Terry Terry Barnum digital OutPost San Diego, CA http://www.dop.com 800/464-6434
Re: [Dovecot] any limitations running on a Mac?
On Feb 25, 2010, at 5:17 PM, Bradley Giesbrecht wrote: On Feb 25, 2010, at 4:31 PM, Terry Barnum wrote: On Feb 25, 2010, at 3:29 PM, Bradley Giesbrecht wrote: On Feb 25, 2010, at 2:17 PM, Terry Barnum wrote: I have postfix/dovecot/mysql installed using MacPorts on a quad-core 2.8GHz MacPro running Snow Leopard (10.6.2). I moved the base mail directory to a pair of 10k RPM Raptors that are mirrored (/Volumes/email/) and everything seems to be working fine on an unused domain with very little traffic. I used imapsync to pull everything from the current mailserver to this test server to play with. Our current mailserver's IMAP performance *really* suffers when an IMAP folder exceeds ~500MB or ~5k messages. I've become very tired of being the mailbox police trying to get my 20 users to delete email or divide into smaller mailboxes. Are there folks on the list running postfix/dovecot on similar Mac hardware that can share their experiences? Specifically, are there any limitations (file descriptors, other?) that can impact performance I should be aware of? How does dovecot on the Mac deal with 500GB maildirs? Is there a recommended tool for loading and testing the server? Postal? I have 20 IMAP users, with a total of ~37GB of mail currently. Thanks for any help and insight. Mac OS X 10.5.8 dovecot 1.2.9 Two mirrored 7200rpm sata drives. dbox mailbox format Thanks Brad. What hardware is this running on? Also, are you using the plain vanilla Apple RAID tool for mirroring or something else like SoftRAID? DP G5 2Ghz. 5.5GB ram. Apple Disk Utility software RAID 1. Users are virtual in mysql. I also have a fairly busy application hitting mysql. Mysql seems to be cacheing around 120 threads which is less the config max and I usually have a around 700MB of free memory. User Inbox count around 200 on 20 domains but we are a Print company and have to except large attachments and store indefinitely. I looked at the wiki on dbox but shied away from it because the compatibility matrix said postfix didn't like it. Did I read that wrong? Would you mind sharing your postfix -n and dovecot -n? Edited and offlist would be fine. Did you use macports to build dovecot? Yes. 1.2.10. If so I'm working of some new packages you may be interested in. Mostly dovecot-sieve and dovecot-managesieve. I have started to look at sieve and can see how it could be useful for my iPhone users. Currently they have to leave Mail.app running on their desktop to sort mail into IMAP folders since the phone doesn't have any sort by rules filters. So yes, I'm interested. Your welcome to hit me up off list if you have some non-dovecot questions. I'm small time compared to many here but I love dovecot and won't be looking elsewhere till I come home broke and bleeding. // Brad Terry Barnum digital OutPost San Diego, CA http://www.dop.com 800/464-6434
Re: [Dovecot] Anyone successfully setup Continous Backup of mailboxes using rsync ?
On Feb 10, 2010, at 11:53 AM, Charles Marcus wrote: On 2010-02-10 2:22 PM, Terry Barnum wrote: There's also rsnapshot which I've been using for hardlinked, incremental backups for a couple years now. Very flexible and has saved us a couple times. http://www.rsnapshot.org +1 for rsnapshot - although it doesn't answer the 'continuous' backup aspect of this thread... I'm pretty sure rsnapshot could be configured to run every x minutes. -Terry Terry Barnum digital OutPost San Diego, CA http://www.dop.com 800/464-6434
