Re: [Dovecot] Change to Maildir format (Was:Corrupted Mail?)
Well, with thanks to everyone on this list I have now successfully (I hope) switched from mbox to Maildir format. Moreover I am now using dovecot-lda to deliver. I went step by step, and tested it with just one account before opening up the whole system. I had to rewrite my procmail recipes twice because I first changed them to: | /usr/libexec/dovecot/deliver -d mark -m .MLists.Fedora/ but after testing with the Maildir format up and running I found that they had to be: | /usr/libexec/dovecot/deliver -d mark -m MLists.Fedora I will now leave it running for a couple of days. I will keep an eye on the logs, but I hope that I have now seen the last of the "Error: Next message unexpectedly corrupted ..." messages! Thanks again for all the help. Much appreciated! Mark
Re: [Dovecot] Change to Maildir format (Was:Corrupted Mail?)
On Thu, 2014-05-29 at 06:53 -0500, Larry Stone wrote: > On May 29, 2014, at 6:35 AM, Arthur Dent wrote: > > > 1) I have tried both /usr/libexec/dovecot/deliver > > and /usr/libexec/dovecot/dovecot-lda and they both work. Are they > > equivalent? If not which should I use? > > A ls -l /usr/libexec/dovecot will (or at least it does on my system) show you > that deliver is a link to dovecot-lda. > lrwxr-xr-x 1 root wheel 11 May 15 22:16 deliver -> dovecot-lda Oh yes. Mine too. Thanks for clearing that up!
[Dovecot] Change to Maildir format (Was:Corrupted Mail?)
On Wed, 2014-05-28 at 14:27 +0200, Sven Hartge wrote: > Larry Stone wrote: > > On May 28, 2014, at 2:03 AM, Arthur Dent > > wrote: > > >> DELIVER=/usr/libexec/dovecot/deliver > >> DESTDIR=/home/mark/mail/ > >> :0 w > >> * ^List-Id:.*users.lists.fedoraproject.org > >> | $DELIVER -m $DESTDIR/MLists/Fedora > > > But that’s not exactly what was suggested above. It’s missing the “-d > > $USER”. > > And I guess the mailbox name at -m is wrong. > > This gets expanded to: > > | /usr/libexec/dovecot/deliver -d mark -m /home/mark/mail/MLists/Fedora > > But with "-m" you have to use the mailbox name dovecot uses, which could > be for example > > "INBOX.MLists.Fedora" or "INBOX/MLists/Fedora" or "MLists.Fedora" or > even "MLists/Fedora" depending on your namespace configuration. > > I suggest to use "doveadm mailbox list" to find out what the dovecot > name of the target mailbox is. Thanks to everyone who helped me sort this out. This was the most helpful reply - I had been trying both with and without the -d switch without success. Using "doveadm mailbox list" showed me that the problem was with the delivery address. This is what I now have - and it works - Thanks to all: :0 w * ^List-Id:.*users.lists.fedoraproject.org | /usr/libexec/dovecot/deliver -d mark -m MLists/Fedora Another two questions now please if I may: 1) I have tried both /usr/libexec/dovecot/deliver and /usr/libexec/dovecot/dovecot-lda and they both work. Are they equivalent? If not which should I use? 2) Switching to maildir, should the new delivery line now read: | /usr/libexec/dovecot/deliver -d mark -m .MLists.Fedora/ (i.e. a dot before and after MLists and a trailing slash)? Thanks again for all the help and support. Much appreciated... Mark
[Dovecot] Change to Maildir format (Was:Corrupted Mail?)
I am seriously considering the change to Maildir format. I am however more than a little concerned that this could all go horribly wrong. This is what I am planning. Does it look sane? Checklist 1) Stop the mail queue (essentially stop fetchmail) 2) Stop Dovecot 3) Backup all mbox files 4) Edit all procmail recipes (actually I will probably do this first - working with copies - because this will be time-consuming) See below. 5) Convert mbox files to maildir using either mb2md or dsync (to be decided) 6) Change Dovecot config Actually - As far as I can tell no configuration change is required. Dovecot should just recognise the directories as maildirs. Is this right? 7) Restart Dovecot 8) Restart Fetchmail Have I missed anything? Procmail scripts: = I am quite concerned about this. I have a directory structure such as: Home Inbox <-mbox Work Inbox <-mbox HobbyFile <-mbox Malware <-directory L> Spam <-mbox L> Virus <-mbox MLists <-directory L> Fedora <-mbox L> Dovecot <-mbox L> Spamassassin <-mbox L> ...etc... My current procmail recipe for one example look like this: :0: * ^List-Id:.*users.lists.fedoraproject.org $DESTDIR/MLists/Fedora I'm guessing it would need to change to: :0 * ^List-Id:.*users.lists.fedoraproject.org $DESTDIR/.MLists.Fedora/ Is that right??? (Note: remove locking colon ":" and use of "." and trailing "/" in path) Please advise - I am rather nervous of making a rather big mess here! Thanks Mark
Re: [Dovecot] Corrupted Mail?
On Tue, 2014-05-27 at 16:31 -0700, Joseph Tam wrote: > Arthur Dent writes: > > > I get my mail from a variety of ISP and online email accounts using > > fetchmail. I guess there's no change needed there? > > > > That feeds into procmail. I think I will have to change the mail > > procmail configuration file and ALL my procmail recipe rc files (I have > > many) - which filter into several different mboxs? > > As others have intimated, the use of procmail as your LDA is causing your > mailbox to become inconsistent with respect to your indices. Procmail is > appending your messages to mailboxes, but is ignorant of the existence > of dovecot indices, and does not update them. When your IMAP service > access those mailboxes, it detects this condition, and that's the log > messages you are seeing. > > I think those messages are benign -- you can probably ignore it. > Dovecot will automatically rebuild them. > > The alternative is to let dovecot's be responsible for message delivery: > it can then update the indices, and IMAP/POP will find consistent indices. > > Try this at the end of procmailrc for regular delivery to the INBOX > > > > # Replace default action using dovecot-lda to maintains caches. > > > :0 w > | /location/of/dovecot-lda -d $USER > > For for non-INBOX filtering, try something like > > :0 Hw > *^Subject: TEST > |/location/of/dovecot-lda -d $USER -m testbox Thanks Joseph - this is really useful. By coincidence (whilst researching all of this) last night I cam across this page: http://wiki2.dovecot.org/procmail which says exactly what you suggested. I tried it as an experiment on just one of my mailing list recipes. It didn't work and dropped through to my default inbox. This is what I have: :0 w * ^List-Id:.*users.lists.fedoraproject.org | $DELIVER -m $DESTDIR/MLists/Fedora where: DELIVER=/usr/libexec/dovecot/deliver DESTDIR=/home/mark/mail/ are defined in a the main procmailrc This is what I found in the procmail log: lda: Fatal: destination user parameter (-d user) not given procmail: Program failure (64) of "/usr/libexec/dovecot/deliver" procmail: Couldn't determine implicit lockfile from "spamc" >From users-boun...@lists.fedoraproject.org Wed May 28 01:39:03 2014 Subject: Five Things in Fedora This Week (2014-05-27) Why did this mail not get delivered to /home/mark/mail/MLists/Fedora? Thanks again Mark
Re: [Dovecot] Corrupted Mail?
On Tue, 2014-05-27 at 16:45 +0200, Patrick De Zordo wrote: > > You would *really* not change do Maildir format? > It's not so difficult to switch over.. and you will have working and simple > mailfolders.. (also backup and restore would be a simple task).. > > So, why won't you change at all? Well - the more I read about this, the more I think you're right. Perhaps I should convert to Maildir - but I'm terrified... I have seen that there is a script - mb2md - that I can use, but what changes do I need to make elsewhere? I get my mail from a variety of ISP and online email accounts using fetchmail. I guess there's no change needed there? That feeds into procmail. I think I will have to change the mail procmail configuration file and ALL my procmail recipe rc files (I have many) - which filter into several different mboxs? Within one of the procmail recipes I call spamassassin and clamscan. Will they be a problem? Then Dovecot. What changes do i need to make within Dovecot? What about the clients? Do I need to configure squirrelmail, Evolution or iPad/iPhone email clients? I archive old mail using archivemail. I think this can cope with maildir format, but it archives to a mbox - which I guess is not a problem except when I want to retrieve something from the archive - but I think I can use Mutt for that? Is there anything else I should consider? I have the day off tomorrow. I might have a go - what do you think? Thanks Mark
Re: [Dovecot] Corrupted Mail?
On Tue, 2014-05-27 at 16:09 +0200, Robert Schetterer wrote:
> Am 27.05.2014 16:04, schrieb Arthur Dent:
> > On Tue, 2014-05-27 at 15:44 +0200, Robert Schetterer wrote:
> >> Am 27.05.2014 14:56, schrieb Arthur Dent:
> >>> Hello All,
> >>>
> >>> I have a Dovecot v2.2.12 installation running on a Fedora 20 box which
> >>> is the mailserver for my family. This means that it is serving only 4
> >>> accounts.
> >>>
> >>> I use fetchmail->procmail to feed Dovecot and I read my mail on clients
> >>> such as Evolution on (another) Fedora 20 machine, iPad/iPhone or
> >>> squirrelmail on the web (running on the same Fedora 20 server).
> >>>
> >>> The average mail throughput for the whole family is around a couple of
> >>> hundred messages per day, but each (and every) day I get between 3 and
> >>> 10 of the following messages:
> >>>
> >>> dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox
> >>> file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s)
> >>>
> >>> I have googled, but with little success. What I have found seemed to
> >>> suggest that removing the .index files and allowing Dovecot to rebuild
> >>> them will solve this. I did:
> >>> find /home -type f -name "dovecot.index.*" -exec rm -f {} \;
> >>> but it has made no difference.
> >>>
> >>> Can anyone help me solve this please?
> >>
> >> at this small setup simply use maildir as format
> >> and use getmail to fetch, sieve may do sorting
> >>
> >> see
> >> https://sys4.de/de/blog/2013/04/12/abholdienst-fur-mail/
> >>
> >> sorry german but setup should speak for its own
> >>
> >> getmail does also deliver into maildir directly ( without dovecot deliver )
> >> if you dont need to sort through sieve
> >
> > Well thanks - but I have been using an mbox setup for about 10 years. It
> > is only since I upgraded to F20 that I have been experiencing this
> > problem. I may look at switching to maildir for the future, but for now
> > that's too big a job...
> >
> > Any other ideas?
> >
> > Thanks
> >
> > Mark
> >
>
> http://wiki2.dovecot.org/MboxProblems
OK - Unless I'm missing something, all that page (which I'd already
read) tells me is that I may have to delete the dovecot.index file(s) -
which I've don (several times). I still have the same problem. Every
single day.
Anything else?
Re: [Dovecot] Corrupted Mail?
On Tue, 2014-05-27 at 15:44 +0200, Robert Schetterer wrote:
> Am 27.05.2014 14:56, schrieb Arthur Dent:
> > Hello All,
> >
> > I have a Dovecot v2.2.12 installation running on a Fedora 20 box which
> > is the mailserver for my family. This means that it is serving only 4
> > accounts.
> >
> > I use fetchmail->procmail to feed Dovecot and I read my mail on clients
> > such as Evolution on (another) Fedora 20 machine, iPad/iPhone or
> > squirrelmail on the web (running on the same Fedora 20 server).
> >
> > The average mail throughput for the whole family is around a couple of
> > hundred messages per day, but each (and every) day I get between 3 and
> > 10 of the following messages:
> >
> > dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox
> > file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s)
> >
> > I have googled, but with little success. What I have found seemed to
> > suggest that removing the .index files and allowing Dovecot to rebuild
> > them will solve this. I did:
> > find /home -type f -name "dovecot.index.*" -exec rm -f {} \;
> > but it has made no difference.
> >
> > Can anyone help me solve this please?
>
> at this small setup simply use maildir as format
> and use getmail to fetch, sieve may do sorting
>
> see
> https://sys4.de/de/blog/2013/04/12/abholdienst-fur-mail/
>
> sorry german but setup should speak for its own
>
> getmail does also deliver into maildir directly ( without dovecot deliver )
> if you dont need to sort through sieve
Well thanks - but I have been using an mbox setup for about 10 years. It
is only since I upgraded to F20 that I have been experiencing this
problem. I may look at switching to maildir for the future, but for now
that's too big a job...
Any other ideas?
Thanks
Mark
[Dovecot] Corrupted Mail?
Hello All,
I have a Dovecot v2.2.12 installation running on a Fedora 20 box which
is the mailserver for my family. This means that it is serving only 4
accounts.
I use fetchmail->procmail to feed Dovecot and I read my mail on clients
such as Evolution on (another) Fedora 20 machine, iPad/iPhone or
squirrelmail on the web (running on the same Fedora 20 server).
The average mail throughput for the whole family is around a couple of
hundred messages per day, but each (and every) day I get between 3 and
10 of the following messages:
dovecot: imap(mark): Error: Next message unexpectedly corrupted in mbox
file /home/mark/mail/MLists/Fail2Ban at 3407473: 1 Time(s)
I have googled, but with little success. What I have found seemed to
suggest that removing the .index files and allowing Dovecot to rebuild
them will solve this. I did:
find /home -type f -name "dovecot.index.*" -exec rm -f {} \;
but it has made no difference.
Can anyone help me solve this please?
Thanks in advance
Mark
Re: [Dovecot] fchown() failed
On Mon, 2009-08-24 at 15:05 -0400, Timo Sirainen wrote: > On Wed, 2009-08-19 at 13:18 +0100, Arthur Dent wrote: > > dovecot: Dovecot v1.2.0 starting up (core dumps disabled): 1 Time(s) > > dovecot: IMAP(mark): fchown() failed with file > > /home/mark/Mail/.imap/INBOX/dovecot.index.log.newlock: Operation not > > permitted: 1 Time(s) > > v1.2.3+ would have given a bit better error message here. Ironically, since I posted my original message last week Fedora issued an updated Dovecot package, so here is yesterday's message in syslog: **Unmatched Entries** dovecot: Dovecot v1.2.3 starting up (core dumps disabled): 1 Time(s) dovecot: IMAP(helena): fchown(/home/wife/mail/.imap/INBOX/dovecot.index.cache.lock, -1, 12(mail)) failed: Operation not permitted (egid=501(helena), group based on /var/mail/helena): 1 Time(s) dovecot: IMAP(mark): chown(/home/mark/Mail/.imap/INBOX, -1, 12(mail)) failed: Operation not permitted (egid=500(mark), group based on /var/mail/mark): 1 Time(s) dovecot: IMAP(mark): fchown(/home/mark/Mail/.imap/INBOX/dovecot.index.cache.lock, -1, 12(mail)) failed: Operation not permitted (egid=500(mark), group based on /var/mail/mark): 1 Time(s) dovecot: IMAP(mark): fchown(/home/mark/Mail/.imap/INBOX/dovecot.index.log.newlock, -1, 12(mail)) failed: Operation not permitted (egid=500(mark), group based on /var/mail/mark): 1 Time(s) dovecot: IMAP(mark): fchown(/home/mark/Mail/.imap/INBOX/dovecot.index.tmp, -1, 12(mail)) failed: Operation not permitted (egid=500(mark), group based on /var/mail/mark): 1 Time(s) dovecot: IMAP(mark): mkdir(/home/mark/Mail/.imap/INBOX) failed: Operation not permitted: 1 Time(s) > Anyway the issue is that nowadays Dovecot tries to preserve mailbox's > permissions > so that shared mailboxes work properly. What permissions do you have in > the actual INBOX file? ls -la /var/mail/ total 202024 drwxrwxr-x. 2 root mail 4096 2009-08-26 10:30 . drwxr-xr-x. 15 root root 4096 2009-08-13 12:35 .. -rw-rw. 1 clamav mail 0 2009-08-12 12:40 clamav -rw-rw. 1 wife mail 190669010 2009-08-26 10:00 wife -rw-rw. 1 mark mail 8421931 2009-08-26 09:46 mark -rw---. 1 root root787636 2009-08-26 10:30 root -rw-rw. 1 rpcmail 0 2009-08-11 16:26 rpc -rw-rw. 1 sonmail 6750285 2009-08-26 00:39 son > > My guess is that you have 0660 permissions of /var/mail/$USER so Dovecot > tries to preserve the group. Easiest fix that also makes your system > more secure is to chmod 0600 it. Hmmm - OK, seems you're right about that. Will changing it break anything else? Why does group "mail" exist? I will try chmod 0600 and see what happens. > > > As I access my mailbox several times a day (and certainly did yesterday) > > from several different mail clients I am surprised that the error count > > is only 1 for most the above entries. Is this being caused by some sort > > of cron job (logrotate perhaps)? > > It happens only when index files are being created/rotated, which > happens automatically "every once in a while". Thanks very much. Your help is greatly appreciated... Mark signature.asc Description: This is a digitally signed message part
[Dovecot] fchown() failed
Hello all, I'm not sure if this question is better posed to a Fedora group because I have only had this problem since I upgraded from F9 to F11, but I figured that the specialised knowledge here might help me more... I have been getting these entries in syslog since upgrading. Today for instance I got: **Unmatched Entries** dovecot: Dovecot v1.2.0 starting up (core dumps disabled): 1 Time(s) dovecot: IMAP(mark): fchown() failed with file /home/mark/Mail/.imap/INBOX/dovecot.index.log.newlock: Operation not permitted: 1 Time(s) dovecot: IMAP(mark): fchown() failed with file /home/mark/Mail/.imap/INBOX/dovecot.index.tmp: Operation not permitted: 1 Time(s) and yesterday I had: **Unmatched Entries** dovecot: IMAP(helena): fchown() failed with file /home/wife/mail/.imap/INBOX/dovecot.index.tmp: Operation not permitted: 1 Time(s) dovecot: IMAP(toby): fchown() failed with file /home/son/mail/.imap/INBOX/dovecot.index.cache.lock: Operation not permitted: 1 Time(s) dovecot: IMAP(toby): fchown() failed with file /home/son/mail/.imap/INBOX/dovecot.index.log.newlock: Operation not permitted: 1 Time(s) dovecot: IMAP(toby): fchown() failed with file /home/son/mail/.imap/INBOX/dovecot.index.tmp: Operation not permitted: 3 Time(s) The permissions are as follows: ls -la /home/mark/Mail/.imap/INBOX/ total 328 drwx--. 2 mark mark 4096 2009-08-18 15:32 . drwx--. 84 mark mark 4096 2009-03-05 21:53 .. -rw---. 1 mark mark 7472 2009-08-18 15:32 dovecot.index -rw---. 1 mark mark 286720 2009-08-19 11:46 dovecot.index.cache -rw---. 1 mark mark 5380 2009-08-19 12:43 dovecot.index.log Those of wife and son are similar. As I access my mailbox several times a day (and certainly did yesterday) from several different mail clients I am surprised that the error count is only 1 for most the above entries. Is this being caused by some sort of cron job (logrotate perhaps)? Any ideas? All help / suggestions gratefully received... Thanks Mark signature.asc Description: This is a digitally signed message part
