Re: dovecot-uidlist is not up-to-date [EXTERNAL]
Yes. That makes imap client to re-download whole mail folder. Not an acceptable solution to me :( Fil On March 22, 2018 5:05:26 AM EDT, Aki Tuomi wrote: >Did you try removing the file? > >Aki > > >On 22.03.2018 06:32, Dmitry Filonov wrote: >> Tried that. It rebuilds index based on dovecot-uidlist. But not the >> dovecot-uidlist based on actual mail data. :( >> >> Fil >> >> >> >> On March 21, 2018 11:58:21 PM EDT, "@lbutlr" >wrote: >> >> On 2018-03-21 (17:15 MDT), Dmitry Filonov > wrote: >> >> Now the question is if there's any way to tell dovecot to >> rebuild dovecot-uidlist files using actual Maildir data. I >> don't want to remove dovecot-uidlist files as this triggers >> the whole mailbox being re-downloaded by the imap client. >With >> some accounts having over 100Gb of mail this is too much of a >> hassle. I just need dovecot itself to fix it's data. >> >> >> >> doveadm index -A '*' >> >> if that doesn't work then perhaps >> >> doveadm force-resync -A '*' >> >> of course you can use -u for a specific user instead of -A >for all users. >> >> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: dovecot-uidlist is not up-to-date
Tried that. It rebuilds index based on dovecot-uidlist. But not the dovecot-uidlist based on actual mail data. :( Fil On March 21, 2018 11:58:21 PM EDT, "@lbutlr" wrote: >On 2018-03-21 (17:15 MDT), Dmitry Filonov > wrote: >> Now the question is if there's any way to tell dovecot to rebuild >dovecot-uidlist files using actual Maildir data. I don't want to remove >dovecot-uidlist files as this triggers the whole mailbox being >re-downloaded by the imap client. With some accounts having over 100Gb >of mail this is too much of a hassle. I just need dovecot itself to fix >it's data. > > > doveadm index -A '*' > >if that doesn't work then perhaps > > doveadm force-resync -A '*' > >of course you can use -u for a specific user instead of -A for >all users. > >-- >On 20-12-2013 Wes Groleau wrote: >>On 12-20-2013, 10:04, George Kerby wrote: >>> 'Tis the Season to bitch and moan. > >> That doesn't fit the tune you @#$%#$&%$*#$@%T#^% ! -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
dovecot-uidlist is not up-to-date
Hi, Am using dovecot-2.2.10-8 on Centos-7 and everything seemed to be totally fine for many years. Now am trying to migrate our on-premises mail server to the cloud and I had to get some statistics. I was doing that using the doveadm utility. Unfortunately I found that the numbers don't match if I check the real messages in my Maildirs. For some account doveadm was reporting number of messages but that account didn't have any messages stored at all. So I looked a bit further and found that doveadm utility reports data form indexes and indexes are built using dovecot-uidlist data. For quite some folders these files have way more data than actual mailbox has. For example, my personal Inbox has 569 messages but dovecot-uidlist has 716 entries. I don't know when and how that happened but this happens a lot on many different mailboxes on my server. I never touched dovecot-uidlist files outside of dovecot. I might have removed messages from Maildirs though, but not at this scale. Now the question is if there's any way to tell dovecot to rebuild dovecot-uidlist files using actual Maildir data. I don't want to remove dovecot-uidlist files as this triggers the whole mailbox being re-downloaded by the imap client. With some accounts having over 100Gb of mail this is too much of a hassle. I just need dovecot itself to fix it's data. Thanks, Fil -- Dmitry Filonov Network Analyst 300 Longwood Ave. Enders-1262.2 Boston, MA 02115 617-919-4702
Re: [Dovecot] Need help configuering Dovecot to support folders that contain sub-folders and messages
Scott Silva wrote: on 6-13-2008 10:24 AM Dan Roberts spake the following: The intent on my part was not to be trapped, but to simplify life. I don't work at a command line or system level on a regular enough basis these days to be 100% confident that I won't screw it up, so relying on a system that is pseudo capable of keeping itself together was a saccrifice I was willing to make. I have had the bad misfortune of running RMPs in the past only to have things go very wrong - and they should not have, but for a screw up in the RPM to begin with. Back then, I was more than capable of fixing things - but that was a long time ago. So I want to proceed cautiously and not blow a hole in the working system. That said, if upgraded is it possible to resolve this issue, or should I truly be contemplating a switch to another IMAP system? Dovecot is very popular so I find it odd that Apple would not foresee this issue. So it would seem that a solution exists, but I don't have that configuration in play - at least I hope so. Cheers - Dan Is your IMAP server using mbox or maildir for the mailstore? Maildir should be able to handle the folders. He has mailbox. And that is the problem. Dovecot doesn't support both folders and messages in mailbox folders. Well, that makes sense given that mailbox folders are FILES on the filesystem and you can't have sub-files. So the only way to go - migrate to Maildir. FiL
Re: [Dovecot] MBOX to Maildir preserving the UIDL (outlook-tested, no redownloading)
Horn Wijaya wrote: TESTED AND RUNNING, MIGRATING from MBOX to MAILDIR, PRESERVE UID and OUTLOOK 1. mb2md-3.20.pl -s /var/mail/hwijaya -d home/hwijaya/Maildir/ that's converting 444 emails from mbox to 444 files in the Maildir/cur folder 2. cd /home/hwijaya/Maildir/cur 3. grep IMAPbase * and I get 1210219860.00.mbox:2,S:X-IMAPbase: 1202794557 19944 The last 2 strings are the KEY to the ORIGINAL UID 4. So in order to prepare the environment to fool it, some calculations are needed: How do we get 19501 as the first counter? as expected in the first POP TEST above: 19944 - 444 emails + 1 = 19501 5. cd /home/hwijaya/Maildir 6. echo '3 V1202794557 N19501' > dovecot-uidlist Hello, First of all, I was looking into this problem of getting the right converter for couple years. You can search list archives for that - there are more problems with perfect migration, than you know of. Second, your calculations assume two things, that are not always true - 1) original mailbox already has X-IMAPbase header (not every mail server puts these headers) 2) first message UID is last message UID-, which is not true, if you have deleted some messages in between. Third, usually servers use different UIDL format, not time+number. Last, if you are migrating from mbox to Maildir on IMAP server, then you have more than one folder. Probably you have "/" as folder separator on mbox-based system. And will have "." on Maildir. That means, Outlook will need to refresh it's folder list anyway. And that involves re-downloading all messages. Yes, in your case you might be able to make a perfect migration tool, but this won't be ideal for everybody. P.S. After couple years of thinking, planning and looking for a ideal solution I have modified mb2md-3.20.pl script a little and was able to migrate about 200Gb of mail with no problem. Yes, most clients had to re-download all their messages (I had to force POP3 clients to empty their server mailboxes), but it wasn't that big of a deal, as I was afraid of. And now, even with much slower hardware I see about 10 times lower LA than with mbox. FiL
Re: [Dovecot] Convert plugin question
Timo Sirainen wrote: On Mon, 2008-02-04 at 15:00 -0500, Dmitry Filonov wrote: Is it possible to change convert plugin, so it would keep old UIDVALIDITY and UIDNEXT after conversion from mbox to Maildir? It would also have to preserve all messages' UIDs. This is something that has been in my TODO for a while, but it would require pretty large changes so it hasn't been done yet. Maildir -> dbox migration can preserve UIDs though. :) Why this require large changes? Isn't it only about creating dovecot-uidlist on migration? Anyway, it's your time and I can't ask you to work on my problem first. What about mbox->dbox migration? Does it preserve UIDs? Thanks, FiL
[Dovecot] Convert plugin question
Hi, Is it possible to change convert plugin, so it would keep old UIDVALIDITY and UIDNEXT after conversion from mbox to Maildir? As far as I can tell, this shouldn't be a big problem as this information already exists in the mbox. Well, at least for mboxes, that have that information. The problem is easy, if UIDVALIDITY changes during conversion, then client has to reload whole mailbox from the server. And this is not easy for users with 5+ Gb of mail. Thanks, FiL
