Re: [Dovecot] How to exec /usr/libexec/dovecot/imap with specific maildir location
On Tue, 2013-04-09 at 18:44 +0100, Ben Morrow wrote: > /usr/libexec/dovecot/imap -o mail_location=maildir:~/Maildir-archive Perfect. Thank you. -- dwmw2 smime.p7s Description: S/MIME cryptographic signature
[Dovecot] How to exec /usr/libexec/dovecot/imap with specific maildir location
Once upon a time I could configure my mailer to access my historical archives by running something like ssh $mailserver MAIL=maildir:~/Maildir-archive dovecot --exec-mail imap I need to run /usr/libexec/dovecot/imap manually now, and I've fixed that a while ago for my normal email access. But I've just had cause to go back to my archives for the first time in a while... and realised that I don't know how to make it work any more. When invoking /usr/libexec/dovecot/imap, is there an environment variable or command line option which can set the mail location? -- dwmw2 smime.p7s Description: S/MIME cryptographic signature
Re: [Dovecot] SSD drives are really fast running Dovecot
On Sat, 2011-01-15 at 10:41 +, Ed W wrote: > > > One of the systems to fail was a firewall running off SSD. > > SSD or CF? That doesn't make a lot of difference. They're all broadly similar. There are better devices and worse devices, but they're mostly crap. And as I said earlier, even if you think you've worked out which is which, it may change from batch to batch of what is allegedly the *same* product. > It would appear it's also possible to damage some flash memory by > powering off at the wrong moment? Almost all of them will fail hard it you do any serious power-fail testing on them. It's not a hardware failure; it's just that their *internal* file system is corrupt and needs a fsck (or just wiping and starting again). But since you can't *access* the underlying medium, all you can do is cry and buy a new one. The fun thing is that their internal garbage collection could be triggered by a *read* from the host computer, or could even happen purely due to a timeout of some kind. So there is *never* a time when you know it's "safe to power off because I haven't written to it for 5 minutes". Yes, it's perfectly possible to design journalling file systems that *are* resilient to power failure. But the "file systems" inside these devices are generally written by the same crack-smoking hobos that write PC BIOSes; you don't expect quality software here. By putting a logic analyser on some of these devices to watch what they're *actually* doing on the flash when they garbage-collect, we've found some really nasty algorithms. When garbage-collecting, one of them would read from the 'victim' eraseblock into RAM, then erase the victim block while the data were still only held in RAM — so that a power failure at that moment would lose it. And then, just to make sure its race window was nice and wide, it would then pick a *second* victim block and copy data from there into the freshly-erased block, before erasing that second block and *finally* writing the data from RAM back to it. It's just scary :) -- dwmw2
Re: [Dovecot] SSD drives are really fast running Dovecot
On Thu, 2011-01-13 at 20:19 +0100, Steve wrote: > > I would not use MLC in a server environment. SLC has much better > program/erase cycles per cell. I wouldn't be overly worried about the underlying medium. I'm more worried about the translation layer they use on top of it, to make it pretend to be spinning rust. It is essentially a file system, on top of which you are expected to layer another file system. Not particularly efficient, but at least TRIM addresses one of the biggest inefficiencies of that gratuitous extra layering. The inefficiency is one thing, but it's the reliability that worries me. It's generally accepted that it takes at least 5 years for a file system implementation to truly reach maturity. And that's for open source code that you can debug, on a medium that you can access directly to do diagnosis and data recovery. But what we're talking about here is a file system implemented inside a black box where you can't do any of that. And what's more, they keep changing it. Even if you manage to find some device that passes your testing, you may find that the next batch of the *same* device (from your point of view) actually contains completely different software *and* hardware if you take it apart. These translation layers are almost always a complete pile of crap. Especially in the face of power failures, since they so often completely fail to implement basic data integrity features (the same kind of journalling features that also have to be implemented in the 'real' file system on top of this fake disk). The best way to use flash is to have a file system that's *designed* for use on flash. The only problem with that is that it wouldn't work with DOS; you can't provide an INT 13h DISK BIOS handler to use it... -- dwmw2
Re: [Dovecot] SSD drives are really fast running Dovecot
On Wed, 2011-01-12 at 09:53 -0800, Marc Perkel wrote: > I just replaced my drives for Dovecot using Maildir format with a pair > of Solid State Drives (SSD) in a raid 0 configuration. It's really > really fast. Kind of expensive but it's like getting 20x the speed for > 20x the price. I think the big gain is in the 0 seek time. You may find ramfs is even faster :) I hope you have backups. -- dwmw2
Re: [Dovecot] Best way to move mail to an archive
On Sat, 2010-12-18 at 12:21 -0500, Richard C. Cox wrote: > I'd like to archive e-mail as it ages into a > single IMAP archive directory. Ideally, I would like to set up a cron job to > search for e-mail in all of my imap directories that older that 'X' days and > move it to the archive directory. I know that dovecot wants to manage e-mail > movement thru the IMAP interface, so I'm leery of just using ordinary Linux > commands to move mail around and bypass the IMAP protocol. Does anybody have > any recommendations about dovecot/IMAP safe commands I could use to > accomplish > this task? Personally, I just do it over IMAP. http://david.woodhou.se/archivemail.sh -- dwmw2
Re: [Dovecot] --exec-mail
On Fri, 2010-11-12 at 01:52 +, Timo Sirainen wrote:
> > Once upon a time in the dim and distant past, I believe I used to
> > run /usr/libexec/dovecot/imap directly. But that doesn't set the
> > environment up correctly, so isn't the right way to do it... is it?
>
>
> Nowadays it does.
Ah, right.
If I create ~/Maildir/{cur,new,tmp} manually, then running
/usr/libexec/dovecot/imap does actually work.
(I did have a ~/Maildir on this test machine, but it only included one
mailing list folder which I'd been using as a test case for a QRESYNC
bug a few months ago.)
Thanks.
--
dwmw2
[Dovecot] --exec-mail
Most of my IMAP access is achieved by 'ssh $mailhost exec dovecot --exec-mail imap'. I'm testing Dovecot 2.0, and this doesn't seem to work any more — and I can't seem to work out how to make it work. The wiki page at http://wiki2.dovecot.org/Upgrading/2.0 has no mention of it. I looked through the git history to find the commit that broke it and see if I could find the replacement that way, but it was the commit entitled 'Initial commit for v2.0 master rewrite.', and wasn't particularly edifying. Once upon a time in the dim and distant past, I believe I used to run /usr/libexec/dovecot/imap directly. But that doesn't set the environment up correctly, so isn't the right way to do it... is it? -- dwmw2
Re: [Dovecot] \" character in folder name results in strange LIST
On Fri, 2010-09-03 at 23:30 +0200, Samuel Kvasnica wrote: > > Well, David, if evolution was not such an instable beast... we did few > evaluations from openSuSE10.3 up to 11.2 recently, > and it was a no-go, except for basic imap usage. As soon as more > connectors/plugins are involved, it is a nightmare. > Plus, we unfortunately want it also on Win32 Platform. Yeah, I've often hated Evolution, but only stayed with it out of inertia. I'm a lot happier with the 2.30.3 release, and will be even happier with 2.32 when it comes out. There have been a lot of improvements. Did you file bugs? The Evolution team seem to be a lot better these days than they used to be, and bugs do actually get looked at. -- dwmw2
Re: [Dovecot] \" character in folder name results in strange LIST
On Fri, 2010-09-03 at 22:48 +0200, Samuel Kvasnica wrote: > most cell phones simply haven't got any ssh > terminal but have own IMAP clients. Any J2ME-capable phone has an SSH client: http://www.xk72.com/midpssh/ My experience of the native IMAP clients in phones has been extremely poor. My N97 at the moment consistently fails to contact the home IMAP server (although it used to work), and gives me *no* coherent error message when it fails. I can't remember the last time I saw a phone mail client which actually bothered to include In-Reply-To: or References: headers in replies, either. -- dwmw2
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Thu, 2010-08-26 at 17:02 +0100, Timo Sirainen wrote: > On Thu, 2010-08-26 at 14:32 +0100, David Woodhouse wrote: > > > > Or if you want flags for messages you haven't even seen yet, > > > 1:4294967295 should work too. > > > > 1:4294967295 doesn't really fill me with joy either -- that assumes that > > a UID is limited to 32 bits unsigned... > > It is, as defined by RFC 3501. > > > and that other servers won't > > fall over when presented with that number. > > Well, those servers would also be broken then.. I think servers can > typically handle such UID better than clients (many clients use signed > 32bit ints). I think there's a good chance it would work fine with all > servers that support QRESYNC. OK, I'll do that then. Thanks. -- dwmw2
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Mon, 2010-08-23 at 18:04 +0100, Timo Sirainen wrote: > On Mon, 2010-08-23 at 17:33 +0100, David Woodhouse wrote: > > > And thanks for not (yet) making it reject the invalid command with the > > 1:* in it > > I changed it in v2.0. > > > -- I'll need to come up with a strategy for migrating to the > > 'correct' command on the client side, given that older versions of > > dovecot won't accept it. > > > > I'll probably make the Evolution client code start off by trying the > > correct command, and then retry with the bogus '1:*' if that fails. > > Can't you simply send 1:? I *might* be able to get away with that, and fetch flags for any newer messages at the same time as I fetch the headers for those messages. I'll check. > Or if you want flags for messages you haven't even seen yet, > 1:4294967295 should work too. 1:4294967295 doesn't really fill me with joy either -- that assumes that a UID is limited to 32 bits unsigned... and that other servers won't fall over when presented with that number. -- dwmw2
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Mon, 2010-08-23 at 15:40 +0100, Timo Sirainen wrote: > Whops. Looks like I read the RFC a bit wrong. Fixed in v2.0 and v1.2 > hg now. I guess I should release 1.2.14 then. Could you try that the > fix works properly? At least it doesn't give any errors anymore. > http://hg.dovecot.org/dovecot-1.2/rev/7e959d397a35 Looks like it's working; thanks. Tested against 1.2.13. And thanks for not (yet) making it reject the invalid command with the 1:* in it -- I'll need to come up with a strategy for migrating to the 'correct' command on the client side, given that older versions of dovecot won't accept it. I'll probably make the Evolution client code start off by trying the correct command, and then retry with the bogus '1:*' if that fails. I don't really want to just 'fix' the client and let people get errors with older versions of Dovecot, even though QRESYNC support is optional. -- David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Thu, 2010-08-19 at 18:37 +0100, Timo Sirainen wrote:
> On Wed, 2010-08-18 at 22:27 +0100, David Woodhouse wrote:
> > Aug 18 22:07:31 twosheds IMAP(dwmw2): : Panic: file
> > mail-index-transaction.c:
> > line 637 (mail_index_transaction_lookup): assertion failed:
> > (seq >= t->first_new_seq && seq <= t->last_new_seq)
> >
> > A00131 SELECT lists.bluez (QRESYNC (1154090296 1861 1:*
> > (1,120,1578,2064,2226,2280,2298 1,120,12037,12523,12685,12739,12757)))
>
> Thanks, fixed: http://hg.dovecot.org/dovecot-1.2/rev/70fa6178380e
Hm, looking at RFC5162 again I realise that SELECT command isn't
actually valid. The '1:*' for the known-uids is not permitted. From the
formal syntax in §6:
capability =/ "QRESYNC"
select-param= "QRESYNC" SP "(" uidvalidity SP
mod-sequence-value [SP known-uids]
[SP seq-match-data] ")"
;; conforms to the generic select-param
;; syntax defined in [IMAPABNF]
seq-match-data = "(" known-sequence-set SP known-uid-set ")"
uidvalidity = nz-number
known-uids = sequence-set
;; sequence of UIDs, "*" is not allowed
known-sequence-set = sequence-set
;; set of message numbers corresponding to
;; the UIDs in known-uid-set, in ascending order.
;; * is not allowed.
known-uid-set = sequence-set
;; set of UIDs corresponding to the messages in
;; known-sequence-set, in ascending order.
;; * is not allowed.
§3.1 says:
If the list of known UIDs was also provided, the server should only
report flag changes and expunges for the specified messages. If the
client did not provide the list of UIDs, the server acts as if the
client has specified "1:", where is the mailbox's
UIDNEXT value minus 1.
So instead of giving the known-uid set '1:*', the client should actually
have omitted the optional known-uid parameter completely. It *should*
have sent this command:
A00131 SELECT lists.bluez (QRESYNC (1154090296 1861
(1,120,1578,2064,2226,2280,2298 1,120,12037,12523,12685,12739,12757)))
Dovecot doesn't like that though:
A00131 BAD Error in IMAP command SELECT: Invalid QRESYNC parameters
--
David WoodhouseOpen Source Technology Centre
david.woodho...@intel.com Intel Corporation
Re: [Dovecot] 1.2.13 QRESYNC crash.
On Thu, 2010-08-19 at 18:37 +0100, Timo Sirainen wrote: > Thanks, fixed: http://hg.dovecot.org/dovecot-1.2/rev/70fa6178380e Thanks. Obviously you've been able to test on exactly the same mailbox I'm using, so you'll not be surprised to hear that the patch works fine here too. Do shout if you see anything that could be improved about the client's behaviour, by the way -- this is Evolution's 'imapx' back end. It's including sequence numbers working backwards exponentially from the end of the folder -- so for a folder which had N mails last time we looked at it, we'll include sequences N-9, N-27, N-81, N-243, N-729, etc. in the QRESYNC request. Does that seem reasonable? -- David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation
Re: [Dovecot] \" character in folder name results in strange LIST
On Sat, 2010-08-14 at 23:36 +0200, Samuel Kvasnica wrote: > Well, it looks like at least Thunderbird v3.1.2 is buggy then. Are there > any non-buggy clients anyway ...? Evolution (>=2.30.2) with the imapx back end gets it right. I know this because I fixed it myself a few weeks ago. -- David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation
Re: [Dovecot] \" character in folder name results in strange LIST
On Wed, 2010-08-18 at 12:48 +0200, Samuel Kvasnica wrote: > This is a very nasty mess. If there are some ancient clients like pine > or whatever we used on vt100 in the 90th that need broken LIST command, > there should be a workaround setting for that. Hey! I still use pine. With mouse-in-xterm mode enabled it works really nicely on devices with a touch screen, and it makes excellent use of small screens and limited network bandwidth. -- dwmw2
[Dovecot] 1.2.13 QRESYNC crash.
Aug 18 22:07:31 twosheds IMAP(dwmw2): : Panic: file mail-index-transaction.c: line 637 (mail_index_transaction_lookup): assertion failed: (seq >= t->first_new_seq && seq <= t->last_new_seq) * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as dwmw2 0 enable condstore qresync * ENABLED CONDSTORE QRESYNC 0 OK Enabled. A00131 SELECT lists.bluez (QRESYNC (1154090296 1861 1:* (1,120,1578,2064,2226,2280,2298 1,120,12037,12523,12685,12739,12757))) Program received signal SIGABRT, Aborted. 0x0034ace326c5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install cyrus-sasl-lib-2.1.23-9.fc12.x86_64 glibc-2.11.2-1.x86_64 keyutils-libs-1.2-6.fc12.x86_64 krb5-libs-1.7.1-9.fc12.x86_64 libcom_err-1.41.9-8.fc12.x86_64 libgcc-4.4.4-10.fc12.x86_64 libselinux-2.0.90-5.fc12.x86_64 nss-softokn-freebl-3.12.6-2.fc12.1.x86_64 openldap-2.4.19-4.fc12.x86_64 openssl-1.0.0a-1.fc12.x86_64 zlib-1.2.3-23.fc12.x86_64 (gdb) bt #0 0x0034ace326c5 in raise () from /lib64/libc.so.6 #1 0x0034ace33ea5 in abort () from /lib64/libc.so.6 #2 0x004a2698 in default_fatal_finish (type=, status=0) at failures.c:160 #3 0x004a273b in i_syslog_fatal_handler (type=LOG_TYPE_PANIC, status=0, fmt=, args=) at failures.c:315 #4 0x004a1d86 in i_panic (format=0x6 ) at failures.c:207 #5 0x004787f0 in mail_index_transaction_lookup ( t=, seq=) at mail-index-transaction.c:637 #6 0x0047be45 in tview_lookup_uid (view=, seq=, uid_r=) at mail-index-transaction-view.c:162 #7 0x004580fb in index_mail_set_seq (_mail=0x7282b8, seq=1578) at index-mail.c:1229 #8 0x00423eb4 in expunges_drop_known (ctx=0x70a628) at imap-fetch.c:178 #9 get_expunges_fallback (ctx=0x70a628) at imap-fetch.c:269 #10 imap_fetch_send_vanished (ctx=0x70a628) at imap-fetch.c:291 #11 imap_fetch_begin (ctx=0x70a628) at imap-fetch.c:314 #12 0x0041e8e2 in select_qresync (cmd=, ---Type to continue, or q to quit--- readonly=) at cmd-select.c:243 #13 select_open (cmd=, readonly=) at cmd-select.c:320 #14 cmd_select_full (cmd=, readonly=) at cmd-select.c:382 #15 0x00420ead in client_command_input (cmd=0x70a458) at client.c:612 #16 0x00420f8d in client_command_input (cmd=0x70a458) at client.c:661 #17 0x004211b5 in client_handle_next_command (client=0x709ff0) at client.c:702 #18 client_handle_input (client=0x709ff0) at client.c:714 #19 0x00421a5c in client_input (client=0x709ff0) at client.c:753 #20 0x004aa7cd in io_loop_handler_run (ioloop=) at ioloop-epoll.c:208 #21 0x004a9c38 in io_loop_run (ioloop=0x707cc0) at ioloop.c:335 #22 0x00429726 in main (argc=, argv=, envp=) at main.c:327 It's a public mailing list folder, so I've uploaded it to http://david.woodhou.se/bluez-maildir.tar.gz Filed as Red Hat bug #625907: https://bugzilla.redhat.com/show_bug.cgi?id=625207 -- dwmw2
Re: [Dovecot] UID FETCH returns results outside requested range
On Mon, 2010-07-12 at 13:58 +0100, Timo Sirainen wrote: > On Mon, 2010-07-12 at 13:51 +0100, David Woodhouse wrote: > > 01 UID FETCH 152280:* (UID) > > * always matches the last message. Ah, thanks. I hadn't realised that the 'start' and 'end' of a range are actually interchangeable. RFC3501 does make that clear, now I'm looking for it. -- dwmw2
[Dovecot] UID FETCH returns results outside requested range
00 SELECT INBOX * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Junk $Label1 $Label2 $Label3 $Label4 $Label5 receipt-handled NotJunk NonJunk $MDNSent $has_cal unknown-6 OldNotJunk $Forwarded) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Junk $Label1 $Label2 $Label3 $Label4 $Label5 receipt-handled NotJunk NonJunk $MDNSent $has_cal unknown-6 OldNotJunk $Forwarded \*)] Flags permitted. * 7126 EXISTS * 0 RECENT * OK [UNSEEN 16] First unseen. * OK [UIDVALIDITY 1154089803] UIDs valid * OK [UIDNEXT 152280] Predicted next UID * OK [HIGHESTMODSEQ 22137] Highest 00 OK [READ-WRITE] Select completed. 01 UID FETCH 152280:* (UID) * 7126 FETCH (UID 152279) 01 OK Fetch completed. Dovecot 1.2.11 on Fedora 12 with Maildir mailboxen. Why is it returning the message with UID 152279, when asked for the range 152280:* ? Is there some reason it has to return at least one result? -- dwmw2
Re: [Dovecot] Dovecot git mirror
On Mon, 2010-07-05 at 11:55 +0100, Timo Sirainen wrote: > I had planned on leaving NOTIFY until I get mailbox list indexes > implemented > (http://www.dovecot.org/list/dovecot/2009-December/045480.html), > otherwise it's going to be annoyingly inefficient. Of course, feel free > to implement an inefficient way and I'll just make it faster later once > the indexes are done. :) Yeah, I'd seen that. For testing the client, though, I don't mind if it's not hugely efficient. I wasn't going to bother with the indices. I was thinking of just setting up inotify watches on each watched folder -- and whenever the inotify triggers, act as if a STATUS command had been sent for the corresponding folder. There are more details to cope with for a properly compliant implementation, but that covers the majority of what I wanted for client testing. -- dwmw2
[Dovecot] Dovecot git mirror
I've been working on Evolution's 'imapx' back end to support QRESYNC, and was going to look at NOTIFY next. Then I realised I may have to look at implementing it on the server side first. Before I can even contemplate that, though, I had to mirror it into git. I've always resisted the one-version-control-system-per-child craze, and these days if I'm even contemplating a project which is using some other version control system, I'll start by importing it into a git repo. Thus: git://git.infradead.org/dovecot/dovecot-1.0 git://git.infradead.org/dovecot/dovecot-1.1 git://git.infradead.org/dovecot/dovecot-1.2 git://git.infradead.org/dovecot/dovecot-2.0 These are browsable in gitweb as: http://git.infradead.org/dovecot/dovecot-1.0 http://git.infradead.org/dovecot/dovecot-1.1 http://git.infradead.org/dovecot/dovecot-1.2 http://git.infradead.org/dovecot/dovecot-2.0 The mirrors are updated hourly, at 15 minutes past the hour. When making such a mirror, I usually try to ensure that it's imported in such a way that that the project maintainer(s) _could_ switch to using it as their primary VCS if they wanted to -- and a number of projects have actually done so. But I'm not arguing for that; I'm just making the repository available in git form in case it's useful to anyone else who, like me, doesn't really want to install and learn to use all of arch/hg/bzr/monotone/svn/etc for various different projects. No VCS flame wars please. -- dwmw2
