[Dovecot] Copying sieve scripts
Hello, I am using squirrelmail with avelsieve to edit my filter rules. The filter rules are like this: 1. If the message contains a virus, delete the mail and notify me by email. 2. If the message is spam, deliver it to my spam folder. 3. If the message is from the dovecot mailing list, deliver it to my dovecot folder Yesterday, I created a new user and wanted to let him have rules 1 and 2 like above, only with another destination for email notification. So, I copied my sieve script into the sieve directory of the new user, deleted rule 3, changed the destination for email notification and then called sievec to compile the rule set. But this did not work like expected: Using squirrelmail, I saw that the new user had only rules 1 and 2, but the destination in rule 1 still contains my email address. My own account still has three rules. So, this simple way of copying sieve scripts does not work. Is there another way to copy sieve rules from one user to another? Regards Christoph
Re: [Dovecot] Best filesystem?
Nick Edwards wrote: xfs is not very nice to you if you lose power, it's not as bad as it used to be, but it still gives you 0 byte files, I began to worry about this after that other thread showed XFS's considerable strengths, and this one weakness. Co-incidentally,we had already just built a couple of XFS servers in raid1 configurations (the second is purely an rdiff-backup server for the first, and both are raid1). Because our work is frequently the subject of very close legal scrutiny, we're utterly paranoid about losing email - that's why we've created those two redundant servers. I remember Stan (in the other thread) saying, also, that write-delays due to caching were more or less built-in to the kernel anyway, so XFS may not be alone in this problem. What I am not (yet) sure about, is whether XFS is any 'more' vulnerable than others, or is any 'more' catastrophically damaged, than others, due to power fail. Has any analysis of this been published? However, like the OP, our scale is quite small and this (potentially) gives us one advantage over those very large users. We could forgo some 'performance' if there were options in XFS that could reduce its 'vulnerability'. I looked at the XFS FAQ, and several of the archived messages on the XFS list, but could not see any create options, or mount options, that would reduce or inhibit the 'vulnerability window' (but I'm no expert on filesystems, or the kernel, so maybe I didn't understand what the FAQ was telling me). Would appreciate any suggestions from those who use and know XFS. so make sure you have a good UPS to issue a safe shutdown of the server, We are very susceptible to power outages, duration anything from 12 seconds to 14 hours (we're not in a city) and never notified in advance. We use APC desktop UPS for workstations and the few servers we have, and we then shut down. For security, the shutdown needs to be automatic so that it takes effect if the site is unmanned - overnight, for example. 'Absolutely secure email' needs the speed of XFS, the performance of XFS on multitudes of small files, and the fault-tolerance of some kind of non-volatile storage coupled with positive confirmation of successful writes. One day, maybe. Until then, email needs UPSs, it seems. regards, Ron
Re: [Dovecot] Copying sieve scripts
Am 30.01.2011 um 10:03 schrieb Stephan Bosch: To my opinion, the RoundCube Sieve plugin handles this much better by parsing the actual Sieve code to retrieve the rules and using plaintext comments for meta data. However, I do agree that some standardized method of encoding meta data in Sieve scripts could solve much of this, also for interoperability. However, this won't be done any time soon. Isn't this already described in RFC5784 ( Sieve Email Filtering: Sieves and Display Directives in XML) ? I just read the RFC some time ago, and it looked quite usable with the XSLT conversion. Regards, Oliver
[Dovecot] inform users's about new spam
Hallo evryone! Mailsetup: postfix+dovecot2 with postgres database backend - about 2,000 virtual mailboxes, growing.. New possible spam will be sorted into a users directory named spam, emails older than 30 day's in that folder will be deleted by using expunge. I would like to inform my users evry two day's (or something like that) about new emails in the spam folder. Can you give me a hint how to handle that ? Are there helpful tools , or have i to wrote my own script, walking through the folders, checking for new emails since last check and finally inform the user if needed. Best wishes Christoph
Re: [Dovecot] dovecot: pop3-login: Disconnected (tried to use disabled plaintext auth): method=PLAIN
Yep - despite the above, I still get mail. And from my conf files, plain auth is disabled. Should I re-port my conf file in case I missed something? John On 1/28/2011 2:48 AM, Mark Sapiro wrote: So you successfully get mail via your pop client in spite of the above. My guess is somehow the client first tries plain authentication without STARTTLS before trying STARTTLS. In my case with pop3 and T'bird I use Port 995 Connection security: SSL/TLS Authentication: Normal password I haven't tried port 110 and STARTTLS (mostly I use IMAP anyway).
Re: [Dovecot] dovecot: pop3-login: Disconnected (tried to use disabled plaintext auth): method=PLAIN
After reading this: http://wiki2.dovecot.org/Authentication/Mechanisms It seems that PLAIN is OK, if I am using STARTTLS, which I believe I am. I mean, I've set it up, and it _seems_ to work. So the question I have, to the list, is... how can I verify that the passwords are being sent over STARTTLS. Quoting: The simplest authentication mechanism is PLAIN. The client simply sends the password unencrypted to Dovecot. All clients support the PLAIN mechanism, but obviously there's the problem that anyone listening on the network can steal the password. For that reason (and some others) other mechanisms were implemented. Today however many people use SSL/TLS, and there's no problem with sending unencrypted password inside SSL secured connections. So if you're using SSL, you probably don't need to bother worrying about anything else than the PLAIN mechanism. On 1/28/2011 2:48 AM, Mark Sapiro wrote: O So you successfully get mail via your pop client in spite of the above. My guess is somehow the client first tries plain authentication without STARTTLS before trying STARTTLS. In my case with pop3 and T'bird I use Port 995 Connection security: SSL/TLS Authentication: Normal password I haven't tried port 110 and STARTTLS (mostly I use IMAP anyway).
[Dovecot] cgroup support
Hi, I am setting up a system that enforces cgroup restrictions when a user logs in via SSH, and for all the services that are run by a particular user. I am also running dovecot to give users IMAP/POP access to their mailboxes. However, to be part of a cgroup, PIDs must be explicitly added to the cgroup tasks file. So for now, all my processes are run with resource restrictions, except for Dovecot processes. It would be really cool if dovecots child/worker processes could be added the a cgroup in addition to the usual setuid/chroot protections that already exists. Adding a process to a cgroup is a matter of writing the PID to the correct cgroup tasks file. If this were implemented as an extra field in userdb, it could be very powerful, and allow for all kinds of resource management/accounting of dovecot processes. This would obviously not be cross-platform, since cgroups are a feature of the Linux kernel. Would that be a problem? Is support for cgroups something that could be considered for dovecot at all? Are there other ways to put dovecot processes in cgroups? I do not really have a patch or a plan for how everything would work out in detail. If this would be useful for dovecot, I would be happy to start hacking on a patch. Cheers Andreas
Re: [Dovecot] Best filesystem?
Hi all, Pardon me for chiming in... Until then, email needs UPSs, it seems. You cannot rely on commercial power without a UPS in any critical system. As a plus, UPS filters the mains giving your PSU a big relief. What's more worrying me about this filesystem-war (let me follow this bad behaviour and throw in ext4...) is the fact no one mentions the battery backed up raid controller. Cheers, Robert
Re: [Dovecot] cgroup support
Have you tried post login scripts ? On Sun, 30 Jan 2011 13:28:22 +0100, Andreas Pelme andr...@pelme.se wrote: Hi, I am setting up a system that enforces cgroup restrictions when a user logs in via SSH, and for all the services that are run by a particular user. I am also running dovecot to give users IMAP/POP access to their mailboxes. However, to be part of a cgroup, PIDs must be explicitly added to the cgroup tasks file. So for now, all my processes are run with resource restrictions, except for Dovecot processes. It would be really cool if dovecots child/worker processes could be added the a cgroup in addition to the usual setuid/chroot protections that already exists. Adding a process to a cgroup is a matter of writing the PID to the correct cgroup tasks file. If this were implemented as an extra field in userdb, it could be very powerful, and allow for all kinds of resource management/accounting of dovecot processes. This would obviously not be cross-platform, since cgroups are a feature of the Linux kernel. Would that be a problem? Is support for cgroups something that could be considered for dovecot at all? Are there other ways to put dovecot processes in cgroups? I do not really have a patch or a plan for how everything would work out in detail. If this would be useful for dovecot, I would be happy to start hacking on a patch. Cheers Andreas -- Emerson Pinter Picture Internet +55 11 5089 8100 http://www.picture.com.br/
Re: [Dovecot] cgroup support
On 30 jan 2011, at 16:05, Emerson Pinter wrote: Have you tried post login scripts ? Wow, I had missed the post login scripts. That will solve this problem very nicely! Is there anything dovecot cannot do? :) Cheers Andreas
Re: [Dovecot] cgroup support
On Sun, 2011-01-30 at 17:03 +0100, Andreas Pelme wrote: On 30 jan 2011, at 16:05, Emerson Pinter wrote: Have you tried post login scripts ? Wow, I had missed the post login scripts. That will solve this problem very nicely! Hmm. With v1.x it works, but in v2.0 the post-login scripts are run in a separate process and I don't think you can get the PID there without some changes to code..
Re: [Dovecot] sql : uid, gid, home ignored ?
On Sun, 2011-01-30 at 11:18 +0100, Hurricane wrote: 2011-01-30 11:06:12auth: Debug: sql(theu...@mysite.com,1.2.3.4): query: SELECT username AS user, password, home AS userdb_home, uid AS userdb_uid, id AS userdb_gid, home, uid, gid FROM user WHERE username = 'theuser' AND domain = 'mysite.com' AND active = 'Y' You have too much stuff in there. userdb_* fields are used if you enable userdb prefetch. It doesn't look like you've enabled it. The uid/home/gid are completely ignored in password_query. 2011-01-30 11:06:12auth: Debug: auth(theu...@mysite.com,1.2.3.4): username changed theu...@mysite.com - theuser It's unlikely you want this either, so select also domain field. You didn't show your dovecot -n output so I don't know what userdb you're using, but I guess it's userdb static. Switch to userdb sql and it'll use user_query. (And maybe for optimization userdb prefetch too.)
Re: [Dovecot] dovecot@dovecot.org
On Sun, 2011-01-30 at 12:17 +1100, Frank Crawford wrote: On Sat, 2011-01-29 at 22:49 +0100, mouss wrote: Le 29/01/2011 21:15, Veronese Claudio a écrit : I apologize, but I can not find a complete list of directives in dovecot.conf possible. where can I find? thanks dunno if it's complete, but http://wiki.dovecot.org/MainConfig That is good, but is for Dovecot 1. Is there a similar listing for Dovecot 2? In v1.x it's only a conversion from dovecot-example.conf to wiki format. In v2.0 such conversion isn't done, but all the stuff is in the example-config/* files.
Re: [Dovecot] Cooperating with dovecot in its Maildir
On Sat, 2011-01-29 at 12:04 -0600, Rob Browning wrote: I saw that, but I wasn't sure if the fact that a message might receive a new UID could be a problem. It's a theoretical problem mostly, especially in your case. It's mainly visible when doing stress testing with large maildirs. I doubt in regular use it matters. Courier doesn't try to prevent it in any way and it seems to have worked mostly ok. Or is the UID supposed to change when the flags change? No. OK, so it sounds like if we wanted to be completely safe, we probably need to know that we're in a dovecot Maildir, and then we need to know where to create the appropriate dovecot-uidlist.lock file whenever renaming files. There's no good way to find out where the uidlist files are, if they're not in the maildir itself. They typically are. Do you happen to know if the liblockfile (lockfile_create(3), etc.) .lock strategy is compatible with dovecot's approach? Should be. It's possible though that in a future version there is no .lock file but rather the uidlist is locked directly with fcntl.
Re: [Dovecot] dovecot: pop3-login: Disconnected (tried to use disabled plaintext auth): method=PLAIN
On 11:59 AM, John Espiro wrote: After reading this: http://wiki2.dovecot.org/Authentication/Mechanisms It seems that PLAIN is OK, if I am using STARTTLS, which I believe I am. I mean, I've set it up, and it _seems_ to work. So the question I have, to the list, is... how can I verify that the passwords are being sent over STARTTLS. Your client is ultimately logging in after STARTTLS because Dovecot is not allowing it to login before, but it apparently is trying and possibly sending a cleartext password or there would be no 'disconnected' log message. Sniff the port 110 packets during a login from your client and see what's going on. One other thought - Is there more than one account on this server configured in your client and if so, are they all using STARTTLS? -- Mark Sapiro m...@msapiro.netThe highway is for gamblers, San Francisco Bay Area, Californiabetter use your sense - B. Dylan
Re: [Dovecot] Cooperating with dovecot in its Maildir
Timo Sirainen t...@iki.fi writes: On Sat, 2011-01-29 at 12:04 -0600, Rob Browning wrote: OK, so it sounds like if we wanted to be completely safe, we probably need to know that we're in a dovecot Maildir, and then we need to know where to create the appropriate dovecot-uidlist.lock file whenever renaming files. There's no good way to find out where the uidlist files are, if they're not in the maildir itself. They typically are. Right, I was assuming we might just have to require the user to tell us whenever they're not in the normal place. Do you happen to know if the liblockfile (lockfile_create(3), etc.) .lock strategy is compatible with dovecot's approach? Should be. It's possible though that in a future version there is no .lock file but rather the uidlist is locked directly with fcntl. OK, though as you're probably aware, there may be some issues cross-platform, and/or with shared FSs. Avery wrote an interesting summary recently: http://apenwarr.ca/log/?m=201012#13 Thanks again -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4
Re: [Dovecot] Best filesystem?
Nick Edwards put forth on 1/29/2011 7:14 PM: On Sun, Jan 30, 2011 at 4:00 AM, Stan Hoeppner s...@hardwarefreak.comwrote: Nick Edwards put forth on 1/28/2011 9:47 PM: xfs is not very nice to you if you lose power, it's not as bad as it used to be, but it still gives you 0 byte files, so make sure you have a good UPS to issue a safe shutdown of the server, if you do, xfs is better, using CentOS. It would be nice if you spoke from experience or documentation instead of rumor. I do speak from experience, it is also a /very/ well known fact, for someone who rants on and on and on with self justification, you sure don't actually know a lot do you. It's interesting that you cut the fact out of my reply, then proceeded to rant against me personally, instead of discussing the subject at hand. Here, I'll add the XFS power loss fact back in, as that is the subject of this thread currently: From: http://xfs.org/index.php/XFS_FAQ Q: Why do I see binary NULLS in some files after recovery when I unplugged the power? Update: This issue has been addressed with a CVS fix on the 29th March 2007 and merged into mainline on 8th May 2007 for 2.6.22-rc1... That proves you factually incorrect. Now, if you were using XFS prior to May 2007, or using an antique kernel in a Linux ditsro (RHEL/CentOS) that shipped well after that date but without the patches, I can see how you might have had issues. Patching systems is not the responsibility of the devs however. If you were running un-patched systems after May 2007, that's not the fault of XFS. Given the problem you describe was fixed almost 4 years ago, do you feel it's proper to continue to denigrate XFS today, spreading FUD 4 years after the issue was resolved? Also, I didn't quote anything from Wikipedia. It seems you have confused the XFS FAQ website with Wikipedia simply because it uses Wiki software and thus has the same look/feel as Wikipedia. Are you saying that one can't trust the content on any site running Wiki software because it simply looks like Wikipedia? -- Stan
Re: [Dovecot] Best filesystem?
On Sun, 2011-01-30 at 16:13 -0600, Stan Hoeppner wrote: be, but it still gives you 0 byte files, so make sure you have a good UPS .. Q: Why do I see binary NULLS in some files after recovery when I unplugged the power? 0 byte files != NULL bytes in files. My guess is it's the same problem as described in http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/
Re: [Dovecot] Best filesystem?
Ron Leach put forth on 1/30/2011 5:00 AM: Nick Edwards wrote: xfs is not very nice to you if you lose power, it's not as bad as it used to be, but it still gives you 0 byte files, I began to worry about this after that other thread showed XFS's considerable strengths, and this one weakness. Co-incidentally,we had already just built a couple of XFS servers in raid1 configurations (the second is purely an rdiff-backup server for the first, and both are raid1). Because our work is frequently the subject of very close legal scrutiny, we're utterly paranoid about losing email - that's why we've created those two redundant servers. I remember Stan (in the other thread) saying, also, that write-delays due to caching were more or less built-in to the kernel anyway, so XFS may not be alone in this problem. What I am not (yet) sure about, is whether XFS is any 'more' vulnerable than others, or is any 'more' catastrophically damaged, than others, due to power fail. Has any analysis of this been published? It's all in the XFS FAQ. See #23 for the power fail issue patch. Did you read the other excellent XFS resources available? Users guide: http://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide//tmp/en-US/html/index.html File system structure: http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure//tmp/en-US/html/index.html Training labs: http://xfs.org/docs/xfsdocs-xml-dev/XFS_Labs/tmp/en-US/html/index.html However, like the OP, our scale is quite small and this (potentially) gives us one advantage over those very large users. We could forgo some 'performance' if there were options in XFS that could reduce its 'vulnerability'. I looked at the XFS FAQ, and several of the archived messages on the XFS list, but could not see any create options, or mount options, that would reduce or inhibit the 'vulnerability window' (but I'm no expert on filesystems, or the kernel, so maybe I didn't understand what the FAQ was telling me). Would appreciate any suggestions from those who use and know XFS. Once again: http://xfs.org/index.php/XFS_FAQ See #23 There is nothing to configure to make XFS 'more resilient' to power failure. There was a bug that caused problems after power failure. Again, the bug was fixed in May 2007, almost 4 years ago. That's the Jurassic period in internet time folks. There is no 'vulnerability window'. Now you understand my frustration with Nick for spreading FUD. so make sure you have a good UPS to issue a safe shutdown of the server, We are very susceptible to power outages, duration anything from 12 seconds to 14 hours (we're not in a city) and never notified in advance. We use APC desktop UPS for workstations and the few servers we have, and we then shut down. For security, the shutdown needs to be automatic so that it takes effect if the site is unmanned - overnight, for example. Aren't you using the net enabled APC units? They have a NIC slot for exactly this purpose. You install software on your physical hosts (some come with such software like Linux) and configure it. When wall power fails the UPS goes into battery mode, and when the battery hits a configurable amount of remaining capacity, it sends a packet to all connected/configured hosts to shut down. This has been available since the mid 1990s, and it's a fabulous, necessary feature. This is not limited to APC. Many UPS vendors offer such network capabilities. 'Absolutely secure email' needs the speed of XFS, the performance of XFS on multitudes of small files, and the fault-tolerance of some kind of non-volatile storage coupled with positive confirmation of successful writes. One day, maybe. Until then, email needs UPSs, it seems. Anything, everything, needs a UPS, except a laptop or smartphone (anything w/an inbuilt battery). Online models are best, and typically more expensive. -- Stan
Re: [Dovecot] Best filesystem?
On Mon, Jan 31, 2011 at 12:40:11AM +0200, Timo Sirainen wrote: On Sun, 2011-01-30 at 16:13 -0600, Stan Hoeppner wrote: be, but it still gives you 0 byte files, so make sure you have a good UPS .. Q: Why do I see binary NULLS in some files after recovery when I unplugged the power? 0 byte files != NULL bytes in files. My guess is it's the same problem as described in http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ But is it relevant for dovecot ? Isn't dovecot doing the necessary fsync()'s, so this should really be a non-issue ? -jf -jf
[Dovecot] Duplicate messages when moving message to inbox
Hello, I am encountering a strange problem but I'm not sure if it's a dovecot problem or a Mac Mail problem. The symptom is that if I move a message from a folder into my inbox the message ends up showing up as a duplicate in Mac Mail. On the server it's still just one message. This ONLY happens when I move a message IN to the inbox. It does not happen when I move a message out of my inbox into a folder, or when I move a message between folders. I can force a sync on Mac Mail and the duplicate is still there. If however I quit Mac Mail and start it up again, the duplicate disappears. I can move one of the duplicated messages out of my inbox into a folder and then back into my inbox repeatedly and end up with (apparently) an arbitrary number of copies of the same message. On the server, though, there is still only one file. Surely I'm not the first person to have encountered this behavior? Does anyone happen to know if this is a dovecot bug or a Mac Mail bug (or, hopefully, a configuration problem)? Thanks, rg
Re: [Dovecot] Best filesystem?
On Mon, 2011-01-31 at 00:03 +0100, Jan-Frode Myklebust wrote: 0 byte files != NULL bytes in files. My guess is it's the same problem as described in http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ But is it relevant for dovecot ? Isn't dovecot doing the necessary fsync()'s, so this should really be a non-issue ? Depends on what mail_fsync has been set to. As long as it's not never, then it should be non-issue for Dovecot.
Re: [Dovecot] Best filesystem?
Timo Sirainen put forth on 1/30/2011 4:40 PM: On Sun, 2011-01-30 at 16:13 -0600, Stan Hoeppner wrote: be, but it still gives you 0 byte files, so make sure you have a good UPS .. Q: Why do I see binary NULLS in some files after recovery when I unplugged the power? 0 byte files != NULL bytes in files. My guess is it's the same problem as described in http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ Yes, very similar Timo. To be clear, for any subscribers who haven't followed all of the various filesystem and data security threads, with any modern *nix system, you WILL lose data when power fails. How much depends on how many writes to disk were in flight when the power failed, and how one has their RAID controller and inside-the-disk caches configured, whether using barriers, etc. I believe I mentioned this when discussing the merits of XFS and ZFS with Frank, who stated Solaris/ZFS were immune to this, to which I called BS. They aren't immune, as Ted T'so clearly states. For those who don't know, Ted T'so is an MIT PH.D., is the creator of EXT2/3, and is to this day an active Linux kernel hacker/developer on filesystems and storage drivers. -- Stan
Re: [Dovecot] Best filesystem?
On Sun, 2011-01-30 at 17:07 -0600, Stan Hoeppner wrote: 0 byte files != NULL bytes in files. My guess is it's the same problem as described in http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ Yes, very similar Timo. To be clear, for any subscribers who haven't followed all of the various filesystem and data security threads, with any modern *nix system, you WILL lose data when power fails. How much depends on how many writes to disk were in flight when the power failed, and how one has their RAID controller and inside-the-disk caches configured, whether using barriers, etc. But it's not just about how much data is lost. It's also about if any existing data is unexpectedly lost. That's why people were complaining about ext4, because suddenly when renaming a file over another might lose both the old and the new file's contents if power got lost, while with ext3 either the old or the new data stayed behind. Then they did all kinds of things to ext4 to fix this / make it less likely. I don't know how likely that is with XFS. Probably one way to test would be something like: 1. Create 100 files of 1 MB size. 2. sync 3. Create a new file of 2 MB size rename() it over a file created in step 1. 4. Repeat 3 until all files are replaced 5. Kill the power immediately after done Then you can compare filesystems based on how many files there are whose size or content doesn't match.
Re: [Dovecot] Smart IMAP proxying with imapc storage
On Fri, 2011-01-28 at 20:17 +0200, Timo Sirainen wrote: I just committed a very early initial implementation of imapc storage backend to v2.1 hg: http://hg.dovecot.org/dovecot-2.1 This is fully functional now, at least in theory. TODO: * Don't read entire message bodies into a string in memory * Fetch more than one message at a time when it's known that more messages are going to be fetched The above features are done now. So the performance shouldn't be too horrible anymore. * SSL/TLS support to remote server * Support locally stored indexes * Error handling could be better * Eventually make lib-storage API more asynchronous - make imapc more asynchronous These are not. Another TODO item is: * Add support for multiple connections to IMAP server. This is required for e.g. THREAD command to work correctly.
Re: [Dovecot] Best filesystem?
Timo Sirainen put forth on 1/30/2011 4:40 PM: On Sun, 2011-01-30 at 16:13 -0600, Stan Hoeppner wrote: be, but it still gives you 0 byte files, so make sure you have a good UPS .. Q: Why do I see binary NULLS in some files after recovery when I unplugged the power? 0 byte files != NULL bytes in files. IIRC, the former is the current XFS (correct) behavior (which you quoted from an old email, not today), and the latter is the behavior fixed by the 2007 patch. The text in the FAQ can be a bit confusing as neither the before or after behavior is thoroughly explained. My guess is it's the same problem as described in http://thunk.org/tytso/blog/2009/03/12/delayed-allocation-and-the-zero-length-file-problem/ Similar but not quite exactly the same. For practical purposes, as seen by an OP, they are functionally the same problem, so the exact cause is a bit irrelevant from an OP's perspective. XFS, along with all the journaling filesystems, still suffers from this delayed allocation dilemma during power loss/crash, as I've stated previously on this list. If you want performance, there is a required sacrifice. In this case, delayed allocation gives the performance, but it sacrifices some on disk guarantees WRT power loss or crash. Again, this isn't the same issue as the XFS bug fixed in 2007. An XFS system today will still suffer data loss due to power loss/crash if there is write data in the Linux buffer cache, which is the case for all Linux journaling filesystems, not just XFS, as Ted T'so so eloquently points out in his blog post. -- Stan
Re: [Dovecot] Best filesystem?
Timo Sirainen put forth on 1/30/2011 5:27 PM: But it's not just about how much data is lost. It's also about if any existing data is unexpectedly lost. That's why people were complaining about ext4, because suddenly when renaming a file over another might lose both the old and the new file's contents if power got lost, while with ext3 either the old or the new data stayed behind. Then they did all kinds of things to ext4 to fix this / make it less likely. People were complaining about EXT4 because EXT2/3 implemented features to save bad programmers from themselves, even though it is NOT the job of the filesystem code to do so. EXT4 removed these safeguards and bad programmers who relied on EXT2/3 to cross their Ts and dot their Is for them threw fits when they realized EXT4 didn't do this for them any longer. Google O_PONIES, and the blog entry from Eric Sandeen, an XFS developer, regarding O_PONIES: http://sandeen.net/wordpress/?p=42 XFS never had such protections for bad programmers. The bulk of IRIX developers were well/over educated and well/over paid, usually working for the US government in one from or another. Such developers knew when to fsync or take other measures to make sure critical data hit the disks. I dare say the average Linux developer didn't/doesn't have quite the same level of education or proper mindset as IRIX devs. If they'd had such skill we'd not have seen the EXT2/3 to EXT4 problem Ted describes. I don't know how likely that is with XFS. Probably one way to test would be something like: 1. Create 100 files of 1 MB size. 2. sync 3. Create a new file of 2 MB size rename() it over a file created in step 1. 4. Repeat 3 until all files are replaced 5. Kill the power immediately after done Then you can compare filesystems based on how many files there are whose size or content doesn't match. Depending on your hardware, you may need a lot larger test set than 100 files. If you don't sync between steps 4/5 you may not see anything except that the 100 overwrites never occurred, as those writes may all still be in the buffer cache when you pull the plug. Assuming you can call pull_the_plug with perfect timing, I can't tell you what the exact results would be, as I've never tested this. You'll likely lose a pre-existing file depending on what inodes were committed to the journal without their respective files being written. I'm pretty sure this is one of those scenarios that prompt programming professors to teach create new/delete old/rename new to old, instead of rename/edit in place/overwrite. I recall back in the day that WordPerfect and Microsoft Word both implemented making a temp copy of every file opened for edit, because crashes of said text editors would routinely corrupt the opened file. I believe MS Word and Open Office Writer still do the same today. It seems some fundamentals haven't really changed that much in 25 years. -- Stan
[Dovecot] disable_plaintext_auth = no ignored by dovecot in Ubuntu 10.04
Help!! I have been trying to get Dovecot configured to allow plaintext auth with no success. After some testing with the mail system I discovered the dovecot -a commant to dump the config file values from the program. Gee, changing the value of disable_plaintext_auth had no effect on what the program reported this value to be! To eliminate possible errors caused by other config file entries I finally restored the /etc/dovecot/dovecot.conf file that was created by installing the package. I then changed the one line to uncomment disable_plaintext_auth and set it equal to no. restart the box and execute dovecot -a The value is still set to the default value of yes... Help! What is happening and how to I get this system to allow plain text auth without TLS? I did also try setting different ports for imap, imaps, pop3, pop3s inside the seperate protocol blocks with no effect. Below you will find a partial of the config file. Below that you will find the output of dovecot -a and dovecot -n Thank you in advance for any help with this problem. Dave dave@mail:~$ cat /etc/dovecot/dovecot.conf ## Dovecot configuration file # If you're in a hurry, see http://wiki.dovecot.org/QuickConfiguration # dovecot -n command gives a clean output of the changed settings. Use it # instead of copypasting this file when posting to the Dovecot mailing list. # '#' character and everything after it is treated as comments. Extra spaces # and tabs are ignored. If you want to use either of these explicitly, put the # value inside quotes, eg.: key = # char and trailing whitespace ... #listen = * disable_plaintext_auth = no dave@mail:~$ sudo dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-24-generic i686 Ubuntu 10.04.1 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 imaps pop3s managesieve ssl_cert_file: /etc/ssl/certs/ssl-mail.pem ssl_key_file: /etc/ssl/private/ssl-mail.key ssl_cipher_list: ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): imap_client_workarounds(managesieve): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): lda: postmaster_address: postmaster mail_plugins: sieve quota_full_tempfail: yes deliver_log_format: msgid=%m: %$ rejection_reason: Your message to %t was automatically rejected:%n%r auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix plugin: sieve: ~/.dovecot.sieve sieve_dir: ~/sieve dave@mail:~$ dave@mail:~$ sudo dovecot -a # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-24-generic i686 Ubuntu 10.04.1 LTS base_dir: /var/run/dovecot log_path: info_log_path: log_timestamp: %Y-%m-%d %H:%M:%S syslog_facility: mail protocols: imap pop3 imaps pop3s managesieve listen: * ssl_listen: ssl: yes ssl_ca_file: ssl_cert_file: /etc/ssl/certs/ssl-mail.pem ssl_key_file: /etc/ssl/private/ssl-mail.key ssl_key_password: ssl_parameters_regenerate: 168 ssl_cipher_list: ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM ssl_cert_username_field: commonName ssl_verify_client_cert: no disable_plaintext_auth: yes verbose_ssl: no
Re: [Dovecot] Thunderbird doesn't popup on new mail - SOLVED
It seems that deleting all files from Maildir with name begining with dovecot , will fix this issue. Ex : rm ~/Maildir/dovecot* Those files will be recreated by dovecot (dovecot-keywords , dovecot-uidlist ,dovecot.index , etc )and from now on , the thunderbird will notify about every new mail. Adrian On Thu, 2011-01-27 at 08:54 +0200, Adrian Stoica wrote: I have a question : i've upgrade dovecot from 1.1.4 to 2.0.9 and everything is fine except thunderbird clients , that no longer show a popup when new mail arrives. Anyone know why ? No, but the main visible difference to clients is the IMAP CAPABILITY string that is sent before login. Although v2.0.9 already has IDLE in there as well, so I don't really know. You could anyway try setting explicitly: imap_capability = IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS QUOTA ACL
