Re: [Dovecot] Certificate Server name!

[WJCarpenter]

Easy!  Either connect to imap.tib.com instead of mail.tib.com, or 
create and install a new security certificate on the server which is 
for mail.tib.com instead.


Another solution is to obtain and install a wildcard certificate (which 
will be good for all *.tib.com). 

That's the good news.  The bad news is that the commercial certificate 
authorities charge extra for wildcard certificates because they know 
they're more valuable to you (and not because it costs them anything 
extra in creating them, except maybe lost sales of certificates for 
specific names). 

BTW, you can get free certificates from http://cacert.org (no 
affiliation except as a user), though the first time your users see them 
they may have to answer a pop-up about a "funny" certificate.  (My 
experience is that most users just click OK and don't give it much 
thought.  The ones who do think about it tend to be more sophisticated 
anyhow, so they can sort it out rather than just switching off the 
computer in a panic and watching TV for the rest of their lives.)

Re: [Dovecot] IMAP SPECIAL-USE extension

[WJCarpenter]

On Fri, 2011-12-02 at 17:12 +0200, Timo Sirainen wrote:

I'm anyway now wondering what the defaults should be? Could someone
check what these defaults are for Outlook and any other clients you
have:

  * "Drafts" is used by all clients
  * "Trash" is used by all clients
  * "Junk" is used by all clients? Or is "Spam" used by some?
  * "Sent" is used by Thunderbird, Evolution
  * "Sent Messages" is used by Apple Mail



SquirrelMail (http://www.squirrelmail.org) and Hastymail2 
(http://www.hastymail.org) webmail clients let you customize 
draft/trash/sent, but the defaults for both are Drafts, Trash, and Sent.

Re: [Dovecot] Storing passwords encrypted... bcrypt?

[WJCarpenter]

Was just perusing this article about how trivial it is to decrypt
passwords that are stored using most (standard) encryption methods (like
MD5), and was wondering - is it possible to use bcrypt with
dovecot+postfix+mysql (or posgres)?


Ooop... forgot the link:

http://codahale.com/how-to-safely-store-a-password/


AFAIK, that web page is correct in a relative sense, but getting bcrypt 
support might not be the most urgent priority.


In his description, he uses the example of passwords which are 
"lowercase, alphanumeric, and 6 characters long" (and in another place 
the example is "lowercase, alphabetic passwords which are ≤7 
characters", I guess to illustrate that things have gotten faster).  If 
you are allowing your users to create such weak passwords, using bcrypt 
will not save you/them.  Attackers will just be wasting more of your CPU 
time making attempts.  If they get a copy of your hashed passwords, 
they'll likely be wasting their own CPU time, but they have plenty of 
that, too.


There are plenty of recommendations for what makes a good password / 
passphrase.  If you are not already enforcing such rules (perhaps also 
with a lookaside to one or more of the leaked tables of passwords 
floating around), then IMHO that's much more urgent.  (One of the best 
twists I read somewhere [sorry, I forget where] was to require at least 
one uppercase and one digit, but to not count them as fulfilling the 
requirement if they were used as the first or last character.)


Side note, but for the sake of precision ... attackers are not literally 
decrypting passwords.  They are guessing passwords and then performing a 
one-way hash to see if they guessed correctly.  As a practical matter, 
that means that you have to ask your users to update their passwords any 
time you change the password storage scheme.  (I don't know enough about 
bcrypt to know if that would be required if you wanted to simply 
increase the work factor.)

Re: [Dovecot] Storing passwords encrypted... bcrypt?

[WJCarpenter]

On 1/3/2012 2:38 PM, Simon Brereton wrote:

http://xkcd.com/936/


As they saying goes, entropy ain't what it used to be.

https://www.grc.com/haystack.htm

However, both links actually illustrate the same point: once you get 
past dictionary attacks, the length of the password is dominant factor 
in the strength of the password against brute force attack.

Re: [Dovecot] Storing passwords encrypted... bcrypt?

[WJCarpenter]

On 1/3/2012 5:25 PM, Charles Marcus wrote:
I think ya'll are missing the point... not sure, because I'm still not 
completely sure that this is saying what I think it is saying (that's 
why I asked)...


I'm sure I'm not missing the point.  My comment was that password length 
and complexity are probably more important than bcrypt versus sha1, and 
you've already addressed those.  Given that you use strong 15-character 
passwords, pretty much all hash functions are already out of reach for 
brute force.  bcrypt is probably better in the same sense that it's 
harder to drive a car to Saturn than it is to drive to Mars.

[Dovecot] using complete email address as username

[WJCarpenter]

I've been poking around the dovecot wiki, and this idea looks
plausible.  I'm just double-checking with the assembled experts to see
if there is some "gotcha" that will happen to me down the road.

I have in mind a deployment of dovecot which will use serve several
unrelated domains.  My plan is to use the complete email address of
the users as the unique userid.  So, for example, a user with email
address <[EMAIL PROTECTED]> is unrelated to a user with email address
<[EMAIL PROTECTED]>.  The "home" directories of such users will look
something like:

  /some/path/[EMAIL PROTECTED]/
  /some/path/[EMAIL PROTECTED]/

I do also plan to use ACLs for some things, so there could be ACL
entries of the form:

  user:[EMAIL PROTECTED] ...

I will probably be using the quota plugin for individual users right
away and group quotas later when that's officially available.

So, is all this likely to work or will it collapse in a miasma of
unmet expectations?

Thanks in advance for any comments!

[Dovecot] rule of thumb for indexing overhead

[WJCarpenter]

I realize it's hard to be precise about this, but does anyone have a
feel or rule of thumb for a couple of aspects of indexing overhead?

1.  Proportionally, how much space does it take for all 4 files?  If I
want to give my users a quota of 100 MB for messages, how much real
space should I plan for so that I won't run out of space for indexing?

2.  What's the overhead in rebuilding index files?  Suppose I use an
LDA other than dovecot, so at least the INBOX index is frequently
getting out of date.  Does it cost much (in CPU, memory, and disk IO)
to rebuild the index files?  (I'm using maildir.)

Re: [Dovecot] rule of thumb for indexing overhead

[WJCarpenter]

wjc> I realize it's hard to be precise about this, but does anyone
wjc> have a feel or rule of thumb for a couple of aspects of indexing
wjc> overhead?

ss> You can always put the indexes in non-quota space like var. That
ss> way the indexes don't get counted against the users files, and
ss> won't get corrupted if a user goes over quota.

Thanks, but I think you misunderstood my question.  I'm trying to
figure out how much disk space I'll actually need for holding the
index files.  In other words, I'm looking for a planning factor to
figure out how much disk to buy when I got to Fry's :-).

(I'll be using the Dovecot quota stuff, so index files are already not
in the count.)

Re: [Dovecot] rule of thumb for indexing overhead

[WJCarpenter]

> Also, are there any drawbacks of using exim to do the local delivery?

I'm very interested in the answer to this question, too.  So far I have found 
(through reading, not trying things yet) that
Dovecot's quota handling is more flexible than Exim's (exim is pretty much 
limited to FS quotas, I think, which is no good for
virtual users).  Dovecot's Sieve implementation has more features than Exim's.  
Both of these happen to matter to me (and led me
to Dovecot in the first place.)

An upside for Exim doing the delivery is that you theoretically can arrange for 
some additional rejections to happen at SMTP
time.  Pragmatically, those additional rejections are typically not done at 
SMTP time anyhow (e.g., an explicit fail from a user
Exim filter).

Re: [Dovecot] rule of thumb for indexing overhead

[WJCarpenter]

> We use exim all the way to local delivery. And it handles Maildir++
> quotas just fine.

Ah, right.  I was misremembering.  It's the DB-stored quotas in dovecot that I 
was thinking of using some time back.  Sorry for
my misstatement.

> of indexing during delivery are negligible for us in general and
> potentially negative in some scenarios (mass mail to many/all users).

Is this negative hypothetical or have you actually seen load spikes in 
situations like this?

> I don't think there are many situations where a MTA can do better
> than a LDA when it comes to rejections during SMTP time. Since at

I agree.  The opportunities are more theoretical than practical.  If you 
include processing user .forward files as part of the
exim recipient verification or if you include a quota check after SMTP DATA 

[Dovecot] passing quota to deliver via command line

[WJCarpenter]

Is there any way to pass things like "quota=" for the quota plugin to
the deliver LDA via the command line or as an environment variable or
whatever?  I've got all the info I need in my MTA to avoid a userdb
lookup in the LDA, but I still have to do the userdb query to get the
per-user quota number (the default quota works fine, of course).
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

[Dovecot] securing dovecot proxy connections

[WJCarpenter]

The wiki <http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy>
page says this: "The connections created to the destination server
can't be TLS/SSL encrypted.".

Hrmm.  Right now, with perdition, I'm forcing the use of STARTTLS on
the internal connections.  I'd just as soon get rid of perdition (to
have one less moving part in my architecture), but I need the secure
connections.  

Is there a way to configure dovecot's internal proxy connections to
use STARTTLS or some other SSL/TLS level of security?  (Without a
bunch more research, I don't know what the interaction is between the
real client, the dovecot proxy, and the destination server.)
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

Re: [Dovecot] Quota bug in deliver?

[WJCarpenter]

bc> When I put a standard quota into the config as follows, the
bc> delivery succeeds (apparently deliver checks the standard quota,
bc> while IMAP correctly checks the quota from the database if it is
bc> returned.  Sep 5 21:46:11 ms4

bc> The mailbox is still over quota here, but deliver does not seem to
bc> notice.

I recently debugged that situation in my own configuration.  Are you
using prefetches for your user query?

If you are using prefetch for your userdb lookups, you still need a
separate user query to be used by deliver (it doesn't do the password
query).  The wiki pages show a configuration for keeping the prefetch
for IMAP but having a user query for deliver.  I don't know if that
works since I simply got rid of my prefetch completely and moved on to
a different problem when that cured it. (I plan to try to put the
prefetch back in later when I get some spare time.)
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

Re: [Dovecot] securing dovecot proxy connections

[WJCarpenter]

wjc> Is there a way to configure dovecot's internal proxy connections
wjc> to use STARTTLS or some other SSL/TLS level of security?
wjc> (Without a

mmj> Just create encrypted tunnel between the peers and send your
mmj> traffic through it.  IPSec, ssh etc..

Thanks for the suggestion.  I had thought of that, but all my
front-end servers are also back-end servers.  (I'm just letting the
users come in on any server -- usually the correct one -- and want to
transparently connect them to the correct back-end if they happen to
come into the wrong server.)  So, even with just 5 servers, that's 20
tunnels to keep afloat through reboots, etc.  In principle no problem,
but it's a lot of balls in the air.

Re: [Dovecot] securing dovecot proxy connections

[WJCarpenter]

ka> per another current thread (o/s tuning for imap), I've installed
ka> imapproxy, and it supports starttls to the backend imap server. It
ka> doesn't use encryption on the incoming connections though, since
ka> they are presumably from localhost (squirrelmail).  Ken

That's an interesting thought.  Have you actually gotten its STARTTLS
to work?  I tried it a couple days ago with no luck, but maybe I just
didn't try hard enough.
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

Re: [Dovecot] dovecot dspam plugin using libdspam

[WJCarpenter]

It's none of my business, but here's a suggestion for more work for
somebody.  :-)

The idea behind the dspam plugin -- watching things move from folder
to folder -- seems very, very clever to me.  I wonder if it might have
wider applicability, even if we can't see what it is today?  Taking
account some of the performance concerns discussed way, way back, it
still seems like a good idea to separate out the watching part from
the processing part.

What if there were a generic "movewatcher" plugin which:

-- observed movements from one folder to another

-- configured with some kind of list or regexp or something for the
   source folders which were interesting/uninteresting (e.g., "SPAM")
   and likewise for the target folders (e.g., "Trash")

-- configured for some list of email header values to log

-- configured for some place/way to log those header values

-- configured for some format for logging with maybe %-escapes for
   interesting information (e.g., "%u" means the userid, "%s" means
   the source folder name, "%t" means the target folder name, "%hFoo"
   means the value of header "Foo", "%HFoo" means the string "Foo: "
   followed by the value of header "Foo").

Assuming everything interesting you would want to process is available
in one or more headers (in keeping with the Postmaster Code of Condut
Rule #2c: Thou shalt not look at content :-), you could log everything
interesting to some flat file (or FIFO) and write a completely
independent program to process that log.

For the particular case of dspam retraining, you're all good if you
have the dspam signature in the headers.  (I don't know how you end up
without that except during transition to dspam, so I'm not worried
about retraining from the raw message.)
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

Re: [Dovecot] Quota Exceeded message

[WJCarpenter]

ts> The first line is also configurable in v1.1. "Quota exceeded" text
ts> itself isn't though.

That's good since not every site wants to tell random outsiders that a
user's quota is full.  I know it's common practice, but it's really a
holdover from the early days of email where there wasn't much concern
for privacy or security.

Re: [Dovecot] What is your dovecot setup?

[WJCarpenter]

> My dovecot.conf sets
>   mail_location = maildir:~/Maildir/

dovecot has sometimes had trouble with "~".  I use "%h" instead.  (I think 
there is a mention of this on one of the wiki pages.)
 I don't recall if I changed to "%h" because I was having trouble or because I 
wanted to avoid trouble.

Here's what I have:

mail_location = maildir:%h/Maildir

On disk, I have ~/Maildir/cur/ (and new/ and tmp/) for the Inbox.  I also have 
things like ~/Maildir/.Trash/cur/ (and new/ and
tmp/).

Re: [Dovecot] do not lose mail when dovecot is dead

[WJCarpenter]

Eg. when a customer says: "Hey, I want my username to be "foo", and my 
email to be "[EMAIL PROTECTED]". :)
  


I encourage that on my servers.  It's just one more layer of defense 
against password guessing zombies; beats me if it makes any difference.  
I don't want to give them the userid for free just because they saw the 
email address on some spam list.

[Dovecot] proxy configuration

[WJCarpenter]

Before I spend some time experimenting with what might be impossible,
maybe someone can just tell me (either "how" or that it's
impossible).

I'd like to get perdition out of my environment (mainly to have one
less moving part in my architecture).  I'm looking at dovecot's
built-in proxying.  In my setup, I don't have dedicated front-end
machines.  A user can connect to any server, but their mail files live
on one particular machine.  I'd like to transparently proxy them from
whatever machine they happen to hit to their home machine.

Except for the "proxy_maybe" feature advertised for dovecot 1.1, I
don't see a surefire way to do this.  Has anyone done it, or can
anyone say for sure that it can't reasonably be done until
"proxy_maybe"?

Tx!
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

Re: [Dovecot] proxy configuration

[WJCarpenter]

> Patrick Ben Koetter wrote:
> Ed W wrote:


Thanks to both for your comments; very helpful.  I just wanted to make 
sure it could work in practice before I spent some time on it.  Sounds 
like it can be done.

[Dovecot] multiple ports, different characteristics

[WJCarpenter]

The wiki <http://wiki.dovecot.org/Iptables> leads me to believe that
the only way to configure dovecot to listen for the same protocol on
multiple ports is via external redirection (iptables or similar).  Is
that so, or can dovecot be directly configured to listen on multiple
ports?

Here's a concrete example of what I'm trying to accomplish.  Our
environment uses plaintext authentication, and so we force our users
to use TLS for IMAP.  For some internal connections (specifically with
dovecot built-in proxying), we need non-TLS connections for IMAP.
It's completely OK here if we need to listen on different ports for
the two flavors of connections.

Can dovecot be configured to handle this and/or do we need to run
multiple instances of dovecot with different config files?

Thanks
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

[Dovecot] NULL for password and proxy fields

[WJCarpenter]

In the course of experimenting with getting dovecot proxying to work,
I took a guess at two things.  These work fine for me, but now I'm
wondering if they are "as designed" or just a lucky accident that
might stop working in the future.

(I'm using dovecot 1.0.rc17, which is the included version in Ubuntu
Feisty.  (I am aware that some details change in v1.1, and I'm not
worried about that right now.)

http://wiki.dovecot.org/PasswordDatabase/ExtraFields/Proxy

1.  The wiki page says about password forwarding, "Make sure that the
authentication succeeds with any given password. You can do this by
using empty passwords."  I didn't know exactly now to interpret that
last part, so I tried returning NULL as the value of the "password"
field for proxy cases.  That seems to make dovecot skip the password
check will forwarding the user-submitted password to the proxy target
host.  Is that the intent of that part of that wiki page?

2.  It looks like any value at all for the "proxy" field in the passdb
lookup turns proxying on.  The one exception is a value of NULL for
"proxy", in which case proxying is not turned on and proxy-related
other fields are ignored.  Is that how it's intended to work?

If someone can give and informend confirmation that this is
as-designed, I can add some clarifying remarks on those points on the
wiki page.

Thanks.
-- 
[EMAIL PROTECTED] (WJCarpenter)PGP 0x91865119
38 95 1B 69 C9 C6 3D 2573 46 32 04 69 D6 ED F3

Re: [Dovecot] NULL for password and proxy fields

[WJCarpenter]

2.  It looks like any value at all for the "proxy" field in the passdb
lookup turns proxying on.  The one exception is a value of NULL for
"proxy", in which case proxying is not turned on and proxy-related
other fields are ignored.  Is that how it's intended to work?



Yes. It might change in some future release, but currently I'm not
planning on changing it. 'Y' would anyway be a future-safe choice.

  


Thanks for the information.  I was actually asking about the other side 
of the coin ... is it safe to assume that "proxy=NULL" will disable the 
proxying, at least for the 1.0 series?  (I saw the "proxy_maybe" for 
1.1, which is also OK for me when I get to that release.)

Re: [Dovecot] NULL for password and proxy fields

[WJCarpenter]

It's even documented (somewhere) that NULL value means the same as
if the field wasn't even selected by the query.

  


Ah, so it is, on 
http://wiki.dovecot.org/PasswordDatabase/ExtraFields>  Not sure how 
I missed that.  Thanks.

[Dovecot] calling dovecot exported auth from Java

[WJCarpenter]

As far as I have been able to figure out, dovecot auth always works over 
a Unix domain socket.  I believe it is not currently possible to operate 
dovecot auth over an Internet domain (TCP) socket.  Am I correct?


I want to call dovecot's exported authentication from a Java 
application.  Java doesn't natively know how to talk to a Unix domain 
socket, so there are inconveniences.  There are 3rd party JNI libraries 
to allow Java to do it, but I'm not too wild about the idea of using 
JNI.  My current thinking is to rig up some kind of proxy/shuttle 
arrangement between a localhost TCP port and the dovecot auth Unix 
domain socket in the filesystem.  I'm looking at using the more or less 
standard tool "socat" to do that.  ('m on a mainstream Linux distribution.)


Has anyone else confronted and solved this problem?  Do tell us the 
details  :-)


TIA!

Re: [Dovecot] calling dovecot exported auth from Java

[WJCarpenter]

On 4/15/2011 5:36 PM, WJCarpenter wrote:
I want to call dovecot's exported authentication from a Java 
application.  Java doesn't natively know how to talk to a Unix domain 
socket, so there are inconveniences.  There are 3rd party JNI 
libraries to allow Java to do it, but I'm not too wild about the idea 
of using JNI.  My current thinking is to rig up some kind of 
proxy/shuttle arrangement between a localhost TCP port and the dovecot 
auth Unix domain socket in the filesystem.  I'm looking at using the 
more or less standard tool "socat" to do that.  ('m on a mainstream 
Linux distribution.)


BTW, one workaround I thought of this situation was to connect to the 
POP3 port and do a normal POP3 login there.  That would be pretty easy, 
but it doesn't happen to work out for me.  My SQL-based login config 
factors in the service name provided by dovecot, and some of the Java 
app users might not be authorized for POP3 and/or IMAP4.  If I could 
figure out a way to run a second POP3 daemon with a different service 
name, that would probably work for me (but I don't see a way to do it 
... at least with dovecot 1.1.11 ... some some later dovecot version 
allow that?)

Re: [Dovecot] calling dovecot exported auth from Java

[WJCarpenter]

On 4/15/2011 5:36 PM, WJCarpenter wrote:
As far as I have been able to figure out, dovecot auth always works 
over a Unix domain socket.  I believe it is not currently possible to 
operate dovecot auth over an Internet domain (TCP) socket.  Am I correct?


I want to call dovecot's exported authentication from a Java 
application.  Java doesn't natively know how to talk to a Unix domain 
socket, so there are inconveniences.  There are 3rd party JNI 
libraries to allow Java to do it, but I'm not too wild about the idea 
of using JNI.  My current thinking is to rig up some kind of 
proxy/shuttle arrangement between a localhost TCP port and the dovecot 
auth Unix domain socket in the filesystem.  I'm looking at using the 
more or less standard tool "socat" to do that.  (I'm on a mainstream 
Linux distribution.)


Here is the solution we came up with.  All of this is, uh, lightly 
tested at this point.  Bug reports welcome (directly or via this mailing 
list if you want.)


First, the "socat" command to connect a localhost TCP socket to the Unix 
domain socket.  This is suitable for somehow running just once since it 
makes its own children for handling each connection instance.


/usr/bin/socat -ly \
  TCP4-LISTEN:1649,bind=localhost,reuseaddr,fork \
  UNIX-CONNECT:/var/run/dovecot/auth-client

We chose port 1649 because it's allocated to the ancient kermit 
protocol, and we're confident we'll never use that here.


Here is the Java class that we use for speaking to the dovecot auth 
process.  It's basically a rework of the C++ code in the exim sources.  
Because we have a Tomcat environment for this, we used a Catalina 
utility for base64 encoding.  If you don't have that, you'll have to 
find one elsewhere (there is no standard base64 encoder thing in Java; 
grrr).  There are some things in here that I don't completely 
understand, but they are in the exim code and (apparently) do no harm.  
You can instantiate one of these objects and then call the doLogin() 
method arbitrarily many times.  As noted, the class is not threadsafe.



package aio.util;
import java.io.*;
import java.net.Socket;

import org.apache.catalina.util.Base64;

/**
 * This class is not threadsafe.
 */
public class DovecotLogin
{
private boolean TRACE = true;
private String host;
private int port;
private String service;
private LineNumberReader lineReader;
private OutputStream outputStream;

public DovecotLogin()
{
this(null, 0, null);
}

public DovecotLogin(String host, int port, String service)
{
this.host = host != null ? host : "localhost";
this.port = port > 0 ? port : 1649;
this.service = service != null ? service : "aiologin";
if (TRACE)
{
System.out.println("CONNECT " + this.host + ":" + this.port 
+ ", service=" + this.service);

}
initDovecotConnection();
}

private void initDovecotConnection()
{
try
{
socket = new Socket(host, port);
int localport = socket.getLocalPort();
InputStream inputStream = socket.getInputStream();
lineReader = new LineNumberReader(new 
InputStreamReader(inputStream, "utf-8"));

outputStream = socket.getOutputStream();
initialDovecotListenTo();
initialDovecotSpeakTo(localport);
}
catch (Exception e)
{
// TODO logme
e.printStackTrace();
}
}

public void close()
{
if (socket != null &&  socket.isConnected())
{
try
{
socket.close();
socket = null;
}
catch (IOException e)
{
// TODO logme
e.printStackTrace();
}
}
}

public boolean doLogin(String userid, String password)
{
try
{
sayThisLogin(userid, password);
return readLoginResponse();
}
catch (Exception e)
{
// TODO logme
e.printStackTrace();
return false;
}
}

private boolean readLoginResponse() throws IOException
{
String line = lineReader.readLine();
if (TRACE)
{
System.out.println("S< " + line);
}
String[] splits = line.split("\t");
String token1 = splits[0];
if ("FAIL".equalsIgnoreCase(token1))
{
return false;
}
if ("OK".equalsIgnoreCase(token1))
{
return true;
}
throw new IOException("unexpected response received from 
dovecot auth: " + line);

}

private void initialDovecotListenTo() throws IOExcepti

Re: [Dovecot] Want to have some users with Maildir, some with mbox

[WJCarpenter]

BTW. Are you also using a script that preserves IMAP UIDs? Otherwise 
IMAP clients will re-download all the mail.


Does the "convert" plugin preserve the UIDs?  The wiki implies that it 
doesn't, but I thought maybe since it's implemented as a plugin that 
maybe it does (or that maybe the caution is about converting to/from 
dovecot and some other server).

Re: [Dovecot] Want to have some users with Maildir, some with mbox

[WJCarpenter]

BTW. Are you also using a script that preserves IMAP UIDs? Otherwise IMAP 
clients will re-download all the mail.
   

Does the "convert" plugin preserve the UIDs?  The wiki implies that it doesn't, 
but I thought maybe since it's implemented as a plugin that maybe it does (or that maybe 
the caution is about converting to/from dovecot and some other server).
 

It doesn't. In v2.0 dsync does. In v2.0 convert plugin has been removed.



So, what is the answer for pre-2.0 folks who want to do mbox->Maildir 
conversion?  I have not yet found a script that claims to preserve 
UIDs.  Is there one that people use?

Re: [Dovecot] Want to have some users with Maildir, some with mbox

[WJCarpenter]

http://wiki.dovecot.org/Migration/MailFormat shows a bunch of scripts, one of 
them claims to preserve UIDs.
   


Does anyone here have direct successful experience with this?  I only 
asked about this because a comment on that same page says, "None of the 
solutions described below preserve the message UIDs."


(Some of my users have > 1 GB of email, so any UID disruption would 
likely be seriously annoying to them.)

Re: [Dovecot] Want to have some users with Maildir, some with mbox

[WJCarpenter]

On 1/29/2010 4:11 PM, WJCarpenter wrote:


http://wiki.dovecot.org/Migration/MailFormat shows a bunch of 
scripts, one of them claims to preserve UIDs.


Does anyone here have direct successful experience with this?  I only 
asked about this because a comment on that same page says, "None of 
the solutions described below preserve the message UIDs."


Answering my own question.  I did some experiments last night, and 
JulianFitzell's patched version of mb2md.pl, along with his 
migrateuser.sh script, looks like it works as advertised.  (Local tweaks 
to the migrate scripts, but that's fair enough.)   Thanks, Julian!

[Dovecot] sha-512 ... shadow blended with database

[WJCarpenter]

The cryptic subject is the outcome of my looking into how to do a 
particular thing.  I wonder if anyone else has solved this problem in a 
way that hasn't occurred to me.


I'm using dovecot 1.1.11 on Ubuntu Server 9.10.  I could consider 
upgrading to my own install of a newer dovecot if it made a difference 
to this problem.


I have two populations of dovecot users.  Some users have Unix accounts 
(with logins disabled), and so their password hashes are stored in 
/etc/shadow.  These days, the default configuration for that is salted 
SHA-512.  It's easy for me to change that scheme to something else if I 
want to, but the important fact is that I already have some users with 
passwords in salted SHA-512.  The other population of users is purely 
virtual, and their password hashes are stored in a MySQL database in 
SHA-1 format (unsalted, but moving to salted wouldn't be a big deal).  
The database also has a column identifying the hash scheme, so SHA-1 
isn't some assumption.


I know that I have have multiple passdb in my dovecot config, but I'm 
looking to unify my two user populations and put them all in the MySQL 
database.  As far as I can tell from the wiki, there is no SHA-512 in 
any version of dovecot.  MySQL also doesn't have SHA-512.  So, I don't 
see a way of reworking my password checking to accomodate the salted 
SHA-512 values currently in /etc/shadow.  I'd prefer to not ask the 
SHA-512 users to update their passwords for this administrative reason 
if I can avoid it (but so far, that looks like the only answer).


Any ideas?

Re: [Dovecot] sha-512 ... shadow blended with database

[WJCarpenter]

I have two populations of dovecot users.  Some users have Unix accounts (with 
logins disabled), and so their password hashes are stored in /etc/shadow.  
These days, the default configuration for that is salted SHA-512.  It's easy 
for me to change that scheme to something else if I want to, but the important 
fact is that I already have some users with passwords in salted SHA-512.  The 
other population of users is purely virtual, and their password hashes are 
stored in a MySQL database in SHA-1 format (unsalted, but moving to salted 
wouldn't be a big deal).  The database also has a column identifying the hash 
scheme, so SHA-1 isn't some assumption.
 

By salted SHA-512 do you mean the $6$salt$sha format that glibc uses? If so, 
you can use CRYPT scheme, which causes Dovecot to use crypt() function. Then 
assuming you're using new enough glibc, it understands it.
   



Ah, I didn't realize that.  Yes, that's exactly what I meant, and the 
glibc will be on an identically-configured system to the one where the 
$6$ shadow entries were created.  Thanks for the info.

Re: [Dovecot] Want to have some users with Maildir, some with mbox

[WJCarpenter]

http://wiki.dovecot.org/Migration/MailFormat shows a bunch of 
scripts, one of them claims to preserve UIDs.


Does anyone here have direct successful experience with this?  I only 
asked about this because a comment on that same page says, "None of 
the solutions described below preserve the message UIDs."


Answering my own question.  I did some experiments last night, and 
JulianFitzell's patched version of mb2md.pl, along with his 
migrateuser.sh script, looks like it works as advertised.  (Local 
tweaks to the migrate scripts, but that's fair enough.)   Thanks, Julian!



Commenting on my answer to my own question... :-)

For one MBOX folder for one user, total of over 5000 messages, a couple 
of dozen messages got a warning to the effect of "can't parse date for 
this message".  Although I imagined that the resulting MAILDIR messages 
would have bogus dates, it turned out that they had no headers at all.  
The message files contained only the message bodies.  (The user 
complained, but I had him whacked so word wouldn't get out.  :-)

Re: [Dovecot] OT: best linux imap client for dovecot

[WJCarpenter]

Example:
* I have 100's of sent-mail mailboxes I don't want to be subscribed 
to, because it is doubtful I will ever use them. These mailboxes are 
unsubscribed because I don't want to see them in any mailbox listings 
by default.


This use of subscriptions is a terrible abuse of IMAP. Like most 
terrible abuses, it's a-ok to choose for yourself if you're an 
advanced user, but anyone who has done support for a broad user base 
knows that a client should *NEVER* act like this as the default. 
Subscriptions are brittle and non-portable and hiding mailboxes based 
on them leads only to floods of "Where is all my mail you screwed up 
my life" interactions.


I'm genuinely confused by this come-back.  Could you elaborate?

Why is having subscriptions (and, specifically, some folders to which 
you are not subscribed) a terrible abuse of IMAP?  What is non-portable 
about subscriptions?  The IMAP protocol supports them directly.

Re: [Dovecot] OT: best linux imap client for dovecot

[WJCarpenter]

Subscriptions themselves aren't an abuse of IMAP, obviously, as they 
are in the spec. A client that *by default* uses them to hide folders 
is abusing them, for exactly the reasons I explained. They are 
non-portable because:


I agree 100% that hiding folders by default is bad, but I've never seen 
a client do that.  I have seen many clients hide folders that are not 
subscribed, and that's exactly the behavior that I want.  (If you don't 
have a subscription list, you see everything.  If you have a 
subscription list, that's what you see.)



1. The interaction that most (all?) clients poorly bake in between 
subscriptions and the "IMAP root" setting means that if your various 
clients are not configured identically, you'll see one set of folders 
in one place and another set in another. This contributes to users 
thinking mail has disappeared to creating mailboxes with the same name 
at different paths. The latter is annoying to begin with, but becomes 
especially bad when yet another client shows the user both of two 
same-name folders and *resolves them in the interface to the same 
directory*, so the user thinks they are simple duplicates and deletes 
one.


2. Different clients interpolate names differently, such that even if 
two clients are identically configured when it comes to the "IMAP 
root" and  namespaces, they map the subscription to inconsistent paths 
(either on the backend or the in the interface). This is especially 
true of moving between Thunderbird and certain versions of Outlook and 
Vista Mail.


Are these problems with subscriptions or folder handling in general?  In 
every client I've looked at, a subscription is just a visibility marker 
for a folder.  When the folder is visible, it shows up in whatever place 
the client would put it if there were no such thing as subscriptions.  
Are you saying there are clients that make these UI problem just for 
subscribed folders?


Or are you maybe talking about subscriptions to public/shared folders?  
If yes, never mind.  I don't have any experience with client behavior 
for those.

Re: [Dovecot] Account lockout option?

[WJCarpenter]

Is there any option available for me to help inhibit/prevent 
brute-force login attempts?


I (and many others) use fail2ban.  It works outside of dovecot, et al, 
by tailing your log files.  When it finds a configurable number of 
failed attempts in a configurable time window, it blocks the remote IP 
address for a configurable amount of time.  It can protect you against a 
lot more than failed email login attempts.  I'm quite happy with it.  I 
typically turn back several brute force SSH login attempts every day.  I 
also have it watching my dovecot logins, but so far don't get many 
attempts there.

Re: [Dovecot] SIS Implementation

[WJCarpenter]

Step 4) Figure out if base64-encoded attachments can be decoded in a way
that allows re-encoding them back to the exact original encoding. If so,
save the attachment decoded and add the necessary encoding info the dbox
metadata.
  


Although you might like to do that for some sort of tidiness or 
whatever, I don't think there's an actual requirement to restore it to 
base64 (or q-p or whatever).  Those are just transfer encodings, and 
intermediate MTAs might in theory have transformed them (though in 
practice that's doubtful).  The one use case for an exact reconstruction 
might be some older digital signature schemes (which were not so robust 
and probably lost if an intermediate MTA messed with things).




Or perhaps just store them compressed. How much of a difference is there
between storing decoded data vs. compressed base64 encoded data?
  


The best you can ever do is break even, and often you will lose.  In any 
case, you have the CPU/memory cost of the compress/decompress.  I would 
get rid of any transfer encoding and try to compress.  If the 
compression was above some threshold of storage benefit, store it 
compressed.  Otherwise, store it uncompressed.

Re: [Dovecot] SIS Implementation

[WJCarpenter]

I was thinking things like: upper vs. lowercase characters, different 
line wrapping lengths, possibly some other weird stuff.. I'd think 
that all digital signatures break if any of those change? Or do they 
really parse the headers and do calculate the signatures using the 
decoded base64?


Yes, you will have to perfectly preserve whatever is inside the base64 
or q-p, but that's a different matter from needing to preserve the 
base64 or q-p itself.  base64 and q-p are just schemes for safely 
transporting the message since there is some mild danger of losing the 
8th bit.


These days, standardized digitial signature schemes take into account 
legal transformations that can happen during message transmission.  Most 
of them have a canonicalization formula so that things still work.  
However, in early days, various schemes didn't take that into account.  
Luckily, MTAs typically didn't rearrange anything even if they were 
legally allowed to. 

So, I think you should regard all MIME parts as binary (after decoding 
any base64, q-p, or whatever).  If some of them happen to contain plain 
text, so what?  Just perfectly preserve every bit, possibly with 
lossless compression for storage, and everything should work.  (Because 
the SMTP spec has the ridiculous requirement that mail be transmitted 
with CRLF line endings, some mail systems do line-ending conversion to 
the local convention.  That's a nightmare; best to avoid it and just 
store everything as binary.)


Another issue is that the MIME structure (MIME part sizes, offsets) 
must match what got saved into dovecot's cache file, but that could be 
fixed with some extra code.




Right.  I assumed that that area of code would need a lot of touching 
anyhow.  If you take my advice and basically discard the base64/q-p 
encoding, you also can't depend on the MIME boundary being unambiguous 
any more.  But if you're reassembling things on the fly from an SIS 
store, you can generate new MIME boundaries if you need them.  All that 
stuff is just wrapping paper.  (Of course, you should check the MIME 
specs to see what you can officially do, but I'm pretty sure most of the 
things that are interesting to do were anticipated.  Even if not, the 
MIME specs only cover message transmission.  You can do whatever you 
want in your local store.)

Re: master: service(imap): child xxxx killed with signal 11

[WJCarpenter]

Thank you, Lukas Matasovsky, for isolating this. I've had dovecot pinned 
to the .16 version since I started getting that crash in the .17 
version. But I've been too slack to do more than scan the mailing list 
for fixes. Your investigation provided the solution that also worked for 
me. Once again, my laziness pays off. Thanks again.


On 1/2/22 6:07 AM, Lukas Matasovsky wrote:
I have isolated the issue: after removing fts_squat dovecot seems to 
be back to normal.
See also: https://doc.dovecot.org/configuration_manual/fts/squat/ 

Re: maildirsize not correct

[WJCarpenter]

A long shot, but here's how I experienced similar symptoms:
http://www.dovecot.org/list/dovecot/2016-April/104091.html


Kamil Madac wrote on 08/23/2016 02:02 PM:

Hi,

One of my email accounts has 1,5gb of emails in INBOX, but maildirsize
shows only 528mb. 'du' also shows 1,5gb and there are 3809 files in cur
directory.

When I use  mailbox status a can see correct values:

doveadm -f table mailbox status -u u...@domain.sk "messages vsize" INBOX*
  messages
vsize

INBOX   3809 1521049349

but when I check the quotas with doveadm:

doveadm quota get -u u...@domain.sk
Quota name Type Value
Limit
%
User quota STORAGE 541391
2048000
26
User quota MESSAGE886
-
0

I tried to delete maildirsize, but it was recalculated again incorrectly.

2097152000S
554384829 886

Dovecot version is 2.2.9 installed from packages on Ubuntu 14.04
I have other accounts on server which are have no problems with maildirsize
and quotas. Does anyone have same experience?

Kamil

MFA 2FA TOTP razz-ma-tazz!

[WJCarpenter]

I'd like to start offering my server's users multi-factor 
authentication. Right now, I funnel all authentication through dovecot. 
Before I get too far down the fantasy design path, I'm wondering if 
anyone else has already done this and could share some details or code. 
(I loaded up the subject line with acronyms to show how serious I am. :-))


I am specifically thinking of two-factor authentication using TOTP 
(time-based one-time passwords) as described in RFC-6238. Those are the 
ones compatible with Google Authenticator and compatible apps. I already 
am a user of those at several sites. Some of them don't have a separate 
opportunity to enter the 6-digit code. Instead, you append the 6-digit 
code to your normal password. If your config on the site shows you as a 
user of TOTP, they peel those trailing 6 digits off your password and 
then validate the rest of the password in the normal way. That is what I 
think I would do for dovecot authentication.


So, who's already done this or something like it?

Re: JMAP support in Dovecot

[WJCarpenter]

I don't know the answer to that question, but I am curious about 
something. What client are you thinking about using with JMAP? I haven't 
found much. (And much of the demo stuff at jmap.io seems to be busted in 
various ways.)


Andrew Jones wrote on 11/26/2016 10:43 AM:

Hi there,

I understand there were discussions to try and develop JMAP support for Dovecot.

Is this still in the pipeline for Dovecot 2.5?

Regards
Andrew

Sent from my iPhone

Re: JMAP support in Dovecot

[WJCarpenter]

Bron Gondwana wrote on 11/27/2016 02:25 PM:
The demo proxy was a pretty quick hack and isn't very efficient, but 
it should be working. It does a fairly slow background import for 
existing accounts, so I'd recommend using small tests accounts. What 
particular bit is broken for you? (apart from the known not-working 
authentication flow) Bron. 
I don't know what you mean by the "known not working authentication 
flow", so possibly that is what I am hitting.


On the "Set Hosts" screen with either Chrome or FireFox, I see two boxes 
each for IMAP and SMTP and a drop-down to toggle plaintext, SSL, or 
STARTLS. The fields are populated initially with "IMAP: 993/1/SSL" and 
"SMTP: 587/1/STARTLS". I don't know what that "1" is supposed to mean, 
so I replaced it with the relevant server names. I got an error "UNABLE 
TO CONNECT for ". To try again, I swapped 
those two fields around. Same result. (I was watching my server logs at 
the same time and saw no connection attempts.) I could not figure out 
how to go further.

Re: Maildirsize not updated

[WJCarpenter]

Who delievers incoming mail, dovecot LDA or something else?

This is what caused a similar problem for me: 
https://dovecot.org/list/dovecot/2016-April/104091.html

expunging messages doesn't update maildirsize

[WJCarpenter]

This might be something that everybody thinks of as a well-known fact, 
but I've been searching around quite a bit without finding anything 
definitive. Maybe I just have the wrong idea of what's supposed to be 
going on.


I'm using maildir+ quotas. For arriving mail, it works as I expect: new 
lines are appended to maildirsize to reflect the size of the new 
messages. But I also expected that mail that is deleted (and expunged, 
yes totally gone from the disk drive, verified from the command line) 
would lead to some kind of update to maildirsize or maybe even a 
complete recalculation. But that does not happen on my system (dovecot 
2.2.9 on a Ubuntu variant).


Now, I know that I can run "doveadm" from a cron job or whatever to have 
the maildirsize info recomputed. I also know that the maildirsize file 
will be recreated when it gets larger than 4k. That's all fine. But I 
don't see why I should have to have this out-of-band cron job, and I 
don't see why my users should have to wait if their maildirsize file 
doesn't tip over the magical 4k mark (which is a lot of individual 
messages being recorded).


Is it by design that dovecot doesn't update maildirsize immediately when 
messages are expunged? If yes, why?

Re: expunging messages doesn't update maildirsize

[WJCarpenter]

On 04/19/2016 07:17 PM, WJCarpenter wrote:
Is it by design that dovecot doesn't update maildirsize immediately 
when messages are expunged? If yes, why?


Replying to my own question since I figured it out. Local config glitch, 
not a dovecot bug or design fault.


I should have mentioned that I don't use dovecot's local delivery agent. 
Updates to maildirsize for newly arriving messages is done by exim.


Boring details follow. They become interesting details if you are having 
the same problem. :-)


I use the SQL backend for keeping track of user data, including quota 
values. Consequently, I have configured 3 very similar (but not 
identical) SQL queries for that user data. One is in my exim config, and 
two (password_query and user_query) are in my dovecot-sql.conf.


Those of you familiar with dovecot's SQL queries will know that the 
password_query can also return values that would ordinarily be looked up 
via user_query, the motivation being that user_query can often be 
skipped completely. The dovecot 2.2.9 release broke that userdb prefetch 
feature (it was fixed soon after, but I am using my distribution's 
dovecot version [Ubuntu 14.04], which is stuck at 2.2.9). Even though 
userdb prefetch doesn't work and both passdb and userdb queries are both 
done every time, my user_query is just a clone of my password_query in 
anticipation of the day when I can get rid of user_query.


It's the cloning of the query where things went wrong. In the 
password_query, the prefetched user quota value is returned as 
"userdb_quota". In the user_query, it's returned as "quota". When I 
cloned the query, I neglected to change "userdb_quota" to "quota" in my 
select list. (That's a bummer, because I did correctly change 
"userdb_home" to "home" in my select list, so I must've known I should 
do it.)


The net is that dovecot did the password_query and got the correct value 
for the user quota. It then ignored that (due to the prefetch bug) and 
did the user_query. The user_query reported the user quota with the 
wrong field name, and dovecot figured there was no quota for that user 
(which means /every/ user).


Once I fixed my 7-character configuration mistake, it started working 
correctly.


For the sake of brevity, I've left out the parts where I went down a 
bunch of blind alleys before finding the correct solution. :-)

Re: [Dovecot] Dovecot MTA

[WJCarpenter]

On 11/08/2013 07:43 AM, Charles Marcus wrote:

On 2013-11-08 10:22 AM, Timo Sirainen  wrote:
Ah, I had actually been mostly just thinking about inbound SMTP 
features.


Hmmm well, I'd hate to see this turn into a huge time-sink for 
you. The fact is, postfix's maturity combined with its new postscreen 
capabilities will make it a very, very hard sell to postfix shops - 
especially the larger ones that rely heavily on postfix's ability to 
filter out the crap with as few resources as possible - and postscreen 
just increased that already powerful capability by at least one or two 
orders of magnitude.


Likewise, exim users aren't likely to be sold. Most of things originally 
listed are either standard or easily accomplished in exim.


I would probably be pretty skeptical and uninterested in a Dovecot MTA. 
No offense. I think you should look at other existing MTAs besides 
postfix before concluding there is a hole that needs filling.


(You know the old Internet saying ... all programs evolve until they can 
send email. I guess you are getting there. :-)

Re: [Dovecot] Dovecot MTA

[WJCarpenter]

On 11/08/2013 10:46 AM, Charles Marcus wrote:

On 2013-11-08 1:35 PM, WJCarpenter  wrote:
I would probably be pretty skeptical and uninterested in a Dovecot 
MTA. No offense. I think you should look at other existing MTAs 
besides postfix before concluding there is a hole that needs filling. 


The one exception to this though, is if it was simply implemented as a 
proxy, but used for internal/local only emails - so, no need to 
involve postfix or Exim for mail submitted to the dovecot_mta if the 
recipient is destined for the dovecot_LDA...


The more I think about it the more I even like this idea...



Well, that's sort of true for operational efficiency kinds of things, 
but do you really one to configure one program for internal mail and 
another for externally sent/received mail? I don't, but tastes vary. :-)

Re: [Dovecot] Dove LDA vs Exim LDA

[WJCarpenter]

The main reason I switched was so that I could move from procmail to 
seive,


I am curious if you investigated Exim's sieve script support. Was there 
some problem you saw with it, or you just didn't look into it?