Re: [Dovecot] Backing Up

[Daniel L. Miller]

Stewart Dean wrote:

Dave McGuire wrote:

On Oct 29, 2008, at 3:42 PM, Scott Silva wrote:

What is the best way to do a (server-side) backup of all mail in a
user's mail?

I usually just rsync the /home directories to another server. The 
inital sync
can take a while, but it gets faster after there is a base to work 
from.


  ...and it's much less painful if you're using maildir instead of mbox!

   -Dave

I have to wonder.  I have a mailserver that I do a bootable complete 
image copy of with all files and O/S in two hours to an Ultrium-2 
tape, 95 GB.  When I switch to maildir, I will go from some 25,000 
mbox files to 2.5 to 3 million files...I can't believe that isn't 
going to hurt and will force me into incrementals.
Well, I can't talk about an installation of that size - but I CAN say 
I've been using an rsync based backup and been thrilled with the 
results.  I'm supporting a measly dozen users, but I'm incrementally 
backing up our complete server on a nightly basis.  I think the script 
now takes less than an hour to run - for a complete backup.


Since I'm doing this via a VPN to a server at my house, it makes it 
rather convenient.  I love having an immediately available backup I can 
ssh to - that now represents a daily snapshot for over a year - so when 
one of my users says, "Um...if I deleted a file that I've been working 
on for the past two weeks...can you get it back?"


--
Daniel

Re: [Dovecot] Backing Up

[mikkel]

> [EMAIL PROTECTED] wrote:
>> Just imagine backing the thing up, exporting 60.000.000 SQL queries.
>> Not to say importing them again if something should go really wrong.
>> Actually I'n not even sure it would be faster. When the index files grow
>> to several gigabytes they kind of loose their purpose.
>
> There are many businesses backing up way-more data than that and it it
> isn't 60,000,000 queries -- it is one command.  But if you use serious
> hardware "backing up" isn't really needed.  RAID, redundant/hot-swap
> servers, etc. make backing up /extra redundancy/.  :-)
>

Why make things complicated and expensive when you can make them cheap and
simple?
Anything is possible if you wanna pay for it (in terms of hardware,
administration and licenses).
I have focused primarily on making it as simple as possible.

And while running a 400 GB with 60.000.000 records database isn't
impossible it would be if it were to run on the same hardware that now
comprises the system.
Roughly 1000 IOPS is plenty to handle all mail operations.

I seriously doubt that it would be enough to even supply one lookup a
second on that huge db (and even less over NFS as is now being used).
And I assume that a hundreds of lookups a second would be required to
handle the load.

So it would require a lot more resources and still give nothing but
trouble (risk of crashed database and backup issues that now aren't
there).

By the way data is stored in a SAN it needs to be backed up.
500 GB SATA disks takes a day to synchronize if one breaks down and we
can't really take that chance (Yes I will eventually move the data to
smaller 15.000 RPM disks but there is no need to pay for them before its
necessary). Also there is the risk of data being deleted by a mistake,
hacker attacks or software malfunctioning.

But we really are moving off-topic here.

Regards, Mikkel

Re: [Dovecot] Startup info message

[Athan Dimoy]

On Thu, 30 Oct 2008 21:59:20 +0200, Timo Sirainen <[EMAIL PROTECTED]> wrote:

Hmm. After a successful imap/pop3 login, you should have /var/lib/
dovecot/auth-success file. Do you have it?


Dovecot always displays it when it starts, even after /var/lib/
dovecot being deleted.


That's right, because then it doesn't have the auth-success file.



You're right, message is not displayed after auth-success initially being  
created.


PS. Could you please consider adding a couple of lines in the wiki docs,  
describing the exact purpose of this message.


Thanks Timo!

Athan

Re: [Dovecot] Problem with Dovecot 1.1.4 and dovecot-antispam 20080829

[Tom Hendrikx]

HILT Guillaume wrote:
> On Thu, 30 Oct 2008 10:51:43 +0100, Guillaume Hilt
> <[EMAIL PROTECTED]> wrote:
>> Hi there,
>>
>> I'm running a server under Gentoo 64 with postfix 2.5.5, Dovecot 
>> 1.1.4-r1 and Dovecot-antispam 20080829 (all are the lastest versions 
>> availables under Portage).
>> I ran into a problem when I log in to my mailbox :
>>
>> Oct 30 10:39:23 ks29138 dovecot: IMAP(hiltg): Loading modules from 
>> directory: /usr/lib64/dovecot/imap
>> Oct 30 10:39:23 ks29138 dovecot: IMAP(hiltg): Module is for different 
>> version 1.1.1: /usr/lib64/dovecot/imap/lib90_antispam_plugin.so
>> Oct 30 10:39:23 ks29138 dovecot: Fatal: IMAP(hiltg): Couldn't load 
>> required plugins
>>
>> Is there another version for Dovecot 1.1.4 out there or must I go back 
>> to 1.1.1 ?
>>
>> Thanks,
>>
>> Guillaume
>>
> 
> Compiling the head snapshot fixed the problem, sounds like portage needs a
> new ebuild for this plugin.


Acutally, the plugin needs to be compiled against the version of dovecot
you're running. So a reinstall of the available version in portage
*after* emerging a new dovecot version would suffice too

-- 
Regards,
Tom



signature.asc
Description: OpenPGP digital signature

Re: [Dovecot] Backing Up

[Roderick A. Anderson]

[EMAIL PROTECTED] wrote:

On Oct 30, 2008, at 2:35 PM, [EMAIL PROTECTED] wrote:

Maildir is nice compared to mbox but it really isn’t optimal. In days
where IOPS is the most difficult resource to get into your server (and
dovecot already using close to nothing in terms of CPU time and
memory)
having one file per e-mail is less than sub-optimal especially when a
large amount of users just downloads the whole mailbox using POP3
(not to
mention backing up Maildirs).

   It seems to me that a database like Postgres or MySQL would be the
best bet.



That's a matter of opinion. Moving mail storage to a database would
probably be the last thing I would ever do (I'm not saying it's not the
right thing for some people. I'm just not one of them).
I'm using mysql for storing the users database but that’s another story.



Adding a database is one additional level of complexity. One more program
to govern. In my opinion it's nice to know that as long as the disk is
readable nothing can go completely wrong.



I have to jump in here and go a bit tangential by saying there are 
databases and want-to-be's.



The database in my case would be roughly 400 GB holding some 60 million
records.


Fair sized but not really big.


Just imagine if one single byte got written to the wrong place. Power
outage, OS crash, software bug or whatever could easily result in this (I
regularly experience mysql tables that crash on their own from heavy use).
Having to run a repair on a table of that size whilst all users are eager
to get to their data must be a nightmare of proportions.


There is the difference between an enterprise database and MySQL.  Yes, 
yes, yes lots of /enterprises/ run applications that use MySQL but most 
of those apps have throw away data or they are not using the free 
version of MySQL.



Just imagine backing the thing up, exporting 60.000.000 SQL queries.
Not to say importing them again if something should go really wrong.
Actually I'n not even sure it would be faster. When the index files grow
to several gigabytes they kind of loose their purpose.


There are many businesses backing up way-more data than that and it it 
isn't 60,000,000 queries -- it is one command.  But if you use serious 
hardware "backing up" isn't really needed.  RAID, redundant/hot-swap 
servers, etc. make backing up /extra redundancy/.  :-)


And I bring this up because  "Archiveopteryx" 
 uses a database - PostgreSQL.



Rod
--



Maildir is very resilient to various errors. It is virtually impossible to
corrupt a maildir (at least I've never experienced anything).
Also you can backup up the thing without worrying about anything accessing
it at the same time.
Mbox less so but still a lot better than having one huge database.

Dbox would be the ultimate compromise between crash resilience and a low
number of files (not to mention the enormous potential for speed gains).

Regards, Mikkel

Re: [Dovecot] Startup info message

[Timo Sirainen]

On Oct 30, 2008, at 9:50 PM, Athan Dimoy wrote:


Thank you both for the explanation.

@Timo: this message doesn't go way even after a successful login;


Hmm. After a successful imap/pop3 login, you should have /var/lib/ 
dovecot/auth-success file. Do you have it?


Dovecot always displays it when it starts, even after /var/lib/ 
dovecot being deleted.


That's right, because then it doesn't have the auth-success file.



PGP.sig
Description: This is a digitally signed message part

Re: [Dovecot] Startup info message

[Athan Dimoy]

Thank you both for the explanation.

@Timo: this message doesn't go way even after a successful login; Dovecot  
always displays it when it starts, even after /var/lib/dovecot being  
deleted. I count it as a bug although a minor one.


Anyway, not something serious, as Dovecot works really great.

Athan

Re: [Dovecot] SIGBART/SIGSEGV while SELECTing virtual folder

[Lars Noschinski]

* Lars Noschinski <[EMAIL PROTECTED]> [08-10-30 20:29]:

* Lars Noschinski <[EMAIL PROTECTED]> [08-10-30 20:13]:

And another one to go (using latest hg tip, changeset 8360:7c615ac48711)


Forgot the dovecot-virtual file:

*
mbox/*
   all

though it also occurs using only "*". And dovecot -n output:


While trying to produce a smaller testcase, I changed the
dovecot-virtual file to

mbox/*
 all

and the problem went away. It did not reoccur after reverting the
change. After this, I also could not reproduce the problem on a copy of
the Mailbox, which I produced using cp -rl. Probably I should not have
used hardlinks there.

So this sounds like a chaching-related problem to me?  Hm, I remember I
renamed the virtual folder yesterday (in the filesystem, using mv).

Re: [Dovecot] Backing Up

[Ed W]

Scott Silva wrote:
> Rsync will use more memory on large filesystems, but it is usually lighter in
> CPU, network, and IO time. But tar gives you multiple backups. To achieve that
> with rsync you need the rbackup script or rsnapshot.
>
>   


Also check snapback2 (similar to tools you mentioned above)

And brackup looks quite interesting for backing up maildir... (same chap
who wrote memcached)

Ed W

Re: [Dovecot] Backing Up

[mikkel]

> On Oct 30, 2008, at 2:35 PM, [EMAIL PROTECTED] wrote:
>> Maildir is nice compared to mbox but it really isn’t optimal. In days
>> where IOPS is the most difficult resource to get into your server (and
>> dovecot already using close to nothing in terms of CPU time and
>> memory)
>> having one file per e-mail is less than sub-optimal especially when a
>> large amount of users just downloads the whole mailbox using POP3
>> (not to
>> mention backing up Maildirs).
>
>It seems to me that a database like Postgres or MySQL would be the
> best bet.
>

That's a matter of opinion. Moving mail storage to a database would
probably be the last thing I would ever do (I'm not saying it's not the
right thing for some people. I'm just not one of them).
I'm using mysql for storing the users database but that’s another story.

Adding a database is one additional level of complexity. One more program
to govern. In my opinion it's nice to know that as long as the disk is
readable nothing can go completely wrong.

The database in my case would be roughly 400 GB holding some 60 million
records.
Just imagine if one single byte got written to the wrong place. Power
outage, OS crash, software bug or whatever could easily result in this (I
regularly experience mysql tables that crash on their own from heavy use).
Having to run a repair on a table of that size whilst all users are eager
to get to their data must be a nightmare of proportions.

Just imagine backing the thing up, exporting 60.000.000 SQL queries.
Not to say importing them again if something should go really wrong.
Actually I'n not even sure it would be faster. When the index files grow
to several gigabytes they kind of loose their purpose.


Maildir is very resilient to various errors. It is virtually impossible to
corrupt a maildir (at least I've never experienced anything).
Also you can backup up the thing without worrying about anything accessing
it at the same time.
Mbox less so but still a lot better than having one huge database.

Dbox would be the ultimate compromise between crash resilience and a low
number of files (not to mention the enormous potential for speed gains).

Regards, Mikkel

Re: [Dovecot] SIGBART/SIGSEGV while SELECTing virtual folder

[Lars Noschinski]

* Lars Noschinski <[EMAIL PROTECTED]> [08-10-30 20:13]:

And another one to go (using latest hg tip, changeset 8360:7c615ac48711)


Forgot the dovecot-virtual file:

*
mbox/*
all

though it also occurs using only "*". And dovecot -n output:

# 1.2.alpha3: /tmp/dd/etc/dovecot.conf
# OS: Linux 2.6.26-rc9-00114-gf2260c5-dirty i686 Debian lenny/sid 
listen: 127.0.0.1

ssl_disable: yes
login_dir: /tmp/dd/var/run/dovecot/login
login_executable: /tmp/dd/libexec/dovecot/imap-login
mail_plugins: fts fts_squat virtual zlib
namespace:
  type: private
  separator: /
  location: maildir:~/Mailbox/Maildir:LAYOUT=fs
  inbox: yes
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: mbox/
  location: mbox:~/Mailbox/mbox/
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: spam/
  location: mbox:~/Mailbox/spam/
  list: yes
  subscriptions: yes
namespace:
  type: private
  separator: /
  prefix: virtual/
  location: virtual:~/Mailbox/virtual
  list: yes
  subscriptions: yes
auth default:
  passdb:
driver: pam
  userdb:
driver: passwd


Greetings, Lars.

Re: [Dovecot] dovecot -n - provide sys info too? - WAS: Re: Dovecot read only for users

[Timo Sirainen]

On Oct 30, 2008, at 9:02 PM, John Lightsey wrote:

A little late, but I don't see any mention of /etc/lsb-release in  
the LSB specification.  You probably want the output of /usr/bin/ 
lsb_release -d


I don't think dovecot should execute external binaries. Sounds scary.



PGP.sig
Description: This is a digitally signed message part

Re: [Dovecot] SIGBART/SIGSEGV while SELECTing virtual folder

[Lars Noschinski]

* Lars Noschinski <[EMAIL PROTECTED]> [08-10-26 15:31]:

* Timo Sirainen <[EMAIL PROTECTED]> [08-10-26 15:28]:

On Sun, 2008-10-26 at 11:49 +0100, Lars Noschinski wrote:
#6  0xb7fbdca3 in virtual_mail_get_header_stream (mail=0x9b5c038, 
headers=0x9885050, stream_r=0x920c7cc) at virtual-mail.c:252

backend_headers = (struct mailbox_header_lookup_ctx *) 0x99823e0
ret = 


Whops, I almost fixed this the last time but left one important part
out. Fixed now. :)


Yeah, looks good now. Thank you very much!


And another one to go (using latest hg tip, changeset 8360:7c615ac48711)

--
% ./sbin/dovecot --exec-mail ext /usr/bin/gdb ./libexec/dovecot/imap
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i486-linux-gnu"...
(gdb) run
Starting program: /tmp/dd/libexec/dovecot/imap 
* PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT THREAD=REFERENCES MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH SEARCHRES WITHIN CONTEXT=SEARCH] Logged in as lars

. SELECT virtual/alle

Program received signal SIGABRT, Aborted.
0xb7dbd556 in raise () from /lib/libc.so.6
(gdb) bt full
#0  0xb7dbd556 in raise () from /lib/libc.so.6
No symbol table info available.
#1  0xb7dbed78 in abort () from /lib/libc.so.6
No symbol table info available.
#2  0x080eaf15 in default_fatal_finish (type=, status=0) 
at failures.c:150
backtrace = 0xa120f60 "/tmp/dd/libexec/dovecot/imap [0x80eaf01] -> 
/tmp/dd/libexec/dovecot/imap(i_syslog_fatal_handler+0x34) [0x80eafd4] -> 
/tmp/dd/libexec/dovecot/imap [0x80ea919] -> /tmp/dd/lib/dovecot/imap/lib20_virtual_"...
#3  0x080eafd4 in i_syslog_fatal_handler (type=LOG_TYPE_PANIC, status=0, 
fmt=0xb7ee38af "UID lost unexpectedly", args=0xbfe09004 "\001") at failures.c:308

No locals.
#4  0x080ea919 in i_panic (format=0xb7ee38af "UID lost unexpectedly") at 
failures.c:197
No locals.
#5  0xb7ee075e in virtual_sync_external_flags (ctx=0x9f130a8, bbox=0x977ea20, vseq=35152, 
real_uid=1) at virtual-sync.c:66

flags = 
kw_names = 
keywords = 
#6  0xb7ee230a in virtual_sync_backend_boxes (ctx=0x9f130a8) at 
virtual-sync.c:977
i = 83
#7  0xb7ee28cf in virtual_storage_sync_init (box=0x977e5e0, flags=65) at 
virtual-sync.c:1199
ret = 
#8  0x080b03b0 in mailbox_sync (box=0x17a8, flags=65, status_items=239, 
status_r=0xbfe093c8)
at mail-storage.c:523
ctx = (struct mailbox_sync_context *) 0x0
#9  0x08064ca8 in cmd_select_full (cmd=0x9775e60, readonly=false) at 
cmd-select.c:273
client = (struct client *) 0x9775be0
box = (struct mailbox *) 0xbfe09438
ctx = (struct imap_select_context *) 0x9775ea8
args = (const struct imap_arg *) 0x977aee0
mailbox = 0x977af68 "alle"
ret = 
__PRETTY_FUNCTION__ = "cmd_select_full"
--
Oct 30 20:09:18 vertikal EXT(lars): : Panic: UID lost unexpectedly
Oct 30 20:09:18 vertikal EXT(lars): : Raw backtrace: /tmp/dd/libexec/dovecot/imap [0x80eaf01] -> /tmp/dd/libexec/dovecot/imap(i_syslog_fatal_handler+0x34) [0x80eafd4] -> /tmp/dd/libexec/dovecot/imap [0x80ea919] -> /tmp/dd/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7ee075e] -> /tmp/dd/lib/dovecot/imap/lib20_virtual_plugin.so [0xb7ee230a] -> /tmp/dd/lib/dovecot/imap/lib20_virtual_plugin.so(virtual_storage_sync_init+0x53f) [0xb7ee28cf] -> /tmp/dd/libexec/dovecot/imap(mailbox_sync+0x30) [0x80b03b0] -> /tmp/dd/libexec/dovecot/imap(cmd_select_full+0x3d8) [0x8064ca8] -> /tmp/dd/libexec/dovecot/imap(cmd_select+0x19) [0x8065409] -> /tmp/dd/libexec/dovecot/imap [0x806758c] -> /tmp/dd/libexec/dovecot/imap [0x8067629] -> /tmp/dd/libexec/dovecot/imap(client_handle_input+0x1d) [0x8067c2d] -> /tmp/dd/libexec/dovecot/imap(client_input+0x63) [0x80680e3] -> /tmp/dd/libexec/dovecot/imap(io_loop_handler_run+0xe0) [0x80f3400] -> /tmp/dd/libexec/dovecot/imap(io_loop_run+0x20) [0x80f2890] -> /tmp/dd/libexec/dovecot/imap(main+0x59d) 
--

Re: [Dovecot] dovecot -n - provide sys info too? - WAS: Re: Dovecot read only for users

[John Lightsey]

On Oct 29, 2008, at 12:39 PM, Timo Sirainen wrote:


First however it checks for /etc/lsb-release and if it exists, prints
DISTRIB_DESCRIPTION contents. I guess Ubuntu is the only distro
currently using that file..



A little late, but I don't see any mention of /etc/lsb-release in the  
LSB specification.  You probably want the output of /usr/bin/ 
lsb_release -d


[EMAIL PROTECTED]:/# /usr/bin/lsb_release -d
Description:CentOS release 5.2 (Final)

[EMAIL PROTECTED]:~$ /usr/bin/lsb_release -d
Description:Debian GNU/Linux unstable (sid)


The lsb_release binary has been in the specification since version 1.0.

Re: [Dovecot] Backing Up

[Scott Silva]

on 10-30-2008 11:42 AM Allen Belletti spake the following:
> I'd like to add my vote here as well; dbox would be *the* feature that
> would make me happy. I'm the guy who asked a few weeks ago about ways to
> speed access on our GFS clustered mail environment.
> 
> Meanwhile, I've done some preliminary testing with mbox. As expected,
> it's vastly faster than the Maildirs that we're using now. Of course it
> pains me to go "backwards" but that may be the interim solution. I got
> stopped temporarily when it seemed that I couldn't nest folders using
> mbox, but hopefully that's untrue.
> 
You can nest folders with mbox, but you can't have "folders" that contain both
messages and other folders. A "folder" in mbox can either hold messages or
other folders, but not both.

-- 
MailScanner is like deodorant...
You hope everybody uses it, and
you notice quickly if they don't



signature.asc
Description: OpenPGP digital signature

Re: [Dovecot] Backing Up

[Dave McGuire]

On Oct 30, 2008, at 2:35 PM, [EMAIL PROTECTED] wrote:

Maildir is nice compared to mbox but it really isn’t optimal. In days
where IOPS is the most difficult resource to get into your server (and
dovecot already using close to nothing in terms of CPU time and  
memory)

having one file per e-mail is less than sub-optimal especially when a
large amount of users just downloads the whole mailbox using POP3  
(not to

mention backing up Maildirs).


  It seems to me that a database like Postgres or MySQL would be the  
best bet.


-Dave

--
Dave McGuire
Port Charlotte, FL

Re: [Dovecot] Backing Up

[Allen Belletti]

I'd like to add my vote here as well; dbox would be *the* feature that
would make me happy. I'm the guy who asked a few weeks ago about ways to
speed access on our GFS clustered mail environment.

Meanwhile, I've done some preliminary testing with mbox. As expected,
it's vastly faster than the Maildirs that we're using now. Of course it
pains me to go "backwards" but that may be the interim solution. I got
stopped temporarily when it seemed that I couldn't nest folders using
mbox, but hopefully that's untrue.

Allen

[EMAIL PROTECTED] wrote:

Timo Sirainen:
  

One possibility is to just wait for dbox with multiple-messages-per-file
feature. I can't really say when it'll be ready (or when I'll even start
implementing it), but I know I want to use it myself and some companies
have also recently been asking about it.





Have you considered making dbox a major priority for v. 1.2?

I have been holding back on v.1.2 because I don’t really see the big
improvements in it that I saw in v.1.0 and v.1.1.
With 1.0 and 1.1 I hurried off using them in production environments even
while they where still in beta (of course only after proper testing)
because they posed so many advantages (primarily speed and stability) over
other solutions.

Since I’m focused almost entirely on stability and speed, and very little
on fancy functionality, what v.1.0 offers in terms of functionality is
just fine. What drove me towards 1.1 were speed improvements (and
stability on NFS).
I remember you made a post about not many people testing v.1.2.
I think the reason may be that most users feel the same as me. They’d like
to se a major feature that benefits their primary needs, which isn’t in
term of functionality but more in term of speed improvements.
Dbox could be that feature as I think there isn’t much room for further
developing the Maildir format (and as far as I can see you have gone as
far as possible with regards to optimizing speed while working within the
boundaries of the Maildir standard).

Maildir is nice compared to mbox but it really isn’t optimal. In days
where IOPS is the most difficult resource to get into your server (and
dovecot already using close to nothing in terms of CPU time and memory)
having one file per e-mail is less than sub-optimal especially when a
large amount of users just downloads the whole mailbox using POP3 (not to
mention backing up Maildirs).

Now don't take this as a critic, I love your software.
I just would really like to se dbox evolve and think it would be a major
driving force for v.1.2 :)

Develop dbox, Do it. Do it naoughw! (preferably pronounced with a
schwarzeneggerish accent like in the last three seconds of this splendid
video http://www.youtube.com/watch?v=adc3MSS5Ydc).

Best regards, Mikkel


  


--
Allen Belletti
[EMAIL PROTECTED] 404-894-6221 Phone
Industrial and Systems Engineering404-385-2988 Fax
Georgia Institute of Technology

Re: [Dovecot] Backing Up

[mikkel]

Timo Sirainen:
> One possibility is to just wait for dbox with multiple-messages-per-file
> feature. I can't really say when it'll be ready (or when I'll even start
> implementing it), but I know I want to use it myself and some companies
> have also recently been asking about it.
>


Have you considered making dbox a major priority for v. 1.2?

I have been holding back on v.1.2 because I don’t really see the big
improvements in it that I saw in v.1.0 and v.1.1.
With 1.0 and 1.1 I hurried off using them in production environments even
while they where still in beta (of course only after proper testing)
because they posed so many advantages (primarily speed and stability) over
other solutions.

Since I’m focused almost entirely on stability and speed, and very little
on fancy functionality, what v.1.0 offers in terms of functionality is
just fine. What drove me towards 1.1 were speed improvements (and
stability on NFS).
I remember you made a post about not many people testing v.1.2.
I think the reason may be that most users feel the same as me. They’d like
to se a major feature that benefits their primary needs, which isn’t in
term of functionality but more in term of speed improvements.
Dbox could be that feature as I think there isn’t much room for further
developing the Maildir format (and as far as I can see you have gone as
far as possible with regards to optimizing speed while working within the
boundaries of the Maildir standard).

Maildir is nice compared to mbox but it really isn’t optimal. In days
where IOPS is the most difficult resource to get into your server (and
dovecot already using close to nothing in terms of CPU time and memory)
having one file per e-mail is less than sub-optimal especially when a
large amount of users just downloads the whole mailbox using POP3 (not to
mention backing up Maildirs).

Now don't take this as a critic, I love your software.
I just would really like to se dbox evolve and think it would be a major
driving force for v.1.2 :)

Develop dbox, Do it. Do it naoughw! (preferably pronounced with a
schwarzeneggerish accent like in the last three seconds of this splendid
video http://www.youtube.com/watch?v=adc3MSS5Ydc).

Best regards, Mikkel

Re: [Dovecot] dovecot expire doesn't work (?)

[LÉVAI Dániel]

On Thursday 30 October 2008 16.42.16 Timo Sirainen wrote:
> On Thu, 2008-10-30 at 16:28 +0100, LÉVAI Dániel wrote:
> > I've got bitten by this:
> > The wiki[1] reads:
> >
> > [...]
> >  - "%" works by matching any number of characters, but it stops at
> > the hierarchy separator. Currently the separator is hardcoded to
> > "/". [...]
> > plugin {
> >   # Trash and its children 7d, Spam 30d
> >   expire = Trash 7 Trash/* 7 Spam 30
> > [...]
> >
> > That is not exactly true. The separator which is working (as told
> > me by e-frog, and as can be seen in the Maildir/ hierarchy) is the
> > dot character (ie.: .).
>
> The "/" hardcoding only means the "%" wildcard matching, meaning if
> you've a mailbox "foo/bar" then "%" would match only "foo" part, but
> if you've a mailbox "foo.bar" then "%" would match the full
> "foo.bar". In any case you'll need to use the separator you've
> configured in your namespaces.
Where have I configured that? I'm just using maildir: in the 
mail_location, and if I create a subdirectory with my MUA, in the 
server on the filesystem it will separate it from the parent with a 
dot. Is this configurable?

> > Yep, now I can understand that, but what *is* weird, that the
> > only "debug" information comes from this expire-tool when ran with
> > the --test option. If I run it without it, it won't output anything
> > anywhere. It would be nice to increase the logging for this (with
> > or without the --test option), eg. when mail_debug=yes.
>
> What should it write? I guess -v parameter could do something.
> mail_debug=yes could affect the plugin's logging.
I think, it should display that it found an "expire = something" entry 
in dovecot.conf, and that it could find a matching directory under the 
mail_location. While iterating over the messages that it has found, it 
would be nice if it would write an info line with each message, 
including the message's path/name, the message's expire date in the 
future, that it has been expunged, or that it would have been expunged, 
but the --test option was set.
Like:
$ .../expire-tool --test
expire = spamassassin.SPAM 1 spamassassin.HAM 1
match: /var/virtualmaildir/user1/Maildir/.spamassassin.SPAM/
cur/1225290969.M266240P9922.host,W=3210,S=3155:2,S will expire on Oct 
31, 13:01
cur/1225291087.M157951P9922.host,W=3267,S=3193:2,S will expire on Oct 
30, 19:25
cur/1225292609.M646577P6712.host,S=2802,W=2874:2,S expunged (not really)
cur/1225316456.M35928P11573.host,S=3760,W=3852:2,S expunged (not really)
cur/1225333013.M644866P4387.host,S=4208,W=4311:2,S expunged (not really)
cur/1225350074.M658507P24283.host,W=2508,S=2462:2,S expunged (not 
really)
cur/1225361450.M896405P30952.host,S=58036,W=58865:2,S expunged (not 
really)
match: /var/virtualmaildir/user1/Maildir/.spamassassin.HAM/
cur/1225381029.M654813P11449.host,W=5325,S=5268:2,S expunged (not 
really)
match: /var/virtualmaildir/user116/Maildir/.spamassassin.HAM/
cur/1225290969.M35928P11573.host,W=5325,S=5268:2,S will expire on Oct 
31, 01:54

Of course without the --test option it wouldn't have to write the (not 
really) part.
Anyway, something like the above :)

Daniel

-- 
LEVAI Daniel
PGP key ID = 0x4AC0A4B1
Key fingerprint = D037 03B9 C12D D338 4412  2D83 1373 917A 4AC0 A4B1

Re: [Dovecot] dovecot expire doesn't work (?)

[Timo Sirainen]

On Thu, 2008-10-30 at 16:28 +0100, LÉVAI Dániel wrote:
> I've got bitten by this:
> The wiki[1] reads:
> 
> [...]
>  - "%" works by matching any number of characters, but it stops at the 
> hierarchy separator. Currently the separator is hardcoded to "/".
> [...]
> plugin {
>   # Trash and its children 7d, Spam 30d
>   expire = Trash 7 Trash/* 7 Spam 30
> [...]
> 
> That is not exactly true. The separator which is working (as told me by 
> e-frog, and as can be seen in the Maildir/ hierarchy) is the dot 
> character (ie.: .).

The "/" hardcoding only means the "%" wildcard matching, meaning if
you've a mailbox "foo/bar" then "%" would match only "foo" part, but if
you've a mailbox "foo.bar" then "%" would match the full "foo.bar". In
any case you'll need to use the separator you've configured in your
namespaces. I guess wiki should explain this clearly. Or better yet, I
could fix the whole issue and remove it from wiki. :)

> Also the dovecot-example.conf says:
> "The following dict block maps dictionary names to URIs when the server 
> is used. These can then be referenced using URIs in 
> format "proxy:"."
> 
> That is not true either, it must be "proxy::" (note the two 
> colons) or else dovecot won't even start.

Fixed now.

> Yep, now I can understand that, but what *is* weird, that the 
> only "debug" information comes from this expire-tool when ran with 
> the --test option. If I run it without it, it won't output anything 
> anywhere. It would be nice to increase the logging for this (with or 
> without the --test option), eg. when mail_debug=yes.

What should it write? I guess -v parameter could do something.
mail_debug=yes could affect the plugin's logging.


signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] Startup info message

[Timo Sirainen]

On Thu, 2008-10-30 at 08:30 -0700, Seth Mattinen wrote:
> > "Info: If you have trouble with authentication failures, enable auth_debug
> > setting. See http://wiki.dovecot.org/WhyDoesItNotWork";

> Ah, the irony. The message that was supposed to generate less questions 
> actually causes questions itself.

I did think about this myself before releasing 1.1.6, but then thought
it'll generate questions only for a short time while people are
upgrading, so it's not worth it to add a new line saying "This message
goes away after the first successful authentication."


signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] Startup info message

[Seth Mattinen]

Athan Dimoy wrote:

Hi folks,

I'm using the latest Dovecot 1.16 with Postfix for my FreeBSD box. Since 
v1.1.4

(if i remember correctly), the following message is displayed whenever
starting Dovecot.

"Info: If you have trouble with authentication failures, enable auth_debug
setting. See http://wiki.dovecot.org/WhyDoesItNotWork";

Besides that, Dovecot works great but I wonder what this message is all 
about. I've

searched wiki and mailing list but didn't find anything related.

PS: I removed Dovecot removing all stalled files and directories and
reinstalled it from scratch. Still the message is displayed whenever it
starts.

Any idea?




Ah, the irony. The message that was supposed to generate less questions 
actually causes questions itself.


If your Dovecot works, just ignore it.

~Seth

Re: [Dovecot] dovecot expire doesn't work (?)

[LÉVAI Dániel]

On Monday 27 October 2008 14.26.50 LÉVAI Dániel wrote:
> Hi!
>
> I'm using dovecot-1.1.5 and trying to make the expire plugin work.
> What I've configured in dovecot.conf is the following:
>
> protocol imap,pop3,lda {
>   mail_plugins = [...] expire
> }
>
> dict {
>   expire = db:/var/dovecot/expire/expire.db
> }
>
>
> plugin {
>expire = spamassassin/SPAM 2 spamassassin/HAM 2
>expire_dict = proxy::expire
> }
>
> I have a sieve rule, to copy certain messages to my
> "spamassassin/SPAM" folder. Then I want to expire those messages
> after 2 days (I think I've configured that under the plugin{} section
> in dovecot.conf). So the actual message saving is done by the
> dovecot's deliver, but I have the plugin loaded under the "protocol
> lda {}" section too. So I thought now I just have to wait 2 days, and
> run the expire-tool, and then it will expire the messages.
> Now I have three messages dated back to 10.25, but running the
> expire-tool outputs nothing.
> # dovecot --exec-mail ext /usr/local/libexec/dovecot/expire-tool
> --test
>
> Nothing in the logfiles, and nothing on the console. I have the
> /var/dovecot/expire directory:
> # ls -la /var/dovecot/expire/
> total 1640
> drwx--  2 root  wheel   512 Oct 26 19:47:53 2008 ./
> drwxr-x---  3 root  wheel   512 Oct 27 07:57:42 2008 ../
> -rw---  1 root  wheel 24576 Oct 27 13:00:01 2008 __db.001
> -rw---  1 root  wheel 57344 Oct 27 13:00:01 2008 __db.002
> -rw---  1 root  wheel270336 Oct 27 13:00:01 2008 __db.003
> -rw---  1 root  wheel 98304 Oct 27 13:00:01 2008 __db.004
> -rw---  1 root  wheel 49152 Oct 27 13:00:01 2008 __db.005
> -rw---  1 root  wheel 32768 Oct 26 19:47:37 2008 expire.db
> -rw---  1 root  wheel  10485760 Oct 27 14:22:08 2008
> log.01
>
> It contains the familiar BDB files, so I think it works, although the
> expire.db's modify time is yesterday, but deliver saved some messages
> also today to the spamassassin/SPAM folder.

I've got bitten by this:
The wiki[1] reads:

[...]
 - "%" works by matching any number of characters, but it stops at the 
hierarchy separator. Currently the separator is hardcoded to "/".
[...]
plugin {
  # Trash and its children 7d, Spam 30d
  expire = Trash 7 Trash/* 7 Spam 30
[...]

That is not exactly true. The separator which is working (as told me by 
e-frog, and as can be seen in the Maildir/ hierarchy) is the dot 
character (ie.: .).
My $USER/spamassassin/SPAM directory is not working as:
expire = spamassassin/SPAM 1
only as:
expire = spamassassin.SPAM 1

Also the dovecot-example.conf says:
"The following dict block maps dictionary names to URIs when the server 
is used. These can then be referenced using URIs in 
format "proxy:"."

That is not true either, it must be "proxy::" (note the two 
colons) or else dovecot won't even start.

Anyway, for the record, I should mention that while it is easy to check 
whether dovecot is fooling around with a mysql database, it is not so 
straightforward with BDB.
One can check if the Berkeley database is being used with db4_dump (or 
on some systems db4.7_dump or db4.6_dump and so on...):

$ db4_dump -d a expire.db
[...]
page 1: btree leaf: LSN [0][1]: level 1
prev:0 next:0 entries:0 offset: 16384

^^^ the above contains no entries, while:
$ db4_dump -da expire.db
[...]
page 1: btree leaf: LSN [1][84670]: level 1
prev:0 next:0 entries:2 offset: 16344
[000] 16352 len:  29 data: shared/leva/spamassa...
[001] 16344 len:   4 data: ��0x08I
^^^ this contains entries. Don't ask me what is the second row, 
though :), and also it is a PITA that the data gets trimmed.


On Wednesday 29 October 2008 15.53.24 Timo Sirainen wrote:
> On Wed, 2008-10-29 at 15:25 +0100, LÉVAI Dániel wrote:
> > When I ran `dovecot --exec-mail ext
> > /usr/local/libexec/dovecot/expire-tool --test', it told me that:
> > Info: leva/spamassassin.SPAM: stop, expire time in future:
> > 1225290174
>
> Sounds like it's working. It just wasn't time yet to expunge the
> oldest mail from there

Yep, now I can understand that, but what *is* weird, that the 
only "debug" information comes from this expire-tool when ran with 
the --test option. If I run it without it, it won't output anything 
anywhere. It would be nice to increase the logging for this (with or 
without the --test option), eg. when mail_debug=yes.


[1] - http://wiki.dovecot.org/Plugins/Expire

Daniel

-- 
LEVAI Daniel
PGP key ID = 0x4AC0A4B1
Key fingerprint = D037 03B9 C12D D338 4412  2D83 1373 917A 4AC0 A4B1

Re: [Dovecot] Backing Up

[Timo Sirainen]

On Thu, 2008-10-30 at 11:00 -0400, Stewart Dean wrote:
> Dave McGuire wrote:
> > On Oct 29, 2008, at 3:42 PM, Scott Silva wrote:
> >>> What is the best way to do a (server-side) backup of all mail in a
> >>> user's mail?
> >>>
> >> I usually just rsync the /home directories to another server. The 
> >> inital sync
> >> can take a while, but it gets faster after there is a base to work from.
> >
> >   ...and it's much less painful if you're using maildir instead of mbox!
> >
> >-Dave
> >
> I have to wonder.  I have a mailserver that I do a bootable complete 
> image copy of with all files and O/S in two hours to an Ultrium-2 tape, 
> 95 GB.  When I switch to maildir, I will go from some 25,000 mbox files 
> to 2.5 to 3 million files...I can't believe that isn't going to hurt and 
> will force me into incrementals.

One possibility is to just wait for dbox with multiple-messages-per-file
feature. I can't really say when it'll be ready (or when I'll even start
implementing it), but I know I want to use it myself and some companies
have also recently been asking about it.


signature.asc
Description: This is a digitally signed message part

Re: [Dovecot] dovecot 1.1.5 mbox bug (From_-line separator related)

[Anton Yuzhaninov]

On 30.10.2008 01:01, Timo Sirainen wrote:

On Wed, 2008-10-29 at 23:31 +0300, Anton Yuzhaninov wrote:

On 29.10.2008 21:24, Timo Sirainen wrote:

On Wed, 2008-10-29 at 20:23 +0300, Anton Yuzhaninov wrote:

If I send mail with content like this:

 From <[EMAIL PROTECTED]> Wed Jan 09 01:33:55 2008

..

So a message whose body begins with "From " line? Fixed:
http://hg.dovecot.org/dovecot-1.1/rev/9c3fa81a721d


This patch help only when message begins with "From " line, but
bug still exits if message begins with empty line, then "From " line.


This should fix both: http://hg.dovecot.org/dovecot-1.1/rev/c89c9d0bc877



Thanks, I can confirm, that this path fixes bug.

--
WBR,
 Anton Yuzhaninov

Re: [Dovecot] Backing Up

[Stewart Dean]

Dave McGuire wrote:

On Oct 29, 2008, at 3:42 PM, Scott Silva wrote:

What is the best way to do a (server-side) backup of all mail in a
user's mail?

I usually just rsync the /home directories to another server. The 
inital sync

can take a while, but it gets faster after there is a base to work from.


  ...and it's much less painful if you're using maildir instead of mbox!

   -Dave

I have to wonder.  I have a mailserver that I do a bootable complete 
image copy of with all files and O/S in two hours to an Ultrium-2 tape, 
95 GB.  When I switch to maildir, I will go from some 25,000 mbox files 
to 2.5 to 3 million files...I can't believe that isn't going to hurt and 
will force me into incrementals.

Re: [Dovecot] mail_executable's process environment

[Mike Malsman]

On 29.Oct.2008, at 11:41 AM, Timo Sirainen wrote:

On Mon, 2008-10-20 at 20:02 -0400, Mike Malsman wrote:

Upon inspection of the processes' environment I'm pleased to see that
there's a load of useful information in there.  However, one  
essential

component in my case is the destination network address, which is
missing.  I added it with the attached patch, exposed as 'LOCAL_IP'.
Works for me.

Is this something that would be useful to anyone else?


OK, added: http://hg.dovecot.org/dovecot-1.1/rev/a5495e3e90c9


Thanks very much, Timo.

I know the patch is trivial but it saves me the effort of remembering  
to patch at all :]


-Mike

Re: [Dovecot] Backing Up

[Ken A]

Calvin Gordon wrote:
I use the tar/bzip method, and have been wondering about the rsync.  All 
my users have system accounts on the dovecot server, and use Maildir 
format.  If i rsync the mail to another box where the users do not have 
system accounts, will the ownerships/ permissions etc. be goofed up ?


Correctly, or incorrectly, I've been using tar to preserve all that 
information.


rsync preserves all that too, but you should preserve uid->username and 
gid->groupname mappings too, otherwise all that information is not as 
useful. Saving the password files is usually sufficient, assuming you 
are doing backups for disaster recovery, and not just for the occasional 
restore after an "oops, I deleted all my mail!" phonecall.


rsnapshot is nice too. It uses rsync and hard links to make as many 
snapshots of the filesystem as you like. This creates many 'restore 
points' with total disk usage being just over what a single full backup 
would take.


Ken



Cal Gordon

Sotiris Tsimbonis wrote:

Scott Silva wrote, On 10/30/2008 12:34 AM:

on 10-29-2008 3:18 PM Dave McGuire spake the following:

On Oct 29, 2008, at 5:32 PM, Arkadiusz Miskiewicz wrote:

What is the best way to do a (server-side) backup of all mail in a
user's mail?

I usually just rsync the /home directories to another server. The
inital sync
can take a while, but it gets faster after there is a base to work
from.

   ...and it's much less painful if you're using maildir instead of
mbox!

Not for rsyncing. Tons of small files means much slower rsync.
  Due to connection turnaround latency, I assume?  (I've never 
looked at

the rsync protocol)  If that's the case, then I stand very much
corrected, thank you.  I was going from the same logic regarding mbox
vs. maildir in the context of backups.  One new message delivered and a
400MB mail spool gets backed up again..

  -Dave


Rsync adds some latency as it indexes and compares files on both ends.
Obviously it would take more time to compare 40,000 1K files then 
1000 40K
files even though the data size is similar. It would still be better 
than
tar/bzip/scp which has to compress everything and transfer the lot 
every time.




Maildirsync it an "Online synchronizer for Maildir-format mailboxes"
See http://hacks.dlux.hu/maildirsync/

Sot.






--
Ken Anderson
Pacific.Net

Re: [Dovecot] Backing Up

[Calvin Gordon]

I use the tar/bzip method, and have been wondering about the rsync.  All 
my users have system accounts on the dovecot server, and use Maildir 
format.  If i rsync the mail to another box where the users do not have 
system accounts, will the ownerships/ permissions etc. be goofed up ?


Correctly, or incorrectly, I've been using tar to preserve all that 
information.


Cal Gordon

Sotiris Tsimbonis wrote:

Scott Silva wrote, On 10/30/2008 12:34 AM:

on 10-29-2008 3:18 PM Dave McGuire spake the following:

On Oct 29, 2008, at 5:32 PM, Arkadiusz Miskiewicz wrote:

What is the best way to do a (server-side) backup of all mail in a
user's mail?

I usually just rsync the /home directories to another server. The
inital sync
can take a while, but it gets faster after there is a base to work
from.

   ...and it's much less painful if you're using maildir instead of
mbox!

Not for rsyncing. Tons of small files means much slower rsync.
  Due to connection turnaround latency, I assume?  (I've never 
looked at

the rsync protocol)  If that's the case, then I stand very much
corrected, thank you.  I was going from the same logic regarding mbox
vs. maildir in the context of backups.  One new message delivered and a
400MB mail spool gets backed up again..

  -Dave


Rsync adds some latency as it indexes and compares files on both ends.
Obviously it would take more time to compare 40,000 1K files then 
1000 40K
files even though the data size is similar. It would still be better 
than
tar/bzip/scp which has to compress everything and transfer the lot 
every time.




Maildirsync it an "Online synchronizer for Maildir-format mailboxes"
See http://hacks.dlux.hu/maildirsync/

Sot.

Re: [Dovecot] [dovecot] Enable logging of all client commands in dovecot-1.2.alpha3

[Mateusz Kijowski]

Dnia czwartek, 30 października 2008, Jonathan Siegle napisał:
> Hello,
>   I would like to log all IMAP client commands sent to dovecot. The
> format would be time pid command arguments. I reviewed
> http://wiki.dovecot.org/Logging and started digging through
> dovecot-1.2.alpha3/src/master . I don't need this turned on all the
> time, just enough to see how clients do things and I don't need to see
> passwords.
>
> Any tips would be appreciated.
>
> -Jonathan

Maybe this will help:

http://wiki.dovecot.org/Debugging/Rawlog

--
Mateusz





signature.asc
Description: This is a digitally signed message part.

[Dovecot] [dovecot] Enable logging of all client commands in dovecot-1.2.alpha3

[Jonathan Siegle]

Hello,
	I would like to log all IMAP client commands sent to dovecot. The 
format would be time pid command arguments. I reviewed 
http://wiki.dovecot.org/Logging and started digging through 
dovecot-1.2.alpha3/src/master . I don't need this turned on all the 
time, just enough to see how clients do things and I don't need to see 
passwords.


Any tips would be appreciated.

-Jonathan


smime.p7s
Description: S/MIME Cryptographic Signature

Re: [Dovecot] Problem with Dovecot 1.1.4 and dovecot-antispam 20080829

[HILT Guillaume]

On Thu, 30 Oct 2008 10:51:43 +0100, Guillaume Hilt
<[EMAIL PROTECTED]> wrote:
> Hi there,
> 
> I'm running a server under Gentoo 64 with postfix 2.5.5, Dovecot 
> 1.1.4-r1 and Dovecot-antispam 20080829 (all are the lastest versions 
> availables under Portage).
> I ran into a problem when I log in to my mailbox :
> 
> Oct 30 10:39:23 ks29138 dovecot: IMAP(hiltg): Loading modules from 
> directory: /usr/lib64/dovecot/imap
> Oct 30 10:39:23 ks29138 dovecot: IMAP(hiltg): Module is for different 
> version 1.1.1: /usr/lib64/dovecot/imap/lib90_antispam_plugin.so
> Oct 30 10:39:23 ks29138 dovecot: Fatal: IMAP(hiltg): Couldn't load 
> required plugins
> 
> Is there another version for Dovecot 1.1.4 out there or must I go back 
> to 1.1.1 ?
> 
> Thanks,
> 
> Guillaume
> 

Compiling the head snapshot fixed the problem, sounds like portage needs a
new ebuild for this plugin.

Re: [Dovecot] benchmark dovecot

[Mathieu Kretchner]

Mark Zealey a écrit :
>> Mark Zealey a écrit :
>>> Thanks; these look interesting. We have a similar nas setup 
>> but we have
>>> 2 frontend dovecot servers connecting to it and store the 
>> indexes over
>>> nfs. 
>> Could you please tell me how have you done this configuration ?
>> 2 frontend dovecot proxy with 10 dovecot mda ? We are looking 
>> for such a
>> configuration : 2 mda frontend with maybe an active and a 
>> passive one !
> 
> I'm not quite sure what you mean? Physically, we have loadbalancers on the 
> frontend so it's all active/active. We use exim with database lookups to find 
> the home directory and then use deliver to drop it onto the filer and update 
> the indexes.
> 
> Mark

Ok sorry for my poor english but you answer to my question ! thanks
begin:vcard
fn:Mathieu Kretchner
n:Kretchner;Mathieu
org:INRIA;Syslog
adr;dom:;;2004 route des lucioles - BP93;Sophia Antipolis;;06902 CEDEX
email;internet:[EMAIL PROTECTED]
tel;work:04 92 38 76 67
x-mozilla-html:FALSE
version:2.1
end:vcard

Re: [Dovecot] benchmark dovecot

[Mark Zealey]

> Mark Zealey a écrit :
> > Thanks; these look interesting. We have a similar nas setup 
> but we have
> > 2 frontend dovecot servers connecting to it and store the 
> indexes over
> > nfs. 
> Could you please tell me how have you done this configuration ?
> 2 frontend dovecot proxy with 10 dovecot mda ? We are looking 
> for such a
> configuration : 2 mda frontend with maybe an active and a 
> passive one !

I'm not quite sure what you mean? Physically, we have loadbalancers on the 
frontend so it's all active/active. We use exim with database lookups to find 
the home directory and then use deliver to drop it onto the filer and update 
the indexes.

Mark

Re: [Dovecot] Enh-Req: Mark As Read When Delivered

[Eduardo M KALINOWSKI]

Neil wrote:
> I'm under the impression bug-reports are supposed to go to the list,  
> so hopefully it's okay if I put in a feature request here too  
> (assuming it's not already implemented; but it doesn't look like it).
>
> Basically, all I would like to do is be able to sometimes deliver mail  
> as already mail into mail boxes.  Is there some way to do this?
>
> If not, could a flag perhaps be added to deliver to do it?  (And  
> Sieve; but for now I think procmail still has much higher adoption,  
> and thus having it in deliver would be rather key...)
>   

You can do that with a sieve script, there's a feature (called imapflags
or something similar) that allows you to mark the e-mail as read.


-- 
The way of the world is to praise dead saints and prosecute live ones.
-- Nathaniel Howe

Eduardo M KALINOWSKI
[EMAIL PROTECTED]
http://move.to/hpkb

Re: [Dovecot] benchmark dovecot

[Mathieu Kretchner]

Mark Zealey a écrit :
> Thanks; these look interesting. We have a similar nas setup but we have
> 2 frontend dovecot servers connecting to it and store the indexes over
> nfs. 
Could you please tell me how have you done this configuration ?
2 frontend dovecot proxy with 10 dovecot mda ? We are looking for such a
configuration : 2 mda frontend with maybe an active and a passive one !


> We also have around 10 mail servers running deliver to try to keep
> the indexes on the nfs store up-to-date. Have you done any tests with
> the speed of multiple boxes each maintaining a local index of the
> mailbox? 
No sorry

> I suspect in this case keeping indexes on nfs would be the best
> bet but I don't have anything to substantiate that claim...
> 
> Also one thing to note with storing things on nfs is that there are a
> large number of broken kernels out there (they issue about 10* more nfs
> lookup requests than they should) - centos 5.1 had these issues iirc
> (though the centosplus kernel and centos 5.2 did fix it).
Good thing to know, I'll try to change kernel before my migration !

> Always give it
> a good test before you change the kernel on your server... I assume you
> are using nfs3; has anyone tried using heavily loaded nfs4 and seeing if
> any better performance can be achieved?
> 
> Another thing - I found that dovecot's pop3 implimentation is worse than
> courier's over nfs (wait state on our boxes is significantly increased).
> I still don't really understand why this is; I suspect it's probably due
> to to the indexes being created/updated though I thought these were
> meant to be discontinued after a while if it is just a simple
> login/fetch all operation. I only mention this because if you are
> offering pop then you should really do the same benchmarks for that.
> 
> Mark
begin:vcard
fn:Mathieu Kretchner
n:Kretchner;Mathieu
org:INRIA;Syslog
adr;dom:;;2004 route des lucioles - BP93;Sophia Antipolis;;06902 CEDEX
email;internet:[EMAIL PROTECTED]
tel;work:04 92 38 76 67
x-mozilla-html:FALSE
version:2.1
end:vcard

Re: [Dovecot] INBOX stored in AFS

[Hans-Werner Paulsen]

On Mon, Oct 20, 2008 at 06:56:55PM +0300, Timo Sirainen wrote:
> On Mon, 2008-10-20 at 16:16 +0200, Hans-Werner Paulsen wrote:
> > When I start my imap-client, I see the content of the INBOX, and I can
> > read, delete, ... the different entries. But when new mail arrives dovecot
> > (and my imap-client) will never notice this. And when I quit my imap-client
> > the dovecot imap server will write back its view of the INBOX, and delete
> > this new mail.
> > This seems to be a problem related to AFS. I poked in the dovecot code and
> > have the following theory: dovecot opens the INBOX R/W, dovecot calls "stat"
> > on the INBOX file periodically to look for modifications. But this does
> > not work, because this machine will stat the local copy (AFS cache) of the
> > INBOX as long as this file is still open R/W.
> 
> I'd have thought that the local view was updated at least after fcntl
> locking the mailbox.
> 
> > My questions:
> > Is it possible to have a configuration, that the INBOX file is not left open
> > when stat-ing this file?
> > Or is it possible to open the INBOX file R/W only when it us locked?
> > Or is it necessary to modify the code?
> 
> It's necessary to modify the code. Probably not difficult (maybe in
> mbox_unlock()), but I'd rather not change that code permanently since
> this is a pretty AFS-specific problem..
> 

I checked flock (fcntl locking is not working at all) semantics on the AFS
filesystem with some small test programs:
Calling flock does NOT update the local view of the AFS file, if it is
opened R/W. But, status information seems to be up-to-date if the file
is opened R/O.
Now I want to modify the dovecot code for mbox processing in the following
way:
open the mbox R/O
before modifying the mbox file: close it, dot-lock the file and open it R/W
after modifying the mbox file: close it, remove the dot-lock, and open it R/O

Any hints in which files I have to look for the necessary modifications?

Thank you for your help,
HW

-- 
Hans-Werner Paulsen [EMAIL PROTECTED]
MPI für Astrophysik Tel 089-3-2602
Karl-Schwarzschild-Str. 1   Fax 089-3-2235  
D-85741 Garching

Re: [Dovecot] benchmark dovecot

[Mark Zealey]

Thanks; these look interesting. We have a similar nas setup but we have
2 frontend dovecot servers connecting to it and store the indexes over
nfs. We also have around 10 mail servers running deliver to try to keep
the indexes on the nfs store up-to-date. Have you done any tests with
the speed of multiple boxes each maintaining a local index of the
mailbox? I suspect in this case keeping indexes on nfs would be the best
bet but I don't have anything to substantiate that claim...

Also one thing to note with storing things on nfs is that there are a
large number of broken kernels out there (they issue about 10* more nfs
lookup requests than they should) - centos 5.1 had these issues iirc
(though the centosplus kernel and centos 5.2 did fix it). Always give it
a good test before you change the kernel on your server... I assume you
are using nfs3; has anyone tried using heavily loaded nfs4 and seeing if
any better performance can be achieved?

Another thing - I found that dovecot's pop3 implimentation is worse than
courier's over nfs (wait state on our boxes is significantly increased).
I still don't really understand why this is; I suspect it's probably due
to to the indexes being created/updated though I thought these were
meant to be discontinued after a while if it is just a simple
login/fetch all operation. I only mention this because if you are
offering pop then you should really do the same benchmarks for that.

Mark

[Dovecot] benchmark dovecot

[Mathieu Kretchner]

Hello,

We would like to do a feed back to this active mailing list. We are
working on a migration project from cyrus to dovecot. And we have just
completed the benchmark sequence.

As I say, this benchmark is here only to show that our old imap server
is out to date. I would not be the source of controversy at all, so I
try to explain my approach.

Because the only thing I found was this old oriented benchmark :
http://www.isode.com/whitepapers/mbox-benchmark.html

We've tried to do our tests, here you could find our results :

http://www-sop.inria.fr/members/Mathieu.Kretchner/dotclear/index.php/2008/10/29/3-dovecot-versus-cyrus


begin:vcard
fn:Mathieu Kretchner
n:Kretchner;Mathieu
org:INRIA;Syslog
adr;dom:;;2004 route des lucioles - BP93;Sophia Antipolis;;06902 CEDEX
email;internet:[EMAIL PROTECTED]
tel;work:04 92 38 76 67
x-mozilla-html:FALSE
version:2.1
end:vcard

[Dovecot] Problem with Dovecot 1.1.4 and dovecot-antispam 20080829

[Guillaume Hilt]

   Hi there,

I'm running a server under Gentoo 64 with postfix 2.5.5, Dovecot 
1.1.4-r1 and Dovecot-antispam 20080829 (all are the lastest versions 
availables under Portage).

I ran into a problem when I log in to my mailbox :

Oct 30 10:39:23 ks29138 dovecot: IMAP(hiltg): Loading modules from 
directory: /usr/lib64/dovecot/imap
Oct 30 10:39:23 ks29138 dovecot: IMAP(hiltg): Module is for different 
version 1.1.1: /usr/lib64/dovecot/imap/lib90_antispam_plugin.so
Oct 30 10:39:23 ks29138 dovecot: Fatal: IMAP(hiltg): Couldn't load 
required plugins


Is there another version for Dovecot 1.1.4 out there or must I go back 
to 1.1.1 ?


Thanks,

   Guillaume

Re: [Dovecot] Startup info message

[Matthias Andree]

On Thu, 30 Oct 2008, Athan Dimoy wrote:

> Hi folks,
>
> I'm using the latest Dovecot 1.16 with Postfix for my FreeBSD box. Since  
> v1.1.4
> (if i remember correctly), the following message is displayed whenever
> starting Dovecot.
>
> "Info: If you have trouble with authentication failures, enable auth_debug
> setting. See http://wiki.dovecot.org/WhyDoesItNotWork";
>
> Besides that, Dovecot works great but I wonder what this message is all  
> about. I've
> searched wiki and mailing list but didn't find anything related.

It was in the release notes and supposed to go away upon first
successful authentication.

-- 
Matthias Andree

[Dovecot] Startup info message

[Athan Dimoy]

Hi folks,

I'm using the latest Dovecot 1.16 with Postfix for my FreeBSD box. Since 
v1.1.4

(if i remember correctly), the following message is displayed whenever
starting Dovecot.

"Info: If you have trouble with authentication failures, enable auth_debug
setting. See http://wiki.dovecot.org/WhyDoesItNotWork";

Besides that, Dovecot works great but I wonder what this message is all 
about. I've

searched wiki and mailing list but didn't find anything related.

PS: I removed Dovecot removing all stalled files and directories and
reinstalled it from scratch. Still the message is displayed whenever it
starts.

Any idea?

Athan 

Re: [Dovecot] read only FS access

[Mathieu Kretchner]

 > mail_location =
maildir:~/Maildir:CONTROL=~/Maildir/dovecot:INDEX=~/Maildir/dovecot
> 
It's ok, I've tried with this configuration and it's working.
Thanks for your help !
begin:vcard
fn:Mathieu Kretchner
n:Kretchner;Mathieu
org:INRIA;Syslog
adr;dom:;;2004 route des lucioles - BP93;Sophia Antipolis;;06902 CEDEX
email;internet:[EMAIL PROTECTED]
tel;work:04 92 38 76 67
x-mozilla-html:FALSE
version:2.1
end:vcard

Re: [Dovecot] dovecot deliver mail bounce problem

[dhaval thakar]

On Wed, Oct 29, 2008 at 8:55 PM, Timo Sirainen <[EMAIL PROTECTED]> wrote:

> On Fri, 2008-10-24 at 17:57 +0530, Dhaval Thakar wrote:
> > but when mail is sent to invalid user, it gets bounced back with error
> > "I'm not going to try again; this message has been in the queue too
> > long." rather "no mailbox here by that name"
> ..
> > Oct 24 23:03:27 backup dovecot: auth(default): master out: NOTFOUND 1
>
> dovecot-auth successfully figures out that the user doesn't exist.
>
> > .qmail-default
> > |/var/qmail/bin/preline -f /usr/local/libexec/dovecot/deliver -n -e -d
> > [EMAIL PROTECTED]
>
> Try manually running:
>
> /usr/local/libexec/dovecot/deliver -n -e -d blabla
> echo $?
>
> What does the echo print? If it prints 67, the problem isn't deliver but
> qmail.
>
thanks
it prints 67.

i'll check with qmail for solution


if somebody is using dovecot deliver with qmail, kindly guide me

Re: [Dovecot] dovecot -n - provide sys info too? - WAS: Re: Dovecot read only for users

[Proskurin Kirill]

Charles Marcus wrote:

On 10/29/2008 2:41 PM, Timo Sirainen wrote:

/etc/gentoo-release

Added.


Hmmm... this doesn't really contain useful info though...

~ # cat /etc/gentoo release
Gentoo Base System release 1.12.11.1

Looks just fine.


maybe some forme of the uname command?

~ # uname -orpm
2.6.23-gentoo-r9 x86_64 AMD Opteron(tm) Processor 244 GNU/Linux

uname() is also used. It would probably print with you:

# OS: Linux 2.6.23-gentoo-r9 x86_64 Gentoo Base System release 1.12.11.1


Ahh... well, there ya go... thanks Timo, maybe this will save you some
few precious seconds... ;)



For FreeBSD you may use:

#sysctl kern.version

kern.version: FreeBSD 7.0-RELEASE-p5 #0: Wed Oct  1 10:10:12 UTC 2008
[EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC

Or:
# uname -srmi
FreeBSD 7.0-RELEASE-p5 i386 GENERIC

--
Best regards,
Proskurin Kirill