Re: Performance

[Roland van Laar]

On 24-04-15 13:26, absolutely_f...@libero.it wrote:

Hi,

at moment I have this environment:

CentOS
nginx + phpfpm
Dovecot, with Maildir format
Postfix
Roundcube
MySQL backend
about 1 mailusers
dual core Intel(R) Pentium(R) D CPU 3.00GHz
8 GB RAM
network storage device (Coraid), ext4 file system

I have no performance issue now, but I need to move to a different  server:

FreeBSD 10.1-RELEASE
nginx + phpfpm
Dovecot
Postfix
Roundcube
dual core Intel(R) Xeon(R) CPU5120  @ 1.86GHz
16 GB RAM
local storage with zfs file system

My question is: is better to use SQLite instead of MySQL?


Do you have a lot of writes?

With SQLite you can run into locking issues:
https://www.sqlite.org/lockingv3.html
Or use the Write-Ahead Logging:
https://www.sqlite.org/wal.html

Regards,

Roland


Should I prefer dbox format?

Thank you in advance for your opinion!

Re: FreeBSD ZFS maildir to mdbox

[Roland van Laar]

On 18-03-15 08:26, Toni Mattila wrote:

Hi,

On 18-Mar-15 00:09, Roland van Laar wrote:

I'm converting my mailbox from Maildir to mdbox..
The Maildir is from an 1.2 server.
The same conversion on a virtual ubuntu 14.04 box works with mentions of
"filename has the wrong S value" and Corrupted index errors.


You should fix the Maildir files first to have correct S= (size) on 
them. Older maildrops and qmail likes to create wrong sizes and newer 
dovecots rely on that S= to be correct.


You can use http://www.dovecot.org/tools/maildir-size-fix.pl or 
similar script to fix your existing maildirs.

That fixed it for me, the migration from maildir to mdbox is working now.

I had trouble running the script at first, I added some extra 
information about the different kind of commandline options.

Feel free to include them in the script on dovecot.org.

LINE: 233
if (scalar @ARGV == 0) {
print STDERR "Usage: maildir-size-fix.pl /path/to/Maildir\n";
  print STDERR "-c :Check if the files are compressed. Use the 
uncompressed size for S=size.\n";
  print STDERR "-f :If S=size already exists, verify that it is 
correct.\n";

  print STDERR "-n :If filename doesn't already have a S=size, add it.\n";
  print STDERR "-p :If UIDLs are based on filename and no P entry 
already

  \t exist for a message, write a P entry so it doesn't
  \t change when renaming a file.\n";
  print STDERR "-r :Recursively scan the maildir for subdirectories.\n";
  print STDERR "-v :Verbose logging.\n";
  exit 1
}


Best Regards,
Toni

FreeBSD ZFS maildir to mdbox

[Roland van Laar]

Hello,

I'm converting my mailbox from Maildir to mdbox..
The Maildir is from an 1.2 server.

The new server is a virtual FreeBSD ZFS server.
The conversion is not working, I'm getting either segfaults or the sync 
just quits with errors.

I also did set with mmap_disable=yes. This didn't resolve the problem.
How can I fix this?

The same conversion on a virtual ubuntu 14.04 box works with mentions of 
"filename has the wrong S value" and Corrupted index errors.


The FreeBSD errors and dovecot -n are included below:

dsync(vagrant): Error: 
read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475)
dsync(vagrant): Error: Maildir filename has wrong S value, renamed the 
file from ./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa to 
./Maildir/cur/1296038598.29562.mail.micite.net,S=20640:2,Sa
dsync(vagrant): Error: Corrupted index cache file 
./Maildir/dovecot.index.cache: Broken physical size for mail UID 475
dsync(vagrant): Error: write(/home/vagrant/mdbox/storage/m.12) failed: 
Invalid argument
dsync(vagrant): Error: copy: 
i_stream_read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475)
dsync(vagrant): Error: 
read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475) (uid=475, box=INBOX)
dsync(vagrant): Error: Mailbox INBOX: read(msg input) failed: Cached 
message size smaller than expected (20542 < 20640, box=INBOX, UID=475)
dsync(vagrant): Error: 
read(./Maildir/cur/1296038598.29562.mail.micite.net,S=20542:2,Sa) 
failed: Cached message size smaller than expected (20542 < 20640, 
box=INBOX, UID=475) (uid=475, box=INBOX)
dsync(vagrant): Panic: file mail-index-transaction-update.c: line 964 
(mail_index_update_ext): assertion failed: (seq > 0 && (seq <= 
mail_index_view_get_messages_count(t->view) || seq <= t->last_new_seq))

Abort (core dumped)

(gdb) core doveadm.core
Core was generated by `doveadm'.
Program terminated with signal 6, Aborted.
#0  0x0008013f8a1a in ?? ()
(gdb) bt
#0  0x0008013f8a1a in ?? ()
#1  0x0008013f7149 in ?? ()
#2  0x in ?? ()


or with no core dump:

% doveadm sync maildir:./Maildir
dsync(vagrant): Error: 
read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899)
dsync(vagrant): Error: Maildir filename has wrong S value, renamed the 
file from ./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa to 
./Maildir/cur/1296466748.64525.mail.micite.net,S=7694:2,Sa
dsync(vagrant): Error: Corrupted index cache file 
./Maildir/dovecot.index.cache: Broken physical size for mail UID 899
dsync(vagrant): Error: write(/home/vagrant/mdbox/storage/m.11) failed: 
Invalid argument
dsync(vagrant): Error: copy: 
i_stream_read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, UID=899)
dsync(vagrant): Error: 
read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, 
UID=899) (uid=899, box=INBOX)
dsync(vagrant): Error: Mailbox INBOX: read(msg input) failed: Cached 
message size smaller than expected (7629 < 7694, box=INBOX, UID=899)
dsync(vagrant): Error: 
read(./Maildir/cur/1296466748.64525.mail.micite.net,S=7629:2,Sa) failed: 
Cached message size smaller than expected (7629 < 7694, box=INBOX, 
UID=899) (uid=899, box=INBOX)




% dovecot -n
# 2.2.16: /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 10.1-RELEASE-p6 amd64
mail_location = mdbox:~/mdbox
namespace {
  inbox = yes
  location =
  prefix =
  separator = .
}
passdb {
  args = /usr/local/etc/dovecot/conf.d/dovecot-sql.conf.ext
  driver = sql
}
protocols = lmtp imap
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
  user = vmail
}
ssl = required
ssl_cert =