[Dovecot] file-buffer_offset + file-buffer-used == file-sync_offset

2009-09-07 Thread Edgar Fuß
I got the following panic on 1.2.3:

Panic: file mail-transaction-log-append.c: line 88 (log_buffer_move_to_memory): 
assertion failed: (file-buffer_offset + file-buffer-used == 
file-sync_offset)

It's probably related to the file system holding the indexes being full.



Re: [Dovecot] dovecot 1.2 and logging start failures

2009-09-07 Thread Michal Hlavinka
On Monday 07 September 2009 04:18:06 Timo Sirainen wrote:
 On Tue, 2009-09-01 at 16:57 +0200, Michal Hlavinka wrote:
  Would it be possible to use syslog for error output sooner or there are
  any cons?
 
 http://hg.dovecot.org/dovecot-1.2/rev/eb4ea59e4657
 

Thanks for looking at this!

Unfortunately this does not work

backtrace:
(gdb) bt
#0  0x0034bca489d7 in _IO_vfprintf_internal (s=0x7fffe020, 
format=value 
optimized out, ap=0x7fffe350) at vfprintf.c:1580
#1  0x0034bca6f232 in _IO_vsnprintf (string=0x63f718 listen(, 
maxlen=value optimized out, format=0x63f6e8 listen(%s, %d) failed: Address 
already in use, args=0x7fffe350)
at vsnprintf.c:120
#2  0x0042605c in t_noalloc_strdup_vprintf (format=0x63f6e8 
listen(%s, %d) failed: Address already in use, args=0x7fffe350, 
size_r=0x7fffe22c) at strfuncs.c:120
#3  0x00426152 in p_strdup_vprintf (pool=0x63c780, format=0x42bd69 
listen(%s, %d) failed: %m, args=0x7fffe350) at strfuncs.c:139
#4  0x004266c2 in t_strdup_vprintf (format=0x42bd69 listen(%s, %d) 
failed: %m, args=0x7fffe350) at strfuncs.c:254
#5  0x004171c6 in syslog_handler (level=2, type=LOG_TYPE_FATAL, 
format=0x42bd69 listen(%s, %d) failed: %m, args=0x7fffe350) at 
failures.c:298
#6  0x00417274 in i_syslog_fatal_handler (type=LOG_TYPE_FATAL, 
status=89, fmt=0x42bd69 listen(%s, %d) failed: %m, args=0x7fffe350) at 
failures.c:311
#7  0x0040d783 in tee_fatal_handler (type=LOG_TYPE_FATAL, status=89, 
fmt=0x42bd69 listen(%s, %d) failed: %m, args=0x7fffe350) at main.c:152
#8  0x00416d87 in i_fatal (format=0x42bd69 listen(%s, %d) failed: 
%m) at failures.c:216
#9  0x00407616 in listener_array_listen_missing (proto=0x42bc64 
imap, listens_arr=0x647720, retry=false) at listener.c:335
#10 0x0040769d in listener_listen_missing (set=0x6474d8, 
proto=0x42bc64 imap, retry=false) at listener.c:346
#11 0x004077e1 in listeners_open_fds (old_set=0x0, retry=false) at 
listener.c:371
#12 0x0040e50c in main (argc=1, argv=0x7fffe658) at main.c:649


reproducer:

listen for ipv4 and ipv6 connections on system, where ipv6 is disabled, start 
dovecot:

Fatal: listen(::, 143) failed: Address already in use

Program received signal SIGSEGV, Segmentation fault.

Cheers,
Michal
(gdb) bt full
#0  0x0034bca489d7 in _IO_vfprintf_internal (s=0x7fffe020, 
format=value optimized out, ap=0x7fffe350) at vfprintf.c:1580
len = value optimized out
string_malloced = value optimized out
step0_jumps = {0, 3491, 3570, 6406, 6492, 2736, 2829, 5759, 3764, 3946, 
4175, 1491, 1570, 1659, 1889, 1936, 3749, 3086, 5665, 5090, 6168, -557, 4254, 
4343, 6127, -2814, 5387, 1748,
  1817, 5680}
step3b_jumps = {0 repeats 11 times, 1570, 0, 0, 1889, 1936, 3749, 
3086, 5665, 5090, 6168, -557, 4254, 4343, 6127, -2814, 5387, 0, 0, 0}
is_negative = value optimized out
alt = 0
space = 0
is_long_double = 0
step2_jumps = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4175, 1491, 1570, 1659, 
1889, 1936, 3749, 3086, 5665, 5090, 6168, -557, 4254, 4343, 6127, -2814, 5387, 
1748, 1817, 0}
the_arg = {pa_wchar = -1130298040 L'H', pa_int = -1130298040, 
pa_long_int = 226502968648, pa_long_long_int = 226502968648, pa_u_int = 
3164669256, pa_u_long_int = 226502968648,
  pa_u_long_long_int = 226502968648, pa_double = 
1.1190733549003374e-312, pa_long_double = invalid float value, pa_string = 
0x34bca10548 ,
  pa_wstring = 0x34bca10548 

Re: [Dovecot] Solr Plugin make only INBOX able to search

2009-09-07 Thread rui . carneiro

Hi again,

Anyone could help me?


Re: [Dovecot] dovecot 1.2 and logging start failures

2009-09-07 Thread Timo Sirainen

On Sep 7, 2009, at 6:01 AM, Michal Hlavinka wrote:


On Monday 07 September 2009 04:18:06 Timo Sirainen wrote:

On Tue, 2009-09-01 at 16:57 +0200, Michal Hlavinka wrote:
Would it be possible to use syslog for error output sooner or  
there are

any cons?


http://hg.dovecot.org/dovecot-1.2/rev/eb4ea59e4657



Thanks for looking at this!

Unfortunately this does not work


It worked in Ubuntu, but I guess these fix it:

http://hg.dovecot.org/dovecot-1.2/rev/cc0d7c00c6ce
http://hg.dovecot.org/dovecot-1.2/rev/5f9782109fcf



[Dovecot] high imap load on large folders

2009-09-07 Thread Ingo Thierack

Hello,

i sync with imapsync from an archiopteryx-server to an dovecot-server, for
testing until now.
The dovecot is inside an VM. storage for mail is on an nfs-share, indexes
are local.
What i now see, is, on large folder( 58000 messages and more) the load 
on the imap-process

is very high, and slowdown the transferprocess.
Is there something, I can do. Whats cause the high load ind the 
imap-process.


Here are the configuration, and below, a partial strace

# 1.2.4: /opt/dovecot/etc/dovecot.conf
# OS: Linux 2.6.18-128.1.14.el5 i686 CentOS release 5.3 (Final) nfs
base_dir: /var/run/dovecot/
ssl_cert_file: /opt/dovecot/etc/ssl/certs/dovecot.pem
ssl_key_file: /opt/dovecot/etc/ssl/private/dovecot.pem
login_dir: /var/run/dovecot/login
login_executable: /opt/dovecot/libexec/dovecot/imap-login
mail_max_userip_connections: 50
mail_location: maildir:/mnt/mail/%u/Maildir:INDEX=/var/spool/indexes/%u
mailbox_idle_check_interval: 5
fsync_disable: yes
mail_nfs_storage: yes
lda:
 postmaster_address: postmas...@ghostdive.org
 hostname: mail.ghostdive.org
 mail_plugins: sieve
 mail_plugin_dir: /opt/dovecot/lib/dovecot/lda
 sendmail_path: /opt/postfix26/sbin/sendmail
auth default:
 verbose: yes
 debug: yes
 debug_passwords: yes
 passdb:
   driver: sql
   args: /opt/dovecot/etc/dovecot-sql.conf
 userdb:
   driver: passwd
 userdb:
   driver: sql
   args: /opt/dovecot/etc/dovecot-sql.conf
 socket:
   type: listen
   master:
 path: /var/run/dovecot/auth-master
 mode: 384
 user: mailaccess
 group: mailaccess
--
strace output


close(7)= 0
munmap(0xb7255000, 2101248) = 0
munmap(0xb7c73000, 1052672) = 0
munmap(0xb7d74000, 528384)  = 0
brk(0x84c1000)  = 0x84c1000
time(NULL)  = 1252311837
stat64(/mnt/mail/mail/ingoiwt/.wx-dev/cur, {st_mode=S_IFDIR|0700, 
st_size=3526656, ...}) = 0
pwrite64(8, 
\0\0\...@\0\0\20\0\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\1\0\0\0\200\200\200\214..., 
92, 32720) = 92

pwrite64(8, \200\200\200\207, 4, 32720) = 4
fstat64(8, {st_mode=S_IFREG|0600, st_size=32812, ...}) = 0
pread64(8, 
\200\200\200\...@\0\0\20\0\0\0\0\0\0\0\0$\0\0\0\0\0\0\0\1\0\0\0\200\200\200\214..., 
1024, 32720) = 92

pread64(8, , 1024, 32812) = 0
stat64(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index, 
{st_mode=S_IFREG|0600, st_size=451180, ...}) = 0
fcntl64(9, F_SETLK64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}, 
0xbfb084c8) = 0
pread64(9, 
\7\2x\0\320\0\0\0\f\0\0\0\1\0\0\0\22o\240J\0\0\0\0k\201\237J\316\222\0\0..., 
120, 0) = 120
pwrite64(9, 
$\0\0\0\0\0\0\0\0\0\0\0\0\0\7\0maildir\0\35\303\244J\22o\240J..., 88, 
120) = 88
pwrite64(9, 
\316\222\0\0\10\0\0\0H\2064\0\317\222\0\0\10\0\0\0\234\2064\0\320\222\0\0\10\0\0\0..., 
1032, 451180) = 1032
pwrite64(9, 
\7\2x\0\320\0\0\0\f\0\0\0\1\0\0\0\22o\240J\0\0\0\0k\201\237J$\223\0\0..., 
120, 0) = 120
fcntl64(9, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 
0xbfb08518) = 0

fstat64(8, {st_mode=S_IFREG|0600, st_size=32812, ...}) = 0
umask(066)  = 077
time(NULL)  = 1252311837
lstat64(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log.newlock, 
0xbfb081c0) = -1 ENOENT (No such file or directory)
open(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log.newlock, 
O_RDWR|O_CREAT|O_EXCL|O_LARGEFILE, 0666) = 7

fstat64(7, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
umask(077)  = 066
stat64(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log, 
{st_mode=S_IFREG|0600, st_size=32812, ...}) = 0
write(7, 
\1\2(\0\22o\240J\226\1\0\0\225\1\0\0,\200\0\0\26\303\244J\0\0\0\0\0\0\0\0..., 
40) = 40

fstat64(7, {st_mode=S_IFREG|0600, st_size=40, ...}) = 0
alarm(180)  = 0
fcntl64(7, F_SETLKW64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}, 
0xbfb08388) = 0

alarm(0)= 180
unlink(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log.2) = 0
stat64(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log, 
{st_mode=S_IFREG|0600, st_size=32812, ...}) = 0
link(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log, 
/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log.2) = 0
rename(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log.newlock, 
/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log) = 0
fcntl64(8, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 
0xbfb08498) = 0
fcntl64(7, F_SETLK64, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}, 
0xbfb08658) = 0

alarm(180)  = 0
fcntl64(7, F_SETLKW64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}, 
0xbfb085b8) = 0

alarm(0)= 180
stat64(/var/spool/indexes/ingoiwt/.wx-dev/dovecot.index.log, 
{st_mode=S_IFREG|0600, st_size=40, ...}) = 0

fstat64(7, {st_mode=S_IFREG|0600, st_size=40, ...}) = 0
pread64(7, , 1024, 40)= 0
fstat64(8, {st_mode=S_IFREG|0600, 

[Dovecot] changing location of index files

2009-09-07 Thread suzuki
Hello,

I think changing index files location of mail_location setting
 to reduce disk I/O.
by adding INDEX=/var/index/%d/%1n/%n to mail_location like below,
mail_location = maildir:~/Maildir:INDEX=/var/index/%d/%1n/%n

Will this change cause any problems or data loss to existing pop3 users?
There are only pop3 users on this server.


% dovecot --version
1.1.18

% dovecot -n
# 1.1.18: /usr/local/dovecot/etc/dovecot.conf
# OS: FreeBSD 7.2-STABLE i386
syslog_facility: local4
protocols: pop3
ssl_disable: yes
disable_plaintext_auth: no
login_dir: /usr/local/dovecot/var/run/dovecot/login
login_executable: /usr/local/dovecot/libexec/dovecot/pop3-login
login_greeting: ready
mail_max_userip_connections: 1
verbose_proctitle: yes
first_valid_uid: 10026
last_valid_uid: 10026
first_valid_gid: 10026
last_valid_gid: 10026
mail_privileged_group: vmail
mail_uid: 10026
mail_gid: 10026
mail_location: maildir:~/Maildir
mail_debug: yes
maildir_copy_preserve_filename: yes
mail_executable: /usr/local/dovecot/libexec/dovecot/pop3
mail_plugins: quota
mail_plugin_dir: /usr/local/dovecot/lib/dovecot/pop3
mail_log_max_lines_per_sec: 30
pop3_enable_last: yes
pop3_reuse_xuidl: yes
pop3_lock_session: yes
pop3_client_workarounds: outlook-no-nuls,oe-ns-eoh
lda:
  postmaster_address: postmaster
  mail_plugins: quota
  sendmail_path: /usr/local/sbin/sendmail
  auth_socket_path: /var/run/dovecot/auth-master
auth default:
  verbose: yes
  passdb:
driver: sql
args: /usr/local/dovecot/etc/dovecot-sql.conf
  userdb:
driver: sql
args: /usr/local/dovecot/etc/dovecot-sql.conf
  socket:
type: listen
master:
  path: /var/run/dovecot/auth-master
  mode: 384
  user: vmail
plugin:
  quota: maildir