Re: [Dovecot] indexer-worker Panic

2012-12-18 Thread Timo Sirainen
On Tue, 2012-12-04 at 13:26 -0500, Dave Abrahams wrote:
> I just found this in my logs.  Any idea?
> 
> --8<---cut here---start->8---
> 12/4/12 12:33:43.831 PM dovecot[2660]: indexer-worker(dave): Panic: file 
> mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
> (mailbox_list_is_valid_pattern(_list, name))
> 12/4/12 12:33:43.831 PM dovecot[2660]: indexer-worker(dave): Error: Raw 
> backtrace: 2   libdovecot.0.dylib  0x00010840e6c4 
> default_fatal_finish + 36 -> 3   libdovecot.0.dylib  
> 0x00010840f3e1 i_internal_fatal_handler + 49 -> 4   libdovecot.0.dylib
>   0x00010840e98e i_panic + 158 -> 5   
> libdovecot-storage.0.dylib  0x000108340693 fs_list_get_path + 467 
> -> 6   indexer-worker  0x0001082e7341 
> master_connection_input + 513 -> 7   libdovecot.0.dylib  
> 0x00010841b5
> 12/4/12 12:33:43.831 PM dovecot[2660]: indexer-worker(dave): Error: ce 
> io_loop_call_io + 46 -> 8   libdovecot.0.dylib  
> 0x00010841c737 io_loop_handler_run + 215 -> 9   libdovecot.0.dylib
>   0x00010841b7e8 io_loop_run + 88 -> 10  libdovecot.0.dylib   
>0x0001084079c8 master_service_run + 24 -> 11  indexer-worker   
>0x0001082e7001 main + 321 -> 12  libdyld.dylib 
>   0x7fff93d117e1 start + 0 -> 13  ??? 
> 0x0001 0x0 + 1

Fixed: http://hg.dovecot.org/dovecot-2.1/rev/04704d1eb6c3




[Dovecot] indexer-worker Panic

2012-12-04 Thread Dave Abrahams

I just found this in my logs.  Any idea?

--8<---cut here---start->8---
12/4/12 12:33:43.831 PM dovecot[2660]: indexer-worker(dave): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
12/4/12 12:33:43.831 PM dovecot[2660]: indexer-worker(dave): Error: Raw 
backtrace: 2   libdovecot.0.dylib  0x00010840e6c4 
default_fatal_finish + 36 -> 3   libdovecot.0.dylib  
0x00010840f3e1 i_internal_fatal_handler + 49 -> 4   libdovecot.0.dylib  
0x00010840e98e i_panic + 158 -> 5   libdovecot-storage.0.dylib  
0x000108340693 fs_list_get_path + 467 -> 6   indexer-worker 
 0x0001082e7341 master_connection_input + 513 -> 7   
libdovecot.0.dylib  0x00010841b5
12/4/12 12:33:43.831 PM dovecot[2660]: indexer-worker(dave): Error: ce 
io_loop_call_io + 46 -> 8   libdovecot.0.dylib  
0x00010841c737 io_loop_handler_run + 215 -> 9   libdovecot.0.dylib  
0x00010841b7e8 io_loop_run + 88 -> 10  libdovecot.0.dylib   
   0x0001084079c8 master_service_run + 24 -> 11  indexer-worker 
 0x0001082e7001 main + 321 -> 12  libdyld.dylib 
  0x7fff93d117e1 start + 0 -> 13  ??? 
0x0001 0x0 + 1
12/4/12 12:33:44.035 PM dovecot[2660]: indexer: Error: Indexer worker 
disconnected, discarding 5 requests for dave
12/4/12 12:33:44.036 PM dovecot[2660]: indexer-worker(dave): Fatal: master: 
service(indexer-worker): child 2665 killed with signal 6 (core dumps disabled)
12/4/12 12:34:17.094 PM dovecot[2660]: indexer-worker(dave): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
12/4/12 12:34:17.094 PM dovecot[2660]: indexer-worker(dave): Error: Raw 
backtrace: 2   libdovecot.0.dylib  0x00010d1dd6c4 
default_fatal_finish + 36 -> 3   libdovecot.0.dylib  
0x00010d1de3e1 i_internal_fatal_handler + 49 -> 4   libdovecot.0.dylib  
0x00010d1dd98e i_panic + 158 -> 5   libdovecot-storage.0.dylib  
0x00010d10d693 fs_list_get_path + 467 -> 6   indexer-worker 
 0x00010d0b7341 master_connection_input + 513 -> 7   
libdovecot.0.dylib  0x00010d1ea5
12/4/12 12:34:17.094 PM dovecot[2660]: indexer-worker(dave): Error: ce 
io_loop_call_io + 46 -> 8   libdovecot.0.dylib  
0x00010d1eb737 io_loop_handler_run + 215 -> 9   libdovecot.0.dylib  
0x00010d1ea7e8 io_loop_run + 88 -> 10  libdovecot.0.dylib   
   0x00010d1d69c8 master_service_run + 24 -> 11  indexer-worker 
 0x00010d0b7001 main + 321 -> 12  libdyld.dylib 
  0x7fff93d117e1 start + 0 -> 13  ??? 
0x0001 0x0 + 1
12/4/12 12:34:17.286 PM dovecot[2660]: indexer: Error: Indexer worker 
disconnected, discarding 5 requests for dave
12/4/12 12:34:17.286 PM dovecot[2660]: indexer-worker(dave): Fatal: master: 
service(indexer-worker): child 2677 killed with signal 6 (core dumps disabled)
12/4/12 12:35:48.633 PM dovecot[2660]: indexer-worker(dave): Panic: file 
mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: 
(mailbox_list_is_valid_pattern(_list, name))
12/4/12 12:35:48.633 PM dovecot[2660]: indexer-worker(dave): Error: Raw 
backtrace: 2   libdovecot.0.dylib  0x000107b606c4 
default_fatal_finish + 36 -> 3   libdovecot.0.dylib  
0x000107b613e1 i_internal_fatal_handler + 49 -> 4   libdovecot.0.dylib  
0x000107b6098e i_panic + 158 -> 5   libdovecot-storage.0.dylib  
0x000107a95693 fs_list_get_path + 467 -> 6   indexer-worker 
 0x000107a3d341 master_connection_input + 513 -> 7   
libdovecot.0.dylib  0x000107b6d5
12/4/12 12:35:48.634 PM dovecot[2660]: indexer-worker(dave): Error: ce 
io_loop_call_io + 46 -> 8   libdovecot.0.dylib  
0x000107b6e737 io_loop_handler_run + 215 -> 9   libdovecot.0.dylib  
0x000107b6d7e8 io_loop_run + 88 -> 10  libdovecot.0.dylib   
   0x000107b599c8 master_service_run + 24 -> 11  indexer-worker 
 0x000107a3d001 main + 321 -> 12  libdyld.dylib 
  0x7fff93d117e1 start + 0 -> 13  ??? 
0x0001 0x0 + 1
12/4/12 12:35:48.838 PM dovecot[2660]: indexer: Error: Indexer worker 
disconnected, discarding 5 requests for dave
12/4/12 12:35:48.838 PM dovecot[2660]: indexer-worker(dave): Fatal: master: 
service(indexer-worker): child 2694 killed with signal 6 (core dumps disabled)
--8<---cut here---end--->8---

-- 
Dave

Re: [Dovecot] indexer-worker

2012-07-02 Thread Timo Sirainen
On 28.6.2012, at 13.38, Wojciech Puchar wrote:

>>> 29413 root 1  760 22820K  9204K kqread  1   0:17  5.86% 
>>> indexer-worker
>> 
>> It runs as root while not really doing anything, but when it starts
>> accessing users' files it temporarily drops privileges. This is
>> necessary if users have multiple different UIDs.
> 
> to showed it with root privilege and 60% CPU load+disk I/O when doing text 
> search over not yet indexed folder.

Maybe your ps/kernel shows the process's "real uid" instead of the "effective 
uid".

>> If you have only one UID e.g. vmail, you could set:
>> 
> i'm not sure what you exactly mean.
> 
> I have simplest possible config - mail accounts are unix accounts and mail is 
> at Maildir

Yes, so the problem is that if the same indexer-worker process wants to index 
mails to two different users, it can't drop root privileges completely because 
otherwise it couldn't switch between the users' UIDs.

Anyway, it would be possible to drop the root privileges completely by 
servicing only a single indexing request and then stopping the process. This 
would be worse for performance of course, but it should still be an option. 
I'll probably add to v2.2.

Re: [Dovecot] indexer-worker

2012-06-28 Thread Wojciech Puchar

29413 root 1  760 22820K  9204K kqread  1   0:17  5.86% 
indexer-worker


It runs as root while not really doing anything, but when it starts
accessing users' files it temporarily drops privileges. This is
necessary if users have multiple different UIDs.


to showed it with root privilege and 60% CPU load+disk I/O when doing text 
search over not yet indexed folder.



If you have only one UID e.g. vmail, you could set:


i'm not sure what you exactly mean.

I have simplest possible config - mail accounts are unix accounts and mail 
is at Maildir


my config below


# 2.1.7: /usr/local/etc/dovecot/dovecot.conf
# OS: FreeBSD 8.3-STABLE amd64 
disable_plaintext_auth = no

listen = *
mail_location = maildir:~/Maildir
mail_plugins = fts fts_squat
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
}

passdb {
  args = /usr/local/etc/dovecot/deny-users
  deny = yes
  driver = passwd-file
}
passdb {
  driver = pam
}
plugin {
  fts = squat
  fts_squat = partial=4 full=10
}
protocols = imap
ssl_cert = 

Re: [Dovecot] indexer-worker

2012-06-27 Thread Timo Sirainen
On Thu, 2012-06-28 at 07:54 +0200, Wojciech Puchar wrote:
> why this process (which most probably do squat index/update) runs as root, 
> not - like imap process - as user?
> 
> 29413 root 1  760 22820K  9204K kqread  1   0:17  5.86% 
> indexer-worker

It runs as root while not really doing anything, but when it starts
accessing users' files it temporarily drops privileges. This is
necessary if users have multiple different UIDs.

If you have only one UID e.g. vmail, you could set:

service indexer-worker {
  user = vmail
}

There are a couple of ways to do this automatically whenever it's
possible.. I guess I'll add those to v2.2.




[Dovecot] indexer-worker

2012-06-27 Thread Wojciech Puchar
why this process (which most probably do squat index/update) runs as root, 
not - like imap process - as user?


29413 root 1  760 22820K  9204K kqread  1   0:17  5.86% 
indexer-worker