doveadm sync subscribed folders only

2015-03-13 Thread Jernej Porenta 

Heya,

is there a simple way to use "doveadm sync" for subscribed folders only?

Since the migration from UW-IMAP, our users can actually subscribe to 
non-mbox files and now, when we are pushing new servers out, we are 
migrating our mboxes to dbox format and we would like to migrate only 
subscribed folders.


I know, there is a -m switch which can be used to transfer subscribed 
folders for each user, but it makes it quite unusable with escaping and 
all other problems with such volume, so just wondering if there is a 
simple way to migrate subscribed folders only.


Thank you in advance,

cheers, Jernej

Re: 2.2.15 Panic in mbox_sync_read_next_mail()

2014-11-27 Thread Jernej Porenta 

On 26/11/14 10:09, Matthias Egger wrote:

Hello Hans

On 11/23/2014 05:57 PM, Hans Morten Kind wrote:

On Tue, Nov 04, 2014 at 12:38:02PM +0100, Matthias Egger wrote:

Has someone of you just found any kind of solution to this problem?


We have been running some days with patches 31262a892ba7 and 80ed82a93c1a
from http://hg.dovecot.org/dovecot-2.2/

They are working fine, handling the previously paniced situations smoothly.
Thanks again to the folks at dovecot.org!


Thank you for sharing this. I will give today or tomorrow a look at
these patches.


At least at our side, these patches have fixed a large number of 
segfaults opening mbox files.


Thank you Timo and dovecot team!!!

cheers, Jernej

Re: 2.2.15 Panic in mbox_sync_read_next_mail()

2014-11-01 Thread Jernej Porenta 

On 30/10/14 15:44, Matthias Egger wrote:

On 10/29/2014 02:50 PM, Matthias Egger wrote:

>As soon as i can catch a coredump i will send a gdb output.

Okay, here is the gdb ouput i could catch and some more information
about the system.


System Infos:
SunOS HOSTNAME 5.10 Generic_150400-10 sun4u sparc SUNW,Sun-Fire-V440


We are experiencing the same issues on CentOS 5:
Linux 2.6.18-371.12.1.el5 x86_64

At first, we were reindexing all users with this issue (doveadm index), 
now we are just forcing mailbox resync (doveadm force-mailbox-resync), 
however it is not fixing it :(


We still don't have the pattern, when does this occur (multiple IMAP 
sessions?, IMAP/POP)...


We might revert to 2.2.13, where such problems were way less frequent.

cheers, Jernej

numeric username

2014-09-22 Thread Jernej Porenta 

Heya,

does dovecot support "numeric only" usernames (in format "^0-9+$")?

We are unable to deliver emails to such usernames via LMTP:
Sep 22 08:41:05 razor4 postfix/lmtp[2120]: D4FD91CDC11F: 
to=<231...@example.org>, relay=example.org[private/dovecot-lmtp], 
delay=217653, delays=217653/0.01/0.02/0, dsn=4.2.0, status=deferred 
(host example.org[private/dovecot-lmtp] said: 451 4.2.0 
<231...@example.org> Permission denied (in reply to end of DATA command))


On normal username format (^[a-z][-a-z0-9]+$) everything is working fine.

Permissions are the same as on other accounts:
# ls -la .mailbox
-rw---+ 1 231588 users 0 Sep 19 20:13 .mailbox

Using dovecot 2.2.13 with local accounts on RHEL and (relevant) configs:
auth_username_chars = 
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@

auth_username_format = %Ln
auth_username_translation =
...
lmtp_address_translate =
...
passdb {
  args = cache_key=%u%s dovecot
  default_fields =
  deny = no
  driver = pam
  master = no
  name =
  override_fields =
  pass = no
  result_failure = continue
  result_internalfail = continue
  result_success = return-ok
  skip = never
}
...
service lmtp {
  chroot =
  client_limit = 1
  drop_priv_before_exec = no
  executable = lmtp
  extra_groups =
  group =
  idle_kill = 0
  privileged_group =
  process_limit = 0
  process_min_avail = 5
  protocol = lmtp
  service_count = 0
  type =
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
  unix_listener lmtp {
group =
mode = 0666
user =
  }
  user =
  vsz_limit = 18446744073709551615 B
}

userdb {
  args =
  default_fields =
  driver = passwd
  name =
  override_fields =
  result_failure = continue
  result_internalfail = continue
  result_success = return-ok
  skip = never
}

Thank you in advance,

cheers, Jernej

[Dovecot] ID command and dovecot

2014-02-19 Thread Jernej Porenta 

Heya,

we are seeing some strange issues with IMAP ID command using latest 
Roundcube builds on dovecot 2.2.10 responding "Input buffer full, aborting".


Roundcube webmail connects to IMAP server and issues ID command first 
with some long arguments to ID command, however shorter than 1024 
octets, which are specified in RFC2971 for ID IMAP extension.


The situation is easily reproducible (see http://pastebin.com/q4HX4wqb):
# telnet hostname 143
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE 
IDLE AUTH=PLAIN AUTH=LOGIN] Server ready.
. ID ("name" "Roundcube" "version" "1.0-git" "php" "5.3.3" "os" "Linux" 
"command" "<$long_less_than_1024_octets_string>")

* BYE Input buffer full, aborting

However, this problem does not occur when user is already logged in.

RFC2971 is saying "Field strings MUST NOT be longer than 30 octets. 
Value strings MUST NOT be longer than 1024 octets. Implementations MUST 
NOT send more than 30 field-value pairs."


IMHO, dovecot is handling ID command before successful login 
successfully only if the length of the whole command is less than 1024 
octets (exactly 861 octets is working fine, over that "Input buffer 
full" occurs).


I tried finding the reference in the source code, however I am unable to 
find where this limit actually occurs (maybe IMAP_ARG_STRING 
imap_arg->type?)


thank you in advance,

regards, Jernej

[Dovecot] 2.2.10 - core dumps

2014-01-13 Thread Jernej Porenta 

Heya,

We are occasionally seeing core dumps with dovecot 2.2.10, which are 
mostly like this:
Jan 11 18:31:08 hostname dovecot: imap(username): Panic: file 
mail-index-transaction-update.c: line 19 
(mail_index_transaction_lookup): assertion failed: (seq >= 
t->first_new_seq && seq <= t->last_new_seq)
Jan 11 18:31:08 hostname dovecot: imap(username): Error: Raw backtrace: 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot.so.0 [0x2b657523aa3a] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot.so.0 [0x2b657523aaa6] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot.so.0 [0x2b6575239b83] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0 [0x2b6574f83458] 
-> /opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0 
[0x2b6574f86458] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0(mail_cache_decision_state_update+0xbe) 
[0x2b6574f7605e] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0(mail_cache_lookup_headers+0x87) 
[0x2b6574f77ba7] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0 [0x2b6574f64919] 
-> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0(index_mail_get_first_header+0x50) 
[0x2b6574f64e80] -> 
/opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0 [0x2b6574f5fda1] 
-> /opt/dovecot-2.2.10/lib/dovecot/libdovecot-storage.so.0 
[0x2b6574f5fee9] -> /opt/dovecot-2.2.10/lib/dovecot/libdove
Jan 11 18:31:08 hostname dovecot: imap(username): Fatal: master: 
service(imap): child 13548 killed with signal 6 (core dumps disabled)


# dovecot -n is available here: http://pastebin.com/RE91aRi5

other raw traces are available here: http://pastebin.com/M1S33NYn

Would you recommend deleting all the index files or rather capturing 
core dump file for further analysis? Or maybe something completely 
different ;)


thank you in advance,

cheers, Jernej

Re: [Dovecot] bug uni_utf8_str_is_valid(vname)

2012-03-20 Thread Jernej Porenta 

On Mar 19, 2012, at 2:53 PM, Timo Sirainen wrote:

> On Mon, 2012-03-19 at 14:27 +0100, Jernej Porenta wrote:
>>> Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: 
>>> line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
>>> 
>>> It is the same. We will try 2.1.3 today and report the results...
> 
>> The home directory of the username is tar.gzipped here: 
>> http://www2.arnes.si/~krklubsls13/username.tar.gz
> 
> Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/c77fbfce438d
> 


Confirmed working…

Thank you again, cheers, Jernej

Re: [Dovecot] bug uni_utf8_str_is_valid(vname)

2012-03-19 Thread Jernej Porenta 

On Mar 19, 2012, at 10:58 AM, Jernej Porenta wrote:

> Mar 19 10:56:35 server dovecot: imap-login: Login: user=, method=PLAIN, 
> rip=193.2.1.110, lip=193.2.1.83, mpid=14732, secured
> Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: line 
> 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
> Mar 19 10:56:40 server dovecot: imap(user): Error: Raw backtrace: 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a91610870] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a916108c6] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a9160fd83] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9134f0e5] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c6cc] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c846] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1)
>  [0x2b7a9135d241] -> 
> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec)
>  [0x2b7a9135619c] -> dovecot/imap [0x40bbb6] -> 
> dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) 
> [0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap 
> [0x40f4ed] -> dovecot/imap [0x40f5a2] -> 
> dovecot/imap(client_handle_input+0x3f) [0x40f6ef] -> 
> dovecot/imap(client_input+0x62) [0x410072] -> /opt
> Mar 19 10:56:40 server dovecot: imap(user): Fatal: master: service(imap): 
> child 14732 killed with signal 6 (core dumps disabled)
> 
> It is the same. We will try 2.1.3 today and report the results...


Same thing with 2.1.3 (. LIST "" "*"):
Mar 19 14:08:59 server dovecot: imap-login: Login: user=, 
method=PLAIN, rip=193.2.1.110, lip=193.2.1.83, mpid=28438, secured
Mar 19 14:09:04 server dovecot: imap(username): Panic: file mail-storage.c: 
line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
Mar 19 14:09:04 server dovecot: imap(username): Error: Raw backtrace: 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot.so.0 [0x2ae071811870] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot.so.0 [0x2ae0718118c6] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot.so.0 [0x2ae071810d83] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0 [0x2ae0715500c5] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0 [0x2ae07155d6ac] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0 [0x2ae07155d826] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) 
[0x2ae07155e221] -> 
/opt/dovecot-2.1.3/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec)
 [0x2ae07155717c] -> dovecot/imap [0x40bbb6] -> 
dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) 
[0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap 
[0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) 
[0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt
Mar 19 14:09:04 server dovecot: imap(username): Fatal: master: service(imap): 
child 28438 killed with signal 6 (core dumps disabled)

The home directory of the username is tar.gzipped here: 
http://www2.arnes.si/~krklubsls13/username.tar.gz

# dovecot -n
# 2.1.3: /opt/dovecot-2.1.3/etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-274.17.1.el5 x86_64 CentOS release 5.7 (Final) 
default_login_user = nobody
disable_plaintext_auth = no
login_greeting = Server ready.
login_trusted_networks = x.y.z.p/32
mail_location = mbox:~/:INBOX=%h/.mailbox:INDEX=/opt/dovecot2-indexes/%1u/%u
mail_plugins = quota
mail_privileged_group = dovecot
mbox_write_locks = fcntl
namespace inbox {
  inbox = yes
  location = 
  prefix = 
  separator = /
  type = private
}
passdb {
  args = session=yes dovecot
  driver = pam
}
plugin {
  quota = fs
}
protocols = imap pop3
service imap-login {
  inet_listener imap {
port = 143
  }
  inet_listener imaps {
ssl = no
  }
}
service pop3-login {
  inet_listener pop3 {
port = 110
  }
  inet_listener pop3s {
ssl = no
  }
}
ssl = no
userdb {
  driver = passwd
}
protocol imap {
  imap_client_workarounds = delay-newmail tb-extra-mailbox-sep
  mail_max_userip_connections = 15
  mail_plugins = quota imap_quota
}
protocol pop3 {
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}

Regards, Jernej

Re: [Dovecot] bug uni_utf8_str_is_valid(vname)

2012-03-19 Thread Jernej Porenta 

On Mar 16, 2012, at 2:14 PM, Timo Sirainen wrote:

> On 16.3.2012, at 11.09, Jernej Porenta wrote:
> 
>>>> Mar  6 13:37:17 machine dovecot: imap(username): Panic: file
>>>> mail-storage.c: line 628 (mailbox_alloc): assertion failed:
>>>> (uni_utf8_str_is_valid(vname))
>>> ..
>> We tried version 2.1.2, which unfortunately does not fix the issue with 
>> weird characters. 
>> 
>> Whenever . LIST "" "*" is issued, dovecot crashes:
>> Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 
>> [0x2b54671eb870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 
>> [0x2b54671eb8c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 
>> [0x2b54671ead83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 
>> [0x2b5466f2a0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 
>> [0x2b5466f376cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 
>> [0x2b5466f37846] -> 
>> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1)
>>  [0x2b5466f38241] -> /opt/dovecot
> 
> I don't think this is the same Panic as the original one? What is the Panic 
> message now?


Mar 19 10:56:35 server dovecot: imap-login: Login: user=, method=PLAIN, 
rip=193.2.1.110, lip=193.2.1.83, mpid=14732, secured
Mar 19 10:56:40 server dovecot: imap(user): Panic: file mail-storage.c: line 
628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
Mar 19 10:56:40 server dovecot: imap(user): Error: Raw backtrace: 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a91610870] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a916108c6] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 [0x2b7a9160fd83] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9134f0e5] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c6cc] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 [0x2b7a9135c846] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) 
[0x2b7a9135d241] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec)
 [0x2b7a9135619c] -> dovecot/imap [0x40bbb6] -> 
dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) 
[0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap 
[0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) 
[0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt
Mar 19 10:56:40 server dovecot: imap(user): Fatal: master: service(imap): child 
14732 killed with signal 6 (core dumps disabled)

It is the same. We will try 2.1.3 today and report the results...

Regards, Jernej

Re: [Dovecot] bug uni_utf8_str_is_valid(vname)

2012-03-16 Thread Jernej Porenta 

On Mar 14, 2012, at 3:29 PM, Timo Sirainen wrote:

> On Tue, 2012-03-06 at 14:28 +0100, Jernej Porenta wrote:
>> Heya,
>> 
>> We are expiriencing issues with dovecot 2.1.1 on Linux with weird
>> filenames in home directory of username. We are using mbox IMAP
>> folders, with no special changes (mail_location = mbox:~/:INBOX=%
>> h/.mailbox).
>> 
>> Mar  6 13:37:17 machine dovecot: imap(username): Panic: file
>> mail-storage.c: line 628 (mailbox_alloc): assertion failed:
>> (uni_utf8_str_is_valid(vname))
> ..
>> AFAIK, the problem lies in processing the file list of home folder,
>> which can contain filenames that do not have proper UTF-8 encoding of
>> filenames, which causes dovecot to crash. 
> 
> Yes, Dovecot shouldn't crash even if there are non-UTF8 mailboxes. This
> should fix it by renaming such mailboxes:
> http://hg.dovecot.org/dovecot-2.1/rev/c077ca9bc306

We tried version 2.1.2, which unfortunately does not fix the issue with weird 
characters. 

Whenever . LIST "" "*" is issued, dovecot crashes:
Error: Raw backtrace: /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 
[0x2b54671eb870] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 
[0x2b54671eb8c6] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot.so.0 
[0x2b54671ead83] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 
[0x2b5466f2a0e5] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 
[0x2b5466f376cc] -> /opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0 
[0x2b5466f37846] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_init+0x4b1) 
[0x2b5466f38241] -> 
/opt/dovecot-2.1.2/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_init_multiple+0xec)
 [0x2b5466f3119c] -> dovecot/imap [0x40bbb6] -> 
dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) 
[0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410497] -> dovecot/imap 
[0x40f4ed] -> dovecot/imap [0x40f5a2] -> dovecot/imap(client_handle_input+0x3f) 
[0x40f6ef] -> dovecot/imap(client_input+0x62) [0x410072] -> /opt

Any clues?

> 
>> On the other hand, UTF-8 filenames created on the system by hand
>> (using touch), are not displayed in IMAP LIST command (sample is
>> included in the folder structure; single letter file).
> 
> This is a bit trickier problem. The mailbox names are currently stored
> in filesystem as IMAP's modified-UTF7. So it's not really even currently
> supposed to work, although it's not very nice that the mailboxes aren't
> visible either. Maybe I'll do something smart in future for this, like
> allowing both mUTF-7 and UTF-8 and remembering per-mailbox which
> formatting it is in.


I think we can leave this issue out, since I don't believe the users will be 
creating folders directly from interactive SSH sessions and rather use IMAP to 
create folders. So, this use-case is not very likely to occur. 

Thank you for your help...

Cheers, Jernej

[Dovecot] bug uni_utf8_str_is_valid(vname)

2012-03-06 Thread Jernej Porenta 
Heya,

We are expiriencing issues with dovecot 2.1.1 on Linux with weird filenames in 
home directory of username. We are using mbox IMAP folders, with no special 
changes (mail_location = mbox:~/:INBOX=%h/.mailbox).

Mar  6 13:37:17 machine dovecot: imap(username): Panic: file mail-storage.c: 
line 628 (mailbox_alloc): assertion failed: (uni_utf8_str_is_valid(vname))
Mar  6 13:37:17 machine dovecot: imap(username): Error: Raw backtrace: 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot.so.0 [0x2ba41cb79450] -> 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot.so.0 [0x2ba41cb794a6] -> 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot.so.0 [0x2ba41cb78963] -> 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0 [0x2ba41c87ebd5] -> 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0 [0x2ba41c88c12c] -> 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0(fs_list_iter_next+0x1b4) 
[0x2ba41c88c494] -> /opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0 
[0x2ba41c885342] -> 
/opt/dovecot-2.1.1/lib/dovecot/libdovecot-storage.so.0(mailbox_list_iter_next+0x234)
 [0x2ba41c885604] -> dovecot/imap [0x40b2d1] -> 
dovecot/imap(cmd_list_full+0x520) [0x40c1f0] -> dovecot/imap(cmd_list+0xb) 
[0x40c3eb] -> dovecot/imap(command_exec+0x37) [0x410427] -> dovecot/imap 
[0x40f4cd] -> dovecot/imap [0x40f582] -> dovecot/imap(client_handle_input+0x3f) 
[0x40f6cf] -> dovecot/imap(client_input+0x62) [0x410052] -> /opt/dovecot
Mar  6 13:37:17 machine dovecot: imap(username): Fatal: master: service(imap): 
child 20873 killed with signal 6 (core dumps disabled)

The bug is reproducible by using home folder structure available from: 
http://bit.ly/x8pTXS

AFAIK, the problem lies in processing the file list of home folder, which can 
contain filenames that do not have proper UTF-8 encoding of filenames, which 
causes dovecot to crash. On the other hand, UTF-8 filenames created on the 
system by hand (using touch), are not displayed in IMAP LIST command (sample is 
included in the folder structure; single letter file).

Cheers, Jernej

Re: [Dovecot] corrupted mbox

2010-03-26 Thread Jernej Porenta 
On Mar 26, 2010, at 1:39 PM, Timo Sirainen wrote:
> On Fri, 2010-03-26 at 13:27 +0100, Jernej Porenta wrote:
>> Heya,
>> 
>> we have two IMAP (UW-imap and dovecot) servers accessing the same mbox
>> files for our users (don't ask why) and sometimes the UW-imap corrupts
>> the mbox file. The mbox looks just like normal mbox except it does not
>> have the proper 'From ' beginning of the file (usually there are some
>> random tidbits in the begining of the file, which can be easily
>> stripped to recover the working mbox).
> 
> The corruption shouldn't be happening if both use the same locking
> mechanisms. I think UW-IMAP uses dotlock + flock. See if changing
> Dovecot to use those avoids the corruption?

The corruptions were happening even before we started using dovecot, so I 
believe the problem exists in old uw-imap release that we (have to) use, 
because of our custom made patches. We will be eliminating uw-imap in April, 
but we have to take care of the corrupted mboxes before that ;)

>> The problem is that if we use dovecot for POP3 server, the dovecot
>> logs the mbox corruption just fine:
>> POP3(username): Couldn't init INBOX: Mailbox isn't a valid mbox file
>> 
>> But if we use IMAP for the same mailbox, dovecot opens the file and
>> does not report any corruption, the user on the other hand sees just
>> empty mailbox.
> 
> What Dovecot version? The IMAP code is pretty much the same as POP3, so
> I don't really see why IMAP wouldn't be noticing it.

Dovecot version is 1.2.11 with almost default settings.

Here are the logs for the user with corrupted mbox, who tried POP3 first and 
afterwards IMAP:
Mar 19 10:34:54 machine dovecot: pop3-login: Login: user=, 
method=PLAIN, rip=X.X.X.Z, lip=X.X.X.W, secured
Mar 19 10:34:54 machine dovecot: POP3(username): Effective uid=15657, gid=3400, 
home=/mnt/./39/username
Mar 19 10:34:54 machine dovecot: POP3(username): mbox: 
data=~/:INBOX=~/.mailbox:INDEX=/mnt/dovecot-index/l/username
Mar 19 10:34:54 machine dovecot: POP3(username): fs: root=/mnt/./39/username, 
index=/mnt/dovecot-index/l/username, control=, 
inbox=/mnt/./39/username/.mailbox/
Mar 19 10:34:54 machine dovecot: POP3(username): Couldn't init INBOX: Mailbox 
isn't a valid mbox file
Mar 19 10:34:54 machine dovecot: POP3(username): Mailbox init failed top=0/0, 
retr=0/0, del=0/0, size=0
Mar 19 10:43:18 machine dovecot: imap-login: Login: user=, 
method=PLAIN, rip=X.X.X.Y, lip=X.X.X.W
Mar 19 10:43:18 machine dovecot: IMAP(username): Loading modules from 
directory: /opt/dovecot/lib/dovecot/imap
Mar 19 10:43:18 machine dovecot: IMAP(username): Module loaded: 
/opt/dovecot/lib/dovecot/imap/lib10_quota_plugin.so
Mar 19 10:43:18 machine dovecot: IMAP(username): Module loaded: 
/opt/dovecot/lib/dovecot/imap/lib11_imap_quota_plugin.so
Mar 19 10:43:18 machine dovecot: IMAP(username): Effective uid=15657, gid=3400, 
home=/mnt/./39/username
Mar 19 10:43:18 machine dovecot: IMAP(username): Quota root: name= backend=fs 
args=
Mar 19 10:43:18 machine dovecot: IMAP(username): mbox: 
data=~/:INBOX=~/.mailbox:INDEX=/mnt/dovecot-index/l/username
Mar 19 10:43:18 machine dovecot: IMAP(username): fs: root=/mnt/./39/username, 
index=/mnt/dovecot-index/l/username, control=, inbox=/mnt/./39/username/.mailbox
Mar 19 10:43:18 machine dovecot: IMAP(username): fs quota add storage dir = 
/mnt/./39/username
Mar 19 10:43:18 machine dovecot: IMAP(username): fs quota block device = 
/dev/dsk/c7t600D023001EBFD22D8275C00d0s2
Mar 19 10:43:18 machine dovecot: IMAP(username): fs quota mount point = /mnt/39
Mar 19 10:43:18 machine dovecot: IMAP(username): Namespace : Using permissions 
from /mnt/./39/username: mode=0711 gid=-1
Mar 19 10:43:18 machine dovecot: IMAP(username): Disconnected: Logged mnt 
bytes=254/909

I would be happy if IMAP would drop a line about the corrupted mbox... :)

thanks in advance, regards, J.

[Dovecot] corrupted mbox

2010-03-26 Thread Jernej Porenta 
Heya,

we have two IMAP (UW-imap and dovecot) servers accessing the same mbox files 
for our users (don't ask why) and sometimes the UW-imap corrupts the mbox file. 
The mbox looks just like normal mbox except it does not have the proper 'From ' 
beginning of the file (usually there are some random tidbits in the begining of 
the file, which can be easily stripped to recover the working mbox).

The problem is that if we use dovecot for POP3 server, the dovecot logs the 
mbox corruption just fine:
POP3(username): Couldn't init INBOX: Mailbox isn't a valid mbox file

But if we use IMAP for the same mailbox, dovecot opens the file and does not 
report any corruption, the user on the other hand sees just empty mailbox.

So I was wondering if there are any possibilities to report mbox corruption 
with IMAP as well?

thanks in advance,
regards, Jernej

[Dovecot] qpopper to dovecot migration

2009-12-21 Thread Jernej Porenta 
Heya,

We are in the process of migrating from qpopper to dovecot as POP3 server (and 
IMAP as well) and we hit the UIDL issue with dovecot. Qpopper uses its own 
algorithm to create UIDL values and if you set it up to write them down in the 
mbox file as X-UIDL, then dovecot can use them (pop_uidl_reuse = yes).

But if you don't use that feature from qpopper, than X-UIDLs are missing and 
there is no way to create the same UIDL values. So, I was wondering if anyone 
else hit that issue and maybe knows how to address it (writing an extension or 
something)?

Thanks in advance, 

regards, Jernej

Re: [Dovecot] compiling issue 1.2.6 - Solaris

2009-10-06 Thread Jernej Porenta 

Timo Sirainen wrote:

On Oct 6, 2009, at 9:00 AM, Bruce Bodger wrote:

Undefined symbols:
 "_SSL_get_current_compression", referenced from:
 _ssl_proxy_get_security_string in 
liblogin-common.a(ssl-proxy-openssl.o)

 "_SSL_COMP_get_name", referenced from:
 _ssl_proxy_get_security_string in 
liblogin-common.a(ssl-proxy-openssl.o)


What OpenSSL version do you have? I thought those compression functions 
were new enough that everyone would have them by now..


Mine is "OpenSSL 0.9.7d" and "OpenSSL 0.9.7i" which might be a little 
old :), but I had no troubles since that...


Anyway, I will build a new one and try to link dovecot with that...

thanks you,

regards, Jernej

[Dovecot] compiling issue 1.2.6 - Solaris

2009-10-06 Thread Jernej Porenta 

Heya,

I am expiriencing compiling issues on Solaris 8 and Solaris 10 boxes 
with dovecot 1.2.6. On Solaris 8 the compiler is gcc 64bit 3.2.2, on 
Solaris 10 gcc 3.4.3.


On both systems the compiling fails with:
/bin/bash ../../libtool --tag=CC--mode=link gcc  -std=gnu99 -g -O2 
-Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith 
-Wchar-subscripts -Wformat=2 -Wbad-function-cast 
-L/opt/gcc-3.1-64bit/lib -R/opt/gcc-3.1-64bit/lib -L/usr/local/ssl/lib 
-R/usr/local/ssl/lib -L/opt/kerberos-heimdal/lib 
-R/opt/kerberos-heimdal/lib -o imap-login client.o client-authenticate.o 
 imap-proxy.o ../login-common/liblogin-common.a  ../lib-imap/libimap.a 
 ../lib-auth/libauth.a  ../lib/liblib.a  -lssl -lcrypto -ldl -lrt -lnsl 
-lsocket  -lrt
libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes 
-Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 
-Wbad-function-cast -o imap-login client.o client-authenticate.o 
imap-proxy.o  -L/opt/gcc-3.1-64bit/lib -L/usr/local/ssl/lib 
-L/opt/kerberos-heimdal/lib ../login-common/liblogin-common.a 
../lib-imap/libimap.a ../lib-auth/libauth.a ../lib/liblib.a -lssl 
-lcrypto -ldl -lnsl -lsocket -lrt -R/opt/gcc-3.1-64bit/lib 
-R/usr/local/ssl/lib -R/opt/kerberos-heimdal/lib

Undefined   first referenced
 symbol in file
SSL_COMP_get_name 
../login-common/liblogin-common.a(ssl-proxy-openssl.o)
SSL_get_current_compression 
../login-common/liblogin-common.a(ssl-proxy-openssl.o)

ld: fatal: Symbol referencing errors. No output written to imap-login
collect2: ld returned 1 exit status

The configure script is the same on both systems:
CPPFLAGS='-I/usr/local/ssl/include -I/opt/kerberos-heimdal/include' 
LDFLAGS='-L/usr/local/ssl/lib -R/usr/local/ssl/lib 
-L/opt/kerberos-heimdal/lib -R/opt/kerberos-heimdal/lib' ./configure 
--prefix=/opt/dovecot --with-ssldir=/usr/local/ssl/certs 
--with-ssl=openssl --without-passwd-file --with-passwd --with-shadow 
--without-bsdauth --without-static-userdb --without-prefetch-userdb 
--without-checkpassword --with-gssapi=yes --without-pgsql --without-sql 
--without-ldap --without-vpopmail --without-sia --enable-header-install


Install prefix . : /opt/dovecot
File offsets ... : 64bit
I/O polling  : poll
I/O notifys  : none
SSL  : yes (OpenSSL)
GSSAPI . : yes
passdbs  : passwd shadow pam
 : -passwd-file -checkpassword -bsdauth -sia -ldap -sql 
-vpopmail

userdbs  : passwd
 : -static -prefetch -passwd-file -checkpassword -ldap 
-sql -vpopmail -nss

SQL drivers  :
 : -pgsql -mysql -sqlite

Version 1.2.4 compiles with no problems...

regards, Jernej

[Dovecot] namespaces / list=no crash

2009-07-28 Thread Jernej Porenta 

Hey everyone,

I am trying to migrate old UW-IMAP setup to dovecot 1.2.2 using 
namespaces as recommended for UW-IMAP backwards compatibility:

http://wiki.dovecot.org/Namespaces

I am trying to use these settings:
mail_location = mbox:~/:INBOX=~/.mailbox
# default namespace
namespace private {
  separator = /
  prefix =
  inbox = yes
}
# for backwards compatibility:
namespace private {
  separator = /
  prefix = mail/
  hidden = yes
  list = no   # for v1.1+
  subscriptions = no
}

With upper settings, dovecot crashes upon login. (GDB trace is here: 
http://www2.arnes.si/~krklubsls13/dovecot-gdb.txt).


If I change "list = no" to list = yes|children, everything is working fine.

This settings were working fine with dovecot 1.1, so I don't know what 
is the deal...


dovecot -n : http://www2.arnes.si/~krklubsls13/dovecot-conf.txt

thanks in advance,
regards Jernej

Re: [Dovecot] v1.2.2 released

2009-07-27 Thread Jernej Porenta 

Timo Sirainen wrote:

On Mon, 2009-07-27 at 15:02 +0200, Jernej Porenta wrote:

Timo Sirainen wrote:

Another issue when building 1.2.2 that wasn't there with 1.2.1 is that
"-lsocket" seems
to be missing causing linking errors. One example:

Fixed: http://hg.dovecot.org/dovecot-1.2/rev/cd29b745c8dd

It is not working for me.


It updated configure.in, so configure needs to be regenerated. Did you
run autogen.sh / autoreconf after patching?


No, missed that step :(

Thanks for your time..

cheers, Jernej

Re: [Dovecot] v1.2.2 released

2009-07-27 Thread Jernej Porenta 

Timo Sirainen wrote:

Another issue when building 1.2.2 that wasn't there with 1.2.1 is that
"-lsocket" seems
to be missing causing linking errors. One example:


Fixed: http://hg.dovecot.org/dovecot-1.2/rev/cd29b745c8dd


It is not working for me. "-lrt" is already set before checking 
clock_gettime. Before that, "-lnsl -lsocket" are set, but after lrt 
checking, they gone missing...


configure:19082: checking for clock_gettime in -lrt
configure:19117: gcc -o conftest -g -O2 -Wall -W -Wmissing-prototypes 
-Wmissing-declarations -Wpointer-arith -
Wchar-subscripts -Wformat=2 -Wbad-function-cast -I/usr/local/ssl/include 
-I/opt/kerberos-heimdal/include -L/us
r/local/ssl/lib -R/usr/local/ssl/lib -L/opt/kerberos-heimdal/lib 
-R/opt/kerberos-heimdal/lib conftest.c -lrt

-lrt -lnsl -lsocket  >&5
configure:19124: $? = 0
configure:19145: result: yes
configure:19264: checking for strtoimax
configure:19291: gcc -o conftest -g -O2 -Wall -W -Wmissing-prototypes 
-Wmissing-declarations -Wpointer-arith -
Wchar-subscripts -Wformat=2 -Wbad-function-cast -I/usr/local/ssl/include 
-I/opt/kerberos-heimdal/include -L/us
r/local/ssl/lib -R/usr/local/ssl/lib -L/opt/kerberos-heimdal/lib 
-R/opt/kerberos-heimdal/lib conftest.c -lrt >

&5
conftest.c: In function `main':
conftest.c:70: warning: implicit declaration of function `strtoimax'
Undefined   first referenced
 symbol in file
strtoimax   /var/tmp/ccsnXIjO.o
ld: fatal: Symbol referencing errors. No output written to conftest


OS: Solaris 8, compiler gcc version 2.95.3 20010315 (release), against 
heimdal kerberos. Compile string:
 CPPFLAGS='-I/usr/local/ssl/include -I/opt/kerberos-heimdal/include' 
LDFLAGS='-L/usr/local/ssl/lib -R/usr/local/ssl/lib 
-L/opt/kerberos-heimdal/lib -R/opt/kerberos-heimdal/lib' ./configure 
--prefix=/opt/dovecot --with-ssldir=/usr/local/ssl/certs 
--with-ssl=openssl --without-passwd-file --with-passwd --with-shadow 
--without-bsdauth --without-static-userdb --without-prefetch-userdb 
--without-checkpassword --with-gssapi=yes --without-pgsql --without-sql 
--without-ldap --without-vpopmail --without-sia


regards, Jernej

[Dovecot] UW-IMAP migration to namespaces

2009-04-17 Thread Jernej Porenta 

Hello,

I am trying to migrate existing UW-IMAP installation to dovecot 1.1.14 
and I am having troubles with namespaces.


As you know UW-IMAP allows full system access to home directory of a 
user and you can set up dovecot to do that as well, but I would like to 
use namespaces.


If I set my namespaces like this:
namespace private {
  separator = /
  prefix =
  inbox = yes
  location = mbox:~/:INBOX=~/.mailbox
  hidden = yes
  list = yes
}

namespace private {
  separator = /
  prefix = mail/
  location = mbox:~/mail
  prefix = mail/
  hidden = yes
}

Everything is working fine, but I am seeing also all other files in home 
directory of a user, but this is annoying if you have users with 
public_html folder with millions of files in it. Dovecot indexing makes 
index.log file for every file seen in namespace, so you have several 
million index files for files that are not mailboxes.


So I was wondering is there a way to skip certain directories in 
namespaces or a way to setup namespaces in a way, that you would have:

- INBOX in home directory
- some folders in home directory (via .subscriptions)
- some folders in %h/mail/ directory
- and not seeing all other files/directories in home dir

Thanks in advance,

regards, Jernej

[Dovecot] failed dovecot 1.2rc2 build Solaris 8, gcc 2.95.3

2009-04-09 Thread Jernej Porenta 

Hello,

I trying to build dovecot 1.2rc2 on ancient Solaris 8 with gcc 2.95.3 
and it fails with:


gcc -DHAVE_CONFIG_H -I. -I../..  -I../../src/lib 
-I../../src/lib-settings  -DSYSCONFDIR=\""/opt/dovecot/etc"\" 
-DPKG_RUNDIR=\""/opt/dovecot/var/run/dovecot"\" 
-DPKG_STATEDIR=\""/opt/dovecot/var/lib/dovecot"\" 
-DPKG_LIBEXECDIR=\""/opt/dovecot/libexec/dovecot"\" 
-DMODULEDIR=\""/opt/dovecot/lib/dovecot"\" 
-DSSLDIR=\""/usr/local/ssl/certs\""  -I/usr/local/ssl/include 
-I/opt/kerberos-heimdal/include  -g -O2 -Wall -W -Wmissing-prototypes 
-Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 
-Wbad-function-cast -c login-process.c

In file included from ../../src/lib/lib.h:28,
 from common.h:6,
 from login-process.c:3:
../../src/lib/data-stack.h:73: warning: `pure' attribute directive ignored
In file included from ../../src/lib/lib.h:39,
 from common.h:6,
 from login-process.c:3:
../../src/lib/strfuncs.h:44: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:54: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:55: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:56: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:57: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:58: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:59: warning: `pure' attribute directive ignored
../../src/lib/strfuncs.h:77: warning: `pure' attribute directive ignored
In file included from master-settings.h:4,
 from common.h:7,
 from login-process.c:3:
../../src/lib/network.h:114: warning: `pure' attribute directive ignored
../../src/lib/network.h:115: warning: `pure' attribute directive ignored
../../src/lib/network.h:123: warning: `pure' attribute directive ignored
In file included from ../../src/lib/array.h:33,
 from login-process.c:4:
../../src/lib/buffer.h:36: warning: `pure' attribute directive ignored
../../src/lib/buffer.h:88: warning: `pure' attribute directive ignored
../../src/lib/buffer.h:106: warning: `pure' attribute directive ignored
In file included from login-process.c:4:
../../src/lib/array.h:96: warning: `pure' attribute directive ignored
../../src/lib/array.h:161: warning: `pure' attribute directive ignored
../../src/lib/array.h:222: warning: `pure' attribute directive ignored
../../src/lib/array.h:229: warning: `pure' attribute directive ignored
In file included from login-process.c:6:
../../src/lib/hash.h:24: warning: `pure' attribute directive ignored
../../src/lib/hash.h:35: warning: `pure' attribute directive ignored
../../src/lib/hash.h:54: warning: `pure' attribute directive ignored
../../src/lib/hash.h:55: warning: `pure' attribute directive ignored
In file included from login-process.c:8:
../../src/lib/ostream.h:72: warning: `pure' attribute directive ignored
login-process.c:52: field `data' has incomplete type
*** Error code 1
make: Fatal error: Command failed for target `login-process.o'
Current working directory /opt/src/dovecot-1.2.rc2/src/master
*** Error code 1

configure looks like this:
CPPFLAGS='-I/usr/local/ssl/include -I/opt/kerberos-heimdal/include' 
LDFLAGS='-L/usr/local/ssl/lib -R/usr/local/ssl/lib 
-L/opt/kerberos-heimdal/lib -R/opt/kerberos-heimdal/lib' ./configure 
--prefix=/opt/dovecot --enable-ssl --with-ssldir=/usr/local/ssl/certs 
--with-ssl=openssl --without-passwd-file --with-passwd --with-shadow 
--without-bsdauth --without-static-userdb --without-prefetch-userdb 
--without-checkpassword --with-gssapi=yes --without-pgsql --without-sql 
--without-ldap --without-vpopmail --without-sia


The build goes fine with gcc 3.2.2.

Dovecot 1.1 builds fine with 2.95.3. Are there any requirements for GCC 
version?


regards, Jernej