Re: [Dovecot] record points outside file error with dovecot revision 9116:9ae55b68cf61
On Tue, 2 Jun 2009, Timo Sirainen wrote: On Tue, 2009-06-02 at 13:24 -0400, jsie...@psu.edu wrote: dovecot: IMAP(testuser): Corrupted index cache file /full/path/ dovecot.index.cache: record points outside file So you're using AIX? Do you also happen to use NFS? Can you reproduce this error by running imaptest for a while? http://imapwiki.org/ImapTest No NFS here. The filesystem is called GPFS. It is a clustered FS. Ah, that probably explains it. Can multiple different servers modify the same mailbox? Yes. I have N+1 machines that access the same filesystem and it looks local to both machines. Cache file is the part of Dovecot that demands the most from the OS/filesystem. The most difficult part is probably that it writes to the file without locking. It first reserves a space and then starts writing there. Multiple processes can write to the same file at the same time. This should probably be changed at some point, since it could just buffer more data to memory and then lock, write, unlock. That would also make the code simpler, since it can currently leave holes to the file because it has to guess initially how much space to reserve.. Ok. Should I disable Cache file or what would be the best fix? Thanks, Jonathan
Re: [Dovecot] Running imaptest revision 209:939fa886391a built against dovecot revision 9116:9ae55b68cf61 on AIX 5.3 core dumps
On Tue, 2 Jun 2009, Timo Sirainen wrote: On Tue, 2009-06-02 at 13:21 -0400, Jonathan Siegle wrote: # ./imaptest host=127.0.0.1 port=143 user=tstem38 pass=pass4you mbox=/ gpfs/users/t/s/tstem38/IMAP/foo4 Panic: file client.c: line 620: assertion failed: (idx >= array_count(&clients) == NULL) IOT/Abort trap(coredump) That's because you're not using gcc.. Hmm. Well, for now you could just delete that assert from the code. Do you have a preferred version of gcc for this? thanks, Jonathan
Re: [Dovecot] record points outside file error with dovecot revision 9116:9ae55b68cf61
On Tue, 2 Jun 2009, Timo Sirainen wrote: On Tue, 2009-06-02 at 09:51 -0400, Jonathan Siegle wrote: I use mbox with mbox-snarf plugin. I get errors like: dovecot: IMAP(testuser): Corrupted index cache file /full/path/ dovecot.index.cache: record points outside file So you're using AIX? Do you also happen to use NFS? Can you reproduce this error by running imaptest for a while? http://imapwiki.org/ImapTest No NFS here. The filesystem is called GPFS. It is a clustered FS. I'll work on reproducing with imaptest. Once I reproduce, I'll give more details about how I run dovecot. thanks, Jonathan
[Dovecot] [OT] merurial Was: Re: Problem with IMAP (rename and delete non existent folder).
On Fri, 8 May 2009, Lenthir wrote: We were waiting for stable version, but now We are going to merge our project with rc3... I've been learning mercurial and it has made life so much easier by using the queues option. The most annoying part is deleting the patch because Timo fixed it in upstream(Thanks Timo!!). I can merge the latest version of dovecot with my patches quickly, rebuild the source tree, and start testing about 15 minutes later. Mozilla has good docs on using mercurial with queues. I haven't used a dovecot*.tar.gz file for months now. The learning curve was not that bad either. svn is still a mystery to me. -- Jonathan
Re: [Dovecot] Problem with IMAP (rename and delete non existent folder).
On Fri, 8 May 2009, Lenthir wrote: Not exacly this way :/ I checked this scenario and It works for me... Don't create this folder! Could you test this scenario? (after login) 1 delete "kj-test-folder" 1 NO [NONEXISTENT] Folder does not exist 2 rename "kj-test-folder" 2 NO Empty mailbox name. 3 rename "kj-test-folder" "other-folder" (Here communication hangs, when GDB is running I've got SIGABRT.) 1 delete "kj-test-folder" 1 NO [NONEXISTENT] Mailbox doesn't exist: kj-test-folder 2 rename "kj-test-folder" 2 NO Empty mailbox name. 3 rename "kj-test-folder" "other-folder" 3 NO [NONEXISTENT] Mailbox doesn't exist: kj-test-folder 4 logout * BYE Logging out 4 OK Logout completed. (after login) 1 delete "some-unexistent-folder" 1 NO [NONEXISTENT] Folder does not exist 2 rename "some-unexistent-folder" "some-other-name" (Here communication hangs, when GDB is running I've got SIGABRT.) Could you test this? 1 delete "some-unexistent-folder" * BAD Error in IMAP command 1: Unknown command. 2 rename "some-unexistent-folder" "some-other-name" 2 NO [NONEXISTENT] Mailbox doesn't exist: some-unexistent-folder 3 logout * BYE Logging out 3 OK Logout completed. If nothing else, build the latest version and install in a different prefix then listen on another port. -Jonathan
Re: [Dovecot] Problem with IMAP (rename and delete non existent folder).
On Fri, 8 May 2009, Lenthir wrote: I don't know if this problem exists only in my version of dovecot, or in all... To reproduce BUG do this (use gdb): telnet localhost 143 o login a...@email.sth verystrong_pass o delete "kj-test-folder" o rename "kj-test-folder" o rename "kj-test-folder" "other-folder" SIGABRT I'm running dovecot 1.2.rc3(hg revision 9018:0bb192fe0abd) on AIX 5.3 with mbox files. Here are my tests: 2 create "kj-test-folder" 2 OK Create completed. 3 rename "kj-test-folder" 3 NO Empty mailbox name. 4 delete "kj-test-folder" 4 OK Delete completed. 5 rename "kj-test-folder" 5 NO Empty mailbox name. 6 rename "kj-test-folder" "other-folder" 6 NO [NONEXISTENT] Mailbox doesn't exist: kj-test-folder So I'm not seeing the error.
Re: [Dovecot] Timeout leak with dovecot version dovecot1.2(8985:f43bebab3dac)
On Wed, 29 Apr 2009, Timo Sirainen wrote: On Wed, 2009-04-29 at 14:52 -0400, jsie...@psu.edu wrote: This is 64bit AIX 5.3. Looking through previous versions of dovecot, I see this warning. I didn't realize this was something bad until today. It's not exactly bad. It gets logged only when the process is exiting. But it shouldn't be happening either. local0.log.20090429:Apr 29 12:41:16 hostname dovecot: IMAP(jsiegle): Timeout leak: 1100054c0 How easily can you reproduce this? For example if you do: telnet localhost 143 1 login user pass 2 select inbox 3 logout Oh lovely. Now when I do these steps I see in the logs: Apr 30 08:06:23 tr27n12.aset.psu.edu syslog: PSU mbox-snarf name is INBOX Apr 30 08:06:23 tr27n12.aset.psu.edu dovecot: Panic: IMAP(tstem38): file istream-mail-stats.c: line 75: assertion failed: (ret != -1 || stream->istream.eof || stream->istream.stream_errno != 0) Apr 30 08:06:23 tr27n12.aset.psu.edu dovecot: child 630886 (imap) killed with signal 6 I don't get this error for other folders, just the INBOX. I don't get the timeout leak very often. Maybe 15 times a day total for multiple users. I'll do some more debugging of that problem and report back. -Jonathan smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] Timeout leak with dovecot version dovecot1.2(8985:f43bebab3dac)
This is 64bit AIX 5.3. Looking through previous versions of dovecot, I see this warning. I didn't realize this was something bad until today. My environment uses mbox/mbox-snarf plugin. The message in the logs is: local0.log.20090429:Apr 29 12:41:16 hostname dovecot: IMAP(jsiegle): Timeout leak: 1100054c0 In dbx(the AIX debugger) I see: $ dbx imap Type 'help' for help. reading symbolic information ... (dbx) gotoi 0x1100054c0 Goto address is not within current function or block. (set $unsafegoto to override) (dbx) set $unsafegoto (dbx) gotoi 0x1100054c0 stopped in cmd-append. at 0x1100054c0 0x1100054c0 (???) 0001 Invalid opcode. (dbx) where cmd-append.() at 0x1100054c0 If you need more info, don't hesitate to ask. Thanks, Jonathan
Re: [Dovecot] Dovecot v2.0 hg tree
On Fri, 24 Apr 2009, Odhiambo ? wrote: On Fri, Apr 24, 2009 at 3:04 AM, Timo Sirainen wrote: http://hg.dovecot.org/dovecot-2.0/ How do I retrieve a copy for testing? Can I do "svn co ..." ? http://www.dovecot.org/download.html Search for Mercurial Repo
Re: [Dovecot] Compiling v1.3 on different OSes
On Tue, 7 Apr 2009, Timo Sirainen wrote: On Mon, 2009-04-06 at 14:08 -0400, Timo Sirainen wrote: http://dovecot.org/tmp/dovecot-1.3.UNSTABLE.tar.gz OK, updated the URL once more. Now iconv problems should be gone? And everyone else is happy with it too? :) Compiles without error on AIX 5.3 & VAC 8 which means that at least you typed in everything correctly. :) smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] Compiling v1.3 on different OSes
On Mon, 6 Apr 2009, Timo Sirainen wrote: I was hoping to finally get shared libdovecot.so and libdovecot-storage.so libraries for v1.3, so Sieve (and maybe others) could link against them. But I'm running into trouble getting it to compile in Solaris 10. Could you non-Linux users test if this compiles with you? On AIX 5.3, I do get compile errors for all-settings.c[1]. libdovecot.so looks great[2]! [1] "all-settings.c", line 492.31: 1506-196 (W) Initialization between types "struct dynamic_settings_parser*" and "unsigned long" is not allowed. "all-settings.c", line 517.31: 1506-196 (W) Initialization between types "struct dynamic_settings_parser*" and "unsigned long" is not allowed. "all-settings.c", line 598.28: 1506-196 (W) Initialization between types "unsigned int" and "char*" is not allowed. "all-settings.c", line 598.28: 1506-1106 (S) Initializer does not evaluate to a constant that fits in the target type. "all-settings.c", line 1355.31: 1506-196 (W) Initialization between types "struct dynamic_settings_parser*" and "unsigned long" is not allowed. [2] # find . |grep libdovecot.so ./src/lib-dovecot/.libs/libdovecot.so.0.0.0 ./src/lib-dovecot/.libs/libdovecot.so.0 ./src/lib-dovecot/.libs/libdovecot.so > http://dovecot.org/tmp/dovecot-1.3.UNSTABLE.tar.gz Or if anyone has ideas why this happens, I'd like to know: libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -I/home/cras/include -o .libs/rawlog rawlog.o ../lib-dovecot/.libs/libdovecot.so -lrt -lsocket -lsendfile -Wl,-rpath -Wl,/usr/local/lib/dovecot ../lib-dovecot/.libs/libdovecot.so: undefined reference to `dler...@sunw_1.22' ../lib-dovecot/.libs/libdovecot.so: undefined reference to `dlo...@sunw_1.22' ../lib-dovecot/.libs/libdovecot.so: undefined reference to `dl...@sunw_1.22' ../lib-dovecot/.libs/libdovecot.so: undefined reference to `dlcl...@sunw_1.22' If I disable plugins (so dl*() functions aren't used) it compiles, so it's only those functions that are problematic. pvs -s /lib/libc.so lists the dl* functions under SUNW_1.22 so I'd think that should have worked.. I don't have a Solaris 10 box. I found this link: http://forums-beta.sun.com/thread.jspa?messageID=9269180 which might help. Dig down close to the bottom(July 1,2005). -Jonathan smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] [Help] How get the complete CAPABILITY list?
On Fri, 27 Mar 2009, Michael M. Slusarz wrote: Quoting tao.6.y...@nokia.com: A Login user1 A CAPABILITY I expect to get the list that contains the following - AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5 AUTH=DIGEST-MD5 But actually I did not get the above content, I only get a short list as below, CAPABILITY IMAP4rev1 SASL -IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS A OK Capability completed Why did I miss the AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5 AUTH=DIGEST-MD5?? Once you login, the list of available authentication methods is irrelevant. The RFC allows the CAPABILITY string to be different in unauthenticated vs. authenticated state. michael You can override the CAPABILITY line after login by modifying imap_capability in dovecot.conf. Whatever you put for that variable is what you will see after login. -Jonathan smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] LIST command claims children exist in empty folder
On Thu, 26 Mar 2009, jsie...@psu.edu wrote: On Thu, 26 Mar 2009, Jonathan Siegle wrote: On Mar 25, 2009, at 8:42 PM, Timo Sirainen wrote: On Wed, 2009-03-25 at 07:52 -0400, Jonathan Siegle wrote: On Mar 24, 2009, at 11:36 AM, Jonathan Siegle wrote: Dovecot 1.2 (8834:5284f45c249a) Should list return \HasChildren if no folders exist under it? I'm using mbox format. Is this really a problem?.. alpine/pine can't delete empty folders because the empty folder \HasChildren. Here is my patch. Problem with that is that it doesn't return any children flags when using LISTEXT command: 1 list (subscribed) "" % return (children) Fixing this would require adding new code to fs_list_subs() to scan the subdirectory if children flags are missing. list_file_subdir() handles that for non-subscription listing, but it can't be directly used for subscription listing. ok thanks. I'll look at that today. Sorry for the tripple! I didn't type what you typed. I understand the problem now. smime.p7s Description: S/MIME Cryptographic Signature
Re: [Dovecot] LIST command claims children exist in empty folder
On Thu, 26 Mar 2009, Jonathan Siegle wrote: On Mar 25, 2009, at 8:42 PM, Timo Sirainen wrote: On Wed, 2009-03-25 at 07:52 -0400, Jonathan Siegle wrote: On Mar 24, 2009, at 11:36 AM, Jonathan Siegle wrote: Dovecot 1.2 (8834:5284f45c249a) Should list return \HasChildren if no folders exist under it? I'm using mbox format. Is this really a problem?.. alpine/pine can't delete empty folders because the empty folder \HasChildren. Here is my patch. Problem with that is that it doesn't return any children flags when using LISTEXT command: 1 list (subscribed) "" % return (children) Fixing this would require adding new code to fs_list_subs() to scan the subdirectory if children flags are missing. list_file_subdir() handles that for non-subscription listing, but it can't be directly used for subscription listing. ok thanks. I'll look at that today. I'm confused on what needs fixing. Maybe my test data doesn't cover the case. Here is output from my "unpatched" dovecot 1.2beta3: 3 list (subscribed) "imaptestdir/" % return (children) * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox1" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox11" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox2" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox3" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox6" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/anotherfolder" 3 OK List completed. 4 list (subscribed) "imaptestdir/" * return (children) * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox1" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox11" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox2" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox3" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox6" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/anotherfolder" * LIST (\Subscribed \Noselect \HasChildren) "/" "imaptestdir/anotherdirectory/" * LIST (\Subscribed \Noselect \HasChildren) "/" "imaptestdir/anotherdirectory/aseconddirectory/" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/anotherdirectory/aseconddirectory/mailbox22" 4 OK List completed. Here is output from the "patched" dovecot 1.2beta3: 3 list (subscribed) "imaptestdir/" % return (children) * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox1" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox11" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox2" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox3" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox6" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/anotherfolder" 3 OK List completed. 4 list (subscribed) "imaptestdir/" * return (children) * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox1" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox11" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox2" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox3" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/mailbox6" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/anotherfolder" * LIST (\Subscribed \Noselect) "/" "imaptestdir/anotherdirectory/" * LIST (\Subscribed \Noselect) "/" "imaptestdir/anotherdirectory/aseconddirectory/" * LIST (\Subscribed \NoInferiors \UnMarked) "/" "imaptestdir/anotherdirectory/aseconddirectory/mailbox22" 4 OK List completed. Thanks, Jonathan smime.p7s Description: S/MIME Cryptographic Signature
[Dovecot] imap SORT core dumps on beta1
Steps to reproduce: 0 login userid password 1 select myfavoritemailbox 2 sort (FROM) UTF-8 ALL I tested alpha5 and it works fine. Here is the trace of beta1. I see many adds in revision 8726(hg annotate mail-storage.c), but that line number(674) is very old. So it might be a local problem. Here is what dbx "where" says about what happened. mailbox_header_lookup_init(box = (nil), headers = 0x0260), line 674 in "mail-storage.c" wanted_fields_get(box = (nil), sort_program = 0x03a0, wanted_fields_r = 0x0308, headers_ctx_r = 0x0310), line 518 in "imap-search.c" imap_search_start(ctx = 0x000110021c70, sargs = 0x0001100848b8, sort_program = 0x03a0), line 536 in "imap-search.c" cmd_sort(cmd = 0x000110021b58), line 136 in "cmd-sort.c" client_command_input(cmd = 0x000110021b58), line 603 in "client.c" client_command_input(cmd = 0x000110021b58), line 652 in "client.c" client_handle_next_command(client = 0x0001100216d0, remove_io_r = 0x0621), line 693 in "client.c" unnamed block in client_handle_input(client = 0x0001100216d0), line 703 in "client.c" client_handle_input(client = 0x0001100216d0), line 703 in "client.c" client_input(client = 0x0001100216d0), line 748 in "client.c" io_loop_handler_run(ioloop = 0x00011001f850), line 203 in "ioloop-poll.c" io_loop_run(ioloop = 0x00011001f850), line 338 in "ioloop.c" main(argc = 1, argv = 0x0918, envp = 0x0928), line 315 in "main.c" (dbx) quit Thanks, Jonathan
Re: [Dovecot] mbox snarf plugin + idle
Still haven't figured it out. Looks like mbox_snarf_sync_init(which calls mbox_snarf) is called once. Looks like cmd_idle only wants to update when: /* check immediately if there are changes. if they came before we added mailbox-notifier, we wouldn't see them otherwise. */ if (client->mailbox != NULL) idle_sync_now(client->mailbox, ctx); So maybe this is by design. So it looks like I should always keep one e-mail in my inbox to delete in order to get new messages when using idle? On Thu, 12 Feb 2009, Jonathan Siegle wrote: I'm having a problem with mbox snarf not looking at /var/spool/mail/ when in idle mode thus never giving me a RECENT line even though there are new messages in /var/spool/mail/ . Here are the imap commands to reproduce the problem: 1 login userid password 2 select inbox 3 idle When I run "select inbox" it does see my messages in /var/spool/mail/ and moves them over fine. When I truss the process, I see it only running stat calls on my "mbox-snarf" file. To get new messages I issue DONE, CLOSE, and SELECT INBOX. I'm not sure why it is reporting alpha5 two lines below. I did an hg pull just the other day and see 1.2.beta1 in the output of hg tags. # /usr/ladmin2/sbin/dovecot -n # 1.2.alpha5: /usr/ladmin2/etc/dovecot.conf Warning: fd limit 2000 is lower than what Dovecot can use under full load (more than 4224). Either grow the limit or change login_max_processes_count and max_mail_processes settings # OS: AIX 3 0001112AD300 syslog_facility: local0 protocols: imap listen: *: ssl: no disable_plaintext_auth: no login_dir: /usr/ladmin2/var/run/dovecot/login login_executable: /usr/ladmin2/libexec/dovecot/imap-login login_greeting: Dovecot ready. login_processes_count: 30 max_mail_processes: 4096 mail_location: mbox:%h/new:INBOX=/var/spool/mail/14/%u mmap_disable: yes dotlock_use_excl: no mbox_write_locks: fcntl mbox_lazy_writes: no mail_plugins: mbox_snarf imap_client_workarounds: delay-newmail auth default: mechanisms: plain gssapi krb5_keytab: /etc/krb5/dovecot.keytab gssapi_hostname: $ALL verbose: yes debug: yes passdb: driver: pam userdb: driver: passwd plugin: mbox_snarf: /gpfs/inbox/14/%u Thanks, Jonathan
