Re: Problem with IMAP Server and Header Cache
On Thu, Jan 26, 2023 at 05:07:50PM +0100, Christian Brabandt wrote:
I tried building building latest master with --with-kyotocabinet
Thank you for rebuilding and testing. Since the problem isn't with the
header cache backend, this smells like a Mutt bug.
Would you mind opening a ticket so we can debug it without causing too
much traffic on mutt-users?
It took me a bit until I found `./prepare` but then I could run
configure (and for some reason, it didn't like my installed
`libidn2-dev` library, so I build without `--with-idn`, but I suppose
for the test it doesn't really matter.)
There is a --with-idn2 configure flag you can use instead. The packaged
mutt used libidn, so I assumed build-dep would install that library.
But I am afraid, I am seeing the same problem (I haven't waited yet 3
hours until mutt gets disconnected from the IMAP server, but the
symptoms look exactly the same.
I'll need some more debug information to try and figure out what's going
on. I'm attaching a patch. If you would apply, re-compile and run with
debug mode (-d 2), it should hopefully give some useful info about
what's happening at the hang point.
You only need to let mutt run for a minute and then kill it. Otherwise
the debug file will get really really big. If you would post that to
the ticket (taking care to remove the authentication parts), I would
appreciate it.
Thank you!
--
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C 5308 ADEF 7684 8031 6BDA
From f5b644b42c418b727ecf758d9c706fa429823cd2 Mon Sep 17 00:00:00 2001
From: Kevin McCarthy
Date: Thu, 26 Jan 2023 11:00:19 -0800
Subject: [PATCH] wip: debug statements
---
imap/message.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/imap/message.c b/imap/message.c
index 376b91dc..e27c0bc3 100644
--- a/imap/message.c
+++ b/imap/message.c
@@ -587,12 +587,15 @@ static int read_headers_qresync_eval_cache (IMAP_DATA *idata, char *uid_seqset)
ctx = idata->ctx;
msn = 1;
+ dprint (2, (debugfile, "seqset: %s\n", uid_seqset));
+
iter = mutt_seqset_iterator_new (uid_seqset);
if (!iter)
return -1;
while ((rc = mutt_seqset_iterator_next (iter, &uid)) == 0)
{
+dprint (2, (debugfile, "read seqset uid %u\n", uid));
/* The seqset may contain more headers than the fetch request, so
* we need to watch and reallocate the context and msn_index */
if (msn > idata->msn_index_size)
@@ -601,6 +604,7 @@ static int read_headers_qresync_eval_cache (IMAP_DATA *idata, char *uid_seqset)
h = imap_hcache_get (idata, uid);
if (h)
{
+ dprint (2, (debugfile, "\tfound in header cache\n"));
idata->max_msn = MAX (idata->max_msn, msn);
idata->msn_index[msn - 1] = h;
@@ -640,7 +644,12 @@ static int read_headers_qresync_eval_cache (IMAP_DATA *idata, char *uid_seqset)
* stored "blank" in the uid seqset.
*/
else if (!uid)
+{
+ dprint (2, (debugfile, "\tblank\n"));
msn++;
+}
+else
+ dprint (2, (debugfile, "\tmissing from header cache\n"));
}
mutt_seqset_iterator_free (&iter);
--
2.39.0
signature.asc
Description: PGP signature
Re: Problem with IMAP Server and Header Cache
Am 2023-01-26 00:33, schrieb Kevin J. McCarthy: Hmmm... I see in the log that even in the "working" case it's taking a good 15 seconds to load in the messages from the seqset/header cache. It's a big mailbox, though, so maybe that's to be expected. I noticed you are using 1.13.2. Are you in a position to try building a newer version of mutt? There have been several important fixes to QRESYNC since then (including a bug in the seqset iterator). It doesn't *look* like that's the issue here, but it would still be a good idea to try 2.2.9 if possible. While you are at it, you might also try building with a different header cache backend. Perhaps tokyocabinet is having issues. I would suggest kyotocabinet or perhaps lmdb and see how they work. On ubuntu, just run $ sudo apt build-dep mutt $ sudo apt install libkyotocabinet-dev liblmdb-dev $ ./configure --enable-compressed --enable-debug --enable-fcntl \ --enable-hcache --enable-gpgme --enable-imap --enable-smtp \ --enable-pop --enable-sidebar --with-curses --with-gnutls \ --with-gss --with-idn --with-sasl \ --with-kyotocabinet (or --with-lmdb) $ make $ sudo make install I tried building building latest master with --with-kyotocabinet It took me a bit until I found `./prepare` but then I could run configure (and for some reason, it didn't like my installed `libidn2-dev` library, so I build without `--with-idn`, but I suppose for the test it doesn't really matter.) #v+ Mutt 2.2.9+62 (90236f5a) (2022-12-20) Copyright (C) 1996-2022 Michael R. Elkins and others. Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'. Mutt is free software, and you are welcome to redistribute it under certain conditions; type `mutt -vv' for details. System: Linux 5.4.0-137-generic (x86_64) ncurses: ncurses 6.2.20200212 (compiled with 6.2) hcache backend: kyotocabinet 1.2.76 Compiler: Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' --with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-10 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --enable-libphobos-checking=release --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-build-config=bootstrap-lto-lean --enable-link-mutex Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 10.2.1 20210110 (Debian 10.2.1-6) Configure options: '--enable-compressed' '--enable-debug' '--enable-fcntl' '--enable-hcache' '--enable-gpgme' '--enable-imap' '--enable-smtp' '--enable-pop' '--enable-sidebar' '--with-curses' '--with-gnutls' '--with-gss' '--with-sasl' '--prefix=/home/chrisbra/local' '--with-kyotocabinet' Compilation CFLAGS: -Wall -pedantic -Wno-long-long -g -O2 #v- I wiped out the header-cache, to make sure there are no incompatibilities, re-connected locally build up the header-cache again (which takes a while to download all messages) browsed quickly and quit mutt and did a fresh reconnection, so that the new header-cache will be used again. But I am afraid, I am seeing the same problem (I haven't waited yet 3 hours until mutt gets disconnected from the IMAP server, but the symptoms look exactly the same. Mutt just shows me "Selecting github..." and does "something" :/ In the debug file I see: #v+ [2023-01-26 16:52:40] Mutt/2.2.9+62 (90236f5a) (2022-12-20) debugging at level 2 [2023-01-26 16:52:40] In mutt_reflow_windows [2023-01-26 16:52:40] In mutt_reflow_windows [2023-01-26 16:52:40] In mutt_reflow_windows [...] # Just so it's clear its the new mutt 2.2 :) [...] [2023-01-26 16:52:45] New mail in =vim, =zsh-users, =Trash, =spam [2023-01-26 16:52:47] Mailbox is unchanged. [2023-01-26 16:52:47] 4> DONE^M a0042 CLOSE^M [2023-01-26 16:52:47] 4< a0041 OK Idle completed (2.993 + 2.993 + 2.992 secs). [2023-01-26 16:52:47] 4< a0042 OK Close completed (0.001 + 0.000 secs). [20
