Re: Cyrus IMAPd 2.3.12 Released
> Simon Matter wrote: >>> Simon Matter wrote: > I am pleased to announce the release of Cyrus IMAPd 2.3.12. This > release should be considered production quality. > > > Noteworthy changes: > > * Added statuscache.db to cache IMAP STATUS data which > significantly reduces the amount of I/O necessary when neither the > mailbox nor \Seen state has changed -- courtesy of Fastmail.fm While upgrading my RPM packages I found that logging is too noisy for my taste. I get alot of syslog messages like so: Apr 24 11:35:30 test imap[10457]: statuscache, 'user.simix', 'simix', '0x13', 'yes' >>> These messages are at the DEBUG level. Do you usually ship your RPM >>> with the logging set to DEBUG? I don't really see a problem leaving >>> these messages alone, since I would expect most production systems to >>> be >>> logging at INFO or higher. >> >> The RPM logs to mail facility and priority of mail is * by default on >> RedHat. That means out of the box we get quite alot messsages in maillog >> now. With statuscache enabled the number of log entries is about 10 >> times >> higher than without. >> I understand that others may want those logs at the DEBUG level, it's >> only >> a problem in my situation. > > > Don't you also get a ton of transaction log messages, regardless of the > statuscache code? Of course I get quite a number of log messages but that's still acceptable. However with statuscache enabled the number of log messages increases dramatically. As I said it increases at a factor of ~10. The other log with skiplist doesn't hurt so much because it's not so often. Still it doesn't seem to make much sense to log that message whenever a seen db is opened, which it looks like in my tests. Maybe that's only here because we run all db's as skiplist. Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Simon Matter wrote: >> Simon Matter wrote: I am pleased to announce the release of Cyrus IMAPd 2.3.12. This release should be considered production quality. Noteworthy changes: * Added statuscache.db to cache IMAP STATUS data which significantly reduces the amount of I/O necessary when neither the mailbox nor \Seen state has changed -- courtesy of Fastmail.fm >>> While upgrading my RPM packages I found that logging is too noisy for my >>> taste. I get alot of syslog messages like so: >>> >>> Apr 24 11:35:30 test imap[10457]: statuscache, 'user.simix', 'simix', >>> '0x13', 'yes' >> These messages are at the DEBUG level. Do you usually ship your RPM >> with the logging set to DEBUG? I don't really see a problem leaving >> these messages alone, since I would expect most production systems to be >> logging at INFO or higher. > > The RPM logs to mail facility and priority of mail is * by default on > RedHat. That means out of the box we get quite alot messsages in maillog > now. With statuscache enabled the number of log entries is about 10 times > higher than without. > I understand that others may want those logs at the DEBUG level, it's only > a problem in my situation. Don't you also get a ton of transaction log messages, regardless of the statuscache code? -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
> Simon Matter wrote: >>> I am pleased to announce the release of Cyrus IMAPd 2.3.12. This >>> release should be considered production quality. >>> >>> >>> Noteworthy changes: >>> >>> * Added statuscache.db to cache IMAP STATUS data which >>> significantly reduces the amount of I/O necessary when neither the >>> mailbox nor \Seen state has changed -- courtesy of Fastmail.fm >> >> While upgrading my RPM packages I found that logging is too noisy for my >> taste. I get alot of syslog messages like so: >> >> Apr 24 11:35:30 test imap[10457]: statuscache, 'user.simix', 'simix', >> '0x13', 'yes' > > These messages are at the DEBUG level. Do you usually ship your RPM > with the logging set to DEBUG? I don't really see a problem leaving > these messages alone, since I would expect most production systems to be > logging at INFO or higher. The RPM logs to mail facility and priority of mail is * by default on RedHat. That means out of the box we get quite alot messsages in maillog now. With statuscache enabled the number of log entries is about 10 times higher than without. I understand that others may want those logs at the DEBUG level, it's only a problem in my situation. Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Simon Matter wrote: >> I am pleased to announce the release of Cyrus IMAPd 2.3.12. This >> release should be considered production quality. >> >> >> Noteworthy changes: >> >> * Added statuscache.db to cache IMAP STATUS data which >> significantly reduces the amount of I/O necessary when neither the >> mailbox nor \Seen state has changed -- courtesy of Fastmail.fm > > While upgrading my RPM packages I found that logging is too noisy for my > taste. I get alot of syslog messages like so: > > Apr 24 11:35:30 test imap[10457]: statuscache, 'user.simix', 'simix', > '0x13', 'yes' These messages are at the DEBUG level. Do you usually ship your RPM with the logging set to DEBUG? I don't really see a problem leaving these messages alone, since I would expect most production systems to be logging at INFO or higher. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
> I am pleased to announce the release of Cyrus IMAPd 2.3.12. This
> release should be considered production quality.
>
>
> Noteworthy changes:
>
> * Added statuscache.db to cache IMAP STATUS data which
> significantly reduces the amount of I/O necessary when neither the
> mailbox nor \Seen state has changed -- courtesy of Fastmail.fm
While upgrading my RPM packages I found that logging is too noisy for my
taste. I get alot of syslog messages like so:
Apr 24 11:35:30 test imap[10457]: statuscache, 'user.simix', 'simix',
'0x13', 'yes'
and
Apr 24 11:59:13 test imap[17299]: skiplist:
/var/lib/imap/user/s/simix.seen is already open 1 time, returning object
I wanted to change things so that those logs are only generated with
CYRUS_VERBOSE > 1 but that seems not be intended for syslog calls other
than from the standalone programs like master.c, make_md5.c and so. I
finally simply disabled those calls I considered too much:
--- cyrus-imapd-2.3.12/lib/cyrusdb_skiplist.c.orig 2008-04-09
19:56:57.0 +0200
+++ cyrus-imapd-2.3.12/lib/cyrusdb_skiplist.c 2008-04-24
12:10:24.0 +0200
@@ -751,8 +751,8 @@
}
if (list_ent) {
/* we already have this DB open! */
- syslog(LOG_NOTICE, "skiplist: %s is already open %d time%s,
returning object",
- fname, list_ent->refcount, list_ent->refcount == 1 ? "" : "s");
+ /* syslog(LOG_NOTICE, "skiplist: %s is already open %d time%s,
returning object",
+ fname, list_ent->refcount, list_ent->refcount == 1 ? "" : "s"); */
*ret = list_ent->db;
++list_ent->refcount;
return 0;
--- cyrus-imapd-2.3.12/imap/index.c.orig2008-03-24
18:09:17.0 +0100
+++ cyrus-imapd-2.3.12/imap/index.c 2008-04-24 11:25:28.0 +0200
@@ -1573,13 +1573,13 @@
*/
if (!r) {
- syslog(LOG_DEBUG, "statuscache, '%s', '%s', '0x%02x', 'yes'",
- mboxname, imapd_userid, statusitems);
+ /* syslog(LOG_DEBUG, "statuscache, '%s', '%s', '0x%02x', 'yes'",
+ mboxname, imapd_userid, statusitems); */
goto statusdone;
}
- syslog(LOG_DEBUG, "statuscache, '%s', '%s', '0x%02x', 'no'",
- mboxname, imapd_userid, statusitems);
+ /* syslog(LOG_DEBUG, "statuscache, '%s', '%s', '0x%02x', 'no'",
+ mboxname, imapd_userid, statusitems); */
}
/* Missing or invalid cache entry */
With those changes the logging quantity is almost the same like with
2.3.11, not ten times more.
Statuscache: I have intended to enable statuscache by default in the RPM
and configure it to skiplist be default. (As usual, the imapd.conf manpage
states that those changes are RPM specific!)
Do I have to expect any troubles with this setup? Unfortunately I don't
have a really large server anymore to test with thousands of users and
very large storage. Bron?
Regards,
Simon
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
> Simon Matter schrieb: >>> Did you copy/paste the patch? Then maybe it's a problem with whitespace >>> and such. Can you try the attached one. > > Yes I did c/p and that seems to be the problem, because with your > patch it did that: > >> imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < >> ../cyrus-imapd-2.3.12-config_parser.patch >> patching file lib/libconfig.c >> patch unexpectedly ends in middle of line >> Hunk #1 succeeded at 344 with fuzz 1. > > and after installing I get: > >> (gdb) file /opt/cyrus-imapd-2.3.12/libexec/master >> Reading symbols from /opt/cyrus-imapd-2.3.12/libexec/master...done. >> Using host libthread_db library "/lib/libthread_db.so.1". >> (gdb) run -d >> Starting program: /opt/cyrus-imapd-2.3.12/libexec/master -d >> >> Program exited normally. >> (gdb) > > and cyrus master is running. OK, that also means that the patch in CVS is really the bad one and Andi's patch is the one to put into CVS. Simon Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Simon Matter schrieb: >> Did you copy/paste the patch? Then maybe it's a problem with whitespace >> and such. Can you try the attached one. Yes I did c/p and that seems to be the problem, because with your patch it did that: > imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < > ../cyrus-imapd-2.3.12-config_parser.patch > patching file lib/libconfig.c > patch unexpectedly ends in middle of line > Hunk #1 succeeded at 344 with fuzz 1. and after installing I get: > (gdb) file /opt/cyrus-imapd-2.3.12/libexec/master > Reading symbols from /opt/cyrus-imapd-2.3.12/libexec/master...done. > Using host libthread_db library "/lib/libthread_db.so.1". > (gdb) run -d > Starting program: /opt/cyrus-imapd-2.3.12/libexec/master -d > > Program exited normally. > (gdb) and cyrus master is running. >> Simon Thanks Thomas Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
> Simon Matter schrieb:
>>> They don't apply togheter, only one can be applied. Try with fresh
>>> sources
>>> and Andy Fiddaman's patch should apply fine.
>>>
>>> Simon
>
> I should have mentioned that that I did it with fresh sources.
>
> 1. fresh sources with first patch
> 2. delete source tree
> 3. unpack sources and second patch
Did you copy/paste the patch? Then maybe it's a problem with whitespace
and such. Can you try the attached one.
Simon
>
> I also verified right now again with fresh sources:
>
>> imap-1:/export/build# tar -xvzf source/cyrus-imapd-2.3.12.tar.gz
> > imap-1:/export/build# cd cyrus-imapd-2-3-12
>> imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < ../libconfig3.patch
>> patching file lib/libconfig.c
>> Hunk #1 FAILED at 344.
>> 1 out of 1 hunk FAILED -- saving rejects to file lib/libconfig.c.rej
>
> Thomas
>
--- cyrus-imapd-2.3.12/lib/libconfig.c.orig 2008-04-11 22:07:00.0 +0200
+++ cyrus-imapd-2.3.12/lib/libconfig.c 2008-04-23 21:27:46.0 +0200
@@ -344,7 +344,7 @@
/* end of line */
buf[--len] = '\0';
- if (buf[len-1] == '\\') {
+ if (len && buf[len-1] == '\\') {
/* line continuation */
len--;
lineno++;
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Simon Matter schrieb: >> They don't apply togheter, only one can be applied. Try with fresh sources >> and Andy Fiddaman's patch should apply fine. >> >> Simon I should have mentioned that that I did it with fresh sources. 1. fresh sources with first patch 2. delete source tree 3. unpack sources and second patch I also verified right now again with fresh sources: > imap-1:/export/build# tar -xvzf source/cyrus-imapd-2.3.12.tar.gz > imap-1:/export/build# cd cyrus-imapd-2-3-12 > imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < ../libconfig3.patch > > patching file lib/libconfig.c > Hunk #1 FAILED at 344. > 1 out of 1 hunk FAILED -- saving rejects to file lib/libconfig.c.rej Thomas Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
> Andy Fiddaman schrieb:
>>> That's the line that the patch changes, and that's the old version of
>>> the
>>> line, so something didn't take with your patching.
>>>
>>> A.
>
> It seems so, but I don't understand, why. Here's the patch:
>
>> --- cyrus-imapd-2.3.12/lib/libconfig.c 2008/04/11 20:07:00 1.18
>> +++ cyrus-imapd-2.3.12/lib/libconfig.c 2008/04/23 11:00:13 1.18.2.1
>> @@ -39,7 +39,7 @@
>> * AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
>> * OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS
>> SOFTWARE.
>> *
>> - * $Id: libconfig.c,v 1.18 2008/04/11 20:07:00 murch Exp $
>> + * $Id: libconfig.c,v 1.18.2.1 2008/04/23 11:00:13 murch Exp $
>> */
>>
>> #include
>> @@ -340,7 +340,7 @@ void config_read_file(const char *filena
>> while (fgets(buf+len, bufsize-len, infile)) {
>> if (buf[len]) {
>> len = strlen(buf);
>> - if (buf[len-1] == '\n') {
>> + if (len && buf[len-1] == '\n') {
>> /* end of line */
>> buf[--len] = '\0';
>> ../libconfig2.patch (END)
>
> and the patching did without error:
>
>> imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < ../libconfig2.patch
>> patching file lib/libconfig.c
>
> So no errors but after installling i still get that segfault.
>
> I also tried the other patch from Andy Fiddaman:
>
>> --- cyrus-imapd-2.3.12/lib/libconfig.cTue Apr 22 20:30:33 2008
>> +++ cyrus-imapd-2.3.12/lib/libconfig.cTue Apr 22 20:30:40 2008
>> @@ -344,7 +344,7 @@
>> /* end of line */
>> buf[--len] = '\0';
>>
>> - if (buf[len-1] == '\\') {
>> + if (len && buf[len-1] == '\\') {
>> /* line continuation */
>> len--;
>> lineno++;
>> ../libconfig3.patch (END)
>
> but this one got rejected:
>
>> imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < ../libconfig3.patch
>> patching file lib/libconfig.c
>> Hunk #1 FAILED at 344.
>> 1 out of 1 hunk FAILED -- saving rejects to file lib/libconfig.c.rej
>
> I'm not such an expert to see why this one got rejected.
They don't apply togheter, only one can be applied. Try with fresh sources
and Andy Fiddaman's patch should apply fine.
Simon
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Thu, Apr 24, 2008 at 09:58:05AM +1000, Bron Gondwana wrote: > Have you looked at: > > http://cyrus.brong.fastmail.fm/patches/cyrus-auditlog-2.3.11.diff > > It's a very detailed logging system which logs all create, delete, > append, copy, expunge, unlink, etc events. Anything which changes > a mailbox or message (but not metadata events like flag changes > at the moment). It also logs noteworthy sieve events. > > It logs everything at LOG_NOTICE. > > If there are other users for it, I'm happy to put some effort into > making auditlog acceptable for upstream, and possibly generalising > it to allow logging of different classes of events. If you do that, please create a dedicated function for emitting audit messages instead of calling syslog directly. That would make it easier to send audit messages to a different destination (socket, fifo, or just a log file) instead of cluttering syslog. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Andy Fiddaman schrieb:
>> That's the line that the patch changes, and that's the old version of the
>> line, so something didn't take with your patching.
>>
>> A.
It seems so, but I don't understand, why. Here's the patch:
> --- cyrus-imapd-2.3.12/lib/libconfig.c 2008/04/11 20:07:00 1.18
> +++ cyrus-imapd-2.3.12/lib/libconfig.c 2008/04/23 11:00:13 1.18.2.1
> @@ -39,7 +39,7 @@
> * AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING
> * OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
> *
> - * $Id: libconfig.c,v 1.18 2008/04/11 20:07:00 murch Exp $
> + * $Id: libconfig.c,v 1.18.2.1 2008/04/23 11:00:13 murch Exp $
> */
>
> #include
> @@ -340,7 +340,7 @@ void config_read_file(const char *filena
> while (fgets(buf+len, bufsize-len, infile)) {
> if (buf[len]) {
> len = strlen(buf);
> - if (buf[len-1] == '\n') {
> + if (len && buf[len-1] == '\n') {
> /* end of line */
> buf[--len] = '\0';
> ../libconfig2.patch (END)
and the patching did without error:
> imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < ../libconfig2.patch
> patching file lib/libconfig.c
So no errors but after installling i still get that segfault.
I also tried the other patch from Andy Fiddaman:
> --- cyrus-imapd-2.3.12/lib/libconfig.cTue Apr 22 20:30:33 2008
> +++ cyrus-imapd-2.3.12/lib/libconfig.cTue Apr 22 20:30:40 2008
> @@ -344,7 +344,7 @@
> /* end of line */
> buf[--len] = '\0';
>
> - if (buf[len-1] == '\\') {
> + if (len && buf[len-1] == '\\') {
> /* line continuation */
> len--;
> lineno++;
> ../libconfig3.patch (END)
but this one got rejected:
> imap-1:/export/build/cyrus-imapd-2.3.12# patch -p1 < ../libconfig3.patch
> patching file lib/libconfig.c
> Hunk #1 FAILED at 344.
> 1 out of 1 hunk FAILED -- saving rejects to file lib/libconfig.c.rej
I'm not such an expert to see why this one got rejected.
Thomas
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Hi! Bron Gondwana wrote: >> + copyargs->name, index_getheader(mailbox, msgno, >> "X-DSPAM-Signature")); > > Wow - that's pretty tricky. I see you're actually logging a specific > header as well. Funky. We don't have anything like that in our yes. > I'd still suggest that this should be a generic mechanism rather than > a hard-coded header if it's going in upstream. Something like > > auditlog_headers: X-DSPAM-Signature X-Something-Else Of course! > What do you think? It will be wonderful. Thank you. I think, currently no chances to see this changes in 2.3.12, but may be it will be possible in 2.3.13? By. Dmitriy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
>
>
> On Wed, 23 Apr 2008, Thomas Robers wrote:
>
> ; Ken Murchison schrieb:
> ; > Can you try this patch:
> ; >
> ; >
> https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/lib/libconfig.c.diff?r1=1.18;r2=1.18.2.1
> ;
> ; I tried the patch. Compiling is ok, but the master segfaults when I try
> to start:
> ; 347 if (buf[len-1] == '\\') {
>
> That's the line that the patch changes, and that's the old version of the
> line, so something didn't take with your patching.
Something doesn't look right to me. The proposed patch and the change in
CVS are not the same. Ken, did you intend to change the fix?
https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059#c1
https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/lib/libconfig.c.diff?r2=1.19&r1=1.18&f=u
Simon
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Wed, Apr 23, 2008 at 03:37:19PM +0400, Dmitriy Kirhlarov wrote: > If dspam miss, user can manually move message from|to "spam" folder. This > fact fixed in cyrus log file. simple script parsing log and relaunch dspam. > +syslog(LOG_ERR, "DSPAM-Hack index_copysetup(): %s -> %s, hdr %s", > mailbox->name, > +copyargs->name, index_getheader(mailbox, msgno, > "X-DSPAM-Signature")); Wow - that's pretty tricky. I see you're actually logging a specific header as well. Funky. We don't have anything like that in our auditlog patch. I'd still suggest that this should be a generic mechanism rather than a hard-coded header if it's going in upstream. Something like auditlog_headers: X-DSPAM-Signature X-Something-Else which would cause a log line: auditlog: copy oldmailbox= mailbox= olduid=<1234> uid=<7> guid=<478920478932fabed74398943243> x_dspam_signature= x_something_else= Of course there would need to be quoting support since these headers could contain <>. Oh the humanity. My favourite is URI encoding because it's really quite simple to parse, but I'm sure everyone has their favourite. What do you think? Bron. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Wed, Apr 23, 2008 at 05:58:27PM +0400, Dmitriy Kirhlarov wrote: > Sebastian Hagedorn wrote: > > --On 23. April 2008 15:37:19 +0400 Dmitriy Kirhlarov <[EMAIL PROTECTED]> > > wrote: > > > >> Attached patch add to log information about moving messages between > >> folders. I am using this information from logs for relaunch dspam. > >> Any chances for add this patch to project tree? > > > > FWIW, logging this at LOG_ERR level certainly isn't the right way to do > > that ... I'd say it should be INFO at best, if not DEBUG. > > And with this correction, patch can be included to cyrus imapd repo? Have you looked at: http://cyrus.brong.fastmail.fm/patches/cyrus-auditlog-2.3.11.diff It's a very detailed logging system which logs all create, delete, append, copy, expunge, unlink, etc events. Anything which changes a mailbox or message (but not metadata events like flag changes at the moment). It also logs noteworthy sieve events. It logs everything at LOG_NOTICE. If there are other users for it, I'm happy to put some effort into making auditlog acceptable for upstream, and possibly generalising it to allow logging of different classes of events. We use it to populate a database which is linked with events from the various login systems and the email tracking information from our Postfix and lmtp setups - it's not finished yet, but the plan is to be able to track the entire lifecycle of every message we receive (probably only for a few weeks to save space!) so users can see what's happening with their emails. Regards, Bron. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Sebastian Hagedorn wrote: > --On 23. April 2008 15:37:19 +0400 Dmitriy Kirhlarov <[EMAIL PROTECTED]> > wrote: > >> Attached patch add to log information about moving messages between >> folders. I am using this information from logs for relaunch dspam. >> Any chances for add this patch to project tree? > > FWIW, logging this at LOG_ERR level certainly isn't the right way to do > that ... I'd say it should be INFO at best, if not DEBUG. And with this correction, patch can be included to cyrus imapd repo? WBR Dmitriy Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Wed, 23 Apr 2008, Thomas Robers wrote:
; Ken Murchison schrieb:
; > Can you try this patch:
; >
; >
https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/lib/libconfig.c.diff?r1=1.18;r2=1.18.2.1
;
; I tried the patch. Compiling is ok, but the master segfaults when I try to
start:
; 347 if (buf[len-1] == '\\') {
That's the line that the patch changes, and that's the old version of the
line, so something didn't take with your patching.
A.
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
--On 23. April 2008 15:37:19 +0400 Dmitriy Kirhlarov <[EMAIL PROTECTED]> wrote: Attached patch add to log information about moving messages between folders. I am using this information from logs for relaunch dspam. Any chances for add this patch to project tree? FWIW, logging this at LOG_ERR level certainly isn't the right way to do that ... I'd say it should be INFO at best, if not DEBUG. -- .:.Sebastian Hagedorn - RZKR-R1 (Gebäude 52), Zimmer 18.:. Zentrum für angewandte Informatik - Universitätsweiter Service RRZK .:.Universität zu Köln / Cologne University - ✆ +49-221-478-5587.:. .:.:.:.Skype: shagedorn.:.:.:. pgpHwdZftQnTl.pgp Description: PGP signature Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Ken Murchison schrieb:
> Can you try this patch:
>
> https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/lib/libconfig.c.diff?r1=1.18;r2=1.18.2.1
>
I tried the patch. Compiling is ok, but the master segfaults when I try to
start:
This GDB was configured as "x86_64-linux-gnu".
(gdb) file /opt/cyrus-imapd-2.3.12/libexec/master
Reading symbols from /opt/cyrus-imapd-2.3.12/libexec/master...done.
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) run -d
Starting program: /opt/cyrus-imapd-2.3.12/libexec/master -d
Program received signal SIGSEGV, Segmentation fault.
0x004076b2 in config_read_file (filename=0x530010
"/var/imap/imapd.conf") at ./../lib/libconfig.c:347
347 if (buf[len-1] == '\\') {
(gdb) bt
#0 0x004076b2 in config_read_file (filename=0x530010
"/var/imap/imapd.conf") at ./../lib/libconfig.c:347
#1 0x00407fbf in config_read (alt_config=) at
./../lib/libconfig.c:215
#2 0x004082cf in masterconf_init (ident=0x40c0ef "master",
alt_config=0x0) at masterconf.c:84
#3 0x00405e0a in main (argc=2, argv=0x7fff73f10b58) at master.c:1681
Thanks
Thomas
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Attached patch add to log information about moving messages between
folders. I am using this information from logs for relaunch dspam.
Any chances for add this patch to project tree?
It's useful schema and I think it can be helpful not only for me.
Some details:
dspam watch all incoming messages and add tag spam|innocent and unique
signature to message headers. sieve use this tag for moving|not-moving
incoming message to USER/spam folder (every user have personal folder
"spam").
If dspam miss, user can manually move message from|to "spam" folder.
This fact fixed in cyrus log file. simple script parsing log and
relaunch dspam.
WBR.
Dmitriy
--- index.c.origMon Mar 3 16:21:13 2008
+++ index.c Mon Mar 3 17:01:48 2008
@@ -1313,6 +1313,7 @@
*copyuidp = NULL;
copyargs.nummsg = 0;
+copyargs.name = name;
index_forsequence(mailbox, sequence, usinguid, index_copysetup,
(char *)©args, NULL);
@@ -3409,6 +3410,9 @@
xrealloc((char *)copyargs->copymsg,
copyargs->msgalloc * sizeof(struct copymsg));
}
+
+syslog(LOG_ERR, "DSPAM-Hack index_copysetup(): %s -> %s, hdr %s",
mailbox->name,
+ copyargs->name, index_getheader(mailbox, msgno,
"X-DSPAM-Signature"));
copyargs->copymsg[copyargs->nummsg].uid = UID(msgno);
copyargs->copymsg[copyargs->nummsg].internaldate = INTERNALDATE(msgno);
--- index.h.origMon Mar 3 16:57:35 2008
+++ index.h Mon Mar 3 16:57:53 2008
@@ -126,6 +126,7 @@
struct copymsg *copymsg;
int nummsg;
int msgalloc;
+char *name;
};
struct mapfile {
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Hi,
thanks for the answer
Bron Gondwana schrieb:
> This would be even more useful:
>
> % gdb /opt/imap/libexec/master
>> run -d
> [ wait for it to segfault ]
>> bt
here's the output
(gdb) file /opt/imap/libexec/master
Reading symbols from /opt/cyrus-imapd-2.3.12/libexec/master...done.
Using host libthread_db library "/lib/libthread_db.so.1".
(gdb) run -d
Starting program: /opt/cyrus-imapd-2.3.12/libexec/master -d
Program received signal SIGSEGV, Segmentation fault.
0x00407416 in config_read_file (filename=0x530010
"/var/imap/imapd.conf") at ./../lib/libconfig.c:347
347 if (buf[len-1] == '\\') {
(gdb) bt
#0 0x00407416 in config_read_file (filename=0x530010
"/var/imap/imapd.conf") at ./../lib/libconfig.c:347
#1 0x00407fbf in config_read (alt_config=) at
./../lib/libconfig.c:215
#2 0x004082cf in masterconf_init (ident=0x40c0ef "master",
alt_config=0x0) at masterconf.c:84
#3 0x00405e0a in main (argc=2, argv=0x7fffa930bf58) at master.c:1681
but I got an answer from Andy Fiddaman which solved the problem:
> Take a look at http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059 and try
> the patch in there. It looks like the same problem.
and
> Take a look at http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059 and try
> the patch in there. It looks like the same problem.
It seems the same problem which is mentioned in bugzilla, because after
using '#' instead of blank lines resp. deleting them in imapd.conf, I was
able to start cyrus 2.3.12.
>> Compiling and running version 2.3.11 with the same options on the same
>> machine works fine.
>
> Are you able to post your /var/imap/imapd.conf file?
>
> I'm guessing there's something in there which is causing the segfault.
Yes as mentioned above I had blank lines in imapd.conf which caused the
segfault.
> Regards,
>
> Bron.
Thomas
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Mon, 21 Apr 2008, Ken Murchison wrote: > I don't have an OpenBSD system to test on, and I don't have any problem on > Linux or Solaris. I'd be happy to accept a patch if you can find the problem. This patch fixes the issue for me: http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059 -- Antoine Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Wed, 23 Apr 2008 10:33:27 +0200, "Thomas Robers - TuTech Innovation GmbH"
<[EMAIL PROTECTED]> said:
> Ken Murchison schrieb:
> > I am pleased to announce the release of Cyrus IMAPd 2.3.12. This
> > release should be considered production quality.
>
> I'm getting a segmentation fault when I try to start the master process.
> Compiling went fine on Debian etch (2.6.18-5-xen-amd64) with gcc version
> (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
>
> [...]
>
> open("/var/imap/imapd.conf", O_RDONLY) = 3
> fstat(3, {st_mode=S_IFREG|0640, st_size=1366, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
> = 0x2b6c1cebe000
> read(3, "# (21.02.2008)\n## directori"..., 4096) = 1366
> --- SIGSEGV (Segmentation fault) @ 0 (0) ---
> +++ killed by SIGSEGV +++
> Process 26096 detached
This would be even more useful:
% gdb /opt/imap/libexec/master
> run -d
[ wait for it to segfault ]
> bt
> Compiling and running version 2.3.11 with the same options on the same
> machine works fine.
Are you able to post your /var/imap/imapd.conf file?
I'm guessing there's something in there which is causing the segfault.
Regards,
Bron.
--
Bron Gondwana
[EMAIL PROTECTED]
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released (fwd)
Andy Fiddaman schrieb: > ; On Wed, 23 Apr 2008, Thomas Robers - TuTech Innovation GmbH wrote: > ; ; Hello, > ; ; > ; ; I'm getting a segmentation fault when I try to start the master process. > ; ; Compiling went fine on Debian etch (2.6.18-5-xen-amd64) with gcc version > ; ; (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21) > ; > ; Take a look at http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059 and try > ; the patch in there. It looks like the same problem. > > Or as a workaround, remove all blank lines from the config file or use > # > instead. > > A. > Hi, I tried removing the blank lines resp. using '#' and now cyrus works. Thanks for the help. Thomas Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released (fwd)
; On Wed, 23 Apr 2008, Thomas Robers - TuTech Innovation GmbH wrote: ; ; Hello, ; ; ; ; I'm getting a segmentation fault when I try to start the master process. ; ; Compiling went fine on Debian etch (2.6.18-5-xen-amd64) with gcc version ; ; (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21) ; ; Take a look at http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059 and try ; the patch in there. It looks like the same problem. Or as a workaround, remove all blank lines from the config file or use # instead. A. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Wed, 23 Apr 2008, Thomas Robers - TuTech Innovation GmbH wrote: ; Hello, ; ; I'm getting a segmentation fault when I try to start the master process. ; Compiling went fine on Debian etch (2.6.18-5-xen-amd64) with gcc version ; (GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21) Take a look at http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3059 and try the patch in there. It looks like the same problem. A. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Ken Murchison schrieb:
> I am pleased to announce the release of Cyrus IMAPd 2.3.12. This
> release should be considered production quality.
>
>
> Noteworthy changes:
>
> * Added statuscache.db to cache IMAP STATUS data which
> significantly reduces the amount of I/O necessary when neither the
> mailbox nor \Seen state has changed -- courtesy of Fastmail.fm
> * Added option to unexpunge to restore messages by time interval --
> courtesy of David Carter
> * Implemented undocumented IMAP SCAN extension, which allows
> Pine/Alpine to do cross-mailbox searches -- based on work of David Carter
> * Implemented incremental squat updates (see squatter.8) -- courtesy of
> David Carter
> * Fixed major bugs in reconstruct -k implementation -- courtesy of David
> Carter
>
> For full details, please see doc/changes.html and
> doc/install-upgrade.html which are included in the distribution.
>
> URLs for this release:
> ftp://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.12.tar.gz
> or
> http://ftp.andrew.cmu.edu/pub/cyrus/cyrus-imapd-2.3.12.tar.gz
>
>
> Questions and comments can be directed to
> info-cyrus@lists.andrew.cmu.edu (public list), or
> [EMAIL PROTECTED]
>
Hello,
I'm getting a segmentation fault when I try to start the master process.
Compiling went fine on Debian etch (2.6.18-5-xen-amd64) with gcc version
(GCC) 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)
configure options were:
[...]
CC=gcc
CFLAGS="-O2 -g -fPIC" \
CPPFLAGS="-I/usr/local/include -I/usr/include"
LDFLAGS="-L/usr/local/lib -L/usr/lib"
export CPPFLAGS LDFLAGS CC CFLAGS
./configure --prefix=$PREFIX \
--exec-prefix=$EPREFIX \
--bindir=$BINDIR \
--sbindir=$SBINDIR \
--libexecdir=$LIBEXECDIR \
--datadir=$DATADIR \
--sysconfdir=$SYSCONFDIR \
--sharedstatedir=$SHAREDSTATEDIR \
--localstatedir=$LOCALSTATEDIR \
--enable-idled \
--enable-sieve \
--enable-nntp \
--enable-murder \
--enable-oldsievename \
--enable-listext \
--enable-replication \
--enable-netscape-hack \
--disable-gssapi \
--with-cyrus-prefix=$PREFIX \
--with-service-path=$EPREFIX/libexec \
--with-cyrus-user=cyrus \
--with-cyrus-group=cyrus \
--with-statedir=$LOCALSTATEDIR \
--with-auth=unix \
--with-bdb=/usr \
--with-bdb-libdir=/usr/lib \
--with-bdb-inc-dir=/usr/include \
--with-ldap=/usr \
--with-openssl=/usr \
--with-snmp=/usr \
--with-pidfile=/var/run/cyrus/cyrmaster.pid \
--with-sasl=/usr \
--with-perl=/usr/bin/perl \
--with-libwrap=/usr
'strace /opt/imap/libexec/master -d' gives (I didn't want to post the whole
output; this is what's printed just before the segmentation fault):
open("/var/imap/imapd.conf", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0640, st_size=1366, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x2b6c1cebe000
read(3, "# (21.02.2008)\n## directori"..., 4096) = 1366
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
Process 26096 detached
Compiling and running version 2.3.11 with the same options on the same machine
works fine.
I'm not such an expert to see what went wrong but perhaps someone can give any
further suggestions.
Thanks!
Thomas
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Antoine Jacoutot wrote: > On Mon, 21 Apr 2008, Ken Murchison wrote: > >> I am pleased to announce the release of Cyrus IMAPd 2.3.12. This >> release should be considered production quality. > > I'm getting a regression here: > > (gdb) run > Starting program: /usr/local/libexec/cyrus-imapd/master > > Program received signal SIGSEGV, Segmentation fault. > 0x0040773c in config_read_file (filename=0x44df > "/etc/imapd.conf") at libconfig.c:347 > 347 libconfig.c: No such file or directory. > in libconfig.c > > This is on OpenBSD-current (tested on different archs with different > configurations. I don't have an OpenBSD system to test on, and I don't have any problem on Linux or Solaris. I'd be happy to accept a patch if you can find the problem. -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Mon, 21 Apr 2008, Ken Murchison wrote: > Is this a clean build and a fresh restart? Yup... -- Antoine Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
Is this a clean build and a fresh restart? Antoine Jacoutot wrote: > On Mon, 21 Apr 2008, Ken Murchison wrote: > >> I am pleased to announce the release of Cyrus IMAPd 2.3.12. This >> release should be considered production quality. > > I'm getting a regression here: > > (gdb) run > Starting program: /usr/local/libexec/cyrus-imapd/master > > Program received signal SIGSEGV, Segmentation fault. > 0x0040773c in config_read_file (filename=0x44df > "/etc/imapd.conf") at libconfig.c:347 > 347 libconfig.c: No such file or directory. > in libconfig.c > > This is on OpenBSD-current (tested on different archs with different > configurations. > -- Kenneth Murchison Systems Programmer Project Cyrus Developer/Maintainer Carnegie Mellon University Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus IMAPd 2.3.12 Released
On Mon, 21 Apr 2008, Ken Murchison wrote: > I am pleased to announce the release of Cyrus IMAPd 2.3.12. This > release should be considered production quality. I'm getting a regression here: (gdb) run Starting program: /usr/local/libexec/cyrus-imapd/master Program received signal SIGSEGV, Segmentation fault. 0x0040773c in config_read_file (filename=0x44df "/etc/imapd.conf") at libconfig.c:347 347 libconfig.c: No such file or directory. in libconfig.c This is on OpenBSD-current (tested on different archs with different configurations. -- Antoine Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
