Rainer Gerhards wrote: > > -----Original Message----- > > From: [email protected] [mailto:rsyslog- > > [email protected]] On Behalf Of Dražen Kacar
> > The second one is followed by ABORT_FINALIZE and it will return an > > error > > which can lead to the situation I described. That's the only such > > condition in the code I found and my disk queues do get filled up from > > time to time, while the testing period lasts. > > > > I have 23 such messages per day, but only one day of testing. I think > > it > > should be logged to give an administrator an indication about what's > > going > > on. > Ah! Logging at this point is a bit problematic, because if the queue is full, > there is no space left for logging. That depends, I hope. I have set a separate OS partition for rsyslog queues. All other partitions on the system will be empty enough, and my log file for errors resides on one of them. The disk assisted action queues are in their own ruleset and the action queue for error messages is in the default ruleset, purely in memory. This works reasonably fine, otherwise the TCP module wouldn't be able to log its error messages. But yes, there are situations when the message wouldn't be written anywhere. > Probably the best approach is to report > the exact error and *hope* that the queue has drained enough so that there is > space left to actually log that. That's in case that the queue which is full and the queue for the error message are the same queue, right? > Please note that the disk queue subsystem is pretty slow, especially after > the last changes. I really needs to be redone, but this is such a big effort > that I currently can not do it without a sponsor :( I've noticed that it takes a lot of time to empty 14 GB of disk queue. :-) Rsyslog barely manages to keep up with the incoming messages while doing that on my system, but it manages. I am more concerned with fail-safe operations, though. Once, while the partition with the queue files was full, rsyslog crashed. I have a monitoring program which restarts it immediately, and it crashed again and again. I cleaned up the mess by deleting everything on disk and rsyslog started working again. I didn't have the time to investigate the problem at that moment. My guess was that one of the files on disk was truncated because there was no space left and that (part of) the rsyslog code doesn't have checks for this possibility. That was on a test system which was intentionally being left to fill up the partition to see what happens. It shouldn't fill up in production. I hope. :-) And the partition has been filled up a number of times since then, but the crashing never happened again. > I'll have a look at the patch in detail. It probably is worth trying to log > an error message, especially if this problem occurs in an action queue. That's where it was, I think. > I am very grateful for all your help, and sorry that I am extremely busy with > a couple of other things during all of our conversations. Hopefully it will > get better in the not so distant future (when I hope to be able to re-focus > on the next iteration of rsyslog tuning). I'm very grateful for your help, which happens to be terse, but it contains all the information that I need. You're one of the most responsive maintainers, as far as my experience goes, even though you don't have much time at the moment. It's a pleasure working with you. > > Rainer > > > > Rainer Gerhards wrote: > > > This from ./tcpsrv.c, not gssapi. There seems to be some condition > > triggered > > > which does not emit a message by itself, maybe removed because it > > occurred > > > too frequently. > > > > > > Rainer > > > > > > > -----Original Message----- > > > > From: [email protected] [mailto:rsyslog- > > > > [email protected]] On Behalf Of Dražen Kacar > > > > Sent: Wednesday, January 19, 2011 12:10 PM > > > > To: [email protected] > > > > Subject: [rsyslog] Tearing down TCP Session in the log file > > > > > > > > Hello. > > > > > > > > I have rsyslog 5.6.2 and I just tried to receive messages via TCP. > > The > > > > sender is syslog-ng, I don't know exactly which version, not under > > my > > > > control. The transfer mostly works, as far as I can see without > > strict > > > > testing. > > > > > > > > But in rsyslog's log file I see several messages which look like > > this: > > > > > > > > rsyslogd-2105: Tearing down TCP Session - see previous messages > > for > > > > reason(s) > > > > [try http://www.rsyslog.com/e/2105 ] > > > > > > > > And then there is no previous message which is different than this > > one > > > > and > > > > if I go back far enough there is the startup message. There's > > nothing > > > > else in the log (produced by the *.* selector at the end of the > > config > > > > file). > > > > > > > > I tried to check the URL above, but I'm getting an error page with > > > > says: > > > > > > > > Fatal error: Call to undefined function SendEmailToAdiscon() in > > > > /home/adisconweb/www/kb.monitorware.com/kbeventdb.php on line > > 217 > > > > > > > > That's generated by > > > > http://kb.monitorware.com/kbeventdb-list-1-Adiscon-rsyslog- > > rsyslogd- > > > > 2105.html > > > > > > > > I checked the source and the only file with the above message is > > > > plugins/imgssapi/imgssapi.c in the > > > > > > > > if(tcps_sess.DataRcvd(pSess, buf, ret) ! = RS_RET_OK) > > > > > > > > block. I didn't even know I was using gssapi plugin. My > > configuration > > > > related to TCP is: > > > > > > > > $ModLoad imtcp > > > > $InputTCPServerBindRuleset indata > > > > $InputTCPServerRun 514 > > > > > > > > and the rest is setup for the UDP receiver and storing messages. I > > > > checked > > > > with Google and rsyslog bug tracker for "Tearing down TCP Session", > > but > > > > with zero results (except for the source code matches). > > > > > > > > I am willing to debug this and submit a patch (which would at least > > log > > > > the missing previous message), but before I do, I'd just like to > > ask if > > > > someone has seen this before or has any idea what's going on. I'm a > > bit > > > > confused with gssapi module being used. My OS is CentOS 5.5, if > > that's > > > > relevant. > > > > > > > > -- > > > > .-. .-. Yes, I am an agent of Satan, but my duties are > > largely > > > > (_ \ / _) ceremonial. > > > > | > > > > | [email protected] > > > > _______________________________________________ > > > > rsyslog mailing list > > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > > http://www.rsyslog.com > > > _______________________________________________ > > > rsyslog mailing list > > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > > http://www.rsyslog.com > > > > -- > > .-. .-. Yes, I am an agent of Satan, but my duties are largely > > (_ \ / _) ceremonial. > > | > > | [email protected] > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com -- .-. .-. Yes, I am an agent of Satan, but my duties are largely (_ \ / _) ceremonial. | | [email protected] _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
