> -----Original Message----- > From: [email protected] [mailto:rsyslog- > [email protected]] On Behalf Of Jason Antman > Sent: Friday, November 05, 2010 5:57 PM > To: rsyslog-users > Subject: Re: [rsyslog] segfault with dynamic filenames > > I haven't run it with valgrind yet... honestly I don't think I've used > valgrind more than once or twice, I'd need to do a bit of reading up to > remember how to use it.
It actually is extemely easy... yum install valgrind valgrind rsyslogd ... your regular rsyslogd command line that's it ;) Rainer > > This isn't a matter of *if* the issue reappears. Every time rsyslogd > tries to write out a message matching a dynafile output, it dies > (within > a few ms of starting, or whenever it actually tries to write the first > line out to disk). > > At the moment, I have the issue localized to the call to > getClockFileAccess() on line 517 of tools/omfile.c, and am continuing > to > trace it through the code. I'll see if one of the more c-skilled guys > can give me a hand with valgrind. > > -Jason > > Rainer Gerhards wrote: > > An you run it under valgrind control? If the issue reappears, it > should give > > us *very* good hints of where to look at... > > > > RAiner > > > > > >> -----Original Message----- > >> From: [email protected] [mailto:rsyslog- > >> [email protected]] On Behalf Of Jason Antman > >> Sent: Friday, November 05, 2010 5:26 PM > >> To: rsyslog-users > >> Subject: Re: [rsyslog] segfault with dynamic filenames > >> > >> Thanks for the info! > >> > >> Just sent this to Tom, forgot to CC the list.... > >> > >> I just rebuilt the RPM with all debugging features enabled and am > >> experiencing the same issue. > >> > >> I have: > >> - debug log output (rsyslogd -c 5 -dn, with RSYSLOG_DEBUG env > variable > >> set to "LogFuncFlow FileTrace=omfile.c Debug") > >> - strace output > >> - RPM spec file including configure options, etc. > >> - the Fedora patch that is being applied > >> - the actual RPMs, if they're of interest, and a source rpm > >> > >> Which of this is of use? Should I email it? Open a ticket and > attach? > >> > >> Here are the last few lines before the crash: > >> 2954.318300688:b7f06b90: Processing next action > >> 2954.318308532:b7f06b90: Called action(NotAllMark), logging to > builtin- > >> file > >> 2954.318315864:b7f06b90: Called action(Batch), logging to builtin- > file > >> 2954.318382400:b7f06b90: omfile.c:682: beginTransaction: enter > >> 2954.318390551:b7f06b90: omfile.c:682: beginTransaction: exit: 0 > >> 2954.318397068:b7f06b90: Action 0x9fbb3a0 transitioned to state: itx > >> 2954.318404578:b7f06b90: entering actionCalldoAction(), state: itx > >> 2954.318411997:b7f06b90: omfile.c:697: doAction: enter > >> 2954.318418085:b7f06b90: file to log to: LocalHost > >> 2954.318424740:b7f06b90: omfile.c:634: writeFile: enter > >> 2954.318431685:b7f06b90: omfile.c:493: prepareDynFile: enter > >> (and then it dies, in the middle of a select() call) > >> > >> I'm going to start looking through omfile.c and perhaps adding some > of > >> my own instrumentation just so I can localize it a bit more. > >> > >> Thanks for any advice you can provide, > >> Jason Antman > >> > >> Tom Bergfeld wrote: > >> > >>> Hi Jason, > >>> > >>> unfortunately we were not able to reproduce your issue in Fedora > 13. > >>> We will create an environment with centos and try it again. > >>> > >>> Tom > >>> > >>> > >>> -----Ursprüngliche Nachricht----- > >>> Von: [email protected] > >>> [mailto:[email protected]] Im Auftrag von Jason > >>> > >> Antman > >> > >>> Gesendet: Donnerstag, 4. November 2010 22:47 > >>> An: rsyslog-users > >>> Betreff: [rsyslog] segfault with dynamic filenames (was: Rsyslog > >>> evaluationquestions) > >>> > >>> Thanks to both of you guys for your answers. FYI, our days have 24 > >>> > >> hours > >> > >>> in them :) > >>> > >>> At this point I just got a config file worked up. I got the regexes > >>> working to do all of the matching I need into separate fields (the > >>> online regex tester was a big help), and upgraded to 5.6.0 (CentOS > >>> > >> 5.5, > >> > >>> in-house RPM build) in order to of bind imudp to a ruleset (so we > can > >>> have one ruleset for localhost/unix socket, and one ruleset for all > >>> remote hosts). > >>> > >>> Unfortunately, I'm getting a segfault almost immediately on startup > >>> > >> due > >> > >>> to the following lines: > >>> $template > >>> > >>> > >> > RemoteHost,"/var/log/HOSTS/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%syslogfa > >> cility > >> > >>> -text%.log" > >>> *.* ?RemoteHost > >>> > >>> I believe I've narrowed the problem down to any templated (dynamic) > >>> filename... even if I dump everything external directly to > >>> /var/log/remote and add the following in my local ruleset: > >>> $template LocalHost,"/var/log/LOCAL/%syslogfacility-text%.log" > >>> *.* ?LocalHost > >>> > >>> I still get a segfault. The last line in the output running with - > dn > >>> > >> is > >> > >>> always: > >>> "file to log to: RemoteHost" (or LocalHost, when I used that > config). > >>> > >>> Has anyone seen anything like this? I have rsyslog 2.0.6 running at > >>> another site and dynamic filenames work fine... sort of troubling > >>> > >> that > >> > >>> 5.6.0 is having issues with them... > >>> > >>> Thanks, > >>> Jason Antman > >>> > >>> Aaron Wiebe wrote: > >>> > >>> > >>>>> 3) Assuming an even distribution over time (not quite accurate), > >>>>> > >> any > >> > >>>>> thoughts on how dumping ~2M lines/day of syslog to MySQL on a VM > >>>>> > >> (Xen) > >> > >>>>> with a single 2.8GHz CPU and 512MB RAM would go? > >>>>> > >>>>> > >>>>> > >>>> 2 million lines a day is 70 lines per second assuming an 8 hour > day. > >>>> MySQL, properly configured, will eat that. Rsyslog won't even > >>>> > >> notice > >> > >>>> that log level. > >>>> > >>>> Your table sizes may be problematic over time, and your ability to > >>>> query may impact things. Regardless, your concern should be > mysql, > >>>> not rsyslog. > >>>> > >>>> -Aaron > >>>> _______________________________________________ > >>>> rsyslog mailing list > >>>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>>> http://www.rsyslog.com > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> _______________________________________________ > >>> rsyslog mailing list > >>> http://lists.adiscon.net/mailman/listinfo/rsyslog > >>> http://www.rsyslog.com > >>> > >>> > >>> > >> _______________________________________________ > >> rsyslog mailing list > >> http://lists.adiscon.net/mailman/listinfo/rsyslog > >> http://www.rsyslog.com > >> > > _______________________________________________ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com > > > > > > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com
