I am using a similar setup (HP proliant, RELP, Centos 5.9 X86_64) for the central syslog receiver. I'm still in the testing phase. I haven't been happy with the client side of things but that is a different story. I have not had problems with the RELP receiver side of things.
Two things to check. 1) I (and others) have seen poor RAID5 performance with the HP 4xx controllers. Do some IO benchmarks if you can reconfigure as a RAID 0+1. search for "hp raid5 performance". 2) It could be the postgres backend. Can you try writing text files on the receiver (instead of to the DB)? That may tell you something. Alan Edmonds -----Original Message----- From: rsyslog-boun...@lists.adiscon.com [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang Sent: 11 March 2013 21:36 To: rsyslog-users Subject: Re: [rsyslog] Big issues with Rsyslog listening on port 514 with IMRELP On Mon, 11 Mar 2013, Nicolas HAHN wrote: > Hello all, > > I'm writing for the first time to this mailing-list in the hope that somebody already experimented the issue I describe below. > > At first, here is the Linux server profile on which is running Rsyslog, if it can have an importance: > - Server type: physical and dedicated (not a VM). HP Proliant > - OS: RHEL 6.4 (yum updated this morning) > - Arch: x86_64 > - Nb CPU cores: 24 > - RAM: 256 GB > - Storage: 3.6 TB with Hardware RAID-5 > - Network: 4 nics with 2 bonding interfaces > - SELINUX is totally disabled > - There are absolutely no iptables rules defined > > Here below are the rsyslog packages installed: > rsyslog-7.2.6-1.el6.x86_64 > rsyslog-pgsql-7.2.6-1.el6.x86_64 > rsyslog-relp-7.2.6-1.el6.x86_64 > libestr-0.1.3-1.el6.x86_64 > librelp-1.0.1-1.el6.x86_64 > libee-0.4.1-1.el6.x86_64 > > Here below is the description of the issue: > > 1) we use Rsyslog with imrelp module, on port TCP-20514 > 2) at first, we don't send traffic on Rsyslog, we can see that the daemon is listening on port 20514: > # netstat -ann | grep 514 > tcp 0 0 0.0.0.0:20514 0.0.0.0:* LISTEN > tcp 0 0 :::20514 :::* LISTEN > > 3) as soon as syslog traffic is sent to the server, imrelp stop to listen. A few remote servers only were successfully bound to the server port, and finally rsyslog connections with the remote senders stop. Remains finally only one or two connected remote servers. Doing a netstat, the first line in copy above is not there: imrelp is not listening any more. > 4) if we do a tcpdump -nni bond0 port 20514 on the rsyslog server, suddenly Rsyslog daemon take 100% of one CPU core. We've seen it eating until 140%. > 5) if we do a telnet from any remote server to the rsyslog server on port 20514, then the connection is refused. It's not possible to connect on the rsyslog daemon, even locally (telnet localhost 20514). This makes me thing that you have SELinux or firewall rules blocking you. > We tried since 5 days, so many things: > 1) we tried originally to use imrelp on port 514 (instead of 20514 now), same issues > 2) we tried versions 5.10.x, latest version 6, same issues > 3) we tried to run rsyslog in debug mode: symptoms are the same in debug mode. > it still loose its ability to LISTEN on the port, and debug mode stops to > display things on the screen. But rsyslog is still running in the process > table with its -d flag. when rsyslog stops, what shows up in the debug logs? David Lang > 4) we tried to configure rsyslog to use imtcp on port 514 instead of imrelp on > port 514 or whatever other port. Here, it never loose its ability to accept > connections and to listen on the network socket. In debug mode, we have > permanently things appearing on the display. > 5) we have other rsyslog servers based on RHEL 5.x (latest 5.10 version, > including also IMRELP and PGSQL modules): they are running like a charm using > exactly the same /etc/rsyslog.conf > > In conclusion: > -this issue is of course very annoying, we cannot understand what is happening: all other daemons correctly listen on their socket and never loose it (postgresql database, ssh daemon, http server, ...) > - we absolutely need to use our rsyslog server with RELP module, and this is the only one not working for now > - we cannot explain why rsyslog cannot keep its network socket and keep listening on it. > > We would really need to know if other users experimented the same behavior, and if yes, if they found solutions. > We would really appreciate any help on that, and if the reason why those issues are found, an urgent fix :-) > > If you ask us to post the rsyslog.conf file, we can do it but we'll have to "hide" various things ;-) > > Thank you very much for any help > > KR. > Nicolas > - > United Nations International Computing Center > Geneva > > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. > _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT. NOTICE AND DISCLAIMER This email (including attachments) is confidential. If you are not the intended recipient, notify the sender immediately, delete this email from your system and do not disclose or use for any purpose. Deutsche Telekom (UK) Limited Company Registered Number: 3951860 Registered Office Address: Hatfield Business Park, Hatfield, Hertfordshire, AL10 9BW Registered in England and Wales _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
