Hello Alan, Well for the story here, I switched from SAN to hardware RAID 5 in the server itself because I've much better performances... This is due the fact our SAN is satured :-X. The IOWait on our SAN was 20 to 70%. Now, the IOWait is between 0 to 2%...
Writing text files on the receiver has always been fine. KR. Nicolas - United Nations International Computing Center Geneva Alan Edmonds <alan.edmo...@telekom.com> a écrit : > I am using a similar setup (HP proliant, RELP, Centos 5.9 X86_64) for > the central syslog receiver. I'm still in the testing phase. I haven't > been happy with the client side of things but that is a different story. > I have not had problems with the RELP receiver side of things. > > Two things to check. > > 1) I (and others) have seen poor RAID5 performance with the HP 4xx > controllers. Do some IO benchmarks if you can reconfigure as a RAID > 0+1. search for "hp raid5 performance". > > 2) It could be the postgres backend. Can you try writing text files on > the receiver (instead of to the DB)? That may tell you something. > > Alan Edmonds > > > > > -----Original Message----- > From: rsyslog-boun...@lists.adiscon.com > [mailto:rsyslog-boun...@lists.adiscon.com] On Behalf Of David Lang > Sent: 11 March 2013 21:36 > To: rsyslog-users > Subject: Re: [rsyslog] Big issues with Rsyslog listening on port 514 > with IMRELP > > On Mon, 11 Mar 2013, Nicolas HAHN wrote: > >> Hello all, >> >> I'm writing for the first time to this mailing-list in the hope that > somebody already experimented the issue I describe below. >> >> At first, here is the Linux server profile on which is running > Rsyslog, if it can have an importance: >> - Server type: physical and dedicated (not a VM). HP Proliant >> - OS: RHEL 6.4 (yum updated this morning) >> - Arch: x86_64 >> - Nb CPU cores: 24 >> - RAM: 256 GB >> - Storage: 3.6 TB with Hardware RAID-5 >> - Network: 4 nics with 2 bonding interfaces >> - SELINUX is totally disabled >> - There are absolutely no iptables rules defined >> >> Here below are the rsyslog packages installed: >> rsyslog-7.2.6-1.el6.x86_64 >> rsyslog-pgsql-7.2.6-1.el6.x86_64 >> rsyslog-relp-7.2.6-1.el6.x86_64 >> libestr-0.1.3-1.el6.x86_64 >> librelp-1.0.1-1.el6.x86_64 >> libee-0.4.1-1.el6.x86_64 >> >> Here below is the description of the issue: >> >> 1) we use Rsyslog with imrelp module, on port TCP-20514 >> 2) at first, we don't send traffic on Rsyslog, we can see that the > daemon is listening on port 20514: >> # netstat -ann | grep 514 >> tcp 0 0 0.0.0.0:20514 0.0.0.0:* > LISTEN >> tcp 0 0 :::20514 :::* > LISTEN >> >> 3) as soon as syslog traffic is sent to the server, imrelp stop to > listen. A few remote servers only were successfully bound to the server > port, and finally rsyslog connections with the remote senders stop. > Remains finally only one or two connected remote servers. Doing a > netstat, the first line in copy above is not there: imrelp is not > listening any more. >> 4) if we do a tcpdump -nni bond0 port 20514 on the rsyslog server, > suddenly Rsyslog daemon take 100% of one CPU core. We've seen it eating > until 140%. >> 5) if we do a telnet from any remote server to the rsyslog server on > port 20514, then the connection is refused. It's not possible to connect > on the rsyslog daemon, even locally (telnet localhost 20514). > > This makes me thing that you have SELinux or firewall rules blocking > you. > >> We tried since 5 days, so many things: >> 1) we tried originally to use imrelp on port 514 (instead of 20514 > now), same issues >> 2) we tried versions 5.10.x, latest version 6, same issues > >> 3) we tried to run rsyslog in debug mode: symptoms are the same in > debug mode. >> it still loose its ability to LISTEN on the port, and debug mode stops > to >> display things on the screen. But rsyslog is still running in the > process >> table with its -d flag. > > when rsyslog stops, what shows up in the debug logs? > > David Lang > >> 4) we tried to configure rsyslog to use imtcp on port 514 instead of > imrelp on >> port 514 or whatever other port. Here, it never loose its ability to > accept >> connections and to listen on the network socket. In debug mode, we > have >> permanently things appearing on the display. > >> 5) we have other rsyslog servers based on RHEL 5.x (latest 5.10 > version, >> including also IMRELP and PGSQL modules): they are running like a > charm using >> exactly the same /etc/rsyslog.conf >> >> In conclusion: >> -this issue is of course very annoying, we cannot understand what is > happening: all other daemons correctly listen on their socket and never > loose it (postgresql database, ssh daemon, http server, ...) >> - we absolutely need to use our rsyslog server with RELP module, and > this is the only one not working for now >> - we cannot explain why rsyslog cannot keep its network socket and > keep listening on it. >> >> We would really need to know if other users experimented the same > behavior, and if yes, if they found solutions. >> We would really appreciate any help on that, and if the reason why > those issues are found, an urgent fix :-) >> >> If you ask us to post the rsyslog.conf file, we can do it but we'll > have to "hide" various things ;-) >> >> Thank you very much for any help >> >> KR. >> Nicolas >> - >> United Nations International Computing Center >> Geneva >> >> >> ---------------------------------------------------------------- >> This message was sent using IMP, the Internet Messaging Program. >> _______________________________________________ >> rsyslog mailing list >> http://lists.adiscon.net/mailman/listinfo/rsyslog >> http://www.rsyslog.com/professional-services/ >> What's up with rsyslog? Follow https://twitter.com/rgerhards >> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST > if you DON'T LIKE THAT. >> > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad > of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you > DON'T LIKE THAT. > NOTICE AND DISCLAIMER > > This email (including attachments) is confidential. If you are not > the intended recipient, notify the sender immediately, delete this > email from your system and do not disclose or use for any purpose. > > Deutsche Telekom (UK) Limited > > Company Registered Number: 3951860 > > Registered Office Address: Hatfield Business Park, Hatfield, > Hertfordshire, AL10 9BW > > Registered in England and Wales > _______________________________________________ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a > myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT > POST if you DON'T LIKE THAT. > ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. _______________________________________________ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
