Bug#689053: rsyslog: slow or dead omysql backend can hang rsyslog

[Michael Biebl]

Control: tags -1 + moreinfo

On Fri, 28 Sep 2012 21:05:53 +0200 Dominik George  wrote:
> Package: rsyslog
> Version: 5.8.11-1+b1
> Severity: important
> 
> We recently added the omysql backend to rsyslog to centralize all logging to 
> a MySQL database.
> The link to the database was a bit flakey lately and a syslog client, slapd, 
> was producing tons
> of log lines for debugging purposes. This lead to rsyslog staying behind with 
> logging for
> nearly 30 minutes and apparently filling up all its buffers. That prevented 
> rsyslog from accepting
> more logs, which even froze slapd (which is a bug in slapd as well, see 
> #689025).
> 
> rsyslog should rather drop messages to a certain backend if it is not 
> available and keep on
> logging to available backends or at least make this behaviour configurable so 
> a broken logging backend
> can not freeze rsyslog.
> 


Is this still an issue with recent versions of rsyslog?
If so, can you please share your rsyslog configuration.


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?



signature.asc
Description: OpenPGP digital signature

Bug#689053: rsyslog: slow or dead omysql backend can hang rsyslog

[Florian Ernst]

Package: rsyslog-pgsql
Version: 7.3.15-1
Followup-For: Bug #689053

Hello there,

FWIW, I just experienced similar behavior after the recent upgrade in
testing, so I'm adding this here instead of filing a new bug report.

Yesterday I upgraded as follows:

[UPGRADE] rsyslog:amd64 5.8.11-3 -> 7.3.15-1
[UPGRADE] rsyslog-pgsql:amd64 5.8.11-3 -> 7.3.15-1

As long as I leave logging to postgres enabled as per the following
config

#  sed -e 's#,[^,]\+$#,password#' /etc/rsyslog.d/pgsql.conf
| ### Configuration file for rsyslog-pgsql
| ### Changes are preserved
| 
| $ModLoad ompgsql
| *.* :ompgsql:localhost,Syslog,rsyslog,password

rsyslog will only log for ~30 seconds and then stop logging. After that
no more entries appear in either DB or logfiles under /var/log.

However, unlike in OP's situation, the DB wasn't flakey at all. But
still, the second paragraph from OP's mail seems to apply here:
"rsyslog should rather drop messages to a certain backend if it is not
available and keep on logging to available backends or at least make
this behaviour configurable so a broken logging backend can not freeze
rsyslog."

Cheers,
Flo

-- System Information:
Debian Release: jessie/sid
  APT prefers testing
  APT policy: (990, 'testing'), (50, 'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.2.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages rsyslog-pgsql depends on:
ii  dbconfig-common1.8.47+nmu1
ii  debconf [debconf-2.0]  1.5.50
ii  libc6  2.17-3
ii  libpq5 9.1.9-1
ii  rsyslog7.3.15-1
ii  ucf3.0025+nmu3

Versions of packages rsyslog-pgsql recommends:
ii  postgresql-client  9.3+142really9.1+143
ii  postgresql-client-9.1 [postgresql-client]  9.1.9-1

Versions of packages rsyslog-pgsql suggests:
ii  postgresql  9.3+142really9.1+143

-- debconf information:
  rsyslog-pgsql/db/app-user: rsyslog
  rsyslog-pgsql/pgsql/manualconf:
* rsyslog-pgsql/pgsql/authmethod-user: ident
  rsyslog-pgsql/db/dbname: Syslog
  rsyslog-pgsql/dbconfig-reinstall: false
  rsyslog-pgsql/missing-db-package-error: abort
  rsyslog-pgsql/pgsql/admin-user: postgres
  rsyslog-pgsql/database-type: pgsql
  rsyslog-pgsql/internal/skip-preseed: false
  rsyslog-pgsql/install-error: abort
  rsyslog-pgsql/remote/port:
  rsyslog-pgsql/dbconfig-upgrade: true
  rsyslog-pgsql/pgsql/no-empty-passwords:
  rsyslog-pgsql/remote/newhost:
  rsyslog-pgsql/remote/host:
  rsyslog-pgsql/upgrade-backup: true
  rsyslog-pgsql/passwords-do-not-match:
  rsyslog-pgsql/purge: false
  rsyslog-pgsql/upgrade-error: abort
  rsyslog-pgsql/pgsql/method: unix socket
  rsyslog-pgsql/remove-error: abort
  rsyslog-pgsql/internal/reconfiguring: false
  rsyslog-pgsql/pgsql/changeconf: false
  rsyslog-pgsql/pgsql/authmethod-admin: ident
  rsyslog-pgsql/dbconfig-remove:
* rsyslog-pgsql/dbconfig-install: true


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#689053: rsyslog: slow or dead omysql backend can hang rsyslog

[Dominik George]

Package: rsyslog
Version: 5.8.11-1+b1
Severity: important

We recently added the omysql backend to rsyslog to centralize all logging to a 
MySQL database.
The link to the database was a bit flakey lately and a syslog client, slapd, 
was producing tons
of log lines for debugging purposes. This lead to rsyslog staying behind with 
logging for
nearly 30 minutes and apparently filling up all its buffers. That prevented 
rsyslog from accepting
more logs, which even froze slapd (which is a bug in slapd as well, see 
#689025).

rsyslog should rather drop messages to a certain backend if it is not available 
and keep on
logging to available backends or at least make this behaviour configurable so a 
broken logging backend
can not freeze rsyslog.

-- System Information:
Debian Release: wheezy/sid
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-3-amd64 (SMP w/2 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages rsyslog depends on:
ii  initscripts  2.88dsf-32
ii  libc62.13-35
ii  lsb-base 4.1+Debian7
ii  zlib1g   1:1.2.7.dfsg-13

Versions of packages rsyslog recommends:
ii  logrotate  3.8.1-4

Versions of packages rsyslog suggests:
pn  rsyslog-doc 
pn  rsyslog-gnutls  
pn  rsyslog-gssapi  
ii  rsyslog-mysql   5.8.11-1+b1
pn  rsyslog-relp

-- no debconf information


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org