Bug#730202: fail2ban should not use space in ident when logging to syslog

2013-11-22 Thread Allan Wind 
On 2013-11-22 15:36:56, Yaroslav Halchenko wrote:
> well -- is that a part of any convention (besides at your work? ;-) )
>
> I do not see in my logs any priority= listed while some other daemons do
> prepend with level:

You are absolutely correct that our (weird) configuration is not 
a good argument, however, most programs will the priority field 
of syslog(3) and not include the it in the msg part.  Here are 
some examples of that:

2013-11-22T00:56:10.463+00:00 vent syslog-ng[2353]: Configuration 
reload request received, reloading configuration;
2013-11-22T00:56:10.579+00:00 vent anacron[2478]: Job 
`cron.daily' terminated
2013-11-22T00:56:40.492+00:00 vent postfix/pickup[3081]: 
782A1680CAD: uid=0 from=
2013-11-22T00:58:38.792+00:00 vent dhclient: DHCPREQUEST on eth1 
to 192.168.0.10 port 67
...

ddclient and named are counter examples, and it appears I have 
none of those on my laptop.

Per your last update we ended up the right place.  Thanks for 
being most helpful.


/Allan
-- 
Allan Wind
Life Integrity, LLC



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730202: fail2ban should not use space in ident when logging to syslog

2013-11-22 Thread Yaroslav Halchenko 
btw -- here it will be
https://github.com/fail2ban/fail2ban/pull/451

PS I removed loglevel completely from msg to syslog

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730202: fail2ban should not use space in ident when logging to syslog

2013-11-22 Thread Yaroslav Halchenko 

On Fri, 22 Nov 2013, Allan Wind wrote:

> On 2013-11-22 13:46:50, Yaroslav Halchenko wrote:
> > what about making it even better:

> > formatter = logging.Formatter("%(name)s[%(process)d]: %(levelname)-7s 
> > %(message)s")

> > so we report the pid as well, stay together with the name while still 
> > keeping
> > indentation for the levelname conssitent for easier visual parsing.  you can
> > try attached script to "play" with it.. it should look like

> Yeah, it's nice touch to include the pid.  I would still suggest 
> using the syslog level instead of encoding it into the message 
> part.

> At work we filter on priority and include that the level as a 
> tagged field like this with syslog-ng:

> template t_message {
>   template("$ISODATE $HOST ${MSGHDR}priority=$PRIORITY $MSG\n");
>   template_escape(no);
> };

well -- is that a part of any convention (besides at your work? ;-) )
I do not see in my logs any priority= listed while some other daemons do
prepend with level:

$> zgrep -i warning /var/log/daemon.log*
/var/log/daemon.log:May 15 13:39:32 washoe named[13929]: Warning: 
'empty-zones-enable/disable-empty-zone' not set: disabling RFC 1918 empty zones
/var/log/daemon.log:May 15 16:05:23 washoe named[3141]: Warning: 
'empty-zones-enable/disable-empty-zone' not set: disabling RFC 1918 empty zones
/var/log/daemon.log:Jun 18 06:33:55 washoe named[9888]: Warning: 
'empty-zones-enable/disable-empty-zone' not set: disabling RFC 1918 empty zones
/var/log/daemon.log:Oct  3 21:41:56 washoe named[19418]: Warning: 
'empty-zones-enable/disable-empty-zone' not set: disabling RFC 1918 empty zones

$> zgrep -i warning /var/log/daemon.log* | head
/var/log/daemon.log:Nov 17 11:11:45 novo ddclient[5539]: WARNING:  file 
/var/cache/ddclient/ddclient.cache, line 3: Invalid Value for keyword 'ip' = ''
...


> so we would end with messages like this:

> Nov 22 13:45:58 novo fail2ban[6554]: priority=info INFOinfo msg kjasdhfk 
> jasdh fkjasdh fkjasdh fkjsdh f 
> laksdjflkasjdflkajsdklfjasdlkfjaslkdjflaskdjflkasdjfklasdjfk

moreover for as simple as downcasing the loglevel in priority=info would
require a bit deeper customization of the formatter which I would prefer to
avoid at this point.

So I guess for now I will submit a PR upstream (to myself and others) for
discussion with the change I have suggested.

-- 
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834   Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730202: fail2ban should not use space in ident when logging to syslog

2013-11-22 Thread Allan Wind 
On 2013-11-22 13:46:50, Yaroslav Halchenko wrote:
> what about making it even better:
> 
> formatter = logging.Formatter("%(name)s[%(process)d]: %(levelname)-7s 
> %(message)s")

> so we report the pid as well, stay together with the name while still keeping
> indentation for the levelname conssitent for easier visual parsing.  you can
> try attached script to "play" with it.. it should look like

Yeah, it's nice touch to include the pid.  I would still suggest 
using the syslog level instead of encoding it into the message 
part.

At work we filter on priority and include that the level as a 
tagged field like this with syslog-ng:

template t_message {
template("$ISODATE $HOST ${MSGHDR}priority=$PRIORITY $MSG\n");
template_escape(no);
};

so we would end with messages like this:

Nov 22 13:45:58 novo fail2ban[6554]: priority=info INFOinfo msg kjasdhfk 
jasdh fkjasdh fkjasdh fkjsdh f 
laksdjflkasjdflkajsdklfjasdlkfjaslkdjflaskdjflkasdjfklasdjfk


/Allan
-- 
Allan Wind
Life Integrity, LLC



-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#730202: fail2ban should not use space in ident when logging to syslog

2013-11-22 Thread Yaroslav Halchenko 
Hi Allan,

got it... indeed we should stay compliant

what about making it even better:

formatter = logging.Formatter("%(name)s[%(process)d]: %(levelname)-7s 
%(message)s")

so we report the pid as well, stay together with the name while still keeping
indentation for the levelname conssitent for easier visual parsing.  you can
try attached script to "play" with it.. it should look like

$> ./test-logging.py
Last log lines
Nov 22 13:45:58 novo fail2ban[6554]: WARNING warning msg kjasdhfk jasdh fkjasdh 
fkjasdh fkjsdh ilkasjdlfkjasdlkfjlaskdjflaskdjf asldkfjaslkdfjskldfj f
Nov 22 13:45:58 novo fail2ban[6554]: INFOinfo msg kjasdhfk jasdh fkjasdh 
fkjasdh fkjsdh f laksdjflkasjdflkajsdklfjasdlkfjaslkdjflaskdjflkasdjfklasdjfk


On Fri, 22 Nov 2013, Allan Wind wrote:

> Package: fail2ban
> Version: 0.8.6-3wheezy2
> Severity: minor

> I configured fail2ban to log to syslog and would get events like these:

> 2013-11-21T04:24:01.077+00:00 pawan fail2ban.filter : INFO   Log  
>   
>
> rotation detected for /var/log/syslog-ng.log  
>   
>
> 2013-11-21T15:12:29.713+00:00 pawan fail2ban.jail   : INFO   Jail 
>   
>
> 'apache' stopped  
>   
>
> 2013-11-21T15:12:29.715+00:00 pawan fail2ban.server : INFO
>   
>
> Changed logging target to SYSLOG for Fail2ban v0.8.6  
>   
>
> 2013-11-21T15:12:29.715+00:00 pawan fail2ban.jail   : INFO
>   
>
> Creating new jail 'apache'
>   
>
> 2013-11-21T15:12:29.716+00:00 pawan fail2ban.jail   : INFO   Jail 
>   
>
> 'apache' uses poller  
>   
>
> 2013-11-21T15:12:29.724+00:00 pawan fail2ban.filter : INFO
>   
>
> Added logfile = /var/log/syslog-ng.log
>   
>
> 2013-11-21T15:12:29.725+00:00 pawan fail2ban.filter : INFO   Set  
>   
>
> maxRetry = 1  
>   
>
> 2013-11-21T15:12:29.726+00:00 pawan fail2ban.filter : INFO   Set  
>   
>
> findtime = 2592000
>   
>
> 2013-11-21T15:12:29.727+00:00 pawan fail2ban.actions: INFO   Set  
>   
>
> banTime = -1  
>   
>

> Gergely tells me in Bug#725668 that syslog(-ng) will split the ident 
> from message on space, so rather than ident being "fail2ban.filter " it 
> will be "fail2ban.filter" and the message becomes " : INFO " instead
> of the expected "INFO ...".  This cause problems for syslog-ng which has
> filtering and flexible logging capabilities.

> The level, "INFO", btw, should not be encoded in the message string, but
> leave that for syslog to record in whatever what it is configured.

> It looks like the action is here:

> server/server.py:
> def setLogTarget(self, target):
> 
> if target == "SYSLOG":
> # Syslog daemons already add date to the 
> message.
> formatter = logging.Formatter("%(name)-16s: 
> %(levelname)-6s %(message)s")
> facility = 
> logging.handlers.SysLogHandler.LOG_DAEMON
> hdlr = 
> logging.handlers.SysLogHandler("/dev/log", 
>

Bug#730202: fail2ban should not use space in ident when logging to syslog

2013-11-22 Thread Allan Wind 
Package: fail2ban
Version: 0.8.6-3wheezy2
Severity: minor

I configured fail2ban to log to syslog and would get events like these:

2013-11-21T04:24:01.077+00:00 pawan fail2ban.filter : INFO   Log
   
rotation detected for /var/log/syslog-ng.log
   
2013-11-21T15:12:29.713+00:00 pawan fail2ban.jail   : INFO   Jail   
   
'apache' stopped
   
2013-11-21T15:12:29.715+00:00 pawan fail2ban.server : INFO  
   
Changed logging target to SYSLOG for Fail2ban v0.8.6
   
2013-11-21T15:12:29.715+00:00 pawan fail2ban.jail   : INFO  
   
Creating new jail 'apache'  
   
2013-11-21T15:12:29.716+00:00 pawan fail2ban.jail   : INFO   Jail   
   
'apache' uses poller
   
2013-11-21T15:12:29.724+00:00 pawan fail2ban.filter : INFO  
   
Added logfile = /var/log/syslog-ng.log  
   
2013-11-21T15:12:29.725+00:00 pawan fail2ban.filter : INFO   Set
   
maxRetry = 1
   
2013-11-21T15:12:29.726+00:00 pawan fail2ban.filter : INFO   Set
   
findtime = 2592000  
   
2013-11-21T15:12:29.727+00:00 pawan fail2ban.actions: INFO   Set
   
banTime = -1
   

Gergely tells me in Bug#725668 that syslog(-ng) will split the ident 
from message on space, so rather than ident being "fail2ban.filter " it 
will be "fail2ban.filter" and the message becomes " : INFO " instead
of the expected "INFO ...".  This cause problems for syslog-ng which has
filtering and flexible logging capabilities.

The level, "INFO", btw, should not be encoded in the message string, but
leave that for syslog to record in whatever what it is configured.

It looks like the action is here:

server/server.py:
def setLogTarget(self, target):

if target == "SYSLOG":
# Syslog daemons already add date to the 
message.
formatter = logging.Formatter("%(name)-16s: 
%(levelname)-6s %(message)s")
facility = 
logging.handlers.SysLogHandler.LOG_DAEMON
hdlr = 
logging.handlers.SysLogHandler("/dev/log", 

  facility = facility)
where the proposed formatting line should be:

formatter = logging.Formatter("%(name)s: 
%(message)s")  

Not sure if there is anything else to do to pass the correct logging level to 
syslog.

-- System Information:
Debian Release: 7.2
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-4-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages fail2ban depends on:
ii  lsb-base4.1+Debian8+deb7u1
ii  python  2.7.3-4+deb7u1
ii  python-central  0.6.17

Versions of packages fail2ban recommends:
ii  iptables  1.4.14-3.1
pn  python-gamin  
ii  whois 5.0.23

Versions of packages fail