Sure will, so here we go.
Attached a modified qmlog script that can be run with the "-c" option
to add colored output for most log entries in Qmail Toaster. If something
is missing or doesn't match correctly it will have the FIXME tag before
the line.
When running in color mode less output will be disabled.
Thanks for pointing the script out, will use that one now since I added
the colors to it :D Maybe even recreate the output and remove the log
file look with a tabled output.
It's written on Bash 3.0.15 again, but with minor changes to the filters
it can be easily ported to higher versions of Bash too (tried it out, but
didn't finish the rewrite since I am running only 3.0.15 servers).
Try it out on your system and let me know what you think!
Cheers,
Sebastian
PS: Yay to Fridays!
Eric Shubert wrote:
Thanks. I'm sure you'll keep us posted! :)
Sebastian Grewe wrote:
After checking out the code in that script I think it might be easier
for me to just start on my script and extend it's functionality to look
for all lines in those logfiles instead of just spamdyke.
I will see what I can do.
Cheers,
Sebastian
Eric Shubert wrote:
Sorry to say that I haven't had a chance to check out your script yet,
Sebastian. :(
Speaking of colored and filtered qmail logfiles though, there's a nice
'qmlog' script at qtp.qmailtoaster.com (part of the qmailtoaster-plus
package). It allows easy viewing and searching of qmail (et al) logs.
I'm wondering if your 'coloring and filtering' might be a nice
enhancement to that script. Care to have a look into it?
Sebastian Grewe wrote:
I totally forgot about that - but I am not using the script to block
them forever, just to monitor qmail when a large amount of connections
is coming in (which happens ever so often). Even so I did turn off
the blocking feature since qmail handles it just fine and connections
clear up after a while. I was just concerned that legitimate e-mail
wouldn't
be coming through - but since they try to resend if no connection could
be established that's not a concern anymore.
So yeah, I use it to see what's being blocked and for what reason - even
added whitelist matches now.
It's basically just colored and filtered output of your qmail logfiles
now :D
Cheers,
Sebastian
Otto Berger wrote:
you could also use fail2ban for that. You just have to specify a custom
rule ("filter") for the spamdyke-log output. Then the sender ip will be
released after a specified timeframe and not blocked forever ;).
(IMHO it is still not a very good idea to block by firewall)
Otto
Sebastian Grewe schrieb:
Hey Guys,
I have been working on a simple bash script that will read from it's
standard input and presents some statistics from the logfile in realtime
(when used with "tail -f .." ).
After a few days that we have been attacked by spambots I got curious
how to avoid these things in the future. The script we use is able to
count the denied connections
per IP and, if desired, adds this IP to the Firewall to reject incoming
connections (brutal, I know). As the firewalling is optional you might
still be interested in it to run just
to see what's going on.
It's written for BASH 3.0.15 but with a little change in the pattern
matcher it runs on higher versions too. To start it in live mode run it
like this:
tail -f /var/log/qmail/smtp/current | qmail_parser.sh
and if you just want to scan some files and see what happened to this:
cat /var/log/qmail/smtp/* | qmail_parser.sh
Since it's BASH it's not very good when it comes to performance but does
the trick well when used with "tail". Also it's not catching everything
(yet) since I was looking for only
some very specific lines in the logfile. Anyhow, try it out and tell me
what you think - attached the current script to this mail.
Cheers,
Sebastian
------------------------------------------------------------------------
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
#!/bin/bash
#
# Copyright (C) 2006-2008 Eric Shubert <e...@shubes.net>
#
# Utility for listing/searching qmail log files
# Original script by Fabio Olaechea
#
# Future Enhancements
# .) find .sed file w/out hard coded path
#
#####################################################################
# Change Log
# 02/27/09 sebastian - added colored output function
# 04/05/08 shubes - changed `` to $()
# 10/17/07 shubes - fixed -t option
# 12/17/06 shubes - added sed, grep, date/time parameters
# 11/24/06 shubes - restructured, added numerous capabilities
# 11/21/06 shubes - added -f option, thanks to phi...@ows.ch
#####################################################################
# Just some color settings
RED="\\033[0;31m"
GREEN="\\033[0;32m"
YELLOW="\\033[1;33m"
BLUE="\\033[0;34m"
VIOLET="\\033[0;35m"
CYAN="\\033[0;36m"
WHITE="\\033[1;37m"
NORMAL="\033[0m"
#####################################################################
## edit service to be sure it exists
#
a1_edit_parameters(){
if [ -z "$service" ] ; then
q1_usage
elif [ ! -d $logdir/$service ]; then
echo "Invalid service: $service"
q2_services
exit 1
fi
}
#####################################################################
## show log file names
#
a2_list_log_files(){
for file in $(find $logdir/$service -name "*.s"); do
filename=$(basename $file)
tempname=$(echo $filename | tai64nlocal | sed 's/ /_/')
datename=${tempname%\.[0-9]*\.s}
echo "$filename - $datename"
done
}
#####################################################################
## show logs content
#
a3_show_logs_content(){
if [ ! -z "$sedtrim" ]; then
b32_get_sed_script
fi
if [ ! -z "$greppattern" ]; then
grepcmd="| grep $greppattern"
fi
if [ ! -z "$namepattern" ] \
|| [ ! -z "$contpattern" ] \
|| [ ! -z "$dateparm" ]; then
b34_select_files
else
showfiles="$logdir/$service/current"
fi
if [ -z "$command" ]; then
command="cat"
else
command="tail $command"
fi
if [ ! -z "$lesscmd" ] && [ ! -z "$position_less" ]; then
lesscmd="$lesscmd $position_less"
fi
# show the log
#echo "$me: eval $command $showfiles $tai64n $sedtrim $grepcmd $lesscmd"
eval $command $showfiles $tai64n $sedtrim $grepcmd $lesscmd $colorize
}
#####################################################################
## lets add some colors to stdin which is passed to this function
#
colorize() {
# Lets make sure less is disabled
if [[ -z $lesscmd ]]; then
while read LINE; do
# spamdyke colors
if [[ $LINE =~ spamdyke ]]; then
if [[ $LINE =~ "(.*)\ (.*)\ (.*)\
(TIMEOUT|DENIED_OTHER|ALLOWED|DENIED_RBL_MATCH|DENIED_RDNS_MISSING|DENIED_RDNS_RESOLVE|DENIED_GRAYLISTED|DENIED_SENDER_NO_MX)\
from:\ (.*)\ to:\ (.*)\ origin_ip:\ (.*)\ origin_rdns:\ (.*)\ auth:\ (.*)(\
.*)?" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}${BASH_REMATCH[4]}${NORMAL} from: ${VIOLET}${BASH_REMATCH[5]}${NORMAL}
to: ${VIOLET}${BASH_REMATCH[6]}${NORMAL} origin_ip:
${RED}${BASH_REMATCH[7]}${NORMAL} origin_rdns:
${RED}${BASH_REMATCH[8]}${NORMAL} auth: ${WHITE}${BASH_REMATCH[9]}${NORMAL}
${NORMAL}${BASH_REMATCH[10]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_RBL_MATCH\ ip:\ (.*)\ rbl:\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_RBL_MATCH${NORMAL} ip: ${RED}${BASH_REMATCH[4]}${NORMAL} rbl:
${WHITE}${BASH_REMATCH[5]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_RDNS_MISSING\ ip:\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_RDNS_MISSING${NORMAL} ip: ${RED}${BASH_REMATCH[4]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_EARLYTALKER\ delay:\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_EARLYTALKER${NORMAL} delay: ${WHITE}${BASH_REMATCH[4]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_RECIPIENT_WHITELIST\ recipient:\ (.*)\ file:\ (.*)\((.*)\)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_RECIPIENT_WHITELIST${NORMAL} recipient:
${VIOLET}${BASH_REMATCH[4]}${NORMAL} file:
${WHITE}${BASH_REMATCH[5]}${NORMAL}(${YELLOW}${BASH_REMATCH[6]}${NORMAL})"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_WHITELIST_NAME\ ip:\ (.*)\ rdns:\ (.*)\ file:\ (.*)\((.*)\)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_WHITELIST_NAME${NORMAL} ip: ${VIOLET}${BASH_REMATCH[4]}${NORMAL}
rdns: ${RED}${BASH_REMATCH[5]}${NORMAL} file:
${WHITE}${BASH_REMATCH[6]}${NORMAL}(${YELLOW}${BASH_REMATCH[7]}${NORMAL})"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_WHITELIST_IP\ ip:\ (.*)\ file:\ (.*)\((.*)\)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_WHITELIST_IP${NORMAL} ip: ${VIOLET}${BASH_REMATCH[4]}${NORMAL}
file:
${WHITE}${BASH_REMATCH[5]}${NORMAL}(${YELLOW}${BASH_REMATCH[6]}${NORMAL})"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_SENDER_NO_MX\ domain:\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_SENDER_NO_MX${NORMAL} domain:
${VIOLET}${BASH_REMATCH[4]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_GRAYLISTED\ sender:\ (.*)\ recipient:\ (.*)\ path:\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_GRAYLISTED${NORMAL} sender: ${VIOLET}${BASH_REMATCH[4]}${NORMAL}
recipient: ${VIOLET}${BASH_REMATCH[5]}${NORMAL} path:
${WHITE}${BASH_REMATCH[6]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\
FILTER_OTHER:\ response:\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}FILTER_OTHER:${NORMAL} response: ${WHITE}${BASH_REMATCH[4]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ ERROR" ]];
then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${RED}ERROR${NORMAL} sender: ${VIOLET}${BASH_REMATCH[4]}${NORMAL} recipient:
${VIOLET}${BASH_REMATCH[5]}${NORMAL} path: ${WHITE}${BASH_REMATCH[6]}${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# tcpserver colors
elif [[ $LINE =~ "tcpserver" ]]; then
echo $LINE
if [[ $LINE =~ "(.*)\ (.*)\ (.*)\ status:\
(.*)/(.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}status:${NORMAL}
${YELLOW}${BASH_REMATCH[4]}${NORMAL}/${RED}${BASH_REMATCH[5]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ pid\ (.*)\
from\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}pid${NORMAL} ${YELLOW}${BASH_REMATCH[4]}${NORMAL} from
${RED}${BASH_REMATCH[5]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ ok\ (.*)\
(.*):(.*):(.*)\ (.*)?:(.*)?:(.*)?:(.*)?" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}ok${NORMAL} ${YELLOW}${BASH_REMATCH[4]}${NORMAL}
${RED}${BASH_REMATCH[5]}${NORMAL}:${RED}${BASH_REMATCH[6]}${NORMAL}:${RED}${BASH_REMATCH[7]}${NORMAL}
${RED}${BASH_REMATCH[8]}${NORMAL}:${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL}:${RED}${BASH_REMATCH[11]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ end\ (.*)\
status\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}end${NORMAL} ${YELLOW}${BASH_REMATCH[4]}${NORMAL} status
${RED}${BASH_REMATCH[5]}${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# simscan colors
elif [[ $LINE =~ "simscan:" ]]; then
if [[ $LINE =~ "(.*)\ (.*)\ (.*):(SPAM\
REJECT|CLEAN)\ (.*):(.*):(.*):(.*):(.*):(.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL}
${YELLOW}${BASH_REMATCH[3]}${NORMAL}:${WHITE}${BASH_REMATCH[4]}${NORMAL}
${WHITE}${BASH_REMATCH[5]}${NORMAL}:${NORMAL}${BASH_REMATCH[6]}${NORMAL}:${RED}${BASH_REMATCH[7]}${NORMAL}:${RED}${BASH_REMATCH[8]}${NORMAL}:${VIOLET}${BASH_REMATCH[9]}${NORMAL}:${VIOLET}${BASH_REMATCH[10]}${NORMAL}"
elif [[ $LINE =~ "(.*)\ (.*)\
(.*):RELAYCLIENT:(.*):(.*):(.*):(.*):(.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL}
${YELLOW}${BASH_REMATCH[3]}${NORMAL}:${GREEN}RELAYCLIENT${NORMAL}:${WHITE}${BASH_REMATCH[4]}${NORMAL}:${BASH_REMATCH[5]}${NORMAL}:${RED}${BASH_REMATCH[6]}${NORMAL}:${VIOLET}${BASH_REMATCH[7]}${NORMAL}:${VIOLET}${BASH_REMATCH[8]}${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# CHKUSE colors
elif [[ $LINE =~ "CHKUSER" ]]; then
if [[ $LINE =~ "(.*)\ (.*)\ (.*)\ (relaying\
rcpt:|accepted\ sender:|accepted rcpt:|rejected rcpt:)\ from\
<(.*)?:(.*)?:(.*)?> remote\ <(.*):(.*):(.*)> rcpt\ <(.*)> :\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL}
${WHITE}${BASH_REMATCH[4]}${NORMAL} from
<${VIOLET}${BASH_REMATCH[5]}${NORMAL}:${RED}${BASH_REMATCH[6]}${NORMAL}:${YELLOW}${BASH_REMATCH[7]}${NORMAL}>
remote
<${RED}${BASH_REMATCH[8]}${NORMAL}:${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL}>
rcpt <${VIOLET}${BASH_REMATCH[11]}${NORMAL}> : ${BASH_REMATCH[12]}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# qmail-smtpd lines
elif [[ $LINE =~ "qmail-smtpd" ]]; then
if [[ $LINE =~ "(.*)\ (.*)\ (.*) qq hard reject
\(Your email is considered spam \((.*) spam-hits\)\):\ MAILFROM:(.*)\
RCPTTO:(.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} (Your email is
considered spam (${WHITE}${BASH_REMATCH[4]}${NORMAL} spam hits))
MAILFROM:${VIOLET}${BASH_REMATCH[5]}${NORMAL}
RCPTTO:${VIOLET}${BASH_REMATCH[6]}${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# spamd
elif [[ $LINE =~ "(spamd:|prefork: child)" ]]; then
if [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ clean\ message\ \((.*)/(.*)\)\
for\ (.*):(.*)\ in\ (.*)\ seconds,\ (.*)\ bytes" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} clean message
(${GREEN}${BASH_REMATCH[7]}${NORMAL}/${RED}${BASH_REMATCH[8]}${NORMAL}) for
${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL} in
${WHITE}${BASH_REMATCH[11]}${NORMAL} seconds,
${RED}${BASH_REMATCH[12]}${NORMAL} bytes."
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ identified\ spam\
\((.*)/(.*)\)\ for\ (.*):(.*)\ in\ (.*)\ seconds,\ (.*)\ bytes" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} identified
spam (${GREEN}${BASH_REMATCH[7]}${NORMAL}/${RED}${BASH_REMATCH[8]}${NORMAL})
for ${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL} in
${WHITE}${BASH_REMATCH[11]}${NORMAL} seconds,
${RED}${BASH_REMATCH[12]}${NORMAL} bytes."
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ processing\ message\ (.*) for
(.*):(.*)" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} processing
message ${VIOLET}${BASH_REMATCH[7]}${NORMAL} for
${RED}${BASH_REMATCH[8]}${NORMAL}:${RED}${BASH_REMATCH[9]}${NORMAL}"
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ connection\ from\ (.*) at port
(.*)" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} connection
from ${RED}${BASH_REMATCH[7]}${NORMAL} at port
${RED}${BASH_REMATCH[8]}${NORMAL}"
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ result:\ (.)\ (.)?(.)\ (.)
(.*)
scantime=(.*),size=(.*),user=(.*),uid=(.*),required_score=(.*),rhost=(.*),raddr=(.*),rport=(.*),mid=(.*),bayes=(.*),autolearn=(.*)"
]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} result:
${WHITE}${BASH_REMATCH[7]}${NORMAL}
${WHITE}${BASH_REMATCH[8]}${NORMAL}${WHITE}${BASH_REMATCH[9]}${NORMAL}
${WHITE}${BASH_REMATCH[10]}${NORMAL} ${WHITE}${BASH_REMATCH[11]}${NORMAL}
scantime=${WHITE}${BASH_REMATCH[12]}${NORMAL},size=${WHITE}${BASH_REMATCH[13]}${NORMAL},user=${RED}${BASH_REMATCH[14]}${NORMAL},uid=${RED}${BASH_REMATCH[15]}${NORMAL},required_score=${WHITE}${BASH_REMATCH[16]}${NORMAL},rhost=${RED}${BASH_REMATCH[17]}${NORMAL},raddr=${RED}${BASH_REMATCH[18]}${NORMAL},rport=${WHITE}${BASH_REMATCH[19]}${NORMAL},mid=${WHITE}${BASH_REMATCH[20]}${NORMAL},bayes=${WHITE}${BASH_REMATCH[21]}${NORMAL},autolearn=${WHITE}${BASH_REMATCH[22]}${NORMAL}"
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ server\ successfully\ spawned\
child\ process,\ pid\ (.*)" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} server
successfully spawned child process, pid ${WHITE}${BASH_REMATCH[7]}${NORMAL}"
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ handled\ cleanup\ of\ child\
pid\ (.*)\ due\ to\ (.*)" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} handled
cleanup of child pid ${WHITE}${BASH_REMATCH[7]}${NORMAL} due to
${WHITE}${BASH_REMATCH[8]}${NORMAL}"
elif [[ $LINE =~ "(.*)-(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ prefork: child states: (.*)" ]]; then
echo -e
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]}
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL}
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}prefork:${NORMAL} child
states: ${WHITE}${BASH_REMATCH[7]}${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# imap
elif [[ $LINE =~ "(DEBUG: Connection,|INFO:
LOGIN,|INFO: DISCONNECTED,)" ]]; then
if [[ $LINE =~ "(.*)\
([0-9]*):([0-9]*):([0-9]*)\ DEBUG:\ Connection,\ ip=\[(.*)\]" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL}
${WHITE}DEBUG:${NORMAL} Connection, ip=[${RED}${BASH_REMATCH[5]}${NORMAL}]"
elif [[ $LINE =~ "(.*)\
([0-9]*):([0-9]*):([0-9]*)\ INFO:\ LOGIN,\ user=(.*),\ ip=\[(.*)\],\
protocol=(.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL}
${WHITE}INFO:${NORMAL} LOGIN, user=${VIOLET}${BASH_REMATCH[5]}${NORMAL},
ip=[${RED}${BASH_REMATCH[6]}${NORMAL}],
protocol=${WHITE}${BASH_REMATCH[7]}${NORMAL} "
elif [[ $LINE =~ "(.*)\
([0-9]*):([0-9]*):([0-9]*)\ INFO:\ DISCONNECTED,\ user=(.*),\ ip=\[(.*)\],\
headers=(.*),\ body=(.*),\ rcvd=(.*),\ sent=(.*),\ time=(.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL}
${WHITE}INFO:${NORMAL} DISCONNECTED, user=${VIOLET}${BASH_REMATCH[5]}${NORMAL},
ip=[${RED}${BASH_REMATCH[6]}${NORMAL}],
headers=${WHITE}${BASH_REMATCH[7]}${NORMAL},
body=${WHITE}${BASH_REMATCH[8]}${NORMAL},
rcvd=${WHITE}${BASH_REMATCH[9]}${NORMAL},
sent=${WHITE}${BASH_REMATCH[8]}${NORMAL},
time=${WHITE}${BASH_REMATCH[9]}${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# clamd
elif [[ $LINE =~ "/var/qmail/simscan/" ]]; then
if [[ $LINE =~ "(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*): OK$" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL}
${WHITE}${BASH_REMATCH[5]}${NORMAL}: ${GREEN}OK${NORMAL}"
elif [[ $LINE =~ "(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*): (.*) FOUND$" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL}
${WHITE}${BASH_REMATCH[5]}${NORMAL}: ${YELLOW}${BASH_REMATCH[6]}${NORMAL}
${RED}FOUND${NORMAL}"
else
echo -e "${RED}FIXME${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
# No hit with any of the above
else
if [[ $LINE =~ "(.*)\
([0-9]*):([0-9]*):([0-9]*)\ (.*)" ]]; then
echo -e "${BLUE}${BASH_REMATCH[1]}
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL}
${WHITE}${BASH_REMATCH[5]}${NORMAL}"
else
# Not a regular formatted logline
echo -e "${RED}NOHIT${NORMAL}
${YELLOW}${LINE}${NORMAL}"
fi
fi
done
else
# Just in case less has not been disabled remove all the color
codes, print the bare lines
while read LINE; do
echo $LINE
done
fi
}
#####################################################################
## get sed commands for trimming the output
#
b32_get_sed_script(){
#sedfile=${mydir%bin}etc/qmlog-trim.sed
sedfile=/opt/qmailtoaster-plus/etc/qmlog-trim.sed
if [ -f $sedfile ]; then
while read sedline; do
sedtrim="$sedtrim $sedline;"
done < $sedfile
else
echo "$me $myver notice - sed file $sedfile not found"
fi
sedtrim="$sedtrim $sedparm'"
}
#####################################################################
## select files to show based on name, dates, and/or content pattern
#
b34_select_files(){
if [ -z "$namepattern" ]; then
filepattern="*.s"
else
filepattern="*$namepattern*.s"
fi
if [ ! -z "$dateparm" ]; then
c342_select_date_time_range
elif [ ! -z "$namepattern" ]; then
c344_select_filename_pattern
else
somefiles="$logdir/$service/$filepattern $logdir/$service/current"
fi
if [ ! -z "$contpattern" ]; then
c346_select_content
else
showfiles="$somefiles"
fi
}
#####################################################################
## select files by date, time range
#
c342_select_date_time_range(){
d3422_setup_date_time_fields
for logfile in $(ls $logdir/$service/$filepattern 2>/dev/null) \
$logdir/$service/current; do
d3425_select_each_file
done
if [ -z "$somefiles" ]; then
echo "$me - no $service log found with date/time matching $dateparm"
exit 1
fi
position_less="-p${from_mon}-${from_day}"
}
#####################################################################
## setup date, time fields for file selection
#
d3422_setup_date_time_fields(){
from_datetime=$(echo $dateparm | cut --delimiter=- --fields=1)
thru_datetime=$(echo $dateparm | cut -s --delimiter=- --fields=2)
from_mmdd=$(echo $from_datetime | cut --delimiter=: --fields=1)
from_hhmm=$(echo $from_datetime | cut -s --delimiter=: --fields=2)
thru_mmdd=$(echo $thru_datetime | cut --delimiter=: --fields=1)
thru_hhmm=$(echo $thru_datetime | cut -s --delimiter=: --fields=2)
: ${thru_mmdd:=$from_mmdd}
: ${from_hhmm:=0000}
: ${thru_hhmm:=2359}
from_mon=$(echo $from_mmdd | cut -b 1-2)
from_day=$(echo $from_mmdd | cut -b 3-4)
thru_mon=$(echo $thru_mmdd | cut -b 1-2)
thru_day=$(echo $thru_mmdd | cut -b 3-4)
curryear=$(date +%Y)
currmmdd=$(date +%m%d)
if [ "$from_mmdd" -gt "$currmmdd" ]; then
from_year=$(($curryear - 1))
else
from_year=$curryear
fi
if [ "$thru_mmdd" -gt "$currmmdd" ]; then
thru_year=$(($curryear - 1))
else
thru_year=$curryear
fi
from_date_time="$from_year$from_mon$from_day$from_hhmm"
thru_date_time="$thru_year$thru_mon$thru_day$thru_hhmm"
}
#####################################################################
## select each file by date/time
#
d3425_select_each_file(){
read logdate logtime restoflog <<!
$(head -n1 $logfile | tai64nlocal)
!
begdate=$(echo $logdate | tr -d -)
begtime=$(echo ${logtime%:[0-9][0-9]\.[0-9]*} | tr -d :)
beg_date_time=$begdate$begtime
read logdate logtime restoflog <<!
$(tail -n1 $logfile | tai64nlocal)
!
enddate=$(echo $logdate | tr -d -)
endtime=$(echo ${logtime%:[0-9][0-9]\.[0-9]*} | tr -d :)
end_date_time=$enddate$endtime
if [ "$end_date_time" -lt "$from_date_time" ] \
|| [ "$beg_date_time" -gt "$thru_date_time" ]; then
continue
else
somefiles="$somefiles $logfile"
fi
}
#####################################################################
## select files by filename pattern
#
c344_select_filename_pattern(){
somefiles=$(ls $logdir/$service/$filepattern 2>1)
if [ -z "$somefiles" ]; then
echo "$me - no $service log found with filename pattern $namepattern"
exit 1
fi
}
#####################################################################
## select files by content
#
c346_select_content(){
showfiles=$(grep -lce "$contpattern" $somefiles 2>/dev/null)
if [ -z "$showfiles" ]; then
if [ -z "$namepattern" ] && [ -z "$dateparm" ]; then
withmsg=""
elif [ -z "$namepattern" ]; then
withmsg=" with date/time matching $dateparm"
elif [ -z "$dateparm" ]; then
withmsg=" with filename pattern $namepattern"
else
withmsg=" with date/time matching $dateparm and filename pattern
$namepattern"
fi
echo "$me - pattern $contpattern not found in any $service logs $withmsg"
exit 1
fi
if [ -z "$position_less" ]; then
position_less="-p$contpattern"
fi
}
#####################################################################
## show usage output
#
q1_usage(){
echo "$me $myver - show current log of service 'service'"
echo "usage: qmlog service [option] ..."
q2_services
echo "options:"
echo " -h[elp] this help"
echo " -l[ist] list saved logs of service"
echo " -f follow as it grows, using 'tail -f'"
echo " -c add colors, disables less"
echo " -t N show (tail) last N lines"
echo " -nl show without using less"
echo " -nt show with no trimming"
echo " -d mmdd[:hhmm][-mmdd[:hhmm]]"
echo " show logs that contain the date 'mmdd' [thru -'mmdd']"
echo " -lc regex show logs that contain a string that matches 'regex'"
echo " -ln pattern show logs with file name containing 'pattern'"
echo " -s command pipe output through sed 'command'"
echo " -g regex show only lines that match the string 'regex'"
exit 1
}
#####################################################################
## show services output
#
q2_services()
{
echo -n "services:"
for opt in `ls $logdir`; do
if [ -d "$logdir/$opt" ]; then
echo -n " $opt"
fi
done
echo
}
#####################################################################
## begin main processing here
#
me=$(basename $0)
myver=v0.4
mydir=$(dirname $0)
: ${logdir:=/var/log/qmail}
option=""
command=""
tai64n="| tai64nlocal"
lesscmd="| less"
grepcmd=""
sedtrim="| sed '"
sedparm=""
dateparm=""
namepattern=""
contpattern=""
service=""
showfiles=""
position_less=""
while (( "$#" )); do
case $1 in
"-h" | "-help" | "--help" )
q1_usage
;;
"-l" | "-list" | "--list" )
option="list"
shift
;;
"-f" )
command="$command $1"
lesscmd=""
shift
;;
"-t" )
command="$command -n $2"
shift 2
;;
"-c" )
lesscmd=""
colorize="| colorize"
shift
;;
"-nl" | "-noless" | "--noless" )
lesscmd=""
shift
;;
"-nt" | "-notrim" | "--notrim" )
sedtrim=""
shift
;;
"-d" | "-date" | "-dates" | "--date" | "--dates" )
dateparm=$2
shift 2
;;
"-lc" | "-logcont" | "--logcont" )
contpattern=$2
shift 2
;;
"-ln" | "-logname" | "--logname" )
namepattern=$2
shift 2
;;
"-s" | "-sed" | "--sed" )
sedparm="$sedparm $2;"
shift 2
;;
"-g" | "-grep" | "--grep" )
greppattern=$2
shift 2
;;
* )
service=$1
shift
;;
esac
done
a1_edit_parameters
case $option in
"list" )
a2_list_log_files
;;
* )
a3_show_logs_content
;;
esac
exit 0
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
