Sure will, so here we go.

Attached a modified qmlog script that can be run with the "-c" option
to add colored output for most log entries in Qmail Toaster. If something
is missing or doesn't match correctly it will have the FIXME tag before
the line.

When running in color mode less output will be disabled.

Thanks for pointing the script out, will use that one now since I added
the colors to it :D Maybe even recreate the output and remove the log
file look with a tabled output.

It's written on Bash 3.0.15 again, but with minor changes to the filters
it can be easily ported to higher versions of Bash too (tried it out, but
didn't finish the rewrite since I am running only 3.0.15 servers).

Try it out on your system and let me know what you think!

Cheers,
Sebastian

PS: Yay to Fridays!

Eric Shubert wrote:
Thanks. I'm sure you'll keep us posted! :)

Sebastian Grewe wrote:
After checking out the code in that script I think it might be easier
for me to just start on my script and extend it's functionality to look
for all lines in those logfiles instead of just spamdyke.

I will see what I can do.

Cheers,
Sebastian

Eric Shubert wrote:
Sorry to say that I haven't had a chance to check out your script yet, Sebastian. :(

Speaking of colored and filtered qmail logfiles though, there's a nice 'qmlog' script at qtp.qmailtoaster.com (part of the qmailtoaster-plus package). It allows easy viewing and searching of qmail (et al) logs. I'm wondering if your 'coloring and filtering' might be a nice enhancement to that script. Care to have a look into it?

Sebastian Grewe wrote:
I totally forgot about that - but I am not using the script to block
them forever, just to monitor qmail when a large amount of connections
is coming in (which happens ever so often). Even so I did turn off
the blocking feature since qmail handles it just fine and connections
clear up after a while. I was just concerned that legitimate e-mail wouldn't
be coming through - but since they try to resend if no connection could
be established that's not a concern anymore.

So yeah, I use it to see what's being blocked and for what reason - even
added whitelist matches now.

It's basically just colored and filtered output of your qmail logfiles now :D

Cheers,
Sebastian

Otto Berger wrote:
you could also use fail2ban for that. You just have to specify a custom
rule ("filter") for the spamdyke-log output. Then the sender ip will be
released after a specified timeframe and not blocked forever ;).

(IMHO it is still not a very good idea to block by firewall)

Otto

Sebastian Grewe schrieb:
Hey Guys,

I have been working on a simple bash script that will read from it's
standard input and presents some statistics from the logfile in realtime
(when used with "tail -f .." ).
After a few days that we have been attacked by spambots I got curious
how to avoid these things in the future. The script we use is able to
count the denied connections
per IP and, if desired, adds this IP to the Firewall to reject incoming
connections (brutal, I know). As the firewalling is optional you might
still be interested in it to run just
to see what's going on.

It's written for BASH 3.0.15 but with a little change in the pattern
matcher it runs on higher versions too. To start it in live mode run it
like this:

 tail -f /var/log/qmail/smtp/current | qmail_parser.sh

and if you just want to scan some files and see what happened to this:

 cat /var/log/qmail/smtp/* | qmail_parser.sh

Since it's BASH it's not very good when it comes to performance but does
the trick well when used with "tail". Also it's not catching everything
(yet) since I was looking for only
some very specific lines in the logfile. Anyhow, try it out and tell me
what you think - attached the current script to this mail.

Cheers,
Sebastian


------------------------------------------------------------------------

_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users



#!/bin/bash
#
# Copyright (C) 2006-2008 Eric Shubert <e...@shubes.net>
#
# Utility for listing/searching qmail log files
# Original script by Fabio Olaechea
#
#   Future Enhancements
#     .) find .sed file w/out hard coded path
#
#####################################################################
# Change Log
# 02/27/09 sebastian - added colored output function
# 04/05/08 shubes - changed `` to $()
# 10/17/07 shubes - fixed -t option
# 12/17/06 shubes - added sed, grep, date/time parameters
# 11/24/06 shubes - restructured, added numerous capabilities
# 11/21/06 shubes - added -f option, thanks to phi...@ows.ch
#####################################################################

# Just some color settings
RED="\\033[0;31m"
GREEN="\\033[0;32m"
YELLOW="\\033[1;33m"
BLUE="\\033[0;34m"
VIOLET="\\033[0;35m"
CYAN="\\033[0;36m"
WHITE="\\033[1;37m"
NORMAL="\033[0m"


#####################################################################
## edit service to be sure it exists
#
a1_edit_parameters(){

if [ -z "$service" ] ; then
  q1_usage
elif [ ! -d $logdir/$service ]; then
  echo "Invalid service: $service"
  q2_services
  exit 1
fi
}

#####################################################################
## show log file names
#
a2_list_log_files(){

for file in $(find $logdir/$service -name "*.s"); do
  filename=$(basename $file)
  tempname=$(echo $filename | tai64nlocal | sed 's/ /_/')
  datename=${tempname%\.[0-9]*\.s}
  echo "$filename - $datename"
done
}

#####################################################################
## show logs content
#
a3_show_logs_content(){

if [ ! -z "$sedtrim" ]; then
  b32_get_sed_script
fi

if [ ! -z "$greppattern" ]; then
  grepcmd="| grep $greppattern"
fi

if [ ! -z "$namepattern" ] \
      || [ ! -z "$contpattern" ] \
      || [ ! -z "$dateparm" ]; then
  b34_select_files
else
  showfiles="$logdir/$service/current"
fi

if [ -z "$command" ]; then
  command="cat"
else
  command="tail $command"
fi

if [ ! -z "$lesscmd" ] && [ ! -z "$position_less" ]; then
  lesscmd="$lesscmd $position_less"
fi

# show the log
#echo "$me: eval $command $showfiles $tai64n $sedtrim $grepcmd $lesscmd"
eval $command $showfiles $tai64n $sedtrim $grepcmd $lesscmd $colorize
}

#####################################################################
## lets add some colors to stdin which is passed to this function
#
colorize() {
        # Lets make sure less is disabled
        if [[ -z $lesscmd ]]; then
                while read LINE; do
                        # spamdyke colors
                        if [[ $LINE =~ spamdyke ]]; then
                                if [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
(TIMEOUT|DENIED_OTHER|ALLOWED|DENIED_RBL_MATCH|DENIED_RDNS_MISSING|DENIED_RDNS_RESOLVE|DENIED_GRAYLISTED|DENIED_SENDER_NO_MX)\
 from:\ (.*)\ to:\ (.*)\ origin_ip:\ (.*)\ origin_rdns:\ (.*)\ auth:\ (.*)(\ 
.*)?" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}${BASH_REMATCH[4]}${NORMAL} from: ${VIOLET}${BASH_REMATCH[5]}${NORMAL} 
to: ${VIOLET}${BASH_REMATCH[6]}${NORMAL} origin_ip: 
${RED}${BASH_REMATCH[7]}${NORMAL} origin_rdns: 
${RED}${BASH_REMATCH[8]}${NORMAL} auth: ${WHITE}${BASH_REMATCH[9]}${NORMAL} 
${NORMAL}${BASH_REMATCH[10]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_RBL_MATCH\ ip:\ (.*)\ rbl:\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_RBL_MATCH${NORMAL} ip: ${RED}${BASH_REMATCH[4]}${NORMAL} rbl: 
${WHITE}${BASH_REMATCH[5]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_RDNS_MISSING\ ip:\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_RDNS_MISSING${NORMAL} ip: ${RED}${BASH_REMATCH[4]}${NORMAL}" 
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_EARLYTALKER\ delay:\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_EARLYTALKER${NORMAL} delay: ${WHITE}${BASH_REMATCH[4]}${NORMAL}" 
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_RECIPIENT_WHITELIST\ recipient:\ (.*)\ file:\ (.*)\((.*)\)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_RECIPIENT_WHITELIST${NORMAL} recipient: 
${VIOLET}${BASH_REMATCH[4]}${NORMAL} file: 
${WHITE}${BASH_REMATCH[5]}${NORMAL}(${YELLOW}${BASH_REMATCH[6]}${NORMAL})" 
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_WHITELIST_NAME\ ip:\ (.*)\ rdns:\ (.*)\ file:\ (.*)\((.*)\)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_WHITELIST_NAME${NORMAL} ip: ${VIOLET}${BASH_REMATCH[4]}${NORMAL} 
rdns: ${RED}${BASH_REMATCH[5]}${NORMAL} file: 
${WHITE}${BASH_REMATCH[6]}${NORMAL}(${YELLOW}${BASH_REMATCH[7]}${NORMAL})" 
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_WHITELIST_IP\ ip:\ (.*)\ file:\ (.*)\((.*)\)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_WHITELIST_IP${NORMAL} ip: ${VIOLET}${BASH_REMATCH[4]}${NORMAL} 
file: 
${WHITE}${BASH_REMATCH[5]}${NORMAL}(${YELLOW}${BASH_REMATCH[6]}${NORMAL})" 
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_SENDER_NO_MX\ domain:\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_SENDER_NO_MX${NORMAL} domain: 
${VIOLET}${BASH_REMATCH[4]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_GRAYLISTED\ sender:\ (.*)\ recipient:\ (.*)\ path:\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_GRAYLISTED${NORMAL} sender: ${VIOLET}${BASH_REMATCH[4]}${NORMAL} 
recipient: ${VIOLET}${BASH_REMATCH[5]}${NORMAL} path: 
${WHITE}${BASH_REMATCH[6]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ 
FILTER_OTHER:\ response:\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}FILTER_OTHER:${NORMAL} response: ${WHITE}${BASH_REMATCH[4]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ ERROR" ]]; 
then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${RED}ERROR${NORMAL} sender: ${VIOLET}${BASH_REMATCH[4]}${NORMAL} recipient: 
${VIOLET}${BASH_REMATCH[5]}${NORMAL} path: ${WHITE}${BASH_REMATCH[6]}${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # tcpserver colors
                        elif [[ $LINE =~ "tcpserver" ]]; then
                                echo $LINE
                                if [[ $LINE =~ "(.*)\ (.*)\ (.*)\ status:\ 
(.*)/(.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}status:${NORMAL} 
${YELLOW}${BASH_REMATCH[4]}${NORMAL}/${RED}${BASH_REMATCH[5]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ pid\ (.*)\ 
from\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}pid${NORMAL} ${YELLOW}${BASH_REMATCH[4]}${NORMAL} from 
${RED}${BASH_REMATCH[5]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ ok\ (.*)\ 
(.*):(.*):(.*)\ (.*)?:(.*)?:(.*)?:(.*)?" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}ok${NORMAL} ${YELLOW}${BASH_REMATCH[4]}${NORMAL} 
${RED}${BASH_REMATCH[5]}${NORMAL}:${RED}${BASH_REMATCH[6]}${NORMAL}:${RED}${BASH_REMATCH[7]}${NORMAL}
 
${RED}${BASH_REMATCH[8]}${NORMAL}:${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL}:${RED}${BASH_REMATCH[11]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ (.*)\ end\ (.*)\ 
status\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}end${NORMAL} ${YELLOW}${BASH_REMATCH[4]}${NORMAL} status 
${RED}${BASH_REMATCH[5]}${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # simscan colors
                        elif [[ $LINE =~ "simscan:" ]]; then
                                if [[ $LINE =~ "(.*)\ (.*)\ (.*):(SPAM\ 
REJECT|CLEAN)\ (.*):(.*):(.*):(.*):(.*):(.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} 
${YELLOW}${BASH_REMATCH[3]}${NORMAL}:${WHITE}${BASH_REMATCH[4]}${NORMAL} 
${WHITE}${BASH_REMATCH[5]}${NORMAL}:${NORMAL}${BASH_REMATCH[6]}${NORMAL}:${RED}${BASH_REMATCH[7]}${NORMAL}:${RED}${BASH_REMATCH[8]}${NORMAL}:${VIOLET}${BASH_REMATCH[9]}${NORMAL}:${VIOLET}${BASH_REMATCH[10]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ (.*)\ 
(.*):RELAYCLIENT:(.*):(.*):(.*):(.*):(.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} 
${YELLOW}${BASH_REMATCH[3]}${NORMAL}:${GREEN}RELAYCLIENT${NORMAL}:${WHITE}${BASH_REMATCH[4]}${NORMAL}:${BASH_REMATCH[5]}${NORMAL}:${RED}${BASH_REMATCH[6]}${NORMAL}:${VIOLET}${BASH_REMATCH[7]}${NORMAL}:${VIOLET}${BASH_REMATCH[8]}${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # CHKUSE colors
                        elif [[ $LINE =~ "CHKUSER" ]]; then
                                if [[ $LINE =~ "(.*)\ (.*)\ (.*)\ (relaying\ 
rcpt:|accepted\ sender:|accepted rcpt:|rejected rcpt:)\ from\ 
<(.*)?:(.*)?:(.*)?> remote\ <(.*):(.*):(.*)> rcpt\ <(.*)> :\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} 
${WHITE}${BASH_REMATCH[4]}${NORMAL} from 
<${VIOLET}${BASH_REMATCH[5]}${NORMAL}:${RED}${BASH_REMATCH[6]}${NORMAL}:${YELLOW}${BASH_REMATCH[7]}${NORMAL}>
 remote 
<${RED}${BASH_REMATCH[8]}${NORMAL}:${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL}>
 rcpt <${VIOLET}${BASH_REMATCH[11]}${NORMAL}> : ${BASH_REMATCH[12]}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # qmail-smtpd lines
                        elif [[ $LINE =~ "qmail-smtpd" ]]; then
                                if [[ $LINE =~ "(.*)\ (.*)\ (.*) qq hard reject 
\(Your email is considered spam \((.*) spam-hits\)\):\ MAILFROM:(.*)\ 
RCPTTO:(.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}${NORMAL} ${YELLOW}${BASH_REMATCH[3]}${NORMAL} (Your email is 
considered spam (${WHITE}${BASH_REMATCH[4]}${NORMAL} spam hits)) 
MAILFROM:${VIOLET}${BASH_REMATCH[5]}${NORMAL} 
RCPTTO:${VIOLET}${BASH_REMATCH[6]}${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # spamd
                        elif [[ $LINE =~ "(spamd:|prefork: child)" ]]; then
                                if [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ clean\ message\ \((.*)/(.*)\)\ 
for\ (.*):(.*)\ in\ (.*)\ seconds,\ (.*)\ bytes" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} clean message 
(${GREEN}${BASH_REMATCH[7]}${NORMAL}/${RED}${BASH_REMATCH[8]}${NORMAL}) for 
${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL} in 
${WHITE}${BASH_REMATCH[11]}${NORMAL} seconds, 
${RED}${BASH_REMATCH[12]}${NORMAL} bytes."
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ identified\ spam\ 
\((.*)/(.*)\)\ for\ (.*):(.*)\ in\ (.*)\ seconds,\ (.*)\ bytes" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} identified 
spam (${GREEN}${BASH_REMATCH[7]}${NORMAL}/${RED}${BASH_REMATCH[8]}${NORMAL}) 
for ${RED}${BASH_REMATCH[9]}${NORMAL}:${RED}${BASH_REMATCH[10]}${NORMAL} in 
${WHITE}${BASH_REMATCH[11]}${NORMAL} seconds, 
${RED}${BASH_REMATCH[12]}${NORMAL} bytes."
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ processing\ message\ (.*) for 
(.*):(.*)" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} processing 
message ${VIOLET}${BASH_REMATCH[7]}${NORMAL} for 
${RED}${BASH_REMATCH[8]}${NORMAL}:${RED}${BASH_REMATCH[9]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ connection\ from\ (.*) at port 
(.*)" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} connection 
from ${RED}${BASH_REMATCH[7]}${NORMAL} at port 
${RED}${BASH_REMATCH[8]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ result:\ (.)\ (.)?(.)\ (.) 
(.*) 
scantime=(.*),size=(.*),user=(.*),uid=(.*),required_score=(.*),rhost=(.*),raddr=(.*),rport=(.*),mid=(.*),bayes=(.*),autolearn=(.*)"
 ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} result: 
${WHITE}${BASH_REMATCH[7]}${NORMAL} 
${WHITE}${BASH_REMATCH[8]}${NORMAL}${WHITE}${BASH_REMATCH[9]}${NORMAL} 
${WHITE}${BASH_REMATCH[10]}${NORMAL} ${WHITE}${BASH_REMATCH[11]}${NORMAL} 
scantime=${WHITE}${BASH_REMATCH[12]}${NORMAL},size=${WHITE}${BASH_REMATCH[13]}${NORMAL},user=${RED}${BASH_REMATCH[14]}${NORMAL},uid=${RED}${BASH_REMATCH[15]}${NORMAL},required_score=${WHITE}${BASH_REMATCH[16]}${NORMAL},rhost=${RED}${BASH_REMATCH[17]}${NORMAL},raddr=${RED}${BASH_REMATCH[18]}${NORMAL},rport=${WHITE}${BASH_REMATCH[19]}${NORMAL},mid=${WHITE}${BASH_REMATCH[20]}${NORMAL},bayes=${WHITE}${BASH_REMATCH[21]}${NORMAL},autolearn=${WHITE}${BASH_REMATCH[22]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ server\ successfully\ spawned\ 
child\ process,\ pid\ (.*)" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} server 
successfully spawned child process, pid ${WHITE}${BASH_REMATCH[7]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ spamd:\ handled\ cleanup\ of\ child\ 
pid\ (.*)\ due\ to\ (.*)" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}spamd:${NORMAL} handled 
cleanup of child pid ${WHITE}${BASH_REMATCH[7]}${NORMAL} due to 
${WHITE}${BASH_REMATCH[8]}${NORMAL}"
                                elif [[ $LINE =~ "(.*)-(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)\ info:\ prefork: child states: (.*)" ]]; then
                                        echo -e 
"${BLUE}${BASH_REMATCH[1]}-${BASH_REMATCH[2]} 
${BASH_REMATCH[3]}:${BASH_REMATCH[4]}:${BASH_REMATCH[5]}${NORMAL} 
${WHITE}${BASH_REMATCH[6]}${NORMAL} info: ${WHITE}prefork:${NORMAL} child 
states: ${WHITE}${BASH_REMATCH[7]}${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # imap
                        elif [[ $LINE =~ "(DEBUG: Connection,|INFO: 
LOGIN,|INFO: DISCONNECTED,)" ]]; then
                                if [[ $LINE =~ "(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ DEBUG:\ Connection,\ ip=\[(.*)\]" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL} 
${WHITE}DEBUG:${NORMAL} Connection, ip=[${RED}${BASH_REMATCH[5]}${NORMAL}]"
                                elif [[ $LINE =~ "(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ INFO:\ LOGIN,\ user=(.*),\ ip=\[(.*)\],\ 
protocol=(.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL} 
${WHITE}INFO:${NORMAL} LOGIN, user=${VIOLET}${BASH_REMATCH[5]}${NORMAL}, 
ip=[${RED}${BASH_REMATCH[6]}${NORMAL}], 
protocol=${WHITE}${BASH_REMATCH[7]}${NORMAL} "
                                elif [[ $LINE =~ "(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ INFO:\ DISCONNECTED,\ user=(.*),\ ip=\[(.*)\],\ 
headers=(.*),\ body=(.*),\ rcvd=(.*),\ sent=(.*),\ time=(.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL} 
${WHITE}INFO:${NORMAL} DISCONNECTED, user=${VIOLET}${BASH_REMATCH[5]}${NORMAL}, 
ip=[${RED}${BASH_REMATCH[6]}${NORMAL}], 
headers=${WHITE}${BASH_REMATCH[7]}${NORMAL}, 
body=${WHITE}${BASH_REMATCH[8]}${NORMAL}, 
rcvd=${WHITE}${BASH_REMATCH[9]}${NORMAL}, 
sent=${WHITE}${BASH_REMATCH[8]}${NORMAL}, 
time=${WHITE}${BASH_REMATCH[9]}${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # clamd
                        elif [[ $LINE =~ "/var/qmail/simscan/" ]]; then
                                if [[ $LINE =~ "(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*): OK$" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL} 
${WHITE}${BASH_REMATCH[5]}${NORMAL}: ${GREEN}OK${NORMAL}"
                                elif [[ $LINE =~ "(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*): (.*) FOUND$" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL} 
${WHITE}${BASH_REMATCH[5]}${NORMAL}: ${YELLOW}${BASH_REMATCH[6]}${NORMAL} 
${RED}FOUND${NORMAL}"
                                else
                                        echo -e "${RED}FIXME${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        # No hit with any of the above
                        else
                                if [[ $LINE =~ "(.*)\ 
([0-9]*):([0-9]*):([0-9]*)\ (.*)" ]]; then
                                        echo -e "${BLUE}${BASH_REMATCH[1]} 
${BASH_REMATCH[2]}:${BASH_REMATCH[3]}:${BASH_REMATCH[4]}${NORMAL} 
${WHITE}${BASH_REMATCH[5]}${NORMAL}"
                                else
                                        # Not a regular formatted logline
                                        echo -e "${RED}NOHIT${NORMAL} 
${YELLOW}${LINE}${NORMAL}"
                                fi
                        fi
                                
                done
        else
                # Just in case less has not been disabled remove all the color 
codes, print the bare lines
                while read LINE; do
                        echo $LINE
                done
        fi
}

#####################################################################
## get sed commands for trimming the output
#
b32_get_sed_script(){

#sedfile=${mydir%bin}etc/qmlog-trim.sed
sedfile=/opt/qmailtoaster-plus/etc/qmlog-trim.sed

if [ -f $sedfile ]; then
  while read sedline; do
    sedtrim="$sedtrim $sedline;"
  done < $sedfile
else
  echo "$me $myver notice - sed file $sedfile not found"
fi

sedtrim="$sedtrim $sedparm'"
}

#####################################################################
## select files to show based on name, dates, and/or content pattern
#
b34_select_files(){

if [ -z "$namepattern" ]; then
  filepattern="*.s"
else
  filepattern="*$namepattern*.s"
fi

if [ ! -z "$dateparm" ]; then
  c342_select_date_time_range
elif [ ! -z "$namepattern" ]; then
  c344_select_filename_pattern
else
  somefiles="$logdir/$service/$filepattern $logdir/$service/current"
fi

if [ ! -z "$contpattern" ]; then
  c346_select_content
else
  showfiles="$somefiles"
fi
}

#####################################################################
## select files by date, time range
#
c342_select_date_time_range(){

d3422_setup_date_time_fields

for logfile in $(ls $logdir/$service/$filepattern 2>/dev/null) \
      $logdir/$service/current; do
  d3425_select_each_file
done

if [ -z "$somefiles" ]; then
  echo "$me - no $service log found with date/time matching $dateparm"
  exit 1
fi

position_less="-p${from_mon}-${from_day}"
}

#####################################################################
## setup date, time fields for file selection
#
d3422_setup_date_time_fields(){

from_datetime=$(echo $dateparm | cut --delimiter=- --fields=1)
thru_datetime=$(echo $dateparm | cut -s --delimiter=- --fields=2)
from_mmdd=$(echo $from_datetime | cut --delimiter=: --fields=1)
from_hhmm=$(echo $from_datetime | cut -s --delimiter=: --fields=2)
thru_mmdd=$(echo $thru_datetime | cut --delimiter=: --fields=1)
thru_hhmm=$(echo $thru_datetime | cut -s --delimiter=: --fields=2)
: ${thru_mmdd:=$from_mmdd}
: ${from_hhmm:=0000}
: ${thru_hhmm:=2359}
from_mon=$(echo $from_mmdd | cut -b 1-2)
from_day=$(echo $from_mmdd | cut -b 3-4)
thru_mon=$(echo $thru_mmdd | cut -b 1-2)
thru_day=$(echo $thru_mmdd | cut -b 3-4)
curryear=$(date +%Y)
currmmdd=$(date +%m%d)

if [ "$from_mmdd" -gt "$currmmdd" ]; then
  from_year=$(($curryear - 1))
else
  from_year=$curryear
fi
if [ "$thru_mmdd" -gt "$currmmdd" ]; then
  thru_year=$(($curryear - 1))
else
  thru_year=$curryear
fi

from_date_time="$from_year$from_mon$from_day$from_hhmm"
thru_date_time="$thru_year$thru_mon$thru_day$thru_hhmm"
}
#####################################################################
## select each file by date/time
#
d3425_select_each_file(){

read logdate logtime restoflog <<!
$(head -n1 $logfile | tai64nlocal)
!
begdate=$(echo $logdate | tr -d -)
begtime=$(echo ${logtime%:[0-9][0-9]\.[0-9]*} | tr -d :)
beg_date_time=$begdate$begtime

read logdate logtime restoflog <<!
$(tail -n1 $logfile | tai64nlocal)
!
enddate=$(echo $logdate | tr -d -)
endtime=$(echo ${logtime%:[0-9][0-9]\.[0-9]*} | tr -d :)
end_date_time=$enddate$endtime

if [ "$end_date_time" -lt "$from_date_time" ] \
      || [ "$beg_date_time" -gt "$thru_date_time" ]; then
  continue
else
  somefiles="$somefiles $logfile"
fi
}

#####################################################################
## select files by filename pattern
#
c344_select_filename_pattern(){

somefiles=$(ls $logdir/$service/$filepattern 2>1)

if [ -z "$somefiles" ]; then
  echo "$me - no $service log found with filename pattern $namepattern"
  exit 1
fi
}
#####################################################################
## select files by content
#
c346_select_content(){

showfiles=$(grep -lce "$contpattern" $somefiles 2>/dev/null)

if [ -z "$showfiles" ]; then
  if [ -z "$namepattern" ] && [ -z "$dateparm" ]; then
    withmsg=""
  elif [ -z "$namepattern" ]; then
    withmsg=" with date/time matching $dateparm"
  elif [ -z "$dateparm" ]; then
    withmsg=" with filename pattern $namepattern"
  else
    withmsg=" with date/time matching $dateparm and filename pattern 
$namepattern"
  fi
  echo "$me - pattern $contpattern not found in any $service logs $withmsg"
  exit 1
fi

if [ -z "$position_less" ]; then
  position_less="-p$contpattern"
fi
}

#####################################################################
## show usage output
#
q1_usage(){

echo "$me $myver -  show current log of service 'service'"
echo "usage: qmlog service [option] ..."
q2_services
echo "options:"
echo "  -h[elp]      this help"
echo "  -l[ist]      list saved logs of service"
echo "  -f           follow as it grows, using 'tail -f'"
echo "  -c           add colors, disables less"
echo "  -t N         show (tail) last N lines"
echo "  -nl          show without using less"
echo "  -nt          show with no trimming"
echo "  -d mmdd[:hhmm][-mmdd[:hhmm]]"
echo "               show logs that contain the date 'mmdd' [thru -'mmdd']"
echo "  -lc regex    show logs that contain a string that matches 'regex'"
echo "  -ln pattern  show logs with file name containing 'pattern'"
echo "  -s command   pipe output through sed 'command'"
echo "  -g regex     show only lines that match the string 'regex'"
exit 1
}

#####################################################################
## show services output
#
q2_services()
{
echo -n "services:"
for opt in `ls $logdir`; do
  if [ -d "$logdir/$opt" ]; then
    echo -n " $opt"
  fi
done
echo
}

#####################################################################
## begin main processing here
#
me=$(basename $0)
myver=v0.4
mydir=$(dirname $0)
: ${logdir:=/var/log/qmail}
option=""
command=""
tai64n="| tai64nlocal"
lesscmd="| less"
grepcmd=""
sedtrim="| sed '"
sedparm=""
dateparm=""
namepattern=""
contpattern=""
service=""
showfiles=""
position_less=""

while (( "$#" )); do
  case $1 in
    "-h" | "-help" | "--help" )
      q1_usage
      ;;
    "-l" | "-list" | "--list" )
      option="list"
      shift
      ;;
    "-f" )
      command="$command $1"
      lesscmd=""
      shift
      ;; 
    "-t" )
      command="$command -n $2"
      shift 2
      ;;
    "-c" )
      lesscmd=""
      colorize="| colorize"
      shift
      ;;
    "-nl" | "-noless" | "--noless" )
      lesscmd=""
      shift
      ;;
    "-nt" | "-notrim" | "--notrim" )
      sedtrim=""
      shift
      ;;
    "-d" | "-date" | "-dates" | "--date" | "--dates" )
      dateparm=$2
      shift 2
      ;;
    "-lc" | "-logcont" | "--logcont" )
      contpattern=$2
      shift 2
      ;;
    "-ln" | "-logname" | "--logname" )
      namepattern=$2
      shift 2
      ;;
    "-s" | "-sed" | "--sed" )
      sedparm="$sedparm $2;"
      shift 2
      ;;
    "-g" | "-grep" | "--grep" )
      greppattern=$2
      shift 2
      ;;
    * )
      service=$1
      shift
      ;;
  esac
done

a1_edit_parameters

case $option in
  "list" )
    a2_list_log_files
    ;;
  * )
    a3_show_logs_content
    ;;
esac

exit 0
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users

Reply via email to