On Mon, Aug 16, 2004 at 08:13:26AM -0700, Robert Spier wrote:
> > > It would be cool to integrate mailgraph (or something like it) into
> > > qpsmtpd.
> > > http://www.onlamp.com/pub/a/onlamp/2004/08/12/mailgraph.html
> >
> > Interesting! My boss just last week suggested we need to do some
> > internal promoting and maintain some uptime graphs of the various
> > systems. This seems like a good addition to that goal. I'll try and
> > take a look at it to see how easy it would be modify to understand
> > qpsmtpd's log files...
>
> I don't really like the idea of parsing qpsmtpd's log files, because
> in high debug modes (like we run) there's a ton of "garbage" in
> there.
>
> Maybe we need some sort of "log_entry" or "statistics" plugin hook or
> something to allow for dynamic logging and statistics gathering.
>
> It gets weirder when you do odder things like we're doing in our
> installation. We don't reject certain things (in order to not create
> more noise) and just silently absorb them -- but don't pass them on.
> This is something we'd want to record, but the log rules would have to
> be custom.
>
> Here's an expansion on this.
> - We create a transaction_end hook, that _always_ gets run at the
> end of a transaction
> - We teach plugins to record things.. so spamassassin could do
> something like $qp->transaction->notes( 'stats_spam' => 1 ); or
> something.
> - The transaction_end hook would then write that out somewhere.
We already have 'queue' and 'deny' hooks - we can just plug in to that.
I've been using the attached plugin to do stats capturing to a dbm
database - I'd probably change the backend now (given some weirdness I've
been seeing with Berkeley DB under load), but the general concept has
worked well. Happy to have it generalised further too.
Cheers,
Gavin
=head1 NAME
qpstats - capture summary statistics of mails queued and denied
=head1 DESCRIPTION
qpstats is a plugin to capture summary statistics for mails queued and
denied by qpsmtpd. Counters are stored in a dbm database, indexed by a
key whose format is configurable.
Two kinds of notes can be passed to qpstats to set additional counters
than just queued and denied emails:
=over 4
=item qpstats_add
This note can be used to increment additional counters in the database.
The format is '<plugin>:<code>'. Code can be either a numeric qpsmtpd
result constant (DENY, DENYSOFT, etc.), or an arbitrary string.
=item qpstats_queue_replace
This note can be used to increment a counter instead of the standard
queue counter in a queue hook. Useful if you want to split a 'queue'
up into categories (e.g. splitting out spam redirects).
The format is '<plugin>:<code>'. Code can be either a numeric qpsmtpd
result constant (DENY, DENYSOFT, etc.), or an arbitrary string.
=back
=head1 AUTHOR
Gavin Carr <[EMAIL PROTECTED]>.
=cut
BEGIN { @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File) }
use AnyDBM_File;
use Fcntl qw(:DEFAULT :flock);
use POSIX qw(strftime);
use strict;
my $VERSION = 0.02;
# The database key format is defined using strftime formats, except that
# %P is used for the plugin name, and %C for the disposition code
my $KEY = "%Y%m%d-%P-%C";
my ($QPHOME) = ($0 =~ m!(.*?)/([^/]+)$!);
my $DBNAME = "qpstats.dbm";
my %CODE = ( DENY => 'deny', DENYSOFT => 'denysoft' );
sub register {
my ($self, $qp, %arg) = @_;
$self->{_defaults} = \%arg;
$self->register_hook("queue", "queue_handler");
$self->register_hook("deny", "deny_handler");
}
sub queue_handler {
my ($self, $transaction) = @_;
# A qpstats_queue_replace note is recorded *instead* of a 'queue' count
if (my $note = $transaction->notes('qpstats_queue_replace')) {
my ($plugin, $code) = split /\s*:\s*/, $note;
$self->record($plugin, $code);
}
else {
$self->record('queue', 'ok');
$self->record('queue', 'rcpt', scalar($transaction->recipients));
}
# A qpstats_add note is always recorded
if (my $note = $transaction->notes("qpstats_add")) {
my ($plugin, $code) = split /\s*:\s*/, $note;
$self->record($plugin, $code);
}
return (DECLINED);
}
sub deny_handler {
my ($self, $transaction, $plugin, $code, $msg) = @_;
$self->log(6, "DENY from $plugin, $code, $msg");
$self->record($plugin, $code);
# A qpstats_add note is always recorded
if (my $note = $transaction->notes("qpstats_add")) {
my ($plugin, $code) = split /\s*:\s*/, $note;
$self->record($plugin, $code);
}
return (DECLINED);
}
sub record
{
my ($self, $plugin, $code, $count) = @_;
$count ||= 1;
$code ||= 'ok';
$code = 'deny' if $code eq DENY;
$code = 'denysoft' if $code eq DENYSOFT;
my $dbdir = -d "$QPHOME/var/db" ? "$QPHOME/var/db" : "$QPHOME/config";
my $db = "$dbdir/$DBNAME";
$self->log(6,"using $db as qpstats database");
# Build key
my $key = $self->{_defaults}->{key} || $KEY;
$key =~ s/%P/$plugin/g;
$key =~ s/%C/$code/g;
$key = strftime($key,localtime);
# Open qpstats db
unless (open LOCK, ">$db.lock") {
$self->log(2, "opening lockfile failed: $!");
return;
}
unless (flock LOCK, LOCK_EX) {
$self->log(2, "flock of lockfile failed: $!");
close LOCK;
return;
}
my %db = ();
unless (tie %db, 'AnyDBM_File', $db, O_CREAT|O_RDWR, 0640) {
$self->log(2, "tie to database $db failed: $!");
close LOCK;
return;
}
# Increment counter
$db{$key} = ($db{$key}||0)+$count;
untie %db;
close LOCK;
}
# arch-tag: 80efb65c-4668-4aea-8ce5-fbd7b4f3a0f1
