Re: Formatting log ouutput (adaptive logging)

[Bob Dodds]

Robin Bowes wrote:

Hi,

I'm reviewing my modifications to the adaptive logging plugin to make
sure it produces sensible output.

Currently, it writes a "rejected" log file using the following code:

    warn( ${$} . ' ' . $self->{_rejectprefix} . ' ' .
        join ",",
        $prev_hook,
        $return,
        $return_text,
        $sender,
        $recipients,
        $mail_size,
        $self->qp->connection->remote_host,
        $self->qp->connection->remote_ip,
        $relay_client,
        $auth_method,
        $auth_user
        . "\n"
    );

This produces output like this (all on one line):

2006-02-19 01:14:02.076939500 11641 -- check_earlytalker,902,Connecting
host started transmitting before SMTP
greeting,,,0,Unknown,72.11.140.231,no,,

I can then post-process the log (it's effectively a CSV file) and
produce stats + reports listing plugin statistics, mails rejected per
IP, etc.

This works fune, unless of course any of the items in the list contain
commas, e.g.:

2006-02-18 19:31:46.184453500 31574 -- check_spamhelo,901,Sorry, I don't
believe that you are
aol.com.,,,0,68-184-58-123.dhcp.mtgm.al.charter.com,68.184.58.123,no,,

As I see it, there are two ways to deal with this:

1. Use some separator character other than comma that can be guaranteed
not to occur in any of the list items

2. Quote each item, if necessary.

I'm currently leaning towards option 2, using something like
Text::CSV_XS, e.g.:

    my @logfields = (
        $prev_hook,
        $return,
        $return_text,
        $sender,
        $recipients,
        $mail_size,
        $self->qp->connection->remote_host,
        $self->qp->connection->remote_ip,
        $relay_client,
        $auth_method,
        $auth_user,
    );

    use Text::CSV_XS;
    my $csv = Text::CSV_XS->new;
    if ($csv->combine(@logfields)) {
        warn( ${$} . ' '
            . $self->{_rejectprefix} . ' '
            . $csv->string
            . "\n",
        );
    } else {
        warn("combine() failed on argument: ", $csv->error_input, "\n");
    }

Any reason why this is not a good idea?

Is it OK to add a dependency on Text::CSV_XS ?

R.

I don't respect other peoples' commas that much.
Maybe that's because I delete them before I get
to know them.

my @logfields = (
$prev_hook,
$return,
( $return_text =~ s/,//g ? $return_text : $return_text ),
$sender,
$recipients,
$mail_size,
$self->qp->connection->remote_host,
$self->qp->connection->remote_ip,
$relay_client,
$auth_method,
$auth_user
);

I need message ID.

-Bob