Re: spamlogd whitelists every logged rdr-to connection
> So, you do agree this is not something that should be happening, right? No, I figure you should not log the connections you're sending to spamd. But we can figure some way to make this a bit clearer.
Re: spamlogd whitelists every logged rdr-to connection
On 2013-W10-4 19:20 -0700, Bob Beck wrote: > I think this should not only be addressed in the documentation (for > pcap-filter, spamd -M / spamlogd, pf.conf log/rdr-to / pflogd, > tcpdump), but in the actual spamlogd code changes, too -- it should > be smart enough to not automatically whitelist the connections that > are rewritten to the default spamd port. Constantine, this statement of yours makes no sense. spamlogd only pays attention to connections to port 25. spamlogd listens on 8025. Unless you are doing something crazy in your pf.conf. Nothing crazier than what's in my prior message, just a few extra rules for gif0. I'm telling you, Bob, spamlogd whitelists those connections that go to spamd, to port 8025! No kidding! Yes, not 25, but 8025! This is a sample rule that causes the default spamlogd to immediately whitelist the spammer: pass in log on re0 proto tcp from any os Windows to any port smtp \ rdr-to 127.0.0.1 port spamd If you don't believe me, just try it out. Else, and although not related to pcap(3), how do you expect that spamd -M works, when the dst address gets rewritten to 127.0.0.1? I'm surprised I'm the first person with this problem; I presume a lot of prior people just thought they were crazy, and gave up. I have a vague recollection of encountering it back in 3.6 days or so. So, you do agree this is not something that should be happening, right? Cheers, Constantine.
Re: spamlogd whitelists every logged rdr-to connection
> I think this should not only be addressed in the documentation (for > pcap-filter, spamd -M / spamlogd, pf.conf log/rdr-to / pflogd, > tcpdump), but in the actual spamlogd code changes, too -- it should > be smart enough to not automatically whitelist the connections that > are rewritten to the default spamd port. Constantine, this statement of yours makes no sense. spamlogd only pays attention to connections to port 25. spamlogd listens on 8025. Unless you are doing something crazy in your pf.conf.
Re: spamlogd whitelists every logged rdr-to connection
On 2013-W10-4 15:08 -0700, Bob Beck wrote:
Show me your pf.conf please - I'd like to address your problem
in the documentation or another way.. I don't like a knob like
this.
According to pf.conf(5), you can only log to one pflog(4) interface
at a time, so, it's not exactly trivial to modify the rules, without
modifying the rest of the workflow.
BTW, don't laugh at these rules. They're highly effective, 99,9% of
my incoming mail does not go through greylisting at all whatsoever,
yet all spammers always end up talking to spamd. Some of these may
seem redundant or a noop, they're mostly used for statistic purposes,
just as you use spamlogd to whitelist already whitelisted netblocks,
and see how much mail individual IPs send / receive.
As for documentation, I think it's counterintuitive that `tcpdump`
shows rewritten addresses / ports, yet spamlogd pcap(3) filter
operates on the original ones. So, I know spamlogd(8) says what it
says, but after running tcpdump on /var/log/pflog, it makes absolutely
no sense that spamlogd would be whitelisting all of those connections
that are redirected to spamd. (It also makes no sense how spamd -M works
with the rules as they are below (although I'm very glad that it does,
and works great at that).)
I think this should not only be addressed in the documentation (for
pcap-filter, spamd -M / spamlogd, pf.conf log/rdr-to / pflogd, tcpdump),
but in the actual spamlogd code changes, too -- it should be smart
enough to not automatically whitelist the connections that are
rewritten to the default spamd port.
Best regards,
Constantine.
# overall policy:
# * always pass BSD, regardless of IP
# * greylist all win, and non-a/r, and trap anyone
pass in log on re0 proto tcp from any to any port smtp label "smtp any / other"
pass in log on re0 proto tcp from any os "Windows .NET" to any port smtp label "smtp
Windows .NET"
pass in log on re0 proto tcp from any os Windows to any port smtp label "smtp
Windows" \
rdr-to 127.0.0.1 port spamd
pass in log on re0 proto tcp from any os Linux to any port smtp label "smtp
Linux"
pass in quick log on re0 proto tcp from any os FreeBSD to any port smtp label "smtp
FreeBSD"
pass in quick log on re0 proto tcp from any os NetBSD to any port smtp label "smtp
NetBSD"
pass in quick log on re0 proto tcp from any os OpenBSD to any port smtp label "smtp
OpenBSD"
pass in log on re0 proto tcp from any os unknown to any port smtp label "smtp
unknown"
# rules for spamd(8)
block return in quick log proto tcp from any to any port 8025
table persist
#table persist file "/etc/mail/nospamd"
#pass in on egress proto tcp from any to any port smtp \
#rdr-to 127.0.0.1 port spamd
pass in on egress proto tcp from {,,} to any port smtp \
rdr-to 127.0.0.1 port spamd
pass in on egress proto tcp from any to spamd.mx.example.su port smtp \
rdr-to 127.0.0.1 port spamd
#pass in on egress proto tcp from to any port smtp
pass in log on egress proto tcp from to any port smtp
pass out log on egress proto tcp to any port smtp
On Thu, Mar 07, 2013 at 12:04:22PM -0800, Constantine A. Murenin wrote:
> On 2013-W10-3 15:46 -0700, Bob Beck wrote:
> > > Yes, one could log stuff into different pflog interfaces, but I don't
> > > understand why pf.conf `pass in ... log ... port smtp ...` is effectively
> > > redefined to mean `add ` when spamlogd is running,
> >
> > http://www.openbsd.org/cgi-bin/man.cgi?query=spamlogd
> >
> > and RTFM for the first two sentences - and it's pretty darn clear.
> >
> > Those of us that whitelist blocks of addresses (and log them) like
> > this behaviour to track what mailservers we are seeing like the
> > current behavior.
>
> Point taken; this should be optional.
> Index: spamlogd.8
> ===
> RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.8,v
> retrieving revision 1.17
> diff -u -d -p -8 -r1.17 spamlogd.8
> --- spamlogd.8 4 Mar 2011 21:01:49 - 1.17
> +++ spamlogd.8 7 Mar 2013 19:41:24 -
> @@ -17,17 +17,17 @@
> .Dd $Mdocdate: March 4 2011 $
> .Dt SPAMLOGD 8
> .Os
> .Sh NAME
> .Nm spamlogd
> .Nd spamd whitelist updating daemon
> .Sh SYNOPSIS
> .Nm spamlogd
> -.Op Fl DI
> +.Op Fl DIU
> .Op Fl i Ar interface
> .Op Fl l Ar pflog_interface
> .Op Fl W Ar whiteexp
> .Op Fl Y Ar synctarget
> .Sh DESCRIPTION
> .Nm
> manipulates the
> .Xr spamd 8
> @@ -73,16 +73,32 @@ target of outbound SMTP connections.
> Specify a network interface on which packets must arrive.
> The default is to watch for connections logged from all interfaces.
> .It Fl l Ar pflog_interface
> Specify a
> .Xr pflog 4
> interface to listen for connection notifications.
> The default is to watch for connections logged on
> .Dq pflog0 .
> +.It Fl U
> +Specify that for inbound SMTP connections,
> +.Nm
> +is only to update existing
> +.Pa /var/db/spamd
> +entries, without adding any new ones.
> +By default
> +.Nm
> +wil
Re: spamlogd whitelists every logged rdr-to connection
Show me your pf.conf please - I'd like to address your problem
in the documentation or another way.. I don't like a knob like
this.
On Thu, Mar 07, 2013 at 12:04:22PM -0800, Constantine A. Murenin wrote:
> On 2013-W10-3 15:46 -0700, Bob Beck wrote:
> > > Yes, one could log stuff into different pflog interfaces, but I don't
> > > understand why pf.conf `pass in ... log ... port smtp ...` is effectively
> > > redefined to mean `add ` when spamlogd is running,
> >
> > http://www.openbsd.org/cgi-bin/man.cgi?query=spamlogd
> >
> > and RTFM for the first two sentences - and it's pretty darn clear.
> >
> > Those of us that whitelist blocks of addresses (and log them) like
> > this behaviour to track what mailservers we are seeing like the
> > current behavior.
>
> Point taken; this should be optional.
> Index: spamlogd.8
> ===
> RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.8,v
> retrieving revision 1.17
> diff -u -d -p -8 -r1.17 spamlogd.8
> --- spamlogd.84 Mar 2011 21:01:49 - 1.17
> +++ spamlogd.87 Mar 2013 19:41:24 -
> @@ -17,17 +17,17 @@
> .Dd $Mdocdate: March 4 2011 $
> .Dt SPAMLOGD 8
> .Os
> .Sh NAME
> .Nm spamlogd
> .Nd spamd whitelist updating daemon
> .Sh SYNOPSIS
> .Nm spamlogd
> -.Op Fl DI
> +.Op Fl DIU
> .Op Fl i Ar interface
> .Op Fl l Ar pflog_interface
> .Op Fl W Ar whiteexp
> .Op Fl Y Ar synctarget
> .Sh DESCRIPTION
> .Nm
> manipulates the
> .Xr spamd 8
> @@ -73,16 +73,32 @@ target of outbound SMTP connections.
> Specify a network interface on which packets must arrive.
> The default is to watch for connections logged from all interfaces.
> .It Fl l Ar pflog_interface
> Specify a
> .Xr pflog 4
> interface to listen for connection notifications.
> The default is to watch for connections logged on
> .Dq pflog0 .
> +.It Fl U
> +Specify that for inbound SMTP connections,
> +.Nm
> +is only to update existing
> +.Pa /var/db/spamd
> +entries, without adding any new ones.
> +By default
> +.Nm
> +will whitelist the source of all inbound SMTP connections that are logged.
> +This option is needed if connections redirected to
> +.Xr spamd 8
> +are logged,
> +and no distinct
> +.Xr pflog 4
> +interface is configured for
> +.Nm .
> .It Fl W Ar whiteexp
> Adjust the time for
> .Ar whiteexp
> in hours.
> The default is 864 hours (approximately 36 days); maximum is 2160 hours
> (approximately 90 days).
> .It Fl Y Ar synctarget
> Add a target to receive synchronisation messages; see
> Index: spamlogd.c
> ===
> RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
> retrieving revision 1.21
> diff -u -d -p -8 -r1.21 spamlogd.c
> --- spamlogd.c18 Mar 2011 22:37:06 - 1.21
> +++ spamlogd.c7 Mar 2013 19:46:44 -
> @@ -63,29 +63,30 @@
> int debug = 1;
> int greylist = 1;
> FILE *grey = NULL;
>
> u_short sync_port;
> int syncsend;
> u_int8_t flag_debug = 0;
> u_int8_t flag_inbound = 0;
> +u_int8_t flag_updateonly = 0;
> char *networkif = NULL;
> char *pflogif = "pflog0";
> char errbuf[PCAP_ERRBUF_SIZE];
> pcap_t *hpcap = NULL;
> struct syslog_datasdata = SYSLOG_DATA_INIT;
> time_twhiteexp = WHITEEXP;
> extern char *__progname;
>
> void logmsg(int , const char *, ...);
> void sighandler_close(int);
> int init_pcap(void);
> void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char *);
> -int dbupdate(char *, char *);
> +int dbupdate(char *, char *, int);
> void usage(void);
>
> void
> logmsg(int pri, const char *msg, ...)
> {
> va_list ap;
> va_start(ap, msg);
>
> @@ -187,22 +188,22 @@ logpkt_handler(u_char *user, const struc
> sizeof(ipstraddr));
> }
>
> if (ipstraddr[0] != '\0') {
> if (hdr->dir == PF_IN)
> logmsg(LOG_DEBUG,"inbound %s", ipstraddr);
> else
> logmsg(LOG_DEBUG,"outbound %s", ipstraddr);
> - dbupdate(PATH_SPAMD_DB, ipstraddr);
> + dbupdate(PATH_SPAMD_DB, ipstraddr, hdr->dir == PF_IN);
> }
> }
>
> int
> -dbupdate(char *dbname, char *ip)
> +dbupdate(char *dbname, char *ip, int inbound)
> {
> HASHINFOhashinfo;
> DBT dbk, dbd;
> DB *db;
> struct gdatagd;
> time_t now;
> int r;
> struct in_addr ia;
> @@ -227,16 +228,20 @@ dbupdate(char *dbname, char *ip)
> /* add or update whitelist entry */
> r = db->get(db, &dbk, &dbd, 0);
> if (r == -1) {
> logmsg(LOG_NOTICE, "db->get failed (%m)");
> goto bad;
> }
>
> if (r) {
> + if (inbound && flag_updateonly) {
> +
Re: spamlogd whitelists every logged rdr-to connection
On 2013-W10-3 15:46 -0700, Bob Beck wrote:
> > Yes, one could log stuff into different pflog interfaces, but I don't
> > understand why pf.conf `pass in ... log ... port smtp ...` is effectively
> > redefined to mean `add ` when spamlogd is running,
>
> http://www.openbsd.org/cgi-bin/man.cgi?query=spamlogd
>
> and RTFM for the first two sentences - and it's pretty darn clear.
>
> Those of us that whitelist blocks of addresses (and log them) like
> this behaviour to track what mailservers we are seeing like the
> current behavior.
Point taken; this should be optional.
Index: spamlogd.8
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.8,v
retrieving revision 1.17
diff -u -d -p -8 -r1.17 spamlogd.8
--- spamlogd.8 4 Mar 2011 21:01:49 - 1.17
+++ spamlogd.8 7 Mar 2013 19:41:24 -
@@ -17,17 +17,17 @@
.Dd $Mdocdate: March 4 2011 $
.Dt SPAMLOGD 8
.Os
.Sh NAME
.Nm spamlogd
.Nd spamd whitelist updating daemon
.Sh SYNOPSIS
.Nm spamlogd
-.Op Fl DI
+.Op Fl DIU
.Op Fl i Ar interface
.Op Fl l Ar pflog_interface
.Op Fl W Ar whiteexp
.Op Fl Y Ar synctarget
.Sh DESCRIPTION
.Nm
manipulates the
.Xr spamd 8
@@ -73,16 +73,32 @@ target of outbound SMTP connections.
Specify a network interface on which packets must arrive.
The default is to watch for connections logged from all interfaces.
.It Fl l Ar pflog_interface
Specify a
.Xr pflog 4
interface to listen for connection notifications.
The default is to watch for connections logged on
.Dq pflog0 .
+.It Fl U
+Specify that for inbound SMTP connections,
+.Nm
+is only to update existing
+.Pa /var/db/spamd
+entries, without adding any new ones.
+By default
+.Nm
+will whitelist the source of all inbound SMTP connections that are logged.
+This option is needed if connections redirected to
+.Xr spamd 8
+are logged,
+and no distinct
+.Xr pflog 4
+interface is configured for
+.Nm .
.It Fl W Ar whiteexp
Adjust the time for
.Ar whiteexp
in hours.
The default is 864 hours (approximately 36 days); maximum is 2160 hours
(approximately 90 days).
.It Fl Y Ar synctarget
Add a target to receive synchronisation messages; see
Index: spamlogd.c
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
retrieving revision 1.21
diff -u -d -p -8 -r1.21 spamlogd.c
--- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
+++ spamlogd.c 7 Mar 2013 19:46:44 -
@@ -63,29 +63,30 @@
int debug = 1;
int greylist = 1;
FILE *grey = NULL;
u_short sync_port;
int syncsend;
u_int8_tflag_debug = 0;
u_int8_tflag_inbound = 0;
+u_int8_tflag_updateonly = 0;
char *networkif = NULL;
char *pflogif = "pflog0";
charerrbuf[PCAP_ERRBUF_SIZE];
pcap_t *hpcap = NULL;
struct syslog_data sdata = SYSLOG_DATA_INIT;
time_t whiteexp = WHITEEXP;
extern char*__progname;
void logmsg(int , const char *, ...);
void sighandler_close(int);
intinit_pcap(void);
void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char *);
-intdbupdate(char *, char *);
+intdbupdate(char *, char *, int);
void usage(void);
void
logmsg(int pri, const char *msg, ...)
{
va_list ap;
va_start(ap, msg);
@@ -187,22 +188,22 @@ logpkt_handler(u_char *user, const struc
sizeof(ipstraddr));
}
if (ipstraddr[0] != '\0') {
if (hdr->dir == PF_IN)
logmsg(LOG_DEBUG,"inbound %s", ipstraddr);
else
logmsg(LOG_DEBUG,"outbound %s", ipstraddr);
- dbupdate(PATH_SPAMD_DB, ipstraddr);
+ dbupdate(PATH_SPAMD_DB, ipstraddr, hdr->dir == PF_IN);
}
}
int
-dbupdate(char *dbname, char *ip)
+dbupdate(char *dbname, char *ip, int inbound)
{
HASHINFOhashinfo;
DBT dbk, dbd;
DB *db;
struct gdatagd;
time_t now;
int r;
struct in_addr ia;
@@ -227,16 +228,20 @@ dbupdate(char *dbname, char *ip)
/* add or update whitelist entry */
r = db->get(db, &dbk, &dbd, 0);
if (r == -1) {
logmsg(LOG_NOTICE, "db->get failed (%m)");
goto bad;
}
if (r) {
+ if (inbound && flag_updateonly) {
+ logmsg(LOG_DEBUG,"ignoring %s", ip);
+ goto bad;
+ }
/* new entry */
memset(&gd, 0, sizeof(gd));
gd.first = now;
gd.bcount = 1;
gd.pass = now;
gd.expire = now + whiteexp;
memset(&dbk, 0, sizeof(dbk));
dbk.size = strlen(ip);
@@ -280,17 +285,17 @@ dbupdate(char *dbname, ch
Re: spamlogd whitelists every logged rdr-to connection
> Yes, one could log stuff into different pflog interfaces, but I don't > understand why pf.conf `pass in ... log ... port smtp ...` is effectively > redefined to mean `add ` when spamlogd is running, http://www.openbsd.org/cgi-bin/man.cgi?query=spamlogd and RTFM for the first two sentences - and it's pretty darn clear. Those of us that whitelist blocks of addresses (and log them) like this behaviour to track what mailservers we are seeing like the current behavior.
Re: spamlogd whitelists every logged rdr-to connection
I'm simply logging greylisted connections;
it's spamlogd that whitelists them just because they're logged.
It doesn't make sense that logging greylisted or blacklisted connections
would immediately turn them into being whitelisted by spamlogd.
Same goes for logging connections that are already whitelisted
through rules and tables other than .
Why would you want them whitelisted several times?
My second patch fixes these issues, without breaking valid setups.
Yes, one could log stuff into different pflog interfaces, but
I don't understand why pf.conf `pass in ... log ... port smtp ...`
is effectively redefined to mean `add `
when spamlogd is running, even when connections are redirected
to spamd for stuttering or greylisting.
That's not something that seems reasonable, and an update-if-exists
logic would make so much more sense.
C.
On 2013-W10-3 14:56 -0700, Bob Beck wrote:
No constatine - that is not the best approach. if you are whitelisting
grelisted connections
or blacklisted connections that are blocked you have your pf.conf or
spamlogd setup wrong.
On Wed, Mar 6, 2013 at 2:54 PM, Constantine A. Murenin wrote:
> Bob, I agree, the hdr->rewritten approach is not good.
>
> I think the best approach here would be to not add any new entries on
> incoming connections in the first place, but only keep updating the existing
> ones (when the connection is incoming).
>
> In addition to not whitelisting greylisted or blocked connection that are
> logged, this would also prevent the case of double-whitelisting the
> connections that are logged and whitelisted through other rules, without any
> adverse side effects or unexpected behaviour.
>
> Patch attached inline.
>
> C.
>
>
> On 2013-W10-3 13:47 -0700, Bob Beck wrote:
>>
>> No constantine - the solution is to simply not use the "log" keyword
>> on such traffic
>>
>> All of my boxen I run this on also rewite the traffic to (pool) of
>> mailservers so this is
>> not accurate.
>>
>> Simply don't log the traffic you don't want spamlogd to see. the
>> *point* of spamlogd
>> is to ensure all continuing valid connections *stay* whitelisted.
>>
>> On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin wrote:
>> > Hi,
>> >
>> > I've started using spamlogd, and since then, every single connection
>> > attempt
>> > results in the host being whitelisted.
>> >
>> > I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog,
>> > and
>> > it would seem like spamlogd filter (for port 25) is picking up the
>> > original
>> > dport, not the rewritten one (with hdr->dport containing original port,
>> > too).
>> >
>> > Not sure of the correct solution, but one of the options is to look at
>> > the
>> > hdr->rewritten field, and only act if it is false. This might impact
>> > someone who does pf rewrites for sendmail itself, but at least it's not
>> > going to let all the spam in for someone who simply logs stuff up. A
>> > patch
>> > is attached.
>> >
>> > Cheers,
>> > Constantine.
>
>
> Index: spamlogd.c
> ===
> RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
> retrieving revision 1.21
> diff -u -d -p -8 -r1.21 spamlogd.c
> --- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
> +++ spamlogd.c 6 Mar 2013 21:14:51 -
> @@ -75,17 +75,17 @@ pcap_t *hpcap = NULL;
> struct syslog_data sdata = SYSLOG_DATA_INIT;
> time_t whiteexp = WHITEEXP;
> extern char*__progname;
>
> void logmsg(int , const char *, ...);
> void sighandler_close(int);
> intinit_pcap(void);
> void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char
> *);
> -intdbupdate(char *, char *);
> +intdbupdate(char *, char *, int);
> void usage(void);
>
> void
> logmsg(int pri, const char *msg, ...)
> {
> va_list ap;
> va_start(ap, msg);
>
> @@ -187,22 +187,22 @@ logpkt_handler(u_char *user, const struc
> sizeof(ipstraddr));
> }
>
> if (ipstraddr[0] != '\0') {
> if (hdr->dir == PF_IN)
> logmsg(LOG_DEBUG,"inbound %s", ipstraddr);
> else
> logmsg(LOG_DEBUG,"outbound %s", ipstraddr);
> - dbupdate(PATH_SPAMD_DB, ipstraddr);
> + dbupdate(PATH_SPAMD_DB, ipstraddr, hdr->dir == PF_IN);
> }
> }
>
> int
> -dbupdate(char *dbname, char *ip)
> +dbupdate(char *dbname, char *ip, int updateonly)
> {
> HASHINFOhashinfo;
> DBT dbk, dbd;
> DB *db;
> struct gdatagd;
> time_t now;
> int r;
> struct in_addr ia;
> @@ -227,16 +227,20 @@ dbupdate(char *dbname, char *ip)
> /* add or update whitelist entry */
> r = db->get(db, &dbk, &dbd, 0);
> if (r == -1) {
> logmsg(LOG_NOTICE, "db->get failed (%m)");
>
Re: spamlogd whitelists every logged rdr-to connection
Indeed, this is what I typically do. and make sure I only log the real mailserver connections to pflog1, and point spamlogd at that. On Wed, Mar 6, 2013 at 2:14 PM, Stuart Henderson wrote: > On 2013/03/06 13:47, Bob Beck wrote: >> No constantine - the solution is to simply not use the "log" keyword >> on such traffic > > Or you can use an alternative log interface > > log (to ) > Send logs to the specified pflog(4) interface instead of pflog0. >
Re: spamlogd whitelists every logged rdr-to connection
No constatine - that is not the best approach. if you are whitelisting
grelisted connections
or blacklisted connections that are blocked you have your pf.conf or
spamlogd setup wrong.
On Wed, Mar 6, 2013 at 2:54 PM, Constantine A. Murenin wrote:
> Bob, I agree, the hdr->rewritten approach is not good.
>
> I think the best approach here would be to not add any new entries on
> incoming connections in the first place, but only keep updating the existing
> ones (when the connection is incoming).
>
> In addition to not whitelisting greylisted or blocked connection that are
> logged, this would also prevent the case of double-whitelisting the
> connections that are logged and whitelisted through other rules, without any
> adverse side effects or unexpected behaviour.
>
> Patch attached inline.
>
> C.
>
>
> On 2013-W10-3 13:47 -0700, Bob Beck wrote:
>>
>> No constantine - the solution is to simply not use the "log" keyword
>> on such traffic
>>
>> All of my boxen I run this on also rewite the traffic to (pool) of
>> mailservers so this is
>> not accurate.
>>
>> Simply don't log the traffic you don't want spamlogd to see. the
>> *point* of spamlogd
>> is to ensure all continuing valid connections *stay* whitelisted.
>>
>> On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin wrote:
>> > Hi,
>> >
>> > I've started using spamlogd, and since then, every single connection
>> > attempt
>> > results in the host being whitelisted.
>> >
>> > I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog,
>> > and
>> > it would seem like spamlogd filter (for port 25) is picking up the
>> > original
>> > dport, not the rewritten one (with hdr->dport containing original port,
>> > too).
>> >
>> > Not sure of the correct solution, but one of the options is to look at
>> > the
>> > hdr->rewritten field, and only act if it is false. This might impact
>> > someone who does pf rewrites for sendmail itself, but at least it's not
>> > going to let all the spam in for someone who simply logs stuff up. A
>> > patch
>> > is attached.
>> >
>> > Cheers,
>> > Constantine.
>
>
> Index: spamlogd.c
> ===
> RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
> retrieving revision 1.21
> diff -u -d -p -8 -r1.21 spamlogd.c
> --- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
> +++ spamlogd.c 6 Mar 2013 21:14:51 -
> @@ -75,17 +75,17 @@ pcap_t *hpcap = NULL;
> struct syslog_data sdata = SYSLOG_DATA_INIT;
> time_t whiteexp = WHITEEXP;
> extern char*__progname;
>
> void logmsg(int , const char *, ...);
> void sighandler_close(int);
> intinit_pcap(void);
> void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char
> *);
> -intdbupdate(char *, char *);
> +intdbupdate(char *, char *, int);
> void usage(void);
>
> void
> logmsg(int pri, const char *msg, ...)
> {
> va_list ap;
> va_start(ap, msg);
>
> @@ -187,22 +187,22 @@ logpkt_handler(u_char *user, const struc
> sizeof(ipstraddr));
> }
>
> if (ipstraddr[0] != '\0') {
> if (hdr->dir == PF_IN)
> logmsg(LOG_DEBUG,"inbound %s", ipstraddr);
> else
> logmsg(LOG_DEBUG,"outbound %s", ipstraddr);
> - dbupdate(PATH_SPAMD_DB, ipstraddr);
> + dbupdate(PATH_SPAMD_DB, ipstraddr, hdr->dir == PF_IN);
> }
> }
>
> int
> -dbupdate(char *dbname, char *ip)
> +dbupdate(char *dbname, char *ip, int updateonly)
> {
> HASHINFOhashinfo;
> DBT dbk, dbd;
> DB *db;
> struct gdatagd;
> time_t now;
> int r;
> struct in_addr ia;
> @@ -227,16 +227,20 @@ dbupdate(char *dbname, char *ip)
> /* add or update whitelist entry */
> r = db->get(db, &dbk, &dbd, 0);
> if (r == -1) {
> logmsg(LOG_NOTICE, "db->get failed (%m)");
> goto bad;
> }
>
> if (r) {
> + if (updateonly) {
> + logmsg(LOG_DEBUG,"ignoring %s", ip);
> + goto bad;
> + }
> /* new entry */
> memset(&gd, 0, sizeof(gd));
> gd.first = now;
> gd.bcount = 1;
> gd.pass = now;
> gd.expire = now + whiteexp;
> memset(&dbk, 0, sizeof(dbk));
> dbk.size = strlen(ip);
>
Re: spamlogd whitelists every logged rdr-to connection
Bob, I agree, the hdr->rewritten approach is not good.
I think the best approach here would be to not add any new entries
on incoming connections in the first place, but only keep updating
the existing ones (when the connection is incoming).
In addition to not whitelisting greylisted or blocked connection that
are logged, this would also prevent the case of double-whitelisting
the connections that are logged and whitelisted through other rules,
without any adverse side effects or unexpected behaviour.
Patch attached inline.
C.
On 2013-W10-3 13:47 -0700, Bob Beck wrote:
No constantine - the solution is to simply not use the "log" keyword
on such traffic
All of my boxen I run this on also rewite the traffic to (pool) of
mailservers so this is
not accurate.
Simply don't log the traffic you don't want spamlogd to see. the
*point* of spamlogd
is to ensure all continuing valid connections *stay* whitelisted.
On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin wrote:
> Hi,
>
> I've started using spamlogd, and since then, every single connection attempt
> results in the host being whitelisted.
>
> I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog, and
> it would seem like spamlogd filter (for port 25) is picking up the original
> dport, not the rewritten one (with hdr->dport containing original port,
> too).
>
> Not sure of the correct solution, but one of the options is to look at the
> hdr->rewritten field, and only act if it is false. This might impact
> someone who does pf rewrites for sendmail itself, but at least it's not
> going to let all the spam in for someone who simply logs stuff up. A patch
> is attached.
>
> Cheers,
> Constantine.
Index: spamlogd.c
===
RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
retrieving revision 1.21
diff -u -d -p -8 -r1.21 spamlogd.c
--- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
+++ spamlogd.c 6 Mar 2013 21:14:51 -
@@ -75,17 +75,17 @@ pcap_t *hpcap = NULL;
struct syslog_data sdata = SYSLOG_DATA_INIT;
time_t whiteexp = WHITEEXP;
extern char*__progname;
void logmsg(int , const char *, ...);
void sighandler_close(int);
intinit_pcap(void);
void logpkt_handler(u_char *, const struct pcap_pkthdr *, const u_char *);
-intdbupdate(char *, char *);
+intdbupdate(char *, char *, int);
void usage(void);
void
logmsg(int pri, const char *msg, ...)
{
va_list ap;
va_start(ap, msg);
@@ -187,22 +187,22 @@ logpkt_handler(u_char *user, const struc
sizeof(ipstraddr));
}
if (ipstraddr[0] != '\0') {
if (hdr->dir == PF_IN)
logmsg(LOG_DEBUG,"inbound %s", ipstraddr);
else
logmsg(LOG_DEBUG,"outbound %s", ipstraddr);
- dbupdate(PATH_SPAMD_DB, ipstraddr);
+ dbupdate(PATH_SPAMD_DB, ipstraddr, hdr->dir == PF_IN);
}
}
int
-dbupdate(char *dbname, char *ip)
+dbupdate(char *dbname, char *ip, int updateonly)
{
HASHINFOhashinfo;
DBT dbk, dbd;
DB *db;
struct gdatagd;
time_t now;
int r;
struct in_addr ia;
@@ -227,16 +227,20 @@ dbupdate(char *dbname, char *ip)
/* add or update whitelist entry */
r = db->get(db, &dbk, &dbd, 0);
if (r == -1) {
logmsg(LOG_NOTICE, "db->get failed (%m)");
goto bad;
}
if (r) {
+ if (updateonly) {
+ logmsg(LOG_DEBUG,"ignoring %s", ip);
+ goto bad;
+ }
/* new entry */
memset(&gd, 0, sizeof(gd));
gd.first = now;
gd.bcount = 1;
gd.pass = now;
gd.expire = now + whiteexp;
memset(&dbk, 0, sizeof(dbk));
dbk.size = strlen(ip);
Re: spamlogd whitelists every logged rdr-to connection
On 2013/03/06 13:47, Bob Beck wrote: > No constantine - the solution is to simply not use the "log" keyword > on such traffic Or you can use an alternative log interface log (to ) Send logs to the specified pflog(4) interface instead of pflog0.
Re: spamlogd whitelists every logged rdr-to connection
No constantine - the solution is to simply not use the "log" keyword
on such traffic
All of my boxen I run this on also rewite the traffic to (pool) of
mailservers so this is
not accurate.
Simply don't log the traffic you don't want spamlogd to see. the
*point* of spamlogd
is to ensure all continuing valid connections *stay* whitelisted.
On Wed, Mar 6, 2013 at 1:08 PM, Constantine A. Murenin wrote:
> Hi,
>
> I've started using spamlogd, and since then, every single connection attempt
> results in the host being whitelisted.
>
> I log some `rdr-to 127.0.0.1 port spamd` connection attempts into pflog, and
> it would seem like spamlogd filter (for port 25) is picking up the original
> dport, not the rewritten one (with hdr->dport containing original port,
> too).
>
> Not sure of the correct solution, but one of the options is to look at the
> hdr->rewritten field, and only act if it is false. This might impact
> someone who does pf rewrites for sendmail itself, but at least it's not
> going to let all the spam in for someone who simply logs stuff up. A patch
> is attached.
>
> Cheers,
> Constantine.
>
> Cns# tail /var/log/spamd
> Mar 6 08:12:53 Cns spamlogd[1082]: inbound 74.122.155.17
> Mar 6 08:50:27 Cns spamd[5220]: 46.53.132.165: connected (1/0)
> Mar 6 08:50:27 Cns spamlogd[1082]: inbound 46.53.132.165
> Mar 6 08:50:30 Cns spamd[5220]: 46.53.132.165: disconnected after 3
> seconds.
> Mar 6 08:51:37 Cns spamd[5220]: 178.127.228.161: connected (1/0)
> Mar 6 08:51:37 Cns spamlogd[1082]: inbound 178.127.228.161
> Mar 6 08:51:40 Cns spamd[5220]: 178.127.228.161: disconnected after 3
> seconds.
> Mar 6 09:21:54 Cns spamlogd[1082]: inbound 46.241.252.81
> Mar 6 09:21:55 Cns spamd[5220]: 46.241.252.81: connected (1/0)
> Mar 6 09:21:58 Cns spamd[5220]: 46.241.252.81: disconnected after 3
> seconds.
>
> Cns# fgrep 46.241.252.81 /var/log/spamd
> Mar 6 09:21:54 Cns spamlogd[1082]: inbound 46.241.252.81
> Mar 6 09:21:55 Cns spamd[5220]: 46.241.252.81: connected (1/0)
> Mar 6 09:21:58 Cns spamd[5220]: 46.241.252.81: disconnected after 3
> seconds.
>
> Cns# tcpdump -o -n -e -ttt -r /var/log/pflog host 46.241.252.81 | tail
> tcpdump: WARNING: snaplen raised from 116 to 160
> Mar 06 09:21:54.834606 rule 43/(match) pass in on re0: 46.241.252.81.3748 >
> 127.0.0.1.8025: S (src OS: Windows 2000 RFC1323, Windows XP RFC1323)
> 4277363076:4277363076(0) win 65535
> (DF)
>
> Cns# fgrep 46.241.252.81 /var/log/maillog
>
> Cns# spamdb | fgrep 46.241.252.81
> WHITE|46.241.252.81|||1362590514|1362590514|1365700914|1|0
>
> Cns# date -r 1362590514
> Wed Mar 6 09:21:54 PST 2013
>
> Cns# uname -rms
> OpenBSD 5.2 amd64
>
> (Logs were rorated several days ago, so, what you see is what's there.)
>
> Index: spamlogd.c
> ===
> RCS file: /cvs/OpenBSD-CVS/src/libexec/spamlogd/spamlogd.c,v
> retrieving revision 1.21
> diff -u -d -p -8 -r1.21 spamlogd.c
> --- spamlogd.c 18 Mar 2011 22:37:06 - 1.21
> +++ spamlogd.c 6 Mar 2013 19:44:32 -
> @@ -174,20 +174,22 @@ logpkt_handler(u_char *user, const struc
>
> /* We're interested in passed packets */
> if (hdr->action != PF_PASS)
> return;
>
> af = hdr->af;
> if (af == AF_INET) {
> ip = (const struct ip *)(sp + hdrlen);
> - if (hdr->dir == PF_IN)
> + if (hdr->dir == PF_IN) {
> + if (hdr->rewritten == 1)
> + return;
> inet_ntop(af, &ip->ip_src, ipstraddr,
> sizeof(ipstraddr));
> - else if (hdr->dir == PF_OUT && !flag_inbound)
> + } else if (hdr->dir == PF_OUT && !flag_inbound)
> inet_ntop(af, &ip->ip_dst, ipstraddr,
> sizeof(ipstraddr));
> }
>
> if (ipstraddr[0] != '\0') {
> if (hdr->dir == PF_IN)
> logmsg(LOG_DEBUG,"inbound %s", ipstraddr);
> else
>
