Bug#383870: fetchmail: Fetchmail duplicates my SPAM every 10 minutes - ARGH!

2006-09-05 Thread Helge Hafting 

Matthias Andree wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Helge Hafting wrote:

  

Still, rejecting on TO address is tricky, mailing list mail is usually not
addressed to me.



Unfortunately, you haven't shown your configuration file yet.
I suspect you're using a wildcarded multidrop configuration.
  

Here is my /etc/fetchmailrc:

#Old mail server
poll popn.c2i.net protocol POP3
user  there has password , is 
[EMAIL PROTECTED] here


#current ISP server
poll mail.broadpark.no protocol POP3
user  there has password , is 
[EMAIL PROTECTED] here




Perhaps this setup isn't the smartest? 
Still, it'd be nice if fetchmail would delete successfully delivered 
messages

when exim goes sour. That would at least avoid message multiplication,
although I would probably not get any mail through if all the x first
messages are bad.  That'd be a exim problem though, not a fetchmail fault.

Looks like I solved the problem by making exim accept 100 bad messages
instead of just 4 before giving up.  Still a DOS waiting to happen.

Helge Hafting


--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Bug#383870: fetchmail: Fetchmail duplicates my SPAM every 10 minutes - ARGH!

2006-08-23 Thread Helge Hafting 

Matthias Andree wrote:

(demoting to serious, tagging upstream, retitling)

Additional info:

fetchmail 6.3.4 and older do NOT log out of the upstream server after
SIGPIPE (I don't know the rationale behind this - it's not in the
comments), which prevents the deletions from happening properly.
Retitling the bug accordingly.
  

Thank you for the response.  I got around the problem temporarily by
reconfiguring exim to accept many more malformed adresses
in a session before hanging up.  (I now allow 100 instead of the
default 3).

I agree that grave was exaggregating, given that it was solveable
with a config change.  I was a bit desperate at the moment, worrying that
I might be unable to download email at all for some time.

There is no need to provide a special package just for me, as I
worked around it by relaxing exim.  Please use the time to
work on a fix instead.  My suggestion is to have fetchmail disconnect in
an orderly way, so that the messages actually downloaded are deleted.

This would solve the problem - I would get first 115 messages, then
a hangup.  After 10min, I'd get some more messages until exim trips up,
and after a few more rounds I'd get all my mail. 


Thanks for all the tips on workarounds and configuration, there is
clearly a lot I could do to avoid downloading that spam in the first place.
Still, rejecting on TO address is tricky, mailing list mail is usually not
addressed to me.

Helge Hafting





--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Bug#383870: fetchmail: Fetchmail duplicates my SPAM every 10 minutes - ARGH!

2006-08-23 Thread Matthias Andree 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Helge Hafting wrote:

 Still, rejecting on TO address is tricky, mailing list mail is usually not
 addressed to me.

Unfortunately, you haven't shown your configuration file yet.
I suspect you're using a wildcarded multidrop configuration.

- --
Matthias Andree
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFE7PeCvmGDOQUufZURAs01AJ9Ax9kCN59Q4rubF11/FMz6qhcb4wCgunuy
3xWJnpbHy+CPt6h7Q7ookc0=
=hZJE
-END PGP SIGNATURE-


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Bug#383870: fetchmail: Fetchmail duplicates my SPAM every 10 minutes - ARGH!

2006-08-20 Thread Helge Hafting 
Package: fetchmail
Version: 6.3.4-4
Severity: grave
Justification: renders package unusable

I have been using fetchmail for many years - it runs every 10 minutes
fetching from a few accounts.  The mail is then fed into a local
exim, because I used to have several users on this machine.

Yesterday, I discovered a hundred-doubling of my spam folder,
which was surprising.  Spam tend to increase, but not so abruptly.

Looking at the spam, the same messages repeated over and over.
Completely identical headers, even the message id. At first   
I thought someone had made new spam software with some sort   
of fault in it, but the same messages kept coming back
after I deleted them.

Then I had a look at what fetchmail was doing:


fetchmail -c showed  893 messages waiting for me at the ISP.
Fetchmail started collecting and delivering,
then one of the messages got an error
response because of a malformed address
with spaces in it.  Example:

reading message [EMAIL PROTECTED]:76 of 893 (1523 octets)
..fetchmail: SMTP error: 501 Denver [EMAIL PROTECTED]: @ or .
expected after Denver

There were many of these in addition to normal mail/spam, and after a while
I got:

reading message [EMAIL PROTECTED]:115 of 893 (1672 octets)
fetchmail: SMTP error: 501 Too many syntax or protocol errors
.. flushed
reading message [EMAIL PROTECTED]:116 of 893 (1800 octets)
..fetchmail: SIGPIPE thrown from an MDA or a stream socket error
fetchmail: socket error while fetching from [EMAIL PROTECTED]
fetchmail: Query status=2 (SOCKET)


Fetchmail gave up.  Fine, I tried a
fetchmail -c
and noticed - still 893 messages left!  But I now have many of
them in my spambox.  Every 10 minutes, fetchmail get me 115   
copies of the _same_ spam, then gives up.


Now, I understand that fetchmail don't want to destroy
the faulty messages - there could be something important.

But it _really_ should remove from the server all those
messages that were delivered successfully!  I should not get
the same (successful) spam messages delivered over and over and over!

I have never seen anything like this before, and it is very annoying.
Now, I can set up a client to access that ISP account directly, and  
delete messages on the server.  But that won't help, for surely
I will get more spam with spaces in the TO: field
that will trip up fetchmail again and again.

Is there a solution for this, or will there
be one soon? I'll try other versions of fetchmail
available from debian, of course.

As it is - fetchmail is unuseable because it both fail to fetch 90%
of my mail, _and_ it fills up the disk slowly threatening the system
with a full disk.  Incoming mail service is denied.


Feel free to yell if I am running fetchmail in a stupid way -
there isn't much thought in this setup, but it worked flawlessly
for many years now.

-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (900, 'testing'), (800, 'unstable'), (700, 'experimental')
Architecture: amd64 (x86_64)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-rc4-mm1
Locale: LANG=nb_NO.UTF-8, LC_CTYPE=nb_NO.UTF-8 (charmap=UTF-8)

Versions of packages fetchmail depends on:
ii  adduser   3.96   Add and remove users and groups
ii  debianutils   2.17   Miscellaneous utilities specific t
ii  gettext   0.14.6-1   GNU Internationalization utilities
ii  libc6 2.3.6-15   GNU C Library: Shared libraries
ii  libssl0.9.8   0.9.8b-2   SSL shared libraries

Versions of packages fetchmail recommends:
ii  ca-certificates   20050804   Common CA Certificates PEM files

-- debconf-show failed


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Bug#383870: fetchmail: Fetchmail duplicates my SPAM every 10 minutes - ARGH!

2006-08-20 Thread Matthias Andree 
Well, Exim's error before it just drops the connection isn't selling
fetchmail a hint that there's no progress beyond that point.

The question is: why does fetchmail not skip over the offending
messages? It may or may not try to bounce them, depending on your
configuration, but should mark them refused - as seen in ..flushed.

But what exactly happens is a matter of your configuration.

Please show your fetchmailrc file and the output of 
/etc/init.d/fetchmail debug-run
up to the error.

Remember to restart fetchmail afterwards with /etc/init.d/fetchmail start

-- 
Matthias Andree


-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]

Bug#383870: fetchmail: Fetchmail duplicates my SPAM every 10 minutes - ARGH!

2006-08-20 Thread Matthias Andree 
(demoting to serious, tagging upstream, retitling)

Additional info:

fetchmail 6.3.4 and older do NOT log out of the upstream server after
SIGPIPE (I don't know the rationale behind this - it's not in the
comments), which prevents the deletions from happening properly.
Retitling the bug accordingly.

Workarounds:

- have your upstream reject spam with improperly formatted envelopes,
  these should be rejected in the first SMTP dialogue

- do not use multidrop with wildcard (*), but list allowed local
  accounts explicitly (since the bogus address will not match, the
  message will be forwarded to the postmaster address, which is under
  your control, and can also be empty to discard the message - this will
  prevent Exim from hanging up).

- set expunge 10  (fetchmail will send QUIT after 10 messages and
   reconnect, to make deletions effective and limit
   the amount of refetching).

Fix: block SIGPIPE; the SIGPIPE handler was wrong from the moment it was
coded as it uses longjmp() which isn't safe in signal handlers.
Functions that write to broken pipes will still report errors, but
fetchmail won't corrupt internal state or defeat the logout.

Patch attached. BEWARE: it is NOT run-time tested,
I'm not fetching from multidrop accounts.

Nico/Héctor, can we provide an inofficial package to Helge with the
patch below to ask feedback if the fix is complete? It shouldn't matter
if it's i386 rather than x86_64.

Rationale for severity demotion: Since the breakage is partially induced
by configuration and for instance works properly for singledrop, I don't
think grave is justified. Perhaps further demotion to important is
in order, but I'm not the Debian maintainer to remove the unsuitable
for release tag that is attached to critical/grave/serious severities.

-- 
Matthias Andree
--- ./fetchmail.c.orig	2006-08-21 01:39:54.0 +0200
+++ ./fetchmail.c	2006-08-21 01:37:47.0 +0200
@@ -577,7 +577,7 @@
 set_signal_handler(SIGINT, terminate_run);
 set_signal_handler(SIGTERM, terminate_run);
 set_signal_handler(SIGALRM, terminate_run);
-set_signal_handler(SIGPIPE, terminate_run);
+set_signal_handler(SIGPIPE, SIG_IGN);
 set_signal_handler(SIGQUIT, terminate_run);
 
 /* here's the exclusion lock */
--- ./driver.c.orig	2006-08-21 01:37:09.0 +0200
+++ ./driver.c	2006-08-21 01:40:41.0 +0200
@@ -55,7 +55,6 @@
 
 /* throw types for runtime errors */
 #define THROW_TIMEOUT	1		/* server timed out */
-#define THROW_SIGPIPE	2		/* SIGPIPE on stream socket */
 
 /* magic values for the message length array */
 #define MSGLEN_UNKNOWN	0		/* length unknown (0 is impossible) */
@@ -113,13 +112,6 @@
 	idletimeout = 1;
 }
 
-static RETSIGTYPE sigpipe_handler (int signal)
-/* handle SIGPIPE signal indicating a broken stream socket */
-{
-(void)signal;
-longjmp(restart, THROW_SIGPIPE);
-}
-
 #define CLEANUP_TIMEOUT 60 /* maximum timeout during cleanup */
 
 static int cleanupSockClose (int fd)
@@ -856,9 +848,6 @@
 alrmsave = set_signal_handler(SIGALRM, timeout_handler);
 mytimeout = ctl-server.timeout;
 
-/* set up the broken-pipe timeout */
-pipesave = set_signal_handler(SIGPIPE, sigpipe_handler);
-
 if ((js = setjmp(restart)))
 {
 	/* exception caught */
@@ -878,14 +867,7 @@
 	sigprocmask(SIG_UNBLOCK, allsigs, NULL);
 #endif /* HAVE_SIGPROCMASK */
 	
-	if (js == THROW_SIGPIPE)
-	{
-	set_signal_handler(SIGPIPE, SIG_IGN);
-	report(stdout,
-		   GT_(SIGPIPE thrown from an MDA or a stream socket error\n));
-	wait(0);
-	}
-	else if (js == THROW_TIMEOUT)
+	if (js == THROW_TIMEOUT)
 	{
 	if (phase == OPEN_WAIT)
 		report(stdout,
@@ -1581,7 +1563,6 @@
 
 set_timeout(0); /* cancel any pending alarm */
 set_signal_handler(SIGALRM, alrmsave);
-set_signal_handler(SIGPIPE, pipesave);
 return(err);
 }