Re: Advice sought on how to convince irresponsible Megapath ISP.
On 8/16/2014 12:08 AM, Linda A. Walsh wrote: The start of filtering out spam is filtering out mail that doesn't have a valid return address -- it's not valid email. I'm stuck because my ISP won't filter it out, I have to download it to find out that it's invalid, then local sendmail rejects it, so it stays in their box. As others have said already, turn off that sendmail feature. It is not useful (as you have discovered) if the mail is being fetched from another MX rather than being delivered directly. Since your ISP's MX has accepted everything, you have no choice but to download it all. My suggestion would be to use SA to score the messages (if you are not already doing so), and then deliver the spam to a spam folder. Once you are confident in SA's filtering, you can set another rule in your delivery agent to delete high-scoring spam. The best solution would be to run your own server rather than relying on your ISP, but the above is the best option if you don't want to do that. -- Bowie
Re: Advice sought on how to convince irresponsible Megapath ISP.
On Fri, 15 Aug 2014 19:06:21 -0700 Linda A. Walsh sa-u...@tlinx.org wrote: My old email service was bought out by Megapath who is letting alot of services slide. [snip] Any ideas on how to get a cheapo-doesn't want to support anything ISP to start blocking all the garbage the pass on? This seems like something more MTA related. Are you using postfix? If so, then the postfix list may be more helpful. The fetchmail list might also help, though I haven't seen this kind of query there. On some anti-spam lists there were debates about spam blocking and the consensus seemed to be that ISP's could get themselves into big trouble if they started blocking everything they considered spam even if the source was obviously fake. More trouble than if they simply let everything pass. A lot of legal trouble and too much work to be worth it. I had once set up fetchmail and postfix to do pretty much the same thing with a before-queue check. It did not bounce mail but it did have fetchmail reject the message on various header and sender checks, then tell the imap server to delete the message. I suspect you'd have better control over your mail with something like a VPS. They seem to be fairly cheap compared to just 2 years ago. As well as having total control, you needn't be chained to one solution and you'd never have to pay a spam filtering service like postini for the privilege of teaching it what spam is. jd
Re: Advice sought on how to convince irresponsible Megapath ISP.
Karsten Br�� wrote: Similarly, your scripts do not reject messages, but choose not to fetch them. === No... fetchmail fetches them, sendmail rejects them because they don't have a resolvable domain. My sorting and spamassassin scripts get called after the email makes it through sendmail. My scripts don't see the email. Pragmatic solution: If you insist on your scripts to not fetch those spam messages (which have been accepted by the MX, mind you), automate the manual download and delete stage, which frankly only exists due to your choice of not downloading them in the first place. Make your scripts delete, instead of skipping over them. 'fetchmail', that I know of, isn't able to tell if a sending domain is invalid until it has fetched the email (that I know of). fetchmail tries to send the email to me via sendmail, which doesn't accept the email because it is invalid. Unfortunately, my ISP doesn't use sendmail or it would reject such emails by default. Be liberal in what you accept, strict in what you send. In particular, later stages simply must not be less liberal than early stages. In this case, I don't even want the invalid email passed on to me. I don't want to accept spam. The first defense is to have the MX reject non-conforming email. Your MX has accepted the message. My ISP's MX has accepted it, because it doesn't do domain checking. My machine's MX rejects it so fetchmail keeps trying to deliver it. While I *could* figure out how to hack sendmail to not reject the message, my preference would be to get the ISP to act responsibly and reject emails without a valid return domainname. It's standard in sendmail, rejection of such email is called for in the RFC's. The choice to not follow RFC's allows spam that would normally be rejected, through to my system which does follow the standards and rejects it -- so it stays in the download queue for my domain. At that point, there is absolutely no way to not accept, reject it later. You can classify, which you use SA for (I guess, given you posting here). You can filter or even delete based on classification, or other criteria. The MX shouldn't accept it based on RFC standards. When I asked for it to be blocked, I was first asked for the name of the offending domain and told I could blacklist the domain by adding it to a list with their web-client. I asked for a scriptable way to do this after a domain lookup, they said they no longer offer scripted solutions as the ISP I signed up with (who they bought) did. The only response my ISP will give is to turn on their spam filtering. I tried that. In about a 2 hour time frame, over 400 messages were blocked as spam. Of those less than 10 were actually spam, the rest were from various lists. So having them censoring my incoming mail isn't gonna work, but neither will the reject the obvious invalid domain email. I can't believe that they insist on forwarding SPAM to their users even though they know it is invalid and is spam. There is no censoring. When I complained about the problem I found that recommended filter rules had been activated on my account. In the couple of days they were active about 80% of the messages they caught were not spam -- and some of the bad domains still got passed through. There is no forwarding. It comes in their MX, and is forwarded to their users. Any ideas on how to get a cheapo-doesn't want to support anything ISP to start blocking all the garbage the pass on? Change ISP. You decided for them to run your MX. I didn't decide for them, I inherited them when they bought out the competition to supply lower quality service for the same price. It is your choice to aim for a cheapo service (your words). It wasn't when I signed up. Cost $100 extra/month. Now only $30 extra/month that I don't host the domain with them. If you're unhappy with the service, take your business elsewhere. Better service doesn't necessarily mean more expensive, but you might need to shell out a few bucks for the service you want. I already am... my ISP (cable company) doesn't have the services I want for mail hosting. I went to another company for that, who was bought out some times ago, with the new company dropping quality as time goes on. In this case, I wanted to try to push back against them accepting the illegal (not to spec) spam and forwarding it to their customers in the first place. There are many compromised solutions that are available. Certainly such choices are not my first, which was why I posted here to see if anyone else had any experience with getting an irresponsible ISP to reject non-compliant email, and barring that, maybe getting offered better choices from the experience of the people on this list. Cheers! '^/
Re: Advice sought on how to convince irresponsible Megapath ISP.
On Sunday 17 August 2014 at 16:37:36 (EU time), Linda Walsh wrote: No... fetchmail fetches them, sendmail rejects them because they don't have a resolvable domain. My sorting and spamassassin scripts get called after the email makes it through sendmail. My scripts don't see the email. My ISP's MX has accepted it, because it doesn't do domain checking. My machine's MX rejects it so fetchmail keeps trying to deliver it. It sounds to me as though you are perfectly capable (and willing) to run an MX server of your own, so why not just register your own domain, and have the MX for it pointed at your own machine, where you can apply whatever filter/reject rules you like, with complete freedom? You might be able to get a combined hosting provider / registrar to act as backup MX for you for a reasonable price too (it's never good to have only a single MX). It wasn't when I signed up. Cost $100 extra/month. Now only $30 extra/month that I don't host the domain with them. You can get a hosted root-server VM with a static IP for less than $30/month. Your own machine, your own choice of software, your own rules, and cheaper too. Regards, Antony. -- Linux is going to be part of the future. It's going to be like Unix was. - Peter Moore, Asia-Pacific general manager, Microsoft Please reply to the list; please *don't* CC me.
Re: Advice sought on how to convince irresponsible Megapath ISP.
On 8/15/2014 9:08 PM, Linda A. Walsh wrote: I was wondering if anyone had any experience in showing their ISP the light... Oh well Linda, is your email domain tlinx.org? I'm assuming that it is because there is an under construction web page on that domain and I cannot imagine a commercial ISP putting an under construction web page on a domain they own. Anyway, getting back to the technical aspects of your problem, I'm gonna assume that for whatever reason (maybe your out in the boonies and are still on a dialup) that you cannot simply setup a Linux box behind a broadband connection and use some provider like dydns.org (or whatever they are calling themselves now) and accept your own email. And because of that your forced to use a commercial mailserver then fetch mail from that. If that is the case, I am suspecting that you are PERHAPS confusing what is called the envelope address with the header address in the incoming email. The envelope address is passed during the SMTP handshake. That is almost certainly being checked for validity by your ISP and it is passing. The header address is supplied during the DATA phase and as such it is part of the mail message content. In general, you cannot filter on this during the SMTP handshake since it isn't supplied until the mailserver already has agreed to accept the message. Instead it must be filtered during the content filtering phase, after the mail has been accepted. I do understand that from a user's POV it is very confusing to see a message in your inbox that says From: fakeaddress@fakedomain but if you actually looked at the full header of the email, you would see the REAL and legitimate from address (assuming your ISP has turned on the feature that displays the envelope address used by the SMTP sender in the header) For example here is the header fragment of your post that is in MY mailbox: Return-Path: users-return-104656-tedm=ipinc@spamassassin.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mail.ipinc.net (8.14.7/8.14.7) with SMTP id s7G49cin016427 for t...@ipinc.net; Fri, 15 Aug 2014 21:09:38 -0700 (PDT) (envelope-from users-return-104656-tedm=ipinc@spamassassin.apache.org) Received: (qmail 23944 invoked by uid 500); 16 Aug 2014 04:09:29 - Mailing-List: contact users-h...@spamassassin.apache.org; run by ezmlm Precedence: bulk list-help: mailto:users-h...@spamassassin.apache.org list-unsubscribe: mailto:users-unsubscr...@spamassassin.apache.org List-Post: mailto:users@spamassassin.apache.org List-Id: users.spamassassin.apache.org Delivered-To: mailing list users@spamassassin.apache.org . . . Received-SPF: unknown ipv4:173.164.175.65 (athena.apache.org: encountered unrecognized mechanism during SPF processing of domain of sa-u...@tlinx.org) Received: from [173.164.175.65] (HELO Ishtar.hs.tlinx.org) (173.164.175.65) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 16 Aug 2014 04:09:23 + Received: from [192.168.4.12] (athenae [192.168.4.12]) by Ishtar.hs.tlinx.org (8.14.7/8.14.4/SuSE Linux 0.8) with ESMTP id s7G48ufp053787 for users@spamassassin.apache.org; Fri, 15 Aug 2014 21:09:00 -0700 Message-ID: 53eed954.50...@tlinx.org Date: Fri, 15 Aug 2014 21:08:52 -0700 From: Linda A. Walsh sa-u...@tlinx.org User-Agent: Thunderbird Please notice something: Envelope sender address: users-return-104656-tedm=ipinc@spamassassin.apache.org Header sender address: Linda A. Walsh sa-u...@tlinx.org The mail message was accepted because the envelope-from address of users-return-blah blah blah was valid, not because the sa-u...@tlinx.org address was valid. iN the typical spam message, the envelope sender is valid, the header sender is not. Spammers do it this way because of sender-ID, domainkeys, and software like milter-callback - all of which checks the envelope email address for validity. They do not want emails sent back to their real address (that is used in the envelope) which was checked for validity, they want emails sent back to the fake address (so the emails don't actually reach them) I hope that explains to you why you erroneously think your ISP is accepting fake emails. In other words, to put it politely, you don't know what your talking about and need education that I have just supplied. Assuming now your properly humbled, and still reading, the good news about all of this is that assuming your ISP is NOT doing ANY KIND of content filtering, that you can post-process your incoming email using SpamAssassin on your own system, and get the same results you would get if your ISP was running SpamAssassin on the mailserver. SA looks at the header address and takes into account whether the From: address is bogus or not. You haven't posted what operating system your using to read mail with or any of that, so I cannot advise you how you would go about running SpamAssassin on whatever your using to script all
Re: Advice sought on how to convince irresponsible Megapath ISP.
On 8/17/2014 7:37 AM, Linda Walsh wrote: Karsten Br�� wrote: Similarly, your scripts do not reject messages, but choose not to fetch them. === No... fetchmail fetches them, sendmail rejects them because they don't have a resolvable domain. My sorting and spamassassin scripts get called after the email makes it through sendmail. My scripts don't see the email. Pragmatic solution: If you insist on your scripts to not fetch those spam messages (which have been accepted by the MX, mind you), automate the manual download and delete stage, which frankly only exists due to your choice of not downloading them in the first place. Make your scripts delete, instead of skipping over them. 'fetchmail', that I know of, isn't able to tell if a sending domain is invalid until it has fetched the email (that I know of). Fetchmail has no other way of doing it, how would it be able to determine if a sending domain is valid or not until it has read in the email message? Fetchmail ATTEMPTS to determine the original envelope senders address when it's processing mail, but can't always do it. fetchmail tries to send the email to me via sendmail, which doesn't accept the email because it is invalid. Unfortunately, my ISP doesn't use sendmail or it would reject such emails by default. No wonder you are frustrated. You are like the person who learned to play gold by watching TV - you have so many misconceptions and wrong ideas of how to play that it would literally take LONGER to teach you to unlearn all of this and learn the proper way to do it, than to start with someone who had never played golf. Your going to need to re-learn much of what you think you know before you can get what you want. Sorry to be harsh but you have been basically lied to for so long - probably by your former ISP - that now your going to have to go through a lot of pain to get rid of all of those lies. email simply does not work the way you think it does. For starters, anyone configuring Fetchmail to pass mail to Sendmail needs to disable all SMTP checks in Sendmail. Why? Because they are utterly useless. The purpose of these SMTP checks for valid domains and suchlike is that when Mr. Spammer opens an SMTP connection to you to spam, if you can detect that it's SPAM before the SMTP transaction is accepted, then you can deny receipt - and if the spam is relay spam from a broken-into mailserver that the spammer has hijacked, those bounced mails will clog up that mailserver, informing the server admin there's a problem and getting the open relay shut down. If the deny is directly back to the spammers server then the spammers server is slowed down, and the spammer has an incentive to remove your email address from his list. Once the mail is successfully delivered by the spammer to the server that fetchmail is pulling mail from, then none of this can happen - you have effectively told the spammer yes you have a valid victim address You need to setup your sendmail so that it accepts everything fetchmail gives it. You can then post-process with SpamAssassin to filter for spam. And that is ALL you can do to block spam. The ENTIRE focus on spamfiltering at the SMTP transaction level is to prevent the acceptance of SPAM before the SMTP transaction phase is completed. But, in order to content filter - to see what is actually inside the mail message - you MUST complete the SMTP transaction phase. So it is like Internet dating. You can view the person's profile all you want, they can have the best looking profile, the prettiest picture, the best reviews - but until the clothes come off, you don't know what you have. Be liberal in what you accept, strict in what you send. In particular, later stages simply must not be less liberal than early stages. In this case, I don't even want the invalid email passed on to me. I don't want to accept spam. The first defense is to have the MX reject non-conforming email. Your MX has accepted the message. My ISP's MX has accepted it, because it doesn't do domain checking. My machine's MX rejects it so fetchmail keeps trying to deliver it. While I *could* figure out how to hack sendmail to not reject the message, my preference would be to get the ISP to act responsibly and reject emails without a valid return domainname. It's standard in sendmail, rejection of such email is called for in the RFC's. The choice to not follow RFC's allows spam that would normally be rejected, through to my system which does follow the standards and rejects it -- so it stays in the download queue for my domain. The SMTP RFCs do not require rejection of email with a bogus senders domain in the header address. At that point, there is absolutely no way to not accept, reject it later. You can classify, which you use SA for (I guess, given you posting here). You can filter or even delete based on classification, or other criteria. The MX shouldn't accept it
Re: Advice sought on how to convince irresponsible Megapath ISP.
On Sun, 2014-08-17 at 07:37 -0700, Linda Walsh wrote:
Karsten Bräckelmann wrote:
Be liberal in what you accept, strict in what you send. In particular,
later stages simply must not be less liberal than early stages.
Your MX has accepted the message.
My ISP's MX has accepted it, because it doesn't do domain checking. My
machine's MX rejects it so fetchmail keeps trying to deliver it.
There is only one MX, run by your ISP. You are running an SMTP relay,
not an MX.
While I *could* figure out how to hack sendmail to not reject the message,
You don't have a choice. That sendmail is an *internal* SMTP relay after
the MX border. While you certainly are not looking at it this way, your
own services *together* with the SMTP run by your ISP form your internal
network.
The internal relay you run must not be stricter than the MX. In fact, it
simply cannot be stricter, without mail ending up in limbo. Exactly what
you have...
There is no forwarding.
It comes in their MX, and is forwarded to their users.
Again, that is not forwarding. (Hint: You are using fetchmail, not
being-forwarded-to-me-mail.)
Any ideas on how to get a cheapo-doesn't want to support anything ISP to
start blocking all the garbage the pass on?
Change ISP. You decided for them to run your MX.
I didn't decide for them, I inherited them when they bought out the
competition to supply lower quality service for the same price.
We're about to split hairs, but it is your decision to try get your ISP
to behave as you want, instead of taking your business elsewhere. So,
yes, it is your decision to let them run your MX.
It is your choice to aim for a cheapo service (your words).
It wasn't when I signed up. Cost $100 extra/month. Now only $30
extra/month that I don't host the domain with them.
But it is now, and all you're doing is complaining about it.
Expenses dropped to a fraction of what it used to be, yet you expect the
same service as before?
If you're unhappy with the service, take your business elsewhere.
Better service doesn't necessarily mean more expensive, but you
might need to shell out a few bucks for the service you want.
I already am... my ISP (cable company) doesn't have the services I want
for mail hosting. I went to another company for that,
It is irrelevant weather your mail service provider happens to also be
your cable provider. You are paying for mail services. And if you want
better service, you might need to pay more -- which is what I said.
Besides, your wording is almost ironic. Your ISP didn't offer the email
service you want, so you went for another company. Now your current
(mail) service provider doesn't offer the service you want...
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Advice sought on how to convince irresponsible Megapath ISP.
On Fri, 2014-08-15 at 19:06 -0700, Linda A. Walsh wrote:
My old email service was bought out by Megapath who is letting alot of
services slide.
My main issue is that my incoming email scripts follow the SMTP RFC's and if
the sender address isn't valid, then it's not a valid email that should be
forwarded.
My script simply check for the domain existing or not - if it doesn't exist,
then it rejects it. This causes about 100-200 messages a month that get
stuck in an IMAP queue waiting for download -- only to be downloaded and
rejected due to the sender domain not existing.
Linda, your are rather vague on details, and definitely confusing terms
and terminology.
You state your ISP would forward mail to you. While on the other hand, a
sub-set of the mail is not accepted by your scripts, thus stuck in an
IMAP account waiting for download. Both, the usage of IMAP as well as
mentioning download shows, your ISP is not forwarding mail, but you
fetching mail.
Similarly, your scripts do not reject messages, but choose not to fetch
them.
Pragmatic solution: If you insist on your scripts to not fetch those
spam messages (which have been accepted by the MX, mind you), automate
the manual download and delete stage, which frankly only exists due to
your choice of not downloading them in the first place. Make your
scripts delete, instead of skipping over them.
Be liberal in what you accept, strict in what you send. In particular,
later stages simply must not be less liberal than early stages.
Your MX has accepted the message. At that point, there is absolutely no
way to not accept, reject it later. You can classify, which you use SA
for (I guess, given you posting here). You can filter or even delete
based on classification, or other criteria.
The only response my ISP will give is to turn on their spam filtering.
I tried that. In about a 2 hour time frame, over 400 messages were
blocked as spam. Of those less than 10 were actually spam, the rest
were from various lists.
So having them censoring my incoming mail isn't gonna work, but neither will
the reject the obvious invalid domain email.
I can't believe that they insist on forwarding SPAM to their users even
though they know it is invalid and is spam.
There is no censoring. There is no forwarding.
Any ideas on how to get a cheapo-doesn't want to support anything ISP to
start blocking all the garbage the pass on?
Change ISP. You decided for them to run your MX.
It is your choice to aim for a cheapo service (your words). If you're
unhappy with the service, take your business elsewhere. Better service
doesn't necessarily mean more expensive, but you might need to shell out
a few bucks for the service you want.
--
char *t=\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4;
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;il;i++){ i%8? c=1:
(c=*++x); c128 (s+=h); if (!(h=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: Advice sought on how to convince irresponsible Megapath ISP.
Hi, The only response my ISP will give is to turn on their spam filtering. I tried that. In about a 2 hour time frame, over 400 messages were blocked as spam. Of those less than 10 were actually spam, the rest were from various lists. So having them censoring my incoming mail isn't gonna work, but neither will the reject the obvious invalid domain email. You're posting to this list because I assume you know your provider is using spamassassin. Perhaps you can filter on the SA headers in the emails, before running your scripts? Dump the spam, and only forward on the good email. Regards, Alex
Re: Advice sought on how to convince irresponsible Megapath ISP.
Alex wrote: Hi, The only response my ISP will give is to turn on their spam filtering. I tried that. In about a 2 hour time frame, over 400 messages were blocked as spam. Of those less than 10 were actually spam, the rest were from various lists. So having them censoring my incoming mail isn't gonna work, but neither will the reject the obvious invalid domain email. Spamassassin wouldn't get things so wrong, in my experience. I have spamassassin on my end and it rarely has a false positive. I don't know what they use, but it certainly wasn't well configured on their recommended setting where it had a very high false positive rate. You're posting to this list because I assume you know your provider is using spamassassin. The start of filtering out spam is filtering out mail that doesn't have a valid return address -- it's not valid email. I'm stuck because my ISP won't filter it out, I have to download it to find out that it's invalid, then local sendmail rejects it, so it stays in their box. Perhaps you can filter on the SA headers in the emails, before running your scripts? Dump the spam, and only forward on the good email. I have the end node -- so I have to take everything they give to filter out the spam. I was wondering if anyone had any experience in showing their ISP the light... Oh well
