Re: [clamav-users] ign2 whitelist don't work
Charles Swiger wrote: > On Jul 19, 2016, at 10:39 AM, Kris Deugau wrote: >> ClamAV hits on any of the Heuristics.* tests get flagged instead of >> treated the same as the signature-based hits, and that flag either >> causes an an adjustment in the SpamAssassin results returned directly to >> MIMEDefang later on, or a header is added which I check for in >> SpamAssassin on mail delivery. > > Are you using LMTP, or did SpamAssassin grow a local delivery agent > capability? Wearing my ISP sysadmin hat, for inbound mail we have a custom delivery agent that calls both ClamAV and SA, along with doing a number of other tasks. We don't currently handle Heuristics.* hits differently, something I'd like to change. On our outbound servers they're flagged and added to the SA results. On my personal server, which happens to still be on sendmail, I use procmail for local delivery. My new server in (very slow) progress will run Postfix, but I'll still use procmail for local delivery. For all that it's not the friendliest tool it does its job quite well and I'm the only user who has any need of complex delivery rules. I'd switch to something using sieve but I don't like the limitation on not calling external programs - it makes it much harder to write a set of delivery rules like this: if (sender is newsletter A) deliver to folder news if (sender is newsletter B) deliver to folder news call a lightweight content filter if the filter says "Spam" deliver to folder spam if (received from a mailing list that allows nonsubscriber posts) deliver to folder spammynews call a process-expensive content filter if the filter says "Spam" deliver to folder spam deliver to the Inbox Which about sums up my .procmailrc, although the live one is much longer. -kgd ___ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
Am 19.07.2016 um 19:19 schrieb Charles Swiger: On Jul 19, 2016, at 1:09 PM, Reindl Harald wrote: False. Assuming that there is only one correct mail architecture is a major fallacy. bla - yes there are more ways but your whole stuff about SPF was entirely wrong from the very begin in case of the messages in question You managed to misinterpet what I actually said, and are now off in the weeds. Have fun with that strawman. you just said nothing but nonsense in context of that messages and trigger a "heuristic phising" in case of a paypal mail from a mailserver listed in the SPF of the paypal spf belonging to the envelope sender is a false postive not be able to whitelist such a misbehaving trigger without disable other things too is a bug and/or design mistake - period If a mail server sends outbound, it needs to be willing to handle bounces and DSNs for those messages/domains which it sends. bullshit - the MX does and this servers outbound mail was *not* for a domain below it's own hostname and so it has no business for inbound mail That, and you are inexcusably rude to people who were trying to help you. trying to help? where? well, agreed, more useful than the first response "You must disable Heuristics using clamd.conf " while such a otpion don't exist you are talking 90% of this thread completly off-topic stuff Good day, sir. I hope your customers find better service elsewhere i doubt - the problem itself is solved for days but in a way which should not be needed and the whole conversation with you was to 90% or more completly off-topic signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
On Jul 19, 2016, at 10:39 AM, Kris Deugau wrote: > Charles Swiger wrote: >> The milter approach is less flexible. With a scoring mechanism, you can >> rate actual viruses sufficiently negative that the scoring algorithm will >> always reject them. > > That depends on the milter you're using. My own favoured milter is > MIMEDefang, which allows you do do anything you like to a message in > transit so long as you can figure out how to code it in Perl. Fair enough. clamav-milter was implied by context, but MIMEDefang is certainly a decent choice, especially if one can do a bit of Perl hacking. For what it is worth, I've been happier with Python-based filters, since they tend to use less resources than the Perl-based software. But that has more to do with how well people can write code in the different languages-- I've also seen some very lightweight and efficient Perl code. > ClamAV hits on any of the Heuristics.* tests get flagged instead of > treated the same as the signature-based hits, and that flag either > causes an an adjustment in the SpamAssassin results returned directly to > MIMEDefang later on, or a header is added which I check for in > SpamAssassin on mail delivery. Are you using LMTP, or did SpamAssassin grow a local delivery agent capability? Last I'd seen, amavisd-new or MailScanner are still recommended for integration between the MTA and SpamAssassin / ClamAV Regards, -- -Chuck ___ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
On Jul 19, 2016, at 1:09 PM, Reindl Harald wrote: >> False. Assuming that there is only one correct mail architecture is a major >> fallacy. > > bla - yes there are more ways but your whole stuff about SPF was entirely > wrong from the very begin in case of the messages in question You managed to misinterpet what I actually said, and are now off in the weeds. Have fun with that strawman. >> If a mail server sends outbound, it needs to be willing to handle bounces >> and DSNs for those messages/domains which it sends. > > bullshit - the MX does and this servers outbound mail was *not* for a domain > below it's own hostname and so it has no business for inbound mail That, and you are inexcusably rude to people who were trying to help you. Good day, sir. I hope your customers find better service elsewhere. Regards, -- -Chuck ___ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
Am 19.07.2016 um 19:00 schrieb Charles Swiger:
On Jul 19, 2016, at 10:28 AM, Reindl Harald wrote:
[ ... ]
2) In the absence of MX records stating otherwise, I expect that any mailserver
which sends outbound email should be willing to accept inbound mail for the
same domains it terminates or relays email on behalf of.
that is not how email works
As I recall, you were either submitting a bug report about ClamAV and SPF, which seems
misguided as you've since acknowledged ("i know that SPF is not relevant for
clamav"), or at the least you were looking for feedback about how to better handle
legitimate email from paypal.at which you were bouncing due to ClamAV's heuristics.
no, i was submitting what the subject says and explained why it's
unacceptable not to be able in a software which tries to make
assumptions about phising by no clue about SPF
a) the sender is @mail.paypal.at and not "@epsl1.com"
True.
b) every smarter setup these days has strictly
seperated outbound and inbound servers
False. Assuming that there is only one correct mail architecture is a major
fallacy.
bla - yes there are more ways but your whole stuff about SPF was
entirely wrong from the very begin in case of the messages in question
If a mail server sends outbound, it needs to be willing to handle bounces and
DSNs for those messages/domains which it sends.
bullshit - the MX does and this servers outbound mail was *not* for a
domain below it's own hostname and so it has no business for inbound mail
why?
because it's much easier to define MTA policies for spamfiltering when you need
not to mix with mail clients and when you do outbound spamfiltering you need
completly different rules (no RBL looksups, no PTR checks, different scorings
and first of all no postscreen in front which a MUA can't handle)
It is reasonable to have different inbound and outbound MTAs to implement
different policies? Sure.
Is that the only mechanism by which one can have different policies? Nope.
far off-topic the whole discussion just because you where unable to look
careful at the one logline and make correct SPF requests while i already
told in the orginal mail that i have verified it and even posted the
spamassassin SPF_PASS line of the message in question
It is reasonable to trust all local mail and push the burden of checking it
upon others? Nope.
did i say that?
You should be applying spamfiltering and especially malware/virus scanning to
outbound email just as rigorously as you do to inbound email. In a few cases
that I am familiar with, outbound email is screened more carefully than inbound
email.
where did i say anything else?
but you need different configs as i explained and it should be pretty
clear why - there is no point makeing dialup-rbl-tests on a submission
client which is typically a enduser somewhere at home
signature.asc
Description: OpenPGP digital signature
___
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
On Jul 19, 2016, at 10:28 AM, Reindl Harald wrote:
[ ... ]
>> 2) In the absence of MX records stating otherwise, I expect that any
>> mailserver which sends outbound email should be willing to accept inbound
>> mail for the same domains it terminates or relays email on behalf of.
>
> that is not how email works
As I recall, you were either submitting a bug report about ClamAV and SPF,
which seems misguided as you've since acknowledged ("i know that SPF is not
relevant for clamav"), or at the least you were looking for feedback about how
to better handle legitimate email from paypal.at which you were bouncing due to
ClamAV's heuristics.
> a) the sender is @mail.paypal.at and not "@epsl1.com"
True.
> b) every smarter setup these days has strictly
> seperated outbound and inbound servers
False. Assuming that there is only one correct mail architecture is a major
fallacy.
What you describe is one reasonable architecture for a large ISP which needs to
have redundant sending and receiving mail servers. However, there are lots of
smaller sites which have no need for that-- they might be better off having an
external MX relay in their firewall DMZ which handles both inbound and outbound
mail, and an internal mailhost / reader box, for example.
> what you expect is completly pointless - as example you have no business to
> deliver mail to our outbound server unless you are a customer with a valid
> username and password since inbound mail is expected at the MX (spamfirewall)
> and not at the submission server
You appear to have skipped past this phrase: "In the absence of MX records
stating otherwise..."
If a mail server sends outbound, it needs to be willing to handle bounces and
DSNs for those messages/domains which it sends.
> why?
>
> because it's much easier to define MTA policies for spamfiltering when you
> need not to mix with mail clients and when you do outbound spamfiltering you
> need completly different rules (no RBL looksups, no PTR checks, different
> scorings and first of all no postscreen in front which a MUA can't handle)
It is reasonable to have different inbound and outbound MTAs to implement
different policies? Sure.
Is that the only mechanism by which one can have different policies? Nope.
It is reasonable to trust all local mail and push the burden of checking it
upon others? Nope.
You should be applying spamfiltering and especially malware/virus scanning to
outbound email just as rigorously as you do to inbound email. In a few cases
that I am familiar with, outbound email is screened more carefully than inbound
email.
Regards,
--
-Chuck
___
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
Charles Swiger wrote: > The milter approach is less flexible. With a scoring mechanism, you can rate > actual viruses sufficiently negative that the scoring algorithm will always > reject them. That depends on the milter you're using. My own favoured milter is MIMEDefang, which allows you do do anything you like to a message in transit so long as you can figure out how to code it in Perl. ClamAV hits on any of the Heuristics.* tests get flagged instead of treated the same as the signature-based hits, and that flag either causes an an adjustment in the SpamAssassin results returned directly to MIMEDefang later on, or a header is added which I check for in SpamAssassin on mail delivery. -kgd ___ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
Am 19.07.2016 um 16:02 schrieb Charles Swiger: Perhaps English isn't your native language? no, it isn't first: i know that SPF is not relevant for clamav, but since it's a clean way to verify the source of a message and clamav can't do this such spoofing rules in clamav which can't be disabled without disable other things too is bad second: i know about scoring - as said i have more than one clamd and what i had to do now is move safebrowsing rules to the other instance because they whould have disabled too by "PhishingScanURLs no" yes, i know that the milter is not flexible and *hence* there are different clamd instances with different configs and different signatures - *but* the last ressort instance for clamav-milter is terrible unflexible to configure because the weakness of ign2 which is waht this topic is about and hence i would nearly need 3 instances to get the granualry i need - but that's not really doable because currently the two running instances eare eating around 800 MB RAM I spoke precisely; I did not say that epsl1.com was the sending domain. Your logs demonstrate that it, or more precisely mta106b.pmx1.epsl1.com was the MTA sending the message to your mailserver and that mail.paypal.at was the SMTP "Mail from" domain. and so what business has "epsl1.com" in context of SPF for @mail.paypal.at? which domain not only doesn't appear in the SPF records for paypal.com / paypal.at, but also doesn't appear to have any published MX records at all (per "dig -t mx epsl1.com.")...? sorry but you don't understand SPF really good and since when have MX records something to do with outbound mail mail.paypal.at. 3600IN TXT "v=spf1 ip4:142.54.244.96/27 ip4:142.54.244.128/29 -all" 142.54.244.106 is the sending IP Two points: 1) I got different SPF results here, although hitting a third-party website to lookup the SPF records for mail.paypal.at does give the result you've shown. (I'm travelling and using someone else's network at the moment, so it's possible that they/their ISP is swizzling DNS lookups.) or you did "dig TXT paypal.at" which is not the same as "dig TXT mail.paypal.at" anyways - "dig NS paypal.at" paypal.at. 300 IN NS ns2.p57.dynect.net. paypal.at. 300 IN NS ns1.p57.dynect.net. paypal.at. 300 IN NS ns3.p57.dynect.net. paypal.at. 300 IN NS ns4.p57.dynect.net. dig TXT mail.paypal.at @ns2.p57.dynect.net mail.paypal.at. 3600IN TXT "v=spf1 ip4:142.54.244.96/27 ip4:142.54.244.128/29 -all" 2) In the absence of MX records stating otherwise, I expect that any mailserver which sends outbound email should be willing to accept inbound mail for the same domains it terminates or relays email on behalf of. that is not how email works a) the sender is @mail.paypal.at and not "@epsl1.com" b) every smarter setup these days has strictly seperated outbound and inbound servers what you expect is completly pointless - as example you have no business to deliver mail to our outbound server unless you are a customer with a valid username and password since inbound mail is expected at the MX (spamfirewall) and not at the submission server why? because it's much easier to define MTA policies for spamfiltering when you need not to mix with mail clients and when you do outbound spamfiltering you need completly different rules (no RBL looksups, no PTR checks, different scorings and first of all no postscreen in front which a MUA can't handle) signature.asc Description: OpenPGP digital signature ___ Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ign2 whitelist don't work
On Jul 18, 2016, at 1:03 PM, Reindl Harald wrote: >> For that specific case, check that OLE2BlockMacros is set to no. > > the point is this should be independent Well, it currently isn't. >>> it makes no sense that you can't disable specific heuristics >> >> This is a reasonable point. One should be able to use the ign2 whitelist to >> disable specific heuristics, as well as having more fine-grained control >> within clamd.conf. > > fine-grained control would be difficult given you have a mix of 3rd party > signatures to know which affects what That's why being able to list the exact signatures to ignore rather than having fixed categories in clamd.conf would be a better option. > i am really shocked that ign2 whitelists don't work in a simple way that > whatever is triggered due the scan is compared agianst the whitelist and just > skipped as it was never there Yes. >> In the meantime, however, if you want to have PhishingScanURLs or PUA >> categories enabled, then you should use any matches returned under >> Heuristics.* or Phishing.Heuristics.* as a soft fail and use it for scoring >> purposes rather than as a hard fail. > > running as milter that's not possible and when you run clamd only as > spamassassin plugin any whitelisting would skip the complete virus scan what > is the reason to have the milter as second instance with different signatures The milter approach is less flexible. With a scoring mechanism, you can rate actual viruses sufficiently negative that the scoring algorithm will always reject them. >>> such false positives are *unacceptable* in case of the monthly account >>> overview and frankly i have not seen any hit which was not very likely a >>> false positive (as example newsletters from payment companies over services >>> like mailchimp) >> [ ... ] >>> Jul 8 14:42:10 mail-gw postfix/cleanup[19493]: 3rmDds0LjczB44: >>> milter-reject: END-OF-MESSAGE from mta106b.pmx1.epsl1.com[142.54.244.106]: >>> 5.7.1 Virus found or dangerous attachment: >>> "Heuristics.Phishing.Email.SpoofedDomain"; >>> from= >>> to=<*> proto=ESMTP helo= >> >> While I'd agree that it should be unacceptable for financial institutions to >> use third parties for email distribution of sensitive content, I suspect >> that wasn't your intended point. > > they use SPF and until clamav can't do a SPF test it must not run phising > tests unconditionally while the real problem is that "PhishingScanURLs no" > also disabled google safe browing signatures ClamAV is a virus scanner; it is not its job to perform SPF lookups. Whatever is calling ClamAV to process mail, whether it be a milter interface, amavisd, etc is the software that should be handling SPF checks. >> I also believe that it should not be acceptable for financial institutions-- >> or anyone who values their reputation-- to send emails containing HTML links >> where the domain of the link text shown to the user does not match the >> domain of the actual A href attribute. > > frankly i am sorry that i can't play police for every institution out there > and explain them how to handle email :-) You have no general obligation to police the entire internet, but one should care about the institutions that you or your userbase interacts with. >> Have you contacted PalPal and asked them to investigate why they are sending >> emails which look like phishing attempts via epsl1.com domain > > since i am mailadmin and don't have access to my users email no > however - "epsl1.com" is *not* the sending domain > you confuse hostnames and sender address Perhaps English isn't your native language? I spoke precisely; I did not say that epsl1.com was the sending domain. Your logs demonstrate that it, or more precisely mta106b.pmx1.epsl1.com was the MTA sending the message to your mailserver and that mail.paypal.at was the SMTP "Mail from" domain. >> which domain not only doesn't appear in the SPF records for paypal.com / >> paypal.at, but also doesn't appear to have any published MX records at all >> (per "dig -t mx epsl1.com.")...? > > sorry but you don't understand SPF really good and since when have MX records > something to do with outbound mail > > mail.paypal.at. 3600IN TXT "v=spf1 ip4:142.54.244.96/27 > ip4:142.54.244.128/29 -all" > > 142.54.244.106 is the sending IP Two points: 1) I got different SPF results here, although hitting a third-party website to lookup the SPF records for mail.paypal.at does give the result you've shown. (I'm travelling and using someone else's network at the moment, so it's possible that they/their ISP is swizzling DNS lookups.) 2) In the absence of MX records stating otherwise, I expect that any mailserver which sends outbound email should be willing to accept inbound mail for the same domains it terminates or relays email on behalf of. >> I did. And I have since stopped using PayPal entirely after they failed to >> improve their practices
