Re: [mailop] Isn't SpamEatingMonkey's SEM-URI broken?
First, Grant, you're far too trusting of institutions and government.
They're especially corrupt these days. Many governments that have had
decades or centuries-long track records for bing mostly trustworthy and
fair - are actually very corrupt these days. Such a governing body would
downward devolve into "what benefits our party" before long. And then
they'l punish those DNSBLs that blocked spam from their party (and orgs
that are their ideological allies), while rewarding those whose actions
benefited their party and ideological allies. They'll also start putting
their own ideological biases as criteria for what constitutes a quality
DNSBL. Remember that spam is about "consent" - NOT "content". But
they'll ultimately make content a criteria. They'll also start rating
DNSBLs by their ability to block what they deem to be "dangerous
misinformation", and the criteria for that will be very ideologically
biased.
And then Michael, the problem with MOST stats - is that this OFTEN
becomes "those stats that rubber stamp what our filter already concluded
must be best, while those that disagreed the most must be wrong".
(circular reasoning)
Merging these two subthreads together - the better solution is simple -
WHICH DNSBLs are economically incentived to be more accurate, as defined
by giving the end users more of what they want, meanwhile erring on the
side of fewer false positives, but not so much so as to create too large
of loopholes for spammers.
So then - how can that be measured?
The best way is to start by taking a DNSBL not previously used - have it
simply mark the messages without actually using the list to block spams
(such as scoring .01 in SA) - then see what they marked as spam that
went into the inbox (so it wouldn't have been blocked without that
DNSBL's usage). THEN - start examining random samplings of THOSE
messages to determine how much this DNSBL is reducing false negatives,
and how many false positives it's causing. Then in "edge cases" or
difficult to determine cases - ask the customer what they think ("do you
recognize this sender?"). But keep in mind that **all** DNSBLs make rare
mistakes, and there are going to be rare situations where a customer
really did engage with a spammer who sent 100% spams to a purchased
list, as well as occasional rare outliers like that.
Then judge that DNSBL on THAT basis. It's a bit tedious and takes some
work collecting such stats - but this isn't rocket science.
If a DNSBL is causing spams that were previously going to the inbox to
significantly lessen (reduction of "false negatives"), while also not
having any noticeable uptick in false positives - then they're going to
improve the spam filtering far more than a DNSBL that only rubber
stamped what a filter was already doing - which is EASY to do if that
DNSBL is only (or mostly) going after the "low hanging fruit". So it's
conceivable for a lower quality DNSBL to block 95% of all spam, not have
a single false positives, but not block a single spam that was otherwise
making it to the inbox - and while such a DNSBL has impressive stats -
it's not adding ANY value to the filtering. So high percentages of hits
on inbound messages can be deceiving. And lower percentages of hits on
the total number of inbound spams can ALSO be deceiving if such a list
is still reducing false negatives without causing false positives - such
a list might be doing an amazing job of that - yet without hitting on a
very high percentage of all inbound spam - since it didn't overly focus
on "lower hanging fruit". So, to summarize, a signficant reduction of
false negatives, without causing false positives - THAT is where value
and improvement is found - but raw spam stats for many systems OFTEN
don't properly measure that.
So, when evaluated in this SUPERIOR way, only a handful of DNSBLs rise
to the top. Those who have done such analysis on ALL of the most
well-known DNSBLs know EXACTLY which few are at the very top. And NONE
of the DNSBLs which aren't incentivized to "give the people what they
want" are there. NONE OF THEM! (and many which are incentivized to do
that are also not there - but, again, ZERO that are not properly
incentivized are at the top most beneficial DNSBLs). And a DNSBL trying
to please a government body that's biased by partisan politics will
NEVER get there (or at least not for long since these institutions
eventually become hopelessly corrupted).
For example - if SORBS has a few too many false positives, ProofPoint
(who owns SORBLS) probably isn't gonna lose a dime. If UCEPROTECT has a
few too many false positives, they'll actually MAKE more money. But, in
contrast, if invaluement or Spamhaus or Abusix ever has an significant
uptick in false positives - they'll all potentially lose much money! But
even then, again, some who are properly incentivized - STILL aren't
particularly good at this. DNSBLs are HARD!
Rob McEwen, invaluement
Re: [mailop] Isn't SpamEatingMonkey's SEM-URI broken?
Actually, what I like is those companies that show real time stats on RBL's, you get to see who is the most accurate, not only who would block the most.. If you get 'inaccuracies', then someone has done something wrong. M3AAWG might be exactly the WRONG organization for this, given it's closed membership.. Need a more altruistic partner for vetting.. Anyone have ideas or contacts? (I know, we have even got on SpamEatingMonkey, love to see their listing criteria, there is suspicion that domains in signatures, or forwarded emails might be enough to trigger it) On 2023-07-10 16:30, Grant Taylor via mailop wrote: On 7/10/23 2:40 PM, Jarland Donnell via mailop wrote: The problem is, running any blacklist and wanting to constantly speak to people who are often just confused about how relevant your list even is, are very often two different things. So there's not anyone to talk to, at least not from a public-facing angle. It would certainly be nice if anyone on this list that might be representing SEM wanted to speak up on the matter. This sounds to be a case worth speaking up on. I found myself wondering if there was anything like the Better Business Bureau or some sort of accreditation that RBL operators can apply for wherein they need to: - demonstrate that they are responsive - publish what is required to be delisted - provide points of contact The intention being that an RBL operator is taking steps / effort to be genuinely good. Yes, mistakes and accidents happen. It's how those mistakes and accidents are responded to that make all the difference. I'd wonder if someone like M3AAWG or the likes could fulfill this function. If such an accreditation existed, then perhaps various filtering software providers could default to only enabling accredited RBLs. I hope it goes without saying that I would want it to be relatively easy to become accredited. I suspect it would need to be even easier to have such accreditation revoked. All players start somewhere small and some grow into big players. Grant. . . . ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop -- "Catch the Magic of Linux..." Michael Peddemors, President/CEO LinuxMagic Inc. Visit us at http://www.linuxmagic.com @linuxmagic A Wizard IT Company - For More Info http://www.wizard.ca "LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd. 604-682-0300 Beautiful British Columbia, Canada This email and any electronic data contained are confidential and intended solely for the use of the individual or entity to which they are addressed. Please note that any views or opinions presented in this email are solely those of the author and are not intended to represent those of the company. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Isn't SpamEatingMonkey's SEM-URI broken?
On 7/10/23 2:40 PM, Jarland Donnell via mailop wrote: The problem is, running any blacklist and wanting to constantly speak to people who are often just confused about how relevant your list even is, are very often two different things. So there's not anyone to talk to, at least not from a public-facing angle. It would certainly be nice if anyone on this list that might be representing SEM wanted to speak up on the matter. This sounds to be a case worth speaking up on. I found myself wondering if there was anything like the Better Business Bureau or some sort of accreditation that RBL operators can apply for wherein they need to: - demonstrate that they are responsive - publish what is required to be delisted - provide points of contact The intention being that an RBL operator is taking steps / effort to be genuinely good. Yes, mistakes and accidents happen. It's how those mistakes and accidents are responded to that make all the difference. I'd wonder if someone like M3AAWG or the likes could fulfill this function. If such an accreditation existed, then perhaps various filtering software providers could default to only enabling accredited RBLs. I hope it goes without saying that I would want it to be relatively easy to become accredited. I suspect it would need to be even easier to have such accreditation revoked. All players start somewhere small and some grow into big players. Grant. . . . ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Guide for setting up a mail server ?
Grant,
Well put.
I was just going to link to Gilles Chehade’s post
(https://poolp.org/posts/2019-12-15/decentralised-smtp-is-for-the-greater-good/)
I don’t find running my own personal email server that hard or time consuming.
The most time consuming element being “keep OpenBSD updated”. :-)
Sean
> On Jul 10, 2023, at 09:44, Grant Taylor via mailop wrote:
>
> Dear ${FELLOW_EMAIL_ADMINIATRATOR},
>
> I don't know how to preface this email other than to say -- I believe the
> following needs to be said lest we loose even more control of our email
> community.
>
> I'm sorry that both 1) I feel that the following needs to be said and 2) that
> I'm the one that's saying it.
>
> ...
>
> I can't say that any of Richard's comments are technically wrong. But I will
> say that the tone of what the email represents is both disappointing and
> concerning to me.
>
> N.B. that Richard is one of many that have said similar things. My comments
> are *NOT* directed at Richard.
>
> Rather many of us are culpable for allowing things to get into this state by
> not actively fighting against it. -- The first step is admitting you have a
> problem.
>
> ...
>
> I agree that email is not easy by any stretch of the imagination. But
> comparatively I suspect it's easier to host your own email than starting a
> company to build a car from scratch. Is this a good comparison, probably
> not. But is it true? I really hop so.
>
> On 7/10/23 7:48 AM, Richard Clayton via mailop wrote:
>> not that I know of -- arguably there should be one, but perhaps it will just
>> encourage unwise activity. I am reminded of Usenet advice of not posting for
>> the first six months and if you ask why that is good advice then add another
>> six months...
>
> I agree with -- what I'll call -- auditing. I don't agree with asking
> questions meaning that you need to extend the audit time. I've seen some
> EXTREMELY intelligent questions asked by people very knew to the situation.
>
> Simply asking a question does not, in and of itself, mean that someone is
> ignorant of the topic. Quite the opposite can be true. E.g. "Is there any
> reason to ever run an open relay? There seem to be LOTS of negative points
> to doing so; ." type thing.
>
>> I recently reviewed an IETF draft on (de)centralisation which observed that
>> running your own mail system, rather than using a centralised provider was
>> far too hard.
>
> This disappoints me, greatly. Both the idea that running a decentralized
> mail server is hard and more so that such recommendations would admit or
> worse support such a stance.
>
>> In discussions with Eliot Lear we ended up with a list of things you had to
>> do:
>> * configure and manage the MTA
>
> This is obvious and not specific to email. You have to configure the service
> for any and all services. Chances of defaults doing exactly what you want
> are quite rare.
>
>> * arrange for a backup MTA
>
> I'll argue that requiring a backup MTA is not strictly required.
>
> I'll absolutely agree that a backup MTA is best practice.
>
> There is a really big delta between strict requirement and best practice.
>
>> * manage DNS MX, DKIM, DMARC and SPF records
>
> None of those are strictly required either. Business to business email can
> function without any of them.
>
> I'll VERY STRONGLY ENCOURAGE at the very least MX record(s).
>
> SPF, DKIM, and DMARC are the order that I'd advise others be implemented.
>
> Chances are quite good that you will be able to exchange email with domains
> that aren't hosted by the email oligarchs.
>
> Aside: Have enough email administrators given up the torch and now started
> praying at the alter of the email oligarchs?
>
>> * manage reverse lookup records, including managing the uncertain chain of
>> authority between the instance and the nearest SOA
>
> I didn't have a problem with reverse DNS being required 22 years ago and I
> have even less of a problem with it today.
>
> Yes, this can make it harder for someone to run an email server at home. But
> if they really want to do this, they can find ways to make it happen. I'm
> happy to help people make it happen too.
>
> There is also the fact that unless the ISP is doing egress filtering --
> that's it's own independent discussion which the lack thereof helps me in
> this discussion -- users can probably have acceptable success with all but
> the email oligarchs if they simply have their MTA hello as the name that the
> ISP assigns to the connection presuming that the ISP has forward and reverse
> DNS configured therefor.
>
>> * manage certificates associated with TLS for SMTP and IMAP
>
> I'll argue that TLS is not strictly required.
>
> I'll absolutely agree that TLS is a best practice.
>
> I'll also posit that IMAP is not required for email.
>
>> * manage DKIM certificate
>
> I maintain that DKIM is not a strict requirement.
>
>> * manage one's upstream to
Re: [mailop] Guide for setting up a mail server ?
Dear ${FELLOW_EMAIL_ADMINIATRATOR},
I don't know how to preface this email other than to say -- I believe
the following needs to be said lest we loose even more control of our
email community.
I'm sorry that both 1) I feel that the following needs to be said and 2)
that I'm the one that's saying it.
...
I can't say that any of Richard's comments are technically wrong. But I
will say that the tone of what the email represents is both
disappointing and concerning to me.
N.B. that Richard is one of many that have said similar things. My
comments are *NOT* directed at Richard.
Rather many of us are culpable for allowing things to get into this
state by not actively fighting against it. -- The first step is
admitting you have a problem.
...
I agree that email is not easy by any stretch of the imagination. But
comparatively I suspect it's easier to host your own email than starting
a company to build a car from scratch. Is this a good comparison,
probably not. But is it true? I really hop so.
On 7/10/23 7:48 AM, Richard Clayton via mailop wrote:
not that I know of -- arguably there should be one, but perhaps it
will just encourage unwise activity. I am reminded of Usenet advice
of not posting for the first six months and if you ask why that is
good advice then add another six months...
I agree with -- what I'll call -- auditing. I don't agree with asking
questions meaning that you need to extend the audit time. I've seen
some EXTREMELY intelligent questions asked by people very knew to the
situation.
Simply asking a question does not, in and of itself, mean that someone
is ignorant of the topic. Quite the opposite can be true. E.g. "Is
there any reason to ever run an open relay? There seem to be LOTS of
negative points to doing so; ." type thing.
I recently reviewed an IETF draft on (de)centralisation which
observed that running your own mail system, rather than using a
centralised provider was far too hard.
This disappoints me, greatly. Both the idea that running a
decentralized mail server is hard and more so that such recommendations
would admit or worse support such a stance.
In discussions with Eliot Lear we ended up with a list of things you
had to do:
* configure and manage the MTA
This is obvious and not specific to email. You have to configure the
service for any and all services. Chances of defaults doing exactly
what you want are quite rare.
* arrange for a backup MTA
I'll argue that requiring a backup MTA is not strictly required.
I'll absolutely agree that a backup MTA is best practice.
There is a really big delta between strict requirement and best practice.
* manage DNS MX, DKIM, DMARC and SPF records
None of those are strictly required either. Business to business email
can function without any of them.
I'll VERY STRONGLY ENCOURAGE at the very least MX record(s).
SPF, DKIM, and DMARC are the order that I'd advise others be implemented.
Chances are quite good that you will be able to exchange email with
domains that aren't hosted by the email oligarchs.
Aside: Have enough email administrators given up the torch and now
started praying at the alter of the email oligarchs?
* manage reverse lookup records, including managing the uncertain
chain of authority between the instance and the nearest SOA
I didn't have a problem with reverse DNS being required 22 years ago and
I have even less of a problem with it today.
Yes, this can make it harder for someone to run an email server at home.
But if they really want to do this, they can find ways to make it
happen. I'm happy to help people make it happen too.
There is also the fact that unless the ISP is doing egress filtering --
that's it's own independent discussion which the lack thereof helps me
in this discussion -- users can probably have acceptable success with
all but the email oligarchs if they simply have their MTA hello as the
name that the ISP assigns to the connection presuming that the ISP has
forward and reverse DNS configured therefor.
* manage certificates associated with TLS for SMTP and IMAP
I'll argue that TLS is not strictly required.
I'll absolutely agree that TLS is a best practice.
I'll also posit that IMAP is not required for email.
* manage DKIM certificate
I maintain that DKIM is not a strict requirement.
* manage one's upstream to address PBL issues
Maybe, maybe not.
* keep the MTA secure and free from DOS attack
I have problems with that statement.
1st: What is "secure" in this context?
2nd: Why is anything other than an MTA less susceptible to a DOS attack
than an MTA?
Also, Michael W. Lucas's I Have A Dream comment about "goals" vs
"dreams" comes to mind.
TL;DR: Focus on goals, things that you can influence, and don't worry
about dreams, things that you have no influence over.
Link - I Have A Dream
- https://mwl.io/archives/22749
ALSO back in 2011 (when the world was a little
Re: [mailop] Guide for setting up a mail server ?
I covered that here: https://www.spamresource.com/2020/07/small-mailserver-best-current-practices.html Anybody who would like to write up a guide, I'd be happy to publish it on Spam Resource (or link to it if you publish it elsewhere). Feel free to reach out. Cheers, Al Iverson -- Al Iverson / Deliverability blogging at https://www.spamresource.com Subscribe to the weekly newsletter at https://ml.spamresource.com DNS Tools: https://xnnd.com / (312) 725-0130 / Chicago (Central Time) ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Guide for setting up a mail server ?
In message <20230709223922.dd59afd9f...@ary.qy>, John Levine via mailop writes >A friend of mine wants to set up a mail server on a VPS and asked me what >he needs to do beyond the obvious setting up postfix and dovecot. Is there >a good summary somewhere? not that I know of -- arguably there should be one, but perhaps it will just encourage unwise activity. I am reminded of Usenet advice of not posting for the first six months and if you ask why that is good advice then add another six months... I recently reviewed an IETF draft on (de)centralisation which observed that running your own mail system, rather than using a centralised provider was far too hard. In discussions with Eliot Lear we ended up with a list of things you had to do: * configure and manage the MTA * arrange for a backup MTA * manage DNS MX, DKIM, DMARC and SPF records * manage reverse lookup records, including managing the uncertain chain of authority between the instance and the nearest SOA * manage certificates associated with TLS for SMTP and IMAP * manage DKIM certificate * manage one's upstream to address PBL issues * keep the MTA secure and free from DOS attack >I'm thinking of things like: > >- choose a provider that has decent mail behavior, e.g., not Digital Ocean > >- make sure the MTA's forward and reverse DNS match > >- set up an SPF record, probably "v=spf1 mx ~all" > >- set up DKIM signing for each domain you host, make the >DKIM domain match the From: domain > >= start slow and look at any bounces > >- maybe collect DMARC stats but for a small volume MTA, not very interesting ALSO back in 2011 (when the world was a little simpler perhaps) I worked on a M3AAWG BCP on this topic -- which eventually went nowhere ... the list then was (and I stress this was not sufficiently peer reviewed then to be authoritative, but it was written by some experts) * Use a static IPv4 address for your email system * Do not share this IPv4 address with user machines * Do not host your email system 'in the cloud' * Make sure that your IP address is not listed in the PBL * Provide an MX record * Provide meaningful and consistent reverse DNS * Your system should say HELO (or EHLO) with its hostname * Keep your software completely up-to-date * Ensure that only authorised users can send email through your system. * Limit outgoing email volumes * Accept reports of problems with your systems * Review the mail system logs on a regular basis * Be reliable (viz have at least 4 9s availability) * Don't be an open relay * Don't create backscatter * Maintain a good reputation -- richard Richard Clayton Those who would give up essential Liberty, to purchase a little temporary Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755 signature.asc Description: PGP signature ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Guide for setting up a mail server ?
Home - maddy https://maddy.email/ -- Von meiner Hängematte aus gesendet. -Original Message- From: "Taavi Eomäe via mailop" To: John Levine , mailop@mailop.org Sent: Mo., 10 Juli 2023 10:12 Subject: Re: [mailop] Guide for setting up a mail server ? Instead of struggling with Postfix, OpenDKIM, Dovecot and friends (and losing out on quite a few features). I'd really recommend looking at Maddy instead. Immensely better "UX" than the currently mentioned approach. ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
Re: [mailop] Guide for setting up a mail server ?
Instead of struggling with Postfix, OpenDKIM, Dovecot and friends (and losing out on quite a few features). I'd really recommend looking at Maddy instead. Immensely better "UX" than the currently mentioned approach. smime.p7s Description: S/MIME Cryptographic Signature ___ mailop mailing list mailop@mailop.org https://list.mailop.org/listinfo/mailop
