Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Helge Kreutzmann]

Hello Adam,
Am Thu, Aug 17, 2023 at 06:52:11PM +0100 schrieb Adam D. Barratt:
> An initial version, rewriting mails to Google-hosted domains from
> "external" e-mail addresses (those for which debian.org's mail relays
> don't consider themselves authoritative, so mostly not *.debian.org and
> *.debconf.org) is now live.
> 
> Please let DSA know if you encounter any issues.

Thanks a lot for the speedy fixing.

I'll report any issues (if any).

Greetings

  Helge

-- 
  Dr. Helge Kreutzmann deb...@helgefjell.de
   Dipl.-Phys.   http://www.helgefjell.de/debian.php
64bit GNU powered gpg signed mail preferred
   Help keep free software "libre": http://www.ffii.de/


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Adam D. Barratt]

On Sun, 2023-08-13 at 10:57 +0100, Adam D. Barratt wrote:
> On Sat, 2023-08-12 at 23:13 +0200, Mattia Rizzolo wrote:
> > On Sat, Aug 12, 2023 at 01:41:46PM -0700, Russ Allbery wrote:
> > > The problem I suspect is with email forwarding, and specifically
> > > email
> > > forwarding to Gmail, which has recently ramped up the amount of
> > > verification it does on messages.  Because of email forwarding,
> > > Gmail sees
> > > a message purportedly from helgefjell.de but actually delivered
> > > by
> > > debian.org mail servers, and has now decided to be suspicious of
> > > that.
> > 
> > This is the exact use case that SRS was developer for, however
> > gmail's documentation does not recommend that (but the situation,
> > as
> > you noted, worsened, so I tried it in some other similar setups and
> > everything is great, so...).
> 
> They sort of recommend it now. But also not. It's complicated. [tm]
> 
> > My understanding is that several DSA members were opposed to using
> > SRS for @debian.org forwarding, but maybe it's now time?
> > 
> 
> That's essentially what's being worked on. But life, and free time,
> and
> other priorities, keep getting in the way.

An initial version, rewriting mails to Google-hosted domains from
"external" e-mail addresses (those for which debian.org's mail relays
don't consider themselves authoritative, so mostly not *.debian.org and
*.debconf.org) is now live.

Please let DSA know if you encounter any issues.

Regards,

Adam

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Marco d'Itri]

On Aug 14, Stephen Frost  wrote:

>If someone has some idea how to get them to care about ARC, I'd love to
>hear about it, as I have folks on the one hand who view DKIM/DMARC as
>too painful to set up but then they end up with bounces from gmail due
>to my forwarding of messages through my server (which are being
>ARC-signed by it and pass on that the SPF check was successful when they
>arrived to my server)...
I do not know of any situation in which DMARC adoption would improve
deliverability, and most people that configure it are just engaging in
cargo cult sysadmining.
DMARC with p=reject is useful when the sender domain is a phishing
victim, e.g. a financial organization, but most users do not need it.

In other words: if these people want to support use cases like
forwarding and participating to mailing lists then they should adopt
DKIM and ignore DMARC.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Stephen Frost]

Greetings,

* Cord Beermann (c...@debian.org) wrote:
> As listmaster i can confirm that it is a big problem to deliver Mails to
> gmail/outlook/yahoo. Yahoo Subscribers are mostly gone by now because they
> bounced a lot, for gmail it is so much that we just ignore bounces because of
> those rules. 

As a maintainer or some pretty big lists ... we don't have *that* much
trouble delivering to gmail, or others for that matter.

> | helgefjell.de descriptive text "v=spf1 ip4:142.132.201.35 mx ~all"
> 
> so you flagged your mail has to come from that IP (or the MX) and from other
> sources it should be considered suspicious.

... but if it's DKIM signed, then it'll generally get delivered
properly.

> SRS/ARC and so on are just dirty patches that try to fix things that were
> broken before, but they will break even more things like Mail signing.

ARC doesn't break DKIM signatures (unless someone's got a very broken
DKIM setup which over-signs ARC headers ... but if so, then that's on
them).

Thanks,

Stephen


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Stephen Frost]

Greetings,

* Mattia Rizzolo (mat...@debian.org) wrote:
> Alternatively, I wonder if ARC nowadays is respected enough (and if
> Google cares about it)... I personally don't have any system with ARC
> under my care.

Sadly, no, they don't seem to care one bit about ARC, except possibly if
it's their own ARC sigs.

If someone has some idea how to get them to care about ARC, I'd love to
hear about it, as I have folks on the one hand who view DKIM/DMARC as
too painful to set up but then they end up with bounces from gmail due
to my forwarding of messages through my server (which are being
ARC-signed by it and pass on that the SPF check was successful when they
arrived to my server)...

I'd encourage everyone running their own email servers to please get
DKIM/DMARC/ARC/SPF set up.  Yeah, it's annoying, but it's not actually
all *that* bad to do.

Thanks,

Stephen


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Cord Beermann]

Hallo! Du (Russ Allbery) hast geschrieben:

>The problem I suspect is with email forwarding, and specifically email
>forwarding to Gmail, which has recently ramped up the amount of
>verification it does on messages.  Because of email forwarding, Gmail sees
>a message purportedly from helgefjell.de but actually delivered by
>debian.org mail servers, and has now decided to be suspicious of that.

>If that's correct, you'll only have this problem with Debian developers
>who forward their @debian.org addresses to Gmail.  Gmail handles some
>large percentage of all email on the Internet, so this probably isn't
>rare, but Debian developers are less likely to use it than random Internet
>users for obvious reasons, so it doesn't surprise me you've not run into
>the problem before.  (In other words, I doubt this is a problem with your
>local configuration.)

As listmaster i can confirm that it is a big problem to deliver Mails to
gmail/outlook/yahoo. Yahoo Subscribers are mostly gone by now because they
bounced a lot, for gmail it is so much that we just ignore bounces because of
those rules. 

If you decide to handle your mails to be curated by someone else you have to
live with an incomplete mailbox. 

| helgefjell.de descriptive text "v=spf1 ip4:142.132.201.35 mx ~all"

so you flagged your mail has to come from that IP (or the MX) and from other
sources it should be considered suspicious.

Thats the result.

SRS/ARC and so on are just dirty patches that try to fix things that were
broken before, but they will break even more things like Mail signing.

As long as we have this Oligopol that doesn't care about what they send out
(i.e. Spamfloods through Outlook) things will only get worse.

Cord

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Adam D. Barratt]

On Sat, 2023-08-12 at 23:13 +0200, Mattia Rizzolo wrote:
> On Sat, Aug 12, 2023 at 01:41:46PM -0700, Russ Allbery wrote:
> > The problem I suspect is with email forwarding, and specifically
> > email
> > forwarding to Gmail, which has recently ramped up the amount of
> > verification it does on messages.  Because of email forwarding,
> > Gmail sees
> > a message purportedly from helgefjell.de but actually delivered by
> > debian.org mail servers, and has now decided to be suspicious of
> > that.
> 
> This is the exact use case that SRS was developer for, however
> gmail's documentation does not recommend that (but the situation, as
> you noted, worsened, so I tried it in some other similar setups and
> everything is great, so...).

They sort of recommend it now. But also not. It's complicated. [tm]

> My understanding is that several DSA members were opposed to using
> SRS for @debian.org forwarding, but maybe it's now time?
> 

That's essentially what's being worked on. But life, and free time, and
other priorities, keep getting in the way.

Regards,

Adam

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Mattia Rizzolo]

On Sat, Aug 12, 2023 at 01:41:46PM -0700, Russ Allbery wrote:
> The problem I suspect is with email forwarding, and specifically email
> forwarding to Gmail, which has recently ramped up the amount of
> verification it does on messages.  Because of email forwarding, Gmail sees
> a message purportedly from helgefjell.de but actually delivered by
> debian.org mail servers, and has now decided to be suspicious of that.

This is the exact use case that SRS was developer for, however gmail's
documentation does not recommend that (but the situation, as you noted,
worsened, so I tried it in some other similar setups and everything is
great, so...).
My understanding is that several DSA members were opposed to using SRS
for @debian.org forwarding, but maybe it's now time?

Alternatively, I wonder if ARC nowadays is respected enough (and if
Google cares about it)... I personally don't have any system with ARC
under my care.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
More about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Russ Allbery]

Helge Kreutzmann  writes:

> It's just that I never had this problem with mails to people with
> @debian.org addresses, so either my new configuration or some other
> change, I don't know.

The problem I suspect is with email forwarding, and specifically email
forwarding to Gmail, which has recently ramped up the amount of
verification it does on messages.  Because of email forwarding, Gmail sees
a message purportedly from helgefjell.de but actually delivered by
debian.org mail servers, and has now decided to be suspicious of that.

If that's correct, you'll only have this problem with Debian developers
who forward their @debian.org addresses to Gmail.  Gmail handles some
large percentage of all email on the Internet, so this probably isn't
rare, but Debian developers are less likely to use it than random Internet
users for obvious reasons, so it doesn't surprise me you've not run into
the problem before.  (In other words, I doubt this is a problem with your
local configuration.)

-- 
Russ Allbery (r...@debian.org)  

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Helge Kreutzmann]

Hello Russ,
Am Sat, Aug 12, 2023 at 11:31:35AM -0700 schrieb Russ Allbery:
> Helge Kreutzmann  writes:
> 
> > I don't know how it worked so far, and the error could be on my side, as
> > I recently switched my e-mail setup; however, I don't see anything I can
> > do to make DKIM/SPF point to @debian.org instead of @helgefjell.de, when
> > transferring e-mail to gmail.
> 
> The mail to which I'm resonding also comes from your @helgefjell.de
> domain, so I'm suspecting some DKIM/SPF issues there if you're using that
> same address in your original mail message.  But just in case you were

Yes, this is my primary e-mail address

> trying to send from your @debian.org address, one option is to send all of
> your outgoing mail that is from your debian.org address through the
> debian.org mail servers.  See:
> 
> https://dsa.debian.org/user/mail-submit/
> 
> I don't think this is the direct answer to your original question, but I
> suspect it would work around the problem.

Thanks for taking care, but I don't have an @debian.org address.

Greetings

 Helge

-- 
  Dr. Helge Kreutzmann deb...@helgefjell.de
   Dipl.-Phys.   http://www.helgefjell.de/debian.php
64bit GNU powered gpg signed mail preferred
   Help keep free software "libre": http://www.ffii.de/


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Helge Kreutzmann]

Hello Adam,
Am Sat, Aug 12, 2023 at 06:11:43PM +0100 schrieb Adam D. Barratt:
> On Sat, 2023-08-12 at 17:08 +, Helge Kreutzmann wrote:
> > Directly gmail accepts it.
> > 
> 
> I'm not sure why the sigh, but in any case your direct mail presumably
> succeeds because it passes the SPF check. I was simply clarifying that
> the DKIM check would fail in both cases.

Well, I did have trouble sending directly to gmail accounts, which now
seems to work. Now the next e-mail problem arises, which I need to see
how much I can configure it to work. That's the sigh.

It's just that I never had this problem with mails to people with
@debian.org addresses, so either my new configuration or some other
change, I don't know.

I hope this explains it a little.

Greetings

Helge

-- 
  Dr. Helge Kreutzmann deb...@helgefjell.de
   Dipl.-Phys.   http://www.helgefjell.de/debian.php
64bit GNU powered gpg signed mail preferred
   Help keep free software "libre": http://www.ffii.de/


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Russ Allbery]

Helge Kreutzmann  writes:

> I don't know how it worked so far, and the error could be on my side, as
> I recently switched my e-mail setup; however, I don't see anything I can
> do to make DKIM/SPF point to @debian.org instead of @helgefjell.de, when
> transferring e-mail to gmail.

The mail to which I'm resonding also comes from your @helgefjell.de
domain, so I'm suspecting some DKIM/SPF issues there if you're using that
same address in your original mail message.  But just in case you were
trying to send from your @debian.org address, one option is to send all of
your outgoing mail that is from your debian.org address through the
debian.org mail servers.  See:

https://dsa.debian.org/user/mail-submit/

I don't think this is the direct answer to your original question, but I
suspect it would work around the problem.

-- 
Russ Allbery (r...@debian.org)  

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Adam D. Barratt]

On Sat, 2023-08-12 at 17:08 +, Helge Kreutzmann wrote:
> Hello Adam,
> Am Sat, Aug 12, 2023 at 05:35:52PM +0100 schrieb Adam D. Barratt:
> > On Sat, 2023-08-12 at 15:54 +, Helge Kreutzmann wrote:
[...]
> > > 550-5.7.26 This mail is unauthenticated, which poses a
> > > security
> > > risk to the
> > > 550-5.7.26 sender and Gmail users, and has been blocked. The
> > > sender must
> > > 550-5.7.26 authenticate with at least one of SPF or DKIM. For
> > > this message,
> > > 550-5.7.26 DKIM checks did not pass and SPF check for
> > > [helgefjell.de] did not
> > > 
[...]
> > > 550-5.7.26  
> > > https://support.google.com/mail/answer/81126#authentication for
> > > 550 5.7.26 instructions on setting up authentication. v26-
> > > 20020aa7d65a00b005231f55294dsi4996663edr.385 - gsmtp
> > > 
> > > The IP 82.195.75.114 resolves to 
> > > 114.75.195.82.in-addr.arpa is an alias for 114.64-
> > > 26.75.195.82.in-
> > > addr.arpa.
> > > 114.64-26.75.195.82.in-addr.arpa domain name pointer
> > > mailly.debian.org.
> > > 
> > > And of course, SPF/DKIM checks for my domain (helgefjell.de) fail
> > > for this IP, which is @debian.org.
> > > 
> > 
> > The DKIM signature warning has nothing to do with the forwarding,
> > or the involvement of debian.org at all. The reason that check
> > fails is that your mail has no DKIM signature, so obviously can't
> > have a valid one. Signing your mail would probably make gmail a lot
> > happier with it in general. (As a side note, the BTS breaks many
> > common DKIM signature strategies, but that's a different issue.)
> 
> Sigh. 
> 
> Directly gmail accepts it.
> 

I'm not sure why the sigh, but in any case your direct mail presumably
succeeds because it passes the SPF check. I was simply clarifying that
the DKIM check would fail in both cases.

Regards,

Adam

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Helge Kreutzmann]

Hello Adam,
Am Sat, Aug 12, 2023 at 05:35:52PM +0100 schrieb Adam D. Barratt:
> On Sat, 2023-08-12 at 15:54 +, Helge Kreutzmann wrote:
> > If I try to mail e.g. Marcos Fouces , this no
> > longer works. I get the following error message:
> > 
> 
> Contacting DSA is generally a better way to ask about infrastructure
> things than filing bugs on high-level pseudo-packages.

Thanks, then I know this for the future. 

> > This message was created automatically by mail delivery software.
> > 
> > A message that you sent could not be delivered to one or more of its
> > recipients. This is a permanent error. The following address(es)
> > failed:
> > 
> >   marcos.fou...@gmail.com
> > host gmail-smtp-in.l.google.com [173.194.79.26]
> > SMTP error from remote mail server after pipelined end of data:
> > 550-5.7.26 This mail is unauthenticated, which poses a security
> > risk to the
> > 550-5.7.26 sender and Gmail users, and has been blocked. The
> > sender must
> > 550-5.7.26 authenticate with at least one of SPF or DKIM. For
> > this message,
> > 550-5.7.26 DKIM checks did not pass and SPF check for
> > [helgefjell.de] did not
> > 550-5.7.26 pass with ip: [82.195.75.114]. The sender should visit
> > 550-5.7.26  
> > https://support.google.com/mail/answer/81126#authentication for
> > 550 5.7.26 instructions on setting up authentication. v26-
> > 20020aa7d65a00b005231f55294dsi4996663edr.385 - gsmtp
> > 
> > The IP 82.195.75.114 resolves to 
> > 114.75.195.82.in-addr.arpa is an alias for 114.64-26.75.195.82.in-
> > addr.arpa.
> > 114.64-26.75.195.82.in-addr.arpa domain name pointer
> > mailly.debian.org.
> > 
> > And of course, SPF/DKIM checks for my domain (helgefjell.de) fail for
> > this IP, which is @debian.org.
> > 
> 
> The DKIM signature warning has nothing to do with the forwarding, or
> the involvement of debian.org at all. The reason that check fails is
> that your mail has no DKIM signature, so obviously can't have a valid
> one. Signing your mail would probably make gmail a lot happier with it
> in general. (As a side note, the BTS breaks many common DKIM signature
> strategies, but that's a different issue.)

Sigh. 

Directly gmail accepts it.

> The general issue is being worked on, as time and resources allow.

Thanks a lot!

Greetings

 Helge

-- 
  Dr. Helge Kreutzmann deb...@helgefjell.de
   Dipl.-Phys.   http://www.helgefjell.de/debian.php
64bit GNU powered gpg signed mail preferred
   Help keep free software "libre": http://www.ffii.de/


signature.asc
Description: PGP signature

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Adam D. Barratt]

Hi,

On Sat, 2023-08-12 at 15:54 +, Helge Kreutzmann wrote:
> If I try to mail e.g. Marcos Fouces , this no
> longer works. I get the following error message:
> 

Contacting DSA is generally a better way to ask about infrastructure
things than filing bugs on high-level pseudo-packages.

> This message was created automatically by mail delivery software.
> 
> A message that you sent could not be delivered to one or more of its
> recipients. This is a permanent error. The following address(es)
> failed:
> 
>   marcos.fou...@gmail.com
> host gmail-smtp-in.l.google.com [173.194.79.26]
> SMTP error from remote mail server after pipelined end of data:
> 550-5.7.26 This mail is unauthenticated, which poses a security
> risk to the
> 550-5.7.26 sender and Gmail users, and has been blocked. The
> sender must
> 550-5.7.26 authenticate with at least one of SPF or DKIM. For
> this message,
> 550-5.7.26 DKIM checks did not pass and SPF check for
> [helgefjell.de] did not
> 550-5.7.26 pass with ip: [82.195.75.114]. The sender should visit
> 550-5.7.26  
> https://support.google.com/mail/answer/81126#authentication for
> 550 5.7.26 instructions on setting up authentication. v26-
> 20020aa7d65a00b005231f55294dsi4996663edr.385 - gsmtp
> 
> The IP 82.195.75.114 resolves to 
> 114.75.195.82.in-addr.arpa is an alias for 114.64-26.75.195.82.in-
> addr.arpa.
> 114.64-26.75.195.82.in-addr.arpa domain name pointer
> mailly.debian.org.
> 
> And of course, SPF/DKIM checks for my domain (helgefjell.de) fail for
> this IP, which is @debian.org.
> 

The DKIM signature warning has nothing to do with the forwarding, or
the involvement of debian.org at all. The reason that check fails is
that your mail has no DKIM signature, so obviously can't have a valid
one. Signing your mail would probably make gmail a lot happier with it
in general. (As a side note, the BTS breaks many common DKIM signature
strategies, but that's a different issue.)

The general issue is being worked on, as time and resources allow.

Regards,

Adam
(part of, but not on behalf of, DSA)

Bug#1043539: project: Forwarding of @debian.org mails to gmail broken

[Helge Kreutzmann]

Package: project
Severity: important

If I try to mail e.g. Marcos Fouces , this no
longer works. I get the following error message:

This message was created automatically by mail delivery software.

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  marcos.fou...@gmail.com
host gmail-smtp-in.l.google.com [173.194.79.26]
SMTP error from remote mail server after pipelined end of data:
550-5.7.26 This mail is unauthenticated, which poses a security risk to the
550-5.7.26 sender and Gmail users, and has been blocked. The sender must
550-5.7.26 authenticate with at least one of SPF or DKIM. For this message,
550-5.7.26 DKIM checks did not pass and SPF check for [helgefjell.de] did 
not
550-5.7.26 pass with ip: [82.195.75.114]. The sender should visit
550-5.7.26  https://support.google.com/mail/answer/81126#authentication for
550 5.7.26 instructions on setting up authentication. 
v26-20020aa7d65a00b005231f55294dsi4996663edr.385 - gsmtp

The IP 82.195.75.114 resolves to 
114.75.195.82.in-addr.arpa is an alias for 114.64-26.75.195.82.in-addr.arpa.
114.64-26.75.195.82.in-addr.arpa domain name pointer mailly.debian.org.

And of course, SPF/DKIM checks for my domain (helgefjell.de) fail for
this IP, which is @debian.org.

I don't know how it worked so far, and the error could be on my side, as I 
recently switched my e-mail setup; however, I don't see anything I can
do to make DKIM/SPF point to @debian.org instead of @helgefjell.de,
when transferring e-mail to gmail.

Greetings

 Helge

-- 
  Dr. Helge Kreutzmann deb...@helgefjell.de
   Dipl.-Phys.   http://www.helgefjell.de/debian.php
64bit GNU powered gpg signed mail preferred
   Help keep free software "libre": http://www.ffii.de/


signature.asc
Description: PGP signature