[dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-04 Thread Juri Haberland via dmarc-discuss 
Hi,

while writing a patch for OpenDMARC, I stumbled accross problems with the
size limit in DMARC URIs that some of the big players have.

Microsoft cannot cope at all with an URI like "rep...@example.org!10m" -
you won't receive a single report.

Yahoo and Google do send a report and respect the size limit - as long as
this URI is the only one in the rua tag.
As soon as one adds another URI (with or without size limit) to the rua
tag, Google and Yahoo forget to strip the size limit from the URI and thus
try to send a mail to "rep...@example.org!10m"!

As OpenDMARC also had problems with the size limit in older versions, it is
best to avoid the use of size limits for now.


  Juri
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-12 Thread Juri Haberland via dmarc-discuss 

Hi,

I hoped to get a reaction here of some sort from Microsoft, Google or 
Yahoo,
but my mail might got burried underneath useless rants about DMARC and 
DNSSEC...


Btw: Did anyone notice that AOL sends DMARC reports with two To: 
headers?



Kind regards,
  Juri


On 2016-10-04 09:21, Juri Haberland via dmarc-discuss wrote:

Hi,

while writing a patch for OpenDMARC, I stumbled accross problems with 
the

size limit in DMARC URIs that some of the big players have.

Microsoft cannot cope at all with an URI like "rep...@example.org!10m" 
-

you won't receive a single report.

Yahoo and Google do send a report and respect the size limit - as long 
as

this URI is the only one in the rua tag.
As soon as one adds another URI (with or without size limit) to the rua
tag, Google and Yahoo forget to strip the size limit from the URI and 
thus

try to send a mail to "rep...@example.org!10m"!

As OpenDMARC also had problems with the size limit in older versions, 
it is

best to avoid the use of size limits for now.

___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-12 Thread Roland Turner via dmarc-discuss 
Consider https://www.ietf.org/mailman/listinfo/dmarc


- Roland




From: dmarc-discuss  on behalf of Juri 
Haberland via dmarc-discuss 
Sent: Wednesday, 12 October 2016 16:32
To: Juri Haberland
Cc: DMARC Discussion List
Subject: Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

Hi,

I hoped to get a reaction here of some sort from Microsoft, Google or
Yahoo,
but my mail might got burried underneath useless rants about DMARC and
DNSSEC...

Btw: Did anyone notice that AOL sends DMARC reports with two To:
headers?


Kind regards,
   Juri


On 2016-10-04 09:21, Juri Haberland via dmarc-discuss wrote:
> Hi,
>
> while writing a patch for OpenDMARC, I stumbled accross problems with
> the
> size limit in DMARC URIs that some of the big players have.
>
> Microsoft cannot cope at all with an URI like "rep...@example.org!10m"
> -
> you won't receive a single report.
>
> Yahoo and Google do send a report and respect the size limit - as long
> as
> this URI is the only one in the rua tag.
> As soon as one adds another URI (with or without size limit) to the rua
> tag, Google and Yahoo forget to strip the size limit from the URI and
> thus
> try to send a mail to "rep...@example.org!10m"!
>
> As OpenDMARC also had problems with the size limit in older versions,
> it is
> best to avoid the use of size limits for now.
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-12 Thread Steven M Jones via dmarc-discuss 
On 10/12/16 01:32, Juri Haberland via dmarc-discuss wrote:
>
> I hoped to get a reaction here of some sort from Microsoft, Google or
> Yahoo,
> but my mail might got burried underneath useless rants about DMARC and
> DNSSEC...

On 10/12/16 02:00, Roland Turner via dmarc-discuss wrote:
>
> Consider https://www.ietf.org/mailman/listinfo/dmarc
> 
>

+1.

There's another question to raise in the IETF working group - do we need
to re-consider the use of HTTPS as an alternative transport for reports?
(Background: HTTP was in the original spec, but hadn't been implemented,
and so was dropped several years ago.)

If we're running into the common size limits on email messages for
reports at the largest senders/receivers today, what should we be
planning for in five years? In ten? Maybe it's time to re-establish an
alternate channel in the spec, so it will be ready when we need it.


> Btw: Did anyone notice that AOL sends DMARC reports with two To: headers?

Looking at the last few reports I received from them for this domain, I
only see one 5322.To header. But the most recent report was
mid-September. Anybody else out there seeing two? It could make tracking
down a bug much easier for them.

I occasionally remind one of the bigger report senders that they're
always missing spaces in the 5322.Subject lines of their aggregate
reports, but it doesn't seem to get fixed...

--Steve.


___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-12 Thread Steven M Jones via dmarc-discuss 
On 10/12/16 03:17, Steven M Jones via dmarc-discuss wrote:
> On 10/12/16 02:00, Roland Turner via dmarc-discuss wrote:
>>
>> Consider https://www.ietf.org/mailman/listinfo/dmarc
>> 
>>
>
> +1.

Let me clarify a bit -- the dmarc-discuss list is very much an
appropriate forum for the kind of operational topic Juri raised.
Implementation issues, operational questions/issues, etc -- all good for
this list.

But for things that appear to be more than that, the IETF WG is a better
place to take them. And I think that if you consider current handling of
size limits, planning for growing report sizes in the near future, and
an additional report transport - all those together - it seems an
appropriate bundle to take to the WG.

--S.

___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-12 Thread Juri Haberland via dmarc-discuss 
On 12.10.2016 12:17, Steven M Jones via dmarc-discuss wrote:
> On 10/12/16 01:32, Juri Haberland via dmarc-discuss wrote:

>> Btw: Did anyone notice that AOL sends DMARC reports with two To: headers?
> 
> Looking at the last few reports I received from them for this domain, I
> only see one 5322.To header. But the most recent report was
> mid-September. Anybody else out there seeing two? It could make tracking
> down a bug much easier for them.

My last report is half a year old, but has two headers, too:

> From: abuse_dm...@abuse.aol.com
> To: r...@dmarc.sapienti-sat.org
> To: pboza...@ag.dmarcian.com

So it seems, AOL is putting every rua URI into a seperate To: header...

  Juri

___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-12 Thread Dave Crocker via dmarc-discuss 

On 10/12/2016 3:31 AM, Steven M Jones via dmarc-discuss wrote:

Let me clarify a bit -- the dmarc-discuss list is very much an
appropriate forum for the kind of operational topic Juri raised.
Implementation issues, operational questions/issues, etc -- all good for
this list.


Yup.



But for things that appear to be more than that, the IETF WG is a better
place to take them.


If it's likely a specification or even a BCP will be needed, the IETF 
list is where that needs to happen.


That said, it's not uncommon for an issue to first surface in a general 
list, such as dmarc-discuss, and only eventually get to the point where 
people decide it is going to need specification work.  At that point, of 
course, it migrates.


d/
--

  Dave Crocker
  Brandenburg InternetWorking
  bbiw.net
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread Alessandro Vesely via dmarc-discuss 

On Wed 12/Oct/2016 21:38:45 +0200 Juri Haberland via dmarc-discuss wrote:

On 12.10.2016 12:17, Steven M Jones via dmarc-discuss wrote:

On 10/12/16 01:32, Juri Haberland via dmarc-discuss wrote:



Btw: Did anyone notice that AOL sends DMARC reports with two To: headers?


Looking at the last few reports I received from them for this domain, I
only see one 5322.To header. But the most recent report was
mid-September. Anybody else out there seeing two? It could make tracking
down a bug much easier for them.


My last report is half a year old, but has two headers, too:

From: abuse_dm...@abuse.aol.com
To: r...@dmarc.sapienti-sat.org
To: pboza...@ag.dmarcian.com

So it seems, AOL is putting every rua URI into a seperate To: header...


I'm surprised no AOL people spoke, so I CC this to the address I found in their 
report_metadata/email field.


Instead, we could add an extra_contact_info entry pointing to this list, no?

Ale
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread Paul Rock via dmarc-discuss 
Sorry for not saying so earlier, but we're looking into the multiple to
thing. We'll roll out a fix asap.

On Thu, Oct 13, 2016 at 3:30 AM, Alessandro Vesely via dmarc-discuss <
dmarc-discuss@dmarc.org> wrote:

> On Wed 12/Oct/2016 21:38:45 +0200 Juri Haberland via dmarc-discuss wrote:
>
>> On 12.10.2016 12:17, Steven M Jones via dmarc-discuss wrote:
>>
>>> On 10/12/16 01:32, Juri Haberland via dmarc-discuss wrote:
>>>
>>
>> Btw: Did anyone notice that AOL sends DMARC reports with two To: headers?

>>>
>>> Looking at the last few reports I received from them for this domain, I
>>> only see one 5322.To header. But the most recent report was
>>> mid-September. Anybody else out there seeing two? It could make tracking
>>> down a bug much easier for them.
>>>
>>
>> My last report is half a year old, but has two headers, too:
>>
>> From: abuse_dm...@abuse.aol.com
>> To: r...@dmarc.sapienti-sat.org
>> To: pboza...@ag.dmarcian.com
>>
>> So it seems, AOL is putting every rua URI into a seperate To: header...
>>
>
> I'm surprised no AOL people spoke, so I CC this to the address I found in
> their report_metadata/email field.
>
> Instead, we could add an extra_contact_info entry pointing to this list,
> no?
>
> Ale
>
> ___
> dmarc-discuss mailing list
> dmarc-discuss@dmarc.org
> http://www.dmarc.org/mailman/listinfo/dmarc-discuss
>
> NOTE: Participating in this list means you agree to the DMARC Note Well
> terms (http://www.dmarc.org/note_well.html)
>



-- 
PAUL ROCK
Principal Software Engineer | AOL Mail
P: 703-265-5734 | C: 703-980-8380
AIM: paulsrock
22070 Broderick Dr.| Dulles, VA | 20166-9305
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread John Levine via dmarc-discuss 
>There's another question to raise in the IETF working group - do we need
>to re-consider the use of HTTPS as an alternative transport for reports?
>(Background: HTTP was in the original spec, but hadn't been implemented,
>and so was dropped several years ago.)
>
>If we're running into the common size limits on email messages for
>reports at the largest senders/receivers today, what should we be
>planning for in five years? In ten? Maybe it's time to re-establish an
>alternate channel in the spec, so it will be ready when we need it.

It's a poor idea to put stuff into a spec if nobody's planning to
implement it, because that generally results in a spec that doesn't
work when someone tries later.  The original http language was
hopelessly broken, so I offered something different that I think
would have worked, but nobody ever tested.

So if DMARC reports are getting too big, I'd be happy to resuscitate
the http language in a short draft to update RFC 7489, but only if
there are a few people who plan to implement each side of it so we can
be sure that it works.

Technically it's really simple, a single HTTP PUT operation which
is not as common as GET or POST, but should be supported by every
web server, and automagically provides for compression and duplicate
report elimination.

R's,
John

___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread Matt Simerson via dmarc-discuss 
Whoah there!

This thread has been hijacked by the lack of reading comprehension. Nobody (in 
this thread) has complained of DMARC reports being too large.

The problem in this thread is an issue with some DMARC report senders failing 
to parse the DMARC URIs properly, if that DMARC URI includes size limits.

I now return you to our normally scheduled programming.

Matt

> On Oct 13, 2016, at 10:53 AM, John Levine via dmarc-discuss 
>  wrote:
> 
>> There's another question to raise in the IETF working group - do we need
>> to re-consider the use of HTTPS as an alternative transport for reports?
>> (Background: HTTP was in the original spec, but hadn't been implemented,
>> and so was dropped several years ago.)
>> 
>> If we're running into the common size limits on email messages for
>> reports at the largest senders/receivers today, what should we be
>> planning for in five years? In ten? Maybe it's time to re-establish an
>> alternate channel in the spec, so it will be ready when we need it.
> 
> It's a poor idea to put stuff into a spec if nobody's planning to
> implement it, because that generally results in a spec that doesn't
> work when someone tries later.  The original http language was
> hopelessly broken, so I offered something different that I think
> would have worked, but nobody ever tested.
> 
> So if DMARC reports are getting too big, I'd be happy to resuscitate
> the http language in a short draft to update RFC 7489, but only if
> there are a few people who plan to implement each side of it so we can
> be sure that it works.
> 
> Technically it's really simple, a single HTTP PUT operation which
> is not as common as GET or POST, but should be supported by every
> web server, and automagically provides for compression and duplicate
> report elimination.
> 
> R's,
> John
> 
> ___
> dmarc-discuss mailing list
> dmarc-discuss@dmarc.org
> http://www.dmarc.org/mailman/listinfo/dmarc-discuss
> 
> NOTE: Participating in this list means you agree to the DMARC Note Well terms 
> (http://www.dmarc.org/note_well.html)


___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread Brandon Long via dmarc-discuss 
Actually, from the code, I'm surprised we handle a single address with !
correctly.  I'll file a bug.

Brandon

On Tue, Oct 4, 2016 at 12:21 AM, Juri Haberland via dmarc-discuss <
dmarc-discuss@dmarc.org> wrote:

> Hi,
>
> while writing a patch for OpenDMARC, I stumbled accross problems with the
> size limit in DMARC URIs that some of the big players have.
>
> Microsoft cannot cope at all with an URI like "rep...@example.org!10m" -
> you won't receive a single report.
>
> Yahoo and Google do send a report and respect the size limit - as long as
> this URI is the only one in the rua tag.
> As soon as one adds another URI (with or without size limit) to the rua
> tag, Google and Yahoo forget to strip the size limit from the URI and thus
> try to send a mail to "rep...@example.org!10m"!
>
> As OpenDMARC also had problems with the size limit in older versions, it is
> best to avoid the use of size limits for now.
>
>
>   Juri
> ___
> dmarc-discuss mailing list
> dmarc-discuss@dmarc.org
> http://www.dmarc.org/mailman/listinfo/dmarc-discuss
>
> NOTE: Participating in this list means you agree to the DMARC Note Well
> terms (http://www.dmarc.org/note_well.html)
>
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread Juri Haberland via dmarc-discuss 

On 2016-10-14 00:26, Brandon Long wrote:
Actually, from the code, I'm surprised we handle a single address with 
!

correctly.  I'll file a bug.


Thanks, Brandon!

  Juri
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-13 Thread Juri Haberland via dmarc-discuss 

On 2016-10-13 20:06, Matt Simerson via dmarc-discuss wrote:


This thread has been hijacked by the lack of reading comprehension.
Nobody (in this thread) has complained of DMARC reports being too
large.


Right.


The problem in this thread is an issue with some DMARC report senders
failing to parse the DMARC URIs properly, if that DMARC URI includes
size limits.


Right again. That's why I hesitated to re-post my findings on the IETF 
dmarc list.


For what it's worth, the largest report I ever got is ~2kB (compressed, 
46kB uncompressed), but I run only a small system with a handful of 
users and lists. Would be interesting to hear what sizes larger sites 
receive (or send), but I doubt it gets into the region of ~1MB 
(compressed) - if the sender has a decent implementation (which 
OpenDMARC currently has not).


So again: Some report senders do not parse reporting URIs correct - 
please check your implementations... That was my point.



Cheers,
  Juri
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)

Re: [dmarc-discuss] Beware of the size limit in DMARC URIs

2016-10-14 Thread Alessandro Vesely via dmarc-discuss 

On Fri 14/Oct/2016 08:37:08 +0200 Juri Haberland via dmarc-discuss wrote:

On 2016-10-13 20:06, Matt Simerson via dmarc-discuss wrote:


The problem in this thread is an issue with some DMARC report senders
failing to parse the DMARC URIs properly, if that DMARC URI includes
size limits.


Right again. That's why I hesitated to re-post my findings on the IETF dmarc 
list.


Please, don't hesitate.  We need to fix bugs, and they're difficult to find in 
a fully automated system where people seldom watch gearings directly.



PROPOSAL:
=

Let's set up a DMARC receiver whose only job is to check the correctness of 
DMARC aggregate reports.  Size limits is just one of the due tests.  For more, 
external addresses, reporting period determination in the face of changed ri= 
tag, compression (.zip vs .gz), authentication, and, last but not least, 
accuracy of data.


I'm willing to dedicate one or two domains to sending a few mails per day just 
to check if/how/what reports get delivered.  If someone volunteers for the main 
job, verification and selection of target domains, that is.  Anyone?



For what it's worth, the largest report I ever got is ~2kB (compressed, 46kB
uncompressed), but I run only a small system with a handful of users and lists.
Would be interesting to hear what sizes larger sites receive (or send), but I
doubt it gets into the region of ~1MB (compressed) - if the sender has a decent
implementation (which OpenDMARC currently has not).


Agreed, correct aggregation has to be tested too.


So again: Some report senders do not parse reporting URIs correct - please
check your implementations... That was my point.


I wrote a C program to send reports, zaggregate[1].  Possibly, I'm its only 
user.  At any rate, I never found a bug in it.  How come?!?  How can I check?


Ale


--
[1] if you're curious:
http://www.tana.it/sw/zdkimfilter/zaggregate.html
http://www.tana.it/svn/zdkimfilter/trunk/src/zaggregate.c
http://www.tana.it/sw/zdkimfilter/database.html
http://www.tana.it/svn/zdkimfilter/trunk/odbx_example.conf
___
dmarc-discuss mailing list
dmarc-discuss@dmarc.org
http://www.dmarc.org/mailman/listinfo/dmarc-discuss

NOTE: Participating in this list means you agree to the DMARC Note Well terms 
(http://www.dmarc.org/note_well.html)