Re: PEM of root certs in Mozilla's root store

2020-10-19 Thread Jakob Bohm via dev-security-policy 

On 2020-10-17 01:38, Ryan Sleevi wrote:

On Fri, Oct 16, 2020 at 5:27 PM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:


RFC4180 section 3 explicitly warns that there are other variants and
specifications of the CSV format, and thus the full generalizations in
RFC4180 should not be exploited to their extremes.



You're referring to this section, correct?

"""
Interoperability considerations:
   Due to lack of a single specification, there are considerable
   differences among implementations.  Implementors should "be
   conservative in what you do, be liberal in what you accept from
   others" (RFC 793 [8]) when processing CSV files.  An attempt at a
   common definition can be found in Section 2.

   Implementations deciding not to use the optional "header"
   parameter must make their own decision as to whether the header is
   absent or present.
"""



Splitting the input at newlines before parsing for quotes and commas is
a pretty common implementation strategy as illustrated by my examples of
common tools that actually do so.



This would appear to be at fundamental odds with "be liberal in what you
accept from others" and, more specifically, ignoring the remark that
Section 2 is an admirable effort at a "common" definition, which is so
called as it minimizes such interoperability differences.

As your original statement was the file produced was "not CSV", I believe
that's been thoroughly dispelled by highlighting that, indeed, it does
conform to the grammar set forward in RFC 4180, and is consistent with the
IANA mime registration for CSV.

Although you also raised concern that naive and ill-informed attempts at
CSV parsing, which of course fail to parse the grammar of RFC 4180, there
are thankfully alternatives for each of those concerns. With awk, you have
FPAT. With Perl, you have Text::CSV. Of course, as the cut command is too
primitive to handle a proper grammar, there are plenty of equally
reasonable alternatives to this, and which take far less time than the
concerns and misstatements raised on this thread, which I believe we can,
thankfully, end.



Please stop trolling, the section you quoted clearly states that section 
2 of the RFC was only an *attempt* at a common definition.


Ideally, a CSV parser should be liberal in what it accepts, but a CSV 
producer (which is the subject of this thread) should be conservative in 
what it provides, thus avoiding the least implemented/tested aspects

of the generalized grammar in that section 2.

Putting line feeds inside CSV fields is like using bang paths in an
RFC822 To: field.  Theoretically permitted and implemented by the most
complete e-mail parsing libraries in the world, but not something that
one can expect to work for a global audience.



Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-16 Thread Ryan Sleevi via dev-security-policy 
On Fri, Oct 16, 2020 at 5:27 PM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> RFC4180 section 3 explicitly warns that there are other variants and
> specifications of the CSV format, and thus the full generalizations in
> RFC4180 should not be exploited to their extremes.
>

You're referring to this section, correct?

"""
   Interoperability considerations:
  Due to lack of a single specification, there are considerable
  differences among implementations.  Implementors should "be
  conservative in what you do, be liberal in what you accept from
  others" (RFC 793 [8]) when processing CSV files.  An attempt at a
  common definition can be found in Section 2.

  Implementations deciding not to use the optional "header"
  parameter must make their own decision as to whether the header is
  absent or present.
"""


> Splitting the input at newlines before parsing for quotes and commas is
> a pretty common implementation strategy as illustrated by my examples of
> common tools that actually do so.


This would appear to be at fundamental odds with "be liberal in what you
accept from others" and, more specifically, ignoring the remark that
Section 2 is an admirable effort at a "common" definition, which is so
called as it minimizes such interoperability differences.

As your original statement was the file produced was "not CSV", I believe
that's been thoroughly dispelled by highlighting that, indeed, it does
conform to the grammar set forward in RFC 4180, and is consistent with the
IANA mime registration for CSV.

Although you also raised concern that naive and ill-informed attempts at
CSV parsing, which of course fail to parse the grammar of RFC 4180, there
are thankfully alternatives for each of those concerns. With awk, you have
FPAT. With Perl, you have Text::CSV. Of course, as the cut command is too
primitive to handle a proper grammar, there are plenty of equally
reasonable alternatives to this, and which take far less time than the
concerns and misstatements raised on this thread, which I believe we can,
thankfully, end.

Enjoy

Ryan
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-16 Thread Jakob Bohm via dev-security-policy 

On 2020-10-16 14:11, Ryan Sleevi wrote:

On Thu, Oct 15, 2020 at 7:44 PM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:


On 2020-10-15 11:57, Ryan Sleevi wrote:

On Thu, Oct 15, 2020 at 1:14 AM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:


For example, embedded new lines are discussed in 2.6 and the ABNF

therein.




The one difference from RFC4180 is that CR and LF are not part of the
alternatives for the inner part of "escaped".



Again, it would do a lot of benefit for everyone if you would be more
precise here.

For example, it seems clear and unambiguous that what you just stated is
factually wrong, because:

escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE



I was stating the *difference* from RFC4180 being precisely that
"simple, traditional CSV" doesn't accept the CR and LF alternatives in
that syntax production.



Ah, that would explain my confusion: you’re using “CSV” in a manner
different than what is widely understood and standardized. The complaint
about newlines would be as technically accurate and relevant as a complaint
that “simple, traditional certificates should parse as JSON” or that
“simple, traditional HTTP should be delivered over port 23”; which is to
say, it seems like this concern is not relevant.

As the CSVs comply with RFC 4180, which is widely recognized as what “CSV”
means, I think Jakob’s concern here can be disregarded. Any implementation
having trouble with the CSVs produced is confused about what a CSV is, and
thus not a CSV parser.



RFC4180 section 3 explicitly warns that there are other variants and 
specifications of the CSV format, and thus the full generalizations in 
RFC4180 should not be exploited to their extremes.


Splitting the input at newlines before parsing for quotes and commas is
a pretty common implementation strategy as illustrated by my examples of
common tools that actually do so.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-16 Thread Ryan Sleevi via dev-security-policy 
On Thu, Oct 15, 2020 at 7:44 PM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> On 2020-10-15 11:57, Ryan Sleevi wrote:
> > On Thu, Oct 15, 2020 at 1:14 AM Jakob Bohm via dev-security-policy <
> > dev-security-policy@lists.mozilla.org> wrote:
> >
> >>> For example, embedded new lines are discussed in 2.6 and the ABNF
> >> therein.
> >>>
> >>
> >> The one difference from RFC4180 is that CR and LF are not part of the
> >> alternatives for the inner part of "escaped".
> >
> >
> > Again, it would do a lot of benefit for everyone if you would be more
> > precise here.
> >
> > For example, it seems clear and unambiguous that what you just stated is
> > factually wrong, because:
> >
> > escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE
> >
>
> I was stating the *difference* from RFC4180 being precisely that
> "simple, traditional CSV" doesn't accept the CR and LF alternatives in
> that syntax production.


Ah, that would explain my confusion: you’re using “CSV” in a manner
different than what is widely understood and standardized. The complaint
about newlines would be as technically accurate and relevant as a complaint
that “simple, traditional certificates should parse as JSON” or that
“simple, traditional HTTP should be delivered over port 23”; which is to
say, it seems like this concern is not relevant.

As the CSVs comply with RFC 4180, which is widely recognized as what “CSV”
means, I think Jakob’s concern here can be disregarded. Any implementation
having trouble with the CSVs produced is confused about what a CSV is, and
thus not a CSV parser.
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-15 Thread Jakob Bohm via dev-security-policy 

On 2020-10-15 11:57, Ryan Sleevi wrote:

On Thu, Oct 15, 2020 at 1:14 AM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:


For example, embedded new lines are discussed in 2.6 and the ABNF

therein.




The one difference from RFC4180 is that CR and LF are not part of the
alternatives for the inner part of "escaped".



Again, it would do a lot of benefit for everyone if you would be more
precise here.

For example, it seems clear and unambiguous that what you just stated is
factually wrong, because:

escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE



I was stating the *difference* from RFC4180 being precisely that
"simple, traditional CSV" doesn't accept the CR and LF alternatives in
that syntax production.



Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-15 Thread Ryan Sleevi via dev-security-policy 
On Thu, Oct 15, 2020 at 1:14 AM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> > For example, embedded new lines are discussed in 2.6 and the ABNF
> therein.
> >
>
> The one difference from RFC4180 is that CR and LF are not part of the
> alternatives for the inner part of "escaped".


Again, it would do a lot of benefit for everyone if you would be more
precise here.

For example, it seems clear and unambiguous that what you just stated is
factually wrong, because:

escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-14 Thread Jakob Bohm via dev-security-policy 

On 2020-10-15 04:52, Ryan Sleevi wrote:

On Wed, Oct 14, 2020 at 7:31 PM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:


Only the CSV form now contains CSV artifacts.  And it isn't really CSV
either (even if Microsoft Excel handles it).



Hi Jakob,

Could you be more precise here? Embedded new lines within quoted values is
well-defined for CSV.

Realizing that there are unfortunately many interpretations of what
“well-defined for CSV” here, perhaps you can frame your concerns in terms
set out in
https://tools.ietf.org/html/rfc4180 . This at least helps make sure we can
understand and are in the same understanding.

For example, embedded new lines are discussed in 2.6 and the ABNF therein.



The one difference from RFC4180 is that CR and LF are not part of the
alternatives for the inner part of "escaped".




Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-14 Thread Ryan Sleevi via dev-security-policy 
On Wed, Oct 14, 2020 at 7:31 PM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> Only the CSV form now contains CSV artifacts.  And it isn't really CSV
> either (even if Microsoft Excel handles it).


Hi Jakob,

Could you be more precise here? Embedded new lines within quoted values is
well-defined for CSV.

Realizing that there are unfortunately many interpretations of what
“well-defined for CSV” here, perhaps you can frame your concerns in terms
set out in
https://tools.ietf.org/html/rfc4180 . This at least helps make sure we can
understand and are in the same understanding.

For example, embedded new lines are discussed in 2.6 and the ABNF therein.
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-14 Thread Jakob Bohm via dev-security-policy 

On 2020-10-15 00:16, Kathleen Wilson wrote:
The text version has been updated to have each line limited to 64 
characters.


Text:

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Websites 



https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Email 



CSV:
https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Websites 



https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Email 




1. The reports that contain /only/ PEM data.  I argue that the
traditional format of concatenated PEM files (as used by e.g. the
openssl command line tool) without CSV embellishments would be
preferable, and that the reports in the latest post by Kathleen
lacked the PEM line wrapping while still containing CSV
artifacts. 


Jakob, please explain what you mean by "still containing CSV artifacts". 
Does the new version of the text report have the problem?




Only the CSV form now contains CSV artifacts.  And it isn't really CSV
either (even if Microsoft Excel handles it).




2. The reports that contain other data in CSV format.  ...


Opening the CSV version of the reports in Excel and Numbers works fine 
for me. So I don't understand what the problem is with having this 
report be a direct extract of the PEM data that the CCADB has for each 
root certificate.




However, it might also be reasonable that if these concerns aren't easily
addressable, perhaps not offering the feed is another option, since it
seems a lot of work for something that should be and is naturally
discouraged.


It's not a lot of work, just a matter of understanding what is most 
useful to folks.


I think it would be good to have downstreams using current data, e.g. 
data published directly via the CCADB. I think that providing the data 
in an easily consumable format is better than having folks extract the 
data from certdata.txt.


Thanks,
Kathleen




Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-14 Thread Kathleen Wilson via dev-security-policy 
The text version has been updated to have each line limited to 64 
characters.


Text:

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Websites

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Email

CSV:
https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Websites

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Email


1. The reports that contain /only/ PEM data.  I argue that the
traditional format of concatenated PEM files (as used by e.g. the
openssl command line tool) without CSV embellishments would be
preferable, and that the reports in the latest post by Kathleen
lacked the PEM line wrapping while still containing CSV
artifacts. 


Jakob, please explain what you mean by "still containing CSV artifacts". 
Does the new version of the text report have the problem?




2. The reports that contain other data in CSV format.  ...


Opening the CSV version of the reports in Excel and Numbers works fine 
for me. So I don't understand what the problem is with having this 
report be a direct extract of the PEM data that the CCADB has for each 
root certificate.




However, it might also be reasonable that if these concerns aren't easily
addressable, perhaps not offering the feed is another option, since it
seems a lot of work for something that should be and is naturally
discouraged.


It's not a lot of work, just a matter of understanding what is most 
useful to folks.


I think it would be good to have downstreams using current data, e.g. 
data published directly via the CCADB. I think that providing the data 
in an easily consumable format is better than having folks extract the 
data from certdata.txt.


Thanks,
Kathleen

___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-14 Thread Jakob Bohm via dev-security-policy 

On 2020-10-13 16:32, Ryan Sleevi wrote:

Jakob,

I had a little trouble following your mail, despite being quite familiar
with PEM, so hopefully you'll indulge me in making sure I've got your
criticisms/complaints correct.

Your objection to the text report is that RFC 7468 requires generators to
wrap lines (except the last line) at exactly 64 characters, right? That is,
the textual report includes the base-64 data with no embedded newlines, and
this causes your PEM decoder trouble, despite being able to easily inject
them programmatically after you download the file.



I was commenting on the /general/ usability of the reports mentioned in 
the first message on this thread, by considering what naive parsers 
would do upon reading the files.  As I have no direct need to parse 
these files, I have no actual parser failing to do so.


My comments fell in two categories:

1. The reports that contain /only/ PEM data.  I argue that the
traditional format of concatenated PEM files (as used by e.g. the
openssl command line tool) without CSV embellishments would be
preferable, and that the reports in the latest post by Kathleen
lacked the PEM line wrapping while still containing CSV
artifacts.

2. The reports that contain other data in CSV format.  I argue that
those reports would be more useful without in-field line breaks, thus
having the Base64 encoded certificates as long strings without PEM
embellishments.  Goal is to make them traditional CSV files with one 
record per line, commas only between fields and optional double quotes

around non-numerical field values.  A sample parser would be awk, cut or
the perl command line option "-F,".

Simply viewing each report in a basic text viewer should make the 
problematic format deviations clear.




I'm not sure I fully understand the CSV objection, despite having inspected
the file, so perhaps you can clarify a bit more.

Perhaps the simplest approach would be that you could attach versions that
look how you'd want.



Here are the top 3 lines of IncludedCACertificateWithPEMReport.csv
reformatted (but with extra line breaks every 68 chars for posting
purposes):

"Owner","Certificate Issuer Organization","Certificate Issuer Organ
izational Unit","Common Name or Certificate Name","Certificate Seri
al Number","SHA-256 Fingerprint","Subject + SPKI SHA256","Valid Fro
m [GMT]","Valid To [GMT]","Public Key Algorithm","Signature Hash Al
gorithm","Trust Bits","Distrust for TLS After Date","Distrust for S
/MIME After Date","EV Policy OID(s)","Approval Bug","NSS Release Wh
en First Included","Firefox Release When First Included","Test Webs
ite - Valid","Test Website - Expired","Test Website - Revoked","Moz
illa Applied Constraints","Company Website","Geographic Focus","Cer
tificate Policy (CP)","Certification Practice Statement (CPS)","Sta
ndard Audit","BR Audit","EV Audit","Auditor","Standard Audit Type",
"Standard Audit Statement Dt","PEM Info"
"AC Camerfirma, S.A.","AC Camerfirma SA CIF A82743287","http://www.
chambersign.org","Chambers of Commerce Root","00","0C258A12A5674AEF
25F28BA7DCFAECEEA348E541E6F5CC4EE63B71B361606AC3","BC2FD9EA61581CB2
2BB859690D61430E7D222D1119E8C41649B9B1D556D439A4","2003.09.30","203
7.09.30","RSA 2048 bits","SHA1WithRSA","Email","","","Not EV","http
s://bugzilla.mozilla.org/show_bug.cgi?id=261778","","Firefox 1","",
"","","","http://www.camerfirma.com","Spain","","https://www.camerf
irma.com/publico/DocumentosWeb/politicas/CPS_eidas_EN_1.2.12.pdf","
https://www.csqa.it/getattachment/Sicurezza-ICT/Documenti/Attestazi
one-di-Audit-secondo-i-requisiti-ETSI/2020-03-CSQA-Attestation-CAME
RFIRMA-rev-2-signed.pdf.aspx?lang=it-IT","https://bugzilla.mozilla.
org/attachment.cgi?id=8995930","","CSQA Certificazioni srl","ETSI E
N 319 411","2020.03.05","MIIEvTCCA6WgAwIBAgIBADANBgkqhkiG9w0BAQUFAD
B/MQswCQYDVQQGEwJFVTEnMCUGA1UEChMeQUMgQ2FtZXJmaXJtYSBTQSBDSUYgQTgyN
zQzMjg3MSMwIQYDVQQLExpodHRwOi8vd3d3LmNoYW1iZXJzaWduLm9yZzEiMCAGA1UE
AxMZQ2hhbWJlcnMgb2YgQ29tbWVyY2UgUm9vdDAeFw0wMzA5MzAxNjEzNDNaFw0zNzA
5MzAxNjEzNDRaMH8xCzAJBgNVBAYTAkVVMScwJQYDVQQKEx5BQyBDYW1lcmZpcm1hIF
NBIENJRiBBODI3NDMyODcxIzAhBgNVBAsTGmh0dHA6Ly93d3cuY2hhbWJlcnNpZ24ub
3JnMSIwIAYDVQQDExlDaGFtYmVycyBvZiBDb21tZXJjZSBSb290MIIBIDANBgkqhkiG
9w0BAQEFAAOCAQ0AMIIBCAKCAQEAtzZV5aVdGDDg2olUkfzIx1L4L1DZ77F1c2VHfRt
bunXF/KGIJPov7coISjlUxFF6tdpg6jg8gbLL8bvZkSM/SAFwdakFKq0fcfPJVD0dBm
pAPrMMhe5cG3nCYsS4No41XQEMIwRHNaqbYE6gZj3LJgqcQKH0XZi/caulAGgq7YN6D
6IUtdQis4CwPAxaUWktWBiP7Zme8a7ileb2R6jWDA+wWFjbw2Y3npuRVDM30pQcakjJ
yfKl2qUMI/cjDpwyVV5xnIQFUZot/eZOKjRa3spAN2cMVCFVd9oKDMyXroDclDZK9D7
ONhMeU+SsTjoF7Nuucpw4i9A5O4kKPnf+dQIBA6OCAUQwggFAMBIGA1UdEwEB/wQIMA
YBAf8CAQwwPAYDVR0fBDUwMzAxoC+gLYYraHR0cDovL2NybC5jaGFtYmVyc2lnbi5vc
mcvY2hhbWJlcnNyb290LmNybDAdBgNVHQ4EFgQU45T1sU3p26EpW1eLTXYGduHRooow
DgYDVR0PAQH/BAQDAgEGMBEGCWCGSAGG+EIBAQQEAwIABzAnBgNVHREEIDAegRxjaGF
tYmVyc3Jvb3RAY2hhbWJlcnNpZ24ub3JnMCcGA1UdEgQgMB6BHGNoYW1iZXJzcm9vdE
BjaGFtYmVyc2lnbi5vcmcwWAYDVR0gBFEwTzBNBgsrBgEEAYGHLgoDATA+MDwGCCsGA

Re: PEM of root certs in Mozilla's root store

2020-10-13 Thread Ryan Sleevi via dev-security-policy 
Jakob,

I had a little trouble following your mail, despite being quite familiar
with PEM, so hopefully you'll indulge me in making sure I've got your
criticisms/complaints correct.

Your objection to the text report is that RFC 7468 requires generators to
wrap lines (except the last line) at exactly 64 characters, right? That is,
the textual report includes the base-64 data with no embedded newlines, and
this causes your PEM decoder trouble, despite being able to easily inject
them programmatically after you download the file.

I'm not sure I fully understand the CSV objection, despite having inspected
the file, so perhaps you can clarify a bit more.

Perhaps the simplest approach would be that you could attach versions that
look how you'd want.

However, it might also be reasonable that if these concerns aren't easily
addressable, perhaps not offering the feed is another option, since it
seems a lot of work for something that should be and is naturally
discouraged.

On Tue, Oct 13, 2020 at 3:26 AM Jakob Bohm via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> On 2020-10-12 20:50, Kathleen Wilson wrote:
> > On 10/7/20 1:09 PM, Jakob Bohm wrote:
> >> Please note that at least the first CSV download is not really a CSV
> >> file, as there are line feeds within each "PEM" value, and only one
> >> column.  It would probably be more useful as a simple concatenated PEM
> >> file, as used by various software packages as a root store input format.
> >>
> >
> >
> > Here's updated reports...
> >
> > Text:
> >
> >
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Websites
> >
> >
> >
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Email
> >
> These two are bad multi-pem files, as each certificate likes the
> required line wrapping.
>
> The useful multi-pem format would keep the line wrapping from the
> old report, but not insert stray quotes and commas.
>
> >
> > CSV:
> >
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Websites
> >
> >
> >
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Email
> >
>
> These two are like the old bad multi-pem reports, with the stray
> internal line feeds after/before the - marker lines, but without the
> internal linefeeds in the Base64 data.
>
> Useful actual-csv reports like IncludedCACertificateWithPEMReport.csv
> would have to omit the marker lines and their linefeeds as well as the
> internal linefeeds in the Base64 data in order to keep each CSV record
> entirely on one line.
>
>
>
> >
> >
> > If the Text reports look good, I'll add them to the wiki page,
> > https://wiki.mozilla.org/CA/Included_Certificates
> >
> >
> > Thanks,
> > Kathleen
> >
>
>
> Enjoy
>
> Jakob
> --
> Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
> Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
> This public discussion message is non-binding and may contain errors.
> WiseMo - Remote Service Management for PCs, Phones and Embedded
> ___
> dev-security-policy mailing list
> dev-security-policy@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-13 Thread Jakob Bohm via dev-security-policy 

On 2020-10-12 20:50, Kathleen Wilson wrote:

On 10/7/20 1:09 PM, Jakob Bohm wrote:
Please note that at least the first CSV download is not really a CSV 
file, as there are line feeds within each "PEM" value, and only one 
column.  It would probably be more useful as a simple concatenated PEM 
file, as used by various software packages as a root store input format.





Here's updated reports...

Text:

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Websites 



https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Email 


These two are bad multi-pem files, as each certificate likes the
required line wrapping.

The useful multi-pem format would keep the line wrapping from the
old report, but not insert stray quotes and commas.



CSV:
https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Websites 



https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Email 



These two are like the old bad multi-pem reports, with the stray
internal line feeds after/before the - marker lines, but without the
internal linefeeds in the Base64 data.

Useful actual-csv reports like IncludedCACertificateWithPEMReport.csv
would have to omit the marker lines and their linefeeds as well as the
internal linefeeds in the Base64 data in order to keep each CSV record
entirely on one line.






If the Text reports look good, I'll add them to the wiki page, 
https://wiki.mozilla.org/CA/Included_Certificates



Thanks,
Kathleen




Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-12 Thread Kathleen Wilson via dev-security-policy 

On 10/7/20 1:09 PM, Jakob Bohm wrote:
Please note that at least the first CSV download is not really a CSV 
file, as there are line feeds within each "PEM" value, and only one 
column.  It would probably be more useful as a simple concatenated PEM 
file, as used by various software packages as a root store input format.





Here's updated reports...

Text:

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Websites

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMTxt?TrustBitsInclude=Email

CSV:
https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Websites

https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEMCSV?TrustBitsInclude=Email


If the Text reports look good, I'll add them to the wiki page, 
https://wiki.mozilla.org/CA/Included_Certificates



Thanks,
Kathleen

___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-07 Thread Kathleen Wilson via dev-security-policy 

On 10/7/20 9:30 AM, Matthew Hardeman wrote:

Would it be unreasonable to also consider publishing, as an "easy to use"
list, that set of only those anchors which are currently trusted in the
program and for which no exceptional in-product policy enforcement is
imposed?  (TLD constraints, provisional distrusts, etc.)

The lazier implementers are going to take the raw set of anchors and none
of the policy associated, and so the default assumption should be that none
of the enhanced policy enforcements from nss or firefox would get copied
along.




These reports are automatically generated by CCADB (Salesforce), so I 
cannot filter out all of the exceptions that may occur or that are 
currently listed in https://wiki.mozilla.org/CA/Additional_Trust_Changes


I could add a report that filters out root certificates that are 
name-constrained. However, there is currently only one name-constrained 
included root cert, and this option ended up not being very popular 
among CAs requesting root inclusion.


Also note that in Mozilla's program being name-constrained does not 
release the CA from following the same rules that all of the other CAs 
have to follow.


Therefore, I'm not currently inclined to add another report to filter 
out name-constrained root certs (currently just the one root cert).


Thanks,
Kathleen



___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-07 Thread Kathleen Wilson via dev-security-policy 

On 10/6/20 7:09 PM, Ryan Sleevi wrote:

It seems like there should be a link to
https://wiki.mozilla.org/CA/FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F
there



I added that link to https://wiki.mozilla.org/CA/Included_Certificates

Thanks,
Kathleen

___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-07 Thread Jakob Bohm via dev-security-policy 

On 2020-10-06 23:47, Kathleen Wilson wrote:

All,

I've been asked to publish Mozilla's root store in a way that is easy to 
consume by downstreams, so I have added the following to 
https://wiki.mozilla.org/CA/Included_Certificates


CCADB Data Usage Terms


PEM of Root Certificates in Mozilla's Root Store with the Websites 
(TLS/SSL) Trust Bit Enabled (CSV)
 



PEM of Root Certificates in Mozilla's Root Store with the Email (S/MIME) 
Trust Bit Enabled (CSV)
 




Please let me know if you have feedback or recommendations about this.



Please note that at least the first CSV download is not really a CSV 
file, as there are line feeds within each "PEM" value, and only one 
column.  It would probably be more useful as a simple concatenated PEM 
file, as used by various software packages as a root store input format.


I have also noted that at least one downstream root store (Debian) takes
all Mozilla-trusted certificates and labels them as simply 
"mozilla/cert-public-name", even though more useful naming can be 
extracted from the last (most complete) report, after finding a non-gui 
tool that can actually parse CSV files with embedded newlines in string 
values.





Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-07 Thread Matthew Hardeman via dev-security-policy 
Would it be unreasonable to also consider publishing, as an "easy to use"
list, that set of only those anchors which are currently trusted in the
program and for which no exceptional in-product policy enforcement is
imposed?  (TLD constraints, provisional distrusts, etc.)

The lazier implementers are going to take the raw set of anchors and none
of the policy associated, and so the default assumption should be that none
of the enhanced policy enforcements from nss or firefox would get copied
along.

On Tue, Oct 6, 2020 at 9:09 PM Ryan Sleevi via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> It seems like there should be a link to
>
> https://wiki.mozilla.org/CA/FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F
> there
>
> I realize there’s a tension between making this easily consumable, and the
> fact that “easily consumed” doesn’t and can’t relieve an organization of
> having to be responsible and being aware of the issues and discussions here
> about protecting their users.
>
> I do worry this is going to encourage one of the things that can make it
> more difficult for Mozilla to protect Mozilla users, which is when vendors
> blindly using/build a PEM file and bake it into a device they never update.
> We know from countless CA incidents that when vendors do that, and aren’t
> using these for “the web”, that it makes it more difficult for site
> operators to replace these certificates. It also makes it harder for
> Mozilla to fix bugs in implementations or policies and otherwise take
> actions that minimize any disruption for users. At the same time, Mozilla
> running a public and transparent root program does indeed mean it’s better
> for users than these vendors doing nothing at all, which is what would
> likely happen if there were too many roadblocks.
>
> While personally, I want to believe it’s “not ideal” to make it so easy, I
> realize the reality is plenty of folks already repackage the Mozilla store
> for just this reason, totally ignoring the above link, and make it easy for
> others to pull in. At least this way, you could reiterate that this list
> doesn’t really absolve these vendors of having to keep users up to date and
> protected and be able to update their root stores for their products, by
> linking to
>
> https://wiki.mozilla.org/CA/FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F
>
> On Tue, Oct 6, 2020 at 5:47 PM Kathleen Wilson via dev-security-policy <
> dev-security-policy@lists.mozilla.org> wrote:
>
> > All,
> >
> > I've been asked to publish Mozilla's root store in a way that is easy to
> > consume by downstreams, so I have added the following to
> > https://wiki.mozilla.org/CA/Included_Certificates
> >
> > CCADB Data Usage Terms
> > 
> >
> > PEM of Root Certificates in Mozilla's Root Store with the Websites
> > (TLS/SSL) Trust Bit Enabled (CSV)
> > <
> >
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEM?TrustBitsInclude=Websites
> > >
> >
> > PEM of Root Certificates in Mozilla's Root Store with the Email (S/MIME)
> > Trust Bit Enabled (CSV)
> > <
> >
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEM?TrustBitsInclude=Email
> > >
> >
> >
> > Please let me know if you have feedback or recommendations about this.
> >
> > Thanks,
> > Kathleen
> > ___
> > dev-security-policy mailing list
> > dev-security-policy@lists.mozilla.org
> > https://lists.mozilla.org/listinfo/dev-security-policy
> >
> ___
> dev-security-policy mailing list
> dev-security-policy@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: PEM of root certs in Mozilla's root store

2020-10-06 Thread Ryan Sleevi via dev-security-policy 
It seems like there should be a link to
https://wiki.mozilla.org/CA/FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F
there

I realize there’s a tension between making this easily consumable, and the
fact that “easily consumed” doesn’t and can’t relieve an organization of
having to be responsible and being aware of the issues and discussions here
about protecting their users.

I do worry this is going to encourage one of the things that can make it
more difficult for Mozilla to protect Mozilla users, which is when vendors
blindly using/build a PEM file and bake it into a device they never update.
We know from countless CA incidents that when vendors do that, and aren’t
using these for “the web”, that it makes it more difficult for site
operators to replace these certificates. It also makes it harder for
Mozilla to fix bugs in implementations or policies and otherwise take
actions that minimize any disruption for users. At the same time, Mozilla
running a public and transparent root program does indeed mean it’s better
for users than these vendors doing nothing at all, which is what would
likely happen if there were too many roadblocks.

While personally, I want to believe it’s “not ideal” to make it so easy, I
realize the reality is plenty of folks already repackage the Mozilla store
for just this reason, totally ignoring the above link, and make it easy for
others to pull in. At least this way, you could reiterate that this list
doesn’t really absolve these vendors of having to keep users up to date and
protected and be able to update their root stores for their products, by
linking to
https://wiki.mozilla.org/CA/FAQ#Can_I_use_Mozilla.27s_set_of_CA_certificates.3F

On Tue, Oct 6, 2020 at 5:47 PM Kathleen Wilson via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> All,
>
> I've been asked to publish Mozilla's root store in a way that is easy to
> consume by downstreams, so I have added the following to
> https://wiki.mozilla.org/CA/Included_Certificates
>
> CCADB Data Usage Terms
> 
>
> PEM of Root Certificates in Mozilla's Root Store with the Websites
> (TLS/SSL) Trust Bit Enabled (CSV)
> <
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEM?TrustBitsInclude=Websites
> >
>
> PEM of Root Certificates in Mozilla's Root Store with the Email (S/MIME)
> Trust Bit Enabled (CSV)
> <
> https://ccadb-public.secure.force.com/mozilla/IncludedRootsPEM?TrustBitsInclude=Email
> >
>
>
> Please let me know if you have feedback or recommendations about this.
>
> Thanks,
> Kathleen
> ___
> dev-security-policy mailing list
> dev-security-policy@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy
>
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy