Re: Policy 2.7.1: MRSP Issue #152: Add EV Audit exception for Policy Constraints

[Kathleen Wilson via dev-security-policy]

>> For this MRSP Issue #152 update to v2.7.1, I propose that we make each
>> occurrence of "capable of issuing EV certificates" link to
>> https://wiki.mozilla.org/CA/EV_Processing_for_CAs#EV_TLS_Capable


In the definition of EV TLS Capable, I'd move the last bullet up to the top.



Done. Thanks!


___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Ryan Sleevi via dev-security-policy]

On Fri, Nov 6, 2020 at 6:08 PM Dimitris Zacharopoulos via
dev-security-policy  wrote:

> Can other people, except Ryan, follow this thread? I certainly can't. Too
> much information, too much text, too many assumptions, makes it impossible
> to meaningfully participate in the discussion.


These are complex topics, for sure, but that’s unavoidable. Participation
requires a degree of understanding both about the goals to be achieved by
auditing, as well as the relevant legal and institutional frameworks for
these audits. So, admittedly, that’s not the easiest to jump into.

Could you indicate what you’re having trouble following? I don’t know that
we can do much about “too much information”, since that can be said about
literally anything unfamiliar, but perhaps if you would simply ask
questions, or highlight what you’d like to more about, it could be more
digestible?

What would you say your desired outcome from your email to be? Accepting,
for a second that this is a complex topic, and so discussion will
inherently be complex, and so a response such as “make it simpler for me”
is a bit unreasonable.
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Dimitris Zacharopoulos via dev-security-policy]

Can other people, except Ryan, follow this thread? I certainly can't. Too much 
information, too much text, too many assumptions, makes it impossible to 
meaningfully participate in the discussion.
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Microsec: Misissuance of 2 CISCO VPN server authentication certificates

[Sándor dr . Szőke via dev-security-policy]

### INCIDENT REPORT - Misissuance of 2 CISCO VPN server authentication 
certificates

---
>I -- How your CA first became aware of the problem (e.g. via a problem report 
>submitted to your Problem Reporting Mechanism, a discussion in 
>mozilla.dev.security.policy, a Bugzilla bug, or internal self-audit), and the 
>time and date.

The Microsec Customer Service Department recovered the misissued certificates 
during the final internal quality check before delivery.

-
>II -- A timeline of the actions your CA took in response. A timeline is a 
>date-and-time-stamped sequence of all relevant events. This may include events 
>before the incident was reported, such as when a particular requirement became 
>applicable, or a document changed, or a bug was introduced, or an audit was 
>done.

**bvpn.mokk.hu**

serial: 03549d12dae60ababce088d2210a
hash  : ce52002bb9bad4f15f0be21df4de7292c16aa985 
issuance  : 2020-11-05 10:20:06 UTC
revocation: 2020-11-06 08:17:23 UTC

**avpn.mokk.hu**

serial: 0354e7785c9a9de1adbc4947c60a
hash  : e64f1d0608e6518933a86a7205e1354511903449
issuance  : 2020-11-06 07:41:05 UTC
revocation: 2020-11-06 08:16:35 UTC


-
>III -- Whether your CA has stopped, or has not yet stopped, issuing 
>certificates with the problem. A statement that you have will be considered a 
>pledge to the community; a statement that you have not requires an explanation.

The affected CISCO VPN Server Authentication certificate profile was suspended, 
the certificate issuance with other certificate profiles has not been stopped.

-
>IV -- A summary of the problematic certificates. For each problem: number of 
>certs, and the date the first and last certs with that problem were issued.

Two certificates were issued for CISCO VPN servers with longer than 398 
days validity
The first certificate was issued on 2020-11-05
The last certificate was issued on 2020-11-06
Booth certificates were revoked on 2020-11-06
The whole problem was solved within 24 hours



-
>V -- The complete certificate data for the problematic certificates. The 
>recommended way to provide this is to ensure each certificate is logged to CT 
>and then list the fingerprints or crt.sh IDs, either in the report or as an 
>attached spreadsheet, with one list per distinct problem.

bvpn.mokk.huhttps://crt.sh/?id=3606063415
avpn.mokk.huhttps://crt.sh/?id=360362

-
>VI -- Explanation about how and why the mistakes were made or bugs introduced, 
>and how they avoided detection until now.

This was the first issuance of CISCO VPN Server Authentication certificates 
since 2020-09-01.

Microsec has a version management system for the certificate profiles, which is 
maintained through SVN. 
The immediate investigation could find an error in this system, one component 
of the affected CISCO VPN Server Authentication certificate profile was not 
covered by the version control, and this way this component left unchanged at 
2020-09-01.

-
>VII -- List of steps your CA is taking to resolve the situation and ensure 
>such issuance will not be repeated in the future, accompanied with a timeline 
>of when your CA expects to accomplish these things.

**Immediate actions**

Microsec revoked the affected two certificates
The affected certificate profile was suspended
An investigation was made to recover the reason of the misissuance
Microsec identified the root of the problem: 
- one component of the affected certificate profile was not covered by 
the configuration management system 
- due to this fault this certificate profile left unchanged at 
2020-09-01
Microsec corrected the affected certificate profile component
Microsec added the missing certificate profile component to the version 
control system
The issuance of CISCO VPN Server certificates was enabled again

**Further actions**

Microsec will review all the certificate profiles for similar problems
Microsec will check the whole certificate profile management system

>   **Planned deadline is 2020-11-20**






___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

RE: Policy 2.7.1: MRSP Issue #152: Add EV Audit exception for Policy Constraints

[Tim Hollebeek via dev-security-policy]

In the definition of EV TLS Capable, I'd move the last bullet up to the top.

This is because the definition is inherently recursive, and it's easy to
miss that 
if the recursion rule isn't first.

For example, I had a question about whether "revoked" meant just the
certificate
itself, or whether a revoked parent (etc) also qualifies.  But the ambiguity
goes 
away once you realize that the parent/cross/etc also needs to be EV TLS
Capable, 
hence not revoked.

-Tim

> -Original Message-
> From: dev-security-policy 
> On Behalf Of Kathleen Wilson via dev-security-policy
> Sent: Thursday, November 5, 2020 7:28 PM
> To: Mozilla 
> Subject: Re: Policy 2.7.1: MRSP Issue #152: Add EV Audit exception for
Policy
> Constraints
> 
> On 10/16/20 11:26 PM, Ryan Sleevi wrote:
> > Because of this, it seems that there is a simpler, clearer,
> > unambiguous path for CAs that seems useful to move to:
> > - If a CA is trusted for purpose X, that certificate, and all
> > subordinate CAs, should be audited against the criteria relevant for X
> >
> 
> I am in favor of this approach for a future version of Mozilla's Root
Store
> Policy, but I prefer not to try to tackle it in this v2.7.1 update.  So I
filed a
> github issue to remind us to consider this in the next version:
> 
> https://github.com/mozilla/pkipolicy/issues/220
> 
> 
> I have added a section called "EV TLS Capable" to the wiki pages, and I
will
> appreciate feedback on it:
> 
> https://wiki.mozilla.org/CA/EV_Processing_for_CAs#EV_TLS_Capable
> 
> For this MRSP Issue #152 update to v2.7.1, I propose that we make each
> occurrence of "capable of issuing EV certificates" link to
> https://wiki.mozilla.org/CA/EV_Processing_for_CAs#EV_TLS_Capable
> 
> Thanks,
> Kathleen
> 
> ___
> dev-security-policy mailing list
> dev-security-policy@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-security-policy



smime.p7s
Description: S/MIME cryptographic signature
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Jakob Bohm via dev-security-policy]

On 2020-11-06 18:31, Jeff Ward wrote:
> ...


Audit reports, whether for WebTrust, financial statements, or other forms of 
engagement reports providing assurance to users of the information, do not 
include specific audit team members’ names.  Simply stated, this desire to 
include individual auditor’s qualifications in a public report is not 
consistent with any other compliance reporting methods or reporting 
requirements for CAs, or any other auditee for that matter.



Most paper-based auditing schemes for company financial records (the
historic work area of auditors) include, on each report, the personal
signature and corresponding printed name of the responsible auditor,
optionally with an abbreviation of their national qualification level
(such as an abbreviation of "Examplarian State Authorized Public
Accountant").  From there, it would be possible for interested parties
to check that a physical person by that name is/was indeed on the roster
of such authorized individuals, but not if/why the State of Exemplar
decided to so include that person.  Furthermore, the auditor person
and/or their company may have voluntarily published further details of
their qualifications (in brochures, on websites etc.) and may have
applicable original degree documents framed and hanging on their walls
for all concerned to readily inspect.

In terms of GDPR, the state would have published rules for how to get
added/removed from the public roster, and each auditor would have the
opportunity, at all times, to retract their self-descriptions and/or
remove some or all of their framed documents from their public office.

A modern equivalent procedure for CA audits could be:

1. Each Auditor has their name and a unique public nickname registered
in a non-public roster at either CPA Canada or the relevant European
counterpart.  This is done to fulfill the contractual obligation of
their professional oath of responsibility.  The roster organization
might optionally provide alias e-mails based on the nicks.

2. Each non-public roster operates a public online service which will
confirm or deny the presence of a name/nick pair, with appropriate
safeguards against attempts to extract the roster by systematic polling
of made up names.

Unless otherwise stated in public by Mozilla (such as the statements
made a few years ago about certain auditors from E), any auditor on
these rosters shall be presumed sufficiently qualified to sign audits
used by Mozilla.

3. Each auditor person signs his public audit letters with his name,
nick, a reference to the roster-keeping organization and any other
honorific titles he/she may legitimately choose to use.  He does this to
satisfy his contractual obligation to provide the CA with that letter.
Any official physical copies will have his physical signature above his
name and may also carry a physical stamp or seal of him or his
organization, as dictated by local legal traditions.

4. Each such public audit letter is submitted to a public repository
operated by the roster-keeping organization, using a procedure that
verifies that the letter was submitted exactly as given, by that named
auditor from their roster.  This is done to satisfy the contractual
obligation of the auditor towards the CA in accordance with a
contractual reference to terms of the roster-keeping organization.

5. The roster-keeping organization publishes the public audit letters in
both a traditional paper journal deposited at major public archives and
as an online readily accessible web site with a Merkel hash tree
providing public verification that each letter was in the public record
on or before the stated inclusion date.  As hash algorithms fail to
future cracks, the roster-keeping organization retains the ability to
regenerate the signatures using new algorithms, based on its offline
archive of originals, including a signed public statement of said
regeneration.  This publication of records that include the identity of
both the actual auditors as well as relevant principal CA Officers is
done to further satisfy the contractual obligations in #4.  As is common
in paper-based book-keeping, retractions can be filed as separate
letters of correction, and the retracted documents may be made invisible
to the public without invalidating the hash-tree.
  For public access, each public letter is given a unique permanent URL
to which the CA may publicly refer, including in the CADB and on its
website.

6. Each auditor shall submit for publication by the roster organization
a self-authored but roster verified statement of qualifications, usually
just a few paragraphs.  Each such statement similarly gets a permanent
URL, but remains visible only until superseded or retracted by either
the auditor or roster-keeper.  This publication is done as part of the
auditor's contractual obligations to the roster-keeping organization,
and the ability to retract provides the GDPR right of deletion of any
included details.  Links to the current 

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Ryan Sleevi via dev-security-policy]

On Fri, Nov 6, 2020 at 12:00 PM Clemens Wanko via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> Hi Ryan, hi all,
> three things to comment on that:
>
> 1.  How is the EU ETSI audit scheme thought and what is it intended to
> provide to Mozilla and the CA/Browser ecosystem?

The European scheme of technical standards for CA/TSP developed by ETSI was
> made and is constantly adopted to integrate CA/Browser requirements. The
> main standards I am referring to are: ETSI EN 319 401, EN 319 411-1 and EN
> 319 411-2. With regard to auditor guidance specifically for the CA/Browser
> ecosystem, ETSI even developed and maintains the TS 119 403-2 “Part 2:
> Additional requirements for Conformity Assessment Bodies auditing Trust
> Service Providers that issue Publicly-Trusted Certificates”.
> For auditors using such technical standards Europe has established a well
> thought through scheme based upon organizations (accredited audit bodies)
> which job it is – amongst others – to ensure auditors (the natural person)
> qualification, independence, etc. This scheme turned out to be of high
> trustworthiness, reliability, robustness, etc. And it works very well over
> here since years.
>

I'm sure something is lost in translation, but this does not address any of
the concerns raised. The ETSI standards here are not relevant; voluntary
standards, in and of themselves, are not binding. The fact that a standard
exists is not relevant, since what matters is how the standard is adhered
to and supervised.

Your subsequent statement is simply "Well, they're auditors, people trust
them to do this", without any form of meaningful engagement on the why.
"It's their job to ensure independence" - yes, but that says nothing about
the performance of the job, that your measure of independence is consistent
with another measure, etc. Your entire appeal here simply is "We say we're
trustworthy, so of course we're trustworthy", which would be farcical if it
wasn't presented so seriously.


> 2.  Transparency
> The transparency lies in the European scheme with independent skilled
> bodies auditing each other in order to ensure conformant implementation of
> technical standards as well as of operational requirements for audit bodies
> as well as for the single auditor (natural person).


This is, again, a restatement of "Trust us, because we declared we're
trustworthy". For something such as trust, there's no question of "but
verify". Your appeal to SDOs is not relevant here, because as you and I are
both aware, the SDO just makes (voluntary) standards, and doesn't enforce
them. And this gets to the heart of the matter: the question about how each
of these dimensions are interpreted is something you would prefer the CABs
to self-declare, and the suggestion here is "Sure, you can say that, but
you need to also help us understand why that's true".


> Not only the requirements for the qualification/independence/etc. of the
> auditor (natural person) are clearly defined within the ISO/ETSI but also
> the management requirements of the body in order to ensure that they are
> kept upright – btw. BSI C5 controls, section 4.4.9 is meant similar to
> that.
> Every accredited body is audited at least once a year by its NAB which
> checks conformant audit processing (e.g. according to ISO/IEC17065 in
> combination with ETSI EN 319 403).
>

This is the first time in this e-mail that you remotely come to
establishing a "why". As we both know full well, the authority of the CAB
(and its assessment along these relevant axis in 319 403) is imbued by the
NAB. The authority of the NAB is imbued by EA, established by Regulation
765/2008. The root of trust is, simply stated, "The state mechanisms of the
European Union trusts us, ergo, we are defacto trustworthy".

As a long-time participant in this group, I'm sure you can understand and
appreciate that the "Trust us, we're the government" argument rarely
improves trust. For CAs, we work on the basis of concrete evidence; after
all, this is why we have audits in the first place, even for government
CAs. The argument you're making here is that EA imbued sovereign member
states the ability to establish their NAB, and the NABs are responsible for
supervising the CABs. If we have a complaint with the CAB, the argument
goes, we should complain to the NAB, consistent with the ISO 17065
framework that EN 319 403 is based on.

Yet this misguided argument overlooks a host of salient details, no doubt
because of the convenience in doing so. Unlike, say, the ISAE 3000 approach
to audits, the approach taking by this EA/NAB/CAB chain is much more
limited with respect to a host of professional duties, and only directly
applicable within the contents of 319 403, and the relevant (supervised)
reports, such as 319 411-1. Thus, there's easy opportunity for there to be
a divergence in needs, by a CAB asserting that they are qualified within
the aegis of 319 403/319 411-1 with the necessary 

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Ryan Sleevi via dev-security-policy]

On Fri, Nov 6, 2020 at 12:31 PM Jeff Ward via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:

> Audit reports, whether for WebTrust, financial statements, or other forms
> of engagement reports providing assurance to users of the information, do
> not include specific audit team members’ names.  Simply stated, this desire
> to include individual auditor’s qualifications in a public report is not
> consistent with any other compliance reporting methods or reporting
> requirements for CAs, or any other auditee for that matter.


Hi Jeff,

Could you help me square this statement with the practical examples? For
example, here's a report [1] from a WebTrust TF member, Ernst and Young,
demonstrating how this works in practice. You can see there hasn't been an
issue for years [2][3], even with the direct involvement of individuals on
the WebTrust TF in preparing such an audit.

So I'm having difficulty squaring your statement that they "do not", when
we've got examples from long-standing members of the WebTrust TF that
demonstrate that, in practice, they do. Could you help highlight what's
inconsistent here?

Alternatively, and as mentioned to ETSI, here's an example of an ISAE 3000
based audit scheme, similar to WebTrust, that also discloses the relevant
professional qualifications and skills to clients [4], as discussed in
4.4.8 and 4.4.9.

[1] https://www.oversight.gov/sites/default/files/oig-reports/18-19.pdf
[2]
https://www.oversight.gov/sites/default/files/oig-reports/Assessment%20Report%2019-12%20GPO%20Federal%20PKI%20Compliance.pdf
[3] https://www.oversight.gov/sites/default/files/oig-reports/17-27.pdf
[4]
https://www.bsi.bund.de/EN/Topics/CloudComputing/Compliance_Criteria_Catalogue/Compliance_Criteria_Catalogue_node.html
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #153: Cradle-to-Grave Contiguous Audits

[Jakob Bohm via dev-security-policy]

On 2020-11-05 22:43, Tim Hollebeek wrote:

So, I'd like to drill down a bit more into one of the cases you discussed.
Let's assume the following:

1. The CAO [*] may or may not have requested removal of the CAC, but removal
has not been completed.  The CAC is still trusted by at least one public
root program.

2. The CAO has destroyed the CAK for that CAC.

The question we've been discussing internally is whether destruction alone
should be sufficient to get you out of audits, and we're very skeptical
that's desirable.

The problem is that destruction of the CAK does not prevent issuance by
subCAs, so issuance is still possible.  There is also the potential
possibility of undisclosed subCAs or cross relationships to consider,
especially since some of these cases are likely to be shutdown scenarios for
legacy, poorly managed hierarchies.  Removal may be occurring *precisely*
because there are doubts about the history, provenance, or scope of previous
operations and audits.

We're basically questioning whether there are any scenarios where allowing
someone to escape audits just because they destroyed the key is likely to
lead to good outcomes as opposed to bad ones.  If there aren't reasonable
scenarios where it is necessary to be able to remove CACs from audit scope
through key destruction while they are still trusted by Mozilla, it's
probably best to require audits as long as the CACs are in scope for
Mozilla.

Alternatively, if there really are cases where this needs to be done, it
would be wise to craft language that limits this exception to those
scenarios.



I believe that destruction of the Root CA Key should only end audit
requirements for the corresponding Root CA itself, not for any of its
still trusted SubCAs.

One plausible (but hypothetical) sequence of events is this:

1. Begin Root ceremony with Auditors present.

1.1 Create Root CA Key pair
1.2 Sign Root CA SelfCert
1.3 Create 5 SubCA Key pairs
1.4 Sign 5 SubCA pre-certificates
1.5 Request CT Log entries for the 5 SubCA pre-certificates
1.6 Sign 5 SubCA certificates with embedded CTs
1.7 Sign, but do not publish a set of post-dated CRLs for various 
contingencies
1.8 Sign, but do not publish a set of post-dated revocation OCSP 
responses for those contingencies
1.9 Sign, but do not yet publish, a set of post-dated non-revocation 
OCSP responses confirming that the SubCAs have not been revoked on each 
date during their validity.

1.10 Destroy Root CA Key pair.

2. Initiate audited storage of the unreleased CRL and OCSP signatures.

3. End Root ceremony, end root CAC audit period.

4. Release public audit report of this ceremony, this ends the ordinary 
audits required for the Root CA Cert.  However audit reports that only 
the correct contingency and continuation OCSP/CRL signatures were 
released from storage remain technically needed.


5. Maintain revocation servers that publish the prepared CRLs and OCSP 
answers according to their embedded dates.  Feed their publication queue

from audited batch releases from the storage.

6. Operate the 5 SubCAs under appropriate security and audit schemes 
detailed in CP/CPS document pairs.


7. Apply for inclusion in the Mozilla root program.


In the above hypothetical scenario, there would be no way for the the 
CAO to misissue new SubCAs or otherwise misuse the root CA Key Pair, but 
still the usual risks associated with the 5 SubCA operations.


Also the CAO would have no way to increase the set of top level SubCAs
or issue revocation statements in any yet-to-be-invented data formats,
even if doing so would be legitimate or even required by the root
programs.

Thus the hypothetical scenario could land the CAO in an impossible 
situation, if root program requirements or common CA protocols change,
and those changes would require even one additional signature by the 
root CA Key Pair.



Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #187: Require disclosure of incidents in Audit Reports

[Jeff Ward via dev-security-policy]

On Thursday, October 22, 2020 at 1:53:40 PM UTC-5, Ben Wilson wrote:
> The purpose of this email is to begin public discussion on the addition of 
> a subsection 11 to section 3.1.4 of the Mozilla Root Store Policy. Issue 
> #187  in GitHub proposes 
> to require audit reports to list all incidents occurring (or open) during 
> the audit period of which the auditor has been made aware or to state that 
> the auditor is unaware of any incidents. This is related to Issue #154 
>  (management assertion 
> disclosures). That proposal is to have section 2.4 read as follows: "If 
> being audited to the WebTrust criteria, the Management Assertion letter 
> MUST include all known incidents that occurred or were still 
> open/unresolved at any time during the audit period." 
> 
> Proposed language may be found in the following commits: 
> 
> - 
> https://github.com/BenWilson-Mozilla/pkipolicy/commit/f6639f503b743aae402dc0f4841dc3dd5ba88753
>  
> - 
> https://github.com/BenWilson-Mozilla/pkipolicy/commit/6c07c44e4db473dc4d34009f1bc955a0e18cb4c1
>  
> - 
> https://github.com/BenWilson-Mozilla/pkipolicy/commit/5dec00e53b4c6361d85af7644660fe185fcf463d
>  
> 
> Proposed language for section 3.1.4 is: 
> 
> "11. all incidents (as defined in section 2.4) that occurred or were still 
> open/unresolved at any time during the audit period, or a statement that 
> the auditor is unaware of any;" 
> 
> I look forward to your comments, suggestions and discussions. 
> 
> Ben

Thanks for bringing this up Ben.  It is important to consider this requirement 
in conjunction with #154 and address them together. It seems reasonable to 
require a CA to disclose all known incidents that are applicable during a given 
period. It would be important, however, to define “known incident” as a 
“verified bug” and exclude items such as bugs closed as a duplicate, invalid, 
etc.  It would also make sense to clarify that an incident should only be 
disclosed once and eliminate duplication when an incident spans two audit 
periods. 

Also keep in mind an auditor typically issues an opinion on management’s 
assertion of its controls. Audit opinions do not make negative assurance 
statements, such as not being aware of any incidents during the period. If the 
CA is required to make this assertion, the auditor’s opinion will consider that 
statement. 

Thanks, 

Jeff
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Jeff Ward via dev-security-policy]

On Tuesday, November 3, 2020 at 5:53:52 PM UTC-6, Ben Wilson wrote:
> Historically, Mozilla Policy required that CAs "provide attestation of 
> their conformance to the stated verification requirements and other 
> operational criteria by a competent independent party or parties with 
> access to details of the CA's internal operations." 
> https://wiki.mozilla.org/CA:CertificatePolicyV1.0 "Competency" was "for 
> whom there is sufficient public information available to determine that the 
> party is competent to judge the CA's conformance to the stated criteria. In 
> the latter case the 'public information' referred to should include 
> information regarding the party's: 
> 
> - knowledge of CA-related technical issues such as public key 
> cryptography and related standards; 
> - experience in performing security-related audits, evaluations, or risk 
> analyses; *and* 
> - honesty and objectivity." 
> 
> Today, section 3.2 of the MRSP 
> 
>  
> states, "In normal circumstances, Mozilla requires that audits MUST be 
> performed by a Qualified Auditor, as defined in the Baseline Requirements 
> section 8.2," but under section 2.3 
> ,
>  
> "Mozilla reserves the right to accept audits by auditors who do not meet 
> the qualifications given in section 8.2 of the Baseline Requirements, or 
> refuse audits from auditors who do." 
> 
> Section 8.2 of the Baseline Requirements states an auditor must have: 
> 1. Independence from the subject of the audit; 
> 2. The ability to conduct an audit that addresses the criteria specified in 
> an Eligible Audit Scheme (see Section 8.1); 
> 3. Employs individuals who have proficiency in examining Public Key 
> Infrastructure technology, information security tools and techniques, 
> information technology and security auditing, and the third-party 
> attestation function; 
> 4. (For audits conducted in accordance with any one of the ETSI standards) 
> accredited in accordance with ISO 17065 applying the requirements specified 
> in ETSI EN 319 403; 
> 5. (For audits conducted in accordance with the WebTrust standard) licensed 
> by WebTrust; 
> 6. Bound by law, government regulation, or professional code of ethics; and 
> 7. Except in the case of an Internal Government Auditing Agency, maintains 
> Professional Liability/Errors & Omissions insurance with policy limits of 
> at least one million US dollars in coverage 
> 
> It is proposed in Issue #192 
>  that information about 
> individual auditor's qualifications be provided--identity, competence, 
> experience and independence. (For those interested as to this independence 
> requirement, Mozilla Policy v.1.0 required either disclosure of the 
> auditor's compensation or the establishment that the auditor "is bound by 
> law, government regulation, and/or a professional code of ethics to render 
> an honest and objective judgement regarding the CA.") 
> 
> While subsection 3 of BR 8.2 requires "individuals who have proficiency in 
> examining Public Key Infrastructure technology, information security tools 
> and techniques, information technology and security auditing, and the 
> third-party attestation function," that fact needs evidence in order to be 
> established. The proposed resolution of this Issue #192 intends to 
> accomplish that. 
> 
> This proposal to require disclosure of individual auditor qualifications is 
> very similar to the approach adopted by the U.S. Federal PKI 
> 
>  
> (see Appendices B-1 and C). E.g., "Did each Audit Opinion Letter identify 
> the auditor and the individuals performing the audit?" In practice, the 
> information about auditor qualifications could be in the form of a separate 
> document, such as a curriculum vitae. 
> 
> Some initial, draft language to address this issue is located here: 
> https://github.com/BenWilson-Mozilla/pkipolicy/commit/d0da7cb2b6db38e66c3a72e5c1db0e78e91d8df6
>  
> 
> A new subsection 3. would be added to the list of audit requirements that 
> would require "[the] name(s) and qualifications of individuals performing 
> the audit, as required by section 3.2" and a new paragrpah would be added 
> to section 3.2 that would say, "A Qualified Auditor MUST have relevant IT 
> Security experience, or have audited a number of CAs, and be independent 
> and not conflicted. Individuals have competence, partnerships and 
> corporations do not. Audit documentation of individual auditor 
> qualifications MUST be provided to Mozilla that is sufficient for Mozilla 
> to determine the competence, experience, and independence of the Qualified 
> Auditor. Mozilla will review each individual 

Re: Policy 2.7.1: MRSP Issue #192: Require information about auditor qualifications in the audit report

[Clemens Wanko via dev-security-policy]

Hi Ryan, hi all,
three things to comment on that:

1.  How is the EU ETSI audit scheme thought and what is it intended to 
provide to Mozilla and the CA/Browser ecosystem?
The European scheme of technical standards for CA/TSP developed by ETSI was 
made and is constantly adopted to integrate CA/Browser requirements. The main 
standards I am referring to are: ETSI EN 319 401, EN 319 411-1 and EN 319 
411-2. With regard to auditor guidance specifically for the CA/Browser 
ecosystem, ETSI even developed and maintains the TS 119 403-2 “Part 2: 
Additional requirements for Conformity Assessment Bodies auditing Trust Service 
Providers that issue Publicly-Trusted Certificates”. 
For auditors using such technical standards Europe has established a well 
thought through scheme based upon organizations (accredited audit bodies) which 
job it is – amongst others – to ensure auditors (the natural person) 
qualification, independence, etc. This scheme turned out to be of high 
trustworthiness, reliability, robustness, etc. And it works very well over here 
since years.

2.  Transparency
The transparency lies in the European scheme with independent skilled bodies 
auditing each other in order to ensure conformant implementation of technical 
standards as well as of operational requirements for audit bodies as well as 
for the single auditor (natural person). Not only the requirements for the 
qualification/independence/etc. of the auditor (natural person) are clearly 
defined within the ISO/ETSI but also the management requirements of the body in 
order to ensure that they are kept upright – btw. BSI C5 controls, section 
4.4.9 is meant similar to that.  
Every accredited body is audited at least once a year by its NAB which checks 
conformant audit processing (e.g. according to ISO/IEC17065 in combination with 
ETSI EN 319 403).
Now, requesting more of transparency with regard to auditor qualification for 
Mozilla insinuates this immediately translates into more confidence in the 
auditor. Please lets be careful here. If we like the community to follow that, 
shouldn’t be clearly stated:
- which documents/evidences are expected from the auditor to be handed in to 
Mozilla, 
- what criteria (details need to be published) shall Mozilla use to check the 
documents/evidences against, 
- a statement of how Mozilla comes to the conclusion: auditor is acceptable / 
not acceptable,
- how Mozilla shall organize themselves to perform this task (skilled staff 
members are required),
- how that can get organized in a way that it is compatible to CA projects (see 
Tim Hollebeeks mail),
and finally: 
what information is expected from Mozilla to be published for every single 
auditor (natural person) to show auditors qualification and make it transparent.
The tasks related to auditor qualification validation and management actually 
are performed by the participants in the European scheme already (and apart 
from that, not very different also in the WebTrust world). 

3.  Requiring all their externally-operated subordinate CAs review their 
auditors 
You said:
quote  <<< That sounds like a great idea, and sounds like a good compliment to 
an approach that several (Root) CAs have taken, of requiring all their 
externally-operated subordinate CAs review their auditors and audit scope with 
the root CAO, before finalizing the audit engagement.
An example of how the public review has been done in the past is available at 
https://groups.google.com/g/mozilla.dev.security.policy/c/sRJOPiePUvI/m/oRmRffaQmcYJ
 >>> 

If we like to suggest that, wouldn’t we then not also need to state at least 
1st  based upon what rule-set we like the CA to review auditors,
2nd what competences are required at CA level to validate auditors and how 
those shall be established and maintained,
3rd how the CA is required to do that with regard to process and timing?
In order to be clear and transparent, things like these would need to be 
defined at CA level then.

Best regards
Clemens
___
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy