Hi Ryan, hi all, well, isn’t the point to make here just, that there are multiple ways to ensure proper auditor qualification? No matter which way you like to go however, you must define the details of your regime: what is the criteria you require the auditor to fulfill, how do you organize that these criteria are checked, how do you ensure the quality of this process, how do you publish any results with regard to the auditors qualification, etc.
Now, what we have in place with the EU ETSI scheme (or with WebTrust) is a regime which provides - normative options specifically adopted to address CA/B-Forum/Browser requirements as well as - a system to ensure auditor qualification is there and kept upright over time. The regime is there, proven and flexible to make proper use out of it. And constant effort is made from ETSI as well as from the auditors (ACAB’c) to the Forum in order to incorporate CA/B Forum requirements. Certainly, there are always alternatives. But the alternative should be clearly defined and described in order to allow an evaluation of all the options before taking a decision. In the current case I like to understand the alternatives you suggest for Mozilla. Best regards Clemens On Friday, 6 November 2020 at 20:35:40 UTC+1, Ryan Sleevi wrote: > On Fri, Nov 6, 2020 at 12:00 PM Clemens Wanko via dev-security-policy < > dev-secur...@lists.mozilla.org> wrote: > > > Hi Ryan, hi all, > > three things to comment on that: > > > > 1. How is the EU ETSI audit scheme thought and what is it intended to > > provide to Mozilla and the CA/Browser ecosystem? > > The European scheme of technical standards for CA/TSP developed by ETSI was > > made and is constantly adopted to integrate CA/Browser requirements. The > > main standards I am referring to are: ETSI EN 319 401, EN 319 411-1 and EN > > 319 411-2. With regard to auditor guidance specifically for the CA/Browser > > ecosystem, ETSI even developed and maintains the TS 119 403-2 “Part 2: > > Additional requirements for Conformity Assessment Bodies auditing Trust > > Service Providers that issue Publicly-Trusted Certificates”. > > For auditors using such technical standards Europe has established a well > > thought through scheme based upon organizations (accredited audit bodies) > > which job it is – amongst others – to ensure auditors (the natural person) > > qualification, independence, etc. This scheme turned out to be of high > > trustworthiness, reliability, robustness, etc. And it works very well over > > here since years. > > > I'm sure something is lost in translation, but this does not address any of > the concerns raised. The ETSI standards here are not relevant; voluntary > standards, in and of themselves, are not binding. The fact that a standard > exists is not relevant, since what matters is how the standard is adhered > to and supervised. > > Your subsequent statement is simply "Well, they're auditors, people trust > them to do this", without any form of meaningful engagement on the why. > "It's their job to ensure independence" - yes, but that says nothing about > the performance of the job, that your measure of independence is consistent > with another measure, etc. Your entire appeal here simply is "We say we're > trustworthy, so of course we're trustworthy", which would be farcical if it > wasn't presented so seriously. > > 2. Transparency > > The transparency lies in the European scheme with independent skilled > > bodies auditing each other in order to ensure conformant implementation of > > technical standards as well as of operational requirements for audit bodies > > as well as for the single auditor (natural person). > This is, again, a restatement of "Trust us, because we declared we're > trustworthy". For something such as trust, there's no question of "but > verify". Your appeal to SDOs is not relevant here, because as you and I are > both aware, the SDO just makes (voluntary) standards, and doesn't enforce > them. And this gets to the heart of the matter: the question about how each > of these dimensions are interpreted is something you would prefer the CABs > to self-declare, and the suggestion here is "Sure, you can say that, but > you need to also help us understand why that's true". > > Not only the requirements for the qualification/independence/etc. of the > > auditor (natural person) are clearly defined within the ISO/ETSI but also > > the management requirements of the body in order to ensure that they are > > kept upright – btw. BSI C5 controls, section 4.4.9 is meant similar to > > that. > > Every accredited body is audited at least once a year by its NAB which > > checks conformant audit processing (e.g. according to ISO/IEC17065 in > > combination with ETSI EN 319 403). > > > This is the first time in this e-mail that you remotely come to > establishing a "why". As we both know full well, the authority of the CAB > (and its assessment along these relevant axis in 319 403) is imbued by the > NAB. The authority of the NAB is imbued by EA, established by Regulation > 765/2008. The root of trust is, simply stated, "The state mechanisms of the > European Union trusts us, ergo, we are defacto trustworthy". > > As a long-time participant in this group, I'm sure you can understand and > appreciate that the "Trust us, we're the government" argument rarely > improves trust. For CAs, we work on the basis of concrete evidence; after > all, this is why we have audits in the first place, even for government > CAs. The argument you're making here is that EA imbued sovereign member > states the ability to establish their NAB, and the NABs are responsible for > supervising the CABs. If we have a complaint with the CAB, the argument > goes, we should complain to the NAB, consistent with the ISO 17065 > framework that EN 319 403 is based on. > > Yet this misguided argument overlooks a host of salient details, no doubt > because of the convenience in doing so. Unlike, say, the ISAE 3000 approach > to audits, the approach taking by this EA/NAB/CAB chain is much more > limited with respect to a host of professional duties, and only directly > applicable within the contents of 319 403, and the relevant (supervised) > reports, such as 319 411-1. Thus, there's easy opportunity for there to be > a divergence in needs, by a CAB asserting that they are qualified within > the aegis of 319 403/319 411-1 with the necessary technical skills, and > which an individual supervisory body unfortunately accepts, while still > being vastly below the quality expected by the broader Mozilla community. > There is no opportunity for relief in the framework of 17065, because the > fact that the NAB/SB have determined the CAB's skills and reporting as > suitable for eIDAS is all that is required. > > All of this, again, rests in a presumption that we "trust, but verify". To > accept the argument that EA/NAB are inviolate and incapable of making > decisions misaligned with community needs is to equally be making the > argument that every government CA should be automatically added, because > they're the government. That's not how this works. That's not how any of > this works. > > Now, requesting more of transparency with regard to auditor qualification > > for Mozilla insinuates this immediately translates into more confidence in > > the auditor. Please lets be careful here. If we like the community to > > follow that, shouldn’t be clearly stated: > > > I can understand and appreciate why, as a CAB used to performing audits > under the ETSI framework, you prefer that everything be a checklist. > Unfortunately, trust is not a checklist, but a preponderance of evidence of > past behavior that can lead to accurate prediction of future behavior. I > can trust you to do X today, because you did X yesterday, the day before, > and the day before that. > > The basis for trust is that the CAB should build the evidence as to *why* > they are trustworthy. What makes the CAB relevant for this problem? > > To date, your appeal has mostly been "Well, we address this other problem, > and they're satisfied, so clearly, we're qualified to address this > problem", but that's comparing apples to orangutans. Concrete evidence as > to /why/ and /how/ are relevant to being able to say "Yes, that's good > enough", but also to help identify trends and patterns, such as "Oh, > auditors with this skill X, or without this skill Y, consistently produce > this result Z, and that is/is not what we want" > > I agree that there is benefit from an appropriate set of principles, but I > reject the assertion at face value that the idea of trust should be reduced > to a checklist, whether it's CAs or auditors. As a long-time participant, > you know exactly that the process of trust is not binary, and considers a > host of factors, some discretionary. Reducing these complex decisions to > binary decisions is exactly the direction root stores have spent > considerable effort moving away from over the past two decades, because of > the myriad security problems, and it'd be naive to suggest it somehow > appropriate for auditors. > > 3. Requiring all their externally-operated subordinate CAs review > > their auditors > > You said: > > quote <<< That sounds like a great idea, and sounds like a good > > compliment to an approach that several (Root) CAs have taken, of requiring > > all their externally-operated subordinate CAs review their auditors and > > audit scope with the root CAO, before finalizing the audit engagement. > > An example of how the public review has been done in the past is available > > at > > https://groups.google.com/g/mozilla.dev.security.policy/c/sRJOPiePUvI/m/oRmRffaQmcYJ > > > > >>> > > > > If we like to suggest that, wouldn’t we then not also need to state at > > least > > 1st based upon what rule-set we like the CA to review auditors, > > 2nd what competences are required at CA level to validate auditors and how > > those shall be established and maintained, > > 3rd how the CA is required to do that with regard to process and timing? > > In order to be clear and transparent, things like these would need to be > > defined at CA level then. > > > This is certainly "an" approach that can be used, and I do want to > acknowledge that. But it's not the only way this problem can be solved, and > I worry that this suffers from the many deficiencies that are addressed > earlier in this mail. _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
