On Wed, Oct 31, 2018 at 4:05 PM Dimitris Zacharopoulos <ji...@it.auth.gr> wrote:
> > For example, when we talk about expectations of CAs, we don't talk about > > what they 'could' do, we talk about what they MUST do, because at the end > > of the day, that's the bar they're being held to. It's certainly true > that > > a given TSP may go above and beyond some bar, but that doesn't mean we > can > > say "CAs do X", because they aren't required to. The same logic applies > in > > the discussion of CABs - it does not make sense to discuss how they > 'could' > > interpret it, but rather, what they MUST do. > > ISO 17065 and ETSI EN 319 403 include normative requirements for CABs > just as the Baseline Requirements and ETSI EN 319 411-1 do for TSPs. I > don't understand why you think it is any different for CABs. For > example, the baseline requirements mandate that "The CA SHALL maintain a > continuous 24x7 ability to respond internally to a high-priority > Certificate Problem Report, and where appropriate, forward such a > complaint to law enforcement authorities, and/or revoke a Certificate > that is the subject of such a complaint". > > It doesn't specify exactly how a CA shall respond internally to such a > request. It must have a process and the CAB will evaluate it. > > Similarly for CABs, under 7.13.1 of 17065 "The certification body shall > have a documented process to receive, evaluate and make decisions on > complaints and appeals. The certification body shall record and track > complaints and appeals, as well as actions undertaken to resolve them". > > It doesn't specify exactly how the CAB shall fulfill this requirement > but each competent CAB must demonstrate to their NAB that they have a > documented process that fulfills this criteria, is effective and efficient. > > Even in the introduction section of 17065, you see the same use of words > SHALL, SHOULD, MAY, CAN as described in RFC2119. > We've now so fully drifted from the original discussion that it's clear to see we're talking about very different things, and thus confusion is understandable. For context, this particular discussion began in https://groups.google.com/d/msg/mozilla.dev.security.policy/Q9whve-HJfM/niS5Y2f0AQAJ , and in particular, the discussion about "If the CAB" does X. The point of my criticism to this statement has been that the CAB is not required to do X, they may do X, but they aren't mandated to do X. I believe you interpreted my remark of "no notification will be made to relying parties" is that it's impossible to notify RPs, and so you raised an example of how a CAB might hypothetically do so. I was not trying to state that - merely, that CABs do not have a normative requirement to notify RPs of this change, and so as a matter of "Can you rely on this", the answer is "No". Yes, some CABs may, but if a CAB doesn't, or if they make mistakes, they've not violated any requirements. And that's just as equally applicable to CAs. When a CA MUST "have a process", we cannot make assumptions or rely on what that process does or how it might result. When a CAB MUST "have a process", we cannot make assumptions or rely on what they do in that process. I would hope we're in violent agreement there. > It's the same way that when we talk about the BRs, it's pointless to talk > > about how some CAs may go above and beyond in their CPS, when discussing > > the CA ecosystem or a particular (different) CA's misissuance. What > matters > > is the baseline expectations here. > > Some baseline expectations are not overly prescriptive in the BRs for > CAs nor in the "BRs for CABs" (i.e. ISO 17065 and ETSI EN 319 403). > Information Security Management Systems can have significantly different > implementations but must maintain specific principles. This is where > illustrative controls and recommended practices come in so that the > auditors can evaluate if the principles are met but it's always subject > to the opinion of the CAB (when auditing TSPs), and to the NAB (when > assessing CABs). > My attempt to explain-by-analogy, to hopefully get us talking about the same thing, has unfortunately lead to even more divergence. I hope that, with the above clarifications to what we're originally talking about, we can get back on the same page. That said, because it contains some confusion, it at least bears highlighting. As mentioned during the recent CABForum, illustrative controls do not serve the purpose you are describing, neither do recommended practices. They have no 'force' to ensure that things must be 'equivalent-or-better'. They're not even hints. They're just that - illustrative. One cannot and should not make any assumptions that the existence of illustrative examples means that you will get the same results. It's entirely valid to wholly ignore those illustrative controls and recommended practices, end up with something completely opposite, and yet have fully fulfilled the requirements. This is why prescriptive controls are more appropriate, or, alternatively, supervision of accreditation schemes that match the desired expectations (e.g. browser supervision and accreditation of CABs for purposes of browser trust, separate from and complementary to NAB supervision of CABs for purpose of legal recognition). > The seal statement which I quoted earlier, leaves the impression -at > least to me- that with the seal, the public gets some assurance "for the > establishment and on-going operation of a Certification Authority > organization of function". > > Also, as an RP that reads ''principles are being followed in conformity > with the WebTrust for Certification Authorities Criteria'', I have > reasonable assurance that "principles are being followed", which is > forward-looking. > As I've been so frequently reminded by WT auditors - you're holding it wrong. The auditors make specific efforts to disclaim this, as part of the discussion of the inherent risks of audits, and as part of the careful wording about what the audit conveys. You may be reading it from a 'plain english' perspective, but I think that's been the point for much of the discussions about audit and audit qualities - what you're interpreting it as is not, in fact, what's being provided, and the language being used is specific in the context (e.g. AICPA for US auditors, CPA Canada for Canadians) and the respective professional standards as to what they are and aren't guaranteeing. > I think this example explains it perfectly, although I have the > impression from previous discussions that in any case of "deviation", > auditors (regardless if they are WT or ETSI) must be notified by the > CA/TSP. > Nope. This is only in the case of ETSI. > This is indeed a different approach between the two schemes. For ETSI, > you might be required to stop all operations if the CAB suspends the > certification, before the end of the on-going audit period. If I > understood correctly, in WT you will be allowed to continue operations > until the next audit, where you will get a qualified opinion. Is that > correct? > Yes. This is one of several fundamental differences between WebTrust and ETSI audits that I discussed at the recent CA/Browser Forum. The thing is, in many ways, WebTrust is providing higher assurance, even in the 'bad' case, because the bar is set adversarially - that is, the CA has to demonstrate sufficient evidence to the auditor that they've been doing things correctly. A good WebTrust auditor will thus ask better, and probing, questions demanding more information. Compare this with ETSI, which, while making reference to historic facts, is predominantly serving as a review of changes and their implementation, without examining the continuous operation or historic evidence. You just have to convince an ETSI auditor that what you're going to do is correct, not that you did it correctly. If you discover your mistake, you notify them, but they're otherwise not obliged to look for that mistake (this is a difference w/r/t product testing vs ISMS testing). A 'good' ETSI auditor can, in theory, carefully examine historic evidence regarding the production, or more aptly, institute multiple ad hoc audits throughout the certification period and following any changes. I have yet to hear of or see an auditor doing that, however, compared to the WebTrust auditors that I know are doing a good (adversarial) job. > Yes, we've seen auditors not appropriately detecting problems in both > schemes. Both schemes should have a complaint procedure for > third-parties to use, if they consider an audit doesn't meet the > appropriate standards. > And they do, and I'm exercising both, as appropriate. I raised the thread about TUVIT because I fundamentally believe their *approach* to auditing the standards does not appropriately meet the requirements. Under the "WebTrust-like" model, we'd expect TUVIT to demonstrate everything they do and have been doing, as a way of showing that no, in fact, they've been doing pretty reasonable things and this is an abberation. Under an "ETSI-like" model, we'd only have to have them promise to do better (as they have, by suggesting they'd add linting), and then consider the matter resolved. I do not consider the "ETSI-like" model to be reassuring for the risk that it entails from having shoddy auditors. _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
