On Wed, Sep 26, 2018 at 07:36:57AM -0700, josselin.allemandou--- via dev-security-policy wrote: > Thank you for your exchanges. We hope that the additions below will answer > your questions.
It appears that your response has removed most indications of what parts of your message are my questions, compared to your responses. For clarity, I've re-added appropriate formatting, but it would be appreciated if you could use standard e-mail formatting in the future. > > Was the action required to manually override the CAA validation failure > > different from what would be required if the CAA validation had succeeded? > > Could an operator have just "clicked the same button they always clicked", > > and have the CAA validation failure overridden? Or was there a separate > > sequence of UI interactions required to override a CAA validation failure? > > > > This answer does not address the controls in place at the time of the > > misissuance. Any action which an operator can manually take should be able > > to be audited for correctness, but I don't see any mention of that being a > > possibility in your response. I take that to mean that there are no > > functional audit logs, or at least that there are no effective audits > > (sampling or otherwise) of those logs, for decisions and actions undertaken > > by registration agents. > > The check on the authorization to be issued for the organization was the > only one that combined both the verification of the consent form and the > verification of DNS CAA alerts. The operator could view the alert but the > same validation button was used to proceed to the next step, whether the > DNS CAA is valid or not. Thank you for this clear statement of your validation interface design. Unfortunately, this sounds like a design which is extremely risky, from an unintentional errors perspective. What form(s) of review for UX, including but not limited to human factors of safety, were applied to this interface prior to it being deployed? > Each operation performed by an operator is traced so that it can be > audited. The periodic audits of registration requests are also intended > to ensure the conduct of controls by RA and the conformity of their > results. Based on your initial report, I got the impression that the misissuance we're discussing was not picked up by an ordinary operational audit, but was instead identified by some sort of extraordinary review. Is that an accurate impression? If so, can you provide more detail around the criteria you use for selecting operations for auditing, and the frequency of those audits, with particular reference to how such an unusual event (overriding a CAA validation failure) wasn't picked up by ordinary auditing? > > This sounds like there may have been a misunderstanding in the BR > > description of CAA validation. > > > > What specific language in the BRs surrounding CAA validation caused you to > > believe that successful CAA validation was optional in the presence of other > > forms of consent? > > > > What changes do you propose in the language of the CAA validation > > requirements in the BRs to ensure that no other CA could possibly > > misunderstand the CAA validation requirements? > > We have no improvement to issue regarding BRs. We understood the > requirement and implemented the DNS CAA control but we made two errors: > > - The first is to have assessed that the existing control having the same > purpose, made it possible to cover the risks and requirements related to > the control of the DNS CAA. This decision led to wrongly allow RA > operators to validate this checkpoint despite DNS CAA alerts. > > - The second is to have accumulated these two controls (consent + DNS CAA) > in the same validation step because they had the same objective, and > this without imposing that the DNS CAA negative result be blocking. I'm unable to reconcile your two statements. "We understood the requirements" cannot be true at the same time as you made errors in understanding the requirements. It's not logically possible for both of those to be true at the same time. That the misunderstand occured is not at issue. The important question that needs answering is: how did the misunderstanding occur? Perhaps the BRs are insufficiently clear, either on the particulars of the need for CAA validation, or in the larger sense that the prescriptive controls laid out in the BRs cannot be substituted for controls which the CA happens to feel are equivalent. If that is the case, then the BRs need to be clarified in some way, so that other CAs cannot suffer from the same misunderstanding in the future. Alternately, if the BRs *are*, in fact, sufficiently clear in all respects, the only other possibility that comes to my mind is that Certigna failed to correctly interpret the BRs, which is far more concerning -- for Certigna, at least. It would mean that there could be any number of other, as yet unidentified, misunderstandings in Certigna's procedures. I would imagine there would need to be a very comprehensive review of Certigna's processes and procedures, with a detailed public report of the findings of that review, for confidence in Certigna to be restored. > > Have you proposed those changes to the CA/B Forum, to improve the BRs for > > all participants? If not, why not? > > We would recommend to the other CAs to segment each control even if their > objective is the same, in order to be sure that the result of each control > is taken into account and can not be circumvented under the pretext that a > control, having the same purpose, is positive. At the risk of re-iterating the previous point, I'm still at a loss to understand what led Certigna management to believe that substitution of controls that were deemed equivalent was permitted by the BRs or Mozilla policy? > Regarding the control on obtaining the consent of the legal > representative, it is imposed by a national standard that is gradually > aligned with the guidelines of the CA\B Forum, but this standard > unfortunately does not evolve quickly due to administrative constraints. > > We did not perceive the interest of suggesting the use of this method > knowing that it is imposed only on the scale of France and not for other > international CA and that the control of the DNS CAA had in our opinion > the same purpose . > > We want as much as possible that the different national, European and > international standards are harmonized in order to avoid the > implementation of different controls but for the same purposes. Fascinating though this is, including this aside does not really improve the impression you're giving of Certigna's commitment to adhere to the requirements of the Mozilla root store policy. That it may be inconvenient, or expensive, to comply with the requirements of multiple regulatory regimes, does not excuse you from doing so. Certigna's choices in operating its business are exactly that -- Certigna's choices -- and expense or inconvenience do not justify failure to adhere to the requirements. If there are fundamental incompatibilities between regulatory regimes, the BRs provide a mechanism for drawing attention to those incompatibilities, and if Certigna has made use of that mechanism, I would appreciate a reference to that. > > That the BRs have been misunderstood in this fashion is somewhat > > concerning. > > it suggests that there may be other misunderstandings latent in your systems > > and processes, which haven't surfaced because they haven't (yet) led to an > > identified misissuance or other incident. > > > > What steps have you taken, or are you planning on taking, to address > > possible misunderstandings of the BRs or Mozilla policy that may cause > > Certigna to unintentionally misissue a certificate or suffer from some other > > incident? > > > > In what ways have you modified your monitoring, internal audits, and annual > > audits by a certification body in light of the fact that the previous > > editions of those were manifestly insufficient to identify the > > non-compliance of your CAA-related procedures ? > > To address the two errors mentioned above, we have updated the RA > Validation Portal to separate the DNS CAA control and make it blocking > automatically. The other validation steps are well separated. Can you expand on how changing the RA portal around the DNS CAA control addresses any *other* potential vectors for misissuance due to Certigna's misunderstanding of the BRs or Mozilla policy? Given that you've made this response in regards to my questions surrounding these other possible misunderstandings, I assume there is a link, but I'm afraid I can't see it. > The certification body in charge of our audit includes in these audit > criteria the ETSI standards but also the BR and EV guidelines as we > requested. As said above, new auditors are selected for our next > certification audit. This statement suggests that Certigna believes there was some degree of blame to be attached to Certigna's auditors regarding this misissuance (otherwise why raise it as part of this discussion?). If Certigna believes that their auditors are partially at fault, can you expand on why that is? Were they simply incompetent in their duties? If so, have you reported this incompetence to either Mozilla or the CA/B Forum, so that appropriate steps can be taken? If you do not believe the auditor to have been incompetent, what leads you to believe that selecting alternate auditors will necessarily lead to a more satisfactory outcome? Are their changes or improvements to the audit criteria that auditors are required to follow that you can suggest, to prevent future occurrences of this kind? Frankly, blaming the auditor smacks of scapegoating to me; it's like that sequence in Monty Python's Quest for the Holy Grail: "the people responsible for sacking the people responsible have themselves been sacked". It doesn't solve any fundamental problem within the organisation to get new auditors. > > I'm also concerned by your use of the word "blocking" in this context. You > > say that "all controls results [...] are now blocking", which I would > > normally take to mean that there is no way that a registration agent would > > be able to influence the issuance of a certificate in contravention of your > > policies. However, in your answer to Q23 (which I've included below) you > > state that there are a number of validation checks which are performed > > manually, which (I presume, and please provide a detailed correction if I am > > wrong) would allow for a certificate to be misissued as a result of > > misunderstanding, mistake, or malfeasance on the part of a registration > > agent. > > > > Could you please explain further what exactly you mean by "blocking" in the > > context of your above answer? > > Controls are automatic such as DCV or DNS CAA and others are not and are > manually performed by the operators as mentioned above. Whether the > control is done automatically or manually, if the result of only one of > these checks is negative, the request is "blocked" and can not be > validated by a RA operator. No validation and authorization to be issued > is possible by a RA operator if the result of one of the manual or > automatic control points is negative. The request is in an "Invalid" > state while waiting for corrections or proofs to be made to the request. Thanks for that clarification. - Matt _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
