Re: Root Store Policy Suggestion
On Thu, Jan 28, 2021 at 7:33 PM Ryan Sleevi wrote: > > > On Thu, Jan 28, 2021 at 1:32 PM Burton wrote: > >> Hi Ryan, >> >> The answer to your questions. >> >> A remediation plan is only useful in cases of slight CA non-compliance to >> the rules set forth by the root store policy. >> >> A remediation plans in cases of slight CA non-compliance provides >> assurance of CA commitment to compliance. >> > > Sure, and I think (and hopefully I'm fairly stating), that the goal is > these should be provided in the Incident Reports themselves. That is, the > remediation should address both the immediate and systemic issues, and > future incidents of the CA will be judged against this. > > The intent is certainly that anyone in the community participates and > reviews these, and I think we see a lot of fantastic activity on the bug > reports from people who do, which is a healthy sign, even though they're > often calling out concerns with the remediation or highlighting how it > fails to meet the expectations. > > >> A CA under investigation of serious non-compliance with detailed >> documented evidence of non-compliance incidents has reach the stage of no >> return. >> >> A remediation plan in the cases of serious non-compliance is a reference >> document in the case of new root inclusion as documented evidence of >> commitment to compliance. >> > >> The CA roots should be removed in the case of serious non-compliance and >> asked to reapply for inclusion again to the root store with new roots and >> new commitment to compliance with new audits from a different auditor and >> reformed practices and management. >> > > Right, and I think this might be premature or giving false hope, at least > to CAs that assume every CA, once removed, can simply reapply with a > remediation plan. I agree with you, it's incredibly valuable to understand > how the CA plans to address the issues, and just like incident reports, > it's useful to understand how the CA views the incidents that might lead up > to distrust and how it plans to mitigate them before reapplying. Yet we've > often seen CAs believe that because a remediation plan exists for the > identified issues, it's sufficient to apply for new roots, when really, > such CAs are working from a serious trust deficit, and so not only need to > remediate the identified issues, but show how they're going above and > beyond addressing the systemic issues, in order to justify the risk of > trusting them again. Understandably, this depends on a case-by-case basis. > > To your original point, historically CA actions (generally) worked in > three phases: > > 1) A pattern is believed to exist (of incidents), or an incident is so > severe it warrants immediate public discussion. The community is asked to > provide details - e.g. of incidents that were overlooked, of other relevant > data - to ensure that a full and comprehensive picture of relevant facts > are gathered and understood. The CA is invited to share details (e.g. how > they mitigated such issues) or to respond to the facts, if they believe > they're not accurate. > > 2) A discussion about the issues themselves, to evaluate the nature of the > incidents, as well as solicit proposals from the community in particular > (rather than the CA, although the CA is welcome to contribute) about how to > mitigate the risks these issues and incidents highlight. > > 3) At least for Mozilla, a proposed plan for Mozilla products, which is > often based on suggestions from the community (in #2) as well as Mozilla's > own product and security considerations. Mozilla may solicit further > feedback on their plan, from the community and the CA, to make sure they've > balanced the concerns and considerations raised in #2 accurately, or may > decide it warrants immediate action. > > This is a rough guide, obviously there are exceptions. For example, > Mozilla and other browsers blocking MITM roots hasn't always involved all > three stages. Similarly, in CA compromise events, Step 2 and 3 may be > skipped entirely, because the only viable solution is obvious. > > Other programs, whether Apple, Google, or Microsoft, don't necessarily > operate the same way. For example, Google, Apple and Microsoft don't > provide any statement at all about public engagement, although they may > closely monitor the discussions in #1 and #2. > > Step #1 has, intentionally and by design, largely been replaced by the > Incident Reporting requirements incorporated into the Root Policies of both > Mozilla and Google Chrome. That is, the incident reports, and the public > discussions of the incidents, serve to contemporaneously address issues, > identify remediations, and understand and identify how well the CA > understands the risks and is able to take meaningful corrective action. > These days, Step #1 is merely summarizing the incidents based on the > information in the incidents, and thus may not need the same lengthy > discussion in the past, prior to the incident
Re: Root Store Policy Suggestion
On Thu, Jan 28, 2021 at 1:32 PM Burton wrote: > Hi Ryan, > > The answer to your questions. > > A remediation plan is only useful in cases of slight CA non-compliance to > the rules set forth by the root store policy. > > A remediation plans in cases of slight CA non-compliance provides > assurance of CA commitment to compliance. > Sure, and I think (and hopefully I'm fairly stating), that the goal is these should be provided in the Incident Reports themselves. That is, the remediation should address both the immediate and systemic issues, and future incidents of the CA will be judged against this. The intent is certainly that anyone in the community participates and reviews these, and I think we see a lot of fantastic activity on the bug reports from people who do, which is a healthy sign, even though they're often calling out concerns with the remediation or highlighting how it fails to meet the expectations. > A CA under investigation of serious non-compliance with detailed > documented evidence of non-compliance incidents has reach the stage of no > return. > > A remediation plan in the cases of serious non-compliance is a reference > document in the case of new root inclusion as documented evidence of > commitment to compliance. > > The CA roots should be removed in the case of serious non-compliance and > asked to reapply for inclusion again to the root store with new roots and > new commitment to compliance with new audits from a different auditor and > reformed practices and management. > Right, and I think this might be premature or giving false hope, at least to CAs that assume every CA, once removed, can simply reapply with a remediation plan. I agree with you, it's incredibly valuable to understand how the CA plans to address the issues, and just like incident reports, it's useful to understand how the CA views the incidents that might lead up to distrust and how it plans to mitigate them before reapplying. Yet we've often seen CAs believe that because a remediation plan exists for the identified issues, it's sufficient to apply for new roots, when really, such CAs are working from a serious trust deficit, and so not only need to remediate the identified issues, but show how they're going above and beyond addressing the systemic issues, in order to justify the risk of trusting them again. Understandably, this depends on a case-by-case basis. To your original point, historically CA actions (generally) worked in three phases: 1) A pattern is believed to exist (of incidents), or an incident is so severe it warrants immediate public discussion. The community is asked to provide details - e.g. of incidents that were overlooked, of other relevant data - to ensure that a full and comprehensive picture of relevant facts are gathered and understood. The CA is invited to share details (e.g. how they mitigated such issues) or to respond to the facts, if they believe they're not accurate. 2) A discussion about the issues themselves, to evaluate the nature of the incidents, as well as solicit proposals from the community in particular (rather than the CA, although the CA is welcome to contribute) about how to mitigate the risks these issues and incidents highlight. 3) At least for Mozilla, a proposed plan for Mozilla products, which is often based on suggestions from the community (in #2) as well as Mozilla's own product and security considerations. Mozilla may solicit further feedback on their plan, from the community and the CA, to make sure they've balanced the concerns and considerations raised in #2 accurately, or may decide it warrants immediate action. This is a rough guide, obviously there are exceptions. For example, Mozilla and other browsers blocking MITM roots hasn't always involved all three stages. Similarly, in CA compromise events, Step 2 and 3 may be skipped entirely, because the only viable solution is obvious. Other programs, whether Apple, Google, or Microsoft, don't necessarily operate the same way. For example, Google, Apple and Microsoft don't provide any statement at all about public engagement, although they may closely monitor the discussions in #1 and #2. Step #1 has, intentionally and by design, largely been replaced by the Incident Reporting requirements incorporated into the Root Policies of both Mozilla and Google Chrome. That is, the incident reports, and the public discussions of the incidents, serve to contemporaneously address issues, identify remediations, and understand and identify how well the CA understands the risks and is able to take meaningful corrective action. These days, Step #1 is merely summarizing the incidents based on the information in the incidents, and thus may not need the same lengthy discussion in the past, prior to the incident disclosure requirements (e.g. StartCom, WoSign). Step #2 is still widely practiced, as we've seen throughout a number of recent and past events. Without wanting to put words into Mozilla's mouth, certainly
Re: Root Store Policy Suggestion
Hi Ryan, The answer to your questions. A remediation plan is only useful in cases of slight CA non-compliance to the rules set forth by the root store policy. A remediation plans in cases of slight CA non-compliance provides assurance of CA commitment to compliance. A CA under investigation of serious non-compliance with detailed documented evidence of non-compliance incidents has reach the stage of no return. A remediation plan in the cases of serious non-compliance is a reference document in the case of new root inclusion as documented evidence of commitment to compliance. The CA roots should be removed in the case of serious non-compliance and asked to reapply for inclusion again to the root store with new roots and new commitment to compliance with new audits from a different auditor and reformed practices and management. Thank you Burton On Wed, 27 Jan 2021, 19:54 Ryan Sleevi, wrote: > > > On Wed, Jan 27, 2021 at 2:45 PM Burton wrote: > >> I included the remediation plan in the proposal because a CA will mostly >> always include a remediation plan when they reach the stage of serious >> non-compliance investigation by root store policy owners. >> > > Sure, but I was more asking: are you aware of any point in the past where > the remediation plan has been valuable, useful or appropriate? I'm not. > > The expectation is continuous remediation, so any remediation plan at a > later stage seems too little, too late, right? The very intentional goal of > the incident reporting was to transition to a continuous improvement > process, where the CA was evaluated based on their > contemporaneous remediation to incidents, rather than waiting until things > get so bad they pile up and a remediation plan is used. > > So I'm trying to understand what a remediation plan would include, during > discussion, that wouldn't (or, more explicitly, shouldn't) have been > included in the incident reports as they happened? > >> ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Root Store Policy Suggestion
Hi Ryan, These are good questions! I'll get back to you tomorrow with the answers to your questions. I want to research and give you the right information. Thank you Burton On Wed, Jan 27, 2021 at 7:54 PM Ryan Sleevi wrote: > > > On Wed, Jan 27, 2021 at 2:45 PM Burton wrote: > >> I included the remediation plan in the proposal because a CA will mostly >> always include a remediation plan when they reach the stage of serious >> non-compliance investigation by root store policy owners. >> > > Sure, but I was more asking: are you aware of any point in the past where > the remediation plan has been valuable, useful or appropriate? I'm not. > > The expectation is continuous remediation, so any remediation plan at a > later stage seems too little, too late, right? The very intentional goal of > the incident reporting was to transition to a continuous improvement > process, where the CA was evaluated based on their > contemporaneous remediation to incidents, rather than waiting until things > get so bad they pile up and a remediation plan is used. > > So I'm trying to understand what a remediation plan would include, during > discussion, that wouldn't (or, more explicitly, shouldn't) have been > included in the incident reports as they happened? > >> ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Root Store Policy Suggestion
On Wed, Jan 27, 2021 at 2:45 PM Burton wrote: > I included the remediation plan in the proposal because a CA will mostly > always include a remediation plan when they reach the stage of serious > non-compliance investigation by root store policy owners. > Sure, but I was more asking: are you aware of any point in the past where the remediation plan has been valuable, useful or appropriate? I'm not. The expectation is continuous remediation, so any remediation plan at a later stage seems too little, too late, right? The very intentional goal of the incident reporting was to transition to a continuous improvement process, where the CA was evaluated based on their contemporaneous remediation to incidents, rather than waiting until things get so bad they pile up and a remediation plan is used. So I'm trying to understand what a remediation plan would include, during discussion, that wouldn't (or, more explicitly, shouldn't) have been included in the incident reports as they happened? > ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Root Store Policy Suggestion
Hi Ryan, I included the remediation plan in the proposal because a CA will mostly always include a remediation plan when they reach the stage of serious non-compliance investigation by root store policy owners. The first remediation plan is always a draft version as it's updated as the discussion is ongoing. Take the recent discussion for example, the remediation plan is version 5 which seems to be the final version. New evidence does sometimes appear in public discussions about the CA that wasn't found or documented earlier and I included that in the proposal as requests for additional information. I do agree that CAs who reach the stage of non-compliance investigation should submit new roots. Thank you Burton On Wed, Jan 27, 2021 at 6:58 PM Ryan Sleevi wrote: > > > On Wed, Jan 27, 2021 at 10:11 AM Burton wrote: > >> Hello, >> >> The Mozilla root store policy should include a section that sets out time >> limit periods in numbered stages for non-compliance CA discussions. That >> way everything is fair, can't be disputed and everyone knows when the >> discussion of the non-compliance CA will conclude. Then the decision from >> the root store policy owners will proceed shortly afterwards to either >> accept the remediation plan from the CA or proceed with the partial or >> complete removal of the CA from the root store. >> >> These time limits below should be enough ample time for the discussion to >> take place between the CA, the community and the root store policy owners. >> >> Stage 1 (Discussion Period: *1 Week*): >> >>- Official notification to all that an investigation regarding the >>non-compliance issues of the CA has started. >>- Requests for additional information, etc. >> >> Stage 2 (Discussion Period: *4 Weeks*): >> >>- The CA to produces a draft remediation plan. >>- The CA answers all questions from the root store policy owners and >>the community. >>- Requests for additional information, etc. >> >> Stage 3 (Discussion Period: *2 Weeks*): >> >>- Discussion of the final remediation plan proposed by the CA. >>- Discussion of whether to partial distrust or full distrust of the >>CA. >>- Requests for anymore additional information. >> >> The decision by the root store policy owners. >> > > From a proposal standpoint, could you explain a bit more about the goals > of a "draft remediation plan"? > > The point and purpose of Incident Reports is so that CAs are continuously > improving, and adopting meaningful improvements and remediations. At the > point at which a CA is being discussed, isn't that already evidence alone > that the CA's failure to remediate not just the incidents, but the systemic > root issues, has failed? > > Recent replies such as > https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg14089.html > and > https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg14090.html > have captured how, given the nature of how PKI works, the only viable > remediation plans are those that transition to new roots. This is something > that, for example, Google Chrome explicitly recommends and encourages, at > https://g.co/chrome/root-policy , as a way of minimizing the risk to > their users. > > Without expressing opinions on the current, ongoing discussion for > Mozilla, it may be useful to examine past CA incidents, as it seems that > there were several remediation plans posed by CAs in the past, yet > consistently, every single one of them failed to adequately address the > very core issues, and every one of them revealed actions the CA was already > expected to be taking, either in light of incidents or as part of the bare > minimum expectation of CAs. > > I mention all of this because your emphasis on remediation plan does seem > to suggest you see there being viable paths forward, other than (at a > minimum), a transition from problematic roots to new roots, so I was hoping > you could expand, using perhaps past CA discussions rather than the current > one, as they provide ample evidence of problematic patterns. > ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
Re: Root Store Policy Suggestion
On Wed, Jan 27, 2021 at 10:11 AM Burton wrote: > Hello, > > The Mozilla root store policy should include a section that sets out time > limit periods in numbered stages for non-compliance CA discussions. That > way everything is fair, can't be disputed and everyone knows when the > discussion of the non-compliance CA will conclude. Then the decision from > the root store policy owners will proceed shortly afterwards to either > accept the remediation plan from the CA or proceed with the partial or > complete removal of the CA from the root store. > > These time limits below should be enough ample time for the discussion to > take place between the CA, the community and the root store policy owners. > > Stage 1 (Discussion Period: *1 Week*): > >- Official notification to all that an investigation regarding the >non-compliance issues of the CA has started. >- Requests for additional information, etc. > > Stage 2 (Discussion Period: *4 Weeks*): > >- The CA to produces a draft remediation plan. >- The CA answers all questions from the root store policy owners and >the community. >- Requests for additional information, etc. > > Stage 3 (Discussion Period: *2 Weeks*): > >- Discussion of the final remediation plan proposed by the CA. >- Discussion of whether to partial distrust or full distrust of the >CA. >- Requests for anymore additional information. > > The decision by the root store policy owners. > >From a proposal standpoint, could you explain a bit more about the goals of a "draft remediation plan"? The point and purpose of Incident Reports is so that CAs are continuously improving, and adopting meaningful improvements and remediations. At the point at which a CA is being discussed, isn't that already evidence alone that the CA's failure to remediate not just the incidents, but the systemic root issues, has failed? Recent replies such as https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg14089.html and https://www.mail-archive.com/dev-security-policy@lists.mozilla.org/msg14090.html have captured how, given the nature of how PKI works, the only viable remediation plans are those that transition to new roots. This is something that, for example, Google Chrome explicitly recommends and encourages, at https://g.co/chrome/root-policy , as a way of minimizing the risk to their users. Without expressing opinions on the current, ongoing discussion for Mozilla, it may be useful to examine past CA incidents, as it seems that there were several remediation plans posed by CAs in the past, yet consistently, every single one of them failed to adequately address the very core issues, and every one of them revealed actions the CA was already expected to be taking, either in light of incidents or as part of the bare minimum expectation of CAs. I mention all of this because your emphasis on remediation plan does seem to suggest you see there being viable paths forward, other than (at a minimum), a transition from problematic roots to new roots, so I was hoping you could expand, using perhaps past CA discussions rather than the current one, as they provide ample evidence of problematic patterns. ___ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
