Here are the results of my triage: *CA* *Bugzilla* *Status* HARICA https://bugzilla.mozilla.org/show_bug.cgi?id=1945389 Too new Entrust https://bugzilla.mozilla.org/show_bug.cgi?id=1943528 Too new GlobalSign https://bugzilla.mozilla.org/show_bug.cgi?id=1942879 Set to close GoDaddy https://bugzilla.mozilla.org/show_bug.cgi?id=1942877 Next update is April 1 KIR https://bugzilla.mozilla.org/show_bug.cgi?id=1922572 Next update is March 7 eMudhra https://bugzilla.mozilla.org/show_bug.cgi?id=1916478 <https://bugzilla.mozilla.org/show_bug.cgi?id=1916478#c10> Posted comment DigiCert https://bugzilla.mozilla.org/show_bug.cgi?id=1910805 Active Discussion Chunghwa Telecom https://bugzilla.mozilla.org/show_bug.cgi?id=1903066 Set to close Entrust https://bugzilla.mozilla.org/show_bug.cgi?id=1898848 Closure Summary requested Netlock https://bugzilla.mozilla.org/show_bug.cgi?id=1891331 <https://bugzilla.mozilla.org/show_bug.cgi?id=1891331#c32> Comment posted GDCA https://bugzilla.mozilla.org/show_bug.cgi?id=1889062 New Action Items needed CFCA https://bugzilla.mozilla.org/show_bug.cgi?id=1888882 New Action Items needed Hong Kong Post https://bugzilla.mozilla.org/show_bug.cgi?id=1887888 New Action Items needed Microsec https://bugzilla.mozilla.org/show_bug.cgi?id=1887110 Closure Summary requested Hong Kong Post https://bugzilla.mozilla.org/show_bug.cgi?id=1886665 New Action Items needed Entrust https://bugzilla.mozilla.org/show_bug.cgi?id=1886532 Closure Summary requested Viking Cloud https://bugzilla.mozilla.org/show_bug.cgi?id=1885568 Closure Summary requested Buypass https://bugzilla.mozilla.org/show_bug.cgi?id=1872738 Set to close Ben
On Mon, Feb 10, 2025 at 12:22 PM Tim Callan <[email protected]> wrote: > I appreciate that this matter already has been discussed and accept > Jeremy’s explanation and retraction, and I have no interest in reopening > that conversation. Nonetheless, based on this thread it feels to me that > we would be wise to articulate our motivations. > > > > About five years ago, Sectigo was in a bad place as a public CA. The > company’s commitment to quality of execution, accurate issuance, > responsibility to the WebPKI, and continuous improvement was abysmal. This > manifested itself in a series of severe errors that should not have > occurred accompanied by terrible management of the resulting Bugzilla > incidents. > > > > I sincerely believe that had Sectigo continued down that path, our roots > would be distrusted today > <https://bugzilla.mozilla.org/show_bug.cgi?id=1714628#c11>. Fortunately, > a few thought leaders inside the company had this epiphany, made their > voices heard, and ultimately convinced those around them. This led to > clear policy changes inside Sectigo and began a process that in the long > term required widespread technical, cultural, procedural, and leadership > reform. We forged new (for us at least) philosophies about information, > quality control, operational efficiency, certificate morphology, policy > governance, internal communication, external communication, automation, and > the proper role of humans in our processes. There is pretty much no aspect > of our public CA operation that we did not scrutinize and renovate in the > past five years. While we don’t think we’re perfect by a long shot, we know > beyond doubt that we have come a great distance. > > > > It's been a lengthy, hard journey that has required deep thinking, > unflinching self-examination, guts, persistence, and lots of sweat. So we > appreciate that it can be difficult for a CA to commit to the kind of > improvement that, unfortunately, many of us still clearly need. > > > > Sectigo would like to help. Looking at the behaviors of the WebPKI CA > community, we perceive a common tendency for CAs not to understand the > importance of self-improvement, or know how to go about it, or even > recognize the incredible privilege they have as stewards of public trust. > We see CAs treating the short-term and short-sighted desires of their > corporate owners and local governments as more important than > five-and-a-half billion internet users around the globe. We routinely see > CAs placing their Subscribers’ convenience over the good of the WebPKI. > > > > We see these things, and we have chosen to act. We are trying to show CAs > that self-improvement not only is possible but is an asset to the strength > of your operation. We are trying to spur improvement through encouragement > and information sharing and where necessary strong rhetoric. We try to > provide concrete coaching while challenging our fellows to become their > better selves. > > > > This is a learning process for us. A sad fact is that many CAs appear > quite intractable and unreceptive. But we also see wins, instances where > some CA will experience an authentic “light bulb” moment or a change of > heart. So we know it’s possible to connect in that way. At least on > occasion we see the idea of “CAs helping CAs” actually work. > > > > Doubtless we have made and will continue to make errors along the way, as > we try to figure out how to be sherpas to other CAs who don’t have the > benefit of our last five years’ experience. Surely some will never > appreciate it, or value it, or care in any way. Or for that matter > improve. But some already have. This makes us think that, however > preliminary our efforts and inconsistent the results, there is something > here worth pursuing. > On Friday, February 7, 2025 at 12:34:00 PM UTC-5 Jeremy Rowley wrote: > >> Thanks Mike! Sorry again about the Sectio comment. My own interest >> (because my name is on them) in closing these bugs is pretty obvious.... >> Not an excuse by any means, but I know it is the main reason I reacted so >> poorly. >> >> Overall, I think a lot of the CAs are confused about what they need to do >> to close the bug. I think with the delayed revocation bugs in particular, >> there's a lack of clarity on what the community needs to close the bug as >> resolved considering it was a conscious decision by the CA to delay >> revocation. Although I do not represent a CA, I cannot think of what the CA >> can say or do to ensure that delayed revocation doesn't happen again, >> especially when you can't get browsers to say "Delayed revocation is never >> acceptable". Is there a bug with intentional delayed revocation where the >> action items are ones that were good technical controls to ensure it didn't >> happen again? I'm struggling to think of any that give real good reasons on >> how delayed revocation won't happen again outside of just promising "No >> more delayed revocation". >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1945389 - No comments so far >>> >> >> >> To me this doesn't mean "there is nothing to do with this bug" but >> more likely "in the 5 days since it opened, nobody has bothered to ask >> anything ahead of the full incident report". These days I'm trying to focus >> my limited root-meddling time on inadequate action items, so I generally >> don't bother with preliminary incident reports because I would have to >> guess what the flaws in the future action items would be. >> >> Yeah - I agree this one needs more time and attention. It isn't old and >> should not be closed. >> >> >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1943528 - Entrust was >>> distrusted so no need to keep it open >>> >> >> >> The purpose of a delayed revocation incident is not only to provide >> root programs with visibility into the practices and reliability of CAs. It >> is also to ensure that all (ahem, remaining) CAs have the opportunity to >> learn from the incident such that they might avoid a similar incident in >> the future. Entrust being distrusted does not necessarily mean that there >> is no value in further discussion of the incident, and indeed I think >> generally many incidents are pretty poorly reasoned and explained in those >> matters. >> >> Makes sense. Is Entrust going to keep responding now that they have sold >> the business to Sectigo? I realise Entrust still owns the roots which means >> we need them to reply to the comments going forward. >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1942879 - Issue identified >>> and the delay was unintentional >>> >> >> >> Note that in this case the issue was "malware filtering blocked CPR", >> which was sufficiently distinct from "spam filter blocked CPR" that >> multiple CAs did not extrapolate from the latter to the former when >> monitoring previous incidents. This makes me feel that perhaps more >> attention could be paid by the root programs to explaining how they expect >> CAs to address classes of issue, rather than just the most >> narrowly-interpreted case specifically implicated in an incident. Closing >> the bug ahead of that clarity coming from root programs or, failing that, >> peer CAs or other community members, seems like a missed opportunity to >> avoid future incidents like "we block things that have a non-ASCII sender >> name" or whatever the next fine speciation would be. >> >> I think so too. Be good to get clarity on how to address different >> situations. IMO, an intentional delay of revocation is more egregious than >> something like this where a tool blocked the CPR. I think it's far more >> acceptable where a technical solution caused the issue instead of a human >> doing something wrong as it shows the CA has technology in place to control >> security and (hopefully) automate a lot of the process. >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1916478 - Delay appears >>> unintentional and was an operational issue >>> >> >> >> This is a great example of where the work hasn't been done to make the >> incident useful for other CAs. It contains the old canards about "improving >> procedures" and "training staff", but doesn't detail exactly what was wrong >> with the previous policies or training, or—more importantly—what caused the >> training or policies to have those defects in the first place. Without >> that, all a CA can take away is "have procedures and train staff" which, I >> still permit myself to hope, they all understand at that level of detail >> already. >> >> Agreed on this one. Unintentional delays need technical controls to >> prevent them from reoccuring. This one doesn't have technical controls. >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1898848 - Acknowleded that >>> it was a mistake to delay yet Tim keeps pushing >>> >> >> >> I don't see Tim continuing to push in this bug at all. The last >> comment was some time ago, and the discussion seems to mostly have been >> between Entrust representatives and Wayne. >> >> You're right. This one is one I would close though. >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1889062 - Acknowledges that >>> it was against the BRs to delay >>> >> >> >> This one is definitely not done, given that the subscriber rationale >> was given as "these agencies take too long to replace stuff and they're >> very important in some way", and there was nothing specific in the action >> items or otherwise to indicate how GDCA will navigate that tension in the >> future other than "encouragement" or "make a policy (of unspecified >> contents)". >> >> Okay - what action item should they have? >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1888882 - Comment 28 pretty >>> much states that will ensure awareness of teh 5 day requirement >>> >> >> >> Yep, for sure *this* time, "customer education" will be sufficient. :P >> "When users apply for or obtain certificates, they will be clearly informed >> of the revocation scenarios and time limits" is the sort of thing that's >> hard for me to take seriously when the BRs already required that the >> Subscriber make a legally binding commitment to that effect. >> >> I think these are hard as I know customers cite the previous Mozilla >> policy as allowing delayed revocation. Getting rid of that language will be >> a significant boon. >> >> https://bugzilla.mozilla.org/show_bug.cgi?id=1887888 - This one never >>> answered the question of what they will do going forward. >>> >> >> >> Indeed. Seems like closing in that situation would send an unfortunate >> message that you can just wait it out and the awkward questions will go >> away. >> >> Yeah - I definitely agree with this one. >> >> On Fri, Feb 7, 2025 at 8:41 AM Mike Shaver <[email protected]> wrote: >> >>> Hi Jeremy, >>> >>> I don't agree with your listed reasoning for these bugs all being >>> suitable for closure, though I do genuinely appreciate you taking the time >>> to enumerate them like this. Some responses to selected entries below! >>> >>> On Thu, Feb 6, 2025 at 4:58 PM Jeremy Rowley <[email protected]> wrote: >>> >>>> Perhaps it would be better to look at each delayed revocation bug: >>>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1945389 - No comments so >>>> far >>>> >>> >>> To me this doesn't mean "there is nothing to do with this bug" but more >>> likely "in the 5 days since it opened, nobody has bothered to ask anything >>> ahead of the full incident report". These days I'm trying to focus my >>> limited root-meddling time on inadequate action items, so I generally don't >>> bother with preliminary incident reports because I would have to guess what >>> the flaws in the future action items would be. >>> >>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1943528 - Entrust was >>>> distrusted so no need to keep it open >>>> >>> >>> The purpose of a delayed revocation incident is not only to provide root >>> programs with visibility into the practices and reliability of CAs. It is >>> also to ensure that all (ahem, remaining) CAs have the opportunity to learn >>> from the incident such that they might avoid a similar incident in the >>> future. Entrust being distrusted does not necessarily mean that there is no >>> value in further discussion of the incident, and indeed I think generally >>> many incidents are pretty poorly reasoned and explained in those matters. >>> >>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1942879 - Issue >>>> identified and the delay was unintentional >>>> >>> >>> Note that in this case the issue was "malware filtering blocked CPR", >>> which was sufficiently distinct from "spam filter blocked CPR" that >>> multiple CAs did not extrapolate from the latter to the former when >>> monitoring previous incidents. This makes me feel that perhaps more >>> attention could be paid by the root programs to explaining how they expect >>> CAs to address classes of issue, rather than just the most >>> narrowly-interpreted case specifically implicated in an incident. Closing >>> the bug ahead of that clarity coming from root programs or, failing that, >>> peer CAs or other community members, seems like a missed opportunity to >>> avoid future incidents like "we block things that have a non-ASCII sender >>> name" or whatever the next fine speciation would be. >>> >>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1916478 - Delay appears >>>> unintentional and was an operational issue >>>> >>> >>> This is a great example of where the work hasn't been done to make the >>> incident useful for other CAs. It contains the old canards about "improving >>> procedures" and "training staff", but doesn't detail exactly what was wrong >>> with the previous policies or training, or—more importantly—what caused the >>> training or policies to have those defects in the first place. Without >>> that, all a CA can take away is "have procedures and train staff" which, I >>> still permit myself to hope, they all understand at that level of detail >>> already. >>> >>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1898848 - Acknowleded >>>> that it was a mistake to delay yet Tim keeps pushing >>>> >>> >>> I don't see Tim continuing to push in this bug at all. The last comment >>> was some time ago, and the discussion seems to mostly have been between >>> Entrust representatives and Wayne. >>> >>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1889062 - Acknowledges >>>> that it was against the BRs to delay >>>> >>> >>> This one is definitely not done, given that the subscriber rationale was >>> given as "these agencies take too long to replace stuff and they're very >>> important in some way", and there was nothing specific in the action items >>> or otherwise to indicate how GDCA will navigate that tension in the future >>> other than "encouragement" or "make a policy (of unspecified contents)". >>> >>> >>>> https://bugzilla.mozilla.org/show_bug.cgi?id=1888882 - Comment 28 >>>> pretty much states that will ensure awareness of teh 5 day requirement >>>> >>> >>> Yep, for sure *this* time, "customer education" will be sufficient. :P >>> "When users apply for or obtain certificates, they will be clearly informed >>> of the revocation scenarios and time limits" is the sort of thing that's >>> hard for me to take seriously when the BRs already required that the >>> Subscriber make a legally binding commitment to that effect. >>> >>> Also, that report describes an action item (maybe? a commitment at >>> least) that was to be completed a month ago, and for which no status has >>> been given: >>> >>> > This action plan will take a long time, but we will still accelerate >>> the progress to encourage as many of our existing customers with larger >>> certificate volumes to use the automated certificate deployment system as >>> much as possible by 2025. >>> >>> Unless the goal is just to complete the *encouragement* by 2025, in >>> which case it's a nothing-burger again. >>> >>> https://bugzilla.mozilla.org/show_bug.cgi?id=1887888 - This one never >>>> answered the question of what they will do going forward. >>>> >>> >>> Indeed. Seems like closing in that situation would send an unfortunate >>> message that you can just wait it out and the awkward questions will go >>> away. >>> >>> >>>> The value to Sectigo and the Sectigo podcast is pretty obvious, >>>> >>> >>> This is bullshit, unwelcome on the list, and quite surprising. I know >>> you apologized but I still can't let it just go by when I'm replying to the >>> message. >>> >>> Mike >>> >>> -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/CA%2B1gtaZBrt4cwKFcdr-m_SRPTWhczojVsAhxo%3DnsrvC--%2BD7SQ%40mail.gmail.com.
