Jeremy, Thanks for attaching the diagrams - this is very useful in helping visualize out the graph! Special thanks for detailing out the validation flow DigiCert both practices and plans to practice - this level of transparency goes a long way to helping assess and understand both risks and mitigations.
I think we can divide the discussion into two parts, similar to the previous mail: How to effectively transition Symantec customers with minimum disruption, whether acting as the Managed CA or as the future operator of Symantec’s PKI, and how to effectively transition DigiCert’s infrastructure. This is a slightly different order than your e-mail message, but given the time sensitivity of the Symantec transition, it seems more effective to discuss that first. I think there may have been some confusion on the Managed CA side. It’s excellent that DigiCert plans to transition Symantec customers to DigiCert roots, as that helps with an expedient reduction in risk, but the plan outlined may create some of the compatibility risks that I was trying to highlight. In the discussions of the proposed remediations, one of the big concerns we heard raised by both Symantec and site operators was related to pinning - both in the Web and in mobile applications. We also heard about embedded or legacy devices, and their needs for particular chains. The plan you’ve outlined doesn’t seem to clearly account for this, which was a key factor in the “Managed CA” design. In the previous reply, I tried to highlight a bit more of at least one technically viable way to accomplish that - what we might call the “Managed CA Lookalike” infrastructure, in that it follows the structural outline and requirements of the Managed CA plan, even if DigiCert may be performing all the validation themselves and not relying on any previous data or documents (which I assume is what you mean by “Transitioned to DigiCert”). To make sure we’re on the same page, when you say “DigiCert Global Root G2”, you mean the certificate at https://crt.sh/?q=DF3C24F9BFD666761B268073FE06D1CC8D4F82A4, right? This CA, in turn, has signed “DigiCert Global CA G2”, https://crt.sh/?caid=5886 , which in turn has signed a number of end entity certificates - https://crt.sh/?Identity=%25&iCAID=5886 . While it does not appear to be a large number of certificates (looks like roughly 84, with some compatibility testing certificates issued), these certificates may stop working - and would need to migrate. However, this doesn’t address the main concern that informed the requirements of the “Managed CA,” in response to community feedback. You propose signing the Root G2 with Verisign Class 3 Public Primary Certificate Authority – G5, https://crt.sh/?id=93 . This could reduce the risk if someone pinned to that root, or https://crt.sh/?caid=443 (because of https://crt.sh/?id=21606053 ), or https://crt.sh/?caid=25 ( because of https://crt.sh/?id=2503596 ), which is fantastic, but wouldn’t address situations like https://crt.sh/?caid=808 (which issued https://crt.sh/?caid=1212 which is actively issuing certs - https://crt.sh/?Identity=%25&iCAID=1212 ). Any of those customers who pinned would potentially be negatively impacted, as no other cross-signs exist for this path. The Managed CA process was meant to address these concerns, by allowing at least a 1:1 transition of the existing roots to new roots without the same risk - by taking the existing roots, signing a new (DigiCert-operated) “Managed CA”, and then transitioning online issuance to a hierarchy within that new “Managed CA”. This doesn’t address situations where applications pinned to intermediates - but unfortunately, I don’t have any suggestions on how to effectively address that use case while also achieving the necessary reduction in risk. It sounds like this plan may have been based on a concern that I’d tried to address in the previous message. That is, the removal of the existing Symantec roots defines a policy goal - the elimination in trust in these legacy roots, due to the unknown scope of issues. However, that goal could be achieved by a number of technical means - for example, ‘whitelisting’ a set of Managed CAs (as proposed by Chrome), or replacing the existing Symantec roots with these new Managed CA roots in a 1:1 swap. Both of these approaches achieve the same policy objective, while reducing the compatibility risk. Note that another part of the Managed CA proposal was to allow for the (limited) reuse of information, due to Symantec’s assertions that no partner could be found that could match the volume of issuance necessary to avoid customer disruption. Based on your description of the plan, it sounds like you intend to transition all of the Symantec customers over on Dec 1 to DigiCert roots - but that seems like it may result in an overload of your support team if path building or pinning issues are discovered, and your policy and compliance team as solutions have to be explored. As communicated in the past, an acquisition of Symantec’s PKI doesn’t change the requirement that the transition certificates be identified and established by Dec 1 - so it would not be possible to address any of these concerns after that point. By using this intermediate "Managed CA" step, and balancing with a reduced certificate lifetime to help limit how long DigiCert would need to operate this infrastructure, would offer a way to smooth out support load and work through any support issues. This is particularly useful when considering what DigiCert’s plans will be for the future and it’s PKI. It’s really great to hear that DigiCert realizes that root segmentation isn’t an effective mitigation strategy, and it’s encouraging to hear there are plans to address that. Hopefully, by looking at the complexity of Symantec’s PKI, you can see how some of these risks can manifest, and why creating a large number of intermediates may also be undesirable. A potentially better way to shard is arguably not by organization or purpose - since they all run on DigiCert’s infrastructure, they would all be treated the same for any matters (like revocation or systemic issues) - but to shard by date. The reason to shard by date allows root programs to effectively respond to any incidents - only having to examine the certificates under the ‘problem period’. It also has a side-benefit to your customers - by the periodic rotation of intermediates, it discourages pinning by intermediate (since they regularly rotate), thus reducing the impact to customers if an intermediate ever does have to be revoked. Hopefully these suggestions help work on a solid transition plan - first, for DigiCert as Symantec’s Managed CA, second, for DigiCert looking to acquire Symantec’s PKI business with minimal disruption to the ecosystem, and third, for DigiCert’s long-term operations. On Wed, Sep 20, 2017 at 12:39 PM, Jeremy Rowley via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Thanks a ton, Ryan! This was very helpful, and we really appreciate the > feedback and suggestions. Here’s what we currently use as publicly-trusted > roots and how we use them: > > > > 1. Baltimore CyberTrust Root – Expires in 2025. Currently only used > to support Verizon customers who have not transitioned to DigiCert’s roots > for whatever reason (SubCAs, Pinned keys, etc) > 2. CyberTrust Global Root – Expires in 2021 (I think) and used > primarily in Japan. This is our EV root for customers who need trust in > Japanese systems. > 3. DigiCert Assured ID Root CA – RSA root that issues primarily > client certs and code signing certificates. The root is trusted in Adobe > and cross-signed by the FBCA (one-way trust). Although it issues some TLS > certs, these TLS are only issued when a special cross-certification is > required that is no available under the other roots. > 4. DigiCert Assured ID Root G2 – RSA rollover root for the Assured ID > root. Once sufficient ubiquity exists, the plan is to retire the Assured ID > Root in favor of this one. > 5. DigiCert Assured ID Root G3 – ECC version of the G2 root. This > root is for entities who want a complete ECC chain. > 6. DigiCert Global Root CA – RSA root that is used for OV and code > signing certificates. This is our primary issuing root and is cross-signed > by Baltimore for additional ubiquity. > 7. DigiCert Global Root G2 – RSA rollover root intended to replace > the Global Root CA. > 8. DigiCert Global Root G3 – ECC version of the G2 root used to > support entities who want a complete ECC chain > 9. DigiCert High Assurance EV Root CA – Our only EV root until this > bug is complete: https://bugzilla.mozilla.org/show_bug.cgi?id=1165472. > This root is primarily used for entities who want one or more EV > certificates and is cross-signed by the Baltimore root for ubiquity. > 10. DigiCert Trusted Root CA – A 4096 bit root in case 2048 is no > longer sufficient. This is cross-signed by the Global Root. > > > > Symantec has the following roots according to CCADB. Symantec would have > to comment on how these are used. > > 1. Symantec Class 1 G4 > 2. Symantec Class 1 G6 > 3. Symantec Class 2 G4 > 4. Symantec Class 2 G6 > 5. Symantec Class 3 G4 > 6. Symantec Class G6 > 7. Symantec Enterprises Mobile > 8. Equifax Secure Certificate Authority > 9. Equifax Secure eBusiness CA-1 > 10. Equifax Secure Global eBusiness CA-1 > 11. GeoTrust Global CA > 12. GeoTrust Global CA 2 > 13. GeoTrust Primary Certificate Authority > 14. GeoTrust Primary Certificate Authority – G2 > 15. GeoTrust Primary Certificate Authority – G3 > 16. GeoTrust Universal CA > 17. GeoTrust Universal CA 2 > 18. Thawte Premium Server CA > 19. Thawte Primary Root CA > 20. Thawte Primary Root CA – G2 > 21. Thawte Primary Root CA – G3 > 22. Thawte Server CA > 23. Thawte Timestamping CA > 24. UTN-Userfirst-Network Applications > 25. Verisign Class 1 Public PCA > 26. Verisign Class 3 Public Primary Certificate Authority – G4 > 27. Verisign Class 3 Public Primary Certificate Authority – G5 > 28. Verisign Class 4 Public Primary Certificate Authority – G3 > 29. Verisign Time Stamping CA > 30. Verisign Universal Root Certificate Authority > 31. Verisign4 > 32. Verisign Class 1 Public PCA – G3 > 33. Verisign Class 1 Public PCA – G2 > 34. Verisign Class 2 Public PCA – G3 > 35. Verisign Class 2 Public PCA – G2 > 36. Verisign Class 3 Public PCA > 37. Verisign Class 3 Public PCA – MD2 > 38. Verisign Class 3 Public PCA – G2 > 39. Verisign Class 2 Public Primary Certification Authority – G3 > > > > The current end-state plan for root cross-signing is provided at > https://bugzilla.mozilla.org/show_bug.cgi?id=1401384. The diagrams there > show all of the existing sub CAs along with the new Sub CAs and root > signings planned for post-close. Some of these don’t have names so they are > lumped in a general “Intermediate” box. > > > > We reached the same conclusion as you about segmentation by root (that > compartmentalization by root won’t work), not to mention that > compartmentalization will be near impossible considering what we’ve > previously issued and how the roots are trusted in various root programs. > Instead, we plan on creating sub CAs based on the number of entities using > the intermediate. For example, every 2000 or so entities will use another > Sub CA. This will roughly segment customers based on excepted volumes. We > also plan on providing a lot more unique intermediates on a per customer > basis. Permitting large companies to have a dedicated intermediate will > help shield them from being revoked if another Sub CA needs to be revoked > and allow browsers to selectively whitelist/blacklist entities. Of course, > not every company will want this, but it’ll be available for anyone who > wants it. > > > > The plan, based on your suggestions, is to cross-sign the DigiCert Global > G2 root with the four Symantec roots: > > > > 1. GeoTrust Global CA > 2. GeoTrust Global CA 2 > 3. Verisign Class 3 Public Primary Certificate Authority – G5 > 4. Thawte Primary Root CA > > > > The exact roots cross-signing the DigiCert root is very much in flux. > Until close, we aren’t reaching out to current Symantec customers for, I > think, obvious reasons. However, we do plan on communicating with these > customers immediately post close to determine which roots are pinned in > applications and what roots are required for custom applications. We are > trying to limit the number of primary roots to six (three ECC, three RSA) > plus one transition root to keep the chains and use manageable. > > > > The Global G2 root will become the transition root to DigiCert for > customers who can’t move fully to an operational DigiCert roots prior to > September 2018. Any customers that require a specific root can use the > transition root for as long as they want, realizing that path validation > may be an issue as Symantec roots are removed by platform operators. > Although we cannot currently move to a single root because of the lack of > EV support and trust in non-Mozilla platforms, we can move to the existing > three roots in an orderly fashion. > > > > If the agreement closes prior to Dec 1, the Managed CA will never exist. > Instead, all issuance will occur through one of the three primary DigiCert > roots mentioned above with the exception of customers required to use a > Symantec root for certain platforms or pinning. The cross-signed Global > root will be only transitory, meaning we’d hope customers would migrate to > the DigiCert roots once the systems requiring a specific Symantec roots are > deprecated or as path validation errors arise. > > > > Jeremy > > > > > > From: Ryan Sleevi [mailto:r...@sleevi.com] > Sent: Thursday, September 14, 2017 1:28 PM > To: Jeremy Rowley <jeremy.row...@digicert.com> > Cc: mozilla-dev-security-pol...@lists.mozilla.org > Subject: Re: DigiCert-Symantec Announcement > > > > > > > > On Wed, Aug 2, 2017 at 5:12 PM, Jeremy Rowley via dev-security-policy < > dev-security-policy@lists.mozilla.org <mailto:dev-security-policy@ > lists.mozilla.org> > wrote: > > Hi everyone, > > > > Today, DigiCert and Symantec announced that DigiCert is acquiring the > Symantec CA assets, including the infrastructure, personnel, roots, and > platforms. At the same time, DigiCert signed a Sub CA agreement wherein we > will validate and issue all Symantec certs as of Dec 1, 2017. We are > committed to meeting the Mozilla and Google plans in transitioning away > from > the Symantec infrastructure. The deal is expected to close near the end of > the year, after which we will be solely responsible for operation of the > CA. > From there, we will migrate customers and systems as necessary to > consolidate platforms and operations while continuing to run all issuance > and validation through DigiCert. We will post updates and plans to the > community as things change and progress. > > > > I wanted to post to the Mozilla dev list to: > > 1. Inform the public, > 2. Get community feedback about the transition and concerns, and > 3. Get an update from the browsers on what this means for the plan, > noting that we fully commit to the stated deadlines. We're hoping that any > changes > > > > Two things I can say we plan on doing (following closing) to address > concerns are: > > a. We plan to segregate certs by type on each root. Going forward, we > will issue all SSL certs from a root while client and email come from > different roots. We also plan on limiting the number of organizations on > each issuing CA. We hope this will help address the "too big to fail" > issue > seen with Symantec. By segregating end entities into roots and sub CAs, > the > browsers can add affected Sub CAs to their CRL lists quickly and without > impacting the entire ecosystem. This plan is very much in flux, and we'd > love to hear additional recommendations. > b. Another thing we are doing is adding a validation OID to all of our > certificates that identifies which of the BR methods were used to issue the > cert. This way the entire community can readily identify which method was > used when issuing a cert and take action if a method is deemed weak or > insufficient. We think this is a huge improvement over the existing > landscape, and I'm very excited to see that OID rolled out. > > > > Thanks a ton for any thoughts you offer. > > > > Jeremy > > > > eremy, > > > > Thanks for sharing details about your rough plans. There’s a lot at play > here, particularly when trying to fully visualize DigiCert’s existing and > proposed hierarchy, so I’m wondering if it might be easier to explore what > the ‘ideal PKI’ may look like, and then try to work backwards to figure out > how this acquisition can help that. > > > > At the core, we can imagine there is a root CA for each major long-term > cryptographic configuration - which, in today’s world, generally means > RSA/2048, RSA/4096, ECC/256, and ECC/384. In tomorrow’s world, this may > also accommodate additional curves Ed25519 and Ed448, such as via > https://tools.ietf.org/id/draft-ietf-curdle-pkix . In total, this means > the ideal PKI only needs four to six root certificates. > > > > Within each root, you can build out the appropriate segmentation. For > performance reasons, it’s likely preferable to have a ‘wide’ PKI (many > sub-CAs off the root, each constrained in capability and used for a limited > amount of time) versus a ‘deep’ PKI (hierarchically reducing the > capabilities at each level in the trust graph- for example, “All TLS” -> > “All DV” -> “All first party DV” -> “All first party DV in Q12017”), even > if a deep PKI can provide better compartmentalization for some use cases. > > > > It isn’t clear that compartmentalizing on root provides any obvious > benefits to users, especially as it’s the same infrastructure and audits, > but it does seem that it increases the management overhead (for root > stores), the configuration challenges (for site operators), not to mention > the management (and, occasionally, network & memory) challenges for users > to support all of those roots. > > > > It would be ideal to see DigiCert streamline its PKI to better align with > that vision, and to understand what challenges might prevent that. For > example, is there a path to transition all new DigiCert issuance to a > single root? If it can’t be done for all certs, can it be done for TLS? > Understanding if there are challenges to that goal can provide invaluable > insight into how the Managed CA transition can look. > > > > A significant reason for the Managed CA plan was to provide a temporary > bridge for those TLS servers who had made risky and fragile technical > decisions - such as pinning to a single CA or only supporting a single CA > on a device - while minimizing the risk to the broader TLS ecosystem. As > Symantec, like other organizations wishing to operate a trusted CA, would > be permitted to apply to have new roots (and a new infrastructure) > included, once it had met the minimum required security standards, the > Managed CA only needed to serve as a transition point until either Symantec > had implemented new infrastructure or the community had migrated away. > > > > Conceptually, this bridge concept is somewhat similar to how SHA-1 was > handled. In the lead-up to SHA-1’s deprecation, CAs self-removed a number > of roots, on the basis that they planned these legacy roots to be used for > SHA-1 issuance, and thus no longer be BR audited. Customers, both existing > and new, were given SHA-256 certificates by default, but, should there be a > demonstrated technical need for the legacy solution, a SHA-1 certificate > could be provided that would work with older devices. > > > > The main difference between that SHA-1 transition and the proposal here is > that the community recognizes that an immediate migration away from > Symantec’s old infrastructure could present challenges, and thus a path was > provided that allowed a limited degree of public trust for those > compatibility certificates, with the assumption and expectation that it > would be an interim solution towards either an eventual reapplication for > trust or an ultimate sunset of this managed infrastructure. > > > > How does this all practically play out? One way to effectively manage this > as follows: Identify the existing Symantec hierarchy, mapping out every > logical CA (same name and key) and all the permutations of certificates for > those names and keys - including cross-signs. Graphing this out is strongly > recommended, given the sheer amount of information here. From there, look > to identify what paths are currently, actively issuing TLS certificates > that are trusted in the Web PKI - I suspect this will be a much smaller > subset, based on Symantec’s past public disclosures. Some of these paths > may transition multiple roots; such as having an older root sign a newer > root sign the active intermediate. For each of the ‘newest’ roots, > establish a new DigiCert-managed private key and issuance infrastructure on > DigiCert’s existing infrastructure, name it in a way that ideally clearly > identifies it, and cross-sign it with that ‘newer’ Symantec root. This > serves as the Managed CA “root”, and in a way that provides the greatest > similarity to Symantec’s existing issuance practices. > > > > Once you have that ‘emergency Managed CA’ infrastructure in place, work to > migrate all existing Symantec customers to DigiCert issuance as quickly as > possible. I know you suggested your goal was 2018, but one way you may be > able to do this quicker is, for every existing Symantec customer, as a new > certificate request comes in, issue them a DigiCert-chaining certificate. > If they identify a compatibility issue - such as a client that only > supports certain Symantec certificates - issue them a certificate off the > Managed CA infrastructure - much like CAs handled the SHA-1/SHA-256 > transition in the lead-up to the SHA-1 sunset. This reduces the use of the > Managed CA infrastructure to just those who absolutely need it, and in a > way that helps DigiCert identify the concrete challenges customers may > face, which can facilitate community discussion on how to solve these > challenges. I would further encourage the limiting the validity period of > these Managed CA certificates to something 13 months or less, to help > ensure that this bridge doesn’t become a crutch, and seems to align with > your stated goal of helping these customers move to an existing DigiCert > root by 2018. > > > > On the Root Program side, this helps greatly simplify the transition > overhead and compatibility risk. If a Root Program takes no action, new > (DigiCert-issued) certificates from the Managed CA infrastructure continue > to work as-is, because they chain to the old Symantec roots. For Root > Programs that are taking steps to reduce trust in Symantec certificates, > they can whitelist those intermediates you identify from such reductions of > trust - by adding the new Managed CA certificates as ‘new’ roots or by > action in code. When steps are taken to fully remove the Symantec > infrastructure (in 2018, as proposed), Root Programs can fully remove the > legacy Symantec roots, and all of the certificates it has issued previously > - while the new certificates issued under the Managed CAs will continue to > work, as the Managed CAs were added as roots (or whitelisted from distrust). > > > > This plan helps transition from “legacy Symantec” to “new DigiCert”, which > reduces the immediate risks to the ecosystem. Unfortunately, this has the > side-effect that it means that DigiCert will now be operating a significant > number of roots - the existing DigiCert roots, the ones it acquired from > Verizon, the new “Symantec” roots (for those that did the > swap-and-replace), and/or the old Symantec roots (for those that did not). > This is not a problem unique to DigiCert - it arises in any form of CA > acquisition or merger, such as Entrust/TrendMicro or WoSign/StartCom - but > is truly special in the sheer number of combined-entity roots that will > exist. Thus, it’s also incredibly useful if, in parallel, DigiCert can look > for ways to harmonize and streamline that PKI towards that longer-goal of > having just four to six roots - one for each cryptographic algorithm. > > > > In thinking about what actions may present compatibility issues - either > to existing certificates or to new ones - I would suggest that signing any > existing DigiCert root or subordinate CAs, or anything other than these few > identified Managed CAs, with the legacy Symantec certificates, would likely > be a recipe for interoperability issues for TLS clients. I’m not saying > it’s expressly forbidden - just that, if done with existing roots, it may > cause those current roots and everything they’ve signed to be rejected. > This would happen for applications which don’t support the notion of trust > anchors (e.g. when the intermediate in the chain presented is the same as a > trusted root, which includes legacy OpenSSL-based products in addition to a > number of macOS and iOS versions) to those that impose policies based on > the root (whether EV recognition or, as a result of past misissuances, > policies that limit trust in Symantec-issued certificates). Avoiding that > for, at a minimum, TLS, is the best success strategy I can recommend, > although I would honestly recommend avoiding that across the board, given > the client issues I’ve seen over the years. > > > > If there are challenges with this outline, it might be useful to diagram a > bit of the existing Symantec and DigiCert infrastructure - both by logical > CA (same name and key) and the actual certificates - so that we can easily > visualize and discuss what paths may be impacted by DigiCert’s proposed > plan or those plans of Root Programs. This would allow easier collaboration > on how best to achieve that integration with minimal risk to the ecosystem > or to DigiCert or Symantec customers, which is a goal I’m certain we all > share. > > > > > _______________________________________________ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > > _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
