Jeremy filed the following incident report at https://bugzilla.mozilla.org/show_bug.cgi?id=1447192 :
1. How your CA first became aware of the problem (e.g. via a problem report submitted to your Problem Reporting Mechanism, via a discussion in mozilla.dev.security.policy, or via a Bugzilla bug), and the date. We received an email from Alex Cohen on March 9, 2018. It was posted to the Mozilla list the next day 2. A timeline of the actions your CA took in response. a) 3/9/3018- Received an email from Alex cohen about the impacted certificates b) 3/10/2018 - Revoked the certificates c) 3/12/2018- Scanned database for any additional certificates. All identified certificates were revoked d) 3/12/2018 - Alex posted information to Mozilla list, and Jeremy responded on what happened. e) 3/14/2018 - Added error handling to detect when a tor descriptor is missing Still to do: Add error handling to check that the cert has sufficient tor descriptors - 1 per onion name. 3. Confirmation that your CA has stopped issuing TLS/SSL certificates with the problem. DigiCert has stopped issuing onion certs that lack a descriptor 4. A summary of the problematic certificates. For each problem: number of certs, and the date the first and last certs with that problem were issued. 20 certificates, all logged to CT, ranging from Oct 2017 to Mar 2018 5. The complete certificate data for the problematic certificates. The recommended way to provide this is to ensure each certificate is logged to CT and then list the fingerprints or crt.sh IDs, either in the report or as an attached spreadsheet, with one list per distinct problem.https://crt.sh/?q=240277340 (revoked 26 October 2017)https://crt.sh/?q=261570255https://crt.sh/?q=261570338https://crt.sh/?q=261570380https://crt.sh/?q=261570384https://crt.sh/?q=261579788https://crt.sh/?q=261601212https://crt.sh/?q=261601280https://crt.sh/?q=261601281https://crt.sh/?q=261601284https://crt.sh/?q=261988060https://crt.sh/?q=326491168https://crt.sh/?q=326830043https://crt.sh/?q=328308725https://crt.sh/?q=328961187https://crt.sh/?q=329559222https://crt.sh/?q=330180704https://crt.sh/?q=351449233https://crt.sh/?id=351449246 6. Explanation about how and why the mistakes were made or bugs introduced, and how they avoided detection until now. > Looking into this, we did not correctly implement the ballot: > 1. We didn't add a check to our backend system too verify the cert > included a descriptor prior to issuance. > 2. On the front end, we missed requiring a Tor descriptor prior to > processing the order. > 3. The validation team received insufficient training on the Tor > descriptor requirement. In reality, the issue was too much reliance on the human component of asserting the Tor descriptors instead of having a technical control in place. We have a central system that manages compliance. The checks for onion certs were never added to this system. They exist now but only to ensure a tor descriptor exists. We are still working on adding checks to ensure at least one descriptor exists for each onion name. 7. List of steps your CA is taking to resolve the situation and ensure such issuance will not be repeated in the future, accompanied with a timeline of when your CA expects to accomplish these things. We revoked the certificates and added preliminary checking for Tor descriptors. We are adding additional checks to ensure certs cannot issue without them. On Mon, Mar 19, 2018 at 5:38 PM, Jeremy Rowley via dev-security-policy < dev-security-policy@lists.mozilla.org> wrote: > Interesting - this escaped our notice because it has a Tor descriptor. > Unfortunately, it looks like the fetch function with v3 is not supported so > we'll have to change how we pull and include the descriptor. Since the key > is > already in the cert, I agree there is nothing gain by including it, but I > doubt there's strong incentives to change the guidelines right now. We'll > modify to include it. > > -----Original Message----- > From: Alex Cohn <a...@alexcohn.com> > Sent: Monday, March 12, 2018 6:55 PM > To: Jeremy Rowley <jeremy.row...@digicert.com> > Cc: mozilla-dev-security-pol...@lists.mozilla.org > Subject: Re: DigiCert .onion certificates without Tor Service Descriptor > Hash > extension > > Thanks, Jeremy. > > I also found a certificate [1] with both 16-character.onion and > 56-character.onion addresses [2] listed in the SAN. The v3 address is not > included in the 2.23.140.1.31 extension, which seems to violate the same > rule > as below. However, v3 addresses include the service's entire public key in > the > address itself, so there's nothing to be gained by embedding a hash of that > key in a certificate. > > I'm not sure what, if anything, needs to happen in this case. Thoughts? > > Alex > > [1] https://crt.sh/?id=351449246 > [2] https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt > > On Mon, Mar 12, 2018 at 7:28 PM, Jeremy Rowley via dev-security-policy > <dev-security-policy@lists.mozilla.org> wrote: > > Thanks Alex. Sorry for the delayed response. I've been traveling today. > > We're reaching out to each of the customers and getting their cert > replaced. > > > > > > Looking into this, we did not correctly implement the ballot: > > 1. We didn't add a check to our backend system too verify the cert > > included a descriptor prior to issuance. > > 2. On the front end, we missed requiring a Tor descriptor prior to > > processing the order. > > 3. The validation team received insufficient training on the Tor > > descriptor requirement. > > > > In reality, the issue was too much reliance on the human component of > > asserting the Tor descriptors instead of having a technical control in > > place. We're working on putting those technical controls in place. > > > > Jeremy > > > > -----Original Message----- > > From: dev-security-policy > > <dev-security-policy-bounces+jeremy.rowley=digicert.com@lists.mozilla. > > org> On Behalf Of Alex Cohn via dev-security-policy > > Sent: Sunday, March 11, 2018 9:37 PM > > To: dev-security-policy@lists.mozilla.org > > Subject: DigiCert .onion certificates without Tor Service Descriptor > > Hash extension > > > > In the EV Guidelines [1], Appendix F states "The CA MUST include the > > CAB Forum Tor Service Descriptor Hash extension in the TBSCertificate > > convey hashes of keys related to .onion addresses." This language was > > added in Ballot 201 [2], which had an effective date of 8 July 2017. > > > > The following certificates (and precertificates if the corresponding > > certificate is not in a public CT log) were issued by DigiCert after 8 > > July for .onion domains, but lack the necessary extension: > > https://crt.sh/?q=240277340 (revoked 26 October 2017) > > https://crt.sh/?q=261570255 > > https://crt.sh/?q=261570338 > > https://crt.sh/?q=261570380 > > https://crt.sh/?q=261570384 > > https://crt.sh/?q=261579788 > > https://crt.sh/?q=261601212 > > https://crt.sh/?q=261601280 > > https://crt.sh/?q=261601281 > > https://crt.sh/?q=261601284 > > https://crt.sh/?q=261988060 > > https://crt.sh/?q=326491168 > > https://crt.sh/?q=326830043 > > https://crt.sh/?q=328308725 > > https://crt.sh/?q=328961187 > > https://crt.sh/?q=329559222 > > https://crt.sh/?q=330180704 > > https://crt.sh/?q=351449233 (revoked 10 March 2018 after initial email > > to > > DigiCert) > > > > This was previously discussed on m.d.s.p about a year ago [3]. > > > > [1] > > https://cabforum.org/wp-content/uploads/CA-Browser- > Forum-EV-Guidelines-v1.6. > > 8.pdf > > [2] https://cabforum.org/2017/06/08/2427/ > > [3] > > https://groups.google.com/d/msg/mozilla.dev.security.policy/6pBLHJBFNt > > s/ZtNI > > D_xfAgAJ > > _______________________________________________ > > dev-security-policy mailing list > > dev-security-policy@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-security-policy > > > > _______________________________________________ > > dev-security-policy mailing list > > dev-security-policy@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-security-policy > > > > _______________________________________________ > dev-security-policy mailing list > dev-security-policy@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-security-policy > > _______________________________________________ dev-security-policy mailing list dev-security-policy@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-security-policy
