On Mon, Apr 10, 2017 at 10:57 AM, Steve Medin via dev-security-policy <
dev-security-policy@lists.mozilla.org> wrote:
> Issue T: RA Program Misissuances (January 2010 - January 2017)
>
> Program Background:
>
> Symantec has operated an RA program designed to deliver a superior
> customer experience in global markets where language skills, understanding
> of local business requirements, and physical local presence are necessary.
> RA partners have supported various certificate types, including those for
> publicly trusted SSL/TLS.
>
> The RA program for publicly trusted SSL/TLS authorized appropriately
> trained personnel at select RA partners to complete all steps for
> authentication, review, and certificate issuance.
>
> In 2011, prior to ratification of the CA/Browser Forum Baseline
> Requirements, Symantec scaled back the scope of the RA program for publicly
> trusted SSL/TLS to support only those partners whose scale of business and
> investment in the future success of that business warranted the additional
> cost associated with supporting the then-new BRs. Since 2013, there have
> been only 4 RA partners still capable of processing and issuing publicly
> trusted SSL/TLS certificates: CrossCert, Certisign, Certsuperior, and
> Certisur.
>
> Symantec has had multiple controls in place to ensure these RAs'
> compliance with the BRs:
>
> Documentation:
> 1. Symantec operates an internal Knowledge Base ("KB") for its
> authentication staff and RA partners that contains detailed step-by-step
> procedures for performing each of the tasks required to validate the
> identity asserted in a certificate request.
> 2. The KB reinforces acceptable and unacceptable sources of validation
> information and processes using a subset of the information in the BRs.
> 3. The KB explains request flagging, flag reasons, and flag clearing
> procedures.
>
> Training & Exams:
> 1. Topics include BR changes, CPS changes, process changes resulting from
> industry incidents regardless of the CA involved, and a review of
> Symantec's procedures that extend the Baseline Requirements.
> 2. Exams are modified and retaken annually as criteria to renew individual
> access certificates or after significant internal or external process
> changes.
>
> Technology During Authentication:
> 1. Each request is screened for trade compliance, high-risk names,
> potential phishing (strings used in scam domains, high-profile brands), and
> other potentially risky content such as "test". Potential failures are
> flagged, preventing RA issuance, until and unless further review and
> analysis is completed.
> 2. Risk flags require manual override by authentication personnel -
> internal or RA personal as appropriate - for certificate issuance to
> proceed. Flag clearing privileges are only granted to personnel who are
> have completed the requisite training and passed appropriate exams.
>
> Technology Pre- and Post- Issuance:
> 1. Each request is screened to ensure elements outside of the subject
> information are BR compliant (e.g. SAN fields are complete, proper validity
> limits are in place, 2048 bit or higher key lengths are used, etc.). This
> scan is done after Authentication personnel approve the request and before
> it is issued. These checks cannot be overridden.
> 2. Daily, we rescan all certificates issued on the prior day using these
> same checks.
>
> Audit:
> 1. We requested independent WebTrust audit reports from RAs and assessed
> them for material findings pursuant to BR 8.4 regarding WebTrust audited
> Delegated Third Parties. See issue V addressing audits.
>
> Customer-Facing Controls:
> 1. Symantec supports Certification Authority Authorization, putting
> control of authorized CAs in the hands of customers.
> 2. Symantec logs publicly trusted certificates to Certificate Transparency
> Logs and offers a CT monitor to provide visibility for all customers to
> enable detection of suspect certificates.
>
> CrossCert Test Certificate Issue:
>
> On January 19, 2017, Andrew Ayer, an independent researcher posted the
> results of an analysis of public Certificate Transparency logs through
> which he identified roughly 270 instances of suspicious certificates issued
> by multiple CAs, including Symantec, containing the words "test" or
> "example" in the subject information.
>
> Symantec determined that 127 of these certificates were issued from
> Symantec operated CAs and that all 127 had been issued by the RA CrossCert.
> All but 31 had already expired or been revoked.
>
> Immediate Response
> Andrew Ayer's report was a certificate problem report under BR 4.9.5 which
> required us to begin an investigation within 24 hours, which we did. We
> determined that 127 certificates were in scope of the problem report.
>
> 1. On January 19, 2017, after becoming aware of this issue, Symantec
> disabled issuance privileges for all CrossCert staff.
> 2. On January 20, 2017, Symantec revoked the 31 still valid and active
> certificates. These certificates had been issued between December 28, 2016
> and January 18, 2017.
> 3. Symantec promptly took over validation and issuance for all pending and
> new orders submitted through CrossCert. Since then, Symantec's
> authentication team has continued to fully process these orders
> independently, without relying on any previous authentication work
> completed by CrossCert.
> 4. Symantec disabled customer issuance from enterprise accounts originally
> authenticated by CrossCert. New issuance from these accounts was blocked
> until the accounts were re-validated by Symantec personnel.
>
> Root Cause
> Following discussions with CrossCert and review of their audit logs,
> Symantec confirmed that all 127 certificates were issued as part of either
> internal or customer-specific testing. In mid-2016, CrossCert created a
> customer-focused feature with good intent to help customers with testing,
> but they did so without following the proper guidelines.
>
> CrossCert personnel had completed the Symantec-required training mandating
> that only fully validated certificates be issued. At no point was CrossCert
> authorized to perform less vetting on certificates for their own internal
> use or for customer testing. The personnel at CrossCert had also completed
> exams designed to verify comprehension of this training. Nonetheless,
> CrossCert management authorized overrides for certificates being used for
> internal and customer-specific testing, and CrossCert authentication
> personnel complied with that authorization.
>
> Audit logs confirm that our controls to flag these certificates for
> compliance failures worked appropriately but CrossCert management overrode
> those flags. CrossCert did not consult with Symantec on the significance of
> the compliance failure flags or their decisions to override the flags for
> any of the certificates. Our flagging process is extensive and a normal
> part of every business day. For example, our process catches popular
> domains/trademarks and variants using fuzzy logic that might otherwise be
> issued by a CA that doesn't test or block trademark abuse. Our flag/hold
> rate is conservative - as a result, the false positive rate is high and,
> following manual analysis, most flags are cleared. We appreciate the
> suggestion that additional flag monitoring and trend analysis would provide
> greater insight and increase the opportunity to catch issues like this more
> quickly in the future.
>
> Review of the 127 certificates showed that all issuances occurred in two
> time frames: Before June, 2012 or after June, 2016. Symantec received
> unqualified WebTrust for CAs and WebTrust for CAs Baseline and Network
> Security audits from Ernst & Young KR for the period July 1, 2015-June 30,
> 2016. Each of the 127 certificates issued for testing purposes were issued
> either years before these audits or after these most recent audits and so
> not detectable as part of these independent audits.
>
> Subsequent Investigation
> As part of investigating this incident, Symantec began a direct review of
> the authentication practices at CrossCert and their engagement with Ernst &
> Young KR. Through that investigation, Symantec identified two additional
> issues:
>
> 1. There were deficiencies in CrossCert's documentation of the validation
> step in the authentication process, failing to meet both Symantec and
> Baseline requirements.
> 2. The CrossCert 2015-2016 audit did not include in its scope all of the
> CAs from which CrossCert was authorized to issue certificates. E&Y KR
> stated that CrossCert provided a list of all their issuing CAs but reduced
> the list of issuing CAs in scope for sampling for budgetary reasons. E&Y KR
> did not call out this known limit in its report.
>
> These additional findings caused Symantec to seriously question not only
> CrossCert, but also the adequacy of E&Y KR's WebTrust for CAs Baseline and
> Network Security audits. This episode also reinforced that we need to more
> closely monitor and review the scope, methodology, and practices of
> external auditors.
>
> Remediation
> 1. Symantec fully terminated CrossCert as an RA partner. They no longer
> participate in the authentication process or issuance of publicly trusted
> SSL/TLS certificates.
> 2. Symantec has hired a team of Korean speaking authentication personnel
> who have completed the requisite training and now independently process all
> new orders from this market.
> 3. Symantec is in the process of fully and independently revalidating
> active SSL/TLS certificates previously approved by CrossCert, and if
> necessary, revoking and replacing certificates if we detect any errors. As
> we posted in response to the assertion about reusing RA validation in
> m.d.s.policy, we reiterate that we are not doing this. For CrossCert, we
> are validating each existing certificate as if it was a new request.
> 4. Symantec announced the decision to wind down the RA program for
> publicly trusted SSL/TLS.
> 5. Symantec is in the process of reviewing 100% of the authentication
> records for the active SSL/TLS certificates approved and issued by
> Certisign, Certsuperior, and Certisur. If we become aware of a certificate
> that is misleading, we follow the Baseline Requirements regarding
> revocation.
>
> It has been suggested that the issues identified above and those
> associated with Issue V warrant immediate revocation of all certificates
> issued by each of these RAs. Symantec disagrees with that assessment.
> Revocation by a CA of certificates, outside of a customer request, can be
> materially disruptive to the individuals and the organizations that rely on
> them, and so should be reserved for cases where there are clear security
> risks.
>
> In the case of CrossCert, Symantec immediately revoked the clearly
> problematic certificates mis-issued for testing. With respect to individual
> certificates, the key additional issue identified with CrossCert related to
> proper record-keeping. Accordingly, using a risk-informed approach,
> Symantec is fully re-validating these certificates.
>
> In the cases of Certisign, Certsuperior, and Certisur, we relied on at
> least one unqualified audit for their operations. With respect to their
> individual certificates, there was no evidence of any problems with the
> certificates they approved and issued. Accordingly, and again using a
> risk-informed approach, Symantec is reviewing all of these certificates,
> which amounts to conducting an audit with 100% sampling.
>
> Symantec will provide updates to the community regarding our revalidation
> of active CrossCert-issued SSL/TLS certificates and our review of the other
> RA partners.
>
>
Hi Steve,
A few followup questions.
1) On the basis of the controls Symantec described, at no point was any
mention made of Symantec performing sampling audits to ensure their RA
partners complied with either the RA partner's CP/CPS or Symantec's CP/CPS.
a) Is it fair to conclude that no such examination if done?
b) If no such examination is done, why not?
c) Does Symantec plan to perform any sampling audits its other RA
partners?
2) Symantec states "We requested independent WebTrust audit reports from
RAs and assessed them for material findings pursuant to BR 8.4 regarding
WebTrust audited Delegated Third Parties."
a) For what years did you request an audits? That is, has Symantec
requested such audits since 2011? If not, when was the first year Symantec
requested such audits?
b) Please describe the process and procedures Symantec employees perform
in order to determine the appropriate status of the auditor.
c) Please describe the process and procedures Symantec employees perform
in order to determine that the RA partner's CP/CPS conforms with Symantec's
CP/CPS
d) Please describe what procedures were in place when an RA partner
changed its CP/CPS
e) Please describe what procedures were in place when Symantec changed
its CP/CPS.
3) Symantec states "Symantec supports Certification Authority
Authorization, putting control of authorized CAs in the hands of customers."
a) Why did you mention this, given that Symantec has acknowledged that
its RA partners performed domain validation independent of Symantec.
b) Do you agree that this statement is materially misleading in the
context in which it is provided, namely the discussion of RA security
controls, and may otherwise be inappropriately interpreted as Symantec
suggesting its RA partners were performing an activity that they may not
have been done?
4) Symantec states "Topics include BR changes, CPS changes, process changes
resulting from industry incidents regardless of the CA involved, and a
review of Symantec's procedures that extend the Baseline Requirements."
a) Did Symantec update its RA partners regarding the set of Symantec
issues related to the Test Certificate Misissuance (Issue D)?
b) Was this the same process used to ensure employee compliance, which
was identified deficient with respect to Issue D?
c) What steps did Symantec take, in response to Issue D, to inform its RA
partners about these issues?
d) What steps did Symantec take, in response to Issue D, to review its RA
partners activities, similar to that of its employees activities?
e) What steps did Symantec take, in response to Issue D, to review its
APIs granted to RA partners, similar to the test tool used to cause
misissuance?
f) What other evidence or facts are you able to share that could
establish Symantec as a competent, reliable, or trustworthy CA partner,
given that the current evidence shared meaningfully suggests it is not any
of these.
5) Symantec states "Symantec is in the process of fully and independently
revalidating active SSL/TLS certificates previously approved by CrossCert,
and if necessary, revoking and replacing certificates if we detect any
errors."
a) Does Symantec believe this process is compliant with the Baseline
Requirements and Symantec's CP/CPS?
b) If Symantec believes this process is consistent with the Baseline
Requirements, please specifically reference the sections that permit this.
c) If not, is Symantec requesting an exception to their obligations under
the Baseline Requirements to promptly revoke these certificates within 24
hours?
d) Has Symantec clearly stated that request as such - a request to
violate the Baseline Requirements - rather than as a statement of intent?
6) Symantec states "Symantec is in the process of reviewing 100% of the
authentication records for the active SSL/TLS certificates approved and
issued by Certisign, Certsuperior, and Certisur."
a) Does Symantec believe this process is compliant with the Baseline
Requirements and Symantec's CP/CPS?
b) If Symantec believes this process is consistent with the Baseline
Requirements, please specifically reference the sections that permit this.
c) If not, is Symantec requesting an exception to their obligations under
the Baseline Requirements to promptly revoke these certificates within 24
hours?
d) Has Symantec clearly stated that request as such - a request to
violate the Baseline Requirements - rather than as a statement of intent?
7) Symantec states "It has been suggested that the issues identified above
and those associated with Issue V warrant immediate revocation of all
certificates issued by each of these RAs. Symantec disagrees with that
assessment. "
a) Please provide the reference to the Baseline Requirements to support
this conclusion.
8) Symantec states that it believes revocation "should be reserved for
cases where there are clear security risks."
a) Please explain how this belief is consistent with the Baseline
Requirements, Section 9.6.3, Item 8.
b) Please explain how this belief is consistent with the Baseline
Requirements, Section 4.9.1.1, Item 5.
c) Please explain how this belief is consistent with the Baseline
Requirements, Section 4.9.1.1, Item 9.
d) Is it fair to conclude that Symantec's belief is that it does not have
to follow the Baseline Requirements that it disagrees with?
e) Please provide any supporting evidence related to this belief that
would be useful for Relying Parties to assess whether Symantec is competent
and trustworthy, as the current evidence suggests it is not.
_______________________________________________
dev-security-policy mailing list
dev-security-policy@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-security-policy
