James Kasten, We apologize for the impact of the security incident on GTS. I want to say that we will not do bad things, but we lack the means of notarization.
But there is a suggestion: GTS could evaluate to disable pre-authorizations for domain names or IP addresses for the next 90 days, to reduce potential impacts of the incident. 在2023年6月13日星期二 UTC+8 13:00:45<James Kasten> 写道： > The final reference should be "3. > https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-v2.0.0.pdf"; > instead of the 1.8.7 link provided. (It doesn't change the discussion.) My > apologies. > > James Kasten > Google Trust Services > > On Mon, Jun 12, 2023 at 8:57 PM James Kasten <jdka...@google.com> wrote: > >> To be explicit, GTS does not have a business relationship with HiCA. >> >> Normally ACME services have protections against these types of MitM-CAs, >> but the remote RCE allowed HiCA to bypass these controls [1, 2]. >> >> For instance, it is possible HiCA replaced the local client's key >> authorization during challenge validation with a key authorization provided >> by HiCA, granting authorization of the domain names to HiCA. HiCA could >> theoretically use these authorizations to continue to issue certificates >> for the affected domain names, or revoke the certificates that were issued. >> >> So clients of HiCA should also consider the lasting effects on the the >> domains in addition to your normal recovery procedures for an RCE. It may >> be prudent to reissue and revoke any certificates with your choice of CA >> directly and to watch certificate transparency logs for any future >> unintended issuance. GTS allows authorization reuse up to 28 days on our >> ACME endpoint and issues certificates with lifetimes up to 90 days. Other >> CAs may differ. By the current baseline requirements CAs can issue 398 day >> certificates and reuse the authorizations for 398 days [3]. >> >> >> James Kasten >> Google Trust Services >> >> >> 1. https://datatracker.ietf.org/doc/html/rfc8555#section-6.4 >> 2. https://datatracker.ietf.org/doc/html/rfc8555#section-10.2 >> 3. https://cabforum.org/wp-content/uploads/CA-Browser-Forum-BR-1.8.7.pdf >> >> >> On Sunday, June 11, 2023 at 6:34:44 AM UTC-7 Xiaohui Lam wrote: >> >>> No, actually we implementing GTS, SSL.com and sometimes Let's Encrypt, >>> Sectigo all the time. and especially SSL.com issued under SSL.com subCA, >>> not under "Quantum Secure Site DV" acutally. >>> The CA >>> >>> 在2023年6月11日星期日 UTC+8 10:02:14<Amir Omidi (aaomidi)> 写道： >>> >>>> Emailing on my personal capacity: >>>> >>>> Xiaohui, can you please confirm that ssl.com was the only actual CA >>>> that was used for issuance through HiCA? >>>> >>>> On Saturday, June 10, 2023 at 2:08:47 PM UTC-4 Kurt Seifried wrote: >>>> >>>>> Forwarding this to the list, I'm not comfortable with off list >>>>> discussions in private. >>>>> >>>>> On Sat, Jun 10, 2023 at 11:18 AM Xiaohui Lam <inao...@gmail.com> >>>>> wrote: >>>>> >>>>>> Mr Seifried, >>>>> >>>>> >>>>>> > Is this really a situation where something extremely suspicious >>>>>> (remote code execution, CA's with multiple entities, some of which don't >>>>>> seem to properly exist, etc.) is going to be swept under the rug with a >>>>>> simple "yeah, we revoked this bad actors certificates, everything is >>>>>> fine"? >>>>>> >>>>>> We are a reseller, not a physical root CA. This is a widely accepted >>>>>> solution for cross border businesses. We have business accepts online >>>>>> payment, the China users needs pay via alipay or wechat, to sign up the >>>>>> merchant we must have a china company, >>>>>> and foreign needs stripe, merchant must be a non-MainlandChina >>>>>> company. this is not suspicious. >>>>>> >>>>>> *. I represent the above opinion of my company >>>>>> >>>>> >>>>>> > If HiCA can do this, how do we know there are not more >>>>>> intermediate/reseller CAs doing this? >>>>>> >>>>>> Most CA has no necessary to exploiting this RCE, because they can >>>>>> natively >>>>>> compatible with RFC 8555, they can define own CPS and CP, which >>>>>> contains validation policy, we does because we are not CA and can't to >>>>>> provider RFC 8555 ACME endpoint like a CA does. so a physical root CA >>>>>> has >>>>>> no necessary to provide ACME simulation by RCE. and also there're more >>>>>> difficulties for a ssl reseller to provide ACME service which real CAs >>>>>> won't undergo. >>>>>> >>>>>> - CSR stage difference: Most CA's subscriber request process or >>>>>> reseller API process, requires CSR be submitted in the `new-order` API, >>>>>> ACME requires CSR be submitted in `finalize` API. I have a topic in >>>>>> letsencrypt community years ago about this - >>>>>> https://community.letsencrypt.org/t/why-acme-requires-domain-auth-first-before-csr/98482 >>>>>> - Challenge difference: Most CA's subscriber request process or >>>>>> reseller API process's DNS validation requires `_<md5>` / `_dnsauth` >>>>>> dnshost, and dnstype possibly CNAME or possibly TXT, But ACME's DNS >>>>>> validation dnshost is constant: `_acme-challenge`, dnstype `TXT`. And >>>>>> in a >>>>>> more deep talk ACME's dnsvalue needs publickey's thumbprint + server >>>>>> token >>>>>> which is totally different than traditional way's dnsvalue. >>>>>> >>>>>> >>>>>> My opinion is community can research how many ACME was publicly >>>>>> provided, and investigate is the provider a physical CA. if is natively >>>>>> compatible with RFC 8555, no worry about that one and continue do >>>>>> investigate >>>>>> next. >>>>>> >>>>>> *. I represent the above opinion of my personal. not my company. >>>>>> >>>>>> >>>>>> Sincere, >>>>>> Bruce. >>>>>> >>>>> >>>>>> >>>>>> >>>>>> >>>>>> 在2023年6月11日星期日 UTC+8 00:39:16<Kurt Seifried> 写道： >>>>>> >>>>>> Is this really a situation where something extremely suspicious >>>>>> (remote code execution, CA's with multiple entities, some of which don't >>>>>> seem to properly exist, etc.) is going to be swept under the rug with a >>>>>> simple "yeah, we revoked this bad actors certificates, everything is >>>>>> fine"? >>>>>> >>>>>> If HiCA can do this, how do we know there are not more >>>>>> intermediate/reseller CAs doing this? >>>>>> >>>>>> https://news.ycombinator.com/item?id=36252310. >>>>>> >>>>>> Just a note, apparently, websites have been shut down and stuff >>>>>> deleted with respect to HiCA. >>>>>> >>>>>> Posting some of the threads here in case they get removed or whatever: >>>>>> >>>>>> ================== >>>>>> egecks 1 day ago | prev | next [–] >>>>>> >>>>>> I think the title buries the most horrifying part of this. The HiCA >>>>>> certificate authority is relying on an RCE to do an end-run around the >>>>>> semantics of the ACME HTTP-01 validation method. >>>>>> Fucked up and they should be booted from every root program for this. >>>>>> >>>>>> ================= >>>>>> >>>>>> 0x0 1 day ago | prev | next [–] >>>>>> >>>>>> Looks like they are issuing under a sub-CA of "ssl.com" according to >>>>>> https://github.com/acmesh-official/acme.sh/issues/4659#issue... >>>>>> Interestingly, the mozilla dev-security-policy group seems to contain >>>>>> a recent discussion about including "ssl.com" in the root store here >>>>>> https://groups.google.com/a/mozilla.org/g/dev-security-polic... >>>>>> >>>>>> Curious to know if this could, maybe it should, have ripple effects >>>>>> to the various SSL Root CA programs. Having someone run a subCA that >>>>>> actually exploits an RCE against ACME clients doesn't seem very >>>>>> trustworthy, and any CA enabling this behaviour should probably be >>>>>> kicked >>>>>> out of the trust stores? >>>>>> >>>>>> reply >>>>>> >>>>>> >>>>>> agwa 1 day ago | parent | next [–] >>>>>> >>>>>> The sub CA is operated by ssl.com, not HiCA (which is not a trusted >>>>>> certificate authority). HiCA is relaying the certificate requests to >>>>>> ssl.com, which is properly validating the requests in accordance >>>>>> with all the requirements. ssl.com isn't doing anything wrong. >>>>>> That's why HiCA needs to exploit an RCE in acme.sh - ACME doesn't >>>>>> support >>>>>> relaying certificate requests to other CAs like this. >>>>>> reply >>>>>> >>>>>> >>>>>> 0x0 23 hours ago | root | parent | next [–] >>>>>> >>>>>> Someone posted a comment on github claiming they are the founder of >>>>>> Quantum (the sub CA of ssl.com - see https://crt.sh/?caid=200960 ) >>>>>> and that they are the provider of the HiCA service. So it does sound >>>>>> like >>>>>> there is a closer link here than your comment would indicate: >>>>>> https://github.com/acmesh-official/acme.sh/issues/4659#issue... >>>>>> >>>>>> reply >>>>>> >>>>>> >>>>>> agwa 23 hours ago | root | parent | next [–] >>>>>> >>>>>> Quantum is not a trusted CA. ssl.com has a white-labeled >>>>>> intermediate CA with the name "Quantum" in it, but this intermediate CA >>>>>> is >>>>>> operated by ssl.com under all the same controls as ssl.com's other >>>>>> intermediate CAs. Quantum has no ability to issue trusted certificates >>>>>> themselves. >>>>>> reply >>>>>> >>>>>> >>>>>> 0x0 23 hours ago | root | parent | next [–] >>>>>> >>>>>> So the person claiming to be the founder of "QuantumCA" does not >>>>>> possess the private key corresponding to https://crt.sh/?caid=200960 >>>>>> - can we be sure the private key is only accessible by ssl.com's CA >>>>>> system? So the certificates listed here aren't issued by this person, >>>>>> but >>>>>> by the ssl.com's system? >>>>>> https://crt.sh/?Identity=%25&iCAID=200960&exclude=expired&de... >>>>>> Also, why would ssl.com even create a subCA named "QuantumCA"? Are >>>>>> they in business with this person claiming to be the founder of >>>>>> "QuantumCA" >>>>>> who appears to be responsible for exploiting this acme.sh 0day? What >>>>>> does >>>>>> this say about ssl.com's trustworthiness? Or is the person in the >>>>>> github comments lying? >>>>>> >>>>>> reply >>>>>> >>>>>> >>>>>> agwa 23 hours ago | root | parent | next [–] >>>>>> >>>>>> > So the person claiming to be the founder of "QuantumCA" does not >>>>>> possess the private key corresponding to https://crt.sh/?caid=200960 >>>>>> - can we be sure the private key is only accessible by ssl.com's CA >>>>>> system? So the certificates listed here aren't issued by this person, >>>>>> but >>>>>> by the ssl.com's system? >>>>>> https://crt.sh/?Identity=%25&iCAID=200960&exclude=expired&de... >>>>>> Correct. You can see the Quantum intermediates listed in ssl.com's >>>>>> most recent audit statement, meaning an auditor has verified that >>>>>> ssl.com has controls to protect the private key: >>>>>> https://www.cpacanada.ca/generichandlers/CPACHandler.ashx?at... >>>>>> >>>>>> (The audit could be flawed, but it's the same amount of assurance we >>>>>> have for any intermediate CA's private key - the fact that "QuantumCA" >>>>>> is >>>>>> in the name does not change the risk calculus) >>>>>> >>>>>> > Also, why would ssl.com even create a subCA named "QuantumCA"? Are >>>>>> they in business with this person claiming to be the founder of >>>>>> "QuantumCA" >>>>>> who appears to be responsible for exploiting this acme.sh 0day? What >>>>>> does >>>>>> this say about ssl.com's trustworthiness? Or is the person in the >>>>>> github comments lying? >>>>>> >>>>>> There is a business relationship between QuantumCA and ssl.com. >>>>>> QuantumCA is a reseller of ssl.com, and they've paid extra to ssl.com >>>>>> so that the certificates they purchase get issued from an intermediate >>>>>> CA >>>>>> named "QuantumCA" rather than one of ssl.com's usual intermediate >>>>>> CAs which have "ssl.com" in the name. This lets QuantumCA pretend to >>>>>> be a real CA. This is a common practice in the industry, and I don't >>>>>> think >>>>>> it says anything about the trustworthiness of ssl.com, because the >>>>>> business relationship with QuantumCA doesn't in any way subvert the >>>>>> integrity of the WebPKI since ssl.com retains control of the >>>>>> issuance. Still, I wish intermediate CA white-labeling were banned >>>>>> because >>>>>> it causes terrible confusion about who is and isn't a CA. >>>>>> >>>>>> reply >>>>>> >>>>>> >>>>>> 0x0 23 hours ago | root | parent | next [–] >>>>>> >>>>>> I find it troubling that a root CA (ssl.com) is apparently OK with >>>>>> lending their name in a business relationship with an actor that is >>>>>> actively exploiting an acme.sh 0day. >>>>>> >>>>>> >>>>>> tptacek 20 hours ago | root | parent | next [–] >>>>>> >>>>>> This feels a little bit like doubling down to find ways to implicate >>>>>> the actual CA instead of the reseller. It's clear how mismanagement by a >>>>>> real CA would make a more interesting story than by this random >>>>>> no-longer-existing pseudo-reseller, but I don't think there's evidence >>>>>> to >>>>>> support that story yet. >>>>>> reply >>>>>> >>>>>> >>>>>> 0x0 20 hours ago | root | parent | next [–] >>>>>> >>>>>> But it's not a random pseudo-reseller? The one github comment from >>>>>> "the founder of Quantum CA" seems to say they are also the creator of >>>>>> HiCA, >>>>>> which is the entity that was exploiting the 0day in acme.sh. And the >>>>>> crt.sh >>>>>> link shows an intermediate CA cert named "QuantumCA", signed by >>>>>> ssl.com. >>>>>> So QuantumCA == HiCA == exploiters of the acme.sh 0day, it's all the >>>>>> same entity? The intermediate CA could just as well be named >>>>>> "0dayexploitersCA"? Why is it not a huge concern that ssl.com is >>>>>> fine with operating such a "0dayexploitersCA" intermediate? >>>>>> >>>>>> Am I missing something here? >>>>>> >>>>>> reply >>>>>> ================= >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Jun 9, 2023 at 1:32 PM Xiaohui Lam <inao...@gmail.com> wrote: >>>>>> >>>>>> Mr mochaaP, >>>>>> >>>>>> We're running businesses under multi entities, one is UK company, and >>>>>> one is CN company, the UK company is registered and running by a former >>>>>> workmate which leaved our team, and CN company is registered and running >>>>>> by >>>>>> me. >>>>>> >>>>>> We do stopped from selling SSL.com certificate due to business >>>>>> concern and the cross-sign root expiration concern, That meantime we do >>>>>> have some cooperates with other CAs without whitelabel/intermediateCA, >>>>>> some >>>>>> CAs are directly implemented and some are tier-2 implements(under other >>>>>> resellers). So, our website is kept running, including HiCA keeps. >>>>>> >>>>>> But we will stop all misleading business to stop provide our Quantum >>>>>> brand products, only contain our China company's materials. >>>>>> >>>>>> My KEY OPINION: our China entity has been kept in existence so we >>>>>> kept the reselling business. >>>>>> >>>>>> Sincere. >>>>>> Bruce Lam >>>>>> 在2023年6月10日星期六 UTC+8 03:13:11<mochaaP> 写道： >>>>>> >>>>>> Hi Xiaohui, >>>>>> >>>>>> I think you may have misunderstood my message. What I meant to convey >>>>>> was that I am skeptical of your intention to resell your own CA for a >>>>>> dissolved Ltd. that was not subject to having its certificate revoked. >>>>>> We >>>>>> believe that this practice is uncommon for a reseller in such a case. >>>>>> >>>>>> Please understand that my message was not intended to be hateful >>>>>> towards you or your team. If you believe that this was an honest >>>>>> mistake, >>>>>> please reply to this thread with more details. The community values >>>>>> transparency and trust, and we would be happy to hear your perspective. >>>>>> >>>>>> Best regards, >>>>>> Zephyr Lykos >>>>>> On Saturday, June 10, 2023 at 1:08:08 AM UTC+8 Xiaohui Lam wrote: >>>>>> >>>>>> Thanks John to share this topic to the dev-security forum. >>>>>> >>>>>> This is HiCA founder, let me to explain your concern, Mr John , >>>>>> the RCE is fully used to finish the challenge which validated by CAs, >>>>>> in another word, the ACME.sh-enrolled certificates which passing this >>>>>> RCE, >>>>>> it does compliant with each CA's BR validation requirements. CA did >>>>>> nothing >>>>>> wrong. And also by this trick can enroll any CA's certificate before >>>>>> acme.sh fix patch. >>>>>> >>>>>> and to Mr @mochaaP, you said to punish our team, we're NOT a public >>>>>> CA or private CA(in my understanding, a CA must manage a or more PKI >>>>>> infrastructure physically), [3]so the clarify relationship to HiCA w/ >>>>>> QuantumCA is no necessary, but we still told we runs HiCA inside >>>>>> QuantumCA >>>>>> project's source code, it's a sub-application inside it. >>>>>> >>>>>> I agree @Andrew's opinion, CAs shouldn't take any responsibilities to >>>>>> the RCE incidents. or there are hundreds acme-tools for CAs need to >>>>>> concern. >>>>>> 在2023年6月10日星期六 UTC+8 00:43:47<mochaaP> 写道： >>>>>> >>>>>> Hello, >>>>>> >>>>>> Although HiCA is not a CA itself, the person own HiCA seems also owns >>>>>> (or at least works for) Quantum CA[1][2]. they also confirmed that >>>>>> Quantum >>>>>> CA is operated by both their team and SSL.com team[3]. >>>>>> >>>>>> I think this probably is not as simple as a white-label intermediate >>>>>> CA being abused, rather a CA that resells their own product to >>>>>> themselves >>>>>> to prevent being punished for bad behaviors. >>>>>> >>>>>> [1]: https://github.com/xiaohuilam (see "Pinned" section) >>>>>> [2]: https://github.com/quantumca (see "People" section) >>>>>> [3]: >>>>>> https://github.com/acmesh-official/acme.sh/issues/4659#issuecomment-1584546150 >>>>>> >>>>>> (note that this person never clearified their relationship with Quantum >>>>>> CA >>>>>> and only replied with "So this isn't the evidence to proof HiCA is a CA >>>>>> which managed PKI.") >>>>>> >>>>>> Regards, >>>>>> Zephyr Lykos >>>>>> >>>>>> On Friday, June 9, 2023 at 9:04:34 PM UTC+8 Andrew Ayer wrote: >>>>>> >>>>>> On Fri, 9 Jun 2023 05:42:22 -0700 (PDT) >>>>>> "John Han (hanyuwei70)" <hanyu...@gmail.com> wrote: >>>>>> >>>>>> > Here is the story. >>>>>> > https://github.com/acmesh-official/acme.sh/issues/4659 >>>>>> > >>>>>> > Seems like they exploited acme.sh and let user to evade certificate >>>>>> > issuing procedure. >>>>>> > >>>>>> > Do we need to discuss this? >>>>>> >>>>>> The party in question (HiCA/QuantumCA) is not a certificate >>>>>> authority, >>>>>> and I don't see any evidence that the actual CAs in question evaded >>>>>> any >>>>>> validation requirements. >>>>>> >>>>>> HiCA/QuantumCA is just acting as an intermediary between subscribers >>>>>> and the issuance APIs operated by actual CAs[1]. Literally anyone can >>>>>> do this and do monumentally stupid/insecure things; it's not >>>>>> productive >>>>>> to have a discussion every time this happens. >>>>>> >>>>>> Regards, >>>>>> Andrew >>>>>> >>>>>> [1] It's true they have a reseller relationship with ssl.com, who >>>>>> are >>>>>> operating a white-label intermediate CA with "QuantumCA" in the >>>>>> subject, but HiCA/QuantnumCA are also fronting other CAs, including >>>>>> GTS, which doesn't require a reseller agreement to access their free >>>>>> ACME API, so I don't see that aspect as being productive to discuss >>>>>> either. >>>>>> >>>>>> -- >>>>>> >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "dev-secur...@mozilla.org" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to dev-security-po...@mozilla.org. >>>>>> >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/0f9174b3-02d6-4ff6-a7fa-3b931375076dn%40mozilla.org >>>>>> >>>>>> <https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/0f9174b3-02d6-4ff6-a7fa-3b931375076dn%40mozilla.org?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Kurt Seifried (He/Him) >>>>>> ku...@seifried.org >>>>>> >>>>>> >>>>> >>>>> -- >>>>> Kurt Seifried (He/Him) >>>>> ku...@seifried.org >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "dev-security-policy@mozilla.org" group. To unsubscribe from this group and stop receiving emails from it, send an email to dev-security-policy+unsubscr...@mozilla.org. To view this discussion on the web visit https://groups.google.com/a/mozilla.org/d/msgid/dev-security-policy/e308eacd-f8ea-4958-bad5-21f9bc658eafn%40mozilla.org.
