Hey Daniel, Thanks for the input. I can see why you might be feeling some pain here, but I'm not sure the proposed solution is quite right.
Note that ACME already has the "new-authorization" flow, which does pretty much exactly what you're proposing. The only difference is that instead of authorizations being created all at once in response to a "new-cert" request, they're created individually by the client. But it's ultimately the same number of RTTs (authz fetch ~ new-authz) -- or rather, one fewer in the new-authz case because there's no new-cert request. To what degree would your problem be solved simply by having the affected integrators move over to that flow? I can sympathize that the new-order flow might not be suitable for all use cases. Maybe it would be useful to write down some thoughts on which use cases are best addressed by which flows. --Richard On Tue, Sep 19, 2017 at 12:26 PM, Daniel McCarney <c...@letsencrypt.org> wrote: > Hi folks, > > Just over a year ago in a thread titled "Proactive vs On-Finalization > Certificate Issuance"[0] we reached the consensus on the question of > whether to > issue a certificate "on-finalization" or "proactively" with the conclusion > to > standardize on proactive issuance. > > Implementation experience shows that proactive issuance is slow in > practice. An > order can be associated with many authorizations and an authorization can > be > associated with many orders. This interacts poorly with optimizations Let's > Encrypt has developed from in-the-field experience with ACME and it is > difficult > to rate-limit effectively. I propose that we return to the on-finalization > style > approach and that we provide the CSR during finalization.. > > ## Issues with proactive issuance > > ### Authorization Reuse > > Proactive issuance requires that when an authorization is switched to a > valid > state by the result of a challenge update that we find the associated > orders and > iterate each order's authorizations to see if all of the authorizations are > valid and issuance can occur. > > Let's Encrypt reuses existing authorizations[1][2] whenever possible, as > encouraged by ACME. This has been of huge value in reducing resource > consumption > (both in terms of database storage and challenge/API requests). Many > clients > create pending authorizations they do not finalize and similarly re-create > pending authorizations for identifiers that the account already has valid > authorizations for. Unchecked this leads to significant database growth and > resource consumption, even with aggressive rate-limits[3]. > > The combination of authorization reuse and proactive issuance means a given > authorization can be associated with many orders. This necessitates an > expensive > many-to-many query at each authorization update to see if any associated > orders are now complete. > > One possible workaround is a rate-limit that would constrain the number of > orders an authorization can be associated with. However establishing the > value > of the limit to both reduce server-side load & also effectively support > Let's > Encrypt's large-volume integrators and their issuance patterns is > challenging. > > ### CSR-first new-order flow > > To support proactive issuance the CA also needs to have the CSR for the > order > at-hand when checking each authorization in case issuance can proceed. The > `new-order` request that creates the order therefore carries the full CSR > that > was previously delivered by the "certificate finalize" message. > > We see users create many more authorizations than they are able to finalize > successfully. With the legacy "new-cert" flow our authorization reuse > dramatically helped reduce the DB space used up by authorizations that will > never lead to issuance because no CSR is stored until all identifiers are > validated. > > With the new-order flow we have to accept and store a full CSR before > handing > back authorizations to the client. Reuse of authorizations does not > prevent the > DB bloat that will occur from users submitting new order requests and not > fulfilling them. The CSR contains a full public key and is one of the > single > largest contributors to DB disk space usage. Addressing this will require > another strict rate limit on outstanding new-orders and is again difficult > to > “dial-in” in to both prevent excessive resource consumption and also > maintain > the ability for big integrators to process large-scale issuance using our > API. > > # Proposed Change > > I believe we should change the `new-order` endpoint to not accept a full > CSR but > to instead accept a list of identifiers that the ACME client wishes to > authenticate & issue for. Authorizations can be created and returned for > those > domains as happens today. > > The order resource should be changed to have a “finalizeURL” field which > provides the client a URL to POST a CSR to indicate that the CA should > issue the > certificate. Proactive issuance should be removed outright. > > This will address both the problem of scanning existing many-to-many orders > & their authorizations as well as the db bloat that will occur from > front-loading the delivery of the CSR. > > It may be tempting to leave proactive issuance as an optional feature but > this > will require maintaining the CSR in the new-order and I also believe that > having > two very distinct methods of issuance will complicate the client ecosystem > and > returns us to the question of how to determine which will occur that > spawned the > original "Proactive vs On-finalization Certificate Issuance" > thread[0]. > > [0] https://mailarchive.ietf.org/arch/msg/acme/0lVmGl8e-rmSH0x7ycDW5dj6GAY > [1] https://community.letsencrypt.org/t/upcoming-api-changes/17947 > [2] https://community.letsencrypt.org/t/automatic-recycling-of- > pending-authorizations/41321 > [3] https://letsencrypt.org/docs/rate-limits/ > > > _______________________________________________ > Acme mailing list > Acme@ietf.org > https://www.ietf.org/mailman/listinfo/acme > >
_______________________________________________ Acme mailing list Acme@ietf.org https://www.ietf.org/mailman/listinfo/acme
