I think being able to leverage FoundationDB's serializability is an awesome idea! +1 (non-binding) on all 4 points. I also support the idea of changing the API in backwards-incompatible ways if that makes things more convenient / streamlined. I wonder, does this mean other, backwards-incompatible changes are also welcome in the next major? (Given that replicator-compatibility (from later on in this thread) is expected.)
Thank you, Donat On Thu, 16 Jul 2020 at 18:26, Paul Davis <[email protected]> wrote: > > From what I'm reading it sounds like we have general consensus on a few > things: > > 1. A single CouchDB API call should map to a single FDB transaction > 2. We absolutely do not want to return a valid JSON response to any > streaming API that hit a transaction boundary (because data > loss/corruption) > 3. We're willing to change the API requirements so that 2 is not an issue. > 4. None of this applies to continuous changes since that API call was > never a single snapshot. > > If everyone generally agrees with that summarization, my suggestion > would be that we just revisit the new pagination APIs and make them > the only behavior rather than having them be opt-in. I believe those > APIs already address all the concerns in this thread and the only > reason we kept the older versions with `restart_tx` was to maintain > API backwards compatibility at the expense of a slight change to > semantics of snapshots. However, if there's a consensus that the > semantics are more important than allowing a blanket `GET > /db/_all_docs` I think it'd make the most sense to just embrace the > pagination APIs that already exist and were written to cover these > issues. > > The only thing I'm not 100% on is how to deal with non-continuous > replications. I.e., the older single shot replication. Do we go back > with patches to older replicators to allow 4.0 compatibility? Just > declare that you have to mediate a replication on the newer of the two > CouchDB deployments? Sniff the replicator's UserAgent and behave > differently on 4.x for just that special case? > > Paul > > On Wed, Jul 15, 2020 at 7:25 PM Adam Kocoloski <[email protected]> wrote: > > > > Sorry, I also missed that you quoted this specific bit about eagerly > > requesting a new snapshot. Currently the code will just react to the > > transaction expiring, then wait till it acquires a new snapshot if > > “restart_tx” is set (which can take a couple of milliseconds on a > > FoundationDB cluster that is deployed across multiple AZs in a cloud > > Region) and then proceed. > > > > Adam > > > > > On Jul 15, 2020, at 6:54 PM, Adam Kocoloski <[email protected]> wrote: > > > > > > Right now the code has an internal “restart_tx” flag that is used to > > > automatically request a new snapshot if the original one expires and > > > continue streaming the response. It can be used for all manner of > > > multi-row responses, not just _changes. > > > > > > As this is a pretty big change to the isolation guarantees provided by > > > the database Bob volunteered to elevate the issue to the mailing list for > > > a deeper discussion. > > > > > > Cheers, Adam > > > > > >> On Jul 15, 2020, at 11:38 AM, Joan Touzet <[email protected]> wrote: > > >> > > >> I'm having trouble following the thread... > > >> > > >> On 14/07/2020 14:56, Adam Kocoloski wrote: > > >>> For cases where you’re not concerned about the snapshot isolation (e.g. > > >>> streaming an entire _changes feed), there is a small performance > > >>> benefit to requesting a new FDB transaction asynchronously before the > > >>> old one actually times out and swapping over to it. That’s a pattern > > >>> I’ve seen in other FDB layers but I’m not sure we’ve used it anywhere > > >>> in CouchDB yet. > > >> > > >> How does _changes work right now in the proposed 4.0 code? > > >> > > >> -Joan > > > > >
