nickva opened a new pull request, #5760:
URL: https://github.com/apache/couchdb/pull/5760
Previously, the internal replicator created twice the number of checkpoints
needed to replicate purges between nodes. An internal replication job first
pulls the purges from the target to the source, then pushes the purges from the
source to the target, then finally pushes the document updates to the target.
During the pull operation, for example, from node `B` (the target) to node
`A` (the source), it creates an `A->B` checkpoint on node `B` (the target).
Then, during the push from `A` to `B` it creates an `A->B` checkpoint on node A
(the source). As a result, after the job finishes there are two checkpoints: an
A->B one on A, and an `A->B` one on B. It may look something like this:
```
[node A] [node B]
<-------pull------ (A->B)
(A->B) --------push------>
```
When the internal replication job runs on node B and _pushes_ purges to node
A, it will create a `B->A` checkpoint on B. After this instant, there will be
two checkpoints on B for replicating purges from B to A: one is `A->B`, from
the first job, and another `B->A`, from the second job. Both of the checkpoints
essentially checkpoint the same thing. It may look like this after both
replication jobs finish:
```
[node A] [node B]
<-------pull------ (A->B) JOB1
(A->B) --------push------>
(B->A) --------pull------>
<-------push------ (B->A) JOB2
```
On B, the checkpoints `A->B` and `B->A` could have a different purge
sequence: one higher than the other, and so the lower one could delay the
compactor from cleaning up purge infos. This also makes it harder to reason
about the replication process, since we have an `A->B` checkpoint on `B`, but
it's for sending changes _from_ B _to_ A, not like one might expect `A->B`
based on its name.
To fix this, make sure to use a single checkpoint per direction of
replication. So, when change are pulled from B to A, the checkpoint is now
B->A, and when changes are pushed from B to A the checkpoint is also B->A.
It should look something like this:
```
[node A] [node B]
<-------pull------ JOB1
(A->B) --------push------>
--------pull------>
<-------push------ (B->A) JOB2
```
Since after this change we'll have some deprecated purge checkpoints to
clean up, it's probably also a good time to introduce purge checkpoint cleanup.
We have this for indexes but we didn't have it for the internal replicator.
That meant that on shard map reconfigurations, or node membership changes, user
would have to manually hunt down local (un-clustered) stale purge checkpoints
and remove them. Now this happens automatically when we compact, and before we
replicate between nodes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
