blackwell-systems commented on issue #6162: URL: https://github.com/apache/couchdb/issues/6162#issuecomment-6150157045
Thanks for running it. My earlier runs were all single-shard (q=1, n=1), which is why I always saw it. On a fresh 3.5.3 with more shards it's intermittent, as you found: the original script diverged in 10/10 runs at q=1, 18/20 at the default q=2, and 3/10 at q=8. The variable is whether the replicator re-sends X through the `_revs_diff` skip in `find_missing`, which needs X to come after enough all-missing batches. I checked at q=8 with the filler ids chosen via `_shards`: - fillers all in X's shard (fixed order): X was re-sent (docs_written 7 instead of 6) and the targets diverged in 10/10 runs; - fillers all in other shards: X was re-sent and they diverged in 2/10. Across those 20 runs, the re-send, the no-VDU target's two roots joining, and the divergence always coincided. When X isn't re-sent, both targets end with X live at the old revision. The server-side part doesn't depend on shards or the replicator. The script below sends the same four replicated writes (`_bulk_docs`, `new_edits: false`) to two targets that differ only in an accept-all VDU, modeling replicas at `_revs_limit` 3 against 1000. It diverged in 30/30 runs at the default q, q=1 and q=8 (t1 ends deleted, t2 ends live at `1-a`). The same happens at 999 vs 1000 with a 1,000-revision history. <details> <summary>repro (curl + jq)</summary> ```sh COUCH=http://admin:[email protected]:5984 J='Content-Type: application/json' A=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa; B=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb; C=cccccccccccccccccccccccccccccccc D=dddddddddddddddddddddddddddddddd; E=eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee for db in t1 t2; do curl -sS -X DELETE "$COUCH/$db" > /dev/null; curl -sS -X PUT "$COUCH/$db" > /dev/null; done curl -sS -X PUT "$COUCH/t2/_design/accept" -H "$J" \ -d '{"validate_doc_update":"function(newDoc, oldDoc, userCtx, secObj) {}"}' > /dev/null put() { curl -sS -X POST "$COUCH/$1/_bulk_docs" -H "$J" -d "{\"new_edits\":false,\"docs\":[$2]}" > /dev/null; } for t in t1 t2; do # 1. from C: rev 1, history [1] put $t "{\"_id\":\"X\",\"_rev\":\"1-$A\",\"_revisions\":{\"start\":1,\"ids\":[\"$A\"]},\"v\":\"a\"}" # 2. from B (revs_limit 3): rev 4, history [2..4], so X now has two roots put $t "{\"_id\":\"X\",\"_rev\":\"4-$D\",\"_revisions\":{\"start\":4,\"ids\":[\"$D\",\"$C\",\"$B\"]},\"v\":\"d\"}" # 3. from A (revs_limit 1000): rev 4 again, full history [1..4] put $t "{\"_id\":\"X\",\"_rev\":\"4-$D\",\"_revisions\":{\"start\":4,\"ids\":[\"$D\",\"$C\",\"$B\",\"$A\"]},\"v\":\"d\"}" # 4. from D (revs_limit 3): deletion rev 5, history [3..5] put $t "{\"_id\":\"X\",\"_rev\":\"5-$E\",\"_deleted\":true,\"_revisions\":{\"start\":5,\"ids\":[\"$E\",\"$D\",\"$C\"]}}" done for t in t1 t2; do echo "$t: $(curl -sS "$COUCH/$t/X")" echo "$t leaves: $(curl -sS "$COUCH/$t/X?open_revs=all&revs=true" -H 'Accept: application/json' \ | jq -c '[.[].ok | {rev: ._rev[0:3], deleted: (._deleted // false), history: (._revisions.ids | length)}]')" done ``` Expected output: ``` t1: {"error":"not_found","reason":"deleted"} t1 leaves: [{"rev":"5-e","deleted":true,"history":5}] t2: {"_id":"X","_rev":"1-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","v":"a"} t2 leaves: [{"rev":"1-a","deleted":false,"history":1},{"rev":"5-e","deleted":true,"history":4}] ``` </details> Step 3 is the one that matters: on t1 it joins the two roots, and on t2 (with the VDU) it's dropped as an already-present leaf before `validate_doc_update` runs, so the roots stay separate and step 4's deletion only covers one of them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
