wohali opened a new pull request #1811: Add new /{db}/_sync_shards endpoint
(admin-only)
URL: https://github.com/apache/couchdb/pull/1811
## Overview
This server admin-only endpoint forces an n-way sync of all shards
across all nodes on which they are hosted.
This can be useful for an administrator adding a new node to the
cluster, after updating _dbs so that the new node hosts an existing db
with content, to force the new node to sync all of that db's shards.
Users may want to bump their `[mem3] sync_concurrency` value to a
larger figure for the duration of the shards sync.
Closes #1807
## Testing recommendations
Testing this requires a non-standard setup, so I have not written an
automated test case for this. (We should think about how this could
possibly be done in the Elixir test suite in the future.)
Manual test script:
```console
# set up 2 nodes not joined together and put data in node1
dev/run -n 2 --no-join &
export AUTH="root:PASSWORD_GOES_HERE"
curl -X PUT http://$AUTH@localhost:15984/foo
curl -X PUT http://$AUTH@localhost:15984/foo/abc -d '{"a":"b"}'
curl -X PUT http://$AUTH@localhost:15984/foo/def -d '{"g":"h"}'
# see the shards with data in node1, here it's shards 00 and 20
ls -lR dev/lib/node1/data/shards/*/foo.*.couch
# join the nodes together
curl -X PUT http://$AUTH@localhost:15986/_nodes/[email protected] -d '{}'
curl http://$AUTH@localhost:15984/_membership
curl http://$AUTH@localhost:25984/_membership
# have node2 participate in serving foo for all shards
curl -X PUT http://$AUTH@localhost:15986/_dbs/foo -d
'{"_id":"foo","_rev":"1-d0f95fcbe46dd04f66bc61906e202d56","shard_suffix":[46,49,53,52,52,56,51,52,53,49,51],"changelog":[["add","00000000-1fffffff","[email protected]"],["add","20000000-3fffffff","[email protected]"],["add","40000000-5fffffff","[email protected]"],["add","60000000-7fffffff","[email protected]"],["add","80000000-9fffffff","[email protected]"],["add","a0000000-bfffffff","[email protected]"],["add","c0000000-dfffffff","[email protected]"],["add","e0000000-ffffffff","[email protected]"]],"by_node":{"[email protected]":["00000000-1fffffff","20000000-3fffffff","40000000-5fffffff","60000000-7fffffff","80000000-9fffffff","a0000000-bfffffff","c0000000-dfffffff","e0000000-ffffffff"],"[email protected]":["00000000-1fffffff","20000000-3fffffff","40000000-5fffffff","60000000-7fffffff","80000000-9fffffff","a0000000-bfffffff","c0000000-dfffffff","e0000000-ffffffff"]},"by_range":{"00000000-1fffffff":["[email protected]","[email protected]"],"20000000-3fffffff":["[email protected]","[email protected]"],"60000000-7fffffff":["[email protected]","[email protected]"],"80000000-9fffffff":["[email protected]","[email protected]"],"a0000000-bfffffff":["[email protected]","[email protected]"],"c0000000-dfffffff":["[email protected]","[email protected]"],"e0000000-ffffffff":["[email protected]","[email protected]"]}}'
# double check shards in node2 are still empty (all should be same file size)
ls -lR dev/lib/node2/data/shards/*/foo.*.couch
# do a shard sync
curl -X POST http://$AUTH@localhost:15984/foo/_sync_shards
# check that shards 00 and 22 on node2 now have data from node1
ls -lR dev/lib/node2/data/shards/*/foo.*.couch
# optionally manually check docs in those shards
# substitute the correct timestamp in the shard name here
curl
http://$AUTH@localhost:25986/shards%2F00000000-1fffffff%2Ffoo.1544834513/_all_docs
curl
http://$AUTH@localhost:25986/shards%2F20000000-3fffffff%2Ffoo.1544834513/_all_docs
```
## Checklist
- [X] Code is written and works correctly;
- [X] Changes are covered by tests;
- [ ] Documentation reflects the changes;
Documentation PR will be issued once this PR (and the bikeshedding over the
name of the new endpoint) is approved.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
