[13/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Remove `couch_replicator_manager` Its logic is now spread amongst a few modules: - `couch_multidb_changes`: Monitor `*/_replicator` dbs, run callbacks for db / doc changes. - `couch_replicator_clustering`: Keep track of cluster membership. Trigger rescans, by restarting `couch_mult

[1/2] fauxton commit: updated refs/heads/master to a9d5d25

Repository: couchdb-fauxton Updated Branches: refs/heads/master c0c0c1c54 -> a9d5d259a travis: remove leftover from debugging this was added some time ago for debugging, but did not fix the error PR: #725 PR-URL: https://github.com/apache/couchdb-fauxton/pull/725 Reviewed-By: Benjamin Keen

[2/2] fauxton commit: updated refs/heads/master to a9d5d25

travis: update node to version 6 PR: #725 PR-URL: https://github.com/apache/couchdb-fauxton/pull/725 Reviewed-By: Benjamin Keen Project: http://git-wip-us.apache.org/repos/asf/couchdb-fauxton/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-fauxton/commit/64294a77 Tree: http://git-wi

[couchdb-couch-replicator] Git Push Summary

Repository: couchdb-couch-replicator Updated Branches: refs/heads/63012-defensive [deleted] 1afa5ea0c

[17/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Add EUnit coverage to replicator tests Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/407b3b85 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/407b3b85 Diff:

[10/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Remove extra process spawn and retry on error from rep. manager * Removed spawning an extra process to start replications. * Removed retrying on error (scheduler will handle that better). Replication manager ended up with no state. `rep.py` tools show replications still trigger and replicate

[21/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #10 from cloudant/63012-remove-infinity-timeout Remove `infinity` timeout from is_stable() call Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/24edd808 Tree: http://g

[30/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

don't start job immediately Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/6a913dc6 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/6a913dc6 Diff: http://git

[15/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Update couch_replicator_sup. Move clustering and event manager to the top. - If `_clustering` crashes we'd have to restart scheduler. Replication used clustering module to decide if they should still run on current. Node. Move clustering before scheduler in child order. If scheduler crashes, clus

[27/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #13 from cloudant/63012-handle-duplicate-docs-better Better handling for duplicate replication docs. Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/343e6f7a Tree: htt

[06/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Stitch everything together. Use scheduler to spawn/kill replication jobs. Replicator manager is now replicator_manager_sup supervisor with 3 children (rest_for_one supervision): Clustering : ownership and cluster configuration Replicator manager : handles replication starts Multi-db change

[35/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #17 from cloudant/63012-fix-log-msg fix log message Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/5dd0cb72 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch

[41/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #20 from cloudant/63012-log-clarity clarify when doc processor is ignoring docs Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/554eac89 Tree: http://git-wip-us.apache

[38/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

clarify when doc processor is ignoring docs Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/e9d4db5a Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/e9d4db5a D

[34/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #15 from cloudant/63012-remove-triggered-doc-update Remove state fields from triggered/error docs Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/9c914df9 Tree: http:/

[20/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Remove `infinity` timeout from is_stable() call Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/b455d26b Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/b455d2

[32/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

fix log message Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/832e36f0 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/832e36f0 Diff: http://git-wip-us.apac

[44/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Remove defensive programming We now explicitly demonitor, so we should never get a DOWN message unless we have a job with a pid. Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/1afa5ea0

[14/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Split couch_replicator.erl into 2 modules. `couch_replicator_doc_processor` - Handles multi-db changes callbacks so it starts replications defined in _replicator docs. `couch_replicator` - Handles `_replicate` http endpoint and creating local `_replicator` db. Project: http://git-wi

[16/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Improve clustering module & rescan handling Now there is a grace period during startup where nodes are expected to be added to the cluster. During that period a shorter quiet time applies (5 seconds by default). This allow replicator change feeds to start quicker, instead of waiting for more than

[24/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Handle error in replication documents Retry on <<"error">> state. Those are documents updated by the older replicator code. Some of those errors were transient. Introduce <<"failed">> state for documents which fail to parse as valid replicator #rep{} records and can't even be submitted as jobs t

[01/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Repository: couchdb-couch-replicator Updated Branches: refs/heads/63012-defensive [created] 1afa5ea0c Multi-db shard change monitoring. Monitors shards which match a suffix for creation, deletion, and doc updates. To use implement `couch_multidb_changes` behavior. Call `start_link` with DbSu

[29/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #14 from cloudant/63012-fix-indentation Fix indentation in doc_processor Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/aa621a8e Tree: http://git-wip-us.apache.org/re

[33/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Remove state fields from triggered/error docs Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/6579c194 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/6579c194

[36/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

remove misleading log noise Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/f76d5cfc Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/f76d5cfc Diff: http://git

[22/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Remove unused variable `Period` Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/a309e8d1 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/a309e8d1 Diff: http:/

[31/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #16 from cloudant/63012-scheduler-dont-start-immediately don't start job immediately Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/2aceb94c Tree: http://git-wip-us.a

[05/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Refactor replication manager, utils before switching to scheduler Cleaned up code and prepared to switch scheduler. This is a refactor as replications still work, eunit tests still pass. Changes: * Code from replication manager which deals with doc updating moved to `couch_replicator_docs.er

[28/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Fix indentation in doc_processor Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/9222dff6 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/9222dff6 Diff: http:

[23/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #11 from cloudant/63012-fixed-unused-period-var Remove unused variable `Period` Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/8dd431dd Tree: http://git-wip-us.apache

[08/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Fix errors in replication manager which prevented replication from runnig. Add a few more dialyzer types to couch_replication_manager. Replications tested with `rep.py` utility: ``` rep.replicate_1_to_n_then_check_replication(2) creating rdyno_src_0001 > created 1 dbs with prefix rdyno_src_ cr

[11/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #6 from cloudant/63012-remove-extra-process-spawn Remove extra process spawn and retry on error from rep. manager Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/e69aaa

[09/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #5 from cloudant/63012-make-replications-work Replications now run Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/46362d3b Tree: http://git-wip-us.apache.org/repos/as

[26/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Better handling for duplicate replication docs. Duplicate replication docs are documents which specify the same replication. Handle 3 cases: 1) Documents are in same database, but potentially differen shards. 2) Documents are in two different replicator databases. For example in `a/_replicat

[03/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Introduce couch_replicator_scheduler BugzID: 63012 Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/bb34ad58 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/bb

[42/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #21 from cloudant/63012-completed clean up completed jobs Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/bd429046 Tree: http://git-wip-us.apache.org/repos/asf/couchdb

[07/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #4 from cloudant/63012-repman-update Update replication manager to use new code Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/30c033d0 Tree: http://git-wip-us.apache

[39/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

clean up completed jobs Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/aa4e80de Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/aa4e80de Diff: http://git-wip

[43/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #22 from cloudant/63012-zero-jobs allow max_jobs == 0 Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/5ca03769 Tree: http://git-wip-us.apache.org/repos/asf/couchdb-cou

[04/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/blob/5218edf8/src/couch_replicator_utils.erl -- diff --git a/src/couch_replicator_utils.erl b/src/couch_replicator_utils.erl index 76bc8e1..8bf350d 100644 --- a/src/co

[02/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Cluster ownership module implementation. Handles owner/2 calls and returns: `{ok, node()} | {error, no_owner} | {error, unstable}` `unstable` means cluster has received nodeup or nodedown recently ( a configurable parameter defaulting to 60 sec). `no_owner` means ownership cannot be determined,

[40/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

allow max_jobs == 0 Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/07ec670d Tree: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/tree/07ec670d Diff: http://git-wip-us.

[37/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #18 from cloudant/63012-log-noise remove misleading log noise Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/ed40d85e Tree: http://git-wip-us.apache.org/repos/asf/cou

[18/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #7 from cloudant/63012-removed-replicator-manager Remove `couch_replicator_manager` Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/f7809707 Tree: http://git-wip-us.ap

[12/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/blob/72e1f1ed/test/couch_replicator_test_helper.erl -- diff --git a/test/couch_replicator_test_helper.erl b/test/couch_replicator_test_helper.erl index 5b9d366..01a60

[19/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #8 from cloudant/63012-add-eunit-coverage-report Add EUnit coverage to replicator tests Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/5858d316 Tree: http://git-wip-u

[25/44] couch-replicator commit: updated refs/heads/63012-defensive to 1afa5ea

Merge pull request #12 from cloudant/63012-handle-error-in-doc-processor Handle error in replication documents Project: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/repo Commit: http://git-wip-us.apache.org/repos/asf/couchdb-couch-replicator/commit/4dfa279f Tree: http://git-w