[
https://issues.apache.org/jira/browse/SOLR-11003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16073572#comment-16073572
]
Amrit Sarkar commented on SOLR-11003:
-------------------------------------
*Proposed design =>*
*Configuration:*
Configure both collections as source and point to their targets. Sample
configurations listed below:
_Cluster2:_
{code}
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
<lst name="replica">
<str name="zkHost">${cdcr.cluster1.zkHost}</str>
<str name="source">cdcr-cluster2</str>
<str name="target">cdcr-cluster1</str>
</lst>
<lst name="replicator">
<str name="threadPoolSize">1</str>
<str name="schedule">1000</str>
<str name="batchSize">1000</str>
</lst>
<lst name="updateLogSynchronizer">
<str name="schedule">1000</str>
</lst>
</requestHandler>
{code}
_Cluster1:_
{code}
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
<lst name="replica">
<str name="zkHost">${cdcr.cluster2.zkHost}</str>
<str name="source">cdcr-cluster1</str>
<str name="target">cdcr-cluster2</str>
</lst>
<lst name="replicator">
<str name="threadPoolSize">1</str>
<str name="schedule">1000</str>
<str name="batchSize">1000</str>
</lst>
<lst name="updateLogSynchronizer">
<str name="schedule">1000</str>
</lst>
</requestHandler>
{code}
*Replication Improvement:*
CdcrReplicator replays the tlogs, creates new UpdateRequests and forwards them
to target collection. *The plan is to persist information in the tlogs' each
update entries telling the collection this update is received as a CDCR forward
and it should be not be forwarded further.*
Each CDCR update forward contains the param key "cdcr.update" and value empty.
Once the target collection received that particular update, before writing it
down to transaction log, we check whether the UpdateCommand -> SolrRequest ->
contains params key "cdcr.update", if there is, apart from the entries listed
for each update, we add a flag, say isCdcr. "true" and write it to tlog. When
CdcrReplicator reads updates from the tlog, and if the flag is present / set,
skip that update for forwarding and read next.
We had to be very careful of where to put the flag so that we don't break the
backward compatibility and hence we added at the last index for DELETE and
DELETEBYQUERY while second last index for ADD. Reason listed below.
> Enabling bi-directional CDCR active-active clusters
> ---------------------------------------------------
>
> Key: SOLR-11003
> URL: https://issues.apache.org/jira/browse/SOLR-11003
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
> Reporter: Amrit Sarkar
>
> The latest version of Solr CDCR across collections / clusters is in
> active-passive format, where we can index into source collection and the
> updates gets forwarded to the passive one and vice-versa is not supported.
> https://lucene.apache.org/solr/guide/6_6/cross-data-center-replication-cdcr.html
> https://issues.apache.org/jira/browse/SOLR-6273
> We are try to get a design ready to index in both collections and the
> updates gets reflected across the collections in real-time.
> ClusterACollectionA => ClusterBCollectionB | ClusterBCollectionB =>
> ClusterACollectionA.
> The best use-case would be to we keep indexing in ClusterACollectionA which
> forwarded the updates to ClusterBCollectionB. If ClusterACollectionA gets
> down, we point the indexer and searcher application to ClusterBCollectionB.
> Once ClusterACollectionA is up, depending updates count, they will be
> bootstrapped or forwarded to ClusterACollectionA from ClusterBCollectionB.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
