[
https://issues.apache.org/jira/browse/SOLR-11718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16322103#comment-16322103
]
Amrit Sarkar edited comment on SOLR-11718 at 1/11/18 12:02 PM:
---------------------------------------------------------------
[~varunthacker]
Thank you for the feedback.
bq. 1. In CdcrRequestHandlerTest#testCheckpointActions why have the asserts
been commented out?
CDCR has a typical behavior when Buffering is enabled, LastProcessedVersion
emits out {{-1}} which is picked as minimum for checkpoint, resulting -1 for
all shards' Checkpoints. The assertion is checkpoint to -1, which won't be the
case NOW as buffering is disabled by default and will emit rightful
tlog_version.
bq. 2. "Since the CdcrReplicationHandlerTest was failing, suggesting typical
Index Replication will take place when followers are numRecordsToKeep count
behind." - Maybe we should modify the test to assert document count instead of
just commenting it out?
Considering this is the default behavior of follower nodes when they fall back
by {{numRecordsToKeep}} w.r.t leader, I didn't write them up. I will add those
tests too in the same {{CdcrReplicationHandlerTest}}.
bq. 3. I don't quite understand the doc changes - "ENABLEBUFFER API has been
deprecated in favor of when buffering is enabled, the Update Logs will grow
without limit; they will never be purged."
Yeah, english. "ENABLEBUFFER API has been deprecated in favor of buffering is
disabled by default. And when buffering is enabled, the Update Logs will grow
without limit; they will never be purged". Is this better? makes sense?
was (Author: sarkaramr...@gmail.com):
[~varunthacker]
Thank you for the feedback.
bq. 1. In CdcrRequestHandlerTest#testCheckpointActions why have the asserts
been commented out?
CDCR has a typical behavior when Buffering is enabled, LastProcessedVersion
emits out {{-1}} which is picked as minimum for checkpoint, resulting -1 for
all shards' Checkpoints. The assertion is checkpoint to -1, which won't be the
case NOW as buffering is disabled by default and will emit rightful
tlog_version.
bq. 2. "Since the CdcrReplicationHandlerTest was failing, suggesting typical
Index Replication will take place when followers are numRecordsToKeep count
behind." - Maybe we should modify the test to assert document count instead of
just commenting it out?
Considering this is the default behavior of follower nodes when they fall back
by {{numRecordsToKeep}} w.r.t leader, I didn't write them up. I will add those
tests too in the same {{CdcrReplicationHandlerTest}}.
bq. 3. I don't quite understand the doc changes - "ENABLEBUFFER API has been
deprecated in favor of when buffering is enabled, the Update Logs will grow
without limit; they will never be purged."
Yeah, english. ENABLEBUFFER API has been deprecated in favor of buffering is
disabled by default. And when buffering is enabled, the Update Logs will grow
without limit; they will never be purged. Is this better? makes sense?
> Deprecate CDCR Buffer APIs
> --------------------------
>
> Key: SOLR-11718
> URL: https://issues.apache.org/jira/browse/SOLR-11718
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
> Affects Versions: 7.1
> Reporter: Amrit Sarkar
> Fix For: master (8.0), 7.3
>
> Attachments: SOLR-11718.patch, SOLR-11718.patch
>
>
> Kindly see the discussion on SOLR-11652.
> Today, if we see the current CDCR documentation page, buffering is "disabled"
> by default in both source and target. We don't see any purpose served by Cdcr
> buffering and it is quite an overhead considering it can take a lot heap
> space (tlogs ptr) and forever retention of tlogs on the disk when enabled.
> Also today, even if we disable buffer from API on source , considering it was
> enabled at startup, tlogs are never purged on leader node of shards of
> source, refer jira: SOLR-11652
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
