[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16195727#comment-16195727
]
Erick Erickson commented on SOLR-11278:
---
Amrit:
One strategy is to put debugging only code into master (not 7x) to test
hypotheses. Be sure to track it carefully and take it out as soon as you can of
course.
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6.1, 7.0
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Fix For: 7.1
>
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16195690#comment-16195690
]
Amrit Sarkar commented on SOLR-11278:
-
[~varunthacker],
Failure is very inconsistent. None of the failuers accessible at
https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/ doesn't have the test
failing and I am unable to produce it for 100 and 500 beasts. Not sure, how to
proceed on this now.
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6.1, 7.0
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Fix For: 7.1
>
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187021#comment-16187021
]
Steve Rowe commented on SOLR-11278:
---
bq. [...] today I see a failure [] Let's not hold up 7.0.1 for this?
+1
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6.1, 7.0
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Fix For: 7.1
>
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186592#comment-16186592
]
Varun Thacker commented on SOLR-11278:
--
Looks like it was happy for 1 day but today I see a failure :
https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/517/
I won't get time to look at this for another till tuesday atleast . Let's not
hold up 7.0.1 for this?
People who hit this race condition in 7.0 can fix it by restarting the target
cluster as the bootstrap will fail. Not the best experience but they are
starting a new cluster so hopefully not a big deal.
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6.1, 7.0
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Fix For: 7.1
>
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186578#comment-16186578
]
Steve Rowe commented on SOLR-11278:
---
Should the commits on this issue be backported to branch_7_0, to be included in
Solr 7.0.1?
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 6.6.1, 7.0
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Fix For: 7.1
>
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183361#comment-16183361
]
ASF subversion and git services commented on SOLR-11278:
Commit 1de25182370dec69a34a721127aee0ecc0fe in lucene-solr's branch
refs/heads/branch_7x from [~varunthacker]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1de2518 ]
SOLR-11278: Fix a race condition in the CDCR bootstrap process which could lead
to bootstraps cancelling itself
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0, 6.6.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Fix For: 7.1
>
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11278) Fix race in cdcr bootstrap process
[
https://issues.apache.org/jira/browse/SOLR-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183297#comment-16183297
]
ASF subversion and git services commented on SOLR-11278:
Commit 26677ab2b041a074fd984457677cbea8b58363ab in lucene-solr's branch
refs/heads/master from [~varunthacker]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=26677ab ]
SOLR-11278: Fix a race condition in the CDCR bootstrap process which could lead
to bootstraps cancelling itself
> Fix race in cdcr bootstrap process
> --
>
> Key: SOLR-11278
> URL: https://issues.apache.org/jira/browse/SOLR-11278
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
>Affects Versions: 7.0, 6.6.1
>Reporter: Amrit Sarkar
>Assignee: Varun Thacker
>Priority: Critical
> Labels: test
> Attachments: master-bs.patch, SOLR-11278-awaits-fix.patch,
> SOLR-11278-cancel-bootstrap-on-stop.patch, SOLR-11278.patch,
> SOLR-11278.patch, SOLR-11278.patch, SOLR-11278.patch, test_results
>
>
> {{CdcrBootstrapTest}} is failing while running beasts for significant
> iterations.
> The bootstrapping is failing in the test, after the first batch is indexed
> for each {{testmethod}}, which results in documents mismatch ::
> {code}
> [beaster] 2> 39167 ERROR
> (updateExecutor-39-thread-1-processing-n:127.0.0.1:42155_solr
> x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2)
> [n:127.0.0.1:42155_solr c:cdcr-target s:shard1 r:core_node2
> x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap
> operation failed
> [beaster] 2> java.util.concurrent.ExecutionException:
> java.lang.AssertionError
> [beaster] 2> at
> java.util.concurrent.FutureTask.report(FutureTask.java:122)
> [beaster] 2> at
> java.util.concurrent.FutureTask.get(FutureTask.java:192)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:654)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
> [beaster] 2> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [beaster] 2> at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [beaster] 2> at
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [beaster] 2> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [beaster] 2> at java.lang.Thread.run(Thread.java:748)
> [beaster] 2> Caused by: java.lang.AssertionError
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:813)
> [beaster] 2> at
> org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:724)
> [beaster] 2> at
> com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
> [beaster] 2> ... 5 more
> {code}
> {code}
> [beaster] [01:37:16.282] FAILURE 153s |
> CdcrBootstrapTest.testBootstrapWithSourceCluster <<<
> [beaster]> Throwable #1: java.lang.AssertionError: Document mismatch on
> target after sync expected:<2000> but was:<1000>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
