RE: CDCR

2020-12-01 Thread Gell-Holleron, Daniel
: 01 December 2020 11:49 To: solr-user@lucene.apache.org Subject: RE: CDCR Hi Shalin, I did try to do that but it hadn't made any difference, the remote clusters did not update. Autocommit is already set on the remote clusters as follows: ${solr.autoCommit.maxTime:15

RE: CDCR

2020-12-01 Thread Gell-Holleron, Daniel
@lucene.apache.org Subject: Re: CDCR EXTERNAL EMAIL - Be cautious of all links and attachments. If you manually issue a commit operation on the remote clusters, do you see any updates? If yes, then you should set autoCommit on the remote clusters. If no, then check the logs on the cluster which

Re: CDCR

2020-11-28 Thread Shalin Shekhar Mangar
:11 PM Gell-Holleron, Daniel < daniel.gell-holle...@gb.unisys.com> wrote: > Hello, > > Does anybody have advice on why CDCR would say its Forwarding updates > (with no errors) even though the solr servers its replicating to aren't > updating? > > We have just

CDCR

2020-11-25 Thread Gell-Holleron, Daniel
Hello, Does anybody have advice on why CDCR would say its Forwarding updates (with no errors) even though the solr servers its replicating to aren't updating? We have just under 50 million documents, that are spread across 4 servers. Each server has a node each. One side is updating happily

Re: UpdateProcessorChains -cdcr processor along with ignore commit processor

2020-07-18 Thread Shawn Heisey
On 7/15/2020 11:39 PM, Natarajan, Rajeswari wrote: Resending this again as I still could not make this work. So would like to know if this is even possible to have both solr.CdcrUpdateProcessorFactory and solr.IgnoreCommitOptimizeUpdateProcessorFactory in solrconfig.xml and get both

Re: CDCR stress-test issues

2020-07-17 Thread Jörn Franke
Instead of CDCR you may simply duplicate the pipeline across both data centers. Then there is no need at each step of the pipeline to replicate (storage to storage, index to index etc.). Instead both pipelines run in different data centers in parallel. > Am 24.06.2020 um 15:46 schrieb Oak

RE: CDCR stress-test issues

2020-07-17 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
Yes, I saw that yesterday. I guess that I was not the only one who noticed the unreliability after all. -Original Message- From: Ishan Chattopadhyaya Sent: Friday, July 17, 2020 1:17 AM To: solr-user Subject: Re: CDCR stress-test issues FYI, CDCR support, as it exists in Solr today

Re: CDCR stress-test issues

2020-07-16 Thread Ishan Chattopadhyaya
FYI, CDCR support, as it exists in Solr today, has been deprecated in 8.6. It suffers from serious design flaws and it allows such things to happen that you observe. While there may be workarounds, it is advisable to not rely on CDCR in production. Thanks, Ishan On Thu, 2 Jul, 2020, 1:12 am

UpdateProcessorChains -cdcr processor along with ignore commit processor

2020-07-15 Thread Natarajan, Rajeswari
, Rajeswari On 7/14/20, 12:40 PM, "Natarajan, Rajeswari" wrote: Hi , Would like to have these two processors (cdcr and ignorecommit) in solrconfig.xml . But cdcr fails with below error , with either cdcr-processor-chain or ignore-commit-from-client chain versio

UpdateProcessorChains -cdcr processor along with ignore commit processor

2020-07-14 Thread Natarajan, Rajeswari
Hi , Would like to have these two processors (cdcr and ignorecommit) in solrconfig.xml . But cdcr fails with below error , with either cdcr-processor-chain or ignore-commit-from-client chain version conflict for 60d35f0850afac66 expected=1671629672447737856 actual=-1, retry=0 commError

RE: CDCR stress-test issues

2020-07-01 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
added after the missing records). Does anyone yet have any suggestion how to get CDCR to work properly? -Original Message- From: Oakley, Craig (NIH/NLM/NCBI) [C] Sent: Wednesday, June 24, 2020 9:46 AM To: solr-user@lucene.apache.org Subject: CDCR stress-test issues In attempting

Re: SOLR CDCR fails with JWT authorization configuration

2020-06-26 Thread Jan Høydahl
I found this in the documentation https://lucene.apache.org/solr/guide/8_5/cdcr-architecture.html#cdcr-limitations <https://lucene.apache.org/solr/guide/8_5/cdcr-architecture.html#cdcr-limitations> : CDCR doesn’t support Basic Authentication features across clusters. Th

RE: SOLR CDCR fails with JWT authorization configuration

2020-06-25 Thread Phatkar, Swapnil (Contractor)
Hi, CDCR might be deprecated really soon now --> In this case will it be there any alternate to this. However, if this turns out to be not supported or a bug, then we can file a JIRA issue. --> it will be great if you raise the JIRA ticket for it. So that we will be more clear th

Re: SOLR CDCR fails with JWT authorization configuration

2020-06-25 Thread Jan Høydahl
I’m mostly trying to identify whether what you are trying to to is a supported option at all, or of perhaps CDCR is only tested without authentication in place. You would also be interested in the fact that CDCR might be deprecated really soon now, see https://issues.apache.org/jira/browse/SOLR

RE: SOLR CDCR fails with JWT authorization configuration

2020-06-25 Thread Phatkar, Swapnil (Contractor)
Hi, 1. Solr is relying on PKI for the request (one cluster sends PKI header to the node in the other cluster) -- > I have not configured anything explicitly. Just followed the steps mention @https://lucene.apache.org/solr/guide/8_4/cdcr-config.html. Is there any additional step ?

Re: SOLR CDCR fails with JWT authorization configuration

2020-06-25 Thread Jan Høydahl
since it is in another cluster I’m not familiar with the CDCR code used here. Have you tried BasicAuth and do you have the same issue then? Jan > 25. jun. 2020 kl. 13:20 skrev Phatkar, Swapnil (Contractor) > : > > > > Whoever is sending calls to /solr/express_shar

RE: SOLR CDCR fails with JWT authorization configuration

2020-06-25 Thread Phatkar, Swapnil (Contractor)
Whoever is sending calls to /solr/express_shard1_replica_n3/cdcr will have to make sure to forward JWT -- How do I forward JWT from source to target server ?? You could try 'forwardCredentials:true' in security.json -- How can I try this ? Can you suggest me sample security.json which

Re: SOLR CDCR fails with JWT authorization configuration

2020-06-25 Thread Jan Høydahl
to /solr/express_shard1_replica_n3/cdcr will have to make sure to forward JWT and not just rely on PKI. PKI won’t work since the two clusters have different ZK and Solr by default only trust PKI between nodes registered in ZK. You could try 'forwardCredentials:true' in security.json, but I’m

SOLR CDCR fails with JWT authorization configuration

2020-06-24 Thread Phatkar, Swapnil (Contractor)
Hi Team , I am trying to configure CDCR for SOLR 8.4.1 . With the provided configuration I can able to replicate the indexes from Source server to Target server. This setup even works with SSL configuration using Https protocol. But the moment I have introduced JWT authorization by enforcing

Re: CDCR stress-test issues

2020-06-24 Thread matthew sporleder
On Wed, Jun 24, 2020 at 9:46 AM Oakley, Craig (NIH/NLM/NCBI) [C] wrote: > > In attempting to stress-test CDCR (running Solr 7.4), I am running into a > couple of issues. > > One is that the tlog files keep accumulating for some nodes in the CDCR > system, particularly for th

CDCR stress-test issues

2020-06-24 Thread Oakley, Craig (NIH/NLM/NCBI) [C]
In attempting to stress-test CDCR (running Solr 7.4), I am running into a couple of issues. One is that the tlog files keep accumulating for some nodes in the CDCR system, particularly for the non-Leader nodes in the Source SolrCloud. No quantity of hard commits seem to cause any of these tlog

RE: CDCR behaviour

2020-06-08 Thread Gell-Holleron, Daniel
be greatly appreciated. Thanks, Daniel -Original Message- From: Jason Gerlowski Sent: 05 June 2020 12:18 To: solr-user@lucene.apache.org Subject: Re: CDCR behaviour Hi Daniel, Just a heads up that attachments and images are stripped pretty aggressively by the mailing list - none

Re: CDCR behaviour

2020-06-05 Thread Jason Gerlowski
-Holleron, Daniel < daniel.gell-holle...@gb.unisys.com> wrote: > Hi, > > > > Looks for some advice, sent a few questions on CDCR the last couple of > days. > > > > I just want to see if this is expected behavior from Solr or not? > > > > When a

CDCR behaviour

2020-06-04 Thread Gell-Holleron, Daniel
Hi, Looks for some advice, sent a few questions on CDCR the last couple of days. I just want to see if this is expected behavior from Solr or not? When a document is added to Site A, it is then supposed to replicate across, however in the statistics page I see the following: Site

Bi-Directional CDCR

2020-06-03 Thread Gell-Holleron, Daniel
Hi there, I need some advice on how Bi-Directional CDCR is properly configured. I've created a collection on Site A (3 Solr nodes, 5 ZooKeepers). I've also created a collection on site B (3 Solr nodes, 5 ZooKeepers). These both have the same number of shards (not sure if that is a factor

Solr TLS for CDCR

2020-05-14 Thread ChienHuaWang
Does anyone have experience to setup TLS for Solr CDCR? I read the documentation: https://lucene.apache.org/solr/guide/7_6/enabling-ssl.html Would this apply to CDCR once enable? or we need additional configuration for CDCR? Appreciate any feedback -- Sent from: https://lucene.472066.n3

Re: Tlogs are not purged when CDCR is enabled

2020-01-08 Thread Louis
Another finding is, no matter how I tried to disable buffer with the following setup on target node, it is always enabled first time. disabled Once I call CDCR API to disable buffer, it turns to be disabled. I wonder if https://issues.apache.org/jira/browse/SOLR-11652 is related

tlogs are not purged when CDCR is enabled

2020-01-08 Thread Louis
Using Solr 7.7.3-snapshot, 1 shard + 3 replicas on source and target cluster When unidirectional CDCR enabled and buffer disabled, my understanding is, when data is successfully forwarded to target and committed, tlogs on both source and target should be purged. However, the source node doesn't

source cluster sends incorrect recovery request to target cluster when CDCR is enabled

2020-01-07 Thread alwaysbluesky
Hi, Running Solr 7.7.2, cluster with 3 replicas When CDCR is enabled, one of the target nodes gets an incorrect recovery request. Below is the content of the state.json file from the zookeeper. "shards":{"shard1":{ "range":"8000

solr unable to locate core on CDCR

2020-01-07 Thread alwaysbluesky
Whenever I create a new collection on solr 7.7.2 cluster CDCR(uni directional), and once I disable buffer on both source and target nodes and start CDCR process on the source node, then I encounter the error message "solr unable to locate core..." on one of target node. On source

Re: Three questions about huge tlog problem and CDCR

2019-12-20 Thread alwaysbluesky
sure. I disabled buffer and started cdcr by calling api on both side. And when I do indexing, I see the size of tlog folder stays within 1MB while the size of index folder is increasing. So I imagined that tlog would be consumed by target node and cleared, and data is being forwarded to target

Re: Three questions about huge tlog problem and CDCR

2019-12-19 Thread damienk
Did you run /cdcr?action=DISABLEBUFFER on both sides? On Fri, 20 Dec 2019 at 05:22, alwaysbluesky wrote: > Thank you for the advice. > > By the way, when I upload a new collectin configuration to zookeepr and > enable bidirectional CDCR for the collections on both prod and dr

Re: Three questions about huge tlog problem and CDCR

2019-12-19 Thread alwaysbluesky
Thank you for the advice. By the way, when I upload a new collectin configuration to zookeepr and enable bidirectional CDCR for the collections on both prod and dr side(/cdcr?action=START), and reload the collections, CDCR usually didn't work. So if I restarted entire nodes in the cluster on both

Re: Three questions about huge tlog problem and CDCR

2019-12-19 Thread Erick Erickson
This usually indicates that the connection between DCs is broken and one or the other is falling behind. Note: “bidirectional” does _not_ mean that you can index to both DCs simultaneously, rather than you can switch from indexing in one DC to the other…. Best, Erick > On Dec 19, 2019, at

Re: Three questions about huge tlog problem and CDCR

2019-12-18 Thread alwaysbluesky
found a typo. correcting "updateLogSynchronizer" is set to 6(1 min), not 1 hour -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Three questions about huge tlog problem and CDCR

2019-12-18 Thread Louis
* Environment: Solr Cloud 7.7.0, 3 nodes / CDCR bidirectional / CDCR buffer disabled Hello All, I have some problem with tlog. They are getting bigger and bigger... They don't seem to be deleted at all even after hard commit, so now the total size of tlog files is more than 21GB.. Actually I

Re: CDCR cpu usage 100% with some errors

2019-10-28 Thread Louis
I just saw this article. https://issues.apache.org/jira/browse/SOLR-13349 Can my issue be related to this? -- Sent from: https://lucene.472066.n3.nabble.com/Solr-User-f472068.html

CDCR cpu usage 100% with some errors

2019-10-28 Thread Louis
* Solr Version 7.7. Using Cloud with CDCR * 3 replicas 1 shard on production and disaster recovery Hi, Last week, I posted a question about tlogs - https://lucene.472066.n3.nabble.com/tlogs-are-not-deleted-td4451323.html#a4451430 I disabled buffer based on the advice, but still, tlogs

CDCR cpu usage 100% with some errors

2019-10-28 Thread Louis
* Solr Version 7.7. Using Cloud with CDCR * 3 replicas 1 shard on production and disaster recovery Hi, Last week, I posted a question about tlogs - https://lucene.472066.n3.nabble.com/tlogs-are-not-deleted-td4451323.html#a4451430 I disabled buffer based on the advice, but still, tlogs

Re: cdcr replicator NPE errors

2019-10-25 Thread Jay Potharaju
Thanks Shawn! Can any of the committers comment about the CDCR error that I posted above? Thanks Jay On Fri, Oct 25, 2019 at 2:56 PM Shawn Heisey wrote: > On 10/25/2019 3:22 PM, Jay Potharaju wrote: > > Is there a solr slack channel? > > People with @apache.org email addres

Re: cdcr replicator NPE errors

2019-10-25 Thread Shawn Heisey
On 10/25/2019 3:22 PM, Jay Potharaju wrote: Is there a solr slack channel? People with @apache.org email addresses can readily join the ASF workspace, I do not know whether it is possible for others. That workspace might be only for ASF members. https://the-asf.slack.com In that

Re: cdcr replicator NPE errors

2019-10-25 Thread Jay Potharaju
Is there a solr slack channel? Thanks Jay Potharaju On Fri, Oct 25, 2019 at 9:00 AM Jay Potharaju wrote: > Hi, > I am frequently seeing cdcr-replicator null pointer exception errors in > the logs. > Any suggestions on how to address this? > *Solr version: 7.7.2* > > Ex

cdcr replicator NPE errors

2019-10-25 Thread Jay Potharaju
Hi, I am frequently seeing cdcr-replicator null pointer exception errors in the logs. Any suggestions on how to address this? *Solr version: 7.7.2* ExecutorUtil Uncaught exception java.lang.NullPointerException thrown by thread: cdcr-replicator-773-thread-3 java.lang.Exception: Submitter stack

RE: CDCR tlog corruption leads to infinite loop

2019-09-11 Thread Webster Homer
-Original Message- From: Webster Homer Sent: Monday, September 09, 2019 4:17 PM To: solr-user@lucene.apache.org Subject: CDCR tlog corruption leads to infinite loop We are running Solr 7.2.0 Our configuration has several collections that are loaded into a solr cloud which is set to replicate

CDCR tlog corruption leads to infinite loop

2019-09-09 Thread Webster Homer
We are running Solr 7.2.0 Our configuration has several collections that are loaded into a solr cloud which is set to replicate using CDCR to 3 different solrclouds. All of our target collections have 2 shards with two replicas per shard. Our source collection has 2 shards, and 1 replica per

Re: Turn off CDCR for only selected target clusters

2019-08-28 Thread Arnold Bronley
@Shawn: You are right. In my case, the collection name is same as configuration name and that is why it works. Do you know if there is some other property that I can use that refers to the collection name instead? On Wed, Aug 28, 2019 at 3:52 PM Shawn Heisey wrote: > On 8/28/2019 1:42 PM,

Re: Turn off CDCR for only selected target clusters

2019-08-28 Thread Shawn Heisey
On 8/28/2019 1:42 PM, Arnold Bronley wrote: I have configured the SolrCloud collection-wise only and there is no other way. The way you have defined 3 zkHosts (comma separated values for zkHost property), I tried that one before as it was more intuitive. But it did not work for me. I had to use

Re: Turn off CDCR for only selected target clusters

2019-08-28 Thread Arnold Bronley
(which does not work in my case and I have tested it), my question was how to NOT send CDCR updates to targetZkHost2 and targetZkHost3 but not targetZkHost1? On Tue, Aug 13, 2019 at 3:23 PM Erick Erickson wrote: > You configure CDCR by _collection_, so this question really makes no > sense. &

Re: Turn off CDCR for only selected target clusters

2019-08-13 Thread Erick Erickson
You configure CDCR by _collection_, so this question really makes no sense. You’d never mention collection.configName. So what I suspect is that you’re misreading the docs. ${targetZkHost1},${targetZkHost2},${targetZkHost3} sourceCollection_on_local_cluster targetCollection_on_targetZkHost1 2

Turn off CDCR for only selected target clusters

2019-08-13 Thread Arnold Bronley
Hi, Is there a way to turn off the CDCR for only selected target clusters. Say, I have a configuration like following. I have 3 target clusters targetZkHost1, targetZkHost2 and targetZkHost3. Is it possible to turn off the CDCR for targetZkHost2 and targetZkHost3 but keep it on for targetZkHost1

Re: [CAUTION] Re: [CAUTION] Re: CDCR Queues API invocation with CloudSolrclient

2019-07-25 Thread Natarajan, Rajeswari
I tried Shawn's suggestion to use SolrQuery Object instead of QT , still it is the same issue. Regards, Rajeswari On 7/24/19, 4:54 PM, "Natarajan, Rajeswari" wrote: Please look at the below test which tests CDCR OPS Api. This has "BadApple" annotation (me

Re: [CAUTION] Re: CDCR Queues API invocation with CloudSolrclient

2019-07-24 Thread Natarajan, Rajeswari
Please look at the below test which tests CDCR OPS Api. This has "BadApple" annotation (meaning the test fails intermittently) https://github.com/apache/lucene-solr/blob/master/solr/core/src/test/org/apache/solr/cloud/cdcr/CdcrOpsAndBoundariesTest.java#L73 This also is because of

Re: CDCR Queues API invocation with CloudSolrclient

2019-07-24 Thread Natarajan, Rajeswari
quot; , sometimes not. > protected static QueryResponse getCdcrQueue(CloudSolrClient client) throws SolrServerException, IOException { > ModifiableSolrParams params = new ModifiableSolrParams(); > params.set(CommonParams.QT, "/cdcr"); > pa

Re: CDCR Queues API invocation with CloudSolrclient

2019-07-24 Thread Natarajan, Rajeswari
ModifiableSolrParams(); > params.set(CommonParams.QT, "/cdcr"); > params.set(CommonParams.ACTION, CdcrParams.QUEUES); > return client.query(params); >} Side note: Setting the handler path with the qt parameter was deprecated i

Re: CDCR Queues API invocation with CloudSolrclient

2019-07-24 Thread Shawn Heisey
ams params = new ModifiableSolrParams(); params.set(CommonParams.QT, "/cdcr"); params.set(CommonParams.ACTION, CdcrParams.QUEUES); return client.query(params); } Side note: Setting the handler path with the qt parameter was deprecated in Solr 3.6, which was released s

CDCR Queues API invocation with CloudSolrclient

2019-07-24 Thread Natarajan, Rajeswari
params.set(CommonParams.QT, "/cdcr"); params.set(CommonParams.ACTION, CdcrParams.QUEUES); return client.query(params); } Invoking http://:/solr//cdcr?action=QUEUES has the same issue But if invoked as http://:/solr//cdcr?action=QUEUES always gets the " lastTimestamp" value.

Re: [CAUTION] CDCR Monitoring - To figure out the latency between source and target replication delay

2019-06-18 Thread Natarajan, Rajeswari
I see below for CDCR Queues API Documentation The output is composed of a list “queues” which contains a list of (ZooKeeper) Target hosts, themselves containing a list of Target collections. For each collection, the current size of the queue and the timestamp of the last update operation

Re: bi-directional CDCR

2019-06-18 Thread Natarajan, Rajeswari
We are using bidirectional CDCR with solr 7.6 and it works for us. Did you look at the logs to see if there are any errors. "Both Cluster 1 and Cluster 2 can act as Source and Target at any given point of time but a cluster cannot be both Source and Target at the same time."

bi-directional CDCR

2019-06-11 Thread Susheel Kumar
in this page doesn't make cdcr works. No data flows from cluster 1 to cluster 2. The Solr 7.7.1. Is there something missing. https://lucene.apache.org/solr/guide/7_7/cdcr-config.html#bi-directional-updates

CDCR Monitoring

2019-05-30 Thread Natarajan, Rajeswari
Hi, Is there a way to monitor the replication delay between Primary/Secondary Cluster for CDCR and raise alerts ,if it exceeds above some threshold. I see below API’s for monitoring. · core/cdcr?action=QUEUES: Fetches statistics about the queue<https://lucene.apache.org/solr/guide/7_6/c

Re: [CDCR]Unable to locate core

2019-05-19 Thread Natarajan, Rajeswari
Thanks Amrith. Created a bug https://issues.apache.org/jira/browse/SOLR-13481 Regards, Rajeswari On 5/19/19, 3:44 PM, "Amrit Sarkar" wrote: Sounds legit to me. Can you create a Jira and list down the problem statement and design solution there. I am confident it will attract

Re: [CDCR]Unable to locate core

2019-05-19 Thread Amrit Sarkar
Sounds legit to me. Can you create a Jira and list down the problem statement and design solution there. I am confident it will attract committers' attention and they can review the design and provide feedback. Amrit Sarkar Search Engineer Lucidworks, Inc. 415-589-9269 www.lucidworks.com Twitter

Re: [CDCR]Unable to locate core

2019-05-19 Thread Natarajan, Rajeswari
Thanks Amrith for creating a patch. But the code in the LBHttpSolrClient.java needs to be fixed too, if the for loop to work as intended. Regards Rajeswari public Rsp request(Req req) throws SolrServerException, IOException { Rsp rsp = new Rsp(); Exception ex = null; boolean

Re: CDCR one source multiple targets

2019-05-19 Thread Amrit Sarkar
Thanks, Arnold, Is the documentation not clear with the manner multiple CDCR targets can be configured? Amrit Sarkar Search Engineer Lucidworks, Inc. 415-589-9269 www.lucidworks.com Twitter http://twitter.com/lucidworks LinkedIn: https://www.linkedin.com/in/sarkaramrit2 Medium: https

Re: CDCR - shards not in sync

2019-05-19 Thread Amrit Sarkar
I have tried stopping and starting CDCR number of times but it > has not helped. > From what i have noticed there is always a shard that is slower than > others. > > Solr version: 7.7.0 > CDCR config > > > 2 > 10 > 4500 &g

[CDCR]Unable to locate core

2019-05-19 Thread Amrit Sarkar
> > Thanks Natrajan, > > Solid analysis and I saw the issue being reported by multiple users in > past few months and unfortunately I baked an incomplete code. > > I think the correct way of solving this issue is to identify the correct > base-url for the respective core we need to trigger

Re: [CDCR]Unable to locate core

2019-05-19 Thread Natarajan, Rajeswari
s not done. To solve this issue , there should be catch around doRequest and then the second time it will re-try the correct request. But in case there are multiple live servers, the request might timeout also. This needs to be fixed to make CDCR bootstrap work reliable. If not sometimes it will

Re: [CDCR]Unable to locate core

2019-05-19 Thread Natarajan, Rajeswari
Hi We are using solr 7.6 and trying out bidirectional CDCR and I also hit this issue. Stacktrace INFO (cdcr-bootstrap-status-17-thread-1) [ ] o.a.s.h.CdcrReplicatorManager CDCR bootstrap successful in 3 seconds

Re: [CDCR]Unable to locate core

2019-05-15 Thread Natarajan, Rajeswari
I am also facing this issue. Any resolution found on this issue, Please update. Thanks On 2/7/19, 10:42 AM, "Tim" wrote: So it looks like I'm having an issue with this fix: https://issues.apache.org/jira/browse/SOLR-11724 So I've messed around with this for a while and every

Re: Seeking advice on SolrCloud production architecture with CDCR

2019-05-14 Thread Shawn Heisey
loses quorum. When ZK loses quorum, SolrCloud loses the ability to react to failures and goes read-only. You mentioned CDCR. This involves two completely separate SolrCloud clusters -- a full ZK ensemble in each location. So you would have 3 ZK servers and at least two Solr servers in one

Seeking advice on SolrCloud production architecture with CDCR

2019-05-14 Thread Cody Burleson
in another data center: US and Europe. For this, the Cross Data Center Replication (CDCR; bi-directional) seems appropriate, but I’m not confident. Also, for the best fault tolerance and high-availability, I’m not real sure how to layout my Zookeper nodes and my Solr instances/shards/replicas physically

CDCR - shards not in sync

2019-04-15 Thread Jay Potharaju
Hi, I have a collection with 8 shards. 6 out of the shards are in sync but the other 2 are lagging behind by more than 10 plus hours. The tlog is only 0.5 GB in size. I have tried stopping and starting CDCR number of times but it has not helped. >From what i have noticed there is always a sh

Re: CDCR one source multiple targets

2019-04-10 Thread Arnold Bronley
This had a very simple solution if anybody else is wondering about the same issue.I had to define separate replica elements inside cdcr. Following is an example. target1:2181 techproducts techproducts target2:2181 techproducts techproducts 8 1000 128 1000disabled On Thu, Mar 21

Re: CDCR issues

2019-03-24 Thread Gus Heck
b151434#diff-e54b251d166135a1afb7938cfe152bb5 > That is related to this JDK bug > https://bugs.openjdk.java.net/browse/JDK-8129861. > > > Thanks > Jay Potharaju > > > > On Thu, Mar 21, 2019 at 10:20 PM Jay Potharaju > wrote: > > > Hi, > > I just enabled CDCR for one co

Re: CDCR issues

2019-03-22 Thread Jay Potharaju
at 10:20 PM Jay Potharaju wrote: > Hi, > I just enabled CDCR for one collection. I am seeing high CPU usage and > the high number of tlog files and increasing. > The collection does not have lot of data , just started reindexing of > data. > . > Solr 7.7.0 , implicit shard

CDCR issues

2019-03-21 Thread Jay Potharaju
Hi, I just enabled CDCR for one collection. I am seeing high CPU usage and the high number of tlog files and increasing. The collection does not have lot of data , just started reindexing of data. . Solr 7.7.0 , implicit sharding 8 shards I have enabled buffer on source side and disabled buffer

Re: CDCR one source multiple targets

2019-03-21 Thread Arnold Bronley
I see a similar question asked but no answers there too. http://lucene.472066.n3.nabble.com/CDCR-Replication-from-one-source-to-multiple-targets-td4308717.html OP there is using multiple cdcr request handlers but in my case I am using multiple zkhost strings. It will be pretty limiting if we

CDCR one source multiple targets

2019-03-20 Thread Arnold Bronley
Hi, is it possible to use CDCR with one source SolrCloud cluster and multiple target SolrCloud clusters? I tried to edit the zkHost setting in source cluster's solrconfig file by adding multiple comma separated values for target zkhosts for multuple target clusters. But the CDCR replication

Re: Bidirectional CDCR not working

2019-03-14 Thread Arnold Bronley
gard if you have already considered that in your configuration. > I had a lot of issues trying to figure out the issue when I realized that > it was a documentation error. > > Thanks > Nishant > > > On Thu, Mar 14, 2019, 2:54 PM Arnold Bronley wrote: > > > Configurati

Re: Bidirectional CDCR not working

2019-03-14 Thread Nish Karve
that in your configuration. I had a lot of issues trying to figure out the issue when I realized that it was a documentation error. Thanks Nishant On Thu, Mar 14, 2019, 2:54 PM Arnold Bronley Configuration is almost identical for both clusters in terms of cdcr except > for zkHost parame

Re: Bidirectional CDCR not working

2019-03-14 Thread Arnold Bronley
Configuration is almost identical for both clusters in terms of cdcr except for zkHost parameter configuration. On Thu, Mar 14, 2019 at 3:45 PM Arnold Bronley wrote: > Exactly. I have it defined in both clusters. I am following the > instructions from here . > https://lucene.apache

Re: Bidirectional CDCR not working

2019-03-14 Thread Arnold Bronley
Exactly. I have it defined in both clusters. I am following the instructions from here . https://lucene.apache.org/solr/guide/7_7/cdcr-config.html#bi-directional-updates On Thu, Mar 14, 2019 at 3:40 PM Amrit Sarkar wrote: > Hi Arnold, > > You need "cdcr-processor-chai

Re: Bidirectional CDCR not working

2019-03-14 Thread Amrit Sarkar
Hi Arnold, You need "cdcr-processor-chain" definitions in solrconfig.xml on both clusters' collections. Both clusters need to act as source and target. Amrit Sarkar Search Engineer Lucidworks, Inc. 415-589-9269 www.lucidworks.com Twitter http://twitter.com/lucidworks Linke

Bidirectional CDCR not working

2019-03-14 Thread Arnold Bronley
Hi, I used unidirectional CDCR in SolrCloud (7.7.1) without any issues. But after setting up bidirectional cdcr configuration, I am not able to index a document. Following is the error that I am getting: Async exception during distributed update: Error from server at http://host1:8983/solr

Re: [CDCR]Unable to locate core

2019-02-07 Thread Tim
So it looks like I'm having an issue with this fix: https://issues.apache.org/jira/browse/SOLR-11724 So I've messed around with this for a while and every time the leader to leader replica portion works fine. But the Recovery portion (implemented as part of the fix above) fails. I've run a few

RE: [EXTERNAL] Re: [CDCR]Unable to locate core

2019-02-02 Thread Timothy Springsteen
to target leader is successful but the target leader is then unsuccessful in replicating to its followers. The "unable to locate core" message is originally coming from the target cluster. Here are the logs being generated from the source for reference: 2019-02-02 20:10:19.551 IN

Re: [CDCR]Unable to locate core

2019-02-02 Thread Tim
to target leader is successful but the target leader is then unsuccessful in replicating to its followers. The "unable to locate core" message is originally coming from the target cluster. *Here are the logs being generated from the source for reference:* 2019-02-02 20:10:19.551 INFO (cdcr

Re: [CDCR]Unable to locate core

2019-02-02 Thread Erick Erickson
CDCR does _not_ replicate to followers, it is a leader<->leader replication of the raw document. Once the document has been forwarded to the target's leader, then the leader on the target system should forward it to followers on that system just like any other update. The Solr JIRA is un

Re: [CDCR]Unable to locate core

2019-02-01 Thread Tim
After some more investigation it seems that we're running into the same bug found here <https://issues.apache.org/jira/browse/SOLR-11724> . However if my understanding is correct that bug in 7.3 was patched out. Unfortunately we're running into the same behavior in 7.5 CDCR is repli

[CDCR]Unable to locate core

2019-01-30 Thread Tim
I'm trying to setup CDCR but I'm running into an issue where one or two shards/replicas will not be replicated but the rest will out of the six cores. The only error that appears in the logs is: "Unable to locate core". Occasionally restarting the instance will fix this but then

Re: CDCR "all" collections

2019-01-24 Thread Erick Erickson
be insuring nothing bad happens if, for instance, the target collection never got created, making sure the tlogs didn't grow, that kind of thing. Best, Erick On Thu, Jan 24, 2019 at 3:51 AM Bram Van Dam wrote: > > Hey folks, > > Is there any way to set up CDCR for *all* collections, i

CDCR "all" collections

2019-01-24 Thread Bram Van Dam
Hey folks, Is there any way to set up CDCR for *all* collections, including any newly created ones? Having to modify the solrconfig in ZK every time a collection is added is a bit of a pain, especially because I'm assuming it requires a restart to activate the config? Basically if I have DC Src

CDCR Replication sensitive to network problems

2018-12-07 Thread Webster Homer
We are using Solr 7.2. We have two solrclouds that are hosted on Google clouds. These are targets for an on Prem solr cloud where we run our ETL loads and have CDCR replicate it to the Google clouds. This mostly works pretty well. However, networks can fail. When the network has a brief outage

Re: Negative CDCR Queue Size?

2018-11-09 Thread Amrit Sarkar
gs or probably check at respective source and target the CDCR is configured and was running ok? without any manual intervention? Also, you mentioned there are a number of intermittent issues with CDCR, I see you have reported few Jiras. I will be grateful if you can report the rest? Code: > for (Cd

RE: Negative CDCR Queue Size?

2018-11-06 Thread Webster Homer
I'm sorry I should have included that. We are running Solr 7.2. We use CDCR for almost all of our collections. We have experienced several intermittent problems with CDCR, this one seems to be new, at least I hadn't seen it before -Original Message- From: Erick Erickson

Re: Negative CDCR Queue Size?

2018-11-06 Thread Erick Erickson
What version of Solr? CDCR has changed quite a bit in the 7x code line so it's important to know the version. On Tue, Nov 6, 2018 at 10:32 AM Webster Homer wrote: > > Several times I have noticed that the CDCR action=QUEUES will return a > negative queueSize. When this happen

Negative CDCR Queue Size?

2018-11-06 Thread Webster Homer
Several times I have noticed that the CDCR action=QUEUES will return a negative queueSize. When this happens we seem to be missing data in the target collection. How can this happen? What does a negative Queue size mean? The timestamp is an empty string. We have two targets for a source. One

Re: SolrCloud CDCR with 3+ DCs

2018-09-07 Thread Amrit Sarkar
Yeah, I am not sure about how the Authentication band aid feature will work, the mentioned stackoverflow link. It is about time we include basic authentication support in CDCR. On Thu, 6 Sep 2018, 8:41 pm cdatta, wrote: > Hi Amrit, Thanks for your response. > > We wiped out our

Solr CDCR replication not working

2018-09-07 Thread Amrit Sarkar
Basic Authentication in clusters is not supported as of today in CDCR. On Fri, 7 Sep 2018, 4:53 pm Mrityunjaya Pathak, wrote: > I have setup two solr cloud instances in two different Datacenters Target > solr cloud machine is copy of source machine with basicAuth enabled on > them. I

Solr CDCR replication not working

2018-09-07 Thread Mrityunjaya Pathak
:65536} ${solr.autoCommit.maxTime:15000} false ${solr.autoSoftCommit.maxTime:-1} ... Target Config Changes ... disabled cdcr-proc-chain ${solr.ulog.dir:} ${solr.ulog.numVersionBuckets:65536

  1   2   3   4   >