ExternalFileField management strategy with SolrCloud

2018-04-26 Thread Tom Peters
Is there a recommended way of managing external files with SolrCloud. At first glance it appears that I would need to manually manage the placement of the external_.txt file in each shard's data directory. Is there a better way of managing this (Solr API, interface, etc?) This message and any

Re: CDCR Bootstrap

2018-04-26 Thread Tom Peters
I'm not sure under what conditions it will be automatically triggered, but if you manually wanted to trigger a CDCR Bootstrap you need to issue the following query to the leader in your target data center. /solr//cdcr?action=BOOTSTRAP&masterUrl= The masterUrl will look something like (change th

Re: Does CDCR Bootstrap sync leaves replica's out of sync

2018-04-16 Thread Tom Peters
There are two ways I've gotten around this issue: 1. Add replicas in the target data center after CDCR bootstrapping has completed. -or- 2. After the bootstrapping has completed, restart the replica nodes one-at-time in the target data center (restart, wait for replica to catch up, then restar

Re: CDCR performance issues

2018-03-23 Thread Tom Peters
olr02-a.svcs.opal.synacor.com:8080 Connection: Keep-Alive {params={cdcr.update=,_stateVer_=synacor:199},delByQ=null,docsMap=[MapEntry[SolrInputDocument(fields: [solr_id=Mytest, _version_=1595749902698151936]):null]]} > > > > Amrit Sarkar > Search Engineer > Lucidworks, Inc. > 4

Re: CDCR performance issues

2018-03-12 Thread Tom Peters
I'm also having issue with replicas in the target data center. It will go from recovering to down. And when one of my replicas go to down in the target data center, CDCR will no longer send updates from the source to the target. > On Mar 12, 2018, at 9:24 AM, Tom Peters wrote: > &g

Re: CDCR performance issues

2018-03-12 Thread Tom Peters
ain? Or is there something else we can do? Thanks. > On Mar 9, 2018, at 3:59 PM, Tom Peters wrote: > > Thanks. This was helpful. I did some tcpdumps and I'm noticing that the > requests to the target data center are not batched in any way. Each update > comes in as an i

Re: CDCR performance issues

2018-03-09 Thread Tom Peters
eckout paper > "Latency performance of SOAP Implementations". Same distribution of skills > - I knew TCP well, but Apache Axis 1.1 not so well. I still improved > response time of Apache Axis 1.1 by 250ms per call with 1-line of code. > > -Original Message

Re: CDCR performance issues

2018-03-08 Thread Tom Peters
number of updates (3805 over two hours). Thanks. > On Mar 7, 2018, at 6:19 PM, Tom Peters wrote: > > I'm having issues with the target collection staying up-to-date with indexing > from the source collection using CDCR. > > This is what I'm getting back in terms

CDCR performance issues

2018-03-07 Thread Tom Peters
I'm having issues with the target collection staying up-to-date with indexing from the source collection using CDCR. This is what I'm getting back in terms of OPS: curl -s 'solr2-a:8080/solr/mycollection/cdcr?action=OPS' | jq . { "responseHeader": { "status": 0, "Q

Re: Issues with CDCR in Solr 7.1

2018-03-05 Thread Tom Peters
You can ignore this. I think I found the issue (I was missing a block of XML in the source ocnfig). I'm going to monitor it over the next day and see if it was resolved. > On Mar 5, 2018, at 4:29 PM, Tom Peters wrote: > > I'm trying to get Solr CDCR setup in Solr 7.1 an

Issues with CDCR in Solr 7.1

2018-03-05 Thread Tom Peters
I'm trying to get Solr CDCR setup in Solr 7.1 and I'm having issues post-bootstrap. I have about 5,572,933 documents in the source cluster (index size is 3.77 GB). I'm enabling CDCR in the following manner: 1. Delete the existing cluster in the target data center admin/collections?actio

Re: /var/solr/data has lots of index* directories

2018-03-05 Thread Tom Peters
perties. The directory name mentioned in > that properties file is the one being used actively. The rest are old > directories that should be cleaned up on Solr restart but you can delete > them yourself without any issues. > > On Mon, Mar 5, 2018 at 11:43 PM, Tom Peters wrote: >

/var/solr/data has lots of index* directories

2018-03-05 Thread Tom Peters
While trying to debug an issue with CDCR, I noticed that the /var/solr/data directories on my source cluster have wildly different sizes. % for i in solr2-{a..e}; do echo -n "$i: "; ssh -A $i du -sh /var/solr/data; done solr2-a: 9.5G /var/solr/data solr2-b: 29G/var/solr/data solr2-

Re: Indexing timeout issues with SolrCloud 7.1

2018-03-01 Thread Tom Peters
oel wrote: >> From the error list, i can see multiple errors: >> >> 1. Failure to recover replica >> 2. Peer sync error >> 3. Failure to download file >> >> On 24 Feb 2018 03:10, "Tom Peters" wrote: >> >> I included the last 25 lin

Re: Indexing timeout issues with SolrCloud 7.1

2018-02-23 Thread Tom Peters
x27;solr-2d' > > On 23 Feb 2018 09:42, "Tom Peters" wrote: > > I'm trying to debug why indexing in SolrCloud 7.1 is having so many issues. > It will hang most of the time, and timeout the rest. > > Here's an example: > >time curl -s 

Indexing timeout issues with SolrCloud 7.1

2018-02-22 Thread Tom Peters
I'm trying to debug why indexing in SolrCloud 7.1 is having so many issues. It will hang most of the time, and timeout the rest. Here's an example: time curl -s 'myhost:8080/solr/mycollection/update/json/docs' -d '{"solr_id":"test_001", "data_type":"test"}'|jq . { "responseHeader"

Re: Issue with CDCR bootstrapping in Solr 7.1

2017-12-04 Thread Tom Peters
t2 > Medium: https://medium.com/@sarkaramrit2 > > On Fri, Dec 1, 2017 at 4:49 AM, Tom Peters wrote: > >> Hi Amrit, I tried issuing hard commits to the various nodes in the target >> cluster and it does not appear to cause the follower replicas to receive >> the initial

Re: Issue with CDCR bootstrapping in Solr 7.1

2017-11-30 Thread Tom Peters
inkedIn: https://www.linkedin.com/in/sarkaramrit2 > Medium: https://medium.com/@sarkaramrit2 > > On Thu, Nov 30, 2017 at 10:06 PM, Tom Peters wrote: > >> Hi Amrit, >> >> Starting with more documents doesn't appear to have made a difference. >> This time I tried w

Re: Issue with CDCR bootstrapping in Solr 7.1

2017-11-30 Thread Tom Peters
;, which is default 100, unless you have modified in > solrconfig.xml. > > Looking forward to your analysis. > > Amrit Sarkar > Search Engineer > Lucidworks, Inc. > 415-589-9269 > www.lucidworks.com > Twitter http://twitter.com/lucidworks > LinkedIn: https://www.

Issue with CDCR bootstrapping in Solr 7.1

2017-11-30 Thread Tom Peters
I'm running into an issue with the initial CDCR bootstrapping of an existing index. In short, after turning on CDCR only the leader replica in the target data center will have the documents replicated and it will not exist in any of the follower replicas in the target data center. All subsequent