You don't need to start cdcr on target cluster. Other steps are exactly what I did. After disable buffer on both target and source, the tlog files are purged according to the specs.
-- Thank you Sean From: Patrick Hoeffel <patrick.hoef...@polarisalpha.com<mailto:patrick.hoef...@polarisalpha.com>> Date: Friday, Jul 28, 2017, 4:01 PM To: solr-user@lucene.apache.org <solr-user@lucene.apache.org<mailto:solr-user@lucene.apache.org>> Cc: jmy...@wayfair.com <jmy...@wayfair.com<mailto:jmy...@wayfair.com>> Subject: [EXTERNAL] RE: CDCR - how to deal with the transaction log files Amrit, Problem solved! My biggest mistake was in my SOURCE-side configuration. The zkHost field needed the entire zkHost string, including the CHROOT indicator. I suppose that should have been obvious to me, but the examples only showed the IP Address of the target ZK, and I made a poor assumption. <!-- CORRECT CDCR CONFIG SECTION --> <requestHandler name="/cdcr" class="solr.CdcrRequestHandler"> <lst name="replica"> <str name="zkHost">10.161.0.7:2181,10.161.0.6:2181,10.161.0.5:2181/chroot/solr</str> <str name="source">ks_v1</str> <str name="target">ks_v1</str> </lst> <!-- MY ORIGINAL INCORRECT CDCR CONFIG SECTION --> <requestHandler name="/cdcr" class="solr.CdcrRequestHandler"> <lst name="replica"> <str name="zkHost">10.161.0.7:2181</str> <=== Problem was here. <str name="source">ks_v1</str> <str name="target">ks_v1</str> </lst> After that, I just made sure I did this: 1. Stop all Solr nodes at both SOURCE and TARGET. 2. $ rm -rf $SOLR_HOME/server/solr/collection_name/data/tlog/*.* 3. On the TARGET: a. $ collection/cdcr?action=DISABLEBUFFER b. $ collection/cdcr?action=START 4. On the Source: a. $ collection/cdcr?action=DISABLEBUFFER b. $ collection/cdcr?action=START At this point any existing data in the SOURCE collection started flowing into the TARGET collection, and it has remained congruent ever since. Thanks, Patrick Hoeffel
 Senior Software Engineer (Direct) 719-452-7371 (Mobile) 719-210-3706 patrick.hoef...@polarisalpha.com PolarisAlpha.com -----Original Message----- From: Amrit Sarkar [mailto:sarkaramr...@gmail.com] Sent: Friday, July 21, 2017 7:21 AM To: solr-user@lucene.apache.org Cc: jmy...@wayfair.com Subject: Re: CDCR - how to deal with the transaction log files Patrick, Yes! You created default UpdateLog which got written to a disk and then you changed it to CdcrUpdateLog in configs. I find no reason it would create a proper COLLECTIONCHECKPOINT on target tlog. One thing you can try before creating / starting from scratch is restarting source cluster nodes, the leaders of shard will try to create the same COLLECTIONCHECKPOINT, which may or may not be successful. Amrit Sarkar Search Engineer Lucidworks, Inc. 415-589-9269 www.lucidworks.com<http://www.lucidworks.com> Twitter http://twitter.com/lucidworks LinkedIn: https://www.linkedin.com/in/sarkaramrit2 On Fri, Jul 21, 2017 at 11:09 AM, Patrick Hoeffel < patrick.hoef...@polarisalpha.com> wrote: > I'm working on my first setup of CDCR, and I'm seeing the same "The > log reader for target collection {collection name} is not initialised" > as you saw. > > It looks like you're creating collections on a regular basis, but for > me, I create it one time and never again. I've been creating the > collection first from defaults and then applying the CDCR-aware > solrconfig changes afterward. It sounds like maybe I need to create > the configset in ZK first, then create the collections, first on the > Target and then on the Source, and I should be good? > > Thanks, > > Patrick Hoeffel > Senior Software Engineer > (Direct) 719-452-7371 > (Mobile) 719-210-3706 > patrick.hoef...@polarisalpha.com > PolarisAlpha.com > > > -----Original Message----- > From: jmyatt [mailto:jmy...@wayfair.com] > Sent: Wednesday, July 12, 2017 4:49 PM > To: solr-user@lucene.apache.org > Subject: Re: CDCR - how to deal with the transaction log files > > glad to hear you found your solution! I have been combing over this > post and others on this discussion board many times and have tried so > many tweaks to configuration, order of steps, etc, all with absolutely > no success in getting the Source cluster tlogs to delete. So > incredibly frustrating. If anyone has other pearls of wisdom I'd love some > advice. > Quick hits on what I've tried: > > - solrconfig exactly like Sean's (target and source respectively) > expect no autoSoftCommit > - I am also calling cdcr?action=DISABLEBUFFER (on source as well as on > target) explicitly before starting since the config setting of > defaultState=disabled doesn't seem to work > - when I create the collection on source first, I get the warning "The > log reader for target collection {collection name} is not > initialised". When I reverse the order (create the collection on > target first), no such warning > - tlogs replicate as expected, hard commits on both target and source > cause tlogs to rollover, etc - all of that works as expected > - action=QUEUES on source reflects the queueSize accurately. Also > *always* shows updateLogSynchronizer state as "stopped" > - action=LASTPROCESSEDVERSION on both source and target always seems > correct (I don't see the -1 that Sean mentioned). > - I'm creating new collections every time and running full data > imports that take 5-10 minutes. Again, all data replication, log > rollover, and autocommit activity seems to work as expected, and logs > on target are deleted. It's just those pesky source tlogs I can't get to > delete. > > > > -- > View this message in context: http://lucene.472066.n3. > nabble.com/CDCR-how-to-deal-with-the-transaction-log- > files-tp4345062p4345715.html > Sent from the Solr - User mailing list archive at Nabble.com. > Confidentiality Notice:: This email, including attachments, may include non-public, proprietary, confidential or legally privileged information. If you are not an intended recipient or an authorized agent of an intended recipient, you are hereby notified that any dissemination, distribution or copying of the information contained in or transmitted with this e-mail is unauthorized and strictly prohibited. If you have received this email in error, please notify the sender by replying to this message and permanently delete this e-mail, its attachments, and any copies of it immediately. You should not retain, copy or use this e-mail or any attachment for any purpose, nor disclose all or any part of the contents to any other person. Thank you.
