Hello, Had to leave the office so didn't get a chance to reply. Nothing in the logs. Just ran one through from the ingest tool.
Same results full copy of the index. Is it something to do with: server.commit(); server.optimize(); I call this at the end of the ingestion. Would optimize then work across the whole index? Thanks Ben -----Original Message----- From: Tomás Fernández Löbbe [mailto:tomasflo...@gmail.com] Sent: 23 March 2012 15:10 To: solr-user@lucene.apache.org Subject: Re: Simple Slave Replication Question Also, what happens if, instead of adding the 40K docs you add just one and commit? 2012/3/23 Tomás Fernández Löbbe <tomasflo...@gmail.com> > Have you changed the mergeFactor or are you using 10 as in the example > solrconfig? > > What do you see in the slave's log during replication? Do you see any > line like "Skipping download for..."? > > > On Fri, Mar 23, 2012 at 11:57 AM, Ben McCarthy < > ben.mccar...@tradermedia.co.uk> wrote: > >> I just have a index directory. >> >> I push the documents through with a change to a field. Im using >> SOLRJ to do this. Im using the guide from the wiki to setup the >> replication. When the feed of updates to the master finishes I call >> a commit again using SOLRJ. I then have a poll period of 5 minutes >> from the slave. When it kicks in I see a new version of the index >> and then it copys the full 5gb index. >> >> Thanks >> Ben >> >> -----Original Message----- >> From: Tomás Fernández Löbbe [mailto:tomasflo...@gmail.com] >> Sent: 23 March 2012 14:29 >> To: solr-user@lucene.apache.org >> Subject: Re: Simple Slave Replication Question >> >> Hi Ben, only new segments are replicated from master to slave. In a >> situation where all the segments are new, this will cause the index >> to be fully replicated, but this rarely happen with incremental >> updates. It can also happen if the slave Solr assumes it has an "invalid" >> index. >> Are you committing or optimizing on the slaves? After replication, >> the index directory on the slaves is called "index" or "index.<timestamp>"? >> >> Tomás >> >> On Fri, Mar 23, 2012 at 11:18 AM, Ben McCarthy < >> ben.mccar...@tradermedia.co.uk> wrote: >> >> > So do you just simpy address this with big nic and network pipes. >> > >> > -----Original Message----- >> > From: Martin Koch [mailto:m...@issuu.com] >> > Sent: 23 March 2012 14:07 >> > To: solr-user@lucene.apache.org >> > Subject: Re: Simple Slave Replication Question >> > >> > I guess this would depend on network bandwidth, but we move around >> > 150G/hour when hooking up a new slave to the master. >> > >> > /Martin >> > >> > On Fri, Mar 23, 2012 at 12:33 PM, Ben McCarthy < >> > ben.mccar...@tradermedia.co.uk> wrote: >> > >> > > Hello, >> > > >> > > Im looking at the replication from a master to a number of slaves. >> > > I have configured it and it appears to be working. When updating >> > > 40K records on the master is it standard to always copy over the >> > > full index, currently 5gb in size. If this is standard what do >> > > people do who have massive 200gb indexs, does it not take a while >> > > to bring the >> > slaves inline with the master? >> > > >> > > Thanks >> > > Ben >> > > >> > > ________________________________________ >> > > >> > > >> > > This e-mail is sent on behalf of Trader Media Group Limited, >> > > Registered >> > > Office: Auto Trader House, Cutbush Park Industrial Estate, >> > > Danehill, Lower Earley, Reading, Berkshire, RG6 4UT(Registered in >> > > England No. >> > 4768833). >> > > This email and any files transmitted with it are confidential and >> > > may be legally privileged, and intended solely for the use of the >> > > individual or entity to whom they are addressed. If you have >> > > received this email in error please notify the sender. This email >> > > message has been swept for the presence of computer viruses. >> > > >> > > >> > >> > ________________________________________ >> > >> > >> > This e-mail is sent on behalf of Trader Media Group Limited, >> > Registered >> > Office: Auto Trader House, Cutbush Park Industrial Estate, >> > Danehill, Lower Earley, Reading, Berkshire, RG6 4UT(Registered in England >> > No. >> 4768833). >> > This email and any files transmitted with it are confidential and >> > may be legally privileged, and intended solely for the use of the >> > individual or entity to whom they are addressed. If you have >> > received this email in error please notify the sender. This email >> > message has been swept for the presence of computer viruses. >> > >> > >> >> ________________________________________ >> >> >> This e-mail is sent on behalf of Trader Media Group Limited, >> Registered >> Office: Auto Trader House, Cutbush Park Industrial Estate, Danehill, >> Lower Earley, Reading, Berkshire, RG6 4UT(Registered in England No. 4768833). >> This email and any files transmitted with it are confidential and may >> be legally privileged, and intended solely for the use of the >> individual or entity to whom they are addressed. If you have received >> this email in error please notify the sender. This email message has >> been swept for the presence of computer viruses. >> >> > ________________________________________ This e-mail is sent on behalf of Trader Media Group Limited, Registered Office: Auto Trader House, Cutbush Park Industrial Estate, Danehill, Lower Earley, Reading, Berkshire, RG6 4UT(Registered in England No. 4768833). This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
