Re: Swapping indexes on disk

2017-06-25 Thread Mike Lissner
Weirdly, this happened again today, deleting a brand new 300GB index that we created after last time, and which had been working for several days. This time, the index was deleted by our log rotation script, which restarted solr, so it's very easy to see what happened before and after the problem.

Re: Swapping indexes on disk

2017-06-20 Thread Mike Lissner
Thanks for the suggestions everybody. Some responses to Shawn's questions: > Does your solr.xml file contain core definitions, or is that information in a core.properties file in each instanceDir? Were using core.properties files. > How did you install Solr Solr is installed just by downloadin

Re: Swapping indexes on disk

2017-06-20 Thread Shawn Heisey
On 6/14/2017 12:26 PM, Mike Lissner wrote: > We are replacing a drive mounted at /old with one mounted at /new. Our > index currently lives on /old, and our plan was to: > > 1. Create a new index on /new > 2. Reindex from our database so that the new index on /new is properly > populated. > 3. Stop

Re: Swapping indexes on disk

2017-06-14 Thread Erick Erickson
Why not just use the replication API and fetchindex? See: https://cwiki.apache.org/confluence/display/solr/Index+Replication#IndexReplication-HTTPAPICommandsfortheReplicationHandler. It's not entirely obvious from the writeup, but you can specify masterUrl as part of the command &masterUrl=some_o

Re: Swapping indexes on disk

2017-06-14 Thread Mike Lissner
I figured Solr would have a native system built in, but since we don't use it already, I didn't want to learn all of its ins and outs just for this disk situation. Ditto, essentially, applies for the swapping strategy. We don't have a Solr expert, just me, a generalist, and sorting out these kinds

Re: Swapping indexes on disk

2017-06-14 Thread Chris Ulicny
Are you physically swapping the disks to introduce the new index? Or having both disks mounted at the same time? If the disks are simultaneously available, can you just swap the cores and then delete the core on the old disk? https://cwiki.apache.org/confluence/display/solr/CoreAdmin+API#CoreAdmin

Re: Swapping indexes on disk

2017-06-14 Thread David Hastings
I dont have an answer to why the folder got cleared, however i am wondering why you arent using basic replication to do this exact same thing, since solr will natively take care of all this for you with no interruption to the user and no stop/start routines etc. On Wed, Jun 14, 2017 at 2:26 PM, Mi

Swapping indexes on disk

2017-06-14 Thread Mike Lissner
We are replacing a drive mounted at /old with one mounted at /new. Our index currently lives on /old, and our plan was to: 1. Create a new index on /new 2. Reindex from our database so that the new index on /new is properly populated. 3. Stop solr. 4. Symlink /old to /new (Solr now looks for the i