On 7/13/2015 10:02 PM, Erick Erickson wrote: > Uggghh. Not persistence again.... > > I'll stay tuned..
While gathering every possible detail I could during the index rebuild, I came across what I believe is the cause of my problems. What I found is no surprise: PEBCAK. All of the core.properties files were owned by root, but I run Solr with a "solr" user for security purposes -- core swaps were unable to update the files. I know how this mistake happened, but I'm amazed that it hasn't been an obvious problem before now. I first set up these machines (handling an index for a new customer) over a year ago. I fixed the permissions, the rebuild finished, and I restarted Solr on both machines. Everything persisted beautifully. My dev server (which sees reboots and restarts *far* more frequently than the production machines) has not been having this problem, and it turns out that all the file permissions were correct on that machine. I am very glad when a problem like this turns out to not be a bug, but it IS embarrassing. If anyone needs any proof of Solr's stability, I can show you servers that run a Solr JVM without interruption for weeks or months at a time. They would probably go longer, but customers keep wanting changes, plus I like to make sure the operating system stays current with security patches. Thanks, Shawn
