True story... Have you tried to stop the process, reindex, and start it every hour on the cron job? I've had a few quirky issues with ts before, and as long as I explicitly stop it then start it up again, it seems to work fine...
I dunno... Beau On Oct 15, 2010, at 2:03 PM, claytonlz wrote: > ts:rebuild works great, but it takes too long with over 100,000 > records. It should be possible to have a consistent index that is only > rebuilt once per day. > > On Oct 15, 11:38 am, Beau Frusetta <[email protected]> wrote: >> Have you tried "ts:rebuild" instead of "ts:reindex"? >> >> On Oct 15, 2010, at 11:34 AM, claytonlz wrote: >> >> >> >>> Here's the output of that ls: >> >>> -rw-r--r-- 1 deploy deploy 171864 Oct 15 13:00 post_core.spa >>> -rw-r--r-- 1 deploy deploy 234975 Oct 15 13:00 post_core.spd >>> -rw-r--r-- 1 deploy deploy 571 Oct 15 13:00 post_core.sph >>> -rw-r--r-- 1 deploy deploy 13421 Oct 15 13:00 post_core.spi >>> -rw-r--r-- 1 deploy deploy 0 Oct 15 13:00 post_core.spk >>> -rw------- 1 deploy deploy 0 Oct 15 13:00 post_core.spl >>> -rw-r--r-- 1 deploy deploy 0 Oct 15 13:00 post_core.spm >>> -rw-r--r-- 1 deploy deploy 273326 Oct 15 13:00 post_core.spp >>> -rw-r--r-- 1 deploy deploy 0 Oct 15 13:00 post_delta.spa >>> -rw-r--r-- 1 deploy deploy 1 Oct 15 13:00 post_delta.spd >>> -rw-r--r-- 1 deploy deploy 571 Oct 15 13:00 post_delta.sph >>> -rw-r--r-- 1 deploy deploy 1 Oct 15 13:00 post_delta.spi >>> -rw-r--r-- 1 deploy deploy 0 Oct 15 13:00 post_delta.spk >>> -rw------- 1 deploy deploy 0 Oct 15 13:00 post_delta.spl >>> -rw-r--r-- 1 deploy deploy 0 Oct 15 13:00 post_delta.spm >>> -rw-r--r-- 1 deploy deploy 1 Oct 15 13:00 post_delta.spp >> >>> We are storing the indexes in a shared path above the current release. >>> That path is symlinked to the current release upon deploy. >> >>> We're basically following your deployment strategy, ours is something >>> like: >> >>> ...regular cap deploy steps... >> >>> 1. deploy:cleanup >>> 2. ts:conf >>> 3. ts:reindex >> >>> So we're not stopping and starting sphinx before we symlink and >>> configure. Could that be the problem? >> >>> Thanks, >>> Clayton >> >>> On Oct 14, 5:20 pm, Pat Allan <[email protected]> wrote: >>>> Hi Clayton >> >>>> I'm not quite sure what's causing this... as the error is pointing to the >>>> core index, not the delta (my first guess was the regular delta updates >>>> were conflicting with the full index - perhaps that's still the case, but >>>> not so convinced at this point). >> >>>> What's the output of the following on your server: >> >>>> ls -alh /home/deploy/apps/my_apps/current/db/sphinx/production/ >> >>>> Also: I recommend storing production indexes outside of the releases >>>> directory - have a read through of the following documentation: >> >>>> http://freelancing-god.github.com/ts/en/deployment.html >> >>>> Cheers >> >>>> -- >>>> Pat >> >>>> On 14/10/2010, at 4:34 AM, claytonlz wrote: >> >>>>> I've got a sphinx installation using thinking sphinx that has ~100,000 >>>>> records distributed across a number of models. One of my models is >>>>> frequently updated causing there to be numerous delta indexes created. >>>>> When ThinkingSphinx runs ts:reindex once per hour, it almost always >>>>> results in this error for my frequently updated model: >> >>>>> WARNING: rotating index 'post_core': cur to old rename failed: rename / >>>>> home/deploy/apps/my_app/releases/20101008210428/db/sphinx/production/ >>>>> post_core.spl to /home/deploy/apps/my_app/releases/20101008210428/db/ >>>>> sphinx/production/post_core.old.spl failed: No such file or directory >> >>>>> This error occurs again and again over the course of 30 minutes with >>>>> multiple messages like this in the log file. It's only when I >>>>> completely rebuild in the index that this problem goes away. >> >>>>> What can I do to prevent this from happening? As it is right now, >>>>> things go smoothly for a while and then the indexes fail to rotate and >>>>> the search functionality becomes useless because of all of the missing >>>>> records. >> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "Thinking Sphinx" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]. >>>>> For more options, visit this group >>>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Thinking Sphinx" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group >>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
