On 18/02/2012, at 5:18 AM, gmoniey wrote: > Thanks Pat, > > In regards to speed, is there anything I can do, or is it just a > limitation of sphinx? Also, any idea why the db is getting hit so > hard. I haven't got a chance to look into the logs yet to see if its > running any slows queries, but I would imagine its just reads. > Whenever the indexer is running, I see a lot of DB timeouts from other > services, so I can't help but feel that something is up.
Generally it should be just reads - the only case where it's not is when you're using deltas, and then it's just a single UPDATE and the rest are SELECTs. I'd recommend looking at the generated query and trying to determine why it's slow. Also worth checking that you have database indices on any foreign keys being used by joins in the generated SQL. > Also, in regards to changing the structure of my indexes, so I need to > run rebuild, or can I just configure and restart (i.e. must I reindex > before restarting with a new config file)? You could configure and index, and *then* restart, but I've heard that's not quite reliable. So, I'd recommend rebuild when there's a structure change - treat it the same as you would a database change, put up a temporary maintenance page. It shouldn't be a common occurrence. Cheers -- Pat > > Thanks. > > On Feb 17, 3:21 am, "Pat Allan" <[email protected]> wrote: >> It's best to use the ts:index task in cron, because all that does is reindex >> the data. ts:rebuild stops Sphinx, reindexes, and *then* restarts Sphinx - >> and you really don't need to restart Sphinx except when the structure of >> your indices change (you've added an index to another model, or added a new >> field or attribute to an existing model, a new config setting, etc). >> >> So, I'd recommend switching to ts:index - although it won't be much >> different in speed (starting/stopping Sphinx won't take much time), but >> it'll mean people can still search while the indexing is happening. >> >> -- >> Pat >> >> On 17/02/2012, at 7:52 PM, gmoniey wrote: >> >> >> >> >> >> >> >>> For the past few years, I have had a cron job that wakes up around >>> 4am, and runs the ts:rebuild rake task. Unfortunately, the size of all >>> of my indexes has grown significantly, and this job takes about 40-60 >>> minutes, and the load on the database is pretty significant. I'm >>> indexing roughly 2-3 million records (some of which have pretty >>> detailed indexes defined). >> >>> I've read on some blogs the index/reindex is significantly faster than >>> rebuild, but haven't been able to find any formal documentation >>> mentioning that. >> >>> Do I have any other options? >> >>> Thanks. >> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Thinking Sphinx" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group >>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
