Hm…I was afraid of that… So, we have 9 application servers with 10-20 processes each. At high traffic times, we'd be seeing 181 processes for searchd? Won't that blow memory up like crazy? Is this normally something that everyone deals with?
Just curious…what are the benefits of persisted connection pools? As always, thanks for the quick response Pat! - Ngan On Saturday, August 10, 2013 at 10:11 PM, Pat Allan wrote: > Hi Ngan > > There's not really any documentation around the changes, I'm afraid… but what > you're seeing is a process per thread of your app, plus a master daemon > process, due to the persisted connection pool. > > All the logic for this connection pool can be found in > ThinkingSphinx::Connection: > https://github.com/pat/thinking-sphinx/blob/v2/lib/thinking_sphinx/connection.rb > > The HISTORY file has a list of all relevant changes though - here's the list > of what's changed between 2.0.14 and 2.1.0 - but this and some delta > refactoring are the biggest items: > > * Removed plugin support - Thinking Sphinx is now gem-only across all > branches. > * ThinkingSphinx::Version and the thinking_sphinx:version task have been > removed - it's a gem, it has a version number. > * Updating Riddle to 1.5.6 or newer. > * Requires ActiveRecord ~> 2.1 for TS 1.x releases (earlier versions were > considered unsupported a few releases ago). > * Allow custom Riddle controllers - useful for Flying Sphinx to take over > management of Sphinx daemon/indexing actions. > * Rejigged delta support to be generic, with local job classes that provide a > clean, simple interface for third-party libraries. > * Add hooks for anything that needs to happen before indexing (such as > clearing out existing delta jobs). > * Connection pool for all Sphinx client communication, with new connections > built if there's any connection-related (as opposed to syntax) issues. > * Multiple-field search conditions can be done with arrays of field names as > keys in the :conditions hash (Alex Dowad). > * Removed named capture in regular expressions to maintain MRI 1.8 support > (Michael Wintrant). > * Support new JDBC configuration style (Kyle Stevens). > > -- > Pat > > On 11/08/2013, at 2:39 PM, Ngan wrote: > > > Hi Pat, > > > > Thanks for the help. I tried upgrading to 2.1.0...and I'm noticing multiple > > instance of searched running now. Is that normal? Would you be able to > > point me to documentation of major changes with 2.1.0? > > > > Thanks, > > Ngan > > > > On Friday, August 9, 2013 7:57:51 AM UTC-7, Pat Allan wrote: > > Hi Ngan > > > > In 2.1.0 there's been some patches that deal with these kinds of errors - > > TS will now retry searches if an error crops up on the client connection > > (which is also persisted per thread in a connection pool, saving socket > > setup/teardown time). If an error crops up, a new connection is made, but > > if the error persists, it'll still get raised… > > > > Give 2.1.0 a spin, see if that helps matters. > > > > -- > > Pat > > > > On 09/08/2013, at 10:46 PM, Ngan wrote: > > > > > Forgot to add the trace: > > > > > > gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:438:in `block > > > in populate' > > > gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:606:in `call' > > > gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:606:in > > > `retry_on_stale_index' > > > gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:426:in > > > `populate' > > > gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:104:in `to_a' > > > > > > > > > > > > On Friday, August 9, 2013 5:45:55 AM UTC-7, Ngan wrote: > > > Hi, > > > > > > We reindex our entire index pretty often (once every 3 minutes) because > > > we have a pretty small data collection and we don't want to use delayed > > > delta. I notice however, that every once in a while, when we deploy our > > > application and it happens to be the same time the reindexing is about to > > > rotate, we'll get this error "no enabled local indexes to search" every > > > time we hit sphinx there afterwards. When this happens, we have to > > > restart our app so that it picks up the new indexes. We are reindexing > > > with rotate so the existing index should still be there and the rotation > > > should be seamless. Any ideas on why this happens? And if there's > > > anything to do about it? > > > > > > Thanks, > > > Ngan > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Thinking Sphinx" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [email protected] (http://googlegroups.com). > > > To post to this group, send email to [email protected] > > > (http://googlegroups.com). > > > Visit this group at http://groups.google.com/group/thinking-sphinx. > > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > > > > > > > > > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Thinking Sphinx" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [email protected] > > (mailto:[email protected]). > > To post to this group, send email to [email protected] > > (mailto:[email protected]). > > Visit this group at http://groups.google.com/group/thinking-sphinx. > > For more options, visit https://groups.google.com/groups/opt_out. > > > > > > > > > -- > You received this message because you are subscribed to a topic in the Google > Groups "Thinking Sphinx" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/thinking-sphinx/rPsXVX8wlHU/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected] > (mailto:[email protected]). > To post to this group, send email to [email protected] > (mailto:[email protected]). > Visit this group at http://groups.google.com/group/thinking-sphinx. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/thinking-sphinx. For more options, visit https://groups.google.com/groups/opt_out.
