Hey Pat, Just had the "no enabled local indexes to search" come back again (after a while of not seeing it after deploy). Any thoughts on why this is still happening?
Thanks, Ngan On Monday, August 12, 2013 9:24:01 AM UTC-7, Ngan wrote: > > You are awesome! Thanks! > > I've deployed to production and everything works great. I'll be keeping > an eye out for my original problem of errors on deployment. > > Can't wait for 2.1.1 :-) > > On Monday, August 12, 2013 5:08:54 AM UTC-7, Pat Allan wrote: >> >> The change was pretty simple: >> >> https://github.com/pat/thinking-sphinx/commit/d5249ea0215128d5d1f916ee42882932ddb86ba7 >> >> >> To use it (for the 2.x releases of Thinking Sphinx): >> >> gem 'thinking-sphinx', '~> 2.1.0', >> :git => 'git://github.com/pat/thinking-sphinx.git', >> :branch => 'v2', >> :ref => '64da4dc7ff' >> >> And then in an initialiser: >> >> ThinkingSphinx.persistence_enabled = false >> >> Will be keen to hear if everything returns to normal for you! >> >> On 12/08/2013, at 12:19 AM, Ngan Pham wrote: >> >> > I see... >> > >> > So I've tested it out and it's a bit of a disaster for my situation. >> Some search requests are getting dropped and the server that runs sphinx is >> hitting memory limits. I could, in theory, solve the problem by increasing >> ram, how ever we have fallback servers...and it would suck to have to >> increase those as well. I'll do some research on how people handle large >> sphinx requests...like airbnb. >> > >> > As for my 2 cents: I never felt that there was that much of an issue >> with the cost of opening a connection to sphinx for every request. It is >> very fast for us currently, and shaving 50-80% off something that already >> only takes milliseconds while trading for more memory usage and less >> scalability isn't worth for me specifically. It would awesome if you added >> a configuration option to disable persistent connections! I will buy you a >> beer! :) >> > >> >> On Aug 10, 2013, at 10:26 PM, Pat Allan <[email protected]> >> wrote: >> >> >> >> And the benefit is faster query times - the socket isn't being >> setup/cleaned up on every single search request. From the quick testing I >> did back when I made this change, this made a noticeable difference with >> Sphinx query times (no numbers at hand, but I think it was at least a 50% >> improvement, if not closer to 80%). >> >> >> >> -- >> >> Pat >> >> >> >>> On 11/08/2013, at 3:15 PM, Ngan Pham wrote: >> >>> >> >>> Hm…I was afraid of that… >> >>> So, we have 9 application servers with 10-20 processes each. At high >> traffic times, we'd be seeing 181 processes for searchd? Won't that blow >> memory up like crazy? Is this normally something that everyone deals with? >> >>> >> >>> Just curious…what are the benefits of persisted connection pools? >> >>> >> >>> As always, thanks for the quick response Pat! >> >>> >> >>> - Ngan >> >>>> On Saturday, August 10, 2013 at 10:11 PM, Pat Allan wrote: >> >>>> >> >>>> Hi Ngan >> >>>> >> >>>> There's not really any documentation around the changes, I'm afraid… >> but what you're seeing is a process per thread of your app, plus a master >> daemon process, due to the persisted connection pool. >> >>>> >> >>>> All the logic for this connection pool can be found in >> ThinkingSphinx::Connection: >> >>>> >> https://github.com/pat/thinking-sphinx/blob/v2/lib/thinking_sphinx/connection.rb >> >> >>>> >> >>>> The HISTORY file has a list of all relevant changes though - here's >> the list of what's changed between 2.0.14 and 2.1.0 - but this and some >> delta refactoring are the biggest items: >> >>>> >> >>>> * Removed plugin support - Thinking Sphinx is now gem-only across >> all branches. >> >>>> * ThinkingSphinx::Version and the thinking_sphinx:version task have >> been removed - it's a gem, it has a version number. >> >>>> * Updating Riddle to 1.5.6 or newer. >> >>>> * Requires ActiveRecord ~> 2.1 for TS 1.x releases (earlier versions >> were considered unsupported a few releases ago). >> >>>> * Allow custom Riddle controllers - useful for Flying Sphinx to take >> over management of Sphinx daemon/indexing actions. >> >>>> * Rejigged delta support to be generic, with local job classes that >> provide a clean, simple interface for third-party libraries. >> >>>> * Add hooks for anything that needs to happen before indexing (such >> as clearing out existing delta jobs). >> >>>> * Connection pool for all Sphinx client communication, with new >> connections built if there's any connection-related (as opposed to syntax) >> issues. >> >>>> * Multiple-field search conditions can be done with arrays of field >> names as keys in the :conditions hash (Alex Dowad). >> >>>> * Removed named capture in regular expressions to maintain MRI 1.8 >> support (Michael Wintrant). >> >>>> * Support new JDBC configuration style (Kyle Stevens). >> >>>> >> >>>> -- >> >>>> Pat >> >>>> >> >>>>> On 11/08/2013, at 2:39 PM, Ngan wrote: >> >>>>> >> >>>>> Hi Pat, >> >>>>> >> >>>>> Thanks for the help. I tried upgrading to 2.1.0...and I'm noticing >> multiple instance of searched running now. Is that normal? Would you be >> able to point me to documentation of major changes with 2.1.0? >> >>>>> >> >>>>> Thanks, >> >>>>> Ngan >> >>>>> >> >>>>> On Friday, August 9, 2013 7:57:51 AM UTC-7, Pat Allan wrote: >> >>>>> Hi Ngan >> >>>>> >> >>>>> In 2.1.0 there's been some patches that deal with these kinds of >> errors - TS will now retry searches if an error crops up on the client >> connection (which is also persisted per thread in a connection pool, saving >> socket setup/teardown time). If an error crops up, a new connection is >> made, but if the error persists, it'll still get raised… >> >>>>> >> >>>>> Give 2.1.0 a spin, see if that helps matters. >> >>>>> >> >>>>> -- >> >>>>> Pat >> >>>>> >> >>>>>> On 09/08/2013, at 10:46 PM, Ngan wrote: >> >>>>>> >> >>>>>> Forgot to add the trace: >> >>>>>> >> >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:438:in >> `block in populate' >> >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:606:in >> `call' >> >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:606:in >> `retry_on_stale_index' >> >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:426:in >> `populate' >> >>>>>> gems/thinking-sphinx-2.0.14/lib/thinking_sphinx/search.rb:104:in >> `to_a' >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Friday, August 9, 2013 5:45:55 AM UTC-7, Ngan wrote: >> >>>>>> Hi, >> >>>>>> >> >>>>>> We reindex our entire index pretty often (once every 3 minutes) >> because we have a pretty small data collection and we don't want to use >> delayed delta. I notice however, that every once in a while, when we deploy >> our application and it happens to be the same time the reindexing is about >> to rotate, we'll get this error "no enabled local indexes to search" every >> time we hit sphinx there afterwards. When this happens, we have to restart >> our app so that it picks up the new indexes. We are reindexing with rotate >> so the existing index should still be there and the rotation should be >> seamless. Any ideas on why this happens? And if there's anything to do >> about it? >> >>>>>> >> >>>>>> Thanks, >> >>>>>> Ngan >> >>>>>> >> >>>>>> -- >> >>>>>> You received this message because you are subscribed to the Google >> Groups "Thinking Sphinx" group. >> >>>>>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>>>>> To post to this group, send email to [email protected]. >> >>>>>> Visit this group at http://groups.google.com/group/thinking-sphinx. >> >> >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> You received this message because you are subscribed to the Google >> Groups "Thinking Sphinx" group. >> >>>>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>>>> To post to this group, send email to [email protected]. >> >>>>> Visit this group at http://groups.google.com/group/thinking-sphinx. >> >> >>>>> For more options, visit https://groups.google.com/groups/opt_out. >> >>>> >> >>>> -- >> >>>> You received this message because you are subscribed to a topic in >> the Google Groups "Thinking Sphinx" group. >> >>>> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/thinking-sphinx/rPsXVX8wlHU/unsubscribe. >> >> >>>> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> >>>> To post to this group, send email to [email protected]. >> >>>> Visit this group at http://groups.google.com/group/thinking-sphinx. >> >>>> For more options, visit https://groups.google.com/groups/opt_out. >> >>> >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> Groups "Thinking Sphinx" group. >> >>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>> To post to this group, send email to [email protected]. >> >>> Visit this group at http://groups.google.com/group/thinking-sphinx. >> >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> -- >> >> You received this message because you are subscribed to a topic in the >> Google Groups "Thinking Sphinx" group. >> >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/thinking-sphinx/rPsXVX8wlHU/unsubscribe. >> >> >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> >> To post to this group, send email to [email protected]. >> >> Visit this group at http://groups.google.com/group/thinking-sphinx. >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Thinking Sphinx" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at http://groups.google.com/group/thinking-sphinx. >> > For more options, visit https://groups.google.com/groups/opt_out. >> >> -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/thinking-sphinx. For more options, visit https://groups.google.com/groups/opt_out.
