No worries - here's hoping you get it figured out. I'm certainly curious! I guess the one thing I'd be interested to see in your logging, is are there any differences between the messages going *to* Sphinx from Riddle when it works, and when it doesn't? Can't think why there would be, but would at least confirm absolutely whether Riddle/TS is at fault or not.
-- Pat On 23/03/2011, at 6:49 PM, Robby Russell wrote: > Moving on... Thanks Pat! > > * http://sphinxsearch.com/forum/view.html?id=7179 > > On Wed, Mar 23, 2011 at 6:32 PM, Robby Russell <[email protected]> wrote: >> Nope, no timing patterns yet. >> >> We've since ripped open Riddle and are outputing raw debug info... >> inspecting... and this is what we're getting in this scenario. >> >> SEARCH: RIDDLE RESULT [:words]={"tour"=>{:docs=>1695, :hits=>15802}} >> SEARCH: RIDDLE RESULT [:total_found]=208 >> SEARCH: RIDDLE RESULT [:error]=nil >> SEARCH: RIDDLE RESULT={:attribute_names=>["sphinx_internal_id", >> "class_crc", "sphinx_deleted", "tour_name_sort", "tour_id_sort", >> "selling_company_id", "brochure_id", "brochure_variation_id"], >> :matches=>[], :status=>0, >> :attributes=>{"brochure_variation_id"=>1073741825, "brochure_id"=>1, >> "tour_name_sort"=>3, "selling_company_id"=>1, "sphinx_deleted"=>1, >> "class_crc"=>1, "tour_id_sort"=>3, "sphinx_internal_id"=>1}, >> :total=>208, :time=>0.002, :total_found=>208, >> :words=>{"tour"=>{:docs=>1695, :hits=>15802}}, >> :fields=>["tour_variation_name", "tour_name", "tour_sightseeing", >> "tour_intro", "tour_transportation", "tour_travels_to", >> "tour_tags_cache", "tour_operating_product_code", "tour_type_name", >> "places_cache", "optionals_cache", "highlights_cache", "tour_code", >> "tour_id"]} >> SEARCH: RIDDLE RESPONSE=#<Riddle::Client::Response:0x7f79a091bf10 >> @str="\000\000\000\000\000\000\000\016\000\000\000\023tour_variation_name\000\000\000\ttour_name\000\000\000\020tour_sightseeing\000\000\000\ntour_intro\000\000\000\023tour_transportation\000\000\000\017tour_travels_to\000\000\000\017tour_tags_cache\000\000\000\etour_operating_product_code\000\000\000\016tour_type_name\000\000\000\fplaces_cache\000\000\000\017optionals_cache\000\000\000\020highlights_cache\000\000\000\ttour_code\000\000\000\atour_id\000\000\000\b\000\000\000\022sphinx_internal_id\000\000\000\001\000\000\000\tclass_crc\000\000\000\001\000\000\000\016sphinx_deleted\000\000\000\001\000\000\000\016tour_name_sort\000\000\000\003\000\000\000\ftour_id_sort\000\000\000\003\000\000\000\022selling_company_id\000\000\000\001\000\000\000\vbrochure_id\000\000\000\001\000\000\000\025brochure_variation_id@\000\000\001\000\000\000\000\000\000\000\000\000\000\000?\000\000?\000\000\002\000\000\000\001\000\000\000\004tour\000\000\006\237\000\000=\272", >> @marker=0> >> >> When it does work... this looks insanely crazy/huge... so I'm going to >> concur that sphinx is probably at fault here, which is discouraging. >> Guess it's time to take our story to the sphinx mailing list. >> >> Robby >> >> On Tue, Mar 22, 2011 at 6:37 AM, Pat Allan <[email protected]> wrote: >>> Any patterns to the timing of the issue? What about load on the server? >>> Harder to track that, I guess... >>> >>> -- >>> Pat >>> >>> On 20/03/2011, at 9:41 PM, Robby Russell wrote: >>> >>>> Thanks for this. Been running in production for a few hours with >>>> that... but still producing the same problem. *scratches head* >>>> >>>> Now to think of some other ways to dissect this problem. >>>> >>>> On Sun, Mar 20, 2011 at 3:40 PM, Pat Allan <[email protected]> >>>> wrote: >>>>> Yeah, unless Sphinx is on the app server, it's best to. This should do >>>>> the trick: >>>>> >>>>> production: >>>>> version: 0.9.9 >>>>> >>>>> That'll make sure Riddle loads its 0.9.9 additions. >>>>> >>>>> -- >>>>> Pat >>>>> >>>>> On 20/03/2011, at 5:35 PM, Robby Russell wrote: >>>>> >>>>>> We're not. Should we be? >>>>>> >>>>>> On Sun, Mar 20, 2011 at 3:25 PM, Pat Allan <[email protected]> >>>>>> wrote: >>>>>>> In a similar vein, are you setting the Sphinx version number in your >>>>>>> config/sphinx.yml file, given that Sphinx isn't on your app servers? >>>>>>> >>>>>>> -- >>>>>>> Pat >>>>>>> >>>>>>> On 20/03/2011, at 5:22 PM, Robby Russell wrote: >>>>>>> >>>>>>>> I noticed that Riddle had some variant code depending on the version >>>>>>>> of the sphinx server. I hadn't restarted the app servers yet, so just >>>>>>>> did that and will see if that makes any difference. >>>>>>>> >>>>>>>> On Sun, Mar 20, 2011 at 3:16 PM, Pat Allan <[email protected]> >>>>>>>> wrote: >>>>>>>>> Fun indeed. >>>>>>>>> >>>>>>>>> Have you posted to the Sphinx forum? Given that post you referred to >>>>>>>>> is three years old, perhaps there's people with a better idea of >>>>>>>>> what's causing it now? >>>>>>>>> >>>>>>>>> And just to check - the timing of the problem doesn't match the >>>>>>>>> timing of your indexing, does it? >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Pat >>>>>>>>> >>>>>>>>> On 20/03/2011, at 4:38 PM, Robby Russell wrote: >>>>>>>>> >>>>>>>>>> A few hours later, it appears that we're still hitting the same >>>>>>>>>> problem with 0.9.9. Fun fun! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Mar 20, 2011 at 11:54 AM, Robby Russell >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> We're adding more memory to the sphinx servers and upgrading to >>>>>>>>>>> 0.9.9. >>>>>>>>>>> Will let you know what we uncover through that process... >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sat, Mar 19, 2011 at 3:45 PM, Pat Allan >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> Well that's frustrating. >>>>>>>>>>>> >>>>>>>>>>>> I agree that the fact that it's only cropping up since you've >>>>>>>>>>>> switched to TS is a fair reason to suspect TS/Riddle. Can't think >>>>>>>>>>>> why it would be so unreliable (and difficult to reproduce). >>>>>>>>>>>> >>>>>>>>>>>> A shot in the dark, but could you try upgrading to Sphinx 0.9.9 or >>>>>>>>>>>> even 1.10-beta? >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Pat >>>>>>>>>>>> >>>>>>>>>>>> On 19/03/2011, at 5:42 PM, Robby Russell wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Sat, Mar 19, 2011 at 12:33 PM, Pat Allan >>>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>>> To be honest, this is at the edge of my Sphinx knowledge. I'd >>>>>>>>>>>>>> recommend posting a question on the Sphinx forum, talk through >>>>>>>>>>>>>> how usually the search works, but sometimes doesn't - and yet >>>>>>>>>>>>>> results from Sphinx say there should be results. >>>>>>>>>>>>>> >>>>>>>>>>>>>> In the meantime, you could add something like the following to >>>>>>>>>>>>>> your controller: >>>>>>>>>>>>>> >>>>>>>>>>>>>> @variations = TourVariation.search 'foo' >>>>>>>>>>>>>> if @variations.empty? && @variations.results[:total] > 0 >>>>>>>>>>>>>> @variations = TourVariation.search 'foo' >>>>>>>>>>>>>> end >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Actually, we tried raising an exception, rescuing and forcing it >>>>>>>>>>>>> to >>>>>>>>>>>>> retry up time five times with a short delay between... and each >>>>>>>>>>>>> time >>>>>>>>>>>>> it was still returning zero results. When it's not cooperating... >>>>>>>>>>>>> it >>>>>>>>>>>>> seems like that individual request is just a lost cause. >>>>>>>>>>>>> >>>>>>>>>>>>> Robby >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>>> Google Groups "Thinking Sphinx" group. >>>>>>>>>>>>> To post to this group, send email to >>>>>>>>>>>>> [email protected]. >>>>>>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>>>>>> [email protected]. >>>>>>>>>>>>> For more options, visit this group at >>>>>>>>>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>>> Groups "Thinking Sphinx" group. >>>>>>>>>>>> To post to this group, send email to >>>>>>>>>>>> [email protected]. >>>>>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>>>>> [email protected]. >>>>>>>>>>>> For more options, visit this group at >>>>>>>>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>> Groups "Thinking Sphinx" group. >>>>>>>>>> To post to this group, send email to >>>>>>>>>> [email protected]. >>>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>>> [email protected]. >>>>>>>>>> For more options, visit this group at >>>>>>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Thinking Sphinx" group. >>>>>>>>> To post to this group, send email to [email protected]. >>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>> [email protected]. >>>>>>>>> For more options, visit this group at >>>>>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Thinking Sphinx" group. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> To unsubscribe from this group, send email to >>>>>>>> [email protected]. >>>>>>>> For more options, visit this group at >>>>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Thinking Sphinx" group. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> To unsubscribe from this group, send email to >>>>>>> [email protected]. >>>>>>> For more options, visit this group at >>>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Thinking Sphinx" group. >>>>>> To post to this group, send email to [email protected]. >>>>>> To unsubscribe from this group, send email to >>>>>> [email protected]. >>>>>> For more options, visit this group at >>>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "Thinking Sphinx" group. >>>>> To post to this group, send email to [email protected]. >>>>> To unsubscribe from this group, send email to >>>>> [email protected]. >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>>> >>>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google Groups >>>> "Thinking Sphinx" group. >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to >>>> [email protected]. >>>> For more options, visit this group at >>>> http://groups.google.com/group/thinking-sphinx?hl=en. >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Thinking Sphinx" group. >>> To post to this group, send email to [email protected]. >>> To unsubscribe from this group, send email to >>> [email protected]. >>> For more options, visit this group at >>> http://groups.google.com/group/thinking-sphinx?hl=en. >>> >>> >> > > -- > You received this message because you are subscribed to the Google Groups > "Thinking Sphinx" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/thinking-sphinx?hl=en. > -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
