Also, I came across update_attribute_values in lib/thinking_sphinx/ active_record/attribute_updates.rb... how exactly is that used?
On Apr 26, 12:58 pm, agibralter <[email protected]> wrote: > Hi Pat, thank you for the response. I'll try to take a stab at a > patch. Just to be clear, when you say updates just take place in > searchd's memory, does that mean if searchd restarts, the attributes > will be stale until the next time the indexer is run? > > Also, on a separate note: do you have any thoughts on whether delayed- > delta or datetime-delta would be more efficient in system that updates > very often? I.e I'd like to have index updates reflected in a minute > if possible. Are there any tradeoffs between > > "set_property :delta => true" with delayed-delta > > and > > "set_property :delta => :datetime, :threshold => 2.minutes" with > datetime-delta? > > Thanks again! > > -ajg- > > On Apr 22, 11:07 pm, Pat Allan <[email protected]> wrote: > > > > > Hi guys > > > Seems there's a few topics being covered here, so it's taking a bit to grok > > it all, but with regards to MVA attribute updating: if it's something that > > Sphinx supports, then I'd like TS to support it natively as well. So Nick, > > if you want to wrap your changes into a patch with specs, that'd be > > fantastic. > > > Obviously, you'll want to check what version of Sphinx is being used, but > > that's do-able via configuration.version. > > > Aaron, what you're wanting to is a little bit more complex, but I like your > > approaches... maybe the more general hook is better, allowing for complex > > situations. A patch for that would be fantastic, too. You can invoke > > updates for Sphinx from any machine - all it actually does is update the > > attributes within searchd's memory, not the index files - and it's all over > > the socket anyway, so even if it did change the file, it can be remote. > > > Hope this helps clear a couple of things up - would love to see what you > > guys come up with. And if there's other questions I've not answered, let me > > know :) > > > -- > > Pat > > > On 23/04/2010, at 2:16 AM, agibralter wrote: > > > > Hmm I'm not sure I follow... As far as integration with the DSL I was > > > thinking of something like:http://gist.github.com/375411 > > > > But in general, delta indexes aside, if I wanted to do live updates on > > > attributes I could create some ActiveRecord before_save callbacks that > > > check for changed attributes and update the Sphinx indexes like so: > > >http://gist.github.com/375427? > > > > Now, my question is, does ThinkingSphinx.update_index need to be > > > called on a server running searchd? Or can it be called remotely? It > > > seems like it just uses the Riddle client and can be called from > > > anywhere... no need to have access to the actual index core files. > > > > Thanks again for the help! > > > > Best, > > > Aaron > > > > On Apr 22, 6:10 am, Nick Sellen <[email protected]> wrote: > > >> My extension is only a very small addition to reveal the attribute > > >> update functionality. It would need quite a bit more work to > > >> incorporate it into sphinx proper (someone was talking about doing it > > >> somewhere I read). > > > >> It suits my needs well enough for now but it's a fairly manual process > > >> to run it though. I added it to get around the process of doing > > >> updates to a lot of records and avoiding this kind of process: > > >> . add a delayed job for each record to remove the it from the main > > >> index (well, set the "sphinx_deleted" attribute using client.update) > > >> . sql to update each single record to delta = 1 > > >> . and then the job to run the delta index > > > >> I was using Model.suspended_delta { # do stuff with Model records } > > >> but that doesn't set the "sphinx_deleted" attribute so there was > > >> problems having duplicate entries in the main and delta indexes. > > > >> Currently I use Model.update_all calls (which doesn't instantiate the > > >> model therefore no callbacks) and manually calculated > > >> ThinkingSphinx.update_index calls. But it's not ideal still because I > > >> still need to read quite a lot of individual rows from the db to > > >> update the MVA attributes (as you can't make a call to add or remove a > > >> value from the array you need to set the whole thing which requires > > >> fetching it first). > > > >> So I think I'm going to go back to running the delta sql for mass > > >> updates going forwards and using the attribute update methods for > > >> smaller updates. e.g.: > > >> Model.update_all("some things I want to update",update conditions) > > >> Model.update_all('delta = 1',update conditions) > > >> Model.update_delta_index_and_wait # another addition I've made to > > >> perform the delta update synchonously > > > >> On Apr 21, 10:52 pm, agibralter <[email protected]> wrote: > > > >>> Hi Nick, > > > >>> This looks awesome! I'm using Sphinx 0.9.9 -- so how exactly would I > > >>> use it? Set up a callback for my models that call the method in your > > >>> pastie when their attributes of interest get updated? Or is there a > > >>> way to hook into ThinkingSphinx's DSL for define_index such that `has > > >>> attribute` could take an option like :update => true? Does anyone know > > >>> how Pat feels about incorporating this into TS proper? > > > >>> Also, does that update have to happen on the app server with searchd > > >>> running? > > > >>> Thanks a ton! > > > >>> -Aaron > > > >>> On Apr 20, 4:14 am, Nick Sellen <[email protected]> wrote: > > > >>>>> multiple app servers but only one Sphinx/searchd > > > >>>> the caveat is "it will only work for a single searchd instance" - so > > >>>> you should be fine to use the delayed delta with multiple app servers > > >>>> (but it looks like you're not using delayed job anyway). > > > >>>> I use sphinx in a simliar way to you and the most useful information > > >>>> in my index are the attributes (not fields) and you can actually live > > >>>> update the index for these. I've stopped doing delta updates where > > >>>> possible to do the live update instead. I've added a small extension > > >>>> to thinking sphinx which makes it easier > > >>>> runhttp://pastebin.com/KgwWREnj. > > > >>>> (You need sphinx 0.9.9 if you want to live update MVA attributes > > >>>> though) > > > >>>> Hope this is helpful. > > > >>>> Nick > > > >>>> On Apr 19, 5:12 pm, agibralter <[email protected]> wrote: > > > >>>>> I use Sphinx for displaying filtered, sortable, and searchable lists > > >>>>> of items in my web app. I use Sphinx even when there is no searching > > >>>>> involved because ThinkingSphinx's sphinx_scopes make it very easy to > > >>>>> chain attribute constraints together and because Sphinx is very fast. > > >>>>> There is an obvious disadvantage though: lists are not displayed in > > >>>>> realtime. This means I have to leverage delta indexing as much as > > >>>>> possible... > > > >>>>> Because I have multiple app servers but only one Sphinx/searchd > > >>>>> server, I cannot use the standard delta indexer. I have seen the > > >>>>> caveat at the bottom > > >>>>> ofhttp://freelancing-god.github.com/ts/en/deltas.html > > >>>>> but I can't figure out a good way to send asynchronous delta update > > >>>>> jobs to my application server with searchd on it. Has anyone figured > > >>>>> out a good way to do this? Also, I'm not using delayed_job in my > > >>>>> app... I'm using Workling/Starling, but thinking of moving to > > >>>>> Resque... has anyone seen a delayed delta that uses Resque? If so, is > > >>>>> there a way to send the "async_update_deltas" jobs to my app server > > >>>>> with searchd? > > > >>>>> Anyway, as an alternative, I've been using ts-datetime-delta to keep > > >>>>> the lists as up-to-date as possible, running the rake ts:index:delta > > >>>>> task every 2 minutes with cron on my server with searchd. I'm thinking > > >>>>> of moving to 1 minute to be more realtime... This seems much more > > >>>>> frequent than any examples of ts-datetime-delta I've seen... does this > > >>>>> sound crazy? Should I try and figure out a way to use the delayed > > >>>>> delta instead? > > > >>>>> Sorry for all the questions! I appreciate any advice! > > > >>>>> Best, > > >>>>> Aaron > > > >>>>> -- > > >>>>> You received this message because you are subscribed to the Google > > >>>>> Groups "Thinking Sphinx" group. > > >>>>> To post to this group, send email to [email protected]. > > >>>>> To unsubscribe from this group, send email to > > >>>>> [email protected]. > > >>>>> For more options, visit this group > > >>>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > >>>> -- > > >>>> You received this message because you are subscribed to the Google > > >>>> Groups "Thinking Sphinx" group. > > >>>> To post to this group, send email to [email protected]. > > >>>> To unsubscribe from this group, send email to > > >>>> [email protected]. > > >>>> For more options, visit this group > > >>>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > >>> -- > > >>> You received this message because you are subscribed to the Google > > >>> Groups "Thinking Sphinx" group. > > >>> To post to this group, send email to [email protected]. > > >>> To unsubscribe from this group, send email to > > >>> [email protected]. > > >>> For more options, visit this group > > >>> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > >> -- > > >> You received this message because you are subscribed to the Google > > >> Groups "Thinking Sphinx" group. > > >> To post to this group, send email to [email protected]. > > >> To unsubscribe from this group, send email to > > >> [email protected]. > > >> For more options, visit this group > > >> athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Thinking Sphinx" group. > > > To post to this group, send email to [email protected]. > > > To unsubscribe from this group, send email to > > > [email protected]. > > > For more options, visit this group > > > athttp://groups.google.com/group/thinking-sphinx?hl=en. > > > -- > > You received this message because you are... > > read more » -- You received this message because you are subscribed to the Google Groups "Thinking Sphinx" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/thinking-sphinx?hl=en.
