Hi Jeff - thanks a lot for looking at this.
I discovered that since last august, will paginate lets you paginate any
arbitrary array - so you can paginate an existing set of results. so what
i'm doing now is this
- get the results using ferret find, passing through the AR conditions and a
large per page which means there's no pagination, effectively. The sorting
(by ferret score) is correct.
- cache the results with memcache, using params (minus the page k-v pair) as
a cache key - this means that the retrieve will be slow the first time but
very quick if the user just wants to look at another page
- then paginate the results, simply doing "@all_results.paginate(:page =>
params[:page], :per_page => 20)"
This is pretty similar to your patch i think - get results with ferret and
then paginate - and it's working fine for me. The size of my result set
returned by ferret is rarely more than 150 big, so getting all results isn't
a problem. Obviously it's not very scaleable though.
thanks!
max
2008/7/18 Jeff Webb <[EMAIL PROTECTED]>:
> Max,
>
> Here is what I changed in the current stable version of AAF in my local
> plugin install to get conditions and pagination working. I have not fully
> tested it except for my particular use cases but it seems to work out.
> Sorting is not factored into these changes.
>
> Basically if there is no pagination OR no AR conditions you can let ferret
> do all the work for you. If you have both of these then you need to return
> all results, merge with ferret IDs, then do your offset and limit. I
> replicated what Jens has done on the AAF trunk.
>
> Hope this works for you. I have not heard if Jens is going to add this to
> the current branch as a hotfix.
>
> replace the existing ar_by_contents method in class_methods.rb with:
>
> ####################################
> #changes to ar_find_by_contents AAF 0.4.3#
> ####################################
>
> def ar_find_by_contents(q, options = {}, find_options = {})
> result_ids = {}
> has_conditions = !find_options[:conditions].blank? || caller.find{
> |call| call =~ %r{active_record/associations} }
>
> # odd case - cannot do pagination combo with AR & Ferret
> # must retrieve all then paginate after
> if options[:per_page] && has_conditions
> late_paginate = true
> offset = find_options.delete(:offset)
> limit = find_options.delete(:limit)
> options.delete(:page)
> options.delete(:per_page)
> find_options.delete(:offset)
> find_options.delete(:limit)
> options[:limit] = :all
> end
>
> total_hits = find_id_by_contents(q, options) do |model, id, score,
> data|
> # stores ids, index and score of each hit for later ordering of
> # results
> result_ids[id] = [ result_ids.size + 1, score ]
> end
>
> result = retrieve_records( { self.name => result_ids }, find_options
> )
>
> if has_conditions
> # what we got from the database is our full result set, so take
> it's size
> total_hits = result.length
>
> if late_paginate
> result = result[offset..offset+limit-1]
> end
> end
>
> [ total_hits, result ]
> end
>
>
> Jeff
>
>
>
>
> On Fri, Jul 18, 2008 at 4:11 AM, Max Williams <[EMAIL PROTECTED]>
> wrote:
>
>> Hi Jens/all
>>
>> I've been playing with latest version, and it seems that the combination
>> of AR conditions, paginating and AR order (eg ":order => "name") seems to be
>> fine now, even on our server, which was were it was broken before (it always
>> worked locally, weirdly).
>>
>> However, i've just tried it with results that are sorted by ferret on the
>> basis of their boost score, and it seems broken.
>>
>> I'm sorry i don't have any tests set up but if i walk through an example
>> can anyone see something that maybe i'm doing wrong? Or shed any light?
>>
>> I use AR conditions to filter the results through a set of ids that an
>> individual user is allowed to view. I just have a method User#allowed_ids
>> for this that returns an array of integers. For the purpose of
>> illustration, though, let's say that the allowed ids are everything between
>> 1000 & 2000.
>> allowed_ids = (1000..2000).to_a
>> => [1000, 1001, 1002, etc, 2000]
>>
>> For a particular search (on 'rhythm') i get these results back - the order
>> is the order calculated by ferret/aaf on the basis of boost values: i've
>> collected them by id for clarity. In this example i've set a big per_page
>> to get all the results.
>>
>> >> ActsAsFerret::find("rhythm", [TeachingObject],{ :page => 1, :per_page
>> => 1000 }, {} ).collect(&:id)
>> => [4038, 698, 4039, 1830, 1831, 1826, 1832, 1825, 1833, 411, 1834, 702,
>> 1827, 1689, 1680, 1688, 1679, 1686, 1684, 1676, 2129, 2130, 2131, 1858,
>> 1859, 1860, 1861, 1865, 2132, 2141, 2345, 2350, 2352, 2353, 2356, 2360,
>> 2362, 2366, 2368, 2371, 2372, 2373, 2376, 2377, 2378, 2384, 2385, 2389,
>> 2407, 2414, 2417, 2419, 2427, 2428, 2438, 2439, 2458, 2459, 2473, 2474,
>> 2475, 2477, 2478, 2133, 2349, 2361, 2363, 2365, 2370, 2375, 2383, 2386,
>> 2392, 2415, 2430, 2431, 2440, 2441, 2442, 2472, 2142, 3751, 2161, 1236]
>>
>> Now, when i apply the condition of only being in the allowed ids, i'd
>> expect the remaining ids to be in the same order as above, and that is in
>> fact the case:
>>
>> >> ActsAsFerret::find("rhythm", [TeachingObject],{ :page => 1, :per_page
>> => 1000 }, {:conditions => ["id in (?)", allowed_ids] } ).collect(&:id)
>> => [1830, 1831, 1826, 1832, 1825, 1833, 1834, 1827, 1689, 1680, 1688,
>> 1679, 1686, 1684, 1676, 1858, 1859, 1860, 1861, 1865, 1236]
>>
>> Just to check, let's do the original again (with no AR conditions) and
>> just do set intersection with the allowed_ids
>> >> ActsAsFerret::find("rhythm", [TeachingObject],{ :page => 1, :per_page
>> => 1000 }, {} ).collect(&:id) & allowed_ids
>> => [1830, 1831, 1826, 1832, 1825, 1833, 1834, 1827, 1689, 1680, 1688,
>> 1679, 1686, 1684, 1676, 1858, 1859, 1860, 1861, 1865, 1236]
>>
>> OK - looks good. But, when i try to paginate into some actual pages, the
>> order breaks: first, without AR conditions:
>>
>> >> ActsAsFerret::find("rhythm", [TeachingObject],{ :page => 1, :per_page
>> => 10 }, {} ).collect(&:id)
>> => [4038, 698, 4039, 1830, 1831, 1826, 1832, 1825, 1833, 411]
>>
>> That all seems to be in order - i get the first ten results from the big
>> list, above.
>>
>> Now, if we were to apply the allowed_ids condition here, we'd expect the
>> results to start
>> [1830, 1831, 1826, 1832, 1825, 1833, ...] - right? Because we should have
>> the same ordering applied to the remaining resources, and then we get the
>> first ten for page 1. But, the ordering is different -
>>
>> >> ActsAsFerret::find("rhythm", [TeachingObject],{ :page => 1, :per_page
>> => 10 }, {:conditions => ["id in (?)", allowed_ids] } ).collect(&:id)
>> => [1826, 1825, 1689, 1680, 1688, 1679, 1686, 1684, 1676, 1236]
>>
>> So, it seems that pagination + ferret score ordering + AR conditions is a
>> bad combination
>>
>> Again, sorry to not supply test cases but we don't use them (i know, i
>> know!!!). Can anyone shed any light?
>>
>> thanks - max
>>
>> 2008/7/9 Jens Kraemer <[EMAIL PROTECTED]>:
>>
>> Hi folks,
>>>
>>> On 09.07.2008, at 06:01, Jeff Webb wrote:
>>>
>>>> Sheldon/Jens,
>>>>
>>>> Any chance that this has been figured out and patched?
>>>>
>>>
>>> I just committed a fix to this problem to trunk, so pagination with
>>> sorting and AR conditions should work now.
>>>
>>>
>>> cheers,
>>> Jens
>>>
>>> --
>>> Jens Krämer
>>> http://www.jkraemer.net/ - Blog
>>> http://www.omdb.org/ - The new free film database
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ferret-talk mailing list
>>> [email protected]
>>> http://rubyforge.org/mailman/listinfo/ferret-talk
>>>
>>
>>
>> _______________________________________________
>> Ferret-talk mailing list
>> [email protected]
>> http://rubyforge.org/mailman/listinfo/ferret-talk
>>
>
>
>
> --
>
> Jeff Webb
> [EMAIL PROTECTED]
> http://boowebb.com/
>
> _______________________________________________
> Ferret-talk mailing list
> [email protected]
> http://rubyforge.org/mailman/listinfo/ferret-talk
>
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
