You can use the built build thread pool to process more than one feed within
the same worker. So within the worker, you'd do,
def parse_feeds
loop do
feed = Feed.find_feed_to_process
thread_pool.defer do
feed.parse
end
end
end
I think the default pool size is 20. You can control the size of the thread
pool using a class level method, as I recall it is
pool_size x
Paul
On Wed, Apr 23, 2008 at 7:30 AM, Julien Genestoux <
[EMAIL PROTECTED]> wrote:
> Thanks Adam,
>
> That sounded weird to me as well to have one worker for each feed...
> However, if I only have one worker, that also means that I am parsing
> one feed only at any moment. An option, is maybe to have a few workers
> (denpending on the number of feeds) that parse feeds concurrently?
>
> If I only have one worker, according to you what should be the
> winnning strategy to choose the "right" parse to feed? Obviously some
> feeds need to be parsed one every few minutes, while some other might
> no need to be parse more than every hour...
>
> Any idea/tip on this?
>
>
>
>
> On 4/23/08, Adam Williams <[EMAIL PROTECTED]> wrote:
> > On Apr 23, 2008, at 1:07 AM, Julien Genestoux wrote:
> >
> > > I still have a few questions : shoud I have one worker for each feed
> > > that is called periodically (add_periodic_timer) or rather one single
> > > worker that calls every feed one by one?
> > >
> > > What is the best solution, perfomance-wise?
> >
> >
> > Good question... I don't suppose I know exactly. I would start by
> > processing all the feeds in one worker invocation - that is what I
> > have done for sending an unknown amount of email. It just seems wrong
> > to me to invoke a worker for one email at a time.
> >
> > The right answer likely lies in understanding the whole MasterWorker,
> > Packet::Reactor/handler_instance.ask_work bits of the puzzle...
> >
> >
> > adam
> >
> > _______________________________________________
> > Backgroundrb-devel mailing list
> > [email protected]
> > http://rubyforge.org/mailman/listinfo/backgroundrb-devel
> >
>
>
> --
> --
> Julien Genestoux
> [EMAIL PROTECTED]
> http://www.ouvre-boite.com
> +1 (415) 254 7340
> +33 (0)8 70 44 76 29
> _______________________________________________
> Backgroundrb-devel mailing list
> [email protected]
> http://rubyforge.org/mailman/listinfo/backgroundrb-devel
>
_______________________________________________
Backgroundrb-devel mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/backgroundrb-devel
