jhaagmans a écrit, le 11/19/2009 03:34 PM : > And yes, I'm still using BRB, I really can't think of a way to avoid > using BRB. I need to query a few thousand hosts every hour, every day. > >
This translates to a few hosts each second. If I had this kind of load I wouldn't use a background job scheduler but a queue manager, several processes picking the requests from the queue and a custom-built scheduler. In fact I'm doing this for several projects and I use ActiveMessaging. > I really regret choosing Ruby/Rails for this particular application at > this point. > Ruby/Rails has not much to do with your problems. Choosing the right tool for the job is the issue. I've a server running a web spider hand-coded in Ruby using the tools I described above and it successfully make 10s of thousands of HTTP queries per hour. The only limit I hit is the RAM available as I use several Ruby processes (which, by the way, internally use threads to handle simultaneous HTTP HEAD/GET requests efficiently). If I had say 32G i could probably hit 100k requests per hour. Given I use a queue manager, I could also add a second server and run queue processors on it to get more capacity... Lionel -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-t...@googlegroups.com. To unsubscribe from this group, send email to rubyonrails-talk+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=.
