I'm having some trouble with using brb for polling for work. Basically, what
I want to do is update the state of a record and have brb notice and start a
process. I do not want to call ask_work for each process because there may
be several places where the object's state is changed, and I only want to
process once. Plus, I can schedule things better this way. My first
thought was to create a worker that polled for records with the correct
state, but it doesn't work as expected. Here is what I have with error
checking, etc. removed:
class RequestQueuePollerWorker < BackgrounDRb::MetaWorker
set_worker_name :request_queue_poller_worker
QUEUE_SLEEP_TIME = 30 # seconds
def create(args = nil)
@running = true
self.poll_queue
end
def build_all_matches(args = nil)
thread_pool.defer(args) do |args|
requests = Request.find_active(:all)
requests.each { |request| request.queue! } # using
acts_as_state_machine
end
end
protected
# Was hoping to get multiple threads processing
def build_matches(args = nil)
thread_pool.defer(args) do |args|
request = Request.find(args.to_i)
request.build_matches
end
end
def poll_queue
Thread.new do
while @running do
sleep_time = QUEUE_SLEEP_TIME
request = nil
Request.transaction do
request = Request.find_in_state(:first, :queued, :order => '
requests.updated_at ASC', :lock => true)
request.search! unless request.nil? # make sure I don't hit again
end
if request
self.build_matches(request.id)
sleep_time = 0
request = nil
end
sleep(sleep_time)
end
end
end
def stop_polling
@running = false
end
end
In my test case, there are 20 Request items. Normally, it would take about 5
seconds to call Request.build_matches on 20 Request items. However, asking
for work like so:
MiddleMan.ask_work(:worker => :request_queue_poller_worker, :worker_method
=> :build_all_matches)
It takes 20 minutes(!) to process the same 20 Request items. Processing
basically c r a w l s along. I expected to immediately see all the states
change to :queued, and then very quickly changed to :searching. However, it
doesn't happen that way. Things go to queued pretty quickly, by searching
takes FOREVER. It's like poll_queue and build_matches don't get any CPU
cycles.
Is there a recommended way to do something like this?
Please note, I do not want to use a periodic timer because I don't know how
long handling a message will take. Plus, if there are more items in the
queue, I don't want to wait around to get them. My thought is to generalize
this as a db queue eventually.
_______________________________________________
Backgroundrb-devel mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/backgroundrb-devel
