Thanks Hemant, you are absolutely correct about threads being too much
overhead for what I need so this looks like a perfect solution!
Cheers,
Chad.
On 24 Jan 2009, at 01:45, hemant wrote:
On Sat, Jan 24, 2009 at 1:46 AM, Chad Thatcher <[email protected]
> wrote:
Thanks Hemant,
Yes I was specifically trying to avoid threads. However, I am
guessing I
could configure it so that only one thread ran at one time right?
That
would cut down on the complexity.
Yes, changing thread pool size to 1 usually helps. Libraries like
mechanize (even Net/HTTP) crap out under concurrent threads but if
your pool size is 1, you won't have these problems, thread pool will
be just a queue. But if you don't want any concurrency, why not just
do something like this:
class FooWorker < BackgrounDRb::MetaWorker
def create(args = nil)
@task_array = []
# check task array for pending tasks and run them
add_periodic_timer(2) { run_from_task_array() }
end
def accept_indexing(args = nil)
klass = Struct.new(:cmd_args)
# add task to the task array
@task_array << klass.new(args)
end
def run_from_task_array
task = @task_array.shift
# run the task
end
def check_count
return @task_array.count
end
end
if you don't want threads and above code will work too. Specially
since thread pool uses Queue data structure from ruby stdlib which is
quite heavyweight and thread safe, but you don't want any thread
safety here, so why pay additional penalty for that? All your tasks
run in same main thread.
Is the enq_xxx stuff only time based? If so I wonder if it would be
difficult to add a feature that allows you to add a whole bunch of
tasks to
the queue and then they just run one after the other in the order
in which
they were added. Maybe I could intercept persistent_job#finished!
to find
the next (grouped) task in the queue and update it to start a few
seconds in
the future so the bgrb picks it up and runs it etc. But that seems
evil
somehow :).
You can totally do that! Check add_timer or add_periodic_timer
I realise this might not be the target usage for backgroundrb but I
look
forward to hearing your thoughts because I'm hoping it won't be too
difficult for me to do. The ultimate fall-back, of course, is to
roll my
own task queue that uses a table similar table to bdrb_job_queues
but that
seems wasteful when there is so much there for it already.
Right.
On 23 Jan 2009, at 18:57, hemant wrote:
That data is in write Queue of master worker. It can't be
meaningfully
extracted to provide how many tasks are in the queue. Put it other
way, its not even in a Queue in traditional sense. Data is just
there
in a buffer so that master will write it to the socket when socket
is
ready for write.
If you want Queue. You will have either use enq_xxx stuff or
provide a
sugar around asyc_index() method so as actual method is not directly
called but is added to the thread pool (thread_pool.defer), this way
you can query how many tasks are there in thread pool queue.
However word "thread" comes with its own meaning and associated
crapiness in ruby, so beware.
On Fri, Jan 23, 2009 at 6:35 PM, Chad Thatcher <[email protected]
>
wrote:
Hello everyone.
I am making several asynchronous calls to one particular method
in my
worker
like this:
MiddleMan.worker(:reindex_worker).async_reindex(:arg => options)
If I call this several times in a row, all subsequent calls are
put in a
queue and I would like to be able to keep track of this. The
behavior is
perfect because I only ever want reindex() running once at any
one time
but
I would like to be able to show my users where in the queue their
request
is. Is there any way to view what async calls are lined up? The
worker_info stuff doesn't seem to give away any details like this.
I also thought about using "enq_" to schedule but is there a way to
specify
that the trigger is the completion of the previous scheduled
instance and
so
on? Or is the scheduling purely time based?
Kind regards,
Chad.
_______________________________________________
Backgroundrb-devel mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/backgroundrb-devel
--
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with
my
own coals.
http://gnufied.org
_______________________________________________
Backgroundrb-devel mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/backgroundrb-devel
--
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.
http://gnufied.org
_______________________________________________
Backgroundrb-devel mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/backgroundrb-devel
