River-344 - replacing TaskManager in SDM

2014-01-24 Thread Peter Firmstone
Notes: ServiceDiscoveryManager NotifyEventTask If the task list contains any RegisterListenerTasks or LookupTasks associated with this task's lookup service (ProxyReg), and if those tasks were queued prior to this task (have lower sequence numbers), then run those tasks before

Replacing TaskManager with ExecutorService

2014-01-13 Thread Peter Firmstone
Two emails are worth reflecting on, as is River-344, this relates to replacing TaskManager with ExecutorService. http://mail-archives.apache.org/mod_mbox/river-dev/201107.mbox/%3cbb4ad312-53c1-4ce6-9bff-01e5cc344...@sorcersoft.org%3e http://mail-archives.apache.org/mod_mbox/river-dev/201105

Re: TaskManager as an ExecutorService

2014-01-08 Thread Gregg Wonderly
. Gregg Wonderly On Jan 4, 2014, at 7:46 PM, Peter j...@zeus.net.au wrote: For a moment, lets consider how we might fix TaskManager, while retaining the existing Task.runAfter method and how the fix would impact TaskManager's users. TaskManager, like most thread pools has a queue

Re: TaskManager as an ExecutorService

2014-01-08 Thread Peter
: For a moment, lets consider how we might fix TaskManager, while retaining the existing Task.runAfter method and how the fix would impact TaskManager's users. TaskManager, like most thread pools has a queue. Before a task is removed from the queue for execution, it is asked

Fixing ServiceDiscoveryManager WAS: Re: TaskManager as an ExecutorService

2014-01-06 Thread Peter
of the API in non-programatic design ways. Gregg Wonderly         On Jan 4, 2014, at 7:46 PM, Peter j...@zeus.net.au wrote: For a moment, lets consider how we might fix TaskManager, while retaining the existing Task.runAfter method and how the fix would impact TaskManager's

TaskManager as an ExecutorService

2014-01-04 Thread Peter Firmstone
Would you like me to add this class, so that existing configurations utilising a TaskManager can also be used? This might be useful for retaining backward compatibility with existing configurations? Regards, Peter. /* * Licensed to the Apache Software Foundation (ASF) under one * or more

Re: TaskManager as an ExecutorService

2014-01-04 Thread Greg Trasuk
I’d like you to make a reasonable case for why TaskManager needs to be replaced, requiring changes to many other classes that depend on TaskManager, rather than stating what the problem is with TaskManager and seeking to fix it, which would only affect TaskManager and not require modifying

Re: TaskManager as an ExecutorService

2014-01-04 Thread Peter
The problem is with TaskManager's public api method Task.runAfter. This is well documented in River-344. The fix requires changing every class that uses it As a thread pool TaskManager is correct provided that no ordering dependencies exist between tasks. TaskManager doesn't compare to Doug

Re: What is wrong with TaskManager?

2013-04-09 Thread Mark Brouwer
Hi Patricia, On 4/8/13 4:32 PM, Patricia Shanahan wrote: Thanks for the background. The general performance of TaskManager, and especially the overhead for runAfter, seemed to me to depend strongly on the queue length. For example, a non-trivial runAfter method scans the list of tasks older

Re: What is wrong with TaskManager?

2013-04-08 Thread Patricia Shanahan
on them to arrange for it themselves.. I concluded runAfter is *not evil* by itself when properly used, although it gives some overhead and it has to perform its work while a lock has been held. ... Thanks for the background. The general performance of TaskManager, and especially the overhead

Re: test failure repeatability - TaskManager

2013-04-05 Thread Peter
probably consider each case individually, I noticed there's a configuration property that allows a TaskManager instance to be injected on a number of occasions too, which suggests there might be some sharing. Peter. On 4/3/2013 2:04 PM, Dan Creswell wrote: I'm with you. My first step was going

Re: test failure repeatability - TaskManager

2013-04-03 Thread Peter
Gut feeling suggests the solution will be executor based, so you're asking good questions, I think we need to understand the use cases better and probably redesign dependant code too. One example of retry, the task will continue attemtping to retry for an entire day. We might need some kind

Re: test failure repeatability - TaskManager

2013-04-03 Thread Patricia Shanahan
I agree with the idea of understanding the use cases before designing the solution, and with using standard API classes as much as possible. The table I sent you was intended as a first step towards that. I'm not convinced that the right solution is a single TaskManager successor. Different

Re: test failure repeatability - TaskManager

2013-04-03 Thread Dan Creswell
with the idea of understanding the use cases before designing the solution, and with using standard API classes as much as possible. The table I sent you was intended as a first step towards that. I'm not convinced that the right solution is a single TaskManager successor. Different TaskManager instances

Re: test failure repeatability - TaskManager

2013-04-02 Thread Patricia Shanahan
is a RetryTask, taskB is supposed to runAfter taskA, and originally appears on the queue after taskA. The first attempt at taskA fails, and it puts itself back on the TaskManager queue, using the normal add call. Now taskA is after taskB. Patricia On 4/2/2013 12:38 AM, Peter Firmstone wrote

Re: test failure repeatability - TaskManager

2013-04-02 Thread Peter Firmstone
, and it puts itself back on the TaskManager queue, using the normal add call. Now taskA is after taskB. Patricia On 4/2/2013 12:38 AM, Peter Firmstone wrote: The formatting didn't work out, I'll create a Jira issue to discuss. Patricia's done a great job detailing the dependencies and issues

Re: test failure repeatability - TaskManager

2013-04-02 Thread Bryan Thompson
I am not clear on the semantics for runAfter, but maybe this can be achieved by wrapping a Runnable within another Runnable such that the 2nd runnable is automatically scheduled after the first has succeeded? Likewise, it is possible to wrap a Runnable in order to automatically retry if it throws

Re: test failure repeatability - TaskManager

2013-04-02 Thread Patricia Shanahan
, because whenever a task finishes the TaskManager has to ask each waiting task whether it still needs to wait for any older task. I wanted to change it so that TaskManager would know which task was being waited for. It could then associate with one task a list of tasks that need to be reconsidered when

RE: TaskManager

2011-06-29 Thread Christopher Dolan
/incubator-river-user/ http://mail-archives.apache.org/mod_mbox/incubator-river-commits/ -Original Message- From: Peter Firmstone [mailto:j...@zeus.net.au] Sent: Sunday, June 26, 2011 5:18 PM To: dev@river.apache.org Subject: Re: TaskManager I think the way to look at this is to examine

TaskManager

2011-06-26 Thread Peter Firmstone
There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting comment from Bob Scheifler: Over the years we've been slowly eliminating use of Task.runAfter, and in any overhaul I'd

Re: TaskManager

2011-06-26 Thread Patricia Shanahan
On 6/26/2011 1:23 PM, Peter Firmstone wrote: There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting comment from Bob Scheifler: Over the years we've been slowly eliminating use

Re: TaskManager

2011-06-26 Thread Peter Firmstone
Patricia Shanahan wrote: On 6/26/2011 1:23 PM, Peter Firmstone wrote: There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting comment from Bob Scheifler: Over the years we've

Re: TaskManager

2011-06-26 Thread Patricia Shanahan
On 6/26/2011 1:58 PM, Peter Firmstone wrote: Patricia Shanahan wrote: On 6/26/2011 1:23 PM, Peter Firmstone wrote: There's an interesting thread of discussion in the Porter mail archive that Mark has uploaded for us (see River-311), relating to the TaskManager. Here's a very interesting