Hi Brett,
From: "Brett Palmer" <brettgpal...@gmail.com>
Adrian,
Thanks for the update. Here are some feedback points on your listed items:
1. JobPoller get out-of-memor error. We've seen this a lot in production
servers when the JobSandbox table is not constantly pruned of old records.
It would be nice if the poller restricted its search for only active
records it could process.
Did you use the purge-job-days setting in serviceengine.xml and the related
purgeOldJobs? If not was there a reason?
2. Queue for capturing missing records would be good. From item 1 above we
have had locks on table when the poller is busy doing a scan and new jobs
cannot be added or time out.
+1
Other wish items:
- Ability to assign different service engines to process specific job
types. We often multiple application servers but want to limit how many
concurrent jobs are run. For example, if I had 4 app servers connected to
the same DB I may only want one app server to service particular jobs. I
thought this feature was possible but when I tried to implement it by
changing some of the configuration files it never worked correctly.
Las time I used this it was with R4.0 and it worked, which problems did you
cross exactly (if you remember) ?
Thanks
- JMS support for the service engine. It would be nice if there was a JMS
interface for those that want to use JMS as their queuing mechanism for
jobs.
+1
Jacques
Brett
On Sun, Aug 5, 2012 at 6:21 AM, Adrian Crum <
adrian.c...@sandglass-software.com> wrote:
On 8/5/2012 11:02 AM, Adrian Crum wrote:
I just committed a bunch of changes to the Job Manager group of classes.
The changes help simplify the code and hopefully make the Job Manager more
robust. On the other hand, I might have broken something. ;) I will monitor
the mailing list for problems.
I believe the JobPoller settings in serviceengine.xml (the <thread-pool>
element) should be changed. I think min-threads should be set to "2" and
max-threads should be set to "5". Creating lots of threads can hurt
throughput because the JVM spends more time managing them. I would be
interested in hearing what others think.
Thinking about this more, there are some other things that need to be
fixed:
1. The JobPoller uses an unbounded queue. In a busy server, there is the
potential the queue will grow in size until it causes an out-of-memory
condition.
2. There is no accommodation for when a job cannot be added to the queue -
it is just lost. We could add a dequeue method to the Job interface that
will allow implementations to recover or reschedule the job when it can't
be added to the queue.
3. There is a JobPoller instance per delegator, and each instance contains
the number of threads configured in serviceengine.xml. With the current
max-threads setting of 15, a multi-tenant installation with 100 tenants
will create up to 1500 threads. (!!!) A smarter strategy might be to have a
single JobPoller instance that services multiple JobManagers.
-Adrian
