I just updated the schema with documentation that should help everyone
understand how to set up the Job Manager/Job Poller. It seems to me it
can accommodate the scenario you described.
-Adrian
On 8/13/2012 2:42 AM, Brett Palmer wrote:
*Adrian,
I think the single JobPoller is a good idea and reduces the chance of too
many JobPoller’s running on a machine.
We often setup multiple delegators to communicate with different databases.
For example, our data warehouse is hosted on a separate server. These
databases usually have a full ofbiz schema on them (jobsandbox table, etc).
Here is how our data warehouse process works:
The application has several ofbiz servers talking to a primary database.
These servers contain all the user information for our application. When
a person logs into the application they are redirected to a secondary ofbiz
server that is used for running the application under heavy loads. The
data is captured on the secondary server.
A data warehouse process is scheduled to run every 5 mins on these
secondary servers. The secondary servers have a delegator that talks to
its local database and a delegator to talk to the data warehouse.
With the new job poller changes would the poller pick up jobs from the data
warehouse database since it has a delegator that points to that instance?
For this example, we would need to make sure the job poller on the
secondary server only serviced jobs from its local database (default
delegator) and not the our configured olap delegator.
Let me know if this will be possible with the new job poller or if you have
any questions from my scenario. I realize this scenario is not the typical
ofbiz e-commerce type of setup everyone is use to, but we use ofbiz to
create lots of different types applications and have found it very flexible
for creating just about any type of ERP application.
Thanks for your work on the job poller.
Brett *
On Sun, Aug 12, 2012 at 5:36 AM, Adrian Crum <
adrian.c...@sandglass-software.com> wrote:
On 8/5/2012 1:21 PM, Adrian Crum wrote:
On 8/5/2012 11:02 AM, Adrian Crum wrote:
I just committed a bunch of changes to the Job Manager group of classes.
The changes help simplify the code and hopefully make the Job Manager more
robust. On the other hand, I might have broken something. ;) I will monitor
the mailing list for problems.
I believe the JobPoller settings in serviceengine.xml (the <thread-pool>
element) should be changed. I think min-threads should be set to "2" and
max-threads should be set to "5". Creating lots of threads can hurt
throughput because the JVM spends more time managing them. I would be
interested in hearing what others think.
Thinking about this more, there are some other things that need to be
fixed:
1. The JobPoller uses an unbounded queue. In a busy server, there is the
potential the queue will grow in size until it causes an out-of-memory
condition.
2. There is no accommodation for when a job cannot be added to the queue
- it is just lost. We could add a dequeue method to the Job interface that
will allow implementations to recover or reschedule the job when it can't
be added to the queue.
3. There is a JobPoller instance per delegator, and each instance
contains the number of threads configured in serviceengine.xml. With the
current max-threads setting of 15, a multi-tenant installation with 100
tenants will create up to 1500 threads. (!!!) A smarter strategy might be
to have a single JobPoller instance that services multiple JobManagers.
I fixed #1 and #2. I am considering working on #3, but I want some
feedback first.
A JobPoller instance is created for each delegator. So, in a multi-tenant
or multi-delegator scenario, multiple JobPollers will be created - which
means one job queue per delegator and (threads per queue) threads per
delegator. In a multi-server installation, things are multiplied: (# of
servers * # of delegators) job queues. Fortunately, in that scenario we can
disable the JobPoller on all but one server.
So, we are left with the potential problem of too many queues/threads
being created on a multi-delegator or multi-tenant server. So, I think
there should be one JobPoller instance that services all delegators. At
each polling interval, the JobPoller gets a list of jobs from each
delegator (JobManager) - creating a list of lists. Then the JobPoller
creates a queue candidate list from the list of lists - using a round-robin
approach so each delegator gets an equal opportunity to queue jobs. The
JobPoller queues the candidate list, and any candidates that don't fit in
the queue are rescheduled. With this approach the JobPoller can service any
number of delegators without saturating the server.
What do you think?
-Adrian
