On 8/23/2012 8:46 AM, Adrian Crum wrote:
On 8/22/2012 7:04 PM, Brett Palmer wrote: We need this functionality
for our data warehouse processing. We try to
provide real time reports but our database cannot handle a high
number of
data warehouse updates during heavy loads. By configuring only one
server
to service a particular pool we can limit the number of concurrent
processes running those services.
<thread-pool send-to-pool="pool"
purge-job-days="4"
failed-retry-min="3"
ttl="120000"
jobs="100"
min-threads="2"
max-threads="5"
wait-millis="1000"
poll-enabled="true"
poll-db-millis="30000">
<run-from-pool name="pool"/>
<run-from-pool name="dwPool"/>
</thread-pool>
That configuration will work. That server will service the two pools.
I forgot to mention, if you're running lots of jobs, then you will want
to increase the jobs (queue size) value. You mentioned in another thread
that your application will run up to 10,000 jobs - in that case you
should increase the jobs value to 1000 or more. The queue size affects
memory, so there is an interaction between responsiveness and memory use.
The potential problem with the Job Poller (before and after the
overhaul) is with asynchronous service calls (not scheduled jobs). When
you run an async service, the service engine converts the service call
to a job and places it in the queue. It is not persisted like scheduled
jobs. If the Job Poller has just filled the queue with scheduled jobs,
then there is no room for async services, and any attempt to queue an
async service will fail (throws an exception "Unable to queue job").
I designed the new code so the service engine can check for that
possibility, but I didn't change the service engine behavior. Instead,
users should configure their <thread-pool> element(s) and applications
carefully. For example, if your application schedules lots of jobs, then
design it in a way that it schedules no more than (queue size - n) jobs
at a time - to leave room for async services. Another option would be to
have a server dedicated to servicing scheduled jobs - that way the
potential clash with async services is not an issue.
-Adrian
