Weird.
I just updated my environment (on Fedora 13) to support 4 Tomcats and it worked
as expected. I verified this by running my test application, watching traffic
via jk-manager, and then shutting down (disabling) one mod_jk connection at a
time. Clustering worked, farm deployment worked, and the application (such as
it
is) worked.
Here are the particulars:
OS: Fedora 13
JRE: 1.6.0_21 (32 bit)
Apache: 2.2.16 (comes with OS - not the latest)
Tomcat: 6.0.29
mod_jk: 1.2.30 (compiled locally)
My workers.properties file (without all the comments)
worker.list=jk-status,jk-manager,lb
worker.jk-status.type=status
worker.jk-status.read_only=true
worker.jk-manager.type=status
worker.template.type=ajp13
worker.template.host=192.168.0.254
worker.template.socket_connect_timeout=5000
worker.template.socket_keepalive=true
worker.template.ping_mode=A
worker.template.ping_timeout=10000
worker.template.connection_pool_minsize=0
worker.template.connection_pool_timeout=600
worker.template.reply_timeout=300000
worker.template.recovery_options=3
worker.deimos.reference=worker.template
worker.deimos.port=8019
worker.mars.reference=worker.template
worker.mars.port=8029
worker.phobos.reference=worker.template
worker.phobos.port=8039
worker.xerxes.reference=worker.template
worker.xerxes.port=8049
worker.lb.type=lb
worker.lb.error_escalation_time=0
worker.lb.max_reply_timeouts=10
worker.lb.balance_workers=deimos,mars,phobos,xerxes
My uriworkermap.properties file is next. The RPets application is a random pet
generator I use for testing clustering.
/examples=lb
/examples/*=lb
/docs=lb
/docs/*=lb
/RPets=lb
/RPets/*=lb
mod_jk is added as an included file to httpd.conf. Here is my configuration.
LoadModule jk_module modules/mod_jk.so
<IfModule jk_module>
JkWorkersFile conf.d/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel info
JkShmFile /var/run/httpd/mod_jk.shm
JkOptions +RejectUnsafeURI
JkWatchdogInterval 60
<Location /jk-status>
JkMount jk-status
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Location>
<Location /jk-manager>
JkMount jk-manager
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Location>
JkMountFile conf.d/uriworkermap.properties
</IfModule>
Here is one (of four) server.xml files stripped of comments and some
miscellaneous global resources. I apologize for any line wrapping. Some of the
package names are quite long.
<?xml version='1.0' encoding='utf-8'?>
<Server port="8015" shutdown="SHUTDOWN">
<Listener className="org.apache.catalina.core.AprLifecycleListener"
SSLEngine="on" />
<Listener className="org.apache.catalina.core.JasperListener" />
<Listener
className="org.apache.catalina.core.JreMemoryLeakPreventionListener"
/>
<Listener className="org.apache.catalina.mbeans.ServerLifecycleListener" />
<Listener
className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
<GlobalNamingResources>
<Resource name="UserDatabase" auth="Container"
type="org.apache.catalina.UserDatabase"
description="User database that can be updated and saved"
factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
pathname="conf/tomcat-users.xml" />
</GlobalNamingResources>
<Service name="Catalina">
<Connector port="8090" protocol="HTTP/1.1"
connectionTimeout="20000"
redirectPort="8453"/>
<Connector port="8019" protocol="AJP/1.3" redirectPort="8453"/>
<Engine name="Catalina" defaultHost="localhost" jvmRoute="deimos">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="6">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership
className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="auto"
port="4000"
selectorTimeout="100"
maxThreads="6"/>
<Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
<Interceptor
className="org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.*\.txt;"/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="${catalina.base}/temp-dir/"
deployDir="${catalina.base}/webapps/"
watchDir="${catalina.base}/watch-dir/"
processDeployFrequency="4"
watchEnabled="true"/>
<ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="logs"
prefix="localhost_access_log." suffix=".txt" pattern="common"
resolveHosts="false"/>
</Host>
</Engine>
</Service>
</Server>
It's all pretty generic. This particular instance happens to be the one that
provides the other three with the web application, so I have
watcheEnabled="true" for the Deployer. In general here's what you have to watch
out for when running all of this on one machine.
1. Change ports for everything
shutdown port
HTTP port (if used - I use it for management)
HTTPS port (if used)
AJP port
Receiver port in cluster (not strictly necessary since Tomcat will figure
this out)
2. Match jvmroute with names used in workers.properties
3. Match the AJP ports between server.xml and workers.properties
I run this using $CATALINA_HOME and $CATALINA_BASE (%CATALINA_HOME% and
%CATALINA_BASE% on Windows) plus cluster-startup.sh and a cluster-shutdown.sh
scripts.
I haven't tried this on Windows yet, so the issue may be with shared memory
allocation.
Take a close look at your workers.properties files and compare it to the one
above (which is based on the stock one provided with the tomcat-connectors
source package).
Hope this helps.
. . . . just my two cents.
/mde/
----- Original Message ----
From: Christopher Schultz <[email protected]>
To: Tomcat Users List <[email protected]>
Sent: Thu, September 9, 2010 11:15:14 AM
Subject: Re: Apache AJP to 4 TCs
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Andrew,
On 9/9/2010 2:00 AM, Andrew Bruno wrote:
> [Wed Sep 08 22:57:36 2010] [6912:6336] [debug] jk_worker.c (242):
> creating worker loadbalancer
> [Wed Sep 08 22:57:36 2010] [6912:6336] [debug] jk_worker.c (146):
> about to create instance loadbalancer of lb
> [Wed Sep 08 22:57:36 2010] [6912:6336] [debug] jk_worker.c (159):
> about to validate and init loadbalancer
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_lb_worker.c (1536):
> allocating lb sub worker record from shared memory
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_worker.c (163):
> validate failed for loadbalancer
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_worker.c (262):
> failed to create worker loadbalancer
D'oh. In your original post, did you include your entire
workers.properties file? Also, can you post all your Jk* directives you
have active in your httpd configuration?
The code that performs validation of the worker is found in
native/common/jk_lb_worker.c:
static int JK_METHOD validate(jk_worker_t *pThis,
jk_map_t *props,
jk_worker_env_t *we, jk_logger_t *l)
> [Wed Sep 08 22:57:36 2010] [6912:6336] [error] jk_lb_worker.c (1536):
> allocating lb sub worker record from shared memory
This occurs on line 1536 (in mod_jk 1.2.30), in this neighborhood:
> 1533 for (i = 0; i < num_of_workers; i++) {
> 1534 p->lb_workers[i].s =
jk_shm_alloc_lb_sub_worker(&p->p);
> 1535 if (p->lb_workers[i].s == NULL) {
> 1536 jk_log(l, JK_LOG_ERROR,
> 1537 "allocating lb sub worker record from
> shared
>memory");
> 1538 JK_TRACE_EXIT(l);
> 1539 return JK_FALSE;
> 1540 }
> 1541 }
This looks like it might be a shared memory allocation error. Once you
get into the jk_shm and jk_pool code, there's no debug logging of any
kind :(
You might have to hack-in some debug logging if Rainer or Mladen can't
comment further. Are you comfortable compiling your own mod_jk? It looks
like compiling with DEBUG #defined to something will at least get you
jk_dump_pool calls when shared memory fails to be allocated.
- -chris
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
