I actually achieved my solution with activation directive, and such a configuration works. The only drawback is:
When PROD recovers all traffic is managed by DR until DR goes down, and my case would be to take back all traffic after PROD is recovered. Milos -----Original Message----- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: 18 October 2016 11:33 To: Tomcat Users List Subject: Re: Load balancing problem with activation=disabled Am 18.10.2016 um 10:10 schrieb Kozak, Milos: > Hi, > > I am debugging a mod_jk load-balancing configuration which has been used a > lot, but for two nodes only. Currently, we made a change for more nodes, and > we are facing problem. > > Original idea was to have one PROD and DR servers such that all requests are > handled by PROD and if PROD goes down DR takes over. In order to do that we > used activation=disabled for DR worker, such that: > > > worker.list=jkstatus,lbhierarchy > worker.jkstatus.type=status > worker.lbhierarchy.type=lb > worker.lbhierarchy.balance_workers=hierarchy-1,hierarchy-2 > > worker.hierarchy-1.type=ajp13 > worker.hierarchy-1.host=host1 > worker.hierarchy-1.port=8009 > worker.hierarchy-1.socket_timeout=0 > worker.hierarchy-1.socket_keepalive=False > worker.hierarchy-1.retries=2 > worker.hierarchy-1.connection_pool_timeout=0 > worker.hierarchy-1.lbfactor=1 > worker.hierarchy-1.redirect=hierarchy-2 > > worker.hierarchy-2.type=ajp13 > worker.hierarchy-2.host=host2 > worker.hierarchy-2.port=8009 > worker.hierarchy-2.socket_timeout=0 > worker.hierarchy-2.socket_keepalive=False > worker.hierarchy-2.retries=2 > worker.hierarchy-2.connection_pool_timeout=0 > worker.hierarchy-2.lbfactor=1 > worker.hierarchy-2.activation=disabled > > > However, the current demand is to have four servers which are chained. > Basically, we try to have one PROD and 3 DR servers. Each DR is activated > when the previous worker goes down. Therefore, we prepared configuration like > this: > > > > worker.list=jkstatus,lbhierarchy,lb2hierarchy > worker.jkstatus.type=status > worker.lbhierarchy.type=lb > worker.lbhierarchy.balance_workers=hierarchy-1,hierarchy-2,hierarchy-3,hierarchy-4 > > worker.hierarchy-1.type=ajp13 > worker.hierarchy-1.host=host > worker.hierarchy-1.port=8009 > worker.hierarchy-1.socket_timeout=0 > worker.hierarchy-1.socket_keepalive=False > worker.hierarchy-1.retries=2 > worker.hierarchy-1.connection_pool_timeout=0 > worker.hierarchy-1.lbfactor=1 > worker.hierarchy-1.redirect=hierarchy-2 > > worker.hierarchy-2.type=ajp13 > worker.hierarchy-2.host=host > worker.hierarchy-2.port=8010 > worker.hierarchy-2.socket_timeout=0 > worker.hierarchy-2.socket_keepalive=False > worker.hierarchy-2.retries=2 > worker.hierarchy-2.connection_pool_timeout=0 > worker.hierarchy-2.lbfactor=1 > worker.hierarchy-2.activation=disabled > worker.hierarchy-2.redirect=hierarchy-3 > > worker.hierarchy-3.type=ajp13 > worker.hierarchy-3.host=host > worker.hierarchy-3.port=8011 > worker.hierarchy-3.socket_timeout=0 > worker.hierarchy-3.socket_keepalive=False > worker.hierarchy-3.retries=2 > worker.hierarchy-3.connection_pool_timeout=0 > worker.hierarchy-3.lbfactor=1 > worker.hierarchy-3.activation=disabled > worker.hierarchy-3.redirect=hierarchy-4 > > worker.hierarchy-4.type=ajp13 > worker.hierarchy-4.host=host12 > worker.hierarchy-4.port=10603 > worker.hierarchy-4.socket_timeout=0 > worker.hierarchy-4.socket_keepalive=False > worker.hierarchy-4.retries=2 > worker.hierarchy-4.connection_pool_timeout=0 > worker.hierarchy-4.lbfactor=1 > worker.hierarchy-4.activation=disabled > worker.hierarchy-4.redirect=hierarchy-1 > > Initially, 3 servers are disabled and redirect is specified. > > Problem occurs when hierarchy-3 worker goes down because hierarchy-4 never > gets initiated and mod_jk log says that: > All tomcat instances failed, no more workers left > > However, workers list is like this: > worker.lbhierarchy.balance_workers=hierarchy-1,hierarchy-2,hierarchy-3,hierarchy-4 > > Which means we have got more than 3 workers hence we have more workers > left... Here is log where I tried to take down workers one by one: > > [Tue Oct 18 09:51:30.623 2016] [31890:139909801125856] [info] > ajp_service::jk_ajp_common.c (2773): (hierarchy-1) sending request to tomcat > failed (recoverable), because of error during request sending (attempt=1) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > jk_open_socket::jk_connect.c (817): connect to HOST:8009 failed (errno=111) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (1068): (hierarchy-1) Failed opening > socket to (HOST:8009) (errno=111) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [error] > ajp_send_request::jk_ajp_common.c (1728): (hierarchy-1) connecting to backend > failed. Tomcat is probably not started or is listening on the wrong port > (errno=111) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > ajp_service::jk_ajp_common.c (2773): (hierarchy-1) sending request to tomcat > failed (recoverable), because of error during request sending (attempt=2) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [error] > ajp_service::jk_ajp_common.c (2794): (hierarchy-1) connecting to tomcat > failed (rc=-3, errors=5, client_errors=0). > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1595): service failed, worker hierarchy-1 is in > error state > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > jk_open_socket::jk_connect.c (817): connect to HOST:8010 failed (errno=111) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (1068): (hierarchy-2) Failed opening > socket to (HOST:8010) (errno=111) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [error] > ajp_send_request::jk_ajp_common.c (1728): (hierarchy-2) connecting to backend > failed. Tomcat is probably not started or is listening on the wrong port > (errno=111) > [Tue Oct 18 09:51:30.724 2016] [31890:139909801125856] [info] > ajp_service::jk_ajp_common.c (2773): (hierarchy-2) sending request to tomcat > failed (recoverable), because of error during request sending (attempt=1) > [Tue Oct 18 09:51:30.824 2016] [31890:139909801125856] [info] > jk_open_socket::jk_connect.c (817): connect to HOST:8010 failed (errno=111) > [Tue Oct 18 09:51:30.824 2016] [31890:139909801125856] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (1068): (hierarchy-2) Failed opening > socket to (HOST:8010) (errno=111) > [Tue Oct 18 09:51:30.824 2016] [31890:139909801125856] [error] > ajp_send_request::jk_ajp_common.c (1728): (hierarchy-2) connecting to backend > failed. Tomcat is probably not started or is listening on the wrong port > (errno=111) > [Tue Oct 18 09:51:30.824 2016] [31890:139909801125856] [info] > ajp_service::jk_ajp_common.c (2773): (hierarchy-2) sending request to tomcat > failed (recoverable), because of error during request sending (attempt=2) > [Tue Oct 18 09:51:30.824 2016] [31890:139909801125856] [error] > ajp_service::jk_ajp_common.c (2794): (hierarchy-2) connecting to tomcat > failed (rc=-3, errors=5, client_errors=0). > [Tue Oct 18 09:51:30.825 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1595): service failed, worker hierarchy-2 is in > error state > [Tue Oct 18 09:51:30.825 2016] [31890:139909801125856] [info] > jk_open_socket::jk_connect.c (817): connect to HOST:8011 failed (errno=111) > [Tue Oct 18 09:51:30.825 2016] [31890:139909801125856] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (1068): (hierarchy-3) Failed opening > socket to (HOST:8011) (errno=111) > [Tue Oct 18 09:51:30.825 2016] [31890:139909801125856] [error] > ajp_send_request::jk_ajp_common.c (1728): (hierarchy-3) connecting to backend > failed. Tomcat is probably not started or is listening on the wrong port > (errno=111) > [Tue Oct 18 09:51:30.825 2016] [31890:139909801125856] [info] > ajp_service::jk_ajp_common.c (2773): (hierarchy-3) sending request to tomcat > failed (recoverable), because of error during request sending (attempt=1) > [Tue Oct 18 09:51:30.925 2016] [31890:139909801125856] [info] > jk_open_socket::jk_connect.c (817): connect to HOST:8011 failed (errno=111) > [Tue Oct 18 09:51:30.925 2016] [31890:139909801125856] [info] > ajp_connect_to_endpoint::jk_ajp_common.c (1068): (hierarchy-3) Failed opening > socket to (HOST:8011) (errno=111) > [Tue Oct 18 09:51:30.925 2016] [31890:139909801125856] [error] > ajp_send_request::jk_ajp_common.c (1728): (hierarchy-3) connecting to backend > failed. Tomcat is probably not started or is listening on the wrong port > (errno=111) > [Tue Oct 18 09:51:30.925 2016] [31890:139909801125856] [info] > ajp_service::jk_ajp_common.c (2773): (hierarchy-3) sending request to tomcat > failed (recoverable), because of error during request sending (attempt=2) > [Tue Oct 18 09:51:30.925 2016] [31890:139909801125856] [error] > ajp_service::jk_ajp_common.c (2794): (hierarchy-3) connecting to tomcat > failed (rc=-3, errors=5, client_errors=0). > [Tue Oct 18 09:51:30.925 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1595): service failed, worker hierarchy-3 is in > error state > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1664): All tomcat instances failed, no more workers > left (attempt=0, retry=1) > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1664): All tomcat instances failed, no more workers > left (attempt=1, retry=1) > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1664): All tomcat instances failed, no more workers > left (attempt=2, retry=1) > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1664): All tomcat instances failed, no more workers > left (attempt=3, retry=1) > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [info] > service::jk_lb_worker.c (1675): All tomcat instances are busy or in error > state > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [error] > service::jk_lb_worker.c (1680): All tomcat instances failed, no more workers > left > [Tue Oct 18 09:51:31.026 2016] [31890:139909801125856] [info] > jk_handler::mod_jk.c (2991): Service error=0 for worker=lbhierarchy > > > I tested similar configuration for 5 workers, and I never got over worker 3.. > > > We are using : > Server Version: Apache/2.2.15 (Unix) mod_jk/1.2.41 > JK Version: mod_jk/1.2.41 > > > I don't know whether is this a bug or did I miss something in documentation? > Thanks for a help! Look at "distance" instead of activation. Set all workers active and use distance values, e.g. "0" for all "normal workers", "1" for the first DR group, "2" etc. See: http://tomcat.apache.org/connectors-doc/reference/workers.html Regards, Rainer --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org ________________________________ Právní informace: Tento e-mail a všechny připojené soubory jsou důvěrné a mohou být chráněny zákonem a jsou určeny pouze oprávněným adresátům. Pokud jste obdržel/a tento e-mail omylem, oznamte to, prosím, neprodleně jeho odesílateli a pak jej vymažte. Tento e-mail ani připojené soubory nejsou návrhem na uzavření smlouvy ani jeho přijetím, ledaže jsou tak výslovně označeny. Commerzbank neručí za bezchybný a úplný přenos zasílaných informací, ani za zpoždění nebo přerušení přenosu a ani za škody způsobené použitím nebo důvěrou v tyto informace. Legal information: This e-mail message and all attached files are confidential, may be protected under the law, and are intended for authorized recipients only. If you have received this e-mail message in error, please notify the sender immediately and subsequently delete the message. This e-mail message and the attached files do not constitute a proposal to enter into an agreement or the acceptance of such a proposal, unless expressly designated as such. COMMERZBANK gives no guarantee that the transmitted information is error-free and complete. COMMERZBANK will not be held liable for any transmission delay or interruption, nor for any loss incurred due to the use of, or reliance on, the transmitted information. COMMERZBANK Aktiengesellschaft, pobočka Praha, se sídlem Praha 2, Jugoslávská 1, 120 21, IČO: 47 61 09 21, zapsaná v Obchodním rejstříku u Městského soudu v Praze, v oddílu A, vložka 7341, pobočka COMMERZBANK Aktiengesellschaft, Frankfurt nad Mohanem, Německo. Povinné údaje: http://www.commerzbank.de/pflichtangaben COMMERZBANK Aktiengesellschaft, pobočka Praha, with its registered office: Praha 2, Jugoslávská 1, 120 21, registered number: 47610921, entered in the Commercial Register at the Municipal Court Prague, Section A, Entry 7341, branch of COMMERZBANK Aktiengesellschaft, Frankfurt am Main, Germany. Mandatory information: http://www.commerzbank.com/pflichtangaben
