Re: cvs commit: jakarta-tomcat-connectors/jk/xdocs menu.idx style.xsl.in
Henri Gomez wrote: [EMAIL PROTECTED] wrote: jfclere 2002/09/09 08:00:58 Modified:jk/xdocs menu.idx style.xsl.in Log: Add code to allow a dummy document/ in menu.idx: To separte jk and jk2. Excellent, we just need now to pass the subpart name and will be ready ;) I'm still waiting comments on jk load-balancing settings after the commited lb patches from May 2002 (Costin, Bern, Matthias) I got your mail and sent it to you directly. Should I send it to the list too? Bernd -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] msg32976/pgp0.pgp Description: PGP signature
Re: cvs commit: jakarta-tomcat-connectors/jk/native2/common jk_worker_lb.c
[EMAIL PROTECTED] wrote: On Tue, 14 May 2002, Bernd Koecke wrote: Hi Costin, the new patch seems to work, but I'll test it more exactly tomorrow. Then I'll create the patches and the functional description. In short, the patched lb_worker uses an additinal flag on the other workers (e.g worker.ajp13.local_worker=1) to determine if it should be moved to the beginning of the balanced_workers. So we don't need to deal with two lists in lb_worker and the lb_value '0' has no special meaning. The flag for sending requests only to local workers is 'local_worker_only' on the lb_worker. More when the patch is tested and ready. Ok. I already commited part of the changes for jk2 - but my version is called 'hwBalanceErr', on worker_lb. If 0 normal selection of non-local workers takes place if all locals are in error state. If non 0, we'll return the value as the error code - for a front-end balancer to detect and stop forwarding requests for this instance. I think that's the behavior you need - and it also allows customization for the returned error code. That sounds great, many thanks! The patch for jk1 is on the way and I added some explanation how it works and about the two config flags. Bernd Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
[PATCH] for lb_worker in jk1 with local workers
Hi,
here is the patch for lb_worker in jk1. I tested it on a testcluster and it
worked for me.
It adds two config params:
'local_worker_only' on the lb_worker
'local_worker' on the balanced workers, e.g. an ajp13 worker
Example environment:
A cluster with two nodes (TC1+TC2), running apache+tomcat tandem on each node, a
loadbalancer in front of the nodes.
Example conf of TC1:
workers.tomcat_home=tomcat-home-dir
workers.java_home=$(JAVA_HOME)
ps=/
worker.list=router
worker.TC1.port=8009
worker.TC1.host=node1.domain.tld
worker.TC1.type=ajp13
worker.TC1.lbfactor=1
worker.TC1.local_worker=1
worker.TC2.port=8009
worker.TC2.host=node2.domain.tld
worker.TC2.type=ajp13
worker.TC2.lbfactor=1
worker.TC2.local_worker=0
worker.router.type=lb
worker.router.balanced_workers=TC1,TC2
worker.router.local_worker_only=1
The 'local_worker' flag on TC1 and TC2 tells the lb_worker which connections are
going to the local worker. If local_worker is an int and is not 0 it is set to
JK_TRUE and marked as local worker, JK_FALSE otherwise. If in minimum one worker
is marked as local worker, lb_worker is in local worker mode. All local worker
are moved to the beginning of the internal worker list in lb_worker during
validation.
This means that if a request with a session id comes in it would be routed to
the appropriate worker. If this worker is down it will be send to the first
local worker which is not in error state.
If a request without a session comes in, it would be routed to the first local
worker. If all local worker are in error state, then the 'local_worker_only'
flag is important. If it was set to an int and this wasn't 0 it is set to
JK_TRUE, JK_FALSE otherwise. With set to JK_TRUE, this request gets an error
response. If set to JK_FALSE lb_worker tries to route the request to another
balanced worker.
If one of the worker was in error state and has recovered nothing changes. The
local worker will be check for requests without a session id (and with a session
on himself) and the other worker will only be checked if a request with a
session id of this worker comes in.
In this environment, with a load balancer in front, it is an error if the
balancer sends a request without a session to an apache without a running local
worker. And if the looad balancer determines that a node is down no other node
is allowed to send a request without a session to it. This is necessary for me,
because on a switched off node apache and tomcat can still be up and running,
but they are in an old state and should only be asked for old sessions.
Defaults:
local_worker: 0
local_worker_only:0
Internals:
The local workers are at the binning of the worker list. Additionaly I don't
change the lb_value for local workers, but because of the workers order this
should not be necessary.
I didn't changed the name of the local_worker_only flag because it suits the
name with local_worker. But the flags are defines in jk_util.c its easy to
change them.
I hope its usefull, the patch was geenrated against cvs with diff -u.
Bernd
--
Dipl.-Inform. Bernd Koecke
UNIX-Entwicklung
Schlund+Partner AG
Fon: +49-721-91374-0
E-Mail: [EMAIL PROTECTED]
Index: jk_lb_worker.c
===
RCS file: /home/cvspublic/jakarta-tomcat-connectors/jk/native/common/jk_lb_worker.c,v
retrieving revision 1.9
diff -u -r1.9 jk_lb_worker.c
--- jk_lb_worker.c 3 May 2002 23:32:43 - 1.9
+++ jk_lb_worker.c 15 May 2002 08:10:50 -
@@ -84,6 +84,7 @@
char*name;
double lb_factor;
double lb_value;
+int is_local_worker;
int in_error_state;
int in_recovering;
time_t error_time;
@@ -100,6 +101,8 @@
char *name;
jk_worker_t worker;
+int in_local_worker_mode;
+int local_worker_only;
};
typedef struct lb_worker lb_worker_t;
@@ -270,28 +273,29 @@
}
for(i = 0 ; i p-num_of_workers ; i++) {
-if(p-lb_workers[i].in_error_state) {
-if(!p-lb_workers[i].in_recovering) {
-time_t now = time(0);
-
-if((now - p-lb_workers[i].error_time) WAIT_BEFORE_RECOVER) {
-
-p-lb_workers[i].in_recovering = JK_TRUE;
-p-lb_workers[i].error_time = now;
+if (!p-in_local_worker_mode || p-lb_workers[i].is_local_worker ||
+!p-local_worker_only) {
+if(p-lb_workers[i].in_error_state) {
+if(!p-lb_workers[i].in_recovering) {
+time_t now = time(0);
+if((now - p-lb_workers[i].error_time) WAIT_BEFORE_RECOVER) {
+p-lb_workers[i].in_recovering = JK_TRUE;
+p-lb_workers[i].error_time = now;
+rc = (p-lb_workers[i]);
+
+break;
+}
+}
+} else {
+if(p-lb_workers[i
Re: local worker patch for JK1
[EMAIL PROTECTED] wrote: Hi, there is something that bothers me in the patch Bernd sent, that is the local_worker property of workers, I think the concept of local worker is linked with that of load balancing worker, and not to that of worker. Nothing forbids to have an ajp13 worker in several load balancing workers, if the local worker property is linked to the ajp13 worker, the worker will be considered local for every load balancing worker it appears in which is something I think should be avoided. Mathias. Sorry, but I asked for, how to handle this flag yesterday and I got no response. Costin said, that he'll wait for my patch, and I don't want to let him wait for days. If we add a list to the lb_worker, how should this be handled? Lets say it is called 'local_workers'. Should the local workers be in the list of balanced workers too? If yes, I think this makes the config look a little bit unclean. If not, we have to change the validate function more than I want to, because it depends on having balanced workers. And with a second list it is possible to have only local workers. By the way, with the same motivation we should ask about the lb_value. It is not possible to have one worker with different values in different lb_workers. But it may be that one worker is the most powerful in one group (lb_worker) and less powerful in another. Ok normaly the lb_values should be choosen in order to the power of all workers and not because of one group. :) I build the patch for the described simple situation. When I understand jk2 right, this would be the right choice for a more complex environment. Which way should be implemented? We should find one position and implement it then. May be I was a little bit to fast this time :). Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-connectors/jk/native2/common jk_worker_lb.c
[EMAIL PROTECTED] wrote: On Mon, 13 May 2002, Bernd Koecke wrote: Sorry, I must say it again, for my environment it is an error, if a _switched off_ tomcat got a request without a sessionid or with a session from another node. Its not necessary that this tomact-apache tandem is In the current code ( in jk2 ), if a worker is in 'disabled' state it'll only get requests with sessionid, as you need. If it is not disabled, but has a higher level ( == distance ), it'll still not get any new requests unless all closer workers are in error state. update and start them up again. If there are no local/main worker I need an error response and no routing to a switched off tomcat. Its possible that this happens once per day. Setting the non-local workers in disabled state should do that. I know this might be a special environment. I spent some time in jk1 to build a working patch. Than I started looking in jk2. I'm not a good C Your patch looks ok. Would it be possible to remove the use of '0' as a special value, and keep only the main_worker_mode flag for that ? Also, what's the meaning of 'reject' flag ? The '0' as lb_value is needed to determine which are the main/local-workers. If we don't have this special value we need an additional config-flag with a list of the local/main-workers like in Mathias patch. Should I add an additional config-flag (I will take it from Mathias patch) or do we stay with the special '0' value? The reject value of the 'main_worker_mode' flag is for the special behavior not to balance even if no main-worker is up. Without this flag you would send a request to a non main-worker if all main-workers are in error state. When the main-workers are only a preference it might be ok to send a request to a non main-worker and lose only the session but don't send an error response. I think this was what Mathias said. But I need an error response if the main-worker is down. The 'main_worker_mode' is not the same like the 'in_main_worker_mode' var in lb_worker struct. If 'main_worker_mode' flag is set to 'reject' in the workers.properties the reject var of lb_worker struct is set to JK_TRUE. The 'in_main_worker_mode' var of lb_worker struct is set to JK_TRUE if there is in minimum one worker with '0' as lb_value. Also it would be nice to get some documentation for the new settings. Thats no problem, I could write a patch for the HTML-page. Regarding jk2 - I just want to know if the current solution is ok or what are still problems. For now the priority is getting the patch in jk1 so it can be released in 4.0.4 final ( so today or early tommorow I would like to close this issue ). This sounds pretty good, many thanks! Bernd Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-connectors/jk/native2/common jk_worker_lb.c
GOMEZ Henri wrote: The '0' as lb_value is needed to determine which are the main/local-workers. If we don't have this special value we need an additional config-flag with a list of the local/main-workers like in Mathias patch. Should I add an additional config-flag (I will take it from Mathias patch) or do we stay with the special '0' value? It will be very usefull to make a short documentation on latest lb work mode and configuration. Like a descriptive mail to this list, I'll commit mod_jk document accordingly... A short description of my latest patch is in my mail from 6.May 18:23 CEST. If we add a new conf-flag I will test it localy and send a detailed description with the patch. If we don't add a flag I'll send a more detailed description about the latest patch. Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-connectors/jk/native2/common jk_worker_lb.c
[EMAIL PROTECTED] wrote: On Tue, 14 May 2002, Bernd Koecke wrote: The '0' as lb_value is needed to determine which are the main/local-workers. If we don't have this special value we need an additional config-flag with a list of the local/main-workers like in Mathias patch. Should I add an additional config-flag (I will take it from Mathias patch) or do we stay with the special '0' value? I think it would be a good idea, it'll make things cleaner. 'local_worker' would be allways selected, and if 'main_worker_mode' ( or maybe 'hw_lb_mode' ) no fallback will happen. The 'main_worker_mode' is not the same like the 'in_main_worker_mode' var in lb_worker struct. If 'main_worker_mode' flag is set to 'reject' in the workers.properties the reject var of lb_worker struct is set to JK_TRUE. The 'in_main_worker_mode' var of lb_worker struct is set to JK_TRUE if there is in minimum one worker with '0' as lb_value. That's a bit confusing. Maybe some better variable names are needed. 2 flags should be enough - 'local_worker' and 'local_worker_only' ( or something that makes it clear that if the flag is set, no fallback will occur but an error is returned for the hw balancer ). Ok, how should we handle the local_worker list? The current code depends on one worker list. And for requests with a session its easier to look into one list. Is it ok to have the balanced_workers and one ore more of these workers could be in the local_worker list? Then we could leave must of the code in validate function untouched and after getting all the workers we go through the local_worker list, if any, and move the worker from this list at the beginning of the balanced_workers and mark them as local. Would this be ok? Oterwise we have to handle two lists and it would be possible to have only local workers and no balanced_workers. Then the lb_module makes no sense, but it is configurable and we have to deal with this. Another solution is to have two lists in config but only one in lb_worker. But then we have to rewrite most of the code in validate and handle memory etc. You know I'm not so experienced in C, so I would prefere the first suggestion :). Bernd I'll implement the same thing in jk2, but I wait your patch for jk1. Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-connectors/jk/native2/common jk_worker_lb.c
[EMAIL PROTECTED] wrote: [...] I'll implement the same thing in jk2, but I wait your patch for jk1. Hi Costin, the new patch seems to work, but I'll test it more exactly tomorrow. Then I'll create the patches and the functional description. In short, the patched lb_worker uses an additinal flag on the other workers (e.g worker.ajp13.local_worker=1) to determine if it should be moved to the beginning of the balanced_workers. So we don't need to deal with two lists in lb_worker and the lb_value '0' has no special meaning. The flag for sending requests only to local workers is 'local_worker_only' on the lb_worker. More when the patch is tested and ready. Bernd Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: cvs commit: jakarta-tomcat-connectors/jk/native2/common jk_worker_lb.c
Mathias Herberts wrote: [EMAIL PROTECTED] wrote: costin 02/05/09 14:06:48 Modified:jk/native2/common jk_worker_lb.c Log: That's the big one. Please review ! It changes the handling of lb_value to int. I also cleaned up the logic so it's easier ( I hope ) to understand what's happening. Levels replace the 'local worker', I thing I got the logic straight for those. I started to add a 'introspection' data, to validate and better report the config. We use one table per level. At the moment the maximum number of workers is hardcoded ( to 255 ), we could make it dynamic but that would make things pretty complex when we add workers dynamically ( it won't work without a CS or atomic operations ) Hi Costin, I read your code throughly and found no problem in get_most_suitable_worker, I think your approach to prioritizing workers is the best. What bernd and I had done was mainly driven by the need to have a frontal load balancer detect the failure of the local worker(s). Since my last patch and having read yours I think I found a better way to make the load balancer detect failures. Configure all Apache instances so they see all Tomcat instances, assign a higher priority to local workers on each Apache, therefore local workers will be chosen first. On each Apache, the load balancing worker is called lb. Another load balancing worker, balancing only the local workers, is called hwlb. The hardware load balancer checks the health of the Apache servers using a URI which is served by hwlb instead of lb, therefore if there are no more local workers left alive, the requests the hardware load balancer dispatches to the associated Apache before it can detect the local workers failure will be rerouted to the other non local workers and the client will only loose its session information, she will not get any errors. When the hardware load balancer ends up detecting the local workers failure (because the hwlb worker rejected the request due to the lack of available worker), it will declare the Apache inactive and will only use the other ones. This setup solves my use cases at least, I don't know for Bernd's. Sorry, I must say it again, for my environment it is an error, if a _switched off_ tomcat got a request without a sessionid or with a session from another node. Its not necessary that this tomact-apache tandem is shut down. We switch off a port on this node and than the balancer wouldn't send a request to it. And than no mod_jk is allowed to send a request to it without a session for this node. It is normal that some nodes are _switched off_. We need this for a a graceful update. We switch off some nodes, wait till there are no active sessions (all timed out) and then we shutdown apache + tomcat, make an update and start them up again. If there are no local/main worker I need an error response and no routing to a switched off tomcat. Its possible that this happens once per day. I know this might be a special environment. I spent some time in jk1 to build a working patch. Than I started looking in jk2. I'm not a good C developer, so I needed some time for looking into jk2. Now I think I understand the internal structure. I don't want to send untested patches or patches which build more problems than it solves. The last patch I sent for jk1 solved my problem, I tested it here on a testcluster and I hope it broke no prior functionality. But it will take some time till I could send a patch for jk2. My boss give me some deadlines for other projects, one is next Wednesday. I would be happy if jk2 make it possible to use local/main-worker with sticky sessions (need only one per node/mod_jk). And if all local/main-worker are down the request gets an error-response. I will do my best to install a jk2 on my test cluster and try to play around with it. May be I misunderstood Mathias suggestion for jk2, than delete the whole mail :). I hope I could send a patch for jk2 or look into the new code shortly. Again, I think its a very good idea to use ints for lb_value, set a maximum and correct the value if one reaches this upper bound. And its a good idea to make the local/main-worker a more general thing. For a cluster environment it is a nice feature :). Thanks Bernd There remains a related problem in jk_requtil in jk2_requtil_getCookieByName, as I mentioned several months ago on the list, the cookie extraction does not work for cookies whose format conforms to RFC 2169, that is the cookie value is enclosed in double quotes. Such cookie format is used by lynx for example. I had submitted a patch into the bug database but cannot find it anymore, I'll have to look up my archives. Good job on the lb worker Costin, Mathias. -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED
Re: jk_lb_worker.c patch
will have priority over one with a lb_value of 0.0 and this seems to break the behavior we had achieved with your patch. Did I miss something or is this really a problem? Mathias. -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: jk_lb_worker.c patch
Hi Costin, here is my patch for jk_lb_worker.c, jk_util.c and jk_util.h. It was diffed against the cvs. How does it work? 1) A new config param for lb_worker exists (main_worker_mode). Two values are possible: reject and balance. 2) This results in two new flags in lb_worker struct. If there was in minimum one worker with lb_value of 0 the flag in_main_worker_mode is set to JK_TRUE. JK_False if there was none. The second flag is reject. It is JK_TRUE if the config param above was set, JK_FALSE otherwise or if in_main_worker_mode is JK_FALSE; Behavior: If there was no worker with lb_value == 0 it should work as it used to. All workers with lb_value == 0 will be moved at the beginning of the worker list. It is possible to have more than one such worker. So you could have a standby server if the first main tomcat goes down. If a request with a session id comes in it will be routed to his tomcat. If that fails it will be routed to the main worker. If a request without a session id comes in it will be routed to the first of the main workers if that fails it goes to the next main worker. If there are no main workers left the reject flag is important. If it is set the request gets an error. If it is not set the lb_worker trys to route it to one of the other workers. All requests without a session id or with an id on a shutdowned node will only be routed to the main worker (reject == JK_TRUE). Its like a graceful shutdown, e.g. for node1. If the lb in front of the cluster didn't send a request without a session id to node 1 it will only get requests with a jvmRoute of TC1. After these sessions timed out, it could be updated, get a shutdown and startup. Even after startup it wouldn't get a request without a session. The lb in front of the cluster must do the first move and send the apache on node one a fresh request. Example workers.properties of node one: workers.tomcat_home=route to Catalina workers.java_home=$(JAVA_HOME) ps=/ worker.list=router worker.TC1.port=8009 worker.TC1.host=node1.domain.tld worker.TC1.type=ajp13 worker.TC1.lbfactor=0 worker.TC2.port=8009 worker.TC2.host=node2.domain.tld worker.TC2.type=ajp13 worker.TC2.lbfactor=1 worker.router.type=lb worker.router.balanced_workers=TC1,TC2 worker.router.main_worker_mode=reject For node two the lbfactor of TC1 is 1 and TC2 is 0. I didn't had a closer look at Mathias patch. It seems to be much shorter. We should take the best of both. This patch is only a suggestion. I'm not an experienced C-Programmer (I know my copy statements in validate are ugly, but I'm not very familiar with moving memory around :) ). Bernd [EMAIL PROTECTED] wrote: Bernd, At this moment I believe we should add flags and stop using the '0' value in the config file. Internally ( in the code ) - it doesn't matter, we can keep 0 or use the flag ( I prefer the second ). I'm waiting for your patch - it seems there is another bug that must be fixed before we can tag - but I hope we can finish all changes in the next few days. Costin On Mon, 6 May 2002, Bernd Koecke wrote: thanks for commiting my patch :). After thinking about it, I found the same problem like Mathias. It's a problem for my environment too. We have the same problem with shutdown and recovering here. I'm on the way of looking in jk2. The question for jk1 is, what want we do if the main worker fails because of an error? Because the normal intention of lb is to switch to another worker in such case. But for the special use of a main worker we don't want that (at least it is an error in my environment here :) ). My suggestion is to add an additional flag to the lb_worker struct where we hold the information that we have a main worker, e.g main_worker_mode. Because of this flag we send only requests with a session id to one of the other worker. And we could change the behavior after an error of an other worker and check his state only if we get a request with his session route. This would be easy if we set the main worker at the begining of the worker list and/or use the flag. But we need the flag if we want to use more the one main worker. But what should happen if the main worker is in error state? In my patch some weeks ago I added an additional flag which causes the module to reject a request if it comes in without a session id and the main worker is down. If this flag wasn't set or was not set to reject the module chooses one of the other worker. For our environment here rejecting the request is ok, because if a request without a session comes to a switched off node, we have a problem with our separated load balancer. This should never happen. We could make this rejecting be the standard if we have a main worker, but with a separate flag it would be more flexible. I will build a patch against cvs to make my intention clearer. Bernd [EMAIL PROTECTED] wrote: Hi Mathias, I think we understand your use case, it is not very uncommon
Re: PROPOSAL: mod_jk2: Group/Instance
Hi Costin, May be I checked out the wrong repository. I checked out jakarta-tomcat-connectors with the CVSROOT=:pserver:[EMAIL PROTECTED]:/home/cvspublic Now to the details, see below. [EMAIL PROTECTED] wrote: On Thu, 2 May 2002, Bernd Koecke wrote: misunderstood it. After you said that my patch is included a had a closer look at mod_jk. I can't see anything of my code but I found the special meaning of the zero lb_factor/lb_value. It seems that I didn't understand it right at the first time. This could solve my problem but after a closer look and some testing I found another problem. When you set the lb_value in workers.properties to 1 for the local tomcat and 0 for the others, you get the desired behavior. But if you switch off the local tomcat for a short time you come into trouble. The problem is the 0 for the other workers. The calculation of lb_worker transforms the 0 to _inf_. Because 1/0 for a double is _inf_. This is greater than any I think there is a piece that checks for 0 and sets it to DEFAULT_VALUE (==1 ) before doing 1/lb. No, I think not :). I checked it yesterday. With some additional log statements in the validate function of jk_lb_worker.c you get the value _inf_ for the lb_factor and lb_value (line 434-444). Because if it would be set to 1, my config hadn't worked. Because I set the local worker to 1 and the others to 0. While looking at the code - I'm not very sure this whole float is needed, I'll try to find a way to simplify it and use ints ( maybe 0..100 with some 'special' values for NEVER and ALLWAYS, or some additional flags ). This is possible, but then you must add a check if the value is 0. Because without it you calc 1/0 with an int and this will give you an error. But the way it works ( or at least how I understand it ) is that if the main worker fails, then we look at all workers in error state and try the one with the oldest error. And the 'main' worker will be tried again when the timeout expires. Thats not the whole story. Its right you will check the main worker when its back again and use it only once. Because when the request was successful handled rec-in_recovering is true (line 332 of jk_lb_worker.c, service function). Than get_max_lb get the value _inf_ from one of the other worker. Than the things happen which I said in my prior mail. I haven't tested this too much, I just applied the patches ( that I understand :-), I'll add some more debugging for this process and maybe we can find a better solution. But this functionality is essential for the JNI worker and very important in general - so I really want to find the best solution. If you have any patch idea, let me know. To avoid further confusion and complexity in the lb-factor/value, I think we should add one more flag ( 'local_worker' ? ) and use it explicitely. Again, patches are wellcome - it's allways good to have different ( and more ) eyes looking at the code. That was it what I did in my sent patch, the additional documentation was sent a few days later. But my additions to the lb_worker were a little bit to complex. You are right we should get it when we use the flag only on the main worker and change the behavior after a failure for this worker. But we need the trick with 0/inf for the other worker, because only with this we have the situation that the other worker wouldn't be asked when there is no session and the main worker is up. I will try to build another patch and send it. I think it could be possible without an additional flag. Another tought about this: When you use double and we fix the handling after an error, the main worker would never reach _inf_. Because the lb_factor is 1 if lb_value wasn't 0. After choosing the worker this value is added to the lb_value. But with a high value for lb_value the differenc between two savable double numbers is greater than the lb_factor. But this is only interessting in theory. I think in real world we will reboot apache before this will happen :). Bernd ( that can go in both jk1, but I can't see a release of jk2 without this functionality ) Costin other lb_value and greater than the lb_value of the local tomcat. But after a failure of the local tomcat he is in error_state. After some time its set to recovering and if the local tomcat is back again the function jk(2)_get_max_lb gets the highest lb_value. This is _inf_ from one of the other workers. The addition of a value to _inf_ is meaningless. You end up with an lb_value of _inf_ for the local worker. If this worker isn't the first in the worker list, it will never be choosen again. Because his lb_value will never be less than another lb_value, because all the other workers have _inf_ as theire lb_values. So every request without a session will be routed to the first of the other tomcats. The only way out is a restart of the local apache after tomcat is up and running. But I don't
[PATHC] jakarta-tomcat-connectors Re: PROPOSAL: mod_jk2: Group/Instance
Hi Costin, now here is my patch. It is very small and it works. And we don't need additional config flags. When the lb_value is read from the config file it is checked against zero. With this a flag in lb_worker is set so that the get_max_lb-function could decide if this worker should be used or not. When you set lb_value of the main or local worker to 1 and 0 for the others all works fine. When you switch off the main worker you will be routed to the first of the other worker. Thats not very balancing, but the load balancer in front of the cluster shouldn't send requestes to a node with a shutdowned tomcat. It is only for requests with sessions on this node and for the time between shutdown of tomcat and the recognition of this by the balancer. When tomcat is up again it will take a little time, in maximum the value of WAIT_BEFORE_RECOVER and the worker will be choosen and because of the flag it wouldn't get _inf_ as his lb_value. the patch was created by cvs diff -u jk_lb_worker.c Bernd Bernd Koecke wrote: Hi Costin, May be I checked out the wrong repository. I checked out jakarta-tomcat-connectors with the CVSROOT=:pserver:[EMAIL PROTECTED]:/home/cvspublic Now to the details, see below. [EMAIL PROTECTED] wrote: On Thu, 2 May 2002, Bernd Koecke wrote: misunderstood it. After you said that my patch is included a had a closer look at mod_jk. I can't see anything of my code but I found the special meaning of the zero lb_factor/lb_value. It seems that I didn't understand it right at the first time. This could solve my problem but after a closer look and some testing I found another problem. When you set the lb_value in workers.properties to 1 for the local tomcat and 0 for the others, you get the desired behavior. But if you switch off the local tomcat for a short time you come into trouble. The problem is the 0 for the other workers. The calculation of lb_worker transforms the 0 to _inf_. Because 1/0 for a double is _inf_. This is greater than any I think there is a piece that checks for 0 and sets it to DEFAULT_VALUE (==1 ) before doing 1/lb. No, I think not :). I checked it yesterday. With some additional log statements in the validate function of jk_lb_worker.c you get the value _inf_ for the lb_factor and lb_value (line 434-444). Because if it would be set to 1, my config hadn't worked. Because I set the local worker to 1 and the others to 0. While looking at the code - I'm not very sure this whole float is needed, I'll try to find a way to simplify it and use ints ( maybe 0..100 with some 'special' values for NEVER and ALLWAYS, or some additional flags ). This is possible, but then you must add a check if the value is 0. Because without it you calc 1/0 with an int and this will give you an error. But the way it works ( or at least how I understand it ) is that if the main worker fails, then we look at all workers in error state and try the one with the oldest error. And the 'main' worker will be tried again when the timeout expires. Thats not the whole story. Its right you will check the main worker when its back again and use it only once. Because when the request was successful handled rec-in_recovering is true (line 332 of jk_lb_worker.c, service function). Than get_max_lb get the value _inf_ from one of the other worker. Than the things happen which I said in my prior mail. I haven't tested this too much, I just applied the patches ( that I understand :-), I'll add some more debugging for this process and maybe we can find a better solution. But this functionality is essential for the JNI worker and very important in general - so I really want to find the best solution. If you have any patch idea, let me know. To avoid further confusion and complexity in the lb-factor/value, I think we should add one more flag ( 'local_worker' ? ) and use it explicitely. Again, patches are wellcome - it's allways good to have different ( and more ) eyes looking at the code. That was it what I did in my sent patch, the additional documentation was sent a few days later. But my additions to the lb_worker were a little bit to complex. You are right we should get it when we use the flag only on the main worker and change the behavior after a failure for this worker. But we need the trick with 0/inf for the other worker, because only with this we have the situation that the other worker wouldn't be asked when there is no session and the main worker is up. I will try to build another patch and send it. I think it could be possible without an additional flag. Another tought about this: When you use double and we fix the handling after an error, the main worker would never reach _inf_. Because the lb_factor is 1 if lb_value wasn't 0. After choosing the worker this value is added to the lb_value. But with a high value for lb_value the differenc between two
Re: PROPOSAL: mod_jk2: Group/Instance
Hi Costin, [EMAIL PROTECTED] wrote: Hi Bernd, First, many thanks for your help :-) your welcome, its a lot of fun :) No, I think not :). I checked it yesterday. With some additional log statements in the validate function of jk_lb_worker.c you get the value _inf_ for the lb_factor and lb_value (line 434-444). Because if it would be set to 1, my config hadn't worked. Because I set the local worker to 1 and the others to 0. I'll check again, and fix it if necesarry. I wrote some code in jk2 that seems to solve the problem, and I can backport this to jk1 if it is correct. Probably this is my mistake - I remember the discussion and the patch that was sent for this problem, and most likely I did something wrong commiting it ( i.e. I did few changes trying to simplify it, and it seems I 'simplified' too much ). But my memory still has the patch's logic which seemed fine :-) This is possible, but then you must add a check if the value is 0. Because without it you calc 1/0 with an int and this will give you an error. Yes, of course. 0 will continue to mean 'default worker'. see below I'm not very comfortable with float calculations in the critical path ( and in an area that is executed concurently !). The only problem is what happens on overflows - the lb_value may become 0 ( or a small value ) and then the worker will take all the load. Thats not the whole story. Its right you will check the main worker when its back again and use it only once. Because when the request was successful handled rec-in_recovering is true (line 332 of jk_lb_worker.c, service function). Than get_max_lb get the value _inf_ from one of the other worker. Than the things happen which I said in my prior mail. That was it what I did in my sent patch, the additional documentation was sent a few days later. But my additions to the lb_worker were a little bit to complex. You are right we should get it when we use the flag only on the main worker and change the behavior after a failure for this worker. But we need the trick with 0/inf for the other worker, because only with this we have the situation that the other worker wouldn't be asked when there is no session and the main worker is up. Ok, can you send the patch again :-) ? For going back to the main worker - if we let it with lb_value=0 at all time ( i.e. we don't alter that at any time ), and only in_error_state is set on failure - then I believe the thing will work fine. Thats the invers from the actual situation. So my patch from a few hours earlier this day depends on the fact that the other worker get a lb_value of 0 in the config file. This will be converted to _inf_ and the main worker gets 1 and this will be the minimal lb_value of the balanced workers. If we want the possibility to switch to ints I could send a new patch which handles 0 as a special value for the main worker. Should I? Bernd I will try to build another patch and send it. I think it could be possible without an additional flag. Great ! Another tought about this: When you use double and we fix the handling after an error, the main worker would never reach _inf_. Because the lb_factor is 1 if lb_value wasn't 0. After choosing the worker this value is added to the lb_value. But with a high value for lb_value the differenc between two savable double numbers is greater than the lb_factor. But this is only interessting in theory. I think in real world we will reboot apache before this will happen :). That may become a problem if we use ints. Costin [...] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
[PATCH] added handling of a main worker in jk_lb_worker, Re: PROPOSAL:mod_jk2: Group/Instance
Hi Costin,
it wasn't difficult, so here is the new patch. The new (old) behavior is:
The main worker is defined by a lb_value of 0. This will never be changed in
jk_lb_worker. The other workers can get a value greater than 0. If the value
from config file is less than 0 it is multiplicated with -1.
Your are right this is a better solution. We can switch from doubles to int and
we get the other worker balanced if the main worker is down.
Bernd
Bernd Koecke wrote:
Hi Costin,
[EMAIL PROTECTED] wrote:
Hi Bernd,
First, many thanks for your help :-)
your welcome, its a lot of fun :)
No, I think not :). I checked it yesterday. With some additional log
statements in the validate function of jk_lb_worker.c you get the
value _inf_ for the lb_factor and lb_value (line 434-444). Because if
it would be set to 1, my config hadn't worked. Because I set the
local worker to 1 and the others to 0.
I'll check again, and fix it if necesarry.
I wrote some code in jk2 that seems to solve the problem, and I can
backport this to jk1 if it is correct.
Probably this is my mistake - I remember the discussion and the patch
that was sent for this problem, and most likely I did something
wrong commiting it ( i.e. I did few changes trying to simplify it, and it
seems I 'simplified' too much ). But my memory still has the patch's
logic
which seemed fine :-)
This is possible, but then you must add a check if the value is 0.
Because without it you calc 1/0 with an int and this will give you an
error.
Yes, of course. 0 will continue to mean 'default worker'.
see below
I'm not very comfortable with float calculations in the critical
path ( and in an area that is executed concurently !). The only problem
is what happens on overflows - the lb_value may become 0 ( or a small
value ) and then the worker will take all the load.
Thats not the whole story. Its right you will check the main worker
when its back again and use it only once. Because when the request
was successful handled rec-in_recovering is true (line 332 of
jk_lb_worker.c, service function). Than get_max_lb get the value
_inf_ from one of the other worker. Than the things happen which I
said in my prior mail.
That was it what I did in my sent patch, the additional documentation
was sent a few days later. But my additions to the lb_worker were a
little bit to complex. You are right we should get it when we use the
flag only on the main worker and change the behavior after a failure
for this worker. But we need the trick with 0/inf for the other
worker, because only with this we have the situation that the other
worker wouldn't be asked when there is no session and the main worker
is up.
Ok, can you send the patch again :-) ?
For going back to the main worker - if we let it with lb_value=0 at all
time ( i.e. we don't alter that at any time ), and only in_error_state
is set on failure - then I believe the thing will work fine.
Thats the invers from the actual situation. So my patch from a few hours
earlier this day depends on the fact that the other worker get a
lb_value of 0 in the config file. This will be converted to _inf_ and
the main worker gets 1 and this will be the minimal lb_value of the
balanced workers. If we want the possibility to switch to ints I could
send a new patch which handles 0 as a special value for the main worker.
Should I?
Bernd
I will try to build another patch and send it. I think it could be
possible without an additional flag.
Great !
Another tought about this:
When you use double and we fix the handling after an error, the main
worker would never reach _inf_. Because the lb_factor is 1 if
lb_value wasn't 0. After choosing the worker this value is added to
the lb_value. But with a high value for lb_value the differenc
between two savable double numbers is greater than the lb_factor. But
this is only interessting in theory. I think in real world we will
reboot apache before this will happen :).
That may become a problem if we use ints.
Costin
[...]
--
Dipl.-Inform. Bernd Koecke
UNIX-Entwicklung
Schlund+Partner AG
Fon: +49-721-91374-0
E-Mail: [EMAIL PROTECTED]
Index: jk_lb_worker.c
===
RCS file: /home/cvspublic/jakarta-tomcat-connectors/jk/native/common/jk_lb_worker.c,v
retrieving revision 1.8
diff -u -r1.8 jk_lb_worker.c
--- jk_lb_worker.c 12 Jan 2002 05:27:39 - 1.8
+++ jk_lb_worker.c 3 May 2002 17:17:35 -
@@ -291,7 +291,12 @@
}
}
-if(rc) {
+/*
+ * The following isn't necessary, because for the 'main' worker
+ * lb_value and lb_factor are == 0. And all other worker have a value 0
+ * But it makes the intention clearer.
+ */
+if(rc rc-lb_value != 0) {
rc-lb_value += rc-lb_factor;
}
@@ -329,7 +334,7
Re: PROPOSAL: mod_jk2: Group/Instance
[EMAIL PROTECTED] wrote: On Thu, 2 May 2002, Amund Elstad wrote: (1) all requests without a session are routed to a specific tomcat instance (if that instance is working). That has been added, and it should work in both jk1 and jk2 ( I don't remember who sent the patch, but I remember adding it ). If it doesn't work yet, is easy to fix. Sorry, may be I'm stupid, but how does it work? I looked at jk and jk2. My understanding is the following: The worker which uses the jvm route is the lb_worker. The others don't use the SessionId extension. You could tweak this worker, but its buggy or I misunderstood it. After you said that my patch is included a had a closer look at mod_jk. I can't see anything of my code but I found the special meaning of the zero lb_factor/lb_value. It seems that I didn't understand it right at the first time. This could solve my problem but after a closer look and some testing I found another problem. When you set the lb_value in workers.properties to 1 for the local tomcat and 0 for the others, you get the desired behavior. But if you switch off the local tomcat for a short time you come into trouble. The problem is the 0 for the other workers. The calculation of lb_worker transforms the 0 to _inf_. Because 1/0 for a double is _inf_. This is greater than any other lb_value and greater than the lb_value of the local tomcat. But after a failure of the local tomcat he is in error_state. After some time its set to recovering and if the local tomcat is back again the function jk(2)_get_max_lb gets the highest lb_value. This is _inf_ from one of the other workers. The addition of a value to _inf_ is meaningless. You end up with an lb_value of _inf_ for the local worker. If this worker isn't the first in the worker list, it will never be choosen again. Because his lb_value will never be less than another lb_value, because all the other workers have _inf_ as theire lb_values. So every request without a session will be routed to the first of the other tomcats. The only way out is a restart of the local apache after tomcat is up and running. But I don't know when tomcat is finished with all his contexts and started the connectors. I didn't looked very deep into jk2, but I found the same get_most_suitable_worker and get_max_lb functions. The jk2_get_max_lb function will always return _inf_. In your answer to some other mails you said, that workers could be removed. Do I understand it right, that if my local tomcat goes down his worker is removed from the list and after he is comming up again added to the worker list with reseted lb_value (only for mod_jk2)? The next days I will look in the docu and code of jk2 and give it a try. May be all my problems gone away with the new module :). Sorry if I ask stupid questions, but I want to make it working for our new cluster. Thanks Bernd This is essential for jk2's JNI worker, which fits perfectly this case ( you don't want to send via TCP when you have a tomcat instance in the same process ). (2) Tomcat instances in standby or soft shutdown mode where they serve requests bound by established sessions, and requests without a session only if all non-standby instances have failed. That's what the SHM scoreboard is going to do ( among other things ). You can register tomcat instances ( which will be added automatically ), or unregister - in which case no new requests ( except the old sessions ) will go to the unregistered tomcat. Costin [EMAIL PROTECTED] wrote: On Tue, 30 Apr 2002, Bernd Koecke wrote: some weeks ago I send a patch for mod_jk for an only routing lb_worker. A few days later I sent the docu. Henry Gomez said, that it should be commited. But it I think it isn't in the repository. But its the same with me here, to mutch work for to less time :). I think it is in mod_jk, I remember seeing the commit. And I think I commited it in jk2 as well ( after some modifications ). I need sticky sessions but no loadbalancing in the module. If a request without a session comes in, it should be routed to the _local_ tomcat. Well, there is another use-case with the exact same behavior - Apache2 with tomcat in JNI mode. All requests without session should be routed to the _jni_ channel ( i.e. in-process, minimal overhead ). It's exacly the same - so be sure I do my best to handle this case :-) Apache2 acts like a 'natural' load-balancer/fail-over, with the parent process monitoring for crashes and it starts/stop childs based on load. I think this could be possible with the associated instance of a channel (item 7). Then I have to configure all four nodes for the same group. Because all nodes will serve the same webapps and associate the channel with this group. But for this I need a non balancing group. I don't see if the default behavior of a group is balancing and if this can be switched off. Is this right or do I miss something
Re: PROPOSAL: mod_jk2: Group/Instance
Hi Costin, some weeks ago I send a patch for mod_jk for an only routing lb_worker. A few days later I sent the docu. Henry Gomez said, that it should be commited. But it I think it isn't in the repository. But its the same with me here, to mutch work for to less time :). Again an example of my environment: Cluster with 4 nodes, tomcat and apache on every node. One Loadbalancer in front of the cluster which does a simple round robbin. I need sticky sessions but no loadbalancing in the module. If a request without a session comes in, it should be routed to the _local_ tomcat. I think this could be possible with the associated instance of a channel (item 7). Then I have to configure all four nodes for the same group. Because all nodes will serve the same webapps and associate the channel with this group. But for this I need a non balancing group. I don't see if the default behavior of a group is balancing and if this can be switched off. Is this right or do I miss something? Thanks Bernd [EMAIL PROTECTED] wrote: One of the major goals of mod_jk2 is simpler configuration. This proposal will cover the 'workers'. 1. The 'worker' name is deprecated. It refers to too many things in mod_jk, and causes too much confusion ( i.e. it is a 'handler', coresponds to a jvmRoute, a protocol, a channel ). 2. We'll use the term 'channel' to define the connections between jk and tomcat. Each channel will follow a standard naming syntax: channel.socket:HOST:PORT channel.unix:/PATH channel.jni:jni Altenatively, the local part can be used as a shortcut. Each channel will use Ajp13 protocol by default ( or whatever is specific to the transport, or overriden ). 3. We'll use the term 'instance' ( XXX or a better name if you can sugest one ) to refer to a single tomcat VM. That's what will be used in the jvmRoute. A tomcat instance can be reached by multiple channels. By default, each instance will have one socket channel and will be named by the local part ( i.e. HOST:port ). That's the name that'll identify the VM. 4. We'll use the term 'group' to reffer to a set of tomcat 'instances' that share a number of applications. The default group will be called 'lb'. One 'instance' can be part of one or many groups. 5. Each URI will be associated with a 'group'. In what used to be: JkMount /examples ajp13 the 'group' will replace ajp13. As I said, the default group is called 'lb' and includes all tomcat instances that are defined. For each application that is installed on all instances, the 'lb' group must be used. If you have a sophisticated setup - and an app is installed only on some instances, you can use names like lb_examples, and explicitely define what tomcat instances belong to the group. 6. The webapps will be defined by a hierarchy of directories ( host/webapp_name ) ( see previous mail ). Each webapp is obviously mapped consistently to one group, which will be defined inside the WEB-INF/jk2/map.properties. 7. Each channel will have an associated 'instance' ( it can't go to 2 tomcats ) and groups. The lb configuration is done automatically based on those attributes. The mappings are done automatically based on the hierarchy. If no map.properties is found, the whole webapp will be forwarded to the default group ( the common case ). I believe this model covers all current use cases. Please, please, send feedback and let me know if you can help ( implementation, documentation, testing ), but feedback is the first step. Costin -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
[PATCH] jakarta-tomcat-connectors/jk/native/common/jk_lb_worker.c,jk_util.c
Hi,
some days ago I sent a question for an only routing lb-worker. Costin asked for
the patch and here it is.
I added two config-directives for the lb-worker:
local_worker
- this is the name of the worker which should get the request if there is no
session or no jvmRoute.
fault_action
- the possible values are 'balance' and 'reject'. Explanation follows.
Behaviour of lb-worker:
a.) there is a jvmRoute and the corresponding node is ok:
- The request goes to the corresponding node
b.) there is no jvmRoute or the corresponding node didn't answered:
- if a local_worker was given. this one will get the request.
- if a local_worker was given, but it is in error state:
fault_action == reject: the request gets an error.
fault_action == balance: try to find another worker from the list of
balanced workers (c.).
- if no local_worker was given, go directly to c.).
c.) if no worker was found in a) and b) the normal balancing behaviour takes place.
I looked at cvs-repopsitory and it seems that jk_lb_worker.c and jk_util.c have
the same version (1.8, 1.12) like mine from
jakarta-tomcat-connectors-4.0.2-src.tar.gz.
I need the fault_action, because our load balancer should not route a request to
a node without a working local tomcat. So it is an error if an unrouteable
request arrives at a node without a working local tomcat. And it is also an
error to route it to one of the other nodes, because sometimes we switch off one
node only by telling the load balancer not to use it for requests. In this case
the modules should use this node only for requests with a session on it and not
for requests without a session. But for other use cases it might be useful if
mod_jk tries to balance the request in case of a failure of the local tomcat.
I hope it is useful. Sorry i haven't looked into jk2 at this time, but may be i
get time to do it soon.
Bernd
--
Dipl.-Inform. Bernd Koecke
UNIX-Entwicklung
Schlund+Partner AG
Fon: +49-721-91374-0
E-Mail: [EMAIL PROTECTED]
--- jk_lb_worker.c.orig Thu Apr 4 13:20:00 2002
+++ jk_lb_worker.c Fri Apr 5 11:03:09 2002
@@ -95,6 +95,9 @@
worker_record_t *lb_workers;
unsigned num_of_workers;
+worker_record_t *default_worker;
+unsigned reject_on_fault;
+
jk_pool_t p;
jk_pool_atom_t buf[TINY_POOL_SIZE];
@@ -269,6 +272,27 @@
}
}
+if (p-default_worker) {
+rc = p-default_worker;
+if (rc-in_error_state) {
+if (!rc-in_recovering) {
+time_t now = time(0);
+
+if ((now - rc-error_time) WAIT_BEFORE_RECOVER) {
+rc-in_recovering = JK_TRUE;
+rc-error_time = now;
+} else {
+rc = NULL;
+}
+} else {
+rc = NULL;
+}
+}
+if (rc || p-reject_on_fault) {
+return rc;
+}
+}
+
for(i = 0 ; i p-num_of_workers ; i++) {
if(p-lb_workers[i].in_error_state) {
if(!p-lb_workers[i].in_recovering) {
@@ -321,6 +345,8 @@
if(rec) {
int is_recoverable = JK_FALSE;
+jk_log(l, JK_LOG_DEBUG, In jk_lb_worker::service routing to worker
+'%s'\n,
+rec-name);
s-jvm_route = jk_pool_strdup(s-pool, rec-name);
@@ -415,6 +441,22 @@
lb_worker_t *p = pThis-worker_private;
char **worker_names;
unsigned num_of_workers;
+char *default_worker_name;
+unsigned reject_on_fault = JK_FALSE;
+
+if (jk_get_lb_default_worker_name(props, p-name, default_worker_name)) {
+jk_log(l, JK_LOG_DEBUG,
+In jk_lb_worker_t::validate found a local worker is %s\n,
+default_worker_name);
+jk_get_lb_fault_action(props, p-name, reject_on_fault);
+} else {
+jk_log(l, JK_LOG_DEBUG,
+In jk_lb_worker_t::validate didn't found a local worker\n);
+}
+p-reject_on_fault = reject_on_fault;
+jk_log(l, JK_LOG_DEBUG,
+In jk_lb_worker_t::validate reject_on_fault is '%u'\n,
+p-reject_on_fault);
if(jk_get_lb_worker_list(props,
p-name,
@@ -449,6 +491,12 @@
we,
l) || !p-lb_workers[i].w) {
break;
+}
+if (default_worker_name 0 == strcmp(p-lb_workers[i].name,
+default_worker_name)) {
+p-default_worker = (p-lb_workers[i]);
+jk_log(l, JK_LOG_DEBUG,
+In jk_lb_worker_t::validate got a local worker '%s'\n,
+p-default_worker-name);
}
}
--- jk_util.c.orig Thu Apr 4 13:15:42 2002
+++ jk_util.c Thu Apr 4 15:31:08
Re: [PATCH] jakarta-tomcat-connectors/jk/native/common/jk_lb_worker.c,jk_util.c
GOMEZ Henri wrote: Seems a good patch. Should be commited. BTW, Bernd could you make a little documentation which could be included in jk doc and add example ? Excellent works ;) Thanks, I'll have a look at the docs in detail and add some example. Should I sent these files to this list? Bernd - Henri Gomez ___[_] EMAIL : [EMAIL PROTECTED](. .) PGP KEY : 697ECEDD...oOOo..(_)..oOOo... PGP Fingerprint : 9DF8 1EA8 ED53 2F39 DC9B 904A 364F 80E6 -Original Message- From: Bernd Koecke [mailto:[EMAIL PROTECTED]] Sent: Friday, April 05, 2002 1:30 PM To: [EMAIL PROTECTED] Subject: [PATCH] jakarta-tomcat-connectors/jk/native/common/jk_lb_worker.c,jk_util.c Hi, some days ago I sent a question for an only routing lb-worker. Costin asked for the patch and here it is. I added two config-directives for the lb-worker: local_worker - this is the name of the worker which should get the request if there is no session or no jvmRoute. fault_action - the possible values are 'balance' and 'reject'. Explanation follows. Behaviour of lb-worker: a.) there is a jvmRoute and the corresponding node is ok: - The request goes to the corresponding node b.) there is no jvmRoute or the corresponding node didn't answered: - if a local_worker was given. this one will get the request. - if a local_worker was given, but it is in error state: fault_action == reject: the request gets an error. fault_action == balance: try to find another worker from the list of balanced workers (c.). - if no local_worker was given, go directly to c.). c.) if no worker was found in a) and b) the normal balancing behaviour takes place. I looked at cvs-repopsitory and it seems that jk_lb_worker.c and jk_util.c have the same version (1.8, 1.12) like mine from jakarta-tomcat-connectors-4.0.2-src.tar.gz. I need the fault_action, because our load balancer should not route a request to a node without a working local tomcat. So it is an error if an unrouteable request arrives at a node without a working local tomcat. And it is also an error to route it to one of the other nodes, because sometimes we switch off one node only by telling the load balancer not to use it for requests. In this case the modules should use this node only for requests with a session on it and not for requests without a session. But for other use cases it might be useful if mod_jk tries to balance the request in case of a failure of the local tomcat. I hope it is useful. Sorry i haven't looked into jk2 at this time, but may be i get time to do it soon. Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: tomcat 4.0.3 + mod_jk
GOMEZ Henri wrote: yes, I know. But when I come in without a session the function get_most_suitable_worker in jk_lb_worker.c will find a worker by using the lb-factor and do weighted round robin. And at this position I would like to switch off the load-balancing and select the worker which is connected to tomcat on the same node. let's resume : you have a tandem apache/tomcat by machine. apache could use also others tomcat from a LB farm. when you've got no session you want to use the local tomcat instead of one grabbed from lb farm. so you propose to add a 'DEFAULT' worker in LB which will bypass the LB settings when no session is present ? Yes. Costin suggested to use a special lb-factor. I would use -1 for the lb-factor. But this wouldn't solve the problem if someone set more than one worker to this value. To use a new config tag for optionally setting a default worker would be a cleaner way. And it would be easier to handle a connection error while connecting to the local tomcat in the service method. Because if my local tomcat has problems I could fallback to the normal selection of a worker by lb-factors. Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: tomcat 4.0.3 + mod_jk
GOMEZ Henri wrote: Yes. Costin suggested to use a special lb-factor. I would use -1 for the lb-factor. But this wouldn't solve the problem if someone set more than one worker to this value. To use a new config tag for optionally setting a default worker would be a cleaner way. And it would be easier to handle a connection error while connecting to the local tomcat in the service method. Because if my local tomcat has problems I could fallback to the normal selection of a worker by lb-factors. will you works on jk2 or good old jk from JTC ? Up to now i Worked only on jk from JTC, but if we get the same functionality from jk2 I could have a look at it. I thought that jk2 was for apache2. But I may misunderstound this. -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: tomcat 4.0.3 + mod_jk
GOMEZ Henri wrote: Up to now i Worked only on jk from JTC, but if we get the same functionality from jk2 I could have a look at it. I thought that jk2 was for apache2. But I may misunderstound this. jk2 is the reworked jk, for ap 1.3/2.0, IIS/iPlanet to come ok, for which should we do it? Can i build and use jk2 or is it in an early development state? -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: tomcat 4.0.3 + mod_jk
yes, I know. But when I come in without a session the function get_most_suitable_worker in jk_lb_worker.c will find a worker by using the lb-factor and do weighted round robin. And at this position I would like to switch off the load-balancing and select the worker which is connected to tomcat on the same node. Because we have a loadbalancer in front of our cluster but it can't handle the sessions. So we need the routing of mod_jk for sticky sessions, but the loadbalancing would indefer with our loadbalancer. With mod_jserv we did it with a little trick. The routing config and the nodeset for loadbalancing were separated. For routing we told mod_jserv all the nodes in the cluster, but for loadbalancing we build a nodeset with only the local node in it. A little perl script build the conffiles suitable for the nodes. But mod_jserv can only work with ajp12 and we would like to use the advanced features of ajp13. For mod_jk of tomcat 3.2.x I build an additional worker which did only the routing. And I added a new config command for declaring the local connected worker. But it is a little bit laborious to add this every time when the connectors change. Id would be nice if this feature was contained in mod_jk. I could do it and send the additional worker to the list/project. Thanks Bernd GOMEZ Henri wrote: in mod_jk, when a session is created the following requests will allways go to the same tomcat. IBM call it Session Affinity ;) Hi, is it possible to use only the session routing of mod_jk without load balancing? Because we have a standalone loadbalancer in front of our cluster. In the past we used mod_jserv with ApacheJServ. But now we want to switch to tomcat. On every node should run an apache and tomcat, connected to each other with mod_jk. We need the routing between the nodes for established sessions, but a request without a session should be routed by mod_jk to the tomcat on the same node. I hope I could reach someone of the mod_jk developers. Sending this question to the user list wasn't successful. Thanks Bernd [...] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
Re: tomcat 4.0.3 + mod_jk
ok, I will patch worker_lb. But it will take a few days. To mutch work and to less time :). Bernd [EMAIL PROTECTED] wrote: This also solves another problem - Apache2.0+JNI worker. It's exactly the same, it acts similary with a load balancer ( multiple processes listening on the same port ), and we want the jni worker to take the load if no session. Wouldn't be simpler if you just patch worker_lb ? Say if a worker has a very high factor ( or special flag ) use it allways. My thinking for jk2 was that the session stickiness should go in front of everything, and be enabled by default, independent of lb. Send the patch please ( if you can just make a smaller change in lb it would be better IMHO ) Costin On Thu, 21 Mar 2002, Bernd Koecke wrote: yes, I know. But when I come in without a session the function get_most_suitable_worker in jk_lb_worker.c will find a worker by using the lb-factor and do weighted round robin. And at this position I would like to switch off the load-balancing and select the worker which is connected to tomcat on the same node. Because we have a loadbalancer in front of our cluster but it can't handle the sessions. So we need the routing of mod_jk for sticky sessions, but the loadbalancing would indefer with our loadbalancer. With mod_jserv we did it with a little trick. The routing config and the nodeset for loadbalancing were separated. For routing we told mod_jserv all the nodes in the cluster, but for loadbalancing we build a nodeset with only the local node in it. A little perl script build the conffiles suitable for the nodes. But mod_jserv can only work with ajp12 and we would like to use the advanced features of ajp13. For mod_jk of tomcat 3.2.x I build an additional worker which did only the routing. And I added a new config command for declaring the local connected worker. But it is a little bit laborious to add this every time when the connectors change. Id would be nice if this feature was contained in mod_jk. I could do it and send the additional worker to the list/project. Thanks Bernd GOMEZ Henri wrote: in mod_jk, when a session is created the following requests will allways go to the same tomcat. IBM call it Session Affinity ;) Hi, is it possible to use only the session routing of mod_jk without load balancing? Because we have a standalone loadbalancer in front of our cluster. In the past we used mod_jserv with ApacheJServ. But now we want to switch to tomcat. On every node should run an apache and tomcat, connected to each other with mod_jk. We need the routing between the nodes for established sessions, but a request without a session should be routed by mod_jk to the tomcat on the same node. I hope I could reach someone of the mod_jk developers. Sending this question to the user list wasn't successful. Thanks Bernd [...] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED] -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED] -- To unsubscribe, e-mail: mailto:[EMAIL PROTECTED] For additional commands, e-mail: mailto:[EMAIL PROTECTED]
tc-3.2.2/3.2.3 Little Bug in ContextAdmin.java
Hi, with tomcat comes the admin-Context. When you use it and select the 'View All Context'-Button you get an IllegalArgumentException. The error arises because of the use of the class org.apache.tomcat.util.RequestUtil in the ContextAdmin-class which is used by the ContextAdmin-JSP. In tomcat 3.2.2 and 3.2.3 the RequestUtil-class throws the Exception when a '/' is URL-encoded in the Parameter-String (explained in the release-notes of 3.2.3). This happens e.g. in lines 76 and 92 of ContextAdmin.java. A quick solution is to use java.net.URLDecoder.decode() instead of RequestUtil. But this class was introduced in JDK 1.2, so users of older JDKs don't have this solution. I'm using 1.3, so the quick solution works fine for me :) If someone is interested in the diffs, I could send them, but they are very simple. If tomcat should run on JDKs prior to 1.2, I could write a small function in ContextAdmin, which translates the '/' before sending it to the RequestUtil-class and send the diffs of this. Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED]
[PATCH Suggestion] tc-3.2.3 Little bug in ContextAdmin.java
Hi,
I got a request for the diff of ContextAdmin.java.
Subject of my original posting was: tc-3.2.2/3.2.3 Little Bug in ContextAdmin.java
Its not to long, so I send it to the list. The orig-class is the one shipped with
tomcat-3.2.3.
But again, it solves the problem only for Java2-Systems.
I hope it's useful
Bernd
--- ContextAdmin.java.orig Wed Jul 18 15:25:25 2001
+++ ContextAdmin.java Wed Jul 18 15:25:53 2001
@@ -2,6 +2,7 @@
import java.util.Enumeration;
import java.io.File;
import java.net.URL;
+import java.net.URLDecoder;
import javax.servlet.http.*;
import org.apache.tomcat.core.Request;
@@ -73,7 +74,8 @@
v.addElement(FULL DOC BASE: + context.getDocumentBase().toString());
v.addElement(PATH: + context.getPath());
if (context.getWorkDir() != null)
- v.addElement(WORK DIR: +
RequestUtil.URLDecode(context.getWorkDir().getName()));
+ v.addElement(WORK DIR: +
+URLDecoder.decode(context.getWorkDir().getName()));
+ //v.addElement(WORK DIR: +
+RequestUtil.URLDecode(context.getWorkDir().getName()));
v.addElement(DESCRIPTION: + context.getDescription());
v.addElement(SESSION TIMEOUT: + new
Integer(context.getSessionTimeOut()).toString());
@@ -89,7 +91,8 @@
while (enum.hasMoreElements()) {
key = (String)enum.nextElement();
v.addElement(ATTRIBUTE NAME: + key);
- v.addElement(ATTRIBUTE: +
RequestUtil.URLDecode(context.getAttribute(key).toString()));
+ v.addElement(ATTRIBUTE: +
+URLDecoder.decode(context.getAttribute(key).toString()));
+ //v.addElement(ATTRIBUTE: +
+RequestUtil.URLDecode(context.getAttribute(key).toString()));
}
v.addElement(SERVER INFO: + context.getEngineHeader());
Re: mod_jk in a cluster
James Courtney wrote: With the exception of failover (case 4 below), I believe that the first three cases can be handled by having your load balancer be "sticky" by client address to any of the app servers (machines running Apache with Tomcat). Thus if your load balancer receives a request from client some.client.net and round robin assigns the request to server B, given servers A, B, and C, then all subsequest requests from some.client.net should be routed to server B. Our load balancer is very poor. He does weighted round robbin and the weights are the system load of the cluster computer. He can't look in the IP-Packets and doesn't know anything about sessions. And he also doesn't cache the client-ip's. So the load balancer can't handle the stickyness of the sessions. This is handled in the past by mod_jserv in the apache-webserver on the cluster. But now we want to use tomcat with mod_jk because of 2.2servlets and the better handling of SSL in ajp13. After playing around with mod_jk I build a new worker, my needs are more like a router (because of the sessions) with failover and not a load balancer (lb-worker). So I think my first idea to add a new property to the lb-worker wasn't clever. mod_jk is compiled, so I have to test it. This will take a little time. If B ever goes down the session information from some.client.net is lost and the client will have to retry their session. To implement a failover system is highly non-trivial and my understanding is that most application servers (even commercial ones) do not support this. In fact I'm told by several coworkers who worked formerly for BEA and know people there up to and including the B, the E, and the A that WebLogic only recently got failover with session replication right. To make this work you will need a somewhat complicated session replication mechanism between all app servers which is n(n-1) replications (in this case 6) if replication is done between all servers and at best n choose 2 (in this case 3, or an extra session "write through" per request if you prefer) replications if each app server has a single failover server. That can be a lot of network traffic and a lot of new code. If I'm wrong and there's a way to make this work in Tomcat then kudos to all involved and let me know how to do it:)! -Jamey I agree. Session replication is a solution but it isn't clever. You have to mutch traffic for the hopefully rare case of a server crash. So we make the following: If you have a session on a server wich is crashed, you get a new one on a server which is up and start at the beginning. This is handled by our servlets. So with my new worker in conjunction with our servlets I should get the desired behaviour. Thanks to the developers of mod_jk! My C-knowledge is not very well, but with the help of the comments and variable/function-names in the source I got it. The code is a little bit special to our purpose, but if there are interests in the small code I could send it to the list. But I think we should wait till it is tested :). Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED]
mod_jk in a cluster
Hi, we want to use tomcat 3.2.1 in a cluster-environment. This is not a request that someone else should code something. I think I have a solution, but may be others are interested in it too. We have, lets say three cluster-computer (server) and one simple load-balancer. The load-balancer doesn't look in the Packets and doesn't know something about Sessions etc. The server are running with apache + tomcat. apache with mod_jk. The following things could happen: - a request arrives at a server without a session = it should be routed to tomcat on this server - a request with a session on another server arrives = should be routed to the other server - a request with a session on this server arrives = should be routed to this server - a request with a session on a server which is down arrives = use another server and send him the request. The session is not present there and our application handles this Is there a way to get this with existing modules? My solution plays around with mod_jk as follows: The easiest way is to switch off load-balancing and recovering with an additional property in the config. But I could understand if someone says this should be a new module. But it could work with mod_jk. For this I use the lb_worker which controls three ajp13-worker on every server. The ajp13-worker are connected to the three tomcats. Now I have two possibilities: - set the lb_value of the worker for the local tomcat to 1 and 1 for the other workers, then switch off load-balancing by setting lb_factor to 0.0. This should give an error because of the validate-function, but it could be fixed with a small patch. - setting the lb_value for the worker which is connected to tomcat on the same server to -1 and 1 for the others and the lb_factor to a negative value. Both solutions are incorrect if one of the tomcats goes down, because of the recovering. So I come to the first idea, to use an additional flag. If it works here, are there interests in a diff of mod_jk? Bernd -- Dipl.-Inform. Bernd Koecke UNIX-Entwicklung Schlund+Partner AG Fon: +49-721-91374-0 E-Mail: [EMAIL PROTECTED]
