Re: Consider support for the Servlet profile of JSR 196 (JASPIC) in Tomcat 7.0.x
On 10/16/2015 5:51 AM, Mark Thomas wrote: On 16/10/2015 11:44, Rémy Maucherat wrote: What's not clear to me at this point is if jaspic should replace the current "classic" auth methods. For example, the supposed benefit of jaspic is that it makes container auth not proprietary, but looking at the code that is there it sounds still quite proprietary in practice and almost impossible to use from webapps. So what's the point exactly ? The main benefit for me is that once Tomcat implements JASPIC, there are a handful of 3rd party modules that users will then be able to use just by adding the lib to Tomcat. It also provides a standard API for users to work against if they want to develop their own custom modules. Those modules would then be usable with any container that implements JASPIC. Today if one wants a portable but custom authentication mechanism, one has to forgo/remove security constraints from one's web.xml and implement the mechanism as a servlet request filter -- with its own security constraint configuration mechanisms. My understanding with JASPIC is that one should be able to add authentication mechanisms that (1) use the same code across any container that supports JASPIC [as Mark notes] and (2) still use standard security constraints in one's web.xml to configure authentication constraints. -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: AJP Protocol enhancements
The one really compelling /usability /bit about AJP proxying is that requests look like you're in/at the web server in question and receiving requests directly from the client of the web server. The fact that you are not is completely and utterly transparent to your web application code. The same cannot be said for HTTP proxying, which throws the proxying details in your face and leaves you scrambling with various valves, etc, to hide these details from web applications that care about the logical, not physical, truth of the situation. -- Jess Holle
Re: Tomcat 9 connector refactoring: NIO vs NIO2
Naively, looking at the Tomcat documentation and note the comparison table there makes it appear that: 1. NIO2 offers everything NIO does 2. NIO cannot offer true blocking IO, whereas NIO2 can If that's not true, then the documentation should be updated at least And, of course, NIO2 just sounds newer/better/fancier than NIO :-) On 5/1/2015 4:55 AM, Mark Thomas wrote: Next up on my list is the NIO2 connector. NIO and APR/native both use a polling approach to non-blocking I/O. You add the socket to the poller, tell it what operation (read/write) you want to perform and then you wait for the poller to tell you the socket is ready to perform that operation. NIO2 uses an asynchronous approach to non-blocking I/O. You perform the read/write and then wait to be told it has finished via either a Future or a CompletionHandler. Servlet 3.1 non-blocking I/O is closest to the polling style (you get a callback when you are allowed to read/write). WebSocket non-blocking I/O uses the asynchronous style. In short, regardless of the underlying approach to non-blocking I/O, we have to support JavaEE APIs that use both styles. Therefore there is no 'obvious' advantage for either NIO or NIO2. As far as I can tell, the performance of NIO and NIO2 are comparable. That raises the question why do we need both NIO and NIO2? And I don't have an answer to that. If I had to pick one, I'd pick NIO because: - it has been around longer and is more stable - it uses the same style as APR/native which may allow further refactoring to reduce duplication. So, should we drop one of NIO or NIO2 in Tomcat 9? If not, why not? If yes, which one? Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
JNDIRealm bug
I just moved from Tomcat 7.0.56 to 8.0.15.
I tried out form-based authentication using CombinedRealm of 2 JNDIRealms.
I get:
java.lang.NullPointerException
org.apache.catalina.realm.JNDIRealm.getUser(JNDIRealm.java:1286)
org.apache.catalina.realm.JNDIRealm.getUser(JNDIRealm.java:1236)
org.apache.catalina.realm.JNDIRealm.authenticate(JNDIRealm.java:1177)
org.apache.catalina.realm.JNDIRealm.authenticate(JNDIRealm.java:1052)
org.apache.catalina.realm.CombinedRealm.authenticate(CombinedRealm.java:157)
org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:272)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:452)
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537)
org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:831)
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658)
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:277)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
java.lang.Thread.run(Thread.java:745)
This tracks directly to the following code added between these two releases:
if (userPassword == null credentials != null) {
// The password is available. Insert it since it may be required for
// role searches.
return new User(user.getUserName(), user.getDN(), credentials,
user.getRoles(), user.getUserRoleId());
}
The new User(...) line is the bad one. Earlier in the method there was
a search for a user:
if (userPatternFormatArray != null curUserPattern = 0) {
user = getUserByPattern(context, username, credentials, attrIds,
curUserPattern);
} else {
user = getUserBySearch(context, username, attrIds);
}
but there's no null check in between these two bodies of code, so this
is an obvious source of NPE's.
I'll patch this in my own Tomcat, but I wanted to notify the community so:
1. I don't have to maintain this patch long term and
2. The Tomcat community can review the correctness/sanity of this code
change and decide whether the correct fix is simply adding a null
check or something larger.
--
Jess Holle
Re: Time for 7.0.48
Given the regression in startup speed due to the web socket scanning, it would be /really/ nice if this release included clear notes on how one can best disable this. On 12/9/2013 4:41 AM, Violeta Georgieva wrote: Hi, 2013/11/16 Violeta Georgieva violet...@apache.org Hi, I'm planning to prepare 7.0.48 for voting. Please reply here if you would like to include something in that release. I'll tag 7.0.48 in the afternoon. Regards Violeta
Tomcat 7 - revision 1427821
Is anyone else seeing regressions due to this change? They appear as NullPointerExceptions from java.io.Writer.write(String) where the writer in question is a JSPWriter. This was addressed long ago by the fix for https://issues.apache.org/bugzilla/show_bug.cgi?id=35410, but the revision reverts the fix as it is not specification compliant. On the flip side, how do we know the fix was not specification compliant? Did a new TCK reject this behavior? Or were other compliant implementations found not to have this behavior? Or...? I'm now left trying to figure out what to do about this change -- push those which ran afoul of this to fix their code or patch the fix back into our Tomcat. -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Tomcat 7 - revision 1427821
Thanks for the feedback. It looks like our issues are tracing back to a single bad tag class implementation. On 1/30/2013 6:54 AM, Konstantin Kolinko wrote: 2013/1/30 Jess Holle je...@ptc.com: Is anyone else seeing regressions due to this change? They appear as NullPointerExceptions from java.io.Writer.write(String) where the writer in question is a JSPWriter. This was addressed long ago by the fix for https://issues.apache.org/bugzilla/show_bug.cgi?id=35410, but the revision reverts the fix as it is not specification compliant. On the flip side, how do we know the fix was not specification compliant? 1. Mainly by carefully reading the specifications. In this case, the javadoc for JspWriter and PrintWriter as well as JSP spec. 2. The fix to BZ 35410 does not make much sense, as a proper way to print null value is null, not . 3. There is no way to distinguish %= Object % and %= String % and %= int % when generating Java code for a JSP page. The Java code will be the same. You can protect from your issue by explicitly using %= String.valueOf(Object) % In this case String.valueOf(Object) will take care of Object being null and out.print(String) will take care of String being null. but personally I would prefer to fix that toString() method. Did a new TCK reject this behavior? Or were other compliant implementations found not to have this behavior? Or...? I'm now left trying to figure out what to do about this change -- push those which ran afoul of this to fix their code or patch the fix back into our Tomcat. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [RESULT] [VOTE] Release Apache Tomcat 7.0.35
On 1/18/2013 3:13 PM, Mark Thomas wrote: On 18/01/2013 19:32, Jeremy Boynes wrote: On Wed, Jan 16, 2013 at 1:41 AM, Mark Thomas ma...@apache.org wrote: Votes were as follows: +1 (binding): kkolinko, rjung, yoavs, olamy, markt No other votes were cast. The vote therefore passes. I'll publish the binaries and announce later today once the mirrors have caught up. Given the regression with issue 54440, should this be withdrawn and a quick release of '36 be done? https://issues.apache.org/bugzilla/show_bug.cgi?id=54440 I'm wasn't planning on it. If that bug is a showstopper then folks can stick on 7.0.34 for a few more weeks. Nothing in 7.0.35 is really that critical. For myself I went to 7.0.35, swore furiously about the bug -- almost just dropped back to 7.0.34 but then applied the patch and moved on. I guess the real question is how many new to Tomcat folk are realistically going to pick up 7.0.35 *and* jump to doing JSP precompilation *and* do so before 7.0.36 is out. If they're /not /new to Tomcat, then they know it *did* work and will start looking for where it stopped, check the mailing lists, etc. If not and they hit this issue before 7.0.36 is announced, then they'll just assume Tomcat is no good -- but how many folk will really fall into that category? -- Jess Holle
[OT] was Re: Time for 7.0.34
On 12/3/2012 5:09 AM, Mark Thomas wrote: On 03/12/2012 11:00, Konstantin Kolinko wrote: 1. I think we need to switch over to the recently released Daemon 1.0.11. There is a regression that needs to be fixed. I think we should wait for 1.0.12. Can you elaborate on the regression? [Having just moved to procrun 1.0.11 for something else, I'm quite interested.] -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [OT] was Re: Time for 7.0.34
On 12/3/2012 5:39 AM, Rainer Jung wrote: On 03.12.2012 12:27, Jess Holle wrote: On 12/3/2012 5:09 AM, Mark Thomas wrote: On 03/12/2012 11:00, Konstantin Kolinko wrote: 1. I think we need to switch over to the recently released Daemon 1.0.11. There is a regression that needs to be fixed. I think we should wait for 1.0.12. Can you elaborate on the regression? [Having just moved to procrun 1.0.11 for something else, I'm quite interested.] https://issues.apache.org/jira/browse/DAEMON-268 Also https://issues.apache.org/jira/browse/DAEMON-245 was solved incompletely, so https://issues.apache.org/jira/browse/DAEMON-269 was raised. That one isn't a regression/showstopper though. So for procrun, 269 is of interest, but not a regression, and 1.0.11 will do for now. [1.0.11 was key in actually allowing automatic restart, etc, actions upon service failure.] Thanks. -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: JSVC 1.0.11
On 11/20/2012 11:42 AM, Mladen Turk wrote: On 11/20/2012 06:25 PM, Jess Holle wrote: I'm also anxiously awaiting 1.0.11 (specifically the procrun piece) -- albeit separately from Tomcat. Originally there was talk of kicking off 1.0.11 this week, but I've not seen any information about test builds yet. Patience :) I'm working on it. Thanks! -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: JSVC 1.0.11
I'm also anxiously awaiting 1.0.11 (specifically the procrun piece) -- albeit separately from Tomcat. Originally there was talk of kicking off 1.0.11 this week, but I've not seen any information about test builds yet. On 11/20/2012 4:49 AM, Ognjen Blagojevic wrote: Konstantin, On 20.11.2012 11:37, Konstantin Kolinko wrote: 1. In what branch of Tomcat ? 7.0.x. 2. According to commons.apache.org, the latest released version of commons-daemon is 1.0.10. Until 1.0.11 is released there is no way to bundle it with Tomcat. D'oh! I didn't check that, sorry. -Ognjen - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Custom compression filters
It is quite possible to write a custom compression filter as a servlet filter that will work across multiple servlet engines. On 9/29/2012 12:33 PM, Florian Fray wrote: Dear TC Devs, while looking around in TC 7 I've found no possibility to plug-in a custom compression filter. So I've had a quick look at the code to find out that the gzip-handling is coded into AbstractHttp11Processor. Though I could create a custom processor, I'd like to know whether there's any interest in having the standard processors supporting custom compressions. Best regards, Florian - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Tomcat 5.5.36
From my perspective only 6.0.x and higher are interesting at this point, but I may be in the minority here. On 7/24/2012 7:40 AM, Jim Jagielski wrote: Any interest in seeing a 5.5.36 release in the near future? - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Timing for 7.0.29
Do you have a list of known regressions (vs. bugs just discovered since 7.0.27)? On 6/26/2012 3:11 PM, Mark Thomas wrote: I'm sure circumstances will conspire against me but I am currently planning to start the 7.0.29 release process early next week. That aligns with my general aim of getting a release out at the start of each month and there are some regressions in 7.0.28 that I'd rather fix sooner than later. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Tomcat 7.0.25 issue
If one's web-app does not contain WEB-INF/classes, Tomcat 7.0.25 logs an
error.
This has never been considered an error in any Tomcat to date that I am
aware of -- nor any other servlet engine I've tried.
This appears to be an accident as ContextConfig had:
webinfClasses = context.getServletContext().getResource(
/WEB-INF/classes);
processAnnotationsUrl(webinfClasses, webXml);
and now has
NamingEnumerationBinding listBindings =
context.getResources().listBindings(/WEB-INF/classes);
while (listBindings.hasMoreElements()) {
Binding binding = listBindings.nextElement();
if (binding.getObject() instanceof FileDirContext) {
File webInfCLassDir =
new File(
((FileDirContext)
binding.getObject()).getDocBase());
processAnnotationsFile(webInfCLassDir, webXml);
}
}
which is all well and good except for the differences in behavior when
/WEB-INF/classes is not found.
Is there a bug and/or patch for this?
If not I can file a bug -- and hack my own patch, I suppose, though I'd
appreciate a stab at a patch by someone who is actually familiar with
this code (which I am not).
--
Jess Holle
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Tomcat 7.0.25 issue
Note that the exception logged says:
2012-01-23 09:13:01,010 ERROR [pool-8-thread-1]
org.apache.catalina.startup.ContextConfig - Unable to determine URL for
WEB-INF/classes
javax.naming.NameNotFoundException: Resource /WEB-INF/classes not found
at
org.apache.naming.resources.BaseDirContext.listBindings(BaseDirContext.java:733)
at
org.apache.naming.resources.ProxyDirContext.listBindings(ProxyDirContext.java:546)
at
org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1197)
at
org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:825)
at
org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:300)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5161)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:897)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Clearly the lack of /WEB-INF/classes is not worthy of an exception, nor
an error.
On 1/23/2012 2:15 PM, Jess Holle wrote:
If one's web-app does not contain WEB-INF/classes, Tomcat 7.0.25 logs
an error.
This has never been considered an error in any Tomcat to date that I
am aware of -- nor any other servlet engine I've tried.
This appears to be an accident as ContextConfig had:
webinfClasses =
context.getServletContext().getResource(
/WEB-INF/classes);
processAnnotationsUrl(webinfClasses, webXml);
and now has
NamingEnumerationBinding listBindings =
context.getResources().listBindings(/WEB-INF/classes);
while (listBindings.hasMoreElements()) {
Binding binding = listBindings.nextElement();
if (binding.getObject() instanceof
FileDirContext) {
File webInfCLassDir =
new File(
((FileDirContext)
binding.getObject()).getDocBase());
processAnnotationsFile(webInfCLassDir,
webXml);
}
}
which is all well and good except for the differences in behavior when
/WEB-INF/classes is not found.
Is there a bug and/or patch for this?
If not I can file a bug -- and hack my own patch, I suppose, though
I'd appreciate a stab at a patch by someone who is actually familiar
with this code (which I am not).
--
Jess Holle
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Tomcat 7.0.25 issue
Okay, I got impatient since I couldn't find any matching bug in the
database and filed bug 52511
https://issues.apache.org/bugzilla/show_bug.cgi?id=52511.
On 1/23/2012 2:56 PM, Jess Holle wrote:
Note that the exception logged says:
2012-01-23 09:13:01,010 ERROR [pool-8-thread-1]
org.apache.catalina.startup.ContextConfig - Unable to determine URL
for WEB-INF/classes
javax.naming.NameNotFoundException: Resource /WEB-INF/classes not found
at
org.apache.naming.resources.BaseDirContext.listBindings(BaseDirContext.java:733)
at
org.apache.naming.resources.ProxyDirContext.listBindings(ProxyDirContext.java:546)
at
org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1197)
at
org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:825)
at
org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:300)
at
org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at
org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5161)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:897)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:615)
at
org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:649)
at
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1581)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Clearly the lack of /WEB-INF/classes is not worthy of an exception,
nor an error.
On 1/23/2012 2:15 PM, Jess Holle wrote:
If one's web-app does not contain WEB-INF/classes, Tomcat 7.0.25 logs
an error.
This has never been considered an error in any Tomcat to date that I
am aware of -- nor any other servlet engine I've tried.
This appears to be an accident as ContextConfig had:
webinfClasses =
context.getServletContext().getResource(
/WEB-INF/classes);
processAnnotationsUrl(webinfClasses, webXml);
and now has
NamingEnumerationBinding listBindings =
context.getResources().listBindings(/WEB-INF/classes);
while (listBindings.hasMoreElements()) {
Binding binding =
listBindings.nextElement();
if (binding.getObject() instanceof
FileDirContext) {
File webInfCLassDir =
new File(
((FileDirContext)
binding.getObject()).getDocBase());
processAnnotationsFile(webInfCLassDir, webXml);
}
}
which is all well and good except for the differences in behavior
when /WEB-INF/classes is not found.
Is there a bug and/or patch for this?
If not I can file a bug -- and hack my own patch, I suppose, though
I'd appreciate a stab at a patch by someone who is actually familiar
with this code (which I am not).
--
Jess Holle
Re: javax.servlet.ServletRequest and thread safety...
On 11/21/2011 5:26 PM, Caldarale, Charles R wrote: From: Jess Holle [mailto:je...@ptc.com] Subject: javax.servlet.ServletRequest and thread safety... Is there an issue with multi-threaded access here or am I imagining things? There is an issue, and it's highly unlikely to ever be changed. The spec defines the APIs as not thread-safe, so there is zero impetus to make them so and thereby impact performance. Okay. So the workaround appears to be to call getParameterMap() [for starters] prior to any potential access by another thread. Clearly there should be *some* way to add a means to peak on requests in progress in detail (including their parameters) -- irrespective of what the spec says. From a quality of implementation standpoint this shouldn't have to copy all the request data -- but forcing it to be obtained once prior to access by other threads is quite reasonable. -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PROPOSAL] Enable session replication by default.
web apps whose web.xml does not specify distributable should certainly not be treated as such -- that would be a spec violation and break lots of web apps. On 6/7/2011 2:08 PM, Christopher Schultz wrote: Mark, On 6/7/2011 2:36 PM, Mark Thomas wrote: On 07/06/2011 19:30, Christopher Schultz wrote: Keiichi, On 6/7/2011 7:37 AM, Keiichi Fujino wrote: Index: conf/web.xml === --- conf/web.xml(revision 1127122) +++ conf/web.xml(working copy) @@ -4176,4 +4176,6 @@ welcome-fileindex.jsp/welcome-file /welcome-file-list +!-- distributable = -- +distributable/ /web-app -1 This will cause web applications to fail when adding session attributes that are not Serializable. Not it won't. This is *not* changing the default value of distributable. Thatdistributable flag is in conf/web.xml, which I assumed set defaults for all web applications. Isdistributable handled differently than, say,servlet-mapping or anything else we put into conf/web.xml? I apologize for not having read the code at this point. -chris - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: 7.0.13 release for HTTP pipelining fix
Fortunately I'm only using Tomcat 7 in development, not production, at this point. That said, this issue would be a showstopper for me moving to Tomcat 7 for production usage. BIO still seems like a fine option for many cases when using Tomcat behind mod_jk. APR connectors are still not as proven as BIO ones overall -- bugs are still being shaken out. APR is also just too painful when dealing with multiple platforms (building additional native code for 7 platforms some of which have loathsome linkers -- yuck!), introduces too great a likelihood of platform specific bugs/behaviors and will likely never be attractive to me. NIO might be (once all the kinks were worked out), but there is no NIO implementation of the AJP connector. -- Jess Holle On 4/6/2011 5:02 PM, Tim Whittington wrote: Given the severity of https://issues.apache.org/bugzilla/show_bug.cgi?id=50957, would it be a good idea to get a 7.0.13 release out soon? 7.0.12 mitigated the issue to a degree, but I'd feel better with a complete fix. cheers tim - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 7.0.4
On 10/20/2010 5:45 AM, Mark Thomas wrote: On 19/10/2010 08:56, Mark Thomas wrote: Ping. Just a gentle reminder that there are ~2 days left for this vote. So far we have 1 vote for beta and no other votes. Sorry - it should have said ~1 day above. I've been traveling and got my dates mixed up. I'll leave the vote open for another 24 hours or so. Currently there are 4 votes for beta (2*PMC, 1*committer, 1*contributor) so we need at least 1 more PMC vote in order to proceed with this release. As someone trying to figure out when to take the plunge into Tomcat 7, but needing something that is definitely stable, is there any sort of list as to what hurdles remain to be cleared before considering Tomcat 7 is considered stable? I'm not trying to rush anyone (most especially a premature labeling of Tomcat 7 as stable), but some insight into the remaining gap between Tomcat 7 and stability would be helpful to me -- and others as well, I suspect. If there's just a collective gut feeling that more experience with Tomcat 7 is needed prior to feeling comfortable with a stable label that's fine, of course, but right now I have no sense as to what makes Tomcat 7 still a beta at this point. -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 7.0.4
Thanks for the detailed reply! On 10/20/2010 7:03 AM, Mark Thomas wrote: On 20/10/2010 06:39, Jess Holle wrote: On 10/20/2010 5:45 AM, Mark Thomas wrote: On 19/10/2010 08:56, Mark Thomas wrote: Ping. Just a gentle reminder that there are ~2 days left for this vote. So far we have 1 vote for beta and no other votes. Sorry - it should have said ~1 day above. I've been traveling and got my dates mixed up. I'll leave the vote open for another 24 hours or so. Currently there are 4 votes for beta (2*PMC, 1*committer, 1*contributor) so we need at least 1 more PMC vote in order to proceed with this release. As someone trying to figure out when to take the plunge into Tomcat 7, but needing something that is definitely stable, is there any sort of list as to what hurdles remain to be cleared before considering Tomcat 7 is considered stable? My own view is that to be considered stable, Tomcat 7 needs to meet the following criteria: 1. Implement all aspects of Servlet 3.0, JSP 2.2, EL 2.2 2. Pass all unit tests with all three HTTP connectors 4. Pass all relevant TCKs with the security manager enabled - Servlet TCK with all three HTTP connectors and both AJP connectors - JSP TCK with any connector - EL TCK (doesn't use web requests) 4. Have no 'significant' open bugs 5. Have reasonable adoption 6. Have a couple of releases with no 'serious' bugs emerging In term of progress: 1. Done (to the best of my knowledge). 2. It does. 3. It does (as have all 7.0.x releases). 4. There is currently 1 (yes one!) open bug without a patch across 5.5.x, 6.0.x and 7.0.x so I think we can call this one done. 5. Based on some analysis of download requests and the number and quality of bug reports I am happy that there is reasonable adoption at this stage. 6. I see this as the only thing between 7.0.x and stability. Serious is subjective but the sort of things I would include are: - anything that requires a major refactoring to fix - anything that breaks typical use cases As an example, I would consider another bug 49884 serious due to both the async issues it caused and the scale of the refactoring required to fix. I wouldn't consider another 50072 serious mainly because that issue has been present in the 6.0.x code base and hasn't been a problem (at least not one folks have reported). So in summary, if 7.0.4 and 7.0.5 go well, things are looking good for 7.0.6. HTH, Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Proposed mod_jk logging patch
I have noticed that mod_jk logging about a dead server is overtly
verbose in some circumstances.
If you use it to load balance over a number of servers and one is dead
you'll get several lines of error logging every time it retries the
server (to see if it's alive yet). This can get rather obnoxious when
you're balancing over a number of ports which may or may not have a
server listening at the time -- and when you're allowing retries of dead
servers with any frequency.
The attached patch changes the level of such logging to debug for
retries of a worker known to be in an error state, leaving the level at
error for other cases. The result is that you get error logging when a
server is first determined to be unavailable -- and then are simply not
bothered about this any longer.
Is there any chance of merging this patch into mod_jk? The current
level of log verbosity just isn't acceptable in cases where one is load
balancing over a sparsely populated range of server ports, for instance.
--
Jess Holle
P.S. I already proposed a similar patch for mod_proxy_balancer/ajp.
There appear to be additional issues there (having to do with load
balancing getting stuck on a subset of the members), however, which
are pushing us back to mod_jk anyway.
--- native/common/jk_ajp_common.c.orig 2009-04-07 12:56:25.926105900 -0500
+++ native/common/jk_ajp_common.c 2009-04-07 12:53:22.408773900 -0500
@@ -1392,7 +1392,8 @@
static int ajp_send_request(jk_endpoint_t *e,
jk_ws_service_t *s,
jk_logger_t *l,
-ajp_endpoint_t * ae, ajp_operation_t * op)
+ajp_endpoint_t * ae, ajp_operation_t * op,
+int probing) /* Added 'probing', which is true
when probing/retrying failed worker [PTC] */
{
int err_conn = 0;
int err_cping = 0;
@@ -1504,6 +1505,14 @@
/* Connect to the backend.
*/
if (ajp_connect_to_endpoint(ae, l) != JK_TRUE) {
+/* Log at debug level rather than error level when 'probing' [PTC]
+*/
+if ( probing )
+jk_log(l, JK_LOG_DEBUG,
+ (%s) connecting to backend failed. Tomcat is probably
not started
+ or is listening on the wrong port (errno=%d),
+ ae-worker-name, ae-last_errno);
+else
jk_log(l, JK_LOG_ERROR,
(%s) connecting to backend failed. Tomcat is probably not
started
or is listening on the wrong port (errno=%d),
@@ -2189,6 +2198,7 @@
int rc = JK_UNSET;
char *msg = ;
int retry_interval;
+int probing; /* Added [PTC] */
JK_TRACE_ENTER(l);
@@ -2286,6 +2296,10 @@
aw-s-busy++;
if (aw-s-state == JK_AJP_STATE_ERROR)
aw-s-state = JK_AJP_STATE_PROBE;
+/* Set 'probing' to true when aw-s-state == JK_AJP_STATE_PROBE;
+ indicates when worker is being probed/retried [PTC]
+*/
+probing = ( aw-s-state == JK_AJP_STATE_PROBE );
if (aw-s-busy aw-s-max_busy)
aw-s-max_busy = aw-s-busy;
retry_interval = p-worker-retry_interval;
@@ -2317,7 +2331,7 @@
log_error = JK_TRUE;
rc = JK_UNSET;
msg = ;
-err = ajp_send_request(e, s, l, p, op);
+err = ajp_send_request(e, s, l, p, op, probing); /* pass 'probing' to
ajp_send_request() [PTC] */
e-recoverable = op-recoverable;
if (err == JK_CLIENT_RD_ERROR) {
*is_error = JK_HTTP_BAD_REQUEST;
@@ -2463,6 +2477,13 @@
ajp_next_connection(p, l);
}
/* Log the error only once per failed request. */
+/* Log at debug level rather than error level when 'probing' [PTC]
+*/
+if ( probing )
+jk_log(l, JK_LOG_DEBUG,
+ (%s) connecting to tomcat failed.,
+ aw-name);
+else
jk_log(l, JK_LOG_ERROR,
(%s) connecting to tomcat failed.,
aw-name);
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Proposed mod_jk logging patch
Rainer Jung wrote: I find it hard to decide between your case (we know the nodes are not available and we don't need a reminder every minute, instead we want to see the real errors), and the most common case (we didn't see the single initial message a few days ago and so we didn't realize our nodes were partially down for a long time). So let me first ask: why don't you stop the nodes, you know are out of service? If you let the balancer know, what your admins know about the state of the system, the balancer will no longer throw errors. This really in some respects a mod_cluster sort of thing. I have a bank of ports in which a smaller number of server processes (embedding Tomcat) will be dynamically started. These will continue to reside on these ports unless/until they hang or die -- at which point a daemon/manager process will start other server processes in the port range -- on whatever ports they can successfully bind to. Having the daemon/manager process message to mod_jk as to which ports to start/stop all the time seems like an undesirable complexity and tight coupling. Ideally the servers shouldn't even know which Apache(s) are targeting them, which module is being used, mod_jk or mod_proxy_balancer/ajp (or possibly mod_cluster at some point), etc. Perhaps there should be a configurable boolean as to whether this should be logged noisily or quietly to meet both use cases? [Note I need IIS and SJWS support as well as Apache 2.2 and so will need to rely on the jk/tc connectors in these cases in any case and will need to be able to configure any such setting in all cases.] -- Jess Holle - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: gsoc project - Tomcat proxy [query]
A Tomcat-based reverse proxy with a pluggable load balancer would be /huge/ in my opinion. Having to resort to Apache (or a hardware load balancer) to distribute load across multiple Tomcats is /really /unfortunate. What's needed is a high-performance Java-based proxy load balancer that makes it easy to get at the balancing algorithm. This should support use cases like balancing new requests to the backend that currently has the fewest active sessions or to avoid any backend which is in a low-memory state, for instance. Trying to hack this sort of thing into mod_proxy_balancer or mod_jk is just so much harder than doing this sort of thing in Java. Plus there's the benefit of build once run anywhere, i.e. cross-platform binaries. -- Jess Holle P.S. Caching at this level is quite uninteresting to me -- but could seemingly be added via servlet request filters assuming the reverse proxy was itself done as a servlet. Ideally the reverse proxy would even be portable across servlet engines and designed for exposure via JMX. [I say designed for as it seems everyone has their own way of actually /doing/ JMX MBeans -- Tomcat uses Modeler and others use something different. Me? I prefer just subclassing StandardMBean -- but with my own value-add subclass thereof, of course :-)] jean-frederic clere wrote: Sharmistha jat wrote: Dear All, I have applied for tomcat proxy project, but i needed a little more information than given in the project idea, to start with the design of proxy. So i would like to know more about the following: 1) Type of proxy [reverse,content filter,caching etc]. Just a reverse proxy. For the content filter I think you mean soemthing like the ProxyPass and ProxyMatch of httpd, do you? If yes some filtering would be nice. Caching is interesting but not a requirement for a reverse proxy (you can have the static resource locally and server them locally). 2) Use [intent] The use would be to front-end TC or cluster of TC. httpd does that but the code is in C and it is a lot of work to have JAVA based administration (JMX for example). The proxy should support HTTP and HTTPS and could support AJP if you want. I have studied a number of servlet proxies like j2ep, noodle Http Proxy Servlet in past days. Could you write a wiki on that? (Somewhere in http://wiki.apache.org/tomcat/). Cheers Jean-Frederic And was puzzled about my intent of making proxy and its designing thereof. So, please guide me a little bit about this. Thanks Sharmistha v javascript://nop/ - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: lastAccessedTime vs. thisAccessedTime
I won't say I understand all the usages perfectly myself, but I also stumbled on this mess. I made a few such changes our Tomcat in the session package in conjunction with changes to PersistentManagerBase, FileStore, JDBCStore, etc (as these do too much thrashing reading all sessions from disk/database on maintenance intervals rather trying to track which might actually be expired, etc). [I provided patches but didn't have time to slice these up into lots of little patches.] Rainer Jung wrote: Hi, I tried to understand the usage of lastAccessedTime (lastAT) vs. thisAccessedTime (thisAT) in Tomcat 6 (I suppost trunk is the same but didn't yet check). As I understand it, the two different timestamps get used, because when a request is associated with a session and asks for the lastAT it should get the previous access time, not the one related to this request itself. The way it seems implemented at the moment is: - if a request for some session is in the state of being processed, lastAT gives the previous access time and thisAT give the time resulting from the request itself - if no such request exists, thisAT returns the last access time of the session, and lastAT returns the time of the second to last request! Whenever we need a lastAT for a session for purposes not directly related to request handling for this session we use lastAT, although thisAT should have the correct semantics in this case. Places where we should replace getLastAccessedTime() by getThisAccessedTime(): session/PersistentManagerBase.java: could swap session out although a request for the session is concurrently being processed on manager/ManagerServlet.java: idleness profile manager/util/SessionUtils.java: inside getUsedTimeForSession(), getTTLForSession(), getInactiveTimeForSession(). Those are used when producing lists with session information. ha/session/mbeans-descriptors.xml: should either expose both values or preferrably return thisAT as a lastAccessedTime, because the use case is not retrieving info for the own session while running a request. Places where I'm not so sure: valves/PersistentValve.java: not sure, whether the valve runs before access() updates the access times. If so, we should switch to thisAT here, if not, keep as is ha/session/SimpleTcpReplicationManager.java: readSession() sets both to current time ? session/JDBCStore.java: maybe should store both values, but not sure where they get read in Remarks? Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: In-process connector?
Having looked into this further it seems clear Tomcat is not designed to provide any reasonably reliable / easy-to-use means of making a JSP or servlet request from within the process but outside a connector request thread. If one wants to properly do this it would appear one has to call org.apache.catalina.core.StandardEngineValve.invoke() oneself -- with Tomcat Request and Response objects. Ideally there would be a way to instead implement HttpServletRequest and HttpServletResponse as desired, mimicking remote client to the degree needed, get a RequestDispatcher [from a cached ServletContext], and call forward(), for instance -- possibly capturing the context class loader in a ContextListener and setting that as the context class loader around all this. That /kind of/ works for include(), but skips all servlet filters and does not work at all for forward(). I fully realize the servlet specification spells out no such mechanism, but I believe there is a clear gap here. One should be able to embed Tomcat in an application and /easily /invoke a JSP or servlet to produce an HTML e-mail body, for instance. This should involve a minimum of Tomcat-specific APIs. It is hard to understand why one should be unable to re-use one's JSPs and servlet investment to produce HTML for other purposes than HTTP responses.// It is also quite unclear to me why there appears to be no interest in addressing this need. -- Jess Holle Jess Holle wrote: I find no user documentation on anything like what I'm looking for. I thus am /assuming/ this would be an extension to Tomcat and thus more appropriate for a development forum than a user forum. I want to call JSPs, etc, from Java code to generate HTML. I'm not looking for a web server connector -- that would be a clear user group question. Remy Maucherat wrote: On Wed, 2008-09-17 at 07:48 -0500, Jess Holle wrote: Is there an in-process connector for Tomcat somewhere out there? Specifically I want to be able to embed Tomcat and use JSPs, etc, to generate HTML but without requiring loopback HTTP(S) requests. Does such a thing exist? If not, would this be hard to add? Pointers and suggestions would be much appreciated. There is a user list for questions on Tomcat usage. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: In-process connector?
Cool. Thanks for the helpful reply. I just wish I could have figured out how to word my question better so as to elicit this helpful information to start with -- instead of getting told I was posting to the wrong group and chasing my tail with it. -- Jess Holle Costin Manolache wrote: Well, there is interest - the tomcat-lite in the sandbox does allow that ( there is even a TomcatLiteNoConnector unit test ). The code is broken right now - in process of moving part of it to trunk and adjusting it to be easier to digest. Will take quite a while before it's ready for an official release - but ability to run 'in process' ( either as unit tests, or as you describe ) is one of the goals. Costin On Mon, Oct 6, 2008 at 11:16 AM, Jess Holle [EMAIL PROTECTED] wrote: Having looked into this further it seems clear Tomcat is not designed to provide any reasonably reliable / easy-to-use means of making a JSP or servlet request from within the process but outside a connector request thread. If one wants to properly do this it would appear one has to call org.apache.catalina.core.StandardEngineValve.invoke() oneself -- with Tomcat Request and Response objects. Ideally there would be a way to instead implement HttpServletRequest and HttpServletResponse as desired, mimicking remote client to the degree needed, get a RequestDispatcher [from a cached ServletContext], and call forward(), for instance -- possibly capturing the context class loader in a ContextListener and setting that as the context class loader around all this. That /kind of/ works for include(), but skips all servlet filters and does not work at all for forward(). I fully realize the servlet specification spells out no such mechanism, but I believe there is a clear gap here. One should be able to embed Tomcat in an application and /easily /invoke a JSP or servlet to produce an HTML e-mail body, for instance. This should involve a minimum of Tomcat-specific APIs. It is hard to understand why one should be unable to re-use one's JSPs and servlet investment to produce HTML for other purposes than HTTP responses.// It is also quite unclear to me why there appears to be no interest in addressing this need. -- Jess Holle Jess Holle wrote: I find no user documentation on anything like what I'm looking for. I thus am /assuming/ this would be an extension to Tomcat and thus more appropriate for a development forum than a user forum. I want to call JSPs, etc, from Java code to generate HTML. I'm not looking for a web server connector -- that would be a clear user group question. Remy Maucherat wrote On Wed, 2008-09-17 at 07:48 -0500, Jess Holle wrote Is there an in-process connector for Tomcat somewhere out there? Specifically I want to be able to embed Tomcat and use JSPs, etc, to generate HTML but without requiring loopback HTTP(S) requests. Does such a thing exist? If not, would this be hard to add? Pointers and suggestions would be much appreciated. There is a user list for questions on Tomcat usage. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
In-process connector?
Is there an in-process connector for Tomcat somewhere out there? Specifically I want to be able to embed Tomcat and use JSPs, etc, to generate HTML but without requiring loopback HTTP(S) requests. Does such a thing exist? If not, would this be hard to add? Pointers and suggestions would be much appreciated. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: In-process connector?
I guess I should clarify. I want Java code to do in-process requests to JSPs (etc) for HTML. This shouldn't involve sockets or JNI -- just a plain simple mechanism to take ServletRequest/HttpServletRequest and ServletResponse/HttpServletResponse objects and toss these (in process) at Tomcat and let it pass these to the normal FilterChain, etc, with the output written to the response stream/writer. Nothing terribly fancy, but this would allow in-process use of JSPs to generate HTML for various non-browser purposes. Otherwise one is stuck with the unnecessary dichotomy of generating browser HTML one way and all other HTML some other way -- unless one forgoes any technology, like JSP, that relies on the servlet API. Henri Gomez wrote: There was a JNI connector sometimes ago. I'm unsure if it's still maintened 2008/9/17 Jess Holle [EMAIL PROTECTED]: Is there an in-process connector for Tomcat somewhere out there? Specifically I want to be able to embed Tomcat and use JSPs, etc, to generate HTML but without requiring loopback HTTP(S) requests. Does such a thing exist? If not, would this be hard to add? Pointers and suggestions would be much appreciated. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: In-process connector?
I find no user documentation on anything like what I'm looking for. I thus am /assuming/ this would be an extension to Tomcat and thus more appropriate for a development forum than a user forum. I want to call JSPs, etc, from Java code to generate HTML. I'm not looking for a web server connector -- that would be a clear user group question. Remy Maucherat wrote: On Wed, 2008-09-17 at 07:48 -0500, Jess Holle wrote: Is there an in-process connector for Tomcat somewhere out there? Specifically I want to be able to embed Tomcat and use JSPs, etc, to generate HTML but without requiring loopback HTTP(S) requests. Does such a thing exist? If not, would this be hard to add? Pointers and suggestions would be much appreciated. There is a user list for questions on Tomcat usage. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat-Lite - part 1
Remy Maucherat wrote: I am interested to look at the code like the proxy, and see if what it can do. I have been longing for a Java-based load-balancing proxy replacement for Apache httpd. Essentially with non-blocking IO it would seem high time to replace Apache httpd with a Java-based web server that can then use flexible, pluggable load balancing algorithms written in Java. There are certain aspects of AJP that would seem nice to preserve (e.g. the notion from a servlet API perspective that you are still receiving the request direct from the client that the front end web server isn't even there). These could also be done in Java, of course. The only other big nicety of Apache httpd is good LDAP authentication (in 2.2) including support for authenticating a single resource against multiple LDAPs, caching LDAP results, etc. Do all this in Tomcat and then you could have a Tomcat front-end load balancing over Tomcats without any need for the native annoyances of httpd. -- Jess Holle P.S. Yes, I like httpd compared to other native alternatives, but having to do any builds and patches in native code, worrying about platform nuances, etc, is such a nuisance compared to just using Java. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_balancer issue
Mladen Turk wrote: Jess Holle wrote: P.S. I'd also like to quiet attempts to recover workers from errors to a lower (and by default unlogged) logging level. The transition of a worker into an error state should certainly be logged, but logging every time we find it to still be in an error state seems to be excessive -- at least for a sparsely populated port bank use case. Everything you said is fine with me. I just did a functional logic via watchdog thread and two basic use cases (maintain and keepalive). The keepalive is meant to deal with firewalls that tends to cut inactive connections. As far as I'm concerned feel free to extend the logic so it can deal with error states more intelligently. ... and of course, the plan is to implement something or all of this to mod_proxy, but using a different mechanism. Many modules needs some sort of out of the request cycle detached maintenance, so I'll add a common mpm maintenance thread to lower down the resource usage. Do you have any /rough/ ideas of the timeframe in which you might get such a thread into mod_proxy? Obviously I can start on mod_jk first. -- Jess Holle
Re: mod_proxy_balancer issue
Jim Jagielski wrote: On Aug 28, 2008, at 12:41 AM, Mladen Turk wrote: Jess Holle wrote: P.S. I'd also like to quiet attempts to recover workers from errors to a lower (and by default unlogged) logging level. The transition of a worker into an error state should certainly be logged, but logging every time we find it to still be in an error state seems to be excessive -- at least for a sparsely populated port bank use case. Everything you said is fine with me. I just did a functional logic via watchdog thread and two basic use cases (maintain and keepalive). The keepalive is meant to deal with firewalls that tends to cut inactive connections. As far as I'm concerned feel free to extend the logic so it can deal with error states more intelligently. ... and of course, the plan is to implement something or all of this to mod_proxy, but using a different mechanism. Many modules needs some sort of out of the request cycle detached maintenance, so I'll add a common mpm maintenance thread to lower down the resource usage. Ummm... since this is regarding mod_proxy, all this should be on the httpd-dev list, and not on the Tomcat dev list... What is, or is not, included in mod_proxy is what the httpd dev community decides :) :) Agreed, but this is concerning *both* mod_jk and mod_proxy and I've been scolded for cross posting... At the core, I'm looking for 2 things: 1. Something to limit the maximum impact of having many dead workers under a load balancer on normal requests * Assuming there are live workers available, of course, i.e. the process of discovering that dead workers are still dead shouldn't overtly impact any normal request. * Sample situation: load balancing over 9 ports, many of which do not have an active Tomcat associated at the time. If there is only one Tomcat alive every 'retry' seconds (in the mod_proxy case, same thing, but different parameters for mod_jk), a normal request is delayed by a period of 8*dead-connection-latency. That's neither necessary nor acceptable. * Possible solutions include having a background thread ping the health of workers rather than allowing normal requests to do so or limiting the number of workers any normal request will attempt to recover. 2. Something to reduce the severity of log messages when discovering that a dead worker is still dead. * There is no need to fill the error logs with notices that a worker that has been dead is still dead. This is good troubleshooting info and should be logged, but at a lower severity level that does not show up in the logs by default. * Depending on the solution to (1), this might just fall out of that. If you believe we should start up a discussion on httpd dev as to how to solve these issues at this point that sounds great to me -- though we also need to solve them in mod_jk (as our current plan is to use mod_proxy_ajp for Apache 2.2 and we have no alternative but the jk/isapi connector for IIS). -- Jess Holle
Re: mod_proxy_balancer issue
Jess Holle wrote: Agreed, but this is concerning *both* mod_jk and mod_proxy and I've been scolded for cross posting... At the core, I'm looking for 2 things: 1. Something to limit the maximum impact of having many dead workers under a load balancer on normal requests * Assuming there are live workers available, of course, i.e. the process of discovering that dead workers are still dead shouldn't overtly impact any normal request. * Sample situation: load balancing over 9 ports, many of which do not have an active Tomcat associated at the time. If there is only one Tomcat alive every 'retry' seconds (in the mod_proxy case, same thing, but different parameters for mod_jk), a normal request is delayed by a period of 8*dead-connection-latency. That's neither necessary nor acceptable. * Possible solutions include having a background thread ping the health of workers rather than allowing normal requests to do so or limiting the number of workers any normal request will attempt to recover. Clearly if we take the approach of a backgroung ping solution, it would be ideal to use cping rather than any sort of real request. 1. Something to reduce the severity of log messages when discovering that a dead worker is still dead. * There is no need to fill the error logs with notices that a worker that has been dead is still dead. This is good troubleshooting info and should be logged, but at a lower severity level that does not show up in the logs by default. * Depending on the solution to (1), this might just fall out of that. If you believe we should start up a discussion on httpd dev as to how to solve these issues at this point that sounds great to me -- though we also need to solve them in mod_jk (as our current plan is to use mod_proxy_ajp for Apache 2.2 and we have no alternative but the jk/isapi connector for IIS). Also to be clear I need solutions to these problems one way or another, so if nothing else I'll have to hack in something into our own fork of the code. I have a fair amount of time to solve these problems, however, so I'd much rather see them solved in a good, general way that can be a value-add part of mod_jk and mod_proxy -- rather than a one-off fork. -- Jess Holle
Re: mod_proxy_balancer issue
Mladen Turk wrote: Jess Holle wrote: I'm quite willing to help, but you're clearly much more familiar with both mod_jk and APR/MPM than I am, so I suspect I'd just get in the way except at the testing level. Well, I plan to create mpm watchdog hook system first. I'll also create a small callback in mod_proxy, just to maintain the connection pool so you are free to implement any additional mod_proxy interval operations (like heath checking) from there. Is there already something along these lines in mod_jk (or more specifically the IIS connector)? I did a little simplistic testing and haven't gotten down and dirty to debug this, but I have no managed to figure out any configuration of mod_jk where the load balancers' dead workers will be tested in the background and avoid testing this with foreground requests. I have a simple ping request that takes many seconds all too often, presumably delayed by re-checking all the dead workers. I have a worker.maintain of 15 and a recover_time for my balanced worker of 30 -- under the theory that background maintenance should be able to test any dead connections and mark them as recently tested without any request threads ever checking the workers. This assumes that the recover_time is only observed by request threads, though. Is there a means of achieving background-only (or nearly so) testing of dead workers with mod_jk? That's what I'm looking for in both jk and mod_proxy_ajp connectors. I guess I was hoping/assuming it was there in mod_jk from reading the docs. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_balancer issue
Mladen Turk wrote: Jess Holle wrote: Mladen Turk wrote: Is there a means of achieving background-only (or nearly so) testing of dead workers with mod_jk? That's what I'm looking for in both jk and mod_proxy_ajp connectors. I guess I was hoping/assuming it was there in mod_jk from reading the docs. There is in the mod_jk (SVN trunk). Ah... I'd been looking in 1.2.26. Add JkWatchdogInterval 60 This will create thread calling maintain each 60 seconds. How does this differ from worker.maintain? If you add worker.xxx.connection_keepalive=60 it will fire a cping/cpong on all connections in connection pool that were not used more then 60 seconds What about for load balance members with no established connections? My primary case here is a fairly large bank of ports which will be sparsely populated with Tomcats. I do not want normal request threads to spend time seeing if the ports have come to life, but I want a background thread to do so periodically. I'm thinking I'd want most of the workers being load balanced over to start in a presumed dead state until the background thread discovered they were alive. Does this sound close? I will take a look at the SVN code. I take it there's no such code in Apache SVN yet for mod_proxy_ajp? [I did look there.] -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_balancer issue
Jess Holle wrote: Add JkWatchdogInterval 60 Also how/where would one specify this for the IIS/Tomcat connector? [I need such a capability for IIS and Apache 2.2, specifically.] -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_balancer issue
Jess Holle wrote: Mladen Turk wrote: Jess Holle wrote: Mladen Turk wrote: Is there a means of achieving background-only (or nearly so) testing of dead workers with mod_jk? That's what I'm looking for in both jk and mod_proxy_ajp connectors. I guess I was hoping/assuming it was there in mod_jk from reading the docs. There is in the mod_jk (SVN trunk). I've been reading this code now... The watchdog thread looks very useful. If I understand it correctly, the watchdog thread can do whatever it feels like but currently mainly calls wc_maintain, which will only do work at most every worker.maintain seconds, right? connection_keepalive does not look like it really my bill, though. I'm most worried about workers in an error state and ensuring that they are rechecked every recover_wait_time -- but only by the watchdog thread and ideally via a ping/pong. Currently recover_workers appears to just put workers into a recovery state where they'll be elligible to be tried again on a future request -- without checking whether the worker is actually accessible. That's fine for some use cases, but explicitly what I want to avoid. Are there any thoughts to have an option to have recover_workers() do a ping prior to returning a working to a non-error state? And, yes, a watchdog thread in mod_proxy_balancer /and /a reasonable means of balancer invoking a ping via mod_proxy_ajp would be really helpful as far as mod_proxy_ajp is concerned. -- Jess Holle
Re: mod_proxy_balancer issue
Jess Holle wrote: Jess Holle wrote: Mladen Turk wrote: Jess Holle wrote: Mladen Turk wrote: Is there a means of achieving background-only (or nearly so) testing of dead workers with mod_jk? That's what I'm looking for in both jk and mod_proxy_ajp connectors. I guess I was hoping/assuming it was there in mod_jk from reading the docs. There is in the mod_jk (SVN trunk). I've been reading this code now... The watchdog thread looks very useful. If I understand it correctly, the watchdog thread can do whatever it feels like but currently mainly calls wc_maintain, which will only do work at most every worker.maintain seconds, right? connection_keepalive does not look like it really my bill, though. I'm most worried about workers in an error state and ensuring that they are rechecked every recover_wait_time -- but only by the watchdog thread and ideally via a ping/pong. Currently recover_workers appears to just put workers into a recovery state where they'll be elligible to be tried again on a future request -- without checking whether the worker is actually accessible. That's fine for some use cases, but explicitly what I want to avoid. Are there any thoughts to have an option to have recover_workers() do a ping prior to returning a working to a non-error state? And, yes, a watchdog thread in mod_proxy_balancer /and /a reasonable means of balancer invoking a ping via mod_proxy_ajp would be really helpful as far as mod_proxy_ajp is concerned. P.S. As far as my other questions go, I noticed that you covered the IIS/isapi base as well. I assume this new option is configured via the registry then. Are you planning to modify the vbs configuration script be extended to cover this?
Re: mod_proxy_balancer issue
Jess Holle wrote: Jess Holle wrote: Mladen Turk wrote: Jess Holle wrote: Mladen Turk wrote: Is there a means of achieving background-only (or nearly so) testing of dead workers with mod_jk? That's what I'm looking for in both jk and mod_proxy_ajp connectors. I guess I was hoping/assuming it was there in mod_jk from reading the docs. There is in the mod_jk (SVN trunk). I've been reading this code now... The watchdog thread looks very useful. If I understand it correctly, the watchdog thread can do whatever it feels like but currently mainly calls wc_maintain, which will only do work at most every worker.maintain seconds, right? connection_keepalive does not look like it really my bill, though. I'm most worried about workers in an error state and ensuring that they are rechecked every recover_wait_time -- but only by the watchdog thread and ideally via a ping/pong. Currently recover_workers appears to just put workers into a recovery state where they'll be elligible to be tried again on a future request -- without checking whether the worker is actually accessible. That's fine for some use cases, but explicitly what I want to avoid. Are there any thoughts to have an option to have recover_workers() do a ping prior to returning a working to a non-error state? And, yes, a watchdog thread in mod_proxy_balancer /and /a reasonable means of balancer invoking a ping via mod_proxy_ajp would be really helpful as far as mod_proxy_ajp is concerned. Another possibly simpler alternative: we could introduce a limit as to how many workers we attempt to do (unforced) recoveries on for any given request. Any request could likely tolerate a recovery attempt or two. None should have to tolerate 6-12 recovery attempts just because of a currently sparsely populated port range. Thoughts? -- Jess Holle
Re: mod_proxy_balancer issue
P.S. I'd also like to quiet attempts to recover workers from errors to a lower (and by default unlogged) logging level. The transition of a worker into an error state should certainly be logged, but logging every time we find it to still be in an error state seems to be excessive -- at least for a sparsely populated port bank use case. Jess Holle wrote: Jess Holle wrote: Jess Holle wrote: Mladen Turk wrote: Jess Holle wrote: Mladen Turk wrote: Is there a means of achieving background-only (or nearly so) testing of dead workers with mod_jk? That's what I'm looking for in both jk and mod_proxy_ajp connectors. I guess I was hoping/assuming it was there in mod_jk from reading the docs. There is in the mod_jk (SVN trunk). I've been reading this code now... The watchdog thread looks very useful. If I understand it correctly, the watchdog thread can do whatever it feels like but currently mainly calls wc_maintain, which will only do work at most every worker.maintain seconds, right? connection_keepalive does not look like it really my bill, though. I'm most worried about workers in an error state and ensuring that they are rechecked every recover_wait_time -- but only by the watchdog thread and ideally via a ping/pong. Currently recover_workers appears to just put workers into a recovery state where they'll be elligible to be tried again on a future request -- without checking whether the worker is actually accessible. That's fine for some use cases, but explicitly what I want to avoid. Are there any thoughts to have an option to have recover_workers() do a ping prior to returning a working to a non-error state? And, yes, a watchdog thread in mod_proxy_balancer /and /a reasonable means of balancer invoking a ping via mod_proxy_ajp would be really helpful as far as mod_proxy_ajp is concerned. Another possibly simpler alternative: we could introduce a limit as to how many workers we attempt to do (unforced) recoveries on for any given request. Any request could likely tolerate a recovery attempt or two. None should have to tolerate 6-12 recovery attempts just because of a currently sparsely populated port range. Thoughts? -- Jess Holle
Re: mod_proxy_balancer issue
Mladen Turk wrote: Jess Holle wrote: I want to use mod_proxy_balancer to load balance over a set of ports that potentially have Tomcats running on them. Unfortunately there will generally be a good number of ports where no Tomcat is running. Every 'retry' seconds I have a request that takes about an extra second for each Tomcat-less port, which is not acceptable. Has there been any thought or effort to provide an option to retry dead/unresponsive members in a periodic background thread instead of with normal, live requests? I am currently very interested in adding such an option if none exists, so I'd appreciate any thoughts on the matter. See the August thread on httpd-dev (Mpm maintenance thread hook) where I proposed exactly that. Among other things it will be potentially used with mod_proxy for out-of-the-request maintenance (Status checks, connection recycle, etc) No major objections, so I'm actively working on that. Also mod_jk in trunk has the watchdog thread (Apache 2+ only via JkWatchdogInterval) that allows to maintain the connection pool, and together with worker.xxx.connection_keepalive it can send CPING/CPONG packets on regular intervals thus checking the connection validity unbound from request frequency. My plan is to add this logic to mod_proxy, but using mpm hooks. I am quite interested in any progress you make here. Do you have any idea when you might have something testable in the mod_proxy area? I'm quite willing to help, but you're clearly much more familiar with both mod_jk and APR/MPM than I am, so I suspect I'd just get in the way except at the testing level. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_proxy_balancer issue
Mladen Turk wrote: Jess Holle wrote: I'm quite willing to help, but you're clearly much more familiar with both mod_jk and APR/MPM than I am, so I suspect I'd just get in the way except at the testing level. Well, I plan to create mpm watchdog hook system first. I'll also create a small callback in mod_proxy, just to maintain the connection pool so you are free to implement any additional mod_proxy interval operations (like heath checking) from there. Ideally I'd like to attempt to re-establish connections to dead servers occasionally there instead of during normal request time. Any pointers and foundation code (as described above) would be greatly appreciated. Thanks. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
mod_proxy_balancer issue
I want to use mod_proxy_balancer to load balance over a set of ports that potentially have Tomcats running on them. Unfortunately there will generally be a good number of ports where no Tomcat is running. Every 'retry' seconds I have a request that takes about an extra second for each Tomcat-less port, which is not acceptable. Has there been any thought or effort to provide an option to retry dead/unresponsive members in a periodic background thread instead of with normal, live requests? I am currently very interested in adding such an option if none exists, so I'd appreciate any thoughts on the matter. I also need to support the mod_jk-based IIS/Tomcat connector in the same scenario, but have not yet investigated how this will work out. Thoughts would be appreciated here as well. If mod_jk would work better than mod_proxy_ajp this presents me with a couple of issues: 1. The ability that mod_proxy_ajp has to queue/backlog requests when all workers are busy rather than returning an immediate 503 is highly desirable here. 2. As an integrated part of Apache, mod_proxy_ajp is in many respects the future here and I'd rather improve mod_proxy_ajp and/or mod_proxy_balancer than deal with mod_jk across all the platforms I have to worry about (various UNIXes and Windows). Also, yes, I know that I could enable status workers and have each Tomcat notify Apache that it is alive and present. That's interesting, but adds end-administrator complexity in that Tomcat must be configured with an appropriate URL for Apache, access to the status worker must be protected via further configuration based on the hosts of the target Tomcats, Tomcat must know if it is being served via mod_proxy_ajp or the IIS/Tomcat connector, etc. Further if Apache must be taken down it will lose track of which members are alive. If other approaches truly seem untenable I can consider this further, but it currently does not seem as attractive. Finally, if this should be addressed to the Apache dev list instead, please let me know. I'm avoiding the temptation to cross-post here :-) -- Jess Holle
Re: 6.0.18 tag
What was wrong with 6.0.17? Or was it essentially a dry run at 6.0.18? Version numbers are cheap, of course, just trying to understand where things are at currently (e.g. whether this is just 6.0.17 plus one or two critical bits). Remy Maucherat wrote: Hi, The plan is to tag it later today. Comments ? Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JConsole and tomcat.
I guess I'm doing something funny then because I see a *lot* of Tomcat MBeans in jconsole. I added -Dcom.sun.management.jmxremote, of course -- to cover Java 5 and left it in place for Java 6, but otherwise I don't think I did anything funny until within my web app. Costin Manolache wrote: This is a proposal for a very small change in tomcat trunk, to make tomcat easier to use with JConsole. Right now if you start tomcat 'out of box' and than try to inspect it with jconsole, you'll only see the 'platform' mbeans (memory etc). That's because tomcat doesn't use the platform mbean server. The fix is simple - in main or anywhere before the first MBeanServerFactory.findMBeanServer(null) we need to add a ManagementFactory.getPlatformMBeanServer(). This way modeler and tomcat will find the platform mbean server as the first server and use it. A better fix would be to replace all usages of findMBeanServer with getPlatformMBeanServer. The options are: - add a flag/system property to enable this behavior - make it default, add a flag to disable and use separate server - just use getPlatformMBeanServer with no flag. The javadoc is: It is recommended that this platform MBeanServer also be used to register other application managed beans besides the platform MXBeans. This will allow all MBeans to be published through the same MBeanServer and hence allow for easier network publishing and discovery. I can't think of any good reason to use a separate MBean server - and using jconsole is quite nice, all other methods of remote JMX documented on our site are more complex ( but good for specific cases ). The extra benefit is that we can see both memory/logging/other platform mbeans in the same place with tomcat beans. Options ? Can I just add the one liner before the first invocation of MBeanServerFactory ? Costin - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Fix #43236
Leech, Jonathan wrote: I read the spec the same way as you do. I have written filters that are even more permissive than this fix, and I have also written filters that strictly adhere to the spec. Strictly adhering to the spec is definitely the way to go. If you want/need your application to violate the spec, you can write a filter that allows it. I wasn't aware that you can call getOutputStream() in a JSP, when did that change? I'm not 100% certain this works. I am rather certain you can have a JSP forward to something else that calls getOutputStream() -- effectively on the same response. I suspect the same sort of thing would work in the page itself. Of course you have to be sure not to fill (and to reset) your JspWriter's buffer in any case. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Fix #43236
P.S. Thanks for the reply. No one had responded yet and I despite much googling I really couldn't seem to find any discussion of the nuances of this area of the servlet spec. Jess Holle wrote: Leech, Jonathan wrote: I read the spec the same way as you do. I have written filters that are even more permissive than this fix, and I have also written filters that strictly adhere to the spec. Strictly adhering to the spec is definitely the way to go. If you want/need your application to violate the spec, you can write a filter that allows it. I wasn't aware that you can call getOutputStream() in a JSP, when did that change? I'm not 100% certain this works. I am rather certain you can have a JSP forward to something else that calls getOutputStream() -- effectively on the same response. I suspect the same sort of thing would work in the page itself. Of course you have to be sure not to fill (and to reset) your JspWriter's buffer in any case. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Fix #43236
I belatedly noticed this bug fix in 5.5.26 and 6.0.16. On the one hand this seems like a nice fix, allowing the following call sequences on ServletResponse: 1. getWriter(), reset(), and then getOutputStream() 2. getOutputStream(), reset(), and then getWriter() 3. getWriter(), reset(), setCharacterEncoding(), and then getWriter() without exceptions and with the effects one would expect if one had not reset() or the methods prior to it. On the other hand, the servlet spec seems to state fairly unequivocally that (1) and (2) should result in IllegalStateExceptions and in (3) the setCharacterEncoding() call should be silently ignored. I'm not arguing for the fix to be rolled back, but I am trying to get some insight as to whether I'm the only one that reads the spec this way. I'm interested this in large part as an occasional (and current) author of rather complex servlet filters wherein I want to be as general and transparent as possible. In this role I want my filter to emulate /standard /ServletResponse behavior as well as possible irrespective of the servlet engine in which I'm running. Behavior like Tomcat's after this bug fix makes me wonder which way to head. For now I've been leaving my filters as per my understanding of the spec -- in large part because trying to emulate Tomcat's new behavior makes at least my filter a real mess. Also in practice I don't see a big need for the new behavior as JSPWriter's usually delay ServletResponse.getWriter() long enough for one to make up one's mind and call getOutputStream() instead, for instance. -- Jess Holle
Embedding question
I'd like to embed Tomcat in an application for which there will be numerous instances running. As such I have 2 basic problems: 1. *catalina.base:* I really want to share /all/ configuration files, web-app context XMLs, etc -- using separate directories only for temp, work, and logs. In an ideal world precompiled JSPs would be shared from one work directory (e.g. CATALINA_HOME/work) to the various instances and the instances would only write their own compilations, sessions, etc, to their own separate directories. Is this possible? Can anyone give me pointers in the sources? 2. *port negotiation:* I need to find an available port -- both for shutdown and for the AJP connector. I tried using the JMX MBean factory, etc, but got no signal when the port was already in use. Pointers to APIs to use that /will/ provide such signals (exceptions, return values, whatever) would be appreciated. Overall I'd like to keep this close to an out-of-the-box usage of 6.0.16, except with port negotiation handled outside server.xml, of course -- and writable directories outside CATALINA_HOME. Overall pointers (documentation?) would be appreciated. -- Jess Holle P.S. I've gotten this working by producing catalina.base directories containing way more of catalina.home's contents than I can see any reason for and pre-computing ports to use into each catalina.base, but this seems wrong overall. I'm looking for something better.
Re: Embedding question
I could see pre-negotiating ports into catalina.base's but I would like to minimize the weight of the catalina.bases in such a case. Jess Holle wrote: I'd like to embed Tomcat in an application for which there will be numerous instances running. As such I have 2 basic problems: 1. *catalina.base:* I really want to share /all/ configuration files, web-app context XMLs, etc -- using separate directories only for temp, work, and logs. In an ideal world precompiled JSPs would be shared from one work directory (e.g. CATALINA_HOME/work) to the various instances and the instances would only write their own compilations, sessions, etc, to their own separate directories. Is this possible? Can anyone give me pointers in the sources? 2. *port negotiation:* I need to find an available port -- both for shutdown and for the AJP connector. I tried using the JMX MBean factory, etc, but got no signal when the port was already in use. Pointers to APIs to use that /will/ provide such signals (exceptions, return values, whatever) would be appreciated. Overall I'd like to keep this close to an out-of-the-box usage of 6.0.16, except with port negotiation handled outside server.xml, of course -- and writable directories outside CATALINA_HOME. Overall pointers (documentation?) would be appreciated. -- Jess Holle P.S. I've gotten this working by producing catalina.base directories containing way more of catalina.home's contents than I can see any reason for and pre-computing ports to use into each catalina.base, but this seems wrong overall. I'm looking for something better.
Re: Tomcat 6.0.16 Startup Listener
Mark Thomas wrote: Jess Holle wrote: Stusynski, Dan wrote: Tomcat devs, Noticed a curiousity during 6.0.16 startup after migrating from 6.0.14. Below is the startup logs for Tomcat 6.0.16 and am curious if this has been seen before (the duplicate Listeners)? This was not encountered in 6.0.14. Not only do we get these duplicate Listener messages for anything defined in web.xml but we also get it for anything we might define additionaly for our softwares Listeners. In other words, we also get such duplicate listener messages any/all listener's in our web app's web.xml as well. [Our web.xml is a perfectly valid 2.4 web.xml that worked fine with 6.0.14.] These info messages were certainly not present in 6.0.14 and even if they're bogus raise concerns and questions in 6.0.16. They are a result of the fix for http://issues.apache.org/bugzilla/show_bug.cgi?id=44268 It means that for some reason, Tomcat is trying to register the listener more than once and subsequent attempts are ignored. This has been the case for all of the 6.0.x series but now it is logged when it happens. Looking back at the original issue http://issues.apache.org/bugzilla/show_bug.cgi?id=3770 it is probably occurring because the TLD is being processed twice. If this is generating a lot of noise then we probably need to look at an alternative solution but one that addresses the noise and 44268 doesn't immediately jump out at me. I'm unclear how listener elements in our web apps' web.xml are being processed multiple times. I only declare a single listener element for each listener class -- nothing odd. Processing them multiple times would seem to be a bug. If Tomcat realizes its mistake and does not actually do this twice, then from an external perspective there is no bug (apart from a slight performance one possibly) -- but telling us noisily about it makes it an externally noticeable bug and erodes confidence in the overall server. Given that our web app didn't mention anything about listeners.ContextListener or listeners.SessionListener I have to assume that this also is a bug -- in that our app never asked to define these at all, much less twice. I'd like to know (1) why this stuff is being attempted twice, (2) whether it can be avoided from our end somehow, (3) whether there are any side-effects [besides a minor delay in startup], (4) whether a near-term fix is likely, and, lastly, (5) how we can shut up just this message if there's no better fix in the short term offing. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 6.0.16 Startup Listener
Stusynski, Dan wrote: Tomcat devs, Noticed a curiousity during 6.0.16 startup after migrating from 6.0.14. Below is the startup logs for Tomcat 6.0.16 and am curious if this has been seen before (the duplicate Listeners)? This was not encountered in 6.0.14. Not only do we get these duplicate Listener messages for anything defined in web.xml but we also get it for anything we might define additionaly for our softwares Listeners. In other words, we also get such duplicate listener messages any/all listener's in our web app's web.xml as well. [Our web.xml is a perfectly valid 2.4 web.xml that worked fine with 6.0.14.] These info messages were certainly not present in 6.0.14 and even if they're bogus raise concerns and questions in 6.0.16. -- Jess Holle Feb 11, 2008 3:01:40 PM org.apache.catalina.core.AprLifecycleListener init INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: C:\Java\jdk1.5.0_11\bin;.;C:\WINNT\system32;C:\WINNT;C:\Java\jdk1.5.0_11 \jre\bin;C:\Java\jdk1.5.0_11\bin;C:\Java\jdk1.5.0_11\bin;C:\Java\jdk1.5. 0_11\jre\bin;C:\WINNT\system32;C:\WINNT;C:\WINNT\System32\Wbem;C:\Progra m Files\Rational\common;T:\sde\bin;T:\gnu\bin;C:\Perl\bin;O:\Oracle102\Ora 102\bin;C:\Program Files\Rational\ClearCase\bin;C:\NASM\nasm-0.98.39;C:\cygwin\bin;O:\Oracl e10g\ora10\bin Feb 11, 2008 3:01:40 PM org.apache.catalina.startup.Catalina load INFO: Initialization processed in 1210 ms Feb 11, 2008 3:01:40 PM org.apache.catalina.core.StandardService start INFO: Starting service Catalina Feb 11, 2008 3:01:40 PM org.apache.catalina.core.StandardEngine start INFO: Starting Servlet Engine: Apache Tomcat/6.0.16 Feb 11, 2008 3:01:41 PM org.apache.catalina.core.StandardContext addApplicationListener INFO: The listener listeners.ContextListener is already configured for this context. The duplicate definition has been ignored. Feb 11, 2008 3:01:41 PM org.apache.catalina.core.StandardContext addApplicationListener INFO: The listener listeners.SessionListener is already configured for this context. The duplicate definition has been ignored. Feb 11, 2008 3:01:42 PM org.apache.catalina.core.StandardContext addApplicationListener INFO: The listener listeners.ContextListener is already configured for this context. The duplicate definition has been ignored. Feb 11, 2008 3:01:42 PM org.apache.catalina.core.StandardContext addApplicationListener INFO: The listener listeners.SessionListener is already configured for this context. The duplicate definition has been ignored. Feb 11, 2008 3:01:44 PM org.apache.jk.common.ChannelSocket init INFO: JK: ajp13 listening on /0.0.0.0:8010 Feb 11, 2008 3:01:44 PM org.apache.jk.server.JkMain start INFO: Jk running ID=0 time=0/218 config=null Feb 11, 2008 3:01:44 PM org.apache.catalina.startup.Catalina start INFO: Server startup in 3985 ms Feb 11, 2008 3:01:59 PM org.apache.catalina.core.StandardService stop INFO: Stopping service Catalina Dan Stusynski - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 6.0.16 Startup Listener
That sounds about right. A patch seems obvious, but I'd (1) like to see this in the source tree for 6.0.17 and so on and (2) like to snag this same version of the source to patch into our 6.0.16. -- Jess Holle Tim Funk wrote: Unless I am misreading something - this looks to be an accident in web.xml processing. See lines 198 - 209 in trunk for org.apache.catalina.startup.WebRuleSet and you'll see this twice: digester.addCallMethod(prefix + web-app/listener/listener-class, addApplicationListener, 0); In a quick test - any listener in web.xml is called out as a dup if declared in web.xml So we went from not reporting to spuriously reporting due to a very old bug which wasn't a bug because dups were ignored but now is a bug since we call out that you tried to register a dup. (But you didn't register a dup, we just called addListener twice) doh. -Tim - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Cookies are broken in 6.0.16?
Or log a really noisy, loud error so you know what's going on at least Jim Manico wrote: we fixed the cookie behavior in this release due to security issues filed against the old parsing. Gotchya, Filip. Makes sense. What about the Runtime exception? That might at least allow legacy systems to debug this problem fast. Fail Quietly doesn't seem like a good solution. - Jim Jim Manico wrote: I guess we could throw a run time exception if the value contained any of those. other than that, I'm not sure how to behave I think this is the best case scenario for v0 cookies. Perhaps, if you really want to get fancy, you can add a flag to let legacy solutions roll back to the old/non-standard cookie handling methodology? no, we wont do that. we fixed the cookie behavior in this release due to security issues filed against the old parsing. Filip - Jim Remy Maucherat wrote: On Sat, 2008-02-09 at 16:14 -0700, Filip Hanik - Dev Lists wrote: no regression, if you do this c = new javax.servlet.http.Cookie(abcv1,123==); c.setVersion(1); response.addCookie(c); then it works just fine. however, if you do c = new javax.servlet.http.Cookie(abcv0,123==); response.addCookie(c); then it doesn't. if we encode it, (which we did at our first attempt for v0 cookies) we actually don't pass the TCK. only v1 cookies should be double quoted, in previous versions of tomcat, I believe everything got double quoted, regardless of version on the cookie. v0 cookies, the spec says /NAME/=/VALUE/ This string is a sequence of characters excluding semi-colon, comma and white space. If there is a need to place such data in the name or value, some encoding method such as URL style %XX encoding is recommended, though no encoding is defined or required. the problem was that encoding wasn't defined nor required. so when we followed the spec, and added %XX encoding, TCK tests failed. at this point I would say, we handle cookies correctly. if one needs == at the end of the cookie, then they need to use v1 cookies, according to spec I find the regressions caused by the new behavior problematic, and it will cause lots of problems with existing applications, since the default cookie version used is version 0. As I'm the only one complaining at the moment, I think I'll take my concerns elsewhere, no problem, I get the idea :) Obviously, when I say encoding, I am not talking about quoting the whole value (or name) as was done before. as always, I'm open to suggestions. it'd be easier if you suggested something. here is the javadoc for the servlet spec setValue public void *setValue*(String http://java.sun.com/j2se/1.5/docs/api/java/lang/String.html newValue) Assigns a new value to a cookie after the cookie is created. If you use a binary value, you may want to use BASE64 encoding. With Version 0 cookies, values should not contain white space, brackets, parentheses, equals signs, commas, double quotes, slashes, question marks, at signs, colons, and semicolons. Empty values may not behave the same way on all browsers. *Parameters:* |newValue| - a |String| specifying the new value *See Also:* |getValue()| http://java.sun.com/javaee/5/docs/api/javax/servlet/http/Cookie.html#getValue%28%29, |Cookie| http://java.sun.com/javaee/5/docs/api/javax/servlet/http/Cookie.html I guess we could throw a run time exception if the value contained any of those. other than that, I'm not sure how to behave Filip Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 6.x feature wishlist
Tim Funk wrote: 11.Timestamps System.currentTimeMillis System.currentTimeMillis is invoked everywhere during the chain of events for a HTTP requests, even though most dates only need precision down to the second. I've received feedback that this could be improved by keeping a time service, that updates a timestamp every second, and therefor reduces the number of system calls I think we would need to prove the theory before committing to the implementation, but that should be pretty easy System.currentTimeMillis() is *very* fast. [System.nanoTime() is not necessarily and can be relatively slow on some hardware.] Many algorithms used to format System.currentTimeMillis() into a human readable date format are fairly slow. I'm not sure what Tomcat is currently using here, but that would be the only thing I'd be concerned about. It would seem sufficient just to make sure this conversion is efficient and/or not called with any ridiculous frequency. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: 6.x feature wishlist
Filip Hanik - Dev Lists wrote: Jess Holle wrote: Tim Funk wrote: 11.Timestamps System.currentTimeMillis System.currentTimeMillis is invoked everywhere during the chain of events for a HTTP requests, even though most dates only need precision down to the second. I've received feedback that this could be improved by keeping a time service, that updates a timestamp every second, and therefor reduces the number of system calls I think we would need to prove the theory before committing to the implementation, but that should be pretty easy System.currentTimeMillis() is *very* fast. [System.nanoTime() is not necessarily and can be relatively slow on some hardware.] Many algorithms used to format System.currentTimeMillis() into a human readable date format are fairly slow. I'm not sure what Tomcat is currently using here, but that would be the only thing I'd be concerned about. It would seem sufficient just to make sure this conversion is efficient and/or not called with any ridiculous frequency. good info, all we use is the millis version, then we don't need to worry about it Yes, if you just call System.currentTimeMillis() and pass along the long this is all but free (essentially it just reads a long out of volatile memory address). If you're repeatedly formatting this into date strings, then some profiling is likely in order. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Serious non-native AJP connector issue
Bill Barker wrote: It seems that we lost building the NIO/AJP connector in Tomcat, but since i recognize you from [EMAIL PROTECTED], you will have seen that already :). Last time I tested, the experimental NIO/AJP connector performed better than either the default or the APR connector on Solaris (surprise, surprise), and came in dead last on Windows. Thanks for all the helpful information, Bill. I wonder if the results of for the NIO/AJP connector would be any different on Windows with a later JVM (e.g. Java 6 Update 1). I would *hope* that Sun would fix the NIO issues on Windows to the point that it would out-perform Java-Blocking-IO/AJP. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Serious non-native AJP connector issue
H I just redid my tests with: BalancerMember ajp://localhost:8010 min=15 max=*50* smax=30 ttl=900 keepalive=Off timeout=900 and Connector port=8010 minSpareThreads=15 maxSpareThreads=30 maxThreads=*51* acceptCount=*0* tomcatAuthentication=false useBodyEncodingForURI=true URIEncoding=UTF-8 enableLookups=false redirectPort=8443 protocol=AJP/1.3 / and ab -n 100 -c *50* http://jessh03l.ptcnet.ptc.com/TestApp/test.jsp?secs=3 I (after about 3 seconds) get SEVERE: All threads (51) are currently busy, waiting. Increase maxThreads (51) or check the servlet status and I eventually get exactly 1 lost request. I'm baffled as to why this can occur. Something's still doesn't seem quite right here. What's even weirder is that I only get this issue with the first ab run after restarting Tomcat. If I do the same test again (any number of times) I don't lose any requests. I can get the same result by restarting and doing 2 ab runs with -n 100 in fairly short secession, so this isn't some ab error. By fairly short, I don't mean very short -- I left a good 5 seconds between runs. I find that using a max of 49 in Apache seems to work reliably, but I'm struggling to understand: (1) why I have to have 2 more Tomcat connector threads allowed than the number of connections I can possibly have and (2) whether 2 is always the safe buffer or whether I'll need 9 on an 8 CPU box or some such. -- Jess Holle
Re: Serious non-native AJP connector issue
Rainer Jung wrote: Jess, I didn't really carefully think about the case of a saturated pool. But nevertheless some hints: 1) There is always one thread waiting for the accept, which is a usual pool thread. So an offset of one between threeads processing requests and the pool size is normal. Got that, but I've accounted for that via maxThreads of 51 and BalanceMember max of 50. I'm left wondering why there's an off-by-one beyond this (and thus an off-by-2 overall). 2) There is no orderly shutdown in AJP13 for a connection, neither from the httpd side, not from the Tomcat side. mod_jk will realize closed connections and reopen them, but I have the impression, that mod_proxy fails a backend when it gets a connection error. Tomcat in my experience reliably closes a connection and frees the thread, when the web server does. Most of the time it makes sense to have a connectionTimeout (Milliseconds) and a connection_pool_timeout (seconds, mod_jk) resp. ttl (seconds, mod_proxy) which are in sync. But: they will never harmonize completely, because mod_jk only checks for the need to throw away closed connections during maintenance (every 60 seconds or whatever is configured with worker.maintain) and I think mod_proxy checks the ttl whenever a connection is put back into the pool. I don't think any of those should be involved in this short test. 3) Does it happen with mod_jk too? I believe so. I did some mod_jk testing to verify the overall nature of the issue remained the same, but I didn't go through all the same detailed tests. I could redo this particular test if that's helpful. 4) Weird guesses: - max is limited with mod_proxy to the number of threads per process configured in your MPM (worker?). This is 25 by default. So if we want to have an easy to understand scenario, set your threads per process to say 60 and min=max=50 as well as removing the smax and the ttl.. That way 50 connections should be started on startup (per httpd process; check with netstat), and we shouldn't see any resizing during your ab test. Now start your ab test before Tomcat will timeout (I remember that connectionTimeout had some default value, even if you don't set it, but it is in the minutes). My Apache MaxClients is 300 and this is on Windows so I only have 1 worker process. If you don't run into trouble then, we know your observation has to do with resizing of connection pools. *Maybe*: ab is too fast and can come back with new requests faster, than the connections go back into the pool after the end of a request in httpd. Not very reasonable, but possible. Of cource the pool is synchronized and the lock doesn't need to behave fair, i.e. when it gets contended, it's not clear if the oldest waiting thread gets it first. I believe I disproved this at some point by running 2 tests with -n 50 and -c 50 manually back to back, but I'd have to re-test to be sure. [I'm wishing I'd taken better notes of various results...] Apart from this weird edge condition (an off-by-2 isn't that devastating if it stays 2 in all cases), the thing that gets me overall is that the documentation really makes it sound like acceptCount works like a fair queue and that there's no harm in exceeding maxThreads except that requests will be queued. As Bill suggested, I should come up with suggested patches to the documentation -- I'm just not yet confident enough in my understanding to propose such patches. At this point all I can propose is strong warning verbage around maxThreads and acceptCount regarding their behavior for the Java AJP connector. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Serious non-native AJP connector issue
We're facing a /serious /issue with the non-native AJP connector. To put it most simply, some requests seem to get lost in Tomcat in various cases involving concurrent requests -- and not egregious numbers of concurrent requests, either. For instance, 1. Use a Tomcat 5.5.23 with a configuration like: Connector port=8010 minSpareThreads=4 maxSpareThreads=12 maxThreads=18 acceptCount=282 tomcatAuthentication=false useBodyEncodingForURI=true URIEncoding=UTF-8 enableLookups=false redirectPort=8443 protocol=AJP/1.3 / (which are intended solely for making it easier to test concurrency issues that to overrun a realistic 'maxThreads' setting) and as a control, similar thread pool settings on the direct HTTP connector: Connector port=8080 maxHttpHeaderSize=8192 minSpareThreads=4 maxSpareThreads=12 maxThreads=18 acceptCount=282 enableLookups=false redirectPort=8443 connectionTimeout=2 disableUploadTimeout=true / 2. Use an Apache 2.2.4 with mod_proxy_ajp with a configuration like: Proxy balancer://ajpWorker BalancerMember ajp://localhost:8010 min=16 max=300 smax=40 ttl=900 keepalive=Off timeout=900 /Proxy RewriteEngine on RewriteRule ^(/TestApp/(.*\.jsp(.*)|servlet/.*|.*\.jar))$ balancer://ajpWorker$1 [P] (on Windows in this case; similar results can be obtained on Linux at least) 3. Use a simple test JSP page (placed in a web app containing nothing else): [EMAIL PROTECTED] session=false %[EMAIL PROTECTED] contentType=text/html pageEncoding=UTF-8 %%! private static final String titleString = Sleepy Test JSP Page; %html head % String sleepSeconds = request.getParameter( secs ); if ( sleepSeconds == null ) sleepSeconds = 1; long secsToSleep = Long.parseLong( sleepSeconds ); Thread.sleep( 1000L * secsToSleep ); % meta http-equiv=Content-Type content=text/html; charset=UTF-8/ title%=titleString%/title /head body center h2%=titleString%: SUCCESS!/h2 [Slept %= secsToSleep % seconds.] /center /body /html 4. Hit the page with ab * First, test direct Tomcat connections: o ab -n 24 -c 24 -A wcadmin:wcadmin http://hostname:*8080*/TestApp/test.jsp?secs=3 + Result: All requests complete sucessfully in 6118 ms. * Second, test connections via Apache: o ab -n 24 -c 24 -A wcadmin:wcadmin http://hostname/TestApp/test.jsp?secs=3 + Result: Only 17 requests complete before ab times out. * Third, test connections via Apache again soon (under the BalancerMember 'timeout' seconds) after the last test o ab -n 24 -c 24 -A wcadmin:wcadmin http://hostname/TestApp/test.jsp?secs=3 + Result: Only 9 requests complete before ab times out. Something is clearly /horribly/ wrong with the handling of any concurrency over 'maxThreads' in this case. Even so, there seems to be some sort of off-by-one error in that only 17 requests complete, not 18. Worse, this has a lingering effect that decreases Tomcat's ability to concurrent requests thereafter (with this impact seemingly being much worse the longer the BalancerMember timeout is set to -- and we have some very long running requests and thus need this timeout to be /very/ large). This is not the only ill effect we've seen when hitting Tomcat 5.5.24 with concurrent requests in this manner, but it is a good place to start. As for the native connector, it just works here. So why don't I just use it? Well, first off, we have to support Tomcat on Windows (32 and 64-bit), Linux, Solaris, HPUX (PA-RISC and Itanium), and AIX. We've been unable to get the connector built on some of these platforms and on some we can't get the resulting binary to function (specifically on AIX). Further, we had some stability issues with the native connector in the past and had considered the Java connector the safest, if not fastest, option -- and to a degree given that everything is Java I still feel that's the case. Finally, however, this connector should just plain work. Tomcat shouldn't be a cripple unless/until you manage to build a native connector for your platform. Any troubleshooting and/or debugging ideas (e.g. where exactly to place breakpoints, what logs to turn on, etc, etc) would be /greatly/ appreciated. -- Jess Holle [EMAIL PROTECTED] P.S. If this should go to the user's mailing list instead that's fine, but this really seemed like a developer-oriented issue to me.
Re: Serious non-native AJP connector issue
There's no intent to handle this case? Really? Tomcat should throw an IllegalArgumentException whenever 'acceptCount' is set to anything other than 0 explaining this if this is the case! If so, this is very unfortunate. We use maxThreads as a throttle to limit the concurrency at this level under the (silly?) assumption that acceptCount behaves as documented in the documentation. [Yes, one could argue that a separate throttle should be used behind this layer, which we might have been inclined to do if the documentation said acceptCount does not work.] Further, I've been able to reproduce such issues using a concurrency of exactly the maxThreads. At *best* there is an off-by-one error here, but I've still seen problems in some testing using a concurrency level 2 or 3 less than maxThreads -- which leads me to believe there is something a bit more nefarious going on here. Another of my colleagues claims that he's encountered issues in simple ab tests quite a ways below maxThreads. I've not managed to reproduce this case myself, but I could certainly focus on this case if the Tomcat contributors have no intent to fix acceptCount behavior for the Java AJP connector. Is there a firm intent to handle this case in the _/native/_ AJP connector? If so, then this distinction should be clearly stated (and an IllegalArgumentException thrown as per above when the native connector cannot be found/initialized). Personally I see utility in having acceptCount working in the AJP case. If one exceeds the mod_proxy_ajp hard maximum the client immediately gets a 503. There are cases when just queuing a few requests makes more sense than either returning 503's or letting them rip -- which is where acceptCount comes in. [Granted the acceptCount numbers below are excessive, but that's another matter...] -- Jess Holle [EMAIL PROTECTED] Bill Barker wrote: Yes, since you know in advance how many connections you are going to get to the AJP connector, you can configure it so that it has enough threads to handle all of those connections. That is why it doesn't attempt to handle the case when the concurrency goes above maxThreads -Original Message- From: Jess Holle [mailto:[EMAIL PROTECTED] Sent: Thursday, June 14, 2007 2:58 PM To: Tomcat Developers List Cc: Dobson, Simon; Wang, Andy; Fenlason, Josh Subject: Serious non-native AJP connector issue We're facing a /serious /issue with the non-native AJP connector. To put it most simply, some requests seem to get lost in Tomcat in various cases involving concurrent requests -- and not egregious numbers of concurrent requests, either. For instance, 1. Use a Tomcat 5.5.23 with a configuration like: Connector port=8010 minSpareThreads=4 maxSpareThreads=12 maxThreads=18 acceptCount=282 tomcatAuthentication=false useBodyEncodingForURI=true URIEncoding=UTF-8 enableLookups=false redirectPort=8443 protocol=AJP/1.3 / (which are intended solely for making it easier to test concurrency issues that to overrun a realistic 'maxThreads' setting) and as a control, similar thread pool settings on the direct HTTP connector: Connector port=8080 maxHttpHeaderSize=8192 minSpareThreads=4 maxSpareThreads=12 maxThreads=18 acceptCount=282 enableLookups=false redirectPort=8443 connectionTimeout=2 disableUploadTimeout=true / 2. Use an Apache 2.2.4 with mod_proxy_ajp with a configuration like: Proxy balancer://ajpWorker BalancerMember ajp://localhost:8010 min=16 max=300 smax=40 ttl=900 keepalive=Off timeout=900 /Proxy RewriteEngine on RewriteRule ^(/TestApp/(.*\.jsp(.*)|servlet/.*|.*\.jar))$ balancer://ajpWorker$1 [P] (on Windows in this case; similar results can be obtained on Linux at least) 3. Use a simple test JSP page (placed in a web app containing nothing else): [EMAIL PROTECTED] session=false %[EMAIL PROTECTED] contentType=text/html pageEncoding=UTF-8 %%! private static final String titleString = Sleepy Test JSP Page; %html head % String sleepSeconds = request.getParameter( secs ); if ( sleepSeconds == null ) sleepSeconds = 1; long secsToSleep = Long.parseLong( sleepSeconds ); Thread.sleep( 1000L * secsToSleep ); % meta http-equiv=Content-Type content=text/html; charset=UTF-8/ title%=titleString%/title /head body center h2%=titleString%: SUCCESS!/h2 [Slept %= secsToSleep % seconds.] /center /body /html 4. Hit the page with ab * First, test direct Tomcat connections: o ab -n 24 -c 24 -A wcadmin:wcadmin http://hostname:*8080*/TestApp/test.jsp
Re: Serious non-native AJP connector issue
Okay that's all starting to make some sense, but it hard to see how someone would come to this understanding from the documentation. I also don't see how |connectionTimeout will help here in that the documentation says: | The number of milliseconds this *Connector* will wait, after accepting a connection, for the request URI line to be presented. The default value is infinite (i.e. no timeout). If a connection is formed between Apache and Tomcat for 24 requests and 17 are immediately processed due to a maxThreads limit of 18, how would setting a low connectionTimeout help the 17 threads process the other 7 requests? I'm trying to understand if there is any useful throttling configuration in which the Java AJP connector has a maxThreads less than Apache's max AJP connections -- plus 1. Put another way, I'm not quite understanding any useful acceptCount scenario from the connectionTimeout description above. I know the Java AJP connector uses a thread-per-connection model. I had assumed that the maxThreads meant maximum /active request processing /threads (e.g. like some old Tomcat releases used to call this maxProcessors) and that connections covered by acceptCount were still allocated threads and were still accepted (as the parameter name implies). I'd further assumed that a fair blocking queue arrangement would allow maxThreads connection threads to run and keep the others waiting until their turn. I don't see anything in the documentation to the contrary of this. Further I'm trying to understand any sort of arrangement with connectionTimeout that would give this level of utility to acceptCount. As it stands it seems like users can easily produce devastatingly bad behavior by making assumptions about acceptCount that seem quite logical from the documentation. Obviously the APR connector can do much better by /not/ allocating threads to connections beyond maxThreads. That's great, but juggling native builds for many different OS's can be a real issue (which is why I still hold out hope that the NIO connection will come through with something better than the non-NIO Java connector's performance even if it is not APR-level performance). I don't mean to be argumentative -- I'm just really struggling to understand and hopefully preventing future misunderstandings through more clarity in the documentation. -- Jess Holle Bill Barker wrote: Jess Holle [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] There's no intent to handle this case? Really? Tomcat should throw an IllegalArgumentException whenever 'acceptCount' is set to anything other than 0 explaining this if this is the case! If so, this is very unfortunate. We use maxThreads as a throttle to limit the concurrency at this level under the (silly?) assumption that acceptCount behaves as documented in the documentation. [Yes, one could argue that a separate throttle should be used behind this layer, which we might have been inclined to do if the documentation said acceptCount does not work.] You are misunderstanding how AJP works. Since you don't have a connectionTimeout on the Connector /, the connections to httpd stay alive waiting to get another request on the same socket. As a result, there won't be any free Threads to handle a new connection so it doesn't matter what acceptCount is. Their isn't anyone there to accept them.
5.5.24?
Doing a quick scan, it looks like a good number of small fixes to 5.5.23 have accumulated over the last number of months. [See http://marc.info/?l=tomcat-devw=2r=1s=5.5.24q=b] I know the Tomcat developers are largely focused on 6.0 now, but is there any chance of a 5.5.24 being rolled to wrap up this backlog. There are a few spec violations in this batch (e.g. 41869) plus the embarrassing missing MailSessionFactory. Overall it would seem good to roll these all up one of these days -- especially given that the number of bugs fixes per month seems to be dropping off -- and thus one could argue that 5.5.24 may stand as is for a long time. There are a good number of open bugs against 5.5.x, but few serious ones seem to: (1) not be in a NEEDINFO state and (2) be in another area than the native connector (which is separately versioned and optional). Thoughts? -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Announcing releases of tomcat native
In general it is not very easy to even notice when new stable tcnative versions become available. For instance, discovering that 1.1.10 is out, what changed since 1.1.8, and verifying that it is intended to work with 5.5.23 are not very trivial from the web site currently... Rainer Jung wrote: Mladen Turk wrote: It is part of Tomcat release and that shouldn't change. Each tomcat release has a detection of the Tomcat native, and it prints out the suggested version compared with the one user is running. For 6.0.13 the required version is 1.1.8 and recommended is 1.1.10 and that is clearly printed whenever Tomcat is started. I'm not sure what more is needed. I think he's trying to provide packages for a distro. So he wants to provide 1.1.8, 1.1.9, 1.1.10, ... whenever they are released (available). At the moment, the only way to learn about the releases for him is looking into a tomcat download, unpacking the included tcnative source and checking, if it is new and he needs to provide a new package. This is not very robust and furthermore that way he only will recognize tcnative versions bundled with a Tomcat download. Regards, Rainer - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tagging 5.5.23
What became of 5.5.23? -- Jess Holle Filip Hanik - Dev Lists wrote: Due to a number of important bug fixes, we will skip the 5.5.22 release and go directly to a 5.5.23. I will tag SVN tomorrow at noon Mountain time. Let me know if this conflicts with anyone Filip - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] Release build 5.5.22
Filip Hanik - Dev Lists wrote: Ok folks, for those that want to see the 5.0.22, please provide some feedback. Without votes, I can't push out a release. [I'm sure you meant 5.5.22, not 5.0.22...] Do votes from non-commiters count in this regard, i.e. in a decisive manner as to whether a release is pushed out? If so, I can try to make some time to test this out. If not I've got a rather full near term plate -- but am still very interested in having a stable 5.5.x release in the somewhat near term. -- Jess Holle Filip Hanik - Dev Lists wrote: Candidate binaries are available here: http://people.apache.org/~fhanik/tomcat/tomcat-5.5/v5.5.22/ According to the (slightly) updated release process, the 5.5.22 tag is: [ ] Broken [ ] Alpha [ ] Beta [ ] Stable Filip - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Let's get 5.5.21 out the door...
William L. Thomson Jr. wrote: On Fri, 2007-02-02 at 17:09 +0100, Remy Maucherat wrote: Mladen Turk wrote: Anyhow, a thing like Remy is doing for 6.0 would be nice to have. It offers a build before official tag Tagged sources are best here :) Tagged build does nothing for us, since we build from source. So if the builds are a pain, maybe just tagging sources is an option. But IMHO, I am more interested in seeing a 6.0.x release than 5.5.21 :) 5.5.21, could be a eternal beta like 5.0.30 for those sticking to that version. FYI, we have 6.0.7 beta, and 6.0.8 alpha packages available on Gentoo ~arch, unstable. Many are already switching to that, and I encourage it. I have run 6.0.x on my development server since 6.0.2. I will likely use on my production systems once there is an official release, non beta/alpha. Just tossing in my 2 cents. For those of us needing the most stable Tomcat possible today and with downstream consumers who can't stand beta labels, 5.5.21 would really be helpful. I'm quite interested in 6.0.x, but in the first stable release... -- Jess Holle
5.5.21?
I would assume that by this time there is a sufficient collection of fixes to 5.5.20 that may well argue for a 5.5.21 release. I know the Tomcat development community has been largely focused on 6.0 releases, but as one who will have to freeze upon a 5.5.x release in the near future, I'm searching for some visibility into the likelihood of a 5.5.21 in the near future. One fix of interest that I am aware of in 5.5.21 is the AJP flush packets fix. We can certainly patch that into our Tomcat, but I'd rather be able to move to a 5.5.21 with any other collected fixes as well -- and avoid discrepancies between our Tomcat and standard releases except where necessary. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Serious mod_jk performance issue
Thanks for the efforts, Rainer. I'll have to look deeper -- perhaps there is some aspect of our configuration that is causing us the issues as they seem to be quite repeatable. -- Jess Holle Rainer Jung wrote: Hi Jess, I did some simple tests and was not able to reproduce your performance observations. Nevertheless I could observe a couple of strange things, but I doubt, if they are relevant to most use cases. First my setup: Apache 2.0.59 worker with mod_jk 1.2.20 and Tomcat 5.5.17 with normal (non-apr) connectors, using Java 1.5.0_06 on an early Release of Solaris 10. Hardware Sun T-2000 (Niagara), which means relatively slow CPU but good scalability. I didn't have the system exclusively, but it was rather idle during the test. Client ab from apache 2.0.59. All ab measurements have been verified with %D in the apache access log. No restarts between measurements, so the file was most likely coming from the file system cache. Client running either on the same machine, or on a SLES 9 SP2, 64Bit AMD Opteron connected by 100MBit Ethernet. Apache and mod_jk compiled with -mcpu=v9 -O2 -g -Wall. Apache, mod_jk and Tomcat configured default (apart from ports and log format), JVM for tomcat with a couple of non-default values: -server \ -Xms64m -Xmx64m \ -XX:NewSize=8m -XX:MaxNewSize=8m \ -XX:SurvivorRatio=6 -XX:MaxTenuringThreshold=31 \ -XX:+UseConcMarkSweepGC -XX:-UseAdaptiveSizePolicy File to test throughput had size 316702480 bytes (some .tar.gz I found lying around). 1) local client, i.e. client running on the same machine as Apache and Tomcat A single request took 15.71 sec (mod_jk) (=153.8 MBit/Sec) and 15.61 (TC HTTP direct) (=154.8 MBit/sec), the same with 10 consecutive - non-parallel - requests gave 157.1 sec resp. 156.8 sec, so this result seems to be stable. Now parallel requests: I used parallelity (-c with ab) of 2 4 8 16 32 and the double amount of requests (4, 8, ...): Throughput results in MBit/sec, depending on concurrency: mod_jk http conc. 1 153.8 154.8 2 306.3 303.6 4 605.5 627.7 8 1090.0 1185.5 161137.7 1161.8 321210.7 1114.3 mod_jk and HTTP direct behave almost the same for the huge file. We saturate the system at about 1100 MBit/second (going via loopback). CPU was busy at most 60% during these tests. This also shows, that mod_jk and HTTP throughput is enough to saturate a lot of bandwidth - as long as your IP stack doesn't add to much overhead to it. 2) remote client, i.e. ab running on the SLES 9 SP2 x86_64 machine, connected via 100MBit to Apache and Tomcat. Throughput results in MBit/sec, depending on concurrency: mod_jk http conc. 1 88.689.1 2 88.989.1 So even with only one request in parallel we saturate the network and it does not make sense to measure more than two parallel requests. 3) Dependancy on file size: Measuring with local client without concurrency for 50, 100, 200, 300, 400, 500, ..., 1000MB: mod_jk http MB 50 167.5 234.9 (5 consecutive requests) 100 168.8 170.1 (5 consecutive requests) 200 168.6 169.8 (2 consecutive requests) 300 169.1 169.7 (2 consecutive requests) 400 168.9 169.7 (2 consecutive requests) 500 168.8 169.4 (2 consecutive requests) 600 167.9 168.0 (2 consecutive requests) 700 167.8 168.9 (2 consecutive requests) 800 168.1 168.6 (2 consecutive requests) 900 168.0 168.0 (2 consecutive requests) 1000 156.2 214.9 (2 consecutive requests) 2000 156.9 214.7 (1 request) Interestingly the result for 1000M and for 2000M is reproducible. But as soon as I switch from the client ab to wget or curl (writing output to /dev/null) I get the same numbers for mod_jk, but for HTTP I get the same result as for mod_jk! The numbers are slightly better than in the first test, I guess because this test was done using a file in the webapps file system, the first test was done using a file in another file system symlinked from within webapps (but still a local fs). Another possibilty would be, that a mkfile generated file has a better block layout in the fs, than a usual file, which was growing over time. All in all I think that throughput for huge files is very good in both cases. I would expect, that most often it would be much more intersting to inspect scalability and system load (cpu/memory) for massive concurrency. When serving large files, downloads will run a long time because most often the client side of the connection is not a fat line. As a result users will add up in parallel, so one might need to serve a few thousands of users. Regards, Rainer Jess Holle schrieb: Mladen Turk wrote: Jess Holle wrote: We're seeing a *serious *performance issue with mod_jk and large (e.g. 500MB+) file transfers. [This is with Apache 2.0.55, Tomcat 5.0.30, and various recent mod_jk including 1.2.20.] SunOS dev12.qa.atl.jboss.com 5.9 Generic_118558-25 sun4u sparc SUNW,Sun-Fire-V210
Re: Serious mod_jk performance issue
Do your results differ when Windows XP is used as the client? I looked back at all our notes and though we tested with Solaris and Windows servers the client was always XP. Rainer Jung wrote: Hi Jess, I did some simple tests and was not able to reproduce your performance observations. Nevertheless I could observe a couple of strange things, but I doubt, if they are relevant to most use cases. First my setup: Apache 2.0.59 worker with mod_jk 1.2.20 and Tomcat 5.5.17 with normal (non-apr) connectors, using Java 1.5.0_06 on an early Release of Solaris 10. Hardware Sun T-2000 (Niagara), which means relatively slow CPU but good scalability. I didn't have the system exclusively, but it was rather idle during the test. Client ab from apache 2.0.59. All ab measurements have been verified with %D in the apache access log. No restarts between measurements, so the file was most likely coming from the file system cache. Client running either on the same machine, or on a SLES 9 SP2, 64Bit AMD Opteron connected by 100MBit Ethernet. Apache and mod_jk compiled with -mcpu=v9 -O2 -g -Wall. Apache, mod_jk and Tomcat configured default (apart from ports and log format), JVM for tomcat with a couple of non-default values: -server \ -Xms64m -Xmx64m \ -XX:NewSize=8m -XX:MaxNewSize=8m \ -XX:SurvivorRatio=6 -XX:MaxTenuringThreshold=31 \ -XX:+UseConcMarkSweepGC -XX:-UseAdaptiveSizePolicy File to test throughput had size 316702480 bytes (some .tar.gz I found lying around). 1) local client, i.e. client running on the same machine as Apache and Tomcat A single request took 15.71 sec (mod_jk) (=153.8 MBit/Sec) and 15.61 (TC HTTP direct) (=154.8 MBit/sec), the same with 10 consecutive - non-parallel - requests gave 157.1 sec resp. 156.8 sec, so this result seems to be stable. Now parallel requests: I used parallelity (-c with ab) of 2 4 8 16 32 and the double amount of requests (4, 8, ...): Throughput results in MBit/sec, depending on concurrency: mod_jk http conc. 1 153.8 154.8 2 306.3 303.6 4 605.5 627.7 8 1090.0 1185.5 161137.7 1161.8 321210.7 1114.3 mod_jk and HTTP direct behave almost the same for the huge file. We saturate the system at about 1100 MBit/second (going via loopback). CPU was busy at most 60% during these tests. This also shows, that mod_jk and HTTP throughput is enough to saturate a lot of bandwidth - as long as your IP stack doesn't add to much overhead to it. 2) remote client, i.e. ab running on the SLES 9 SP2 x86_64 machine, connected via 100MBit to Apache and Tomcat. Throughput results in MBit/sec, depending on concurrency: mod_jk http conc. 1 88.689.1 2 88.989.1 So even with only one request in parallel we saturate the network and it does not make sense to measure more than two parallel requests. 3) Dependancy on file size: Measuring with local client without concurrency for 50, 100, 200, 300, 400, 500, ..., 1000MB: mod_jk http MB 50 167.5 234.9 (5 consecutive requests) 100 168.8 170.1 (5 consecutive requests) 200 168.6 169.8 (2 consecutive requests) 300 169.1 169.7 (2 consecutive requests) 400 168.9 169.7 (2 consecutive requests) 500 168.8 169.4 (2 consecutive requests) 600 167.9 168.0 (2 consecutive requests) 700 167.8 168.9 (2 consecutive requests) 800 168.1 168.6 (2 consecutive requests) 900 168.0 168.0 (2 consecutive requests) 1000 156.2 214.9 (2 consecutive requests) 2000 156.9 214.7 (1 request) Interestingly the result for 1000M and for 2000M is reproducible. But as soon as I switch from the client ab to wget or curl (writing output to /dev/null) I get the same numbers for mod_jk, but for HTTP I get the same result as for mod_jk! The numbers are slightly better than in the first test, I guess because this test was done using a file in the webapps file system, the first test was done using a file in another file system symlinked from within webapps (but still a local fs). Another possibilty would be, that a mkfile generated file has a better block layout in the fs, than a usual file, which was growing over time. All in all I think that throughput for huge files is very good in both cases. I would expect, that most often it would be much more intersting to inspect scalability and system load (cpu/memory) for massive concurrency. When serving large files, downloads will run a long time because most often the client side of the connection is not a fat line. As a result users will add up in parallel, so one might need to serve a few thousands of users. Regards, Rainer Jess Holle schrieb: Mladen Turk wrote: Jess Holle wrote: We're seeing a *serious *performance issue with mod_jk and large (e.g. 500MB+) file transfers. [This is with Apache 2.0.55, Tomcat 5.0.30, and various recent mod_jk including 1.2.20.] SunOS dev12.qa.atl.jboss.com 5.9 Generic_118558-25 sun4u sparc SUNW,Sun-Fire-V210 Tomcat:8080 Total
Re: Serious mod_jk performance issue
Apache and tomcat are both on the same Solaris 10 box and the network between client (XP) is 100Mbit. -- Jess Holle Rainer Jung wrote: If noone finds a reason for it, I can go into it during the weekend. I would try to reproduce and research on Solaris. Concerning your data for Solaris: Apache and Tomcat were both on Solaris? The same machine or different? Network between Client (Browser?) and Apache was 100MBit or 1GBit? Regards, Rainer Jess Holle schrieb: We're seeing a *serious *performance issue with mod_jk and large (e.g. 500MB+) file transfers. [This is with Apache 2.0.55, Tomcat 5.0.30, and various recent mod_jk including 1.2.20.] The performance of downloading the file via Apache is good, as is the performance when downloading directly from Tomcat. The performance when downloading from Tomcat through Apache via mod_jk is, however, quite abysmal. I'd obviously expect *some* degradation due to the extra interprocess hop, but given that this is a just a single-user, single-request test, I'd expect that the network would still be the limiting factor -- or at least that the degradation would be in the order of 25% of less. What we're seeing, however, is far worse: On Windows: * Apache 2.0.55, Tomcat 5.0.30, and mod_jk 1.2.20 - Started at 10 MB/sec ended at 3 MB/sec with mod_deflate disabled (1.5 MB/sec with mod_deflate enabled) * Apache 2.0.55, Tomcat 5.0.30, and mod_jk 1.2.19 - Disabling JkFlushPackets only slightly improved performance. * Apache 2.2.3 with Tomcat 5.5.20 w/ the native connector - Didn't work period. I didn't have a chance to look into it, but the download failed after getting serveral packets (!) * Apache 2.2.3 with Tomcat 5.5.20 w/o the native connector - Was only slightly slower than going straight through Apache about 7-8 MB/sec On Solaris: * Apache 2.0.55, Tomcat 5.0.30, recent mod_jk - Fairly constant 4MB/s when going through mod_jk, 10MB/s when just downloading via Apache [This issue originally was thought to be Windows specific, which is why we have many more results for Windows.] Obviously if our end goal was simple static file transfers we'd just share/mirror them to Apache to solve this (we need the load balancing flexibility, etc, of mod_jk, so directly using Tomcat is not really an option -- nor is doing non-AJP-proxying). The static file case is the simplified reproduction of our real issue, however, which is large file downloads from our (Java-based) content store. We had much better results with Apache 2.2.3 and Tomcat 5.5.20 with tcnative, but we really don't want to force a move to 2.2.x and Tomcat 5.5.x in this case and we've had issues with tcnative (which we *hope* may be resolved with 1.1.8). Overall we'd much prefer to get mod_jk working reasonably than to force a disruptive move to 2.2.x right now. Is this a known issue? Any pointers as to where/how to look for the performance bottleneck? Some VTune examination showed that almost all of Apache's CPU time during this time was in libapr.dll, but that's obviously not terribly specific. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Serious mod_jk performance issue
We're seeing a *serious *performance issue with mod_jk and large (e.g. 500MB+) file transfers. [This is with Apache 2.0.55, Tomcat 5.0.30, and various recent mod_jk including 1.2.20.] The performance of downloading the file via Apache is good, as is the performance when downloading directly from Tomcat. The performance when downloading from Tomcat through Apache via mod_jk is, however, quite abysmal. I'd obviously expect *some* degradation due to the extra interprocess hop, but given that this is a just a single-user, single-request test, I'd expect that the network would still be the limiting factor -- or at least that the degradation would be in the order of 25% of less. What we're seeing, however, is far worse: On Windows: * Apache 2.0.55, Tomcat 5.0.30, and mod_jk 1.2.20 - Started at 10 MB/sec ended at 3 MB/sec with mod_deflate disabled (1.5 MB/sec with mod_deflate enabled) * Apache 2.0.55, Tomcat 5.0.30, and mod_jk 1.2.19 - Disabling JkFlushPackets only slightly improved performance. * Apache 2.2.3 with Tomcat 5.5.20 w/ the native connector - Didn't work period. I didn't have a chance to look into it, but the download failed after getting serveral packets (!) * Apache 2.2.3 with Tomcat 5.5.20 w/o the native connector - Was only slightly slower than going straight through Apache about 7-8 MB/sec On Solaris: * Apache 2.0.55, Tomcat 5.0.30, recent mod_jk - Fairly constant 4MB/s when going through mod_jk, 10MB/s when just downloading via Apache [This issue originally was thought to be Windows specific, which is why we have many more results for Windows.] Obviously if our end goal was simple static file transfers we'd just share/mirror them to Apache to solve this (we need the load balancing flexibility, etc, of mod_jk, so directly using Tomcat is not really an option -- nor is doing non-AJP-proxying). The static file case is the simplified reproduction of our real issue, however, which is large file downloads from our (Java-based) content store. We had much better results with Apache 2.2.3 and Tomcat 5.5.20 with tcnative, but we really don't want to force a move to 2.2.x and Tomcat 5.5.x in this case and we've had issues with tcnative (which we *hope* may be resolved with 1.1.8). Overall we'd much prefer to get mod_jk working reasonably than to force a disruptive move to 2.2.x right now. Is this a known issue? Any pointers as to where/how to look for the performance bottleneck? Some VTune examination showed that almost all of Apache's CPU time during this time was in libapr.dll, but that's obviously not terribly specific. -- Jess Holle
Re: Tomcat 6.0.5 and tcnative 1.1.8 ?
Just an note: We're also having odd problems with tcnative -- but on 5.5.20... All problems disappear without tcnative enabled, but performance suffers... -- Jess Holle Remy Maucherat wrote: Henri Gomez wrote: Hi to all, There are some problems with tcnative and I see today some fixes in Tomcat and tcnative. Should we expect a release of Tomcat 6 and tcnative in the next days ? I understand you have a problem. However, can you try to stop flooding when that happens ? Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_jk 1.2.17 vs. mod_proxy_ajp
It would probably make sense to broadcast a list of improvements, etc, when 1.2.18 is released at least... Henri Gomez wrote: 2006/7/18, Jim Jagielski [EMAIL PROTECTED]: On Jul 14, 2006, at 10:34 AM, Jess Holle wrote: What is the plan for mod_proxy_ajp catching up with all the improvements in the latest mod_jk? As I understand it, mod_proxy_ajp is currently lagging, which is understandable -- one or the other has to. I'm just wanting to get some grasp of the probable/planned timeframe for mod_proxy_ajp catching up. Since most of the people working on the mod_proxy_* stuff aren't also on the TC dev lists, it is hard for them to know of the improvements, etc... :) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_jk 1.2.17+ Recover time
Is the 60 seconds hard-coded? I'd hope not... Once you have some interesting web apps in Tomcat it often takes a bit longer than 10 seconds -- and on my laptop just took a full 60 seconds, but that is rather unusual (a restart thereafter only took 18). David Rees wrote: I've been testing the 1.2.17 (soon to be 1.2.18) release and have noticed a problem in worker recovery. If I restart a Tomcat instance and mod_jk notices that it went down, mod_jk waits 60 seconds recovery time before it tries again (see jk_lb_worker.h WAIT_BEFORE_RECOVER and struct jk_shm_worker recover_wait_time). However, Tomcat will typically recover in just a handful of seconds so this results in nearly a minute of downtime when downtime should only be perhaps 10 seconds. Compounding this problem it doesn't appear to be possible to override this behavior either through the worker configuration and the status module forces a minimum of 60 seconds. The only workaround to this problem I see is to setup a Tomcat cluster, but this isn't feasible in all cases. I'm more than happy to help work up a patch to allow configuring of this parameter in the workers file and to allow a lower minimum recover_wait_time as well if such a patch would be accepted. Of course, if a mod_jk developer can cook it up quickly that is fine too, I will test it. -Dave - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
mod_jk 1.2.17 vs. mod_proxy_ajp
What is the plan for mod_proxy_ajp catching up with all the improvements in the latest mod_jk? As I understand it, mod_proxy_ajp is currently lagging, which is understandable -- one or the other has to. I'm just wanting to get some grasp of the probable/planned timeframe for mod_proxy_ajp catching up. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: mod_jk 1.2.17 vs. mod_proxy_ajp
Rainer Jung wrote: I think Mladen and I both want people to move to mod_proxy_*. Mladen already offered help and httpd-dev for porting mod_jk improvements to mod_proxy_* and I definitely want to join him in that. But first I need to manage the pending mod_jk release, something we are right now doing. Understood. That makes perfect sense. I don't know mod_proxy_* code good enough to make a precise statement about schedule for porting features, but I assume, that mod_proxy_* will be very close to mod_jk (if not better) around october. Thanks. That sort of rough timeframe is very helpful -- even if is slips by a couple of months. There are also plans by jfclere, Mladen and others to improve mod_proxy_* and possibly AJP much further than what mod_jk/AJP13 is able to do at the moment, but I'm sure this part is much more complicated and will take a couple of months. That can wait -- as much as I'd like to see support for 8K headers! -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: JK and AJP protocol enhancements
Mladen Turk wrote: You see how desperate I am when writing this on Sunday :) Anyhow, we are pretty close to the new JK release that I hope will be the most usable and stable whatsoever. The things we agreed so many times before, but having obviously too little resources to actually create are the 1.3 branch (aka JK3) and the AJP protocol stuff. Now there is the problem with that. Henri even created a AJP1.4 protocol enhancements with all that login, discovery etc... (never implement but thats another story). Although we got close to the AJP 1.4 protocol conclusion last year, nowadays all that looks strange to me. All those things might be implemented, but IMHO only as a AJP1.5 protocol. What we desperately need right now are 3 things: 1. Allowing to have +8K headers 2. Allowing to have +0x single header limit 3. Mechanism to tell the Tomcat to gracefully close the connection. Now, the number 3 is very easy. A simple message like we have for SHUTDOW, but instead shutting down the entire Tomcat instance, closing down the socket/channel. OTOH first two are little bit tricky :) I have some ideas: 1. Larger headers can be treated as we handle the POST data. If there are more headers then 8K, then a servlet container should send GIVE_ME_MORE_HEADERS message. 2. If the single header is larger then 39321 bytes, then it should be send as POST data, with servlet container requesting 8K packets. Those headers would be treated as multiple POST sequences, after the initial header(s) packet(s) have been read and before the actual POST data is read. Any comments? Only that item (1) is a short-term must for some... -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: NIO Connector, please review
Remy Maucherat wrote: Filip Hanik - Dev Lists wrote: Remy Maucherat wrote: Filip Hanik - Dev Lists wrote: Alright, fixed, my last night checkin, the one I was unable to diff, caused the problem, I have reverted that one, and the Connector since we can use the fully qualified classname here. I am able to diff to see what got fixed, and it's trivial: all requests were processed as Comet events. yup, noticed that when I ran it in a debugger, I wanted to consolidate the two processSocket method into one, i did it wrong. I can confirm it now works. The two methods are fundamentally different, and one of them was actually named processSocketEvent until I changed it back to processSocket. However, the throughput is very bad at the moment (/usr/sbin/ab.exe -n 2 -c 10 -k http://127.0.0.1:8081/tomcat.gif): 800 req/s (with keepalive actually working) vs 6000+ for regular java.io. That's unfortunate. So regular is better? What are they doing with Grizzly? -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Core webapps and clustering
Remy Maucherat wrote: Henri Gomez wrote: May be not related, but did there is plan in TC 6.x to make use at some time OSGI framework, like the one used in Eclipse and RCP applications ? I really like this concept and it seems a good candidate to provide a modular kernel / micro-architecture. If we do that, what doesn't make it a Geronimo clone ? The services that are shipped by default, maybe ? ;) It also most likely make Tomcat more heavyweight, although I don't know whether or not it would make it more difficult to integrate. As someone who really wants to integrate Tomcat into a larger application soon (I had a working prototype for a while), I really don't want Eclipse or its RCP -- or anything of the sort -- in Tomcat. I like the fact that Tomcat is still relatively lightweight and brings relatively few extra libraries and version conflicts thereof into the picture. As long as I keep a few of the Apache libraries I use up-to-date, all is well (and probably would be otherwise -- it's just really easy to remove any possible issues by version matching). Future NetBeans versions may cease to embed Tomcat and embed the whole Glassfish thing instead, I don't know, but the embedding of Tomcat in current NetBeans releases is a perfect example of why no IDE's faddish RCP (Eclipse's, NetBeans', or new-sprocket-fad-xyz) should not make its way into Tomcat. [Sorry for any cynicism, but I've seen a rash of wouldn't our simple, lightweight open source component X be oh so much better if we just stuck in an entire IDE framework underneath knee jerks in open source communities ranging as far afield as jManage. Let's K.I.S.S!] -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Core webapps and clustering
Ah, I think I misread your suggestion... Having support for clean extension modules where appropriate would be a fine thing for Tomcat. [For instance, the ability to easily replace the form-based authentication mechanism so as to be transparently compatible with basic authentication while still supporting session-based authentication -- all in the same web app...] Henri Gomez wrote: Well being modular, components oriented won't be bad. It's not about cloning Geronimo, but allowing tomcat to get more and more modules and extensions, like does Apache HTTPD 2006/5/5, Remy Maucherat [EMAIL PROTECTED]: Henri Gomez wrote: May be not related, but did there is plan in TC 6.x to make use at some time OSGI framework, like the one used in Eclipse and RCP applications ? I really like this concept and it seems a good candidate to provide a modular kernel / micro-architecture. If we do that, what doesn't make it a Geronimo clone ? The services that are shipped by default, maybe ? ;) It also most likely make Tomcat more heavyweight, although I don't know whether or not it would make it more difficult to integrate. Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC 6: What needs to be done?
Version #'s are cheap and Tomcat 6.5 and 7 have nice rings to them... Rainer Jung wrote: So people think it TC 6 should be focused on JEE 5 compatibility and other major changes should only be done, if they pose no risk on an early date. Somehow we are to late in the J2EE time frame. On the other hand, major changes to e.g. the configuration management should not be done for a release that looks like a minor release (6.1), because then users will be very irritated that they have to learn a very new configuration world. So as a consequence we might need to prepare for a TC 6.5 or TC 7 until end of this year? This will look strange at a first glance (why two major releases so shortly after each other), but if we decide to focus TC 6 on JEE 5 compatibility and early delivery it looks like a logical consequence. - Rainer Jess Holle wrote: [EMAIL PROTECTED] wrote: Would it be suitable to focus on JEE 5 compliance in the short run for a soon to be released TC 6, then enhance the value adds of TC for a 6.1 release later this year? That sounds a lot better to me... -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC 6: What needs to be done?
Costin Manolache wrote: On 5/5/06, Jess Holle [EMAIL PROTECTED] wrote: Costin Manolache wrote: On 5/5/06, Rainer Jung [EMAIL PROTECTED] wrote: 1) Configuration Management === My impreesion is, that to much configuration is hard-coded in RuleSet classes. Of course everyone can easily add properties to existing components, but adding subcomponents nedds changes in core RuleSet classes. Am I wrong? Should we change that to allow more complex subsystems handling their configuration rules independently of the core? One such example is the current stable clusster module. IMO the entire server.xml and RuleSet should be deprecated, and replaced with JMX. We could keep current server.xml around, for compat - but at least not extend it. Even the very limited MLET syntax can support most of our needs. RuleSets are just a way to set attributes ( == jmx setters ) and define hierarchy of componets ( == can be done based on JMX names, in a more dynamic way ) I'd generally tend to agree, but suggest that thought be given not just on elegant configuration but also on the ability to change things via JMX and then *save* the results as an updated configuration. 'save' is on my list for about 2 years now :-) But this would be the second step - after we use JMX to set up tomcat instead of server.xml. I just bring it up as I have completed a JMX framework for our use with round trip persistence to XML. The biggest limitation in my case is that the configuration files are not (nearly) so elegant as something like Spring's -- though they're rather similar to a point (I use Java's built-in XMLEncoder/XMLDecoder stuff). This is primarily so that the original configuration file can be reproduced reliably with very little persistence architecture on my part. Trying to provide the richness of something like Spring and provide round-trip saving would seem to be the holy grail -- and possibly just as unattainable. My old plan was to use something like MLET, and DOM to read/save ( so most of comments/structure is preserved ). Use of XMLEncoder/XMLDecoder allows persistence of arbitrarily complex substructures, which is nice. The result is not extraordinarily reader-friendly, but it is quite general. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC 6: What needs to be done?
Costin Manolache wrote: On 5/5/06, Jess Holle [EMAIL PROTECTED] wrote: AFAIK XMLEncoder/XMLDecoder will save all the attributes, and lose comments in the round trip. Yep. I'm not trying to save comments in my case -- just configuration values. What I feel is missing is that I persist *values*. I cannot persist the notion that the value was obtained via $(propertyName) substitution in the XML, for example. With DOM - you have a good round trip, and if all config happens via JMX, you know what attributes have changed. True enough. One could also merge these in via XSLT. Apple and Mozilla configs are the models I'm considering, even if some of the structure is lost in the round trip, I think having only the modified elements is essential. I don't think you can do this with XMLEncoder, and our server.xml saving doesn't seem to support this either. I'm not sure what you mean by having only the modified elements. XMLEncoder only saves attributes that differ from their default values, if that's what you mean. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: TC 6: What needs to be done?
Costin Manolache wrote: I didn't know this - how does it find the defaults ? I'm not talking only about default values of a property, but also in the object hierarchy, tomcat creates a lot of components by default, if you don't specify a manager or logger, a default one will be created. Seems pretty specific to tomcat - can't see how XMLEncoder would guess that. It can't guess. For each object that is being encoded it has to be able to create another from scratch and then it basically diffs the two and writes the differences. Sorry for the poor explanation -- it's all I have time for at the moment. It's probably possible to arrange things such that XMLEncoder would capture some level of hierarchy aspects as differences and defaults, but I'm not sure how workable this would be. [I have a top-level MBean that keeps a list of top-level entities (primarily other MBeans) that then are delegated to for everything below that point, so the defaulting stuff only occurs below the top-level in my case -- and the hierarchy below that point is empty by default...] -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Why is IBM still using Tomcat 5.0?
tc-dev wrote: My guess is that it was probably developed with an IBM 1.4.2 jvm so running on Tomcat 5.5 would be out of the question. As plenty of folk will tell you, Tomcat 5.5 runs just fine on 1.4.2... You just need the compatibility add-on pack. My own guess -- IBM may have changed a lot, but at least parts of big things still move slowly. For instance, WebSphere still won't officially support Java 5 until some time next month. -- Jess Holle Hola, Do we have anyone from IBM here who can tell us what's stopping them from using Tomcat 5.5, as opposed to 5.0, for new tools like their Web Ontology Manager that was announced earlier today? (http://www.alphaworks.ibm.com/tech/wom/requirements?openS_TACT=105AGX59S_CMP=GR) You can clearly see they're up to date on details, like the site name having changed from jakarta.apache.org/tomcat to tomcat.apache.org, by looking at their installation instructions... Weird. -- Yoav Shapira Nimalex LLC 1 Mifflin Place, Suite 310 Cambridge, MA, USA [EMAIL PROTECTED] / www.yoavshapira.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Memory usage Tomcat 3.3 vs 5.5
Hmm... So model MBeans are not such a pancea after all :-) They're certainly the most complex and unapproachable form of MBean, which Commons Modeler tries to address, of course. I alway suspected that they had an overtly heavy nature at runtime, though -- and it appears that there is some truth to this at least in Tomcat's use case. When I first dove into JMX, I expended an annoying amount of time just trying to figure out which type of MBean to use. I ended up writing my own drop-in replacement class for javax.management.StandardMBean and immediately subclassing this to add more functionality for my other MBeans to inherit. [The issues that caused me to write a replacement have since been addressed in Java 6, but are not all addressed in Java 5 last I knew.] This approach gives standard MBean ease of use wherever possible with any dynamic or open MBean stuff added as needed. [By the way, is it just me or do others also find the PersistentMBean interface useless?] I end up caching and thus sharing MBeanInfo for each MBean interface and implementation class (since implementation class MBeanInfo is generally almost the same as that generated from the MBean interface). Even where the MBeanInfo is more dynamic it is based on *something* else, e.g. another target object's class, and is thus cached and shared on that basis. In the end each MBean needs very few additional fields and these just point to shared objects. The costs of not taking this kind of approach seems clear below... -- Jess Holle Abhi Karmos wrote: Hi all, Following up on this thread, we took a dump of the heap running tomcat 5.5. There is no good reason for JMX to consume 9-10 Meg. 97,236 5,178,736 array of char 25,494 3,463,312 array of java/util/HashMap$Entry 105,908 3,389,056 java/util/HashMap$Entry 98,510 3,152,320 java/lang/String 98,200 2,356,800 javax/management/modelmbean/DescriptorSupport$ValueHolder 25,343 1,216,464 java/util/HashMap 348 830,888 array of byte 18,222 728,880 javax/management/modelmbean/ModelMBeanAttributeInfo 21,937 526,488 javax/management/modelmbean/DescriptorSupport Has anybody looked in the code where the DescriptorSupport class is used? 22 thousand of these things is simply absurd. Nearly 100 thousand DescriptorSupport.ValueHolder objects is even crazier. My guess is that the HashMap usage is related to the DescriptorSupport. Thanks, Abhi. OS: Linux 2.4.18 / RedHat 7.3 JVM: IBM 1.4.2 SR3 Henri Gomez [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] We're also using TC 3.3 on our production systems and are switching to TC 5.5. The memory penalty is real but you'll be using up to date and actively maintained stuff with TC 5.5. Also you could try to reduce the AJP13 / HTTP 11 threads and remove some JMX Listeners 2006/4/19, Abhi Karmos [EMAIL PROTECTED]: Hello all, Sorry, could not find an answer on tomcat-users. We run tomcat inside an embedded system and we are planning to upgrade the tomcat version from 3.3.1 to 5.5.16. We noticed that the memory footprint of a barebones tomcat 5.5 installation is 44 MB. Plain and simple Tomcat 3.3 used to run with 25 MB. We have limited memory on our system and the memory usage of tomcat 5.5 is a concern for us. The numbers above are the resident set size (RSS) on a linux system. When we took the above numbers, both the tomcats had the same heap setting (min: 25 - max: 30 mb). When we run our webapps, which add another 20 jars in the classpath, the steady state memory usage is 50 MB with Tomcat 3.3 and 66 MB with Tomcat 5.5. Has anyone else solved a similar problem before ? Can I try and tweak tomcat 5.5 to run in less memory ? Our server.xml is the minimal configuration possible. Regards, Abhi. OS: Linux / RedHat 7.3 JVM: IBM 1.4.2 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Memory usage Tomcat 3.3 vs 5.5
Remy Maucherat wrote: Abhi Karmos wrote: Has anybody looked in the code where the DescriptorSupport class is used? 22 thousand of these things is simply absurd. Nearly 100 thousand DescriptorSupport.ValueHolder objects is even crazier. My guess is that the HashMap usage is related to the DescriptorSupport. It is completely irrelevant compared to the memory usage of a single modern web application, and at the same time it provides a useful monitoring feature without being intrusive coding wise. Sure, we could use standard MBeans, but the code would then be a mess. I don't understand the point of bothering with this sort of stuff when there's so little benefit. Despite my tone in my previous e-mail on this thread, I tend to agree with Remy that the Tomcat folk have bigger fish to fry. An extra 10MB of overhead is not the top priority for most Tomcat users. Even without using standard MBeans I could certainly see dynamic MBeans with per-target-object-class caching eliminating most of this additional memory footprint. Modeled MBeans are likely as capable of this as any other dynamic MBeans -- and if not that would be a reason not to use them in *future* projects. [Commons Dynamic anyone?] All the same, the MBeans generally do what's needful in Tomcat and I'd thus hate to see any cleanup in this area take precendence over real stuff. If you want the smallest web container in terms of memory usage or size, I think you should not be using Tomcat. Tomcat is more geared towards speed, GC friendliness and feature completeness than memory usage. Agreed. I notice that many of the Tomcat MBeans actually provide fairly useful attribute and MBean descriptions. I also notice a good number of MBeans with no MBean-level description, however. Same for operations, operation parameter names and operation parameter descriptions. Fixing this seems like a higher priority than introducing MBean descriptor/MBeanInfo sharing at this point. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Memory usage Tomcat 3.3 vs 5.5
Costin Manolache wrote: I have to disagree with any statement like this - nobody can decide that his use case is the 'main' use case for tomcat. Yes, tomcat is used in a lot of production environments where speed and manageability are important. But it is also used by developers ( and I suspect for each production site you have few dozen developer instances ), it is used in IDEs like eclipse/idea/etc. Tomcat is also used to deploy large number of very small apps, or small number of large apps, or small set of small apps :-) Not sure what is the plan for commons-modeler, I think it is worth to bring it back to tomcat or fork a separate implementation, not using model mbeans and simplifying the model. I think JMX is a great tool, and is essential to support it - but there are ways to do it without bloat, and it's a valid target for optimizations. More power to such efforts as well. I just see performance/scalability, management functionality, robustness, spec compliance, etc, as higher priorities. If there is time and energy, I'd love see to all redundant MBeanInfo objects eliminated such that a single instance was used in all cases where the MBeanInfo is identical. There certainly is no point to having such a see of small, long-lived, redundant data objects cluttering up the heap. -- Jess Holle On 4/25/06, Jess Holle [EMAIL PROTECTED] wrote: Remy Maucherat wrote: Abhi Karmos wrote: Has anybody looked in the code where the DescriptorSupport class is used? 22 thousand of these things is simply absurd. Nearly 100 thousand DescriptorSupport.ValueHolder objects is even crazier. My guess is that the HashMap usage is related to the DescriptorSupport. It is completely irrelevant compared to the memory usage of a single modern web application, and at the same time it provides a useful monitoring feature without being intrusive coding wise. Sure, we could use standard MBeans, but the code would then be a mess. I don't understand the point of bothering with this sort of stuff when there's so little benefit. Despite my tone in my previous e-mail on this thread, I tend to agree with Remy that the Tomcat folk have bigger fish to fry. An extra 10MB of overhead is not the top priority for most Tomcat users. Even without using standard MBeans I could certainly see dynamic MBeans with per-target-object-class caching eliminating most of this additional memory footprint. Modeled MBeans are likely as capable of this as any other dynamic MBeans -- and if not that would be a reason not to use them in *future* projects. [Commons Dynamic anyone?] All the same, the MBeans generally do what's needful in Tomcat and I'd thus hate to see any cleanup in this area take precendence over real stuff. If you want the smallest web container in terms of memory usage or size, I think you should not be using Tomcat. Tomcat is more geared towards speed, GC friendliness and feature completeness than memory usage. Agreed. I notice that many of the Tomcat MBeans actually provide fairly useful attribute and MBean descriptions. I also notice a good number of MBeans with no MBean-level description, however. Same for operations, operation parameter names and operation parameter descriptions. Fixing this seems like a higher priority than introducing MBean descriptor/MBeanInfo sharing at this point. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Memory usage Tomcat 3.3 vs 5.5
Hmm... Sharing redundant MBeanInfo's shouldn't be that hard to wedge in. Unfortunately, I don't know when I'll have time to take a proper look at this... -- Jess Holle Remy Maucherat wrote: Costin Manolache wrote: Not sure what is the plan for commons-modeler, I think it is worth to bring it back to tomcat or fork a separate implementation Modeler is now in the tomcat.util package in the new repository, so you can try to improve it if you'd like to. One first change I made is that it uses the digester to read the descriptors (rather than DOM). Rémy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Memory usage Tomcat 3.3 vs 5.5
Costin Manolache wrote: On 4/25/06, Remy Maucherat [EMAIL PROTECTED] wrote: Costin Manolache wrote: Not sure what is the plan for commons-modeler, I think it is worth to bring it back to tomcat or fork a separate implementation Modeler is now in the tomcat.util package in the new repository, so you can try to improve it if you'd like to. One first change I made is that it uses the digester to read the descriptors (rather than DOM). I'll take a look, maybe we don't really need model mbeans. Standard MBeans default to caching MBeanInfo on a per-instance, rather than per-interface or per-implementation-class basis. I'd suspect that ModelMBeans have a similar issue by default. I'd guess the same MBeanInfo is used for all instances based on a given target class. In that case, it should be possible to cache/share on that basis, which should reduce the memory MBeanInfo related portion of the footprint from sum(targetType1*nObjectsOfType1,...)*mbeanInfoSize to simply nTargetTypes*mbeanInfoSize. -- Jess Holle
Re: configurable AJP Buffer Size
Henri Gomez wrote: Just take a look at the AJP protocol documentation : http://tomcat.apache.org/connectors-doc/common/ajpv13a.html Packets sent from the server to the container begin with 0x1234. Packets sent from the container to the server begin with AB (that's the ASCII code for A followed by the ASCII code for B). After those first two bytes, there is an integer (encoded as above) with the length of the payload. Although this might suggest that the maximum payload could be as large as 2^16, in fact, the code sets the maximum to be 8K. So to support more than 64k bytes, we should have a whole new code on Apache and Tomcat side and break compatibility with all current implementations. And that's bad ;( We recently saw a customer run into this limitation in that all of their headers, etc, would not fit in the first 8K. Waiting for a new AJP version sounds like a non-starter. Can we just have something like a JkOption that allows one to explicitly opt into this arrangement? Address this for mod_jk and Tomcat and one is done for now (though a mod_proxy_ajp shouldn't be far behind). Take a look at the AJP13 extension proposal : http://tomcat.apache.org/connectors-doc/common/ajpv13ext.html We could plug many usefull informations in LOGIN INIT for example ! These proposals all look interesting. The problem is that there are lots of proposals and this would seem to inevitably take quite some time to stabilize all of this in mod_jk, mod_proxy_ajp, and Tomcat (native and Java). I'd love to be proven wrong, though. -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: configurable AJP Buffer Size
Henri Gomez wrote: These proposals all look interesting. The problem is that there are lots of proposals and this would seem to inevitably take quite some time to stabilize all of this in mod_jk, mod_proxy_ajp, and Tomcat (native and Java). I'd love to be proven wrong, though. Not necessary, if you take a look at curent jk/ajp implementations (native and java), there is allready code in it for AJP14 support :) Including the ability to increase the packet size (or specifically to use a larger packet if necessary to capture all the headers)? How stable is AJP13 functionality in this code? I currently only need this one additional feature :-) -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] Tomcat v5.5.15 Stability
We had issues with APR on Solaris, but: * These were with 5.5.12 (and I believe were related to shutdown). * We've not had time to investigate further, so we've not bothered anyone (as this may be our APR build or some such). Overall, even if this was reproducible I'd not consider these worth withholding the stable label unless this could be shown to be a regression from previous stable releases containing APR. [Thus I wouldn't have considered this worth holding up 5.5.12 at any rate and only worth holding up 5.5.15 if it worked in 5.5.12 (and was reproducible, of course).] -- Jess Holle Mladen Turk wrote: M A wrote: I have issue with the APR not delivering a page that was approx 14KB in size, I did post it on here but no one responded. The thing you've posted makes no sense without some way how to reproduce the bug. This means that you would need to give us the jsp and the exact procedure how to duplicate the bug. BTW, what happens if you use some 'normal' client? Is it a show stopper well for me it is it means i cant use APR .. Well, the APR works both on RHEL kernels and Debian one, because I use regulary both of those platforms for testing. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [VOTE] Tomcat v5.5.15 Stability
P.S. We've not had a chance to test this on 5.5.15 to the best of my knowledge. Jess Holle wrote: We had issues with APR on Solaris, but: * These were with 5.5.12 (and I believe were related to shutdown). * We've not had time to investigate further, so we've not bothered anyone (as this may be our APR build or some such). Overall, even if this was reproducible I'd not consider these worth withholding the stable label unless this could be shown to be a regression from previous stable releases containing APR. [Thus I wouldn't have considered this worth holding up 5.5.12 at any rate and only worth holding up 5.5.15 if it worked in 5.5.12 (and was reproducible, of course).] -- Jess Holle Mladen Turk wrote: M A wrote: I have issue with the APR not delivering a page that was approx 14KB in size, I did post it on here but no one responded. The thing you've posted makes no sense without some way how to reproduce the bug. This means that you would need to give us the jsp and the exact procedure how to duplicate the bug. BTW, what happens if you use some 'normal' client? Is it a show stopper well for me it is it means i cant use APR .. Well, the APR works both on RHEL kernels and Debian one, because I use regulary both of those platforms for testing. Regards, Mladen. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Tomcat 6 plans (JSP 2.1)
Remy Maucherat wrote: Allistair Crossley wrote: Hi, Personally I am less interested in a small footprint Tomcat and more interested in tools that help manage and report on the internals of Tomcat. Instrumentation, JMX, effective and stable debugging and deployment, clustering and load balancing are the types of areas that would help me out with our corporate intranet. I doubt any of these issues are of real concern to the development team because they are supporting of the main container, but they matter on the front-line and often piecing together different technologies via modules which have varying amounts of documentation and stability is tough and time-consuming. I've been fighting getting in WebLogic, WebSphere and Jboss but it looks like it's going that way in 2006 :( Personally, I am also not interested at all in changes to the container architecture either, and I plan to keep using the current codebase to do the Servlet 2.5 support in JBoss. It mostly does what I need, and I think changing it a lot would break more things than what it would fix. - Debugging: No idea, do you have some ? - Deployment: Ok, it's an independent module (HostConfig, and the associated manager webapp, basically); I did it twice already, I think it's half decent, but I am sick of it, so if anyone feels like doing it again, go ahead. - Clustering: We have clustering already. What's wrong with it besides bugs ? (if it had bugs, it's because of not enough users, testing, and bugfix contributors: you can help) - Load balancing: We have mod_jk, and now mod_proxy in Apache 2.2. Do you need more ? I had ideas for an AJP APR client implementation, but I am not sure there's an actual demand for that (I don't see the point of having a Tomcat as a front end server, given it's a single purpose task and exisitng ASF software seem to be doing it just as well). The main item you didn't mention is instrumentation/JMX. This is an area that should not require any substantive rearchitecture and could greatly benefit most users. I know JBoss has more JMX stuff, but having the Tomcat end of things quite well instrumented in Tomcat proper seems like the right way to go. I have to support a number of servlet engines, so I ended up doing my own MBeans for things I can get at via the servlet APIs, i.e. so I have portable request and session monitoring/timing/logging, etc. There are, however, things that are not accessible via the servlet APIs or are just a royal pain to do at that level (e.g. accurate per request I/O byte counting/logging). I'm also uncertain what debugging improvements should be made if any -- apart from the fact that precompilation of JSPs does not seem to generate full SMAP information even when told to do so (yes, I filed this on BZ, but I've not had a chance to delve further myself). That's either user error on my part (though I'm baffled as to how this could be) or a plain/simple bug, though. The only bit with deployment I can think of is easing/automating/documenting means of deploying to a cluster of Tomcats at once. I could just be overlooking wonderful capabilities in this regard, however, as I've not really looked all that hard here. BTW, I am biased, but I don't see a move to JBoss as being that bad. If you use a web oriented configuration, you end up with something that has the same web capabilities as Tomcat, but with a few more robust components for important functionality (TM, datasource, clustering, etc). Unfortunately, it uses more resources ;) There is something to be said for using just enough hammer. Unless you need EJBs, a TM (i.e. other than JOTM), or JMS, it's not clear why you'd add the extra layers. [I suspect someone will pipe in with info on a nice open source, maybe even XA-aware JMS provider that can be used without an app server as well.] -- Jess Holle - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Who is working on the ISAPI connector code?
In IIS 6, you also have to run in IIS 5 compatibility isolation mode. I have no love for IIS myself but we also have customers that have some inexplicable emotional attachment to it. Is it just NTLM? In which case someone should shove mod_auth_sspi into Apache and be done. If not, what's the fascination with IIS? In our case our application is purely Java, so in direct conjunction with it ASP, .Net, etc, are of no benefit whatsoever. It would be helpful if either: (1) all remaining rational reasons for using IIS (e.g. NTLM) were answered by Apache or (2) the IIS/Tomcat connector was not the ugly step child that no one wanted. -- Jess Holle Preston L. Bannister wrote: So - in other words there are no toes to step on :). Unfortunately recommending Apache2 over IIS is pretty much a non-starter in my world, so a working IIS connector is needed (at least to bring server-side Java into this world). On 12/12/05, Tim Whittington [EMAIL PROTECTED] wrote: The ISAPI connector has been somewhat unloved over the past few years. We're recommending Apache 2 over IIS to all our customers, so if that's widespread it'd explain the lack of interest. I've been submitting patches from our local branch over the past year, trying to improve some aspects of it (still waiting for the next dev branch to add chunked encoding support). I've also updated some of the docs to mention features that have been there for years (i.e. isapi_redirect.properties). The only other person I've seen touch the IIS code in recent memory is Mladen. Binary JTC builds from Jakarta have always been a bit hit and miss - I build all ours locally. The only IIS6 issue I know of is that you have to add a web services extension for the redirector for it to work - new locked down IIS rules or something. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
