Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
Oh, I knew that one :( Thanks anyway, y un saludo, Alex. Steve Downey wrote: It's one of the alternate names for Abstract Factory, from the GoF book. AKA toolkit. The idea is that you have a an abstract class with methods such as createThing1 and createThing2, which return abstract Thing1s and Thing2s. A concrete implementation of the factory, xFactory, returns an xThing1 and an xThing2, and a yFactory returns yThing1 and yThing2. A classic example is GUI widgets, where you might have MotifButton, MotifScrollbar, MotifWindow, ..., or Win32Button, Win32Scrollbar, Win32Window, ..., or GtkButton, GtkScrollbar, GtkWindow, ... Here we have CommandLineMangler, CommmandLineContext, CommandLineCompiler, and JspMangler, JspEngineContext, JspCompiler, ... They don't seem to be pure mix and match. That is, it doesn't necessarily make sense to use an XCompiler with an XContext, although it might. So the overhead of having to create a new Factory implementation for a new combination or style isn't that excessive. -Original Message- From: Alex Fernndez [mailto:[EMAIL PROTECTED]] Sent: Friday, March 30, 2001 2:49 AM To: [EMAIL PROTECTED] Subject: Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet Hi Steve! Steve Downey wrote: Perhaps a Kit pattern is in order? Wow, a Kit pattern. I never heard of that one (or never got that far in the Patterns books :) Is it a standard one? If so, I'll check it out later at home. Un saludo, Alex. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
It's one of the alternate names for Abstract Factory, from the GoF book. AKA toolkit. The idea is that you have a an abstract class with methods such as createThing1 and createThing2, which return abstract Thing1s and Thing2s. A concrete implementation of the factory, xFactory, returns an xThing1 and an xThing2, and a yFactory returns yThing1 and yThing2. A classic example is GUI widgets, where you might have MotifButton, MotifScrollbar, MotifWindow, ..., or Win32Button, Win32Scrollbar, Win32Window, ..., or GtkButton, GtkScrollbar, GtkWindow, ... Here we have CommandLineMangler, CommmandLineContext, CommandLineCompiler, and JspMangler, JspEngineContext, JspCompiler, ... They don't seem to be pure mix and match. That is, it doesn't necessarily make sense to use an XCompiler with an XContext, although it might. So the overhead of having to create a new Factory implementation for a new combination or style isn't that excessive. -Original Message- From: Alex Fernndez [mailto:[EMAIL PROTECTED]] Sent: Friday, March 30, 2001 2:49 AM To: [EMAIL PROTECTED] Subject: Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet Hi Steve! Steve Downey wrote: Perhaps a Kit pattern is in order? Wow, a Kit pattern. I never heard of that one (or never got that far in the Patterns books :) Is it a standard one? If so, I'll check it out later at home. Un saludo, Alex. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
--- Steve Downey [EMAIL PROTECTED] wrote:
-Original Message-
From: [EMAIL PROTECTED]
On Thu, 29 Mar 2001, Mel Martinez wrote:
activities that should be orthogonal. The only
dependency should be that the Compiler is a
consumer
of the products of the Mangler.
+1
If we treat the embedded implementation of Mangler
as a separate class,
which I think is a good idea, each style of JSP
compilation works with a
triad of somewhat dependant classes. An X-Compiler,
an X-Context and an
X-Mangler. If we're playing games with the name of
the generated class, like
decorating it with a version number, then some other
part of the system
needs to understand the Mangling scheme as well.
ClassName does some of this
work now, finding out what the 'real' name of the
class is.
Perhaps a Kit pattern is in order?
Ah yes, the venerable ToolKit!
This is basically what I'm going for, with two tiers:
- a static, default layer of factory methods that is
independent of the runtime adaptor scheme. This is
done via factories in a subclass of Constants. This
layer has default behavior that is configurable via
standard configuration properties (both via -D and via
resource file).
- a run-time layer of factory methods that overrides
the default layer based on run-time options such as
init-params. This is to be implemented in a subclass
of Options.
This pattern allows each adapter (i.e. JspC,
JspInterceptor) to adopt just the shared behavior that
is necessary by extending/re-implementing it's own
'options'-layer factories.
I must admit to not being totally sure whether
co-opting the current 'Constants' and 'Options' class
families is the best naming for this, but they do
clearly indicate the difference in scope. I could
break the factory methods out into separate, new
'ConstantsToolkit' and 'OptionsToolkit'
class/interface types, but since they would parallel
the original, I don't see the real advantage.
Refactoring Mangler is highly desireable from my day
job point of view, too,
since the current mangling scheme can cause problems
on Windows. Although we
might be able to move to Linux for developers
desktops soon.
I ran into that some time back (mangling issues) and
was one of my motivations for doing this refactoring
earlier in my day-job implementation of JspServlet
(tc3.3). The result of that effort works very well
and I'm borrowing heavily from it for this effort, but
this time I'm trying to put more support for (and
steal ideas from) other adapter options.
to be thorough, I took our worst behaved JSP page,
which happens to be our
home page, and benchmarked it under several engines.
TC33 blew the doors off
the competition, up to levels of 177 concurrent
connections, at which point
the benchmark tool starting melting down.]
That is always good stuff to hear! Costin deserves a
lot of the credit here, since that performance gain is
primarily (probably) coming from his JspInterceptor
implementation. I know we can never get JspServlet to
be completely as fast, but we should be able to get it
much closer than it currently is.
But in any case, should be independent of
everything
else - just a component that may be used by the
"container adapter" classes.
Costin
DIP, the dependencies should be on interfaces, not
on classes.
90% of the time I agree. Some abstractions require
base or abstract classes, though, if part of the
specification fixes actual behavior. This is often
the case when initialization must follow a fixed
pattern (i.e. a certain type of constructor signature
or other life-cycle methods must be supported).
This can sometimes be avoided if the interface is
well-thought out. For example, the problem with the
current Mangler interface is that it does not support
any way to input the core parameters (the jsp URI and
the scratchdir) from which it's method must derive
their data. In other words, we want a factory method
that works like so:
publc static Mangler createMangler(
String uri,String dir){..}
but we need a predictable way to communicate the
params to the created instance. There are three ways
to fix this in order to support abstract creation
through factory methods.
1) rewrite the Mangler interface to have an
init(String,String) method or 'setJspUri(String)' and
'setOutputDir(String)' methods - this would be
optimal, but would break existing implementations of
Mangler.
2) create a default basic or abstract implementation
of Mangler (call it 'JspMangler' or whatever) and give
it a public constructor that takes the params. The
factory method then becomes:
public static JspMangler createJspMangler(
String, String){...}
The problem with this one is that it forces all new
Mangler implementations to subclass JspMangler. This
is not necessarily a bad thing.
3) create a new interface called JspMangler (or
whatever) and adding the missing initialization/setter
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Fri, 30 Mar 2001, Mel Martinez wrote: --- Steve Downey [EMAIL PROTECTED] wrote: I must admit to not being totally sure whether co-opting the current 'Constants' and 'Options' class families is the best naming for this, but they do clearly indicate the difference in scope. I could My preference would be to completely drop the Constants, and keep each "constant" in the class where it belongs ( but this is personal taste ). One big issue is that we should avoid over-engineering this. We need to improve jasper and do a refactoring - as a number of small code moves and interface changes. Then we can evaluate the result and maybe repeat it. The goal is to simplify and modularize, and IMHO the only way that can be done right is via incremental steps, not a "whaterfall" model ( and I don't want to start another evolution/revolution, XP/waterfall flame war here :-). I would sugest to move to a vote, create the proposal/jasper34, do a first step ( like the JspServlet changes you want, or the new packages - but with minimal interface changes ), evaluate the result and maybe do another iteration. Mel, please don't try to resolve all the problems in one step or as a "block" design ( or a "perfect" initial design ) That is always good stuff to hear! Costin deserves a lot of the credit here, since that performance gain is primarily (probably) coming from his JspInterceptor implementation. I know we can never get JspServlet to Thanks - but I haven't started yet :-) Jsp "real" performance tunning can start only after refactoring. The goal of 3.3 was to make the code simple and modular enough so we can _start_ optimizing it. Costin
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Fri, 30 Mar 2001, Mel Martinez wrote: I don't have the time to do much more than that! That Damn Day Job(tm) keeps getting in the way! :-) Yea, believe me, I know what you mean, you're not the only one :-) Costin
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Wed, 28 Mar 2001, Matthew L Daniel wrote: ( BTW, my interest is more in the jsp-java convertor area, I would be interested to try a more customizable generator that would use XSL templates, but that depends on a modularization and refactoring that would I remember back in the "old days" when the JSP spec actually contained a representation of JSP syntax as XML tags (jsp:expression and whatnot), but Tomcat didn't support it because (as I recall) Sun deprecated it. I'm with Costin, I think that would be/was just super-cool. I just wanted you to know you're not alone, Costin. :-) The XML syntax for JSP pages was not required in JSP 1.1, but is now required in JSP 1.2. Note that this has nothing to do with what happens *inside* of Jasper -- it is only an alternative syntax that is the input to the parser. -- /v\atthew Craig
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
-Original Message- From: Mel Martinez [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 28, 2001 6:41 PM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet Wow! I go away for a day and there is some great discussion on this thread! I've saved everybody's comments and will incorporate them into the proposal, which I will be working on tonight to formalize. When I've got it ready for review, I will put a proposal doc and any related files in html form into .../jakarta-tomcat/proposals/tomcat33/ doing some static analysis, based on class use of other classes, it looks like this constellation of classes could easily be factored out into their own package: org.apache.jasper.compiler.Compiler org.apache.jasper.compiler.Mangler org.apache.jasper.compiler.CommandLineCompiler org.apache.jasper.compiler.JspCompiler org.apache.jasper.compiler.JavaCompiler org.apache.jasper.compiler.JikesJavaCompiler org.apache.jasper.compiler.SunJavaCompiler org.apache.jasper.compiler.JspCompilationContext org.apache.jasper.JspEngineContext org.apache.jasper.CommandLineContext org.apache.jasper.JasperEngineContext Based on actual use, it looks to me like Mangler and Compiler should be merged. There are no instances of Compiler that do not implement the Mangler interface. The classes that implement JspCompilationContext look to me like they don't all belong in the same package, unless everything here is in the same package. They are interface classes between the Jasper compiler and the outside invoker of the compiler. Here are the classes I'm talking about http://www.panix.com/~sdowney/compilers.gif and the entire compiler package http://www.panix.com/~sdowney/org.apache.jasper.compiler.gif (courtesy Together J) This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Thu, 29 Mar 2001, Steve Downey wrote: doing some static analysis, based on class use of other classes, it looks like this constellation of classes could easily be factored out into their own package: org.apache.jasper.compiler.Compiler org.apache.jasper.compiler.Mangler org.apache.jasper.compiler.CommandLineCompiler org.apache.jasper.compiler.JspCompiler org.apache.jasper.compiler.JavaCompiler org.apache.jasper.compiler.JikesJavaCompiler org.apache.jasper.compiler.SunJavaCompiler org.apache.jasper.compiler.JspCompilationContext org.apache.jasper.JspEngineContext org.apache.jasper.CommandLineContext org.apache.jasper.JasperEngineContext Based on actual use, it looks to me like Mangler and Compiler should be merged. There are no instances of Compiler that do not implement the Mangler interface. I don't know - the Mangler has a very nice and clear role - generating correct java package/class name out of some strings. It is quite generic, and independent - I would rather move it in a util package ( i.e. independent of jasper ). I also think that Compiler should _use_ Mangler instead of implementing it ( that's also true for JspEngineContext, and probably few other cases). The java compiler is a nice one ( since ant has a nice collection of compilers ), this should definitely go in a util package and maybe someday we could sync it with the ant javac ( which is far more powerfull IMHO ). And of course we decouple ourself from the javac problems. The classes that implement JspCompilationContext look to me like they don't all belong in the same package, unless everything here is in the same package. They are interface classes between the Jasper compiler and the outside invoker of the compiler. +1 Probably they belong in the same package with the "adapter" ( JspC, JspServlet, JspInterceptor, an eventual JspValve ). I would even split JspCompilationContext based on use ( i.e. find what is used in the java Generator, in Parser, in compiler - simpler and more specialized interfaces would help ). Costin
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Thu, 29 Mar 2001, Mel Martinez wrote: activities that should be orthogonal. The only dependency should be that the Compiler is a consumer of the products of the Mangler. +1 I think the problem starts with the idea to make a JspLoader that 'loads JSP files just as if they were JspLoader is a special thing - it's a great idea ( well, the new model used by JspInterceptor does the same thing in a much cleaner way ). But in any case, should be independent of everything else - just a component that may be used by the "container adapter" classes. Costin classes'. That then broke down into creating a Jsp Compiler that turned JSP files into classes. That finally breaks down when you realize that you have to have an actual javac compiler to turn the .java files into classes. The JspCompilationContext (originally the JspEngineContext) was an attempt to tie all these together with one big payload to carry information around with. But all that information isn't needed in every place and further, some of that information would be better and more efficiently retrieved elsewhere than from the JspCompilationContext. By this I mean that currently the JspCompilationContext provides information relevant to the page life cycle such as the JspFile, the OutputDir, Servlet Class and Package names, etc. etc. - yet JspCompilationContext is a _request_ level object! It is created once for every request! This has got to be refactored. Basically, there are some distinctly separate services used during the jsp page life cycle and the request life cycle that should be factored out and made distinct from each other. More later. I've got to go interview someone. Cheers, Mel The classes that implement JspCompilationContext look to me like they don't all belong in the same package, unless everything here is in the same package. They are interface classes between the Jasper compiler and the outside invoker of the compiler. Here are the classes I'm talking about http://www.panix.com/~sdowney/compilers.gif and the entire compiler package http://www.panix.com/~sdowney/org.apache.jasper.compiler.gif (courtesy Together J) This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail. __ Do You Yahoo!? Get email at your own domain with Yahoo! Mail. http://personal.mail.yahoo.com/?.refer=text
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Thursday, March 29, 2001 3:04 PM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet On Thu, 29 Mar 2001, Mel Martinez wrote: activities that should be orthogonal. The only dependency should be that the Compiler is a consumer of the products of the Mangler. +1 If we treat the embedded implementation of Mangler as a separate class, which I think is a good idea, each style of JSP compilation works with a triad of somewhat dependant classes. An X-Compiler, an X-Context and an X-Mangler. If we're playing games with the name of the generated class, like decorating it with a version number, then some other part of the system needs to understand the Mangling scheme as well. ClassName does some of this work now, finding out what the 'real' name of the class is. Perhaps a Kit pattern is in order? Refactoring Mangler is highly desireable from my day job point of view, too, since the current mangling scheme can cause problems on Windows. Although we might be able to move to Linux for developers desktops soon. [BTW, I have to thank you for the performance work on Tomcat 3.3. We're starting a new strategic cycle, and evaluating what platforms and technologies to persue in the next 12 months. We were preparing arguments about the value access to source of open software, the recruiting value of working with 'cool' technology, career development, and so forth. But just to be thorough, I took our worst behaved JSP page, which happens to be our home page, and benchmarked it under several engines. TC33 blew the doors off the competition, up to levels of 177 concurrent connections, at which point the benchmark tool starting melting down.] I think the problem starts with the idea to make a JspLoader that 'loads JSP files just as if they were JspLoader is a special thing - it's a great idea ( well, the new model used by JspInterceptor does the same thing in a much cleaner way ). But in any case, should be independent of everything else - just a component that may be used by the "container adapter" classes. Costin DIP, the dependencies should be on interfaces, not on classes. -SMD -- Someone else added this stuff g: This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
Hi Steve! Steve Downey wrote: Perhaps a Kit pattern is in order? Wow, a Kit pattern. I never heard of that one (or never got that far in the Patterns books :) Is it a standard one? If so, I'll check it out later at home. Un saludo, Alex.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] So far jasper has been one of the most stable pieces on tomcat ( most bugs I know about are related with the interfacing between jasper and the container ). And it has a huge potential for performance improvement - if we do the right refactoring and provide the right internal APIs. Costin The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
Yes - from my experience, this was the problem deploying Jasper in another container. Tim Julien HP Middleware -Original Message- From: Steve Downey [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 28, 2001 11:33 AM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] So far jasper has been one of the most stable pieces on tomcat ( most bugs I know about are related with the interfacing between jasper and the container ). And it has a huge potential for performance improvement - if we do the right refactoring and provide the right internal APIs. Costin The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Wed, 28 Mar 2001, Steve Downey wrote: The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. I would say this is also part of interfacing the jasper with the container, and not core to the jsp parser and jsp-java convertor ( which in my opinion represents the "core" of jasper ). There is the problem of importing jsps ( and few other cases where files are needed), but again that's related with the API that is needed to plug jasper into a ( cooperating ) container. Costin ( BTW, my interest is more in the jsp-java convertor area, I would be interested to try a more customizable generator that would use XSL templates, but that depends on a modularization and refactoring that would make the generator a clear and replaceable component. This is a long-term plan, of course - some people will kill me if I don't start working on charsets and encodings )
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 28, 2001 3:46 PM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet On Wed, 28 Mar 2001, Steve Downey wrote: The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. I would say this is also part of interfacing the jasper with the container, and not core to the jsp parser and jsp-java convertor ( which in my opinion represents the "core" of jasper ). There is the problem of importing jsps ( and few other cases where files are needed), but again that's related with the API that is needed to plug jasper into a ( cooperating ) container. SNIP ... importing jsp text, determining the file to write java to, finding tag library descriptors, finding the directory to direct the java compiler to write class files to... Files are inherent, at least as long as javac insists on having files. I would love to be able to compile a stream, or array of streams, but that's not happening near term. Almost everything else uses URI's to refer to things. So, for the most part, jasper should not be looking at, or using, files. Streams, yes, as in getResourceAsStream, but mostly not Files. Most current uses of File are wrong, and other parts of Jasper try to compensate. Actually, I don't think that importing needs Files. It should be able to use getResourceAsStream. That may need to work with Files, but that's something that the container deals with, not Jasper. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
Wow! I go away for a day and there is some great discussion on this thread! I've saved everybody's comments and will incorporate them into the proposal, which I will be working on tonight to formalize. When I've got it ready for review, I will put a proposal doc and any related files in html form into .../jakarta-tomcat/proposals/tomcat33/ From the discussion I think it is safe to say that the goals of this refactoring are: 1) focused primarily with cleaning up the object model of the org.apache.jasper.servlet package (i.e. JspServlet) so that we can reduce coupling between JspServlet and the core of Jasper. 2) we will try to increase separation between the JspServlet entry point into jasper from the various life-cycle services (class loading, mangling, compiling, etc) to make those services both replaceable for JspServlet and also reusable for alternate entry points, such as JspC or JspInterceptor. 3) we will try to clarify classpath and init-params configuration issues. 4) despite all this, we will strive to improve performance by reducing the call stack and by utilizing caching where possible. 5) fix more bugs than we introduce. I should have a first-draft class diagram and probably initial java code ready for the proposal tonight or early tomorrow. I tentatively propose to introduce the changes through the package name 'org.apache.jasper.servlet33', unless anyone objects AND has a better package name. :-) Cheers, Dr. Mel Martinez G1440, Inc. --- Steve Downey [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 28, 2001 3:46 PM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet On Wed, 28 Mar 2001, Steve Downey wrote: The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. I would say this is also part of interfacing the jasper with the container, and not core to the jsp parser and jsp-java convertor ( which in my opinion represents the "core" of jasper ). There is the problem of importing jsps ( and few other cases where files are needed), but again that's related with the API that is needed to plug jasper into a ( cooperating ) container. SNIP ... importing jsp text, determining the file to write java to, finding tag library descriptors, finding the directory to direct the java compiler to write class files to... Files are inherent, at least as long as javac insists on having files. I would love to be able to compile a stream, or array of streams, but that's not happening near term. Almost everything else uses URI's to refer to things. So, for the most part, jasper should not be looking at, or using, files. Streams, yes, as in getResourceAsStream, but mostly not Files. Most current uses of File are wrong, and other parts of Jasper try to compensate. Actually, I don't think that importing needs Files. It should be able to use getResourceAsStream. That may need to work with Files, but that's something that the container deals with, not Jasper. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail. __ Do You Yahoo!? Get email at your own domain with Yahoo! Mail. http://personal.mail.yahoo.com/?.refer=text
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
Mel, Regarding the names and the scope of the changes. If we are going to do some refactoring ( and that's going to require work and testing anyway), then we should do it on a slightly broader scope, not only local to the servlet interface. Instead of org.apache.jasper.servlet33, I would suggest something like org.apache.jasperX.*, and have separate packages for the different components that we identify ( see my proposal for 5 components - some of them like the java-class compiler or even the mangler are general-purpose for any scripting system and could even be part of the an util package ). What would be really great is to coordinate this with someone on jasper4.0 side - we should be able to share most components, except probably few classes in the JSP generator ( and of course the runtime - which implements security and is heavily dependent on jsp1.2 ). Most of the parser and generator is identical, and I think we would all benefit by sharing as much as is possible. In other words - it would be great to try to make this proposal not limited to jasper33 servlet, but a jasper7.3 ( 4.0 + 3.3 ) proposal, refactoring and combining the 2 ( now divergent ) versions ( while keeping and increasing the ability to have alternative implementations for all components - like the runtime and some of the generator ). Of course, it's your choice - I will support a limited refactoring if that's what you want, but I think we should at least try to combine our efforts and re-unite ( at least ) jasper. Then all the optimizations ( like tag pooling ) and fixes and the sandboxing will be common for most part, with clear container-specific modules ( that would also prove and show how other containers can use jasper ), Costin On Wed, 28 Mar 2001, Mel Martinez wrote: Wow! I go away for a day and there is some great discussion on this thread! I've saved everybody's comments and will incorporate them into the proposal, which I will be working on tonight to formalize. When I've got it ready for review, I will put a proposal doc and any related files in html form into .../jakarta-tomcat/proposals/tomcat33/ From the discussion I think it is safe to say that the goals of this refactoring are: 1) focused primarily with cleaning up the object model of the org.apache.jasper.servlet package (i.e. JspServlet) so that we can reduce coupling between JspServlet and the core of Jasper. 2) we will try to increase separation between the JspServlet entry point into jasper from the various life-cycle services (class loading, mangling, compiling, etc) to make those services both replaceable for JspServlet and also reusable for alternate entry points, such as JspC or JspInterceptor. 3) we will try to clarify classpath and init-params configuration issues. 4) despite all this, we will strive to improve performance by reducing the call stack and by utilizing caching where possible. 5) fix more bugs than we introduce. I should have a first-draft class diagram and probably initial java code ready for the proposal tonight or early tomorrow. I tentatively propose to introduce the changes through the package name 'org.apache.jasper.servlet33', unless anyone objects AND has a better package name. :-) Cheers, Dr. Mel Martinez G1440, Inc. --- Steve Downey [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 28, 2001 3:46 PM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet On Wed, 28 Mar 2001, Steve Downey wrote: The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. I would say this is also part of interfacing the jasper with the container, and not core to the jsp parser and jsp-java convertor ( which in my opinion represents the "core" of jasper ). There is the problem of importing jsps ( and few other cases where files are needed), but again that's related with the API that is needed to plug jasper into a ( cooperating ) container. SNIP ... importing jsp text, determining the file to write java to, finding tag library descriptors, finding the directory to direct the java compiler to write class files to... Files are inherent, at least as long as javac insists on having files. I would love to be able to compile a stream, or array of streams, but that's not happening near term. Almost everything else uses URI's to refer to things. So, for the most part, jasper should not be looking at, or using, files. Streams, yes, as in getResourceAsStream, but mostly not F
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
( BTW, my interest is more in the jsp-java convertor area, I would be interested to try a more customizable generator that would use XSL templates, but that depends on a modularization and refactoring that would I remember back in the "old days" when the JSP spec actually contained a representation of JSP syntax as XML tags (jsp:expression and whatnot), but Tomcat didn't support it because (as I recall) Sun deprecated it. I'm with Costin, I think that would be/was just super-cool. I just wanted you to know you're not alone, Costin. :-) -- /v\atthew -- Matthew L DanielNever put off until tomorrow what you can do today. Wizard There might be a law against it by that time. [EMAIL PROTECTED]-- `/usr/bin/fortune`
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Wed, 28 Mar 2001, Matthew L Daniel wrote: ( BTW, my interest is more in the jsp-java convertor area, I would be interested to try a more customizable generator that would use XSL templates, but that depends on a modularization and refactoring that would I remember back in the "old days" when the JSP spec actually contained a representation of JSP syntax as XML tags (jsp:expression and whatnot), but Tomcat didn't support it because (as I recall) Sun deprecated it. I'm with Costin, I think that would be/was just super-cool. The XML syntax is part of JSP1.2, but the JSP parser can easily generate SAX events from a "normal" JSP that match the 1.2 XML representation. Just to make clear - this is more of an experimental/cool/maybe project, and a refactoring that would allow other generator modules to be plugged in would make possible to do this kind of experiments while sharing the same infrastructure and without hurting jasper's stability ( i.e. I'm not proposing to replace the current stable code with some cool stuff - just to make things more modular so experiments in code generation can happen). I think Mel's original proposal had this right - developing the new JspServlet interface along with the old one, and allowing the user to use either ( until the new one is clearly ready and we can deprecate the old one ). Same should happen in the generator, compiler and for all other components of jasper. And the good news is that the refactoring shouldn't be very difficult - the most important thing is to agree on the components, move the code in new packages and then try to simplify the interfaces. After that we can test and make sure we still have the good jasper, and start playing with new JspServlet, parsers, generators, manglers, etc. Costin
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
I acknowledge your suggestion and am going to put it in org.apache.jasper34.* That symbolically combines 3 and 4 while deferring more towards an incremental change from 3.3, which is the main focus. I think your other points are well put and that (commonality with tc4.0 jasper) will be a motivator and influence, but the primary goal should and will be to simply improve what is in tc3.3 jasper. Cheers, mel --- [EMAIL PROTECTED] wrote: Mel, Regarding the names and the scope of the changes. If we are going to do some refactoring ( and that's going to require work and testing anyway), then we should do it on a slightly broader scope, not only local to the servlet interface. Instead of org.apache.jasper.servlet33, I would suggest something like org.apache.jasperX.*, and have separate packages for the different components that we identify ( see my proposal for 5 components - some of them like the java-class compiler or even the mangler are general-purpose for any scripting system and could even be part of the an util package ). What would be really great is to coordinate this with someone on jasper4.0 side - we should be able to share most components, except probably few classes in the JSP generator ( and of course the runtime - which implements security and is heavily dependent on jsp1.2 ). Most of the parser and generator is identical, and I think we would all benefit by sharing as much as is possible. In other words - it would be great to try to make this proposal not limited to jasper33 servlet, but a jasper7.3 ( 4.0 + 3.3 ) proposal, refactoring and combining the 2 ( now divergent ) versions ( while keeping and increasing the ability to have alternative implementations for all components - like the runtime and some of the generator ). Of course, it's your choice - I will support a limited refactoring if that's what you want, but I think we should at least try to combine our efforts and re-unite ( at least ) jasper. Then all the optimizations ( like tag pooling ) and fixes and the sandboxing will be common for most part, with clear container-specific modules ( that would also prove and show how other containers can use jasper ), Costin On Wed, 28 Mar 2001, Mel Martinez wrote: Wow! I go away for a day and there is some great discussion on this thread! I've saved everybody's comments and will incorporate them into the proposal, which I will be working on tonight to formalize. When I've got it ready for review, I will put a proposal doc and any related files in html form into .../jakarta-tomcat/proposals/tomcat33/ From the discussion I think it is safe to say that the goals of this refactoring are: 1) focused primarily with cleaning up the object model of the org.apache.jasper.servlet package (i.e. JspServlet) so that we can reduce coupling between JspServlet and the core of Jasper. 2) we will try to increase separation between the JspServlet entry point into jasper from the various life-cycle services (class loading, mangling, compiling, etc) to make those services both replaceable for JspServlet and also reusable for alternate entry points, such as JspC or JspInterceptor. 3) we will try to clarify classpath and init-params configuration issues. 4) despite all this, we will strive to improve performance by reducing the call stack and by utilizing caching where possible. 5) fix more bugs than we introduce. I should have a first-draft class diagram and probably initial java code ready for the proposal tonight or early tomorrow. I tentatively propose to introduce the changes through the package name 'org.apache.jasper.servlet33', unless anyone objects AND has a better package name. :-) Cheers, Dr. Mel Martinez G1440, Inc. --- Steve Downey [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Wednesday, March 28, 2001 3:46 PM To: [EMAIL PROTECTED] Subject: RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet On Wed, 28 Mar 2001, Steve Downey wrote: The second most common cause of bugs in Jasper is confusion over when to use File.separator and when to use '/'. It's hard to keep track of, since Jasper does deal with both files and URIs. And the File methods are used to regularize some URIs. Another reason to refactor. I would say this is also part of interfacing the jasper with the container, and not core to the jsp parser and jsp-java convertor ( which in my opinion represents the "core" of jasper ). There is the problem of importing jsps ( and few other cases where files are needed), but again that's related with the API that is needed to plug
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
--- Steve Downey [EMAIL PROTECTED] wrote: -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Monday, March 26, 2001 1:08 PM To: [EMAIL PROTECTED] Subject: Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet Hi Mel, In my view, jasper is composed from at least 5 big components: 1. The jsp-java translator. 2. The java-class compiler 3. The Mangler ( managing name mappings ) 4. Runtime - that should be completely independent of all other pieces, since jasper-generated servlets should run without jasper ( as simple servlets ) 5. Interface with the serlvet container ( JspServlet, JspInterceptor and the associated classes). ( putting all other components togheter ) Layer 5 has to include the JspC/CommandLineXXX components, also. They have to provide an environment that looks to the rest of the JSP compiler like a web context, while actually interacting more closely with the filesystem. I'm not going to make it a primary goal to fix JspC, but I should be able to make the components reusable so that a new version of JspC could be refactored to use the same classes as this new version of JspServlet. Mangler needs some work, too. JspC and JspServlet have drifted in their implementations. The bane of code sharing by cut and paste. I will try to address this as well. Mel __ Do You Yahoo!? Get email at your own domain with Yahoo! Mail. http://personal.mail.yahoo.com/?.refer=text
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
--- Glenn Nielsen [EMAIL PROTECTED] wrote: I have made some changes to the Jasper code in Tomcat 4 that you might want to look at. I will definitely be looking at TC 4. 1. In general the Java SecurityManager implemenation in Tomcat 4 and Jasper has significant improvements and is much cleaner. I copied some of this in my current implementation. Shoudn't be too hard to keep it in when I port. 2. Jasper class loading is much simpler in the Tomcat 4 version. It uses a single URLClassLoader for each JSP page, this allowed me to simplify a great deal how Jasper handles generation of class and package names (no more need to do incremental .java and .class files). And it removed alot of overhead at both JSP translation and runtime when Jasper tries to figure out which incremental class file to use/check. This sped up JSP compiles by 33% and runtime execution by 25%. The refactoring I propose would indeed use one class-loader per page. My current implementation does this leveraging a subclass of JasperLoader but I'll explore using URLClassLoader. I would need to make sure that it is compatible with jdk1.1 (a tc3.3 requirement) so if I used URLClassLoader, I'd need to provide a jdk1.1 replacement ala the way tomcat startup does. This actually should be pretty easy since I'm getting the loader via a factory method. And before you ask, I just don't have the time to back port these changes in TC3.3. Don't worry - I totally understand this! :-) mel __ Do You Yahoo!? Get email at your own domain with Yahoo! Mail. http://personal.mail.yahoo.com/?.refer=text
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
1) I don't off-hand know of any other generalized way to make a portable JSP compiler that can be plugged into any servlet 2.2 engine other than as a servlet. I'm not sure how that limits us feature-wise, other than the fact that it adds a layer of indirection between the request on the container and the actual ultimate jsp servlet. A small performance price to pay for portability. The only problem is that the price is much higher that that: - JspServlet must manage the servlet lifecycle - the container can no longer treat jsp-generate-servlets the same as regular servlets - initialization - how can you pass init parameters to the jsp ? This is one of the worst hacks and source of counteless problems ( AFAIK - I couldn't find any clean way to do that ) - class loader problems - again, the JspServlet must duplicate and override the class loading decisions of the container - other problems and issues My point is that the servlet abstraction may not provide enough API support for what jasper need - in order to make it work you have to duplicate services that should be ofered by containers ( class loading, init, reloading, admin, etc ). I don't want to discourage you from doing that - any refactoring will help and we badly need that. I just want to give you an alternate view, based on my experience - before you repeat mistakes we've already done :-) 2) One major point of the refactoring is to minimize the role that the JspServlet plays. In the model I'm advocating it only does a couple of things: initializes jasper (i.e., creates the JspFactory) and maintain a cache of JspPageHandlers to which it passes the requests. Given this, the same role could be played by other entry mechanisms. +1 Costin
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
--- [EMAIL PROTECTED] wrote: 1) I don't off-hand know of any other generalized way to make a portable JSP compiler that can be plugged into any servlet 2.2 engine other than as a servlet. I'm not sure how that limits us feature-wise, other than the fact that it adds a layer of indirection between the request on the container and the actual ultimate jsp servlet. A small performance price to pay for portability. The only problem is that the price is much higher that that: - JspServlet must manage the servlet lifecycle - the container can no longer treat jsp-generate-servlets the same as regular servlets This is a limitation of the servlet api and thus the lack of access to a consistent set of servlet engine features necessary for full servlet life-cycle management that can be relied upon. Besides, who said that the container was guaranteed full-lifecycle management of ANY servlet? Any servlet could be created/loaded dynamically and managed by some other servlet without the container ever knowing about it (assuming the SecurityManager allowed it, of course :-). - initialization - how can you pass init parameters to the jsp ? This is one of the worst hacks and source of counteless problems ( AFAIK - I couldn't find any clean way to do that ) How exactly do you solve this with JspInterceptor? The specification doesn't really seem to cover init-params for individual JSP pages. The JSP container has no responsibility to provide individualized init-params for JSP servlets, only to invoke jspInit() prior to service(). In the absence of a specified mechanism, it is the responsibility of the JSP page author to initialize his page by implementing the jspInit() method (from which he/she can optionally load params from whereever he/she sees fit). In other words, this is not a limitation of JspServlet, but rather a limitation of the JSP specification. - class loader problems - again, the JspServlet must duplicate and override the class loading decisions of the container - other problems and issues This, again is more a limitation (or too much freedom) in how servlet containers are implemented. My point is that the servlet abstraction may not provide enough API support for what jasper need - in order to make it work you have to duplicate services that should be ofered by containers ( class loading, init, reloading, admin, etc ). The glass is half-full as well - because at least with the servlet api it is _possible_ to implement those services portably and thus provide a portable JSP compiler. And yes, it would be great if the JspServlet didn't have to implement redundant container services, that in no way really limits JspServlet's ability to fullfill all JSP specification features. And again - what is the alternative? I can't write a JspInterceptor solution for WebLogic or JRun. I don't want to discourage you from doing that - any refactoring will help and we badly need that. I just want to give you an alternate view, based on my experience - before you repeat mistakes we've already done :-) The mistakes aren't in using a servlet-based approach. This sort of feature (a plug-in to provid jsp compilation and loading services to a web server) is exactly what servlets are for. This in itself in no way limits what you can do with JSPs because all features of the JSP spec should be attainable through a servlet implementation. This DOES limit the reliance Jasper can have on value-added container features beyond the servlet API, but hey, you have to draw the line somewhere. The way to look at it is that a JspInterceptor-style solution is a value-added feature of the tomcat servlet engine. That is a positive. The portability of JspServlet is also a positive. Cheers, Mel __ Do You Yahoo!? Get email at your own domain with Yahoo! Mail. http://personal.mail.yahoo.com/?.refer=text
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
- JspServlet must manage the servlet lifecycle - the container can no longer treat jsp-generate-servlets the same as regular servlets This is a limitation of the servlet api and thus the lack of access to a consistent set of servlet engine features necessary for full servlet life-cycle management that can be relied upon. Besides, who said that the container was guaranteed full-lifecycle management of ANY servlet? Any servlet could be created/loaded dynamically and managed by some other servlet without the container ever knowing about it (assuming the SecurityManager allowed it, of course :-). True, but a container can be smart ( and unload unused servlets, provide nice monitoring, etc ). - initialization - how can you pass init parameters to the jsp ? This is one of the worst hacks and source of counteless problems ( AFAIK - I couldn't find any clean way to do that ) How exactly do you solve this with JspInterceptor? A JSP is treated the same way as a servlet - it gets a ServletConfig with the right info from web.xml, etc. There is no distinction between the way a regular servlet is treated and a jasper-generated servlet. The specification doesn't really seem to cover init-params for individual JSP pages. The JSP ??? servletjsp-file/foo.jsp/jsp-file init-paramnamefoo/name.. AFAIK you should be able to define init params for jsps the same as you do for servlets ( unless I'm missing something here). container has no responsibility to provide individualized init-params for JSP servlets, only to invoke jspInit() prior to service(). In the absence I thought it does... Even tought that never worked very well :-) The glass is half-full as well - because at least with the servlet api it is _possible_ to implement those services portably and thus provide a portable JSP compiler. Almost everything is possible ( in a turing machine :-). The problem is at what cost, and if that's the best solution. And again - what is the alternative? I can't write a JspInterceptor solution for WebLogic or JRun. Unfortunately that's true - and it's a good reason to have JspServlet. But if you have control over the servlet container ( for example using an internal extension mechanism - like the interceptor in tc3 or valve in tc4 ) it would be much better to use a rich API that allows full control over jasper, and lets the container do what it is supposed to do - manage servlets. The mistakes aren't in using a servlet-based approach. This sort of feature (a plug-in to provid jsp compilation and loading services to a web server) is exactly what servlets are for. This in itself in no way limits what you can do with JSPs because all features of the JSP spec should be attainable through a servlet implementation. I'm still looking for a solution to deal with the init-param ( except for re-parsing web.xml ) I'm not saying it's a mistake to have a servlet-based approach, but that having a full API and letting the container manager servlets and jasper do the compilation could be a better solution. The way to look at it is that a JspInterceptor-style solution is a value-added feature of the tomcat servlet engine. That is a positive. The portability of JspServlet is also a positive. +1 ( and to my defense - I did the fixes to allow JspServlet to be still be used in tomcat 3.3, for people who want a slower jsp :-) Mel, what I'm trying to explain is not that JspServlet is bad, but that we need a refactoring to expose more if Jasper API and cleanly separate the components - so we can use JspServlet and interceptors ( or valves ). A lot of the code in jasper is written with the assumption that JspServlet is the entry point ( i.e. with workarounds and hacks for this case ) - and then we have JspC and JspInterceptor that adds another round of hacks and workarounds. And it would help a lot to keep this in mind while doing the refactoring - i.e. to distinguish between what's jasper's core functionality ( convert a .jsp file to a java file ) and what's a hack and duplication of container's functionality for JspServlet's use. For example JspLoader, some of the manglers, some of the JspEngine and context. If we are going for a refactoring, it would be a good idea to clean up and fix more than just JspServlet. Costin
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Tue, 27 Mar 2001 [EMAIL PROTECTED] wrote: - initialization - how can you pass init parameters to the jsp ? This is one of the worst hacks and source of counteless problems ( AFAIK - I couldn't find any clean way to do that ) From the application developer's perspective, you do this: servlet servlet-nameMyJspPageServlet/servlet-name jsp-file/path/to/myjsppage.jsp/jsp-file init-param param-namename/param-name param-valuevalue/param-value /init-param /servlet The approach to implementing this in Catalina is also pretty ugly, but it does work ... Craig
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Tue, 27 Mar 2001, Mel Martinez wrote: The glass is half-full as well - because at least with the servlet api it is _possible_ to implement those services portably and thus provide a portable JSP compiler. When Tomcat 4.0 was created, one of the goals was to eliminate all the Jasper dependencies on servlet container code (with the goal of a portable JSP environment). We were completely successful except for two (initially) and now only one issue: specifying a classpath for the Java compiler. In a JDK 1.1 environment, there is no portable solution to this problem. In JDK 1.2 or later, you can have a portable solution as long as the web app classloader is a subclass of URLClassLoader (as it is in Tomcat 4.0), because you can ask the URLClassLoader what it's repositories are and therefore reconstruct a reasonable class path in most circumstances. The portability goal is definitely achievable in a Java2 world. Craig
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Tue, 27 Mar 2001, Craig R. McClanahan wrote: The glass is half-full as well - because at least with the servlet api it is _possible_ to implement those services portably and thus provide a portable JSP compiler. When Tomcat 4.0 was created, one of the goals was to eliminate all the Jasper dependencies on servlet container code (with the goal of a portable JSP environment). We were completely successful except for two (initially) and now only one issue: specifying a classpath for the Java compiler. ... The portability goal is definitely achievable in a Java2 world. The only problem is that maybe trying to make jasper independent of the container is the wrong goal... The price is ugly ( or at least complex ) and inefficient code ( not only because of the overhead, but also because of the missed oportunities for optimizations ). A quick test is very simple - in 3.3 compare the speed of running jsps via JspServlet and JspInterceptor. ( or with the JspServlet in 4.0 if you think the 3.x JspServlet is bad ). I'm not saying that having a JSP implementation packaged as a servlet and reimplementing functions of the servlet container is bad or impossible - just that a solution where the container does servlet management and jasper deals with converting jsps to servlets may result in cleaner code and better performance. I don't remember reading anywhere in the JSP spec that jasper must be implemented as a servlet, or that it must be independent of the servlet container. And I don't think having a good design for jasper's API that allows it to take advantage of the container would make it less portable - and JspServlet can still be used to plug jasper in un-cooperating containers. So far jasper has been one of the most stable pieces on tomcat ( most bugs I know about are related with the interfacing between jasper and the container ). And it has a huge potential for performance improvement - if we do the right refactoring and provide the right internal APIs. Costin
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
Hi Mel, In my view, jasper is composed from at least 5 big components: 1. The jsp-java translator. 2. The java-class compiler 3. The Mangler ( managing name mappings ) 4. Runtime - that should be completely independent of all other pieces, since jasper-generated servlets should run without jasper ( as simple servlets ) 5. Interface with the serlvet container ( JspServlet, JspInterceptor and the associated classes). ( putting all other components togheter ) My understanding is that your proposal is related with (5), and it seems it has the great property that it can be done as a proper refactoring - without chaning any functionality, just by providing better communication and code organization ( as the first stage ), and then by creating one module ( that will eventually replace JspServlet ). Since this is my favorite "modus operandis" I can't say anything than a big +1... There are few issues: - Impact on 3.3 release cycle. I hate delaying it - it's clear we need another milestone, but I believe in the "release early and often" ( and on schedule :-). I tried very hard to decouple the components as much as possible, so development on any component shouldn't affect the overall release ( and the rest of the code ). That would be resolved by your proposal to use a separate package name - the new ( "in development mode" ) code can be developed in a proposal space and released separately and be included in a 3.3.1 for example. I think keeping "old, stable" code in paralel with "new, better" implementations and doing a gradual replacement is a very good strategy ( AJP1.1 - AJP1.2 - Ajp1.3 - Ajp1.4, mod_jserv - mod_jk - mod_jk+webapp, facade22 - facade23, etc ). - The other 4 components of jasper are also in a big need for a refactoring. I think we'll see a lot of improvements in jsp-java generation, and it would help a lot to be able to modularize it even more. We clearly need a refactoring to allow you to plug the new JspServlet, and we should extend it to at least the point where we cleanly separate the components. - Jasper 4. This will increase the divergence between jasper3 and jasper4. I spent some time trying to find a way to merge the 2, and I gave up because of the current complexity. If a similar refactoring is done in both sides ( since at least 90% of the code is still common ), and as result of refactoring we can merge the common code in a set of common components - then we might get more people involved and avoid a lot of time waste. - Class loading and other interfacing problems. As I said many times, I don't think JspServlet is the right way to plug jasper in a container, but a richer interface exposing more of jasper. Having a working JspServlet for quick-plugin is great, but I think we should rather focus on keeping it just a small facade to a better designed and more powerfull internal API. In any case, a refactoring can only help, and you have my +1 ( i.e. I think it's a good idea and I'll help !). ( BTW, I'm looking into an alternative/experimental implementation for the jsp-java component, probably after 3.3 - as a standalone add-on module. I have few ideas - but I want to first do a prototype ) Costin On Mon, 26 Mar 2001, Mel Martinez wrote: As hinted at last week, I'd like to propose refactoring some of the classes in Jasper to improve the OO model a bit, make maintenance/extendability a bit easier and hopefully help the performance a bit as well for those of us using jasper as the JSP engine in other servlet engines (other than tomcat, that is). There are problems with using the current Jasper in other ServletEngines that do not show up with the tomcat JspIntercepter that need to be addressed. I'd like to refactor JspServlet, JspServletWrapper, JasperLoader, JasperEngineContext and JasperCompiler to better divide responsibilities along the following lines: JspServlet - entry point for JSP page (*.jsp), initialize Jasper and maintain cache of jsp servlet wrapper/handlers, otherwise do very little but pass on the request to the handler. I.E. represents the scope of the jasper engine. Possibly rename this to JasperServlet or just 'Jasper'. JspServletWrapper - rename as 'JspServlet/PageHandler' (or similar) and make this a full class (i.e. take it out of JspServlet). This class manages the life cycle of a single Jsp file, including methods to resolve relevant file paths (i.e. to .jsp file, .java file, .class file), package/class name mangling and modification checking. This class would be responsible for managing dynamic reloading of an out-of-date page. Only one instance of this class is created per page for the life JVM (or more accurately, for the life of the JspServlet). JspRequestWrapper/Handler/Context - encapsulate information relevant to one request - this would implement JspCompilationContext, since a compilation would be one possible event during the request. However, we'd remove some of the
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Sent: Monday, March 26, 2001 1:08 PM To: [EMAIL PROTECTED] Subject: Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet Hi Mel, In my view, jasper is composed from at least 5 big components: 1. The jsp-java translator. 2. The java-class compiler 3. The Mangler ( managing name mappings ) 4. Runtime - that should be completely independent of all other pieces, since jasper-generated servlets should run without jasper ( as simple servlets ) 5. Interface with the serlvet container ( JspServlet, JspInterceptor and the associated classes). ( putting all other components togheter ) Layer 5 has to include the JspC/CommandLineXXX components, also. They have to provide an environment that looks to the rest of the JSP compiler like a web context, while actually interacting more closely with the filesystem. Mangler needs some work, too. JspC and JspServlet have drifted in their implementations. The bane of code sharing by cut and paste. This electronic mail transmission may contain confidential information and is intended only for the person(s) named. Any use, copying or disclosure by any other person is strictly prohibited. If you have received this transmission in error, please notify the sender via e-mail.
Re: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
1. In general the Java SecurityManager implemenation in Tomcat 4 and Jasper has significant improvements and is much cleaner. True. The SecurityManager in 3.3 is working fine for now ( Glenn is also the main author for the 3.x sandboxing ), with all watchdog passing - but a refactoring will help cleaning up the code. 2. Jasper class loading is much simpler in the Tomcat 4 version. It uses a single URLClassLoader for each JSP page, this allowed me to simplify a great deal how Jasper handles generation of class and package names (no more need to do incremental .java and .class files). And it removed alot of overhead at both JSP translation and runtime when Jasper tries to figure out which incremental class file to use/check. This sped up JSP compiles by 33% and runtime execution by 25%. Well, I have to disagree here :-) It is indeed much simpler than the 3.2 jsp class loading, and using URLClassLoader is a step in the right direction. Tomcat 3.3 is also using URLClassLoader ( for everything - except the JDK1.1 replacement that has the same interface and replicates URLClassLoader ). I would say the incremental .java is not a bad idea, and the code to support that is reasonably simple ( and provide some unique benefits related with reloading and avoiding some special cases ) - it should be trivial to make it optional ( if anyone feel the need ). We don't use a class loader per jsp page ( since it isn't needed - given the versioning ), but again it would be easy to add it. In any case - at least the class loading is just fine in 3.3, no need to fix it ( but I think everyone would be happy to have more options - like what you mention about one class loader per jsp and no versioning - as long as this is not mandatory and doesn't replace the other scheme ). And before you ask, I just don't have the time to back port these changes in TC3.3. Well, it would help a lot if you could at least take a look and review the sandboxing for 3.3 :-) Security and sandboxing are very difficult, and you probably have the most experience with that. On a related issue, I would be very interested in re-commiting some of the original patches ( defining the permissions in server.xml in addition to the policy file ), this would help a lot in the future admin interface. Again, we might need your help ( if you still have them around ) :-) Costin Regards, Glenn Mel Martinez wrote: As hinted at last week, I'd like to propose refactoring some of the classes in Jasper to improve the OO model a bit, make maintenance/extendability a bit easier and hopefully help the performance a bit as well for those of us using jasper as the JSP engine in other servlet engines (other than tomcat, that is). There are problems with using the current Jasper in other ServletEngines that do not show up with the tomcat JspIntercepter that need to be addressed. I'd like to refactor JspServlet, JspServletWrapper, JasperLoader, JasperEngineContext and JasperCompiler to better divide responsibilities along the following lines: JspServlet - entry point for JSP page (*.jsp), initialize Jasper and maintain cache of jsp servlet wrapper/handlers, otherwise do very little but pass on the request to the handler. I.E. represents the scope of the jasper engine. Possibly rename this to JasperServlet or just 'Jasper'. JspServletWrapper - rename as 'JspServlet/PageHandler' (or similar) and make this a full class (i.e. take it out of JspServlet). This class manages the life cycle of a single Jsp file, including methods to resolve relevant file paths (i.e. to .jsp file, .java file, .class file), package/class name mangling and modification checking. This class would be responsible for managing dynamic reloading of an out-of-date page. Only one instance of this class is created per page for the life JVM (or more accurately, for the life of the JspServlet). JspRequestWrapper/Handler/Context - encapsulate information relevant to one request - this would implement JspCompilationContext, since a compilation would be one possible event during the request. However, we'd remove some of the current behavior in JasperEngineContext that is really page-lifecycle behavior that belongs in the JspServlet/PageHandler. This class should be a lightweight information payload used by the page handler, loader and compiler for fullfillment of the request. By reducing the number of fields to only those that are request-specific and possibly by object pooling, the impact of instance creation (necessary for each request) can be greatly reduced. JasperLoader, Compiler, and a few other classes would be refactored (actually simply extended slightly through subclassing) to support these mods. Some of the current behavior would be refactored out to the page life-cycle handler. Compilers can probably be pooled for reuse. Along with the
RE: TC3.3 Proposal: Refactoring org.apache.jasper.servlet
On Mon, 26 Mar 2001, Steve Downey wrote: 1. The jsp-java translator. 2. The java-class compiler 3. The Mangler ( managing name mappings ) 4. Runtime - that should be completely independent of all other pieces, since jasper-generated servlets should run without jasper ( as simple servlets ) 5. Interface with the serlvet container ( JspServlet, JspInterceptor and the associated classes). ( putting all other components togheter ) Layer 5 has to include the JspC/CommandLineXXX components, also. They have to provide an environment that looks to the rest of the JSP compiler like a web context, while actually interacting more closely with the filesystem. Yes. In fact a lot from JspInterceptor is inspired/derived from JspC. But the JspC is very special because it can be use to pre-compile applications - that would be great for special configurations or only the jsp runtime is included ( say an embeded tomcat, or a minimal environment). It should be possible ( after a bit of refactoring ) to create a mini distribution using only the core, a sub-set of the modules, jasper-runtime ( no xml parser or jasper converter or javac ), plus JspC-processed webapplications ( what's missing is a piece to serialize web.xml - for server.xml there is EmbededTomcat, and it shouldn't be difficult to automatically generate a java class that will have the same effect with reading web.xml - maybe using serialization ). I would bet on a result below 500K :-) ( most of the code is easy to port to J2ME, BTW - we just need volunteers :-). Mangler needs some work, too. JspC and JspServlet have drifted in their implementations. The bane of code sharing by cut and paste. Refactoring is the keyword :-) Merge what's duplicated, use the mistakes to define better interfaces between components. The mangler in 3.3 is slightly better ( forgive my lack of modesty here ), and I would also say the JspInterceptor is at least a simpler way to plug jasper ( compared with JspServlet ). Costin
