Re: Question regarding tomcat class path handling
On 1/12/06, Oded Arbel <[EMAIL PROTECTED]> wrote: > On Wednesday, 11 בJanuary 2006 18:38, Sriram Narayanan wrote: > > I wrote something that works for me: > > http://issues.apache.org/bugzilla/show_bug.cgi?id=38223 > > Thanks. I didn't use your implementation for the reason noted in the > comment I attached to the issue, but you pointed me in the right > direction. > > I solved the problem by also implementing a web app class loader and > attached the source to the above issue. > http://issues.apache.org/bugzilla/show_bug.cgi?id=38223 I just added the following functionality: Picks classpath from an XML file You can specify the classpath via an XML file that can be placed anywhere on the file system. The context.xml contains: The XML structure is as follows: file:/d:/temp/bin/ file:/c:/temp/deleteThis/ > Thanks again :-) Let me know if this proves useful to you. > > -- > Oded > -- Sriram
Re: Question regarding tomcat class path handling
On Friday, 13 בJanuary 2006 01:19, [EMAIL PROTECTED] wrote: > On Mon, Jan 09, 2006 at 08:16:00PM +0200, Oded Arbel wrote: > > The problem is like this (the actual system is far more complex): > > - suppose two web applications, app1 and app2, both use some API > > (which I developed myself). > > - I don't want to develop the API twice on both web applications, > > so I have a third project for the API itself, which is a not a web > > application and so isn't in WEB-INF/classes of neither app1 nor > > app2. - I don't want to build a jar for that API library to put in > > app1 and app2's WEB-INF/lib because its also constantly being > > developed - if I wanted to do this then after each commit, in > > addition to the build stage I would have to copy the jar by hand to > > all the applications that use it, which is an error prone process. > > You've got to be kidding. Copy by hand!? Why? Just build the jar > file to some predefined location, and have the build process for each > app copy it from there. There's no need to copy it by hand, have > ant, or whatever you're using to build, do it for you. The build file that I use to build the web app in the development env is the exact same build file that eventually builds it for distribution. The reason this is so, is that everything in the SCM (Software configuration management - CVS and the likes) for the web app is exactly the same as we use for the final build, so that checking out from SCM and running ant are the only steps required to complete a distribution package - that way we get to do all the code testing in the development environment and the QC is used only for integration testing. > That seems much simpler than playing with custom class loaders, or > startup classpath dependencies, or loose class files. On the contrary. due to the reasons noted above, everything that differs in the development env from the QC env has to be done by hand (when I say by hand, I also mean special development env only scripts that are not stored in SCM. Due to the "not stored in SCM" requirement we strive to keep them as simple as possible so we don't need to track them). > e.g. say you've got 4 different api's. When you build those, a jar > file gets created in (e.g.) /myjars: > Let's say app1 uses api's 1, 3 and 4, so it has something like this > somewhere in its build.xml: (assuming you're using ant) > The dependency on the build env being "just right" and custom to each application (i.e. lack of standard deployment techniques) is the major down side with Java development. In order to solve this, the JPackage project has developed some strict conventions as to what goes where, and I'm not in the habit of violating these as keeping up with the conventions is the number one reducer of development and integration time. The JPackage conventions call for all jars to be installed in a specific directory - /usr/share/java - and be sym-linked from there to where you want. the symlinking IS part of the ant build script, but it works because the locations of the 3rd party jars is known ahead of time because they are installed in the same location for all environments. The problem is that development projects are not "installed" as they are in constant development, and everything which is "installed" has to be installed using the platform package management solution. So, the way our system works is like this: Dev has: 3rd party system packages, development projects, custom stuff Build: get development projects, run ant -> create system package QC and Production: 3rd party and our system packages. What way production does not have any custom hand-installed software, only stuff that plays nice with the built-in software management tools. That allows us to have the operating system supports software versioning and software dependencies and unified software distribution in a transparent way across all of the software products we deploy (3rd party or locally developed), the QC is exactly the same so it allows real testing of what its like to be in production, and the build system is a very simple two step process which is easy to monitor and reproduce on a clean system (hacking the build is kept to minimum and everything is in SCM for easy tracking). This causes the development to have some custom stuff, but again - no hacking of anything that is in SCM. -- Oded ::.. "If I have seen farther than other men, it is because I stood on the shoulders of giants." -- Sir Isaac Newton - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On Mon, Jan 09, 2006 at 08:16:00PM +0200, Oded Arbel wrote: > The problem is like this (the actual system is far more complex): > - suppose two web applications, app1 and app2, both use some API (which > I developed myself). > - I don't want to develop the API twice on both web applications, so I > have a third project for the API itself, which is a not a web > application and so isn't in WEB-INF/classes of neither app1 nor app2. > - I don't want to build a jar for that API library to put in app1 and > app2's WEB-INF/lib because its also constantly being developed - if I > wanted to do this then after each commit, in addition to the build > stage I would have to copy the jar by hand to all the applications that > use it, which is an error prone process. You've got to be kidding. Copy by hand!? Why? Just build the jar file to some predefined location, and have the build process for each app copy it from there. There's no need to copy it by hand, have ant, or whatever you're using to build, do it for you. That seems much simpler than playing with custom class loaders, or startup classpath dependencies, or loose class files. e.g. say you've got 4 different api's. When you build those, a jar file gets created in (e.g.) /myjars: /myjars/api1.jar /myjars/api2.jar /myjars/api3.jar /myjars/api4.jar Let's say app1 uses api's 1, 3 and 4, so it has something like this somewhere in its build.xml: (assuming you're using ant) and app2 uses 2 and 4, so it instead has: etc... of course, if you don't want a global location for the api jar files, you can always use a relative path. This is what I do in my app. I've got a directory structure that looks like this: mycode/ myapi/ build.xml (builds the java files and creates myapi.jar) build/myapi.jar myapp/ build.xml (copies ../myapi/build/myapi.jar to build/WEB-INF/lib e.g.: build/WEB-INF/lib/myapi.jar eric - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On 1/10/06, Oded Arbel <[EMAIL PROTECTED]> wrote: > > > > What you could do is to write your own WebAppClassLoader that'd add > > these folders to its lookup list. > > Hmm. interesting. How can I let such a class loader implementation to be > used instead of tomcat's WebAppClassLoader ? > The following links should help http://tomcat.apache.org/tomcat-5.5-doc/config/loader.html http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/loader/WebappLoader.html http://svn.apache.org/repos/asf/tomcat/container/tc5.5.x/catalina/src/share/org/apache/catalina/loader/WebappClassLoader.java -- Sriram - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On Wednesday, 11 בJanuary 2006 18:38, Sriram Narayanan wrote: > I wrote something that works for me: > http://issues.apache.org/bugzilla/show_bug.cgi?id=38223 Thanks. I didn't use your implementation for the reason noted in the comment I attached to the issue, but you pointed me in the right direction. I solved the problem by also implementing a web app class loader and attached the source to the above issue. Thanks again :-) -- Oded ::.. Software: Making your computer come alive so it can attack you -- Dave Barry in Cyberspace (probably using Windows) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On 1/10/06, Sriram Narayanan <[EMAIL PROTECTED]> wrote: > On 1/10/06, Oded Arbel <[EMAIL PROTECTED]> wrote: > > > > > > > What you could do is to write your own WebAppClassLoader that'd add > > > these folders to its lookup list. > > > > Hmm. interesting. How can I let such a class loader implementation to be > > used instead of tomcat's WebAppClassLoader ? > > > > The following links should help > http://tomcat.apache.org/tomcat-5.5-doc/config/loader.html > http://tomcat.apache.org/tomcat-5.5-doc/catalina/docs/api/org/apache/catalina/loader/WebappLoader.html > http://svn.apache.org/repos/asf/tomcat/container/tc5.5.x/catalina/src/share/org/apache/catalina/loader/WebappClassLoader.java > I wrote something that works for me: http://issues.apache.org/bugzilla/show_bug.cgi?id=38223 Essentially: You need to 1. Place a class in CATALINA_HOME/server/classes 2. Place a context.xml for your web app at CATALINA_HOME/conf/Catalina/localhost 3. Place a myclasses.properties file at some location and edit the context.xml to point to this file. 4. Edit the myclasses.properties file to contain a list of all the folders that contain your classes. 5. Start Tomcat and use your Web App. Your WebApp need not have all the classes in WEB-INF/classes. They can be at the locations specified in the myclasses.properties file. Let me know if you find this useful -- Sriam - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On 1/9/06, Oded Arbel <[EMAIL PROTECTED]> wrote: > > So now if an application uses a local library, and that library required > a third party jar, it can't find it. > The way I see it, I have two options - either put all the 3rd party > libraries back in the JVM classpath, or build jars from my locally > developed libraries and copy them by hand to the web applications' > folder. I don't like either option, and I would really like a third one > - something like configuring the default web.xml file to add external > directories to the web-app class loader ? can something like that be > done ? > Based on my reading of the Context element at http://tomcat.apache.org/tomcat-5.5-doc/config/context.html, no such thing can be done. Your best bets are: 1. Alter the Tomcat startup batch file to Include those folders that you need. 2. Write a batch file that'll copy the classes you need to the web apps' classes fodlers, and then call Tomcat via the startup batch file. This way, you'll not need to change Tomcat's startup batch file, plus with a bit of experimentation and care, you'll avoid the error prone manual copynig. 3. You might even wish to see if you could copy classes into the classes folder while Tomcat is running. That way the web apps would get reloaded. You'd need to configure Tomcat to recognize the Web apps to be reloadable, of course. -- Sriram - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
> On 1/10/06, Oded Arbel <[EMAIL PROTECTED]> wrote: > >> >> An optimal setup for me, I think, is: >> - for each application to put the 3rd party dependencies in WEB-INF/lib >> (I'm using JPackage's build-jar-repository, which I auto invoke from >> the ant script, so its even automatic). > > I thought you said you were not able to the the classes into jar files > (as that was part of a separate build automation process) ? No - I'm talking here about the 3rd party libraries, which I did not develop and are not part of tomcat - these come as jars. >> - for each application that users locally developed libraries to list >> their build directories in the web.xml so what the WebAppClassLoader >> can get at them without them being shared. >> > > There's no attribute in web.xml that let's you specify separate class > paths to load from. > > What you could do is to write your own WebAppClassLoader that'd add > these folders to its lookup list. Hmm. interesting. How can I let such a class loader implementation to be used instead of tomcat's WebAppClassLoader ? -- Oded - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On 1/10/06, Oded Arbel <[EMAIL PROTECTED]> wrote: > > An optimal setup for me, I think, is: > - for each application to put the 3rd party dependencies in WEB-INF/lib > (I'm using JPackage's build-jar-repository, which I auto invoke from > the ant script, so its even automatic). I thought you said you were not able to the the classes into jar files (as that was part of a separate build automation process) ? > - for each application that users locally developed libraries to list > their build directories in the web.xml so what the WebAppClassLoader > can get at them without them being shared. > There's no attribute in web.xml that let's you specify separate class paths to load from. What you could do is to write your own WebAppClassLoader that'd add these folders to its lookup list. -- Sriram - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On Monday, 9 בJanuary 2006 21:34, Boris Unckel wrote: > > But then, if the API library calls a 3rd party library, that > > library can't be put in WEB-INF/lib either - it has to be put in > > the tomcat's startup classpath as well. > > Ok. I will repeat to ensure I understand it: > You have a development system where you do not want to jar for each > test of your API, you also do not want to war (ant task) before you > test. Your API depends on 3rd party JARs which are not in the default > distribution of Tomcat. Yes, more or less. The problem is not a single locally developed library - the problem is that I have many of those, and each required different third party libraries. > For your development system only (please do not use this on a > production machine): > Put your classes in > TOMCAT_ROOT/shared/classes > (create the folder if not existing) As mentioned above, I have more then one project that requires this treatment - putting the products of all projects in one place is not the way I want to go, as its a mess and it will be hard to know which class belongs to which project, and I may also encounter name conflicts (I don't believe that I have those, but its not guaranteed). > put your libs in > TOMCAT_ROOT/shared/lib I'm trying to avoid the need to put the 3rd party dependencies of all the web applications in a single place - due to the reasons you noted. An optimal setup for me, I think, is: - for each application to put the 3rd party dependencies in WEB-INF/lib (I'm using JPackage's build-jar-repository, which I auto invoke from the ant script, so its even automatic). - for each application that users locally developed libraries to list their build directories in the web.xml so what the WebAppClassLoader can get at them without them being shared. This kind of setup would leave all the mess out of the JVM classpath, would allow me to define on a per web application what I want it to have on the classpath by once editing a simple file, and won't require me to do complex hand-eye coordination routines each time new code is introduced. My current interim solution (which is bad) is to have the continuous build process for each locally developed library also create the jar packages, and then I symbolically link them into the WEB-INF/lib directory. The main down side is that this lengthens the continuous build process by almost twice as much, and so a lot of times I have to wait for eclipse to finish a source code commit (that blocks the user interface) before I can continue to work. -- Oded ::.. I've never been canoeing before, but I imagine there must be just a few simple heuristics you have to remember... Yes, don't fall out, and don't hit rocks. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
> But then, if the API library calls a 3rd party library, that library > can't be put in WEB-INF/lib either - it has to be put in the tomcat's > startup classpath as well. Ok. I will repeat to ensure I understand it: You have a development system where you do not want to jar for each test of your API, you also do not want to war (ant task) before you test. Your API depends on 3rd party JARs which are not in the default distribution of Tomcat. For your development system only (please do not use this on a production machine): Put your classes in TOMCAT_ROOT/shared/classes (create the folder if not existing) put your libs in TOMCAT_ROOT/shared/lib Avoid any JAR duplication (JAR x.y-1.0.jar is in two places: bootclasspath, common/lib, shared/lib, WEB-INF/lib), it will cause you a lot of pain to find out which JAR causes the (potential) runtime error and why. Read for http://tomcat.apache.org/tomcat-5.5-doc/class-loader-howto.html for details. For any production environment: Avoid the use of the sharded classloader as much as you can. If you have a version conflict (WebApp A relies on 3rd party JAR 1.0, WebApp B relies on 3rd party JAR 2.1) you have no chance with the shared classloader or will have conflicts. Recommended is use of WEB-INF/lib Distribute your application (WebApp) with any JARs you need in WEB-INF/lib. Regards Boris - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
On Monday, 9 בJanuary 2006 20:02, Boris Unckel wrote: > > The way I see it, I have two options - either put all the 3rd party > > libraries back in the JVM classpath, or build jars from my locally > > developed libraries and copy them by hand to the web applications' > > folder. I don't like either option, and I would really like a third > > one - something like configuring the default web.xml file to add > > external directories to the web-app class loader ? can something > > like that be done ? > > I do not fully understand your problem, but maybe something simple: > WEB-INF/lib for your jars > WEB-INF/classes for your classes The problem is like this (the actual system is far more complex): - suppose two web applications, app1 and app2, both use some API (which I developed myself). - I don't want to develop the API twice on both web applications, so I have a third project for the API itself, which is a not a web application and so isn't in WEB-INF/classes of neither app1 nor app2. - I don't want to build a jar for that API library to put in app1 and app2's WEB-INF/lib because its also constantly being developed - if I wanted to do this then after each commit, in addition to the build stage I would have to copy the jar by hand to all the applications that use it, which is an error prone process. - so instead I add the directory where the API's class files are built to the tomcat's startup classpath. - But then, if the API library calls a 3rd party library, that library can't be put in WEB-INF/lib either - it has to be put in the tomcat's startup classpath as well. Now imagine about 20 such locally developed API libraries (not all used in all web applications, but a lot of web applications use more then one such library), with inter-dependencies between themselves and dependencies on more then 50 3rd party libraries - the situation gets way out of hand very quickly. Before I put a stop to the madness, the tomcat startup classpath was taking about 15 rows of text in my console. -- Oded ::.. The point of philosophy is to start with something so simple as not to seem worth stating, and to end with something so paradoxical that no one will believe it. -- Bertrand Russell, The Philosophy of Logical Atomism - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Question regarding tomcat class path handling
> The way I see it, I have two options - either put all the 3rd party > libraries back in the JVM classpath, or build jars from my locally > developed libraries and copy them by hand to the web applications' > folder. I don't like either option, and I would really like a third one > - something like configuring the default web.xml file to add external > directories to the web-app class loader ? can something like that be > done ? I do not fully understand your problem, but maybe something simple: WEB-INF/lib for your jars WEB-INF/classes for your classes ? Regards Boris - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
