Hi, Firstly, this is actually a continuation of a previous thread (specifically the email from Make Thomas, dated 21/02/2022 08:10) discussing the specification for enhancing the reloading of webapps after changes to the deployed files. This thread is titled ‘<Context … reloadable = "true"> is too quick to respond’ and remains active with outstanding discussion points. However, in order to try and implement the suggested changes, I needed to create a development environment and so the thread quickly shifted into a bunch of configuration issues which took the thread off topic.
Secondly, this email is quite long as I wanted to describe the steps I took to get to where I am now (briefly, nearly there, just can't fully test, unless all of them!). I have also included some suggestions for alterations to the NetBeans configuration files that I am happy to commit back to git if my understanding is correct./ deductions are correct although one of them appears to be an issue with the ant build and I am not confident enough to research this. There are a couple of attachments that, as before, I have placed in my shared Dropbox folder, accessed from https://www.dropbox.com/sh/2ewipogzr48qcxi/AAAf3Rqv6WoRO9hyMC0W7P2za?dl=0 Mark, your insightful email has proved invaluable and having spent a couple of days working through this, it has provided the catalyst for me to be able to start joining the dots to achieving a full development environment. So, responding to each of your suggestions has taken me on a massive learning curve, but.... Firstly, I started back with a blank piece of paper. > If your eventual aim is to provide a pull request to address this issue then > I'd recommend: > - Fork the Tomcat project in GitHub > - Checkout your clone As a life-long supporter of Subversion, this was my first foray into the world of git, but I believe, after a quick crash course, I have managed to have forked and cloned Tomcat onto my laptop! > This is where using a clone would speed things up as you'd be able to fetch > the update from Git. Therefore, you can now assume that I can speed things up :) I have built Tomcat using Ant. Ran the amended 'ant ide-netbeans' script (checked GitHub to see what changes had been made - the details of this change log request turned out to be very useful later!) > Hmm. You shouldn't see any warnings about modules. A handful of warnings > about deprecation are expected. If you still see these warnings then yes, > please provide a copy so we can take a look. Launched NetBeans and did a clean > deploy. The results are in my shared DropBox folder (along with everything else) called "Ant Clean and Build from Netbeans.txt" and it did actually look to have done a successful build if you believe the summary line! While I did get warnings about deprecated features as expected, I also had quite a number of other warnings that looked a bit concerning (which you asked me to submit to you) e.g. - WARN aQute.bnd.plugin.jpms.JPMSModuleInfoPlugin - Can't find a module name for imported package: jakarta.websocket At this point, I still had a lot of red [!] against the Tests folder, just as before and so the version amendment for jUnit didn't appear to work, although I verified that it is the latest jUnit v4 build from Mavens repository. As an aside, as I am sure you are aware, NetBeans didn't support any later versions of jUnit using the Ant framework as they believe that no-one should be using Ant anymore and should migrate to Maven / Gradle! So, rather than push back immediately, I thought I would spend some time yesterday trying to figure it out, given that you had (by your change) hinted which of the files might be related to my issue. Also, your reference to tomcat\res\ide-support\netbeans\README.txt was enlightening! Anyway, to cut a long investigation short, I found two environments that were out of sync. As far as I can tell, in nbproject\project.xml, the classpath just underneath the <unit-tests/> element <classpath mode="compile">output/classes:output/testclasses:${base.path}/junit-4.13.2/junit-4.13.2.jar:${base.path}/easymock-4.3/easymock-4.3.jar:${base.path}/objenesis-3.2/objenesis-3.2.jar:${base.path}/cglib-3.3.0/cglib-nodep-3.3.0.jar:${base.path}/hamcrest/hamcrest-2.2.jar</classpath> should look more like <classpath mode="compile">output/classes:output/testclasses:${base.path}/junit-4.13.2/junit-4.13.2.jar:${base.path}/easymock-4.3/easymock-4.3.jar:${base.path}/objenesis-3.2/objenesis-3.2.jar:${base.path}/cglib-3.3.0/cglib-nodep-3.3.0.jar:${base.path}/hamcrest-2.2/hamcrest-2.2.jar:${base.path}/unboundid-ldapsdk-6.0.3/unboundid-ldapsdk-6.0.3.jar</classpath> There was a typo in the hamcrest folder name (missing version) and some of the classes the test sources referred to were missing. I used the Maven repository to identify the latest versions and downloaded them. The reason that the jUnit classes weren't found is that they weren't being downloaded into the {base.path} folder as part of the 'ant' script, used to build Tomcat, but looks like only the jars needed to build the project were (but that's an assumption of mine). I do not know enough about Ant to try and determine why this may be. Also, while looking here, should the <source-level> be at least updated to '11' as '1.7' gives a bucketload of exceptions when loaded into NetBeans? So, my folder structure now looks like below, with jUnit, easyMock, hamcrest, odjenesis, unboundid having being downloaded manually. I may have missed one out of that list as I forgot to take a directory listing prior to adding the additional folders to compare with. C:\Community\DownloadedTomcatLibraries\bnd-6.1.0 C:\Community\DownloadedTomcatLibraries\cglib-3.3.0 C:\Community\DownloadedTomcatLibraries\commons-daemon-1.2.4 C:\Community\DownloadedTomcatLibraries\derby-10.15.2.0 C:\Community\DownloadedTomcatLibraries\easymock-4.3 C:\Community\DownloadedTomcatLibraries\ecj-4.22 C:\Community\DownloadedTomcatLibraries\hamcrest-2.2 C:\Community\DownloadedTomcatLibraries\jacoco-0.8.7 C:\Community\DownloadedTomcatLibraries\jaxrpc-1.1-rc4 C:\Community\DownloadedTomcatLibraries\junit-4.13.2 C:\Community\DownloadedTomcatLibraries\migration-1.0.0 C:\Community\DownloadedTomcatLibraries\objenesis-3.2 C:\Community\DownloadedTomcatLibraries\osgi-annotations-1.1.1 C:\Community\DownloadedTomcatLibraries\unboundid-6.0.3 C:\Community\DownloadedTomcatLibraries\unboundid-ldapsdk-6.0.3 C:\Community\DownloadedTomcatLibraries\wsdl4j-1.6.3 C:\Community\DownloadedTomcatLibraries\bnd-6.1.0\biz.aQute.bnd-6.1.0.jar C:\Community\DownloadedTomcatLibraries\cglib-3.3.0\cglib-nodep-3.3.0.jar I still have a handful of red ! against some of the test files. org.apache.catalina.ant.TestDeployTask - I have a red ! flag against this file, but the source file looks to be fine (no markers against any of the lines to backup the red ! against the file). I will ask the Netbeans community if I can find out what that means! org.apache.coyote.http2.TestStream - package trailers does not exist - import trailers.ResponseTrailers; util.TestCookieFilter - cannot find symbol CookieFilter in (e.g., multiple instances) Assert.assertEquals("a=b", CookieFilter.filter("a=b", null)); (e.g.) webapp/bug36923.jsp, but there are quite a few similar. - This may be Netbeans validating against specific HTML versions. I have taken 2 screenshots (prefixed JSP Exception) of the exception and proposed fix but not done anything as not sure whether these are historic or used as part of the test framework. It would be nice to start with a clean sheet though, more in case others later come on board and get distracted by exceptions that aren't important. Apart from those remaining exceptions, the tests folder looks to be acceptable to NetBeans. I then ran 'ant test' and encouragingly, all the tests appeared to pass!!!! > Following instructions in points 7 & 8 of the > tomcat\res\ide-support\netbeans\README.txt (run and debug an individual test) >From NetBeans, I then went to see if I could run an individual test and came unstuck. OK, I picked a test at random that looked interesting so I tried org.apache.catalina.servlets.ServletOptionsBaseTest. The log (Test output for org.apache.catalina.servlets.ServletOptionsBaseTest.txt) reported an Error and so FAILED the test. I have uploaded the log to DropBox. I also tried adding a breakpoint on the first line (as described in point 8 of the document) and got the same response and the breakpoint wasn't triggered. Using details in the BUILDING.txt, I tried implementing the instructions for a running a single test from the CMD using 'ant test', but first, amending the build.properties file and adding the line test.entry=org.apache.catalina.util.TestServerInfo which is the one documented in BUILDING.txt. That failed as well, with the following (similar to NetBeans) error Testsuite: org.apache.catalina.servlets.ServletOptionsBaseTest Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.128 sec ------------- Standard Error ----------------- java.util.logging.ErrorManager: 0: Unable to delete log files older than [90] days java.util.logging.ErrorManager: 0: Unable to delete log files older than [90] days java.util.logging.ErrorManager: 0: Unable to delete log files older than [90] days java.util.logging.ErrorManager: 0: Unable to delete log files older than [90] days ------------- ---------------- --------------- Testcase: testOptions took 0.002 sec Caused an ERROR null java.lang.InstantiationException at java.base/jdk.internal.reflect.InstantiationExceptionConstructorAccessorImpl.newInstance(InstantiationExceptionConstructorAccessorImpl.java:48) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) So it may be the specific test I picked (at random) that only makes sense when run as part of a suite of tests so any help would be appreciated. I also tried the simple test case detailed in BUILDING.txt (test.entry=org.apache.catalina.util.TestServerInfo) and this worked both in NetBeans and Ant standalone, as well as adding a breakpoint and so that adds weight to the thought that I picked a test that is not meant to be run in isolation. > The simplest way to test the build is to try the following: > cd output\build\bin > catalina.bat run I then tried to launch Tomcat from a CMD line (again, thanks for the 'obvious' starting script but these hints really help so that I know that taking the right path) The first time I attempted it, it appeared to work beautifully, until I realised that it had actually launched my production copy of Tomcat as I could see the listeners that I have setup reporting that they had been triggered successfully! So I amended the environment variable CATALINA_HOME = C:\Community\tomcat (the location of the webapps directory) and tried again and came to an abrupt halt. The output was as follows C:\Community\tomcat\output\build\bin>catalina.bat run Using CATALINA_BASE: "C:\Community\tomcat" Using CATALINA_HOME: "C:\Community\tomcat" Using CATALINA_TMPDIR: "C:\Community\tomcat\temp" Using JRE_HOME: "C:\Development\JDK\jdk-17" Using CLASSPATH: "C:\Community\tomcat\bin\bootstrap.jar;C:\Community\tomcat\bin\tomcat-juli.jar" Using CATALINA_OPTS: "" NOTE: Picked up JDK_JAVA_OPTIONS: --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED Error: Could not find or load main class org.apache.catalina.startup.Bootstrap Caused by: java.lang.ClassNotFoundException: org.apache.catalina.startup.Bootstrap This brings me to a point I noticed earlier - that there are two 'bin' folders - C:\Community\tomcat\bin & C:\Community\tomcat\output\build\bin and the first didn't have bootstrap.jar within it, and from the date stamps are 21/02/2022 at 11:23 and no jars, and so this may be an outline folder, also, while the dates are yesterday, that may be around the time that I cloned Tomcat onto my laptop although I would have thought that the date/time would represent when they were last amended rather than when they were copied to me (as I would expect from Subversion). So I amended the CATALINA_HOME = C:\Community\tomcat\output\build C:\Community\tomcat\output\build\bin>catalina.bat run Using CATALINA_BASE: "C:\Community\tomcat\output\build" Using CATALINA_HOME: "C:\Community\tomcat\output\build" Using CATALINA_TMPDIR: "C:\Community\tomcat\output\build\temp" Using JRE_HOME: "C:\Development\JDK\jdk-17" Using CLASSPATH: "C:\Community\tomcat\output\build\bin\bootstrap.jar;C:\Community\tomcat\output\build\bin\tomcat-juli.jar" Using CATALINA_OPTS: "" and it launched!!! I checked http://localhost:8080/examples and that all looks OK So, out of interest, why are there two bin folders? > I believe the way most Tomcat committers work is running Tomcat on the > command line and remote debugging from their IDE of choice. > Following instructions in points 9 of the > tomcat\res\ide-support\netbeans\README.txt (run and debug an individual test) In short, I was successful in this. Again, I had never done remote debugging before! I needed to change the port to 8080 everywhere but that was about it. I fudged a few parameters in catalina.bat and tweaked the source of org.apache.catalina.util.ServerInfo (added a console keyboard read to pause the routine) in the first instance as I could test the remote debugging on a simple example. Netbeans initially claimed that the source with the breakpoint in wasn't from tomcat\java (it was) but mysteriously, that message went away and got the "lineBreakpoint ServerInfo:117 successfully submitted" in the status bar and all was well. Finally (you will be glad to know), I repeated the exercise, setting the debugger to break at WebappLoaded.backgroundProcess(), as advised and that worked a treat. This Java Remote Debugging is pretty impressive! > Tomcat 10.1.x (the current main branch) requires Java 11 as a minimum. You > should be fine setting the source level to anything 11 or above. Personally, > I'd set it to 11 as that way you avoid the (very low) risk of proposing a fix > that doesn't work on Java 11. If I set the build version to 11, do I need to install JDK 11 as well and update my 'env' JAVA_HOME, or is the Java compiler clever enough to limit the byte code generation to that which will run on Java 11 while still allowing me to have JDK 17 as my environment. I think the answer is Yes, but would be nice to have confirmation. > Tell NetBeans to ignore that folder. It should not be part of the source you > are trying to compile. It is part of a (rather old) tutorial on building a > web application. To do this, I amended the project.xml in nbproject to remove the three references to sample. I also removed the single reference to 'tomcat-lite'. The folder wasn't cloned from git and I could't find any reference to a 'lite' version of Tomcat and so I was wondering if this is an historic / abandoned project. Should the master \tomcat\res\ide-support\netbeans\project.xml file be edited to remove these permanently (again to provide a clean experience for others). In summary, I am assuming from the lack of response from my request about NetBeans, that this is not a favourite environment for Tomcat development. It may just have been a long time since anyone deployed the development framework into NetBeans and so my efforts might be worthwhile as a refresher check on the current documentation. I am keeping notes of the process I have been through so far as well as how to resolve issues I have encountered and so will look to update the README for NetBeans to help future developers. John --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
