Robert,
Apologies for the delayed reply. Having found the root cause of the
issue I wanted to confirm whether or not the Docker desktop team viewed
it as a security vulnerability or not. I received confirmation yesterday
that they do not.
TL;DR
Running Tomcat in a container via Docker Desktop on a Windows host with
the web application served from a location on the host mounted/bound to
the container is insecure.
Longer version:
To ensure that things like security constraints are correctly applied,
Tomcat has to process URLs in a case sensitive manner. "/Test.jsp" is
not he same as "/test.jsp".
URLs are often mapped to resources on the file system. This mapping also
needs to be applied in a case sensitive manner. If the file system is
case sensitive, this is relatively straight forward. If the file system
is case insensitive, Tomcat performs some additional checks. These look
something like this:
- Request for "/Test.jsp"
- Find file "$appbase/Test.jsp"
- Get the canonical path
- Confirm the case matches the original request
On a case insensitive file system, looking up "$appbase/Test.jsp" will
match "$appbase/test.jsp" but the canonical path will return
"$appbase/test.jsp" which doesn't match so a 404 will be returned.
The issue with Docker Desktop is that paths on the Windows host
mounted/bound to the container behave like this:
- The path - as far as Windows is concerned - is "$appbase/Test.jsp"
- Code running in the container looks up "$appbase/Test.jsp"
- The file is found
- The canonical path is "$appbase/Test.jsp"
- So far, so good
- Code running in the container looks up "$appbase/test.jsp"
- The file is found
- The canonical path is "$appbase/test.jsp"
The issue is that the canonical path returned the second time matches
the path used to obtain the resource, not the canonical path on the
Windows file system. This means Tomcat cannot perform the case
sensitivity tests.
This creates the following security issues:
- security constraints can be bypasses
- JSP source code disclosure
It also creates class loading issues as Java class and package names are
also case sensitive.
The view of the Docket Desktop team is that this is, at best, a bug not
a security vulnerability because Docker Desktop is a developer tool, not
a tool for running production instances. Further, the expectation is
that the web application would be included in the container in production.
If you do continue to use this approach in development, keep in mind that:
- you may see issues like the original EL issue you reported
- security testing may report false positives
HTH,
Mark
On 08/02/2022 15:11, Robert Turner wrote:
Okay. Yep, my most recent suspicion was correct -- it's related to the
Docker bind to a local folder containing the webapps. As such, I believe
it's a Docker issue of some sort and not Tomcat specific. However, you may
want to understand it more completely in any case.
Thanks for your help Mark, Rob S and Neil.
Here are the full details of the reproduction scenario:
Host system: MacOS 12.2; Docker Desktop v4.4.2 (73305), Engine 20.10.12
1. Using Maven Archetypes, do the following:
mvn archetype:generate -DarchetypeGroupId=org.apache.maven.archetypes
-DarchetypeArtifactId=maven-archetype-webapp -DarchetypeVersion=1.4
When prompted for configuration values, use the following:
groupId: com.example.rt
artifactId: test-el-resolver
version: 1.0-SNAPSHOT
package: com.example.rt
2. Switch to the new folder created for the web app
cd test-el-resolver
3. Modify the index.jsp file (src/main/webapp/index.jsp) to have the
following contents:
<%@page contentType="text/html" pageEncoding="UTF-8" isELIgnored="false" %>
<%@page import="com.example.rt.Failing"%>
<%
final Failing failing = null;
pageContext.setAttribute("failing", failing);
%>
<html>
<body>
<h2>Hello World!</h2>
<p>field1=${failing.field1}</p>
</body>
</html>
4. Add the following contents to
file src/main/java/com/example/rt/Failing.java:
package com.example.rt;
public class Failing {
private final String field1 = "field1_value";
public String getField1() {
return field1;
}
}
5. Build the web app
mvn package
6. Create a local folder to mount/bind into the Docker container
mkdir webapps
7. Copy the WAR file to the new folder
cp target/test-el-resolver.war webapps/
8. Start a docker container binding the local web apps folder
docker run -d -p 8075:8080 --mount
type=volume,src=`pwd`/webapps,dst=/usr/local/tomcat/webapps
tomcat:9.0.58-jre11-openjdk
9. Using a browser (or equivalent), access the app:
curl http://localhost:8075/test-el-resolver/
10. Observe the following exception in the error page:
Stacktrace:
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:610)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:489)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
</pre><p><b>Root Cause</b></p><pre>javax.servlet.ServletException:
java.lang.NoClassDefFoundError: com/example/rt/Failing (wrong
name: com/example/rt/failing)
org.apache.jasper.runtime.PageContextImpl.handlePageException(PageContextImpl.java:657)
org.apache.jsp.index_jsp._jspService(index_jsp.java:145)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:466)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
</pre><p><b>Root Cause</b></p><pre>java.lang.NoClassDefFoundError:
com/example/rt/Failing (wrong name:
com/example/rt/failing)
java.base/java.lang.ClassLoader.defineClass1(Native Method)
java.base/java.lang.ClassLoader.defineClass(Unknown Source)
java.base/java.security.SecureClassLoader.defineClass(Unknown Source)
org.apache.catalina.loader.WebappClassLoaderBase.findClassInternal(WebappClassLoaderBase.java:2478)
org.apache.catalina.loader.WebappClassLoaderBase.findClass(WebappClassLoaderBase.java:870)
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1371)
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1215)
javax.el.ImportHandler.findClass(ImportHandler.java:477)
javax.el.ImportHandler.resolveClass(ImportHandler.java:421)
javax.servlet.jsp.el.ScopedAttributeELResolver.getValue(ScopedAttributeELResolver.java:85)
org.apache.jasper.el.JasperELResolver.getValue(JasperELResolver.java:124)
org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:93)
org.apache.el.parser.AstValue.getValue(AstValue.java:136)
org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:189)
org.apache.jasper.runtime.PageContextImpl.proprietaryEvaluate(PageContextImpl.java:692)
org.apache.jsp.index_jsp._jspService(index_jsp.java:130)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:466)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
</pre><p><b>Note</b> The full stack trace of the root cause is available in
the server logs.</p><hr class="line" /><h3>Apache
Tomcat/9.0.58</h3></body></html>
And a "working" scenario, replace step 8 with the following:
docker run -d -p 8075:8080 tomcat:9.0.58-jre11-openjdk
docker cp target/test-e-resolver.war
<container_id>:/usr/local/tomcat/webapps/
Accessing the page with the curl command above will result in the expect
page being displayed (and no exception).
Robert
On Tue, Feb 8, 2022 at 9:39 AM Robert Turner <rtur...@e-djuster.ca> wrote:
Mark,
Thanks for the quick follow up.
Based on your comments I have some ideas on what it might be, and I should
be able to narrow that down further for you, and provide all the details
that you requested.
A few notes on the environment I'm using:
Docker (the host) is running on my Mac (12.2), not in AWS. AWS Linux is
only involved in the Docker image (as per the Dockerfile I shared earlier).
Moreover, it happens with Debian in the Docker image as well.
What I think is important is that I have "<tomcat>/work" and
"<tomcat>/webapps" mounted to file folders on the Mac, so I am suspecting
it's something to do with that "mounting" of the volume to the host OS, and
how Docker is mapping those and handling filename casing, etc. My MacOS
file system is APFS, Encrypted (and I thought I had case sensitivity
enabled, but I can no longer see that option -- maybe not an option for
APFS).
I will try to confirm suspicions and provide details in a few hours
(hopefully -- got a few meetings today that will get in the way).
Thanks again,
Robert
On Tue, Feb 8, 2022 at 8:51 AM Mark Thomas <ma...@apache.org> wrote:
Robert,
I agree this is something to do with the Docker environment.
I think case insensitivity is involved somewhere as I can trigger the
error if I copy my equivalent of Failure.class to failure.class and then
call the JSP.
I understand why it only occurs for * imports. In that instance, Tomcat
has to check if the class can be loaded from each of the imported
packages. Tomcat checks the file system first as that is faster than
trying (and failing) to load a class. The file system checks are
designed to be case sensitive - even on case insensitive file systems.
Something seems to be going wrong here and I'm still not sure what.
I have tried to recreate this on AWS without success. If you have time,
I think we'd need the following to dig into this further:
- Source for the simplest possible test WAR that demonstrates this
issue. I think a single class and a single JSP page should be
sufficient.
- The WAR you created from the above (to rule out any build issues).
- A minimal Dockerfile to create a Tomcat instance that demonstrates
this issue. Should just copy the WAR to the container and start it
with JPDA enabled.
- Which AMI you used to create the AWS instance. I'm using the AWS free
tier so I used a t2.micro instance with
amzn2-ami-kernel-5.10-hvm-2.0.20220121.0-x86_64-gp2
Thanks,
Mark
On 08/02/2022 13:24, Robert Turner wrote:
One thing to add is that my "conclusion" about OS variances I believe
to be
incorrect. Our tests typically run on Linux, so I think it's still
something to do with a difference in the Docker-based environment.
Let me know if you need any more details on anything...(but I suspect
with
a debugger up on the Expression Resolvers, you will at least narrow it
down
quickly...)
On Tue, Feb 8, 2022 at 7:55 AM Robert Turner <rtur...@e-djuster.ca>
wrote:
Thanks Mark. Much appreciated.
On Tue., Feb. 8, 2022, 04:06 Mark Thomas, <ma...@apache.org> wrote:
Robert,
Thank you for putting the effort in to debugging this. Narrowing down
the issue to a simple test case is extremely helpful.
The behaviour you describe looks odd to me. I'd expect consistent
behaviour across platforms irrespective of the case sensitivity of the
file system in use.
I'm going to use your test case to investigate this further. I'll
report
back here with my findings - hopefully later today.
Mark
On 08/02/2022 03:29, Robert Turner wrote:
Okay, so I have finally narrowed it down the trivial failure case,
and I
think I have an explanation as a result:
[1] works (in docker), and [2] fails (in docker) but works outside.
The
difference between the two is the import directive being a wildcard
(ugly,
but historical in our app in some places we haven't yet cleaned up).
I am therefore speculating based on the Expression Language
specification
that because the class wasn't explicitly imported, it's not in the
list
of
available classes for static class resolution, and thus it fails.
Combine this with MacOS and Windows not caring about filename cases,
and
Linux caring, then I suspect it's just matching differently in both
cases.
Workaround/fix would be:
- to ensure we explicitly import the class (instead or in addition to
the
wildcard)
OR
- rename the attribute so it doesn't map directly to the class name.
So I think I can bring my overly-detailed thread to an end...unless
my
guess at the reasoning is incorrect and someone has a better
explanation.
Thanks Rob S and Neil for having a look and providing suggestions --
in
part, it was something related to what you both said, but I believe
in
different contexts than you expected.
Robert
[1]
$ cat src/main/webapp/index.jsp
<%@page contentType="text/html" pageEncoding="UTF-8"
isELIgnored="false" %>
<%@page import="com.example.rt.Failing" %>
<%
final Failing failing = null;
pageContext.setAttribute("failing", failing);
%>
<html>
<body>
<h2>Hello World!</h2>
<p>field1=${failing.field1}</p>
</body>
[2]
$ cat src/main/webapp/index.jsp
<%@page contentType="text/html" pageEncoding="UTF-8"
isELIgnored="false" %>
<%@page import="com.example.rt.*" %>
<%
final Failing failing = null;
pageContext.setAttribute("failing", failing);
%>
<html>
<body>
<h2>Hello World!</h2>
<p>field1=${failing.field1}</p>
</body>
On Mon, Feb 7, 2022 at 10:14 PM Robert Turner <rtur...@e-djuster.ca>
wrote:
So back to a divide and conquer approach I think.
I just created a trivial "example" [1] and it works as expected
(i.e.
no
exception was generated) (on the same servers I was testing the
complex JAR
file) -- so possibly something else modifying the behaviour -- a JAR
on the
classpath, or something in the JSP file...
[1]
$ cat src/main/webapp/index.jsp
<%@page contentType="text/html" pageEncoding="UTF-8"
isELIgnored="false" %>
<%@page import="com.example.rt.Failing"%>
<%
final Failing failing = null;
pageContext.setAttribute("failing", failing);
%>
<html>
<body>
<h2>Hello World!</h2>
<p>field1=${failing.field1}</p>
</body>
</html>
$ cat src/main/java/com/example/rt/Failing.java
package com.example.rt;
public class Failing {
private final String field1 = "field1_value";
public String getField1() {
return field1;
}
}
On Mon, Feb 7, 2022 at 9:51 PM Robert Turner <rtur...@e-djuster.ca>
wrote:
I'm just avoiding sharing product details or things I think only
serves
to confuse the problem. Sorry if you felt I wasn't sharing. It
wasn't
my
intention to be obtuse. I didn't believe they added any value for
the
diagnostics (of course that assumes I know enough about the
problem).
However, since you think they might be useful, here they the name
mapping
from the exception and the Java and JSP code excerpts:
"package1" -> "model"
"Class1" -> "Organization"
"class1" -> "organization"
The class is present in the package (see [1]) -- otherwise it
wouldn't
work in one environment and not the other -- I believe I have
confirmed
that carefully. Full class paths (with path-names relativised to
compare
between environments) listed below [2], and the code that emitted
the
listing is here [3]. I post-processed the log lines as follows [4].
Docker
file for the Tomcat container provided [5]. JDK details listed in
[6].
Tomcat version is 9.0.58 in all cases.
What seems to be different is:
- the way the EL resolver is working
OR
- the behaviour of the class loader differs in the different
environments.
OR
- something else I do not understand is relevant
The working environments are MacOS and Windows, and the failing
environment is either Debian or AWS Linux 2 running in a docker
container.
If the class loaders behaviour differently, then that could explain
the
issues, however, that would surprise me if they differed in any
material
way on the different platforms.
I hope that helps provide more detail that might be useful...
Robert
[1]
$ unzip -l target/app.war | grep "model\/Organization\.class"
66246 02-07-2022 20:17
WEB-INF/classes/model/Organization.class
[2]
java.class.path=/bin/bootstrap.jar:/bin/tomcat-juli.jar
Class loader URLs:
Class loader
name=org.apache.catalina.loader.ParallelWebappClassLoader,
URL count=146
URL=file:/WEB-INF/classes/
URL=file:/WEB-INF/lib/FastInfoset-1.2.18.jar
URL=file:/WEB-INF/lib/SparseBitSet-1.2.jar
URL=file:/WEB-INF/lib/VeracodeAnnotations-1.2.1.jar
URL=file:/WEB-INF/lib/activation-1.1.jar
URL=file:/WEB-INF/lib/animal-sniffer-annotations-1.20.jar
URL=file:/WEB-INF/lib/annotations-4.1.1.4.jar
URL=file:/WEB-INF/lib/api-common-2.0.1.jar
URL=file:/WEB-INF/lib/auto-value-annotations-1.8.2.jar
URL=file:/WEB-INF/lib/avatax-rest-v2-api-java_2.11-21.12.1.jar
URL=file:/WEB-INF/lib/aws-java-sdk-core-1.12.145.jar
URL=file:/WEB-INF/lib/aws-java-sdk-kms-1.12.145.jar
URL=file:/WEB-INF/lib/aws-java-sdk-s3-1.12.145.jar
URL=file:/WEB-INF/lib/bcmail-jdk15on-1.70.jar
URL=file:/WEB-INF/lib/bcpkix-jdk15on-1.70.jar
URL=file:/WEB-INF/lib/bcprov-jdk15on-1.70.jar
URL=file:/WEB-INF/lib/bcutil-jdk15on-1.70.jar
URL=file:/WEB-INF/lib/castor-core-1.4.1.jar
URL=file:/WEB-INF/lib/castor-xml-1.4.1.jar
URL=file:/WEB-INF/lib/checker-qual-3.5.0.jar
URL=file:/WEB-INF/lib/commons-beanutils-1.9.4.jar
URL=file:/WEB-INF/lib/commons-codec-1.15.jar
URL=file:/WEB-INF/lib/commons-collections-3.2.2.jar
URL=file:/WEB-INF/lib/commons-collections4-4.4.jar
URL=file:/WEB-INF/lib/commons-compress-1.21.jar
URL=file:/WEB-INF/lib/commons-digester-2.1.jar
URL=file:/WEB-INF/lib/commons-fileupload-1.4.jar
URL=file:/WEB-INF/lib/commons-io-2.11.0.jar
URL=file:/WEB-INF/lib/commons-lang3-3.12.0.jar
URL=file:/WEB-INF/lib/commons-logging-1.2.jar
URL=file:/WEB-INF/lib/commons-math3-3.6.1.jar
URL=file:/WEB-INF/lib/commons-text-1.9.jar
URL=file:/WEB-INF/lib/conscrypt-openjdk-uber-2.5.1.jar
URL=file:/WEB-INF/lib/curvesapi-1.06.jar
URL=file:/WEB-INF/lib/ecj-3.21.0.jar
URL=file:/WEB-INF/lib/encoder-1.2.3.jar
URL=file:/WEB-INF/lib/encoder-jsp-1.2.3.jar
URL=file:/WEB-INF/lib/error_prone_annotations-2.9.0.jar
URL=file:/WEB-INF/lib/failureaccess-1.0.1.jar
URL=file:/WEB-INF/lib/fontbox-2.0.25.jar
URL=file:/WEB-INF/lib/gax-2.3.0.jar
URL=file:/WEB-INF/lib/gax-grpc-2.3.0.jar
URL=file:/WEB-INF/lib/gmbal-api-only-4.0.3.jar
URL=file:/WEB-INF/lib/google-auth-library-credentials-1.1.0.jar
URL=file:/WEB-INF/lib/google-auth-library-oauth2-http-1.1.0.jar
URL=file:/WEB-INF/lib/google-cloud-speech-1.30.5.jar
URL=file:/WEB-INF/lib/google-http-client-1.39.2.jar
URL=file:/WEB-INF/lib/google-http-client-gson-1.39.2.jar
URL=file:/WEB-INF/lib/googleauth-1.5.0.jar
URL=file:/WEB-INF/lib/grpc-alts-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-api-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-auth-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-context-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-core-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-grpclb-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-netty-shaded-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-protobuf-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-protobuf-lite-1.40.0.jar
URL=file:/WEB-INF/lib/grpc-stub-1.40.0.jar
URL=file:/WEB-INF/lib/gson-2.8.9.jar
URL=file:/WEB-INF/lib/guava-31.0.1-jre.jar
URL=file:/WEB-INF/lib/ha-api-3.1.13.jar
URL=file:/WEB-INF/lib/handlebars-4.3.0.jar
URL=file:/WEB-INF/lib/httpclient-4.5.13.jar
URL=file:/WEB-INF/lib/httpcore-4.4.15.jar
URL=file:/WEB-INF/lib/ion-java-1.0.2.jar
URL=file:/WEB-INF/lib/ipaddress-5.3.3.jar
URL=file:/WEB-INF/lib/itext-2.1.7.js9.jar
URL=file:/WEB-INF/lib/j2objc-annotations-1.3.jar
URL=file:/WEB-INF/lib/jackson-annotations-2.13.1.jar
URL=file:/WEB-INF/lib/jackson-core-2.13.1.jar
URL=file:/WEB-INF/lib/jackson-databind-2.13.1.jar
URL=file:/WEB-INF/lib/jackson-dataformat-cbor-2.13.1.jar
URL=file:/WEB-INF/lib/jai-imageio-core-1.4.0.jar
URL=file:/WEB-INF/lib/jakarta.activation-2.0.1.jar
URL=file:/WEB-INF/lib/jakarta.annotation-api-1.3.5.jar
URL=file:/WEB-INF/lib/jakarta.jws-api-2.1.0.jar
URL=file:/WEB-INF/lib/jakarta.mail-1.6.7.jar
URL=file:/WEB-INF/lib/jakarta.xml.bind-api-2.3.3.jar
URL=file:/WEB-INF/lib/jakarta.xml.soap-api-1.4.2.jar
URL=file:/WEB-INF/lib/jakarta.xml.ws-api-2.3.3.jar
URL=file:/WEB-INF/lib/jasperreports-6.18.1.jar
URL=file:/WEB-INF/lib/jasperreports-fonts-6.18.1.jar
URL=file:/WEB-INF/lib/java-saml-2.8.0.jar
URL=file:/WEB-INF/lib/java-saml-core-2.8.0.jar
URL=file:/WEB-INF/lib/javax.activation-api-1.2.0.jar
URL=file:/WEB-INF/lib/javax.annotation-api-1.3.2.jar
URL=file:/WEB-INF/lib/javax.inject-1.jar
URL=file:/WEB-INF/lib/javax.mail-1.6.2.jar
URL=file:/WEB-INF/lib/javax.xml.soap-api-1.4.0.jar
URL=file:/WEB-INF/lib/jaxb-api-2.3.1.jar
URL=file:/WEB-INF/lib/jaxb-core-2.3.0.1.jar
URL=file:/WEB-INF/lib/jaxb-impl-2.3.5.jar
URL=file:/WEB-INF/lib/jaxws-api-2.3.1.jar
URL=file:/WEB-INF/lib/jaxws-rt-2.3.5.jar
URL=file:/WEB-INF/lib/jbig2-imageio-3.0.3.jar
URL=file:/WEB-INF/lib/jcommon-1.0.24.jar
URL=file:/WEB-INF/lib/jempbox-1.8.16.jar
URL=file:/WEB-INF/lib/jettison-1.4.1.jar
URL=file:/WEB-INF/lib/jfreechart-1.0.19.jar
URL=file:/WEB-INF/lib/jmespath-java-1.12.145.jar
URL=file:/WEB-INF/lib/joda-time-2.10.6.jar
URL=file:/WEB-INF/lib/jsch-0.1.55.jar
URL=file:/WEB-INF/lib/jsoup-1.14.3.jar
URL=file:/WEB-INF/lib/jsr181-api-1.0-MR1.jar
URL=file:/WEB-INF/lib/jsr305-3.0.2.jar
URL=file:/WEB-INF/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
URL=file:/WEB-INF/lib/log4j-api-2.17.1.jar
URL=file:/WEB-INF/lib/log4j-core-2.17.1.jar
URL=file:/WEB-INF/lib/log4j-slf4j-impl-2.17.1.jar
URL=file:/WEB-INF/lib/log4j-web-2.17.1.jar
URL=file:/WEB-INF/lib/management-api-3.2.3.jar
URL=file:/WEB-INF/lib/metadata-extractor-2.16.0.jar
URL=file:/WEB-INF/lib/mimepull-1.9.15.jar
URL=file:/WEB-INF/lib/opencensus-api-0.28.0.jar
URL=file:/WEB-INF/lib/opencensus-contrib-http-util-0.28.0.jar
URL=file:/WEB-INF/lib/owasp-java-html-sanitizer-20211018.2.jar
URL=file:/WEB-INF/lib/pdfbox-2.0.25.jar
URL=file:/WEB-INF/lib/perfmark-api-0.23.0.jar
URL=file:/WEB-INF/lib/poi-4.1.2.jar
URL=file:/WEB-INF/lib/poi-ooxml-4.1.2.jar
URL=file:/WEB-INF/lib/poi-ooxml-schemas-4.1.2.jar
URL=file:/WEB-INF/lib/policy-2.7.10.jar
URL=file:/WEB-INF/lib/postgresql-42.3.1.jar
URL=file:/WEB-INF/lib/proto-google-cloud-speech-v1-1.30.5.jar
URL=file:/WEB-INF/lib/proto-google-cloud-speech-v1p1beta1-0.83.5.jar
URL=file:/WEB-INF/lib/proto-google-common-protos-2.3.2.jar
URL=file:/WEB-INF/lib/protobuf-java-3.17.3.jar
URL=file:/WEB-INF/lib/protobuf-java-util-3.17.3.jar
URL=file:/WEB-INF/lib/saaj-impl-1.5.3.jar
URL=file:/WEB-INF/lib/scala-library-2.11.12.jar
URL=file:/WEB-INF/lib/slf4j-api-1.7.33.jar
URL=file:/WEB-INF/lib/stax-api-1.0.1.jar
URL=file:/WEB-INF/lib/stax-ex-1.8.3.jar
URL=file:/WEB-INF/lib/stax2-api-4.2.1.jar
URL=file:/WEB-INF/lib/streambuffer-2.0.2.jar
URL=file:/WEB-INF/lib/taglibs-standard-impl-1.2.5.jar
URL=file:/WEB-INF/lib/taglibs-standard-spec-1.2.5.jar
URL=file:/WEB-INF/lib/threetenbp-1.3.5.jar
URL=file:/WEB-INF/lib/tika-core-2.2.1.jar
URL=file:/WEB-INF/lib/tika-parser-image-module-2.2.1.jar
URL=file:/WEB-INF/lib/tika-parser-xmp-commons-2.2.1.jar
URL=file:/WEB-INF/lib/woodstox-core-6.2.6.jar
URL=file:/WEB-INF/lib/xmlbeans-3.1.0.jar
URL=file:/WEB-INF/lib/xmlsec-2.2.3.jar
URL=file:/WEB-INF/lib/xmpcore-6.1.11.jar
Class loader name=java.net.URLClassLoader, URL count=33
URL=file:/lib/
URL=file:/lib/annotations-api.jar
URL=file:/lib/catalina-ant.jar
URL=file:/lib/catalina-ha.jar
URL=file:/lib/catalina-ssi.jar
URL=file:/lib/catalina-storeconfig.jar
URL=file:/lib/catalina-tribes.jar
URL=file:/lib/catalina.jar
URL=file:/lib/ecj-4.20.jar
URL=file:/lib/el-api.jar
URL=file:/lib/jasper-el.jar
URL=file:/lib/jasper.jar
URL=file:/lib/jaspic-api.jar
URL=file:/lib/jsp-api.jar
URL=file:/lib/servlet-api.jar
URL=file:/lib/tomcat-api.jar
URL=file:/lib/tomcat-coyote.jar
URL=file:/lib/tomcat-dbcp.jar
URL=file:/lib/tomcat-i18n-cs.jar
URL=file:/lib/tomcat-i18n-de.jar
URL=file:/lib/tomcat-i18n-es.jar
URL=file:/lib/tomcat-i18n-fr.jar
URL=file:/lib/tomcat-i18n-ja.jar
URL=file:/lib/tomcat-i18n-ko.jar
URL=file:/lib/tomcat-i18n-pt-BR.jar
URL=file:/lib/tomcat-i18n-ru.jar
URL=file:/lib/tomcat-i18n-zh-CN.jar
URL=file:/lib/tomcat-jdbc.jar
URL=file:/lib/tomcat-jni.jar
URL=file:/lib/tomcat-util-scan.jar
URL=file:/lib/tomcat-util.jar
URL=file:/lib/tomcat-websocket.jar
URL=file:/lib/websocket-api.jar
Class loader
name=jdk.internal.loader.ClassLoaders$AppClassLoader
Class loader
name=jdk.internal.loader.ClassLoaders$PlatformClassLoader
[3]
private static void
logClassLoaderUrls(org.apache.logging.log4j.Level
logLevel, ClassLoader cl) {
if (null == cl) {
return;
}
boolean loaderNameLogged = false;
if (cl instanceof URLClassLoader) {
final URLClassLoader urlCl = (URLClassLoader) cl;
final URL[] urls = urlCl.getURLs();
if (null != urls) {
LOG.log(logLevel, " Class loader name={}, URL
count={}",
cl.getClass().getName(), urls.length);
loaderNameLogged = true;
for (int idx = 0; idx < urls.length; idx++) {
final URL url = urls[idx];
LOG.log(logLevel, " URL[{}]={}", idx,
url);
}
}
}
if (!loaderNameLogged) {
LOG.log(logLevel, " Class loader name={}",
cl.getClass().getName());
}
logClassLoaderUrls(logLevel, cl.getParent());
}
..snip..
LOG.info("java.class.path={}",
System.getProperty("java.class.path"));
LOG.debug("Class loader URLs:");
logClassLoaderUrls(org.apache.logging.log4j.Level.DEBUG,
ctx.getClassLoader());
[4]
cat log-extract.txt | cut -c137- | sed -e
"s+/usr/local/tomcat++g" -e
"s+/usr/local/tomcat/webapps/exclaim++g" -e "s+\[[0-9]*\]++"
and I manually sorted the "sections" using vim, with the "sort"
command.
[5]
# Debian with OpenJDK
#FROM tomcat:9.0.58-jre11-openjdk AS tomcat_package
# AWS Linux 2 with Corretto
FROM tomcat:9.0.58-jdk11-corretto AS tomcat_package
# Set the timezone to Canada/Eastern
# For Debian
#RUN ln -sf /usr/share/zoneinfo/Canada/Eastern /etc/localtime &&
echo
"Canada/Eastern" > /etc/timezone && dpkg-reconfigure -f
noninteractive
tzdata
# For AWS Linux
RUN ln -sf /usr/share/zoneinfo/Canada/Eastern /etc/localtime &&
sed -i
's/\"UTC\"/\"Canada\/Eastern\"/' /etc/sysconfig/clock # and reboot,
but
will skip as container will be created and ran separately
# Install a postgresql client for testing and the required font
packages
needed by JasperReports
# For Debian:
#RUN echo "deb http://deb.debian.org/debian bullseye contrib
non-free"
/etc/apt/sources.list && apt-get update && apt-get install -y
postgresql-client ttf-mscorefonts-installer
# For AWS Linux:
RUN yum -q list installed epel-release.noarch &>/dev/null && echo
'epel-release already installed' || yum install -y
https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
RUN yum install -y cabextract xorg-x11-font-utils fontconfig
RUN rpm -q msttcore-fonts-installer-2.6-1 &>/dev/null && echo
'msttcore-fonts-installer-2.6-1 already installed' || rpm -i
https://iweb.dl.sourceforge.net/project/mscorefonts2/rpms/msttcore-fonts-installer-2.6-1.noarch.rpm
# Add a health-check script to wait for exclaim to be running --
allow at
least 10 minutes for it to start
COPY health_check.sh /usr/local/bin/
HEALTHCHECK --interval=10s --timeout=5s --start-period=10m
--retries=10
CMD /usr/local/bin/health_check.sh
# Set up the resource folders needed by application and make it a
# volume so it persists across container restarts
COPY resources/ /resources/
VOLUME /resources
# Map some of the Tomcat folders to volumes so they persist between
restarts of the container
VOLUME /usr/local/tomcat/logs
VOLUME /usr/local/tomcat/webapps
VOLUME /usr/local/tomcat/work
VOLUME /usr/local/tomcat/conf
# Enable connecting from anywhere for JPDA (req'd for Java 9+)
ENV JPDA_ADDRESS=*:8000
CMD ["catalina.sh","jpda","run"]
[6]
JDK on MacOS is:
openjdk version "11.0.13" 2021-10-19 LTS
OpenJDK Runtime Environment Corretto-11.0.13.8.1 (build
11.0.13+8-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.13.8.1 (build 11.0.13+8-LTS,
mixed
mode)
JDK on Windows is:
openjdk 11.0.14 2022-01-18 LTS
OpenJDK Runtime Environment Corretto-11.0.14.9.1 (build
11.0.14+9-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.14.9.1 (build 11.0.14+9-LTS,
mixed
mode)
JDK on Docker with AWS Linux 2 is:
openjdk 11.0.14 2022-01-18 LTS
OpenJDK Runtime Environment Corretto-11.0.14.9.1 (build
11.0.14+9-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.14.9.1 (build 11.0.14+9-LTS,
mixed
mode)
On Mon, Feb 7, 2022 at 9:24 PM Rob Sargent <rsarg...@xmission.com>
wrote:
On 2/7/22 19:13, Robert Turner wrote:
So, I've gone back and double-checked as much as I can (Tomcat
version, JDK
version), and the classpath, and I have identical classpaths in
both
environments (except the sort order of the URLs per "level" of
ClassLoader), and I've re-verified the behaviour:
- fails in the docker environment
- works locally
(Same WAR file on both).
I guess I'm in to one of the following approaches next:
- build a debug version of Tomcat, and step through the code
that
breaks
and see if I can figure out why (not that I have time to do this
of
course...but might be necessary)
- construct a trivial application reproduction, along with
docker
layout,
and see if anyone else can reproduce... (assuming anyone else has
time
to
do that of course...)
Anyone got any suggestions of what to look into next?
On Mon, Feb 7, 2022 at 5:05 PM Rob Sargent <
rsarg...@xmission.com>
wrote:
As you appear averse to sharing names of things, seems you will
need
to
do a lot of extra clean-up before you can share a docker image.
Make
sure you're clear on what NoClassDefFoundError indicates (present
at
compile time, absent at runtime) and how that translates to
docker-ness
(of which I know naught).
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
