And my apologies again, I was reading on the phone and didn't see the second half of Andy's reply. Thank you for a great summary!
/Andrew On 2021-12-14, at 17:06, Andrii Berezovskyi <[email protected]<mailto:[email protected]>> wrote: Just to correct myself, there is no other CVE in log4j. The new log4j release simply disabled JNDI functionality by default, making the fix more robust. My apologies. /Andrew ________________________________ From: Andy Seaborne <[email protected]<mailto:[email protected]>> Sent: Tuesday, December 14, 2021 4:46 PM To: [email protected]<mailto:[email protected]> Subject: Re: Log4j2 class loading problems when deploying Fuseki under Tomcat9 On 14/12/2021 14:35, Andrii Berezovskyi wrote: Hello Andy, Thank you so much, this indeed helped, I am now happily running Fuseki 4.3.1 under Tomcat 9. I used the 2.16.0 versions of the log4j libraries as they seem to address yet another CVE [1]. I placed the two JARs under /WEB-INF/lib inside the archive in case others on the list are wondering. Good to know that works. /Andrew [1]: https://lists.apache.org/thread/t72msv9cpxw9q5zw8rfkhx52v24z57f1 It's the same CVE 2021-44228. I understood it to be a better fix (remove all functionality) at the expense of being a more breaking change. We'll upgarde to 2.16.0. With 2.15.0 and Jena 4.3.1 server operators have to actively opt-in to using the suspect "lookup" functionality and by default don't get it. For Fuseki, that would be need a compromised machine. On 2021-12-14, 14:17, "Andy Seaborne" <[email protected]<mailto:[email protected]>> wrote: Hi Andrew, It looks like a 4.1->4.2 change - that's where log4j-api, and log4j-core stop appearing in the war file. So not related the log4j chnages this release. I haven't found a change that caused this yet. I noticed they were optional while clearing up yesterday yet still getting into the shaded jars which looked odd, so did PR 1113. It might be an order-dependent effect in the build or some maven upgrade changed and had an effect on the war-plugin. A fix should be to is to edit the war file (it's a zip file) and add the jars into WEB-INF/lib/. WEB-INF/lib/log4j-core-2.15.0.jar WEB-INF/lib/log4j-api-2.15.0.jar or add to the Tomcat server (untested). I've added it to the PR: https://github.com/apache/jena/pull/1131 ---- Some nearby questions if I may: Is the Tomcat installation running only Fuseki? Would it be reasonable to distribute the war file via a link from the download page or link in README? The apache-jena-fuseki file is getting big because it has two copies of Fuseki in it. Andy On 14/12/2021 11:40, Andrii Berezovskyi wrote: Hello, I decided to upgrade our Fuseki 3.17.0 running on Tomcat 9 to the latest version released yesterday (thank you, Andy, for your hard work and a late night release!). I ran into the “java.lang.NoClassDefFoundError: org/apache/logging/log4j/Logger” exception. The closest issue I found was https://stackoverflow.com/questions/20909446/caused-by-java-lang-noclassdeffounderror-org-apache-log4j-logger but I don’t think Fuseki WAR file has these problems. I am able to start the newest Fuseki server in the standalone mode without problems: FUSEKI_BASE=/etc/fuseki sudo /opt/apache-jena-fuseki-4.3.1/fuseki-server 21:57:56 INFO Server :: Apache Jena Fuseki 4.3.1 21:57:56 INFO Config :: FUSEKI_HOME=/opt/apache-jena-fuseki-4.3.1 21:57:56 INFO Config :: FUSEKI_BASE=/etc/fuseki 21:57:56 INFO Config :: Shiro file: file:///etc/fuseki/shiro.ini 21:57:56 INFO Config :: Load configuration: file:///etc/fuseki/configuration/a.ttl ... 21:57:57 INFO Config :: Load configuration: file:///etc/fuseki/configuration/z.ttl 21:57:57 INFO Server :: Configuration file: /etc/fuseki/config.ttl 21:57:57 INFO Server :: Path = /a ... 21:57:57 INFO Server :: Path = /z 21:57:57 INFO Server :: System 21:57:57 INFO Server :: Memory: 4.0 GiB 21:57:57 INFO Server :: Java: 11.0.11 21:57:57 INFO Server :: OS: Linux 5.4.0-91-generic amd64 21:57:57 INFO Server :: PID: 245617 21:57:58 INFO Server :: Started 2021/12/13 21:57:58 CET on port 3030 The Tomcat server is quite a vanilla tomcat9 package installation under Ubuntu 20.04 LTS. Do you have an idea where I should look at next? I checked out https://jena.apache.org/documentation/fuseki2/fuseki-webapp.html#fuseki-with-tomcat9-and-systemd and already have the systemd edits suggested. Thank you in advance! Tomcat startup info: 13-Dec-2021 21:10:46.889 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name: Apache Tomcat/9.0.31 (Ubuntu) 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built: Oct 20 2020 12:27:39 UTC 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.31.0 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name: Linux 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version: 5.4.0-91-generic 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture: amd64 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home: /usr/lib/jvm/java-11-openjdk-amd64 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version: 11.0.11+9-Ubuntu-0ubuntu2.20.04 13-Dec-2021 21:10:46.890 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor: Ubuntu 13-Dec-2021 21:10:46.891 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE: /var/lib/tomcat9 13-Dec-2021 21:10:46.891 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME: /usr/share/tomcat9 13-Dec-2021 21:10:46.939 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED 13-Dec-2021 21:10:46.939 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io<http://java.io>=ALL-UNNAMED 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/var/lib/tomcat9/conf/logging.properties 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs= 13-Dec-2021 21:10:46.940 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/var/lib/tomcat9 13-Dec-2021 21:10:46.941 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/share/tomcat9 13-Dec-2021 21:10:46.941 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/tmp 13-Dec-2021 21:10:46.941 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded APR based Apache Tomcat Native library [1.2.23] using APR version [1.6.5]. 13-Dec-2021 21:10:46.941 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true]. 13-Dec-2021 21:10:46.941 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true] 13-Dec-2021 21:10:46.945 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.1f 31 Mar 2020] 13-Dec-2021 21:10:47.402 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8081"] 13-Dec-2021 21:10:47.441 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [931] milliseconds 13-Dec-2021 21:10:47.526 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina] 13-Dec-2021 21:10:47.527 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.31 (Ubuntu)] Fuseki WAR deployment error: 13-Dec-2021 21:11:04.321 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/var/lib/tomcat9/webapps/fuseki.war] 13-Dec-2021 21:11:04.373 SEVERE [main] org.apache.catalina.startup.HostConfig.deployWAR Error deploying web application archive [/var/lib/tomcat9/webapps/fuseki.war] java.lang.IllegalStateException: Error starting child at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:720) at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690) at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705) at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:978) at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1849) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118) at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:773) at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:427) at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1576) at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:309) at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423) at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366) at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:936) at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:841) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75) at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140) at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909) at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardService.startInternal(StandardService.java:421) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.startup.Catalina.start(Catalina.java:633) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:343) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:478) Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/fuseki]] at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:440) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:198) at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717) ... 37 more Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/Logger at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:398) at org.apache.catalina.startup.WebappServiceLoader.loadServices(WebappServiceLoader.java:203) at org.apache.catalina.startup.WebappServiceLoader.load(WebappServiceLoader.java:174) at org.apache.catalina.startup.ContextConfig.processServletContainerInitializers(ContextConfig.java:1669) at org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1137) at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:774) at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:301) at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5051) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) ... 38 more Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.Logger at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1365) at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1188) ... 49 more 13-Dec-2021 21:11:04.374 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/var/lib/tomcat9/webapps/fuseki.war] has finished in [53] ms Best regards, Andrew
