Mike, Thanks for the additional details. According to the original output, the problem appears to be in the nifi-kite-processors module. The nifi-kite-processors module includes a dependency on hive-hcatalog-core, the version of which is based on the hive.version property. We should be able to add an exclusion for log4j-core within that dependency definition, which will allow it to compile with Hive 3.1.2. For the moment, if you don't need the Kite processors, you could skip building nifi-kite-nar, or you could manually add the following exclusion:
<exclusion> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <exclusion> Regards, David Handermann On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski < michal.tomaszew...@cca.pl> wrote: > Compilation with -Dhive.version=3.1.0 makes the problem. > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 > is not successful. > We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive > site. Also without success. > > Please note that without this parameter probably hive2 version is compiled > into nifi and HiveStreaming is not working then. > And hive 3 is most popular novadays. > > Regards, > Mike > > -----Original Message----- > From: Bryan Bende <bbe...@gmail.com> > Sent: Wednesday, January 19, 2022 3:03 PM > To: users@nifi.apache.org > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors > > We did ban all use of log4j-core in favor of log4j-to-slf4j... > > https://issues.apache.org/jira/browse/NIFI-9483 > > With your build specifying the -D versions, it created a different > classpath during your build than what existed during the normal build of > the release. > > We should try to fix any issues like this though, so if you can figure out > which dependency path is the issue, the fix is to add some exclusions and > then to ensure that log4j-to-slf4j dependency is added. > > On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski < > michal.tomaszew...@cca.pl> wrote: > > > > OK, but there is information that log4j-core:jar:2.17.1 is blocked: > > > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies > failed with message: > > > > Found Banned Dependency: > > org.apache.logging.log4j:log4j-core:jar:2.17.1 > > > > while this version is not impacted by log4j security vulnerability: > > > > https://logging.apache.org/log4j/2.x/security.html > > > > > > > > So for sure it shouldn’t be banned. > > > > > > > > Regards, > > > > Mike > > > > From: Pierre Villard <pierre.villard...@gmail.com> > > Sent: Wednesday, January 19, 2022 1:11 PM > > To: users@nifi.apache.org > > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors > > > > > > > > We recently changed the build to fail if it's including problematic > transitive dependencies for log4j. I think those are coming with the > specific versions of HBase/Hive you're using. Not a maven expert but I > guess there is a way to exclude the step "enforce-banned-dependencies" from > the build. > > > > > > > > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski < > michal.tomaszew...@cca.pl> a écrit : > > > > Hi Edward, > > > > Full mvn dependency:tree log enclosed. > > > > We’re compiling using command: > > > > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 > > -Dhbase.version=2.0.2 > > > > > > > > Regards, > > > > Mike > > > > > > > > From: Edward Armes <edward.ar...@gmail.com> > > Sent: Wednesday, January 19, 2022 12:34 PM > > To: users@nifi.apache.org > > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors > > > > > > > > Hi Mike, > > > > > > > > Can you give the full output you get from mvn dependency:tree as we 15.2 > and 3 was done to remove log4j dependences from Nifi as a safety measure > due the recent log4shell issues. > > > > > > > > Thanks > > > > > > > > Edward > > > > > > > > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski < > michal.tomaszew...@cca.pl> wrote: > > > > Hi, > > > > We are trying to compile nifi from sources. > > > > There is no problem with 1.15.1 compilation but in case of 1.15.2 and > 1.15.3 there are compilation errors. > > > > > > > > Can you suggest how to solve this problem? > > > > > > > > mvn dependency:tree has no errors: > > > > > > > > > > > > $ tail log1.txt > > > > [INFO] nifi-toolkit-flowfile-repo ......................... SUCCESS [ > > 0.005 s] > > > > [INFO] nifi-toolkit-flowanalyzer .......................... SUCCESS [ > > 0.008 s] > > > > [INFO] nifi-toolkit-assembly .............................. SUCCESS [ > > 0.463 s] > > > > [INFO] nifi-toolkit-api ................................... SUCCESS [ > > 0.023 s] > > > > [INFO] > > ---------------------------------------------------------------------- > > -- > > > > [INFO] BUILD SUCCESS > > > > [INFO] > > ---------------------------------------------------------------------- > > -- > > > > [INFO] Total time: 02:41 min > > > > [INFO] Finished at: 2022-01-19T08:51:46+01:00 > > > > > > > > > > > > Compilation of branch rel/nifi-1.15.3: > > > > > > > > [DEBUG] Adding ERROR message due to exception > > > > org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned > > Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1 > > > > Use 'mvn dependency:tree' to locate the source of the banned > dependencies. > > > > at > > org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute > > (AbstractBanDependencies.java:113) > > > > at org.apache.maven.plugins.enforcer.EnforceMojo.execute > > (EnforceMojo.java:200) > > > > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > > (DefaultBuildPluginManager.java:137) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:210) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:156) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:148) > > > > at > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec > > t (LifecycleModuleBuilder.java:117) > > > > at > > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > > dBuilder$1.call (MultiThreadedBuilder.java:196) > > > > at > > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > > dBuilder$1.call (MultiThreadedBuilder.java:186) > > > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > > > at java.util.concurrent.Executors$RunnableAdapter.call > > (Executors.java:511) > > > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker > > (ThreadPoolExecutor.java:1149) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run > > (ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run (Thread.java:748) > > > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies > failed with message: > > > > Found Banned Dependency: > > org.apache.logging.log4j:log4j-core:jar:2.17.1 > > > > Use 'mvn dependency:tree' to locate the source of the banned > dependencies. > > > > [INFO] > > > > > > > > …. > > > > > > > > [INFO] > > ---------------------------------------------------------------------- > > -- > > > > [ERROR] Failed to execute goal > > org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce > > (enforce-banned-dependencies) on project nifi-kite-processors: Some > > Enforcer rules have failed. Look above for specific messages > > explaining why the rule failed. -> [Help 1] > > > > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to > execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce > (enforce-banned-dependencies) on project nifi-kite-processors: Some > Enforcer rules have failed. Look above for specific messages explaining why > the rule failed. > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:215) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:156) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:148) > > > > at > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec > > t (LifecycleModuleBuilder.java:117) > > > > at > > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > > dBuilder$1.call (MultiThreadedBuilder.java:196) > > > > at > > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > > dBuilder$1.call (MultiThreadedBuilder.java:186) > > > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > > > at java.util.concurrent.Executors$RunnableAdapter.call > > (Executors.java:511) > > > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker > > (ThreadPoolExecutor.java:1149) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run > > (ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run (Thread.java:748) > > > > Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer > rules have failed. Look above for specific messages explaining why the rule > failed. > > > > at org.apache.maven.plugins.enforcer.EnforceMojo.execute > > (EnforceMojo.java:255) > > > > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > > (DefaultBuildPluginManager.java:137) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:210) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:156) > > > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > > (MojoExecutor.java:148) > > > > at > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec > > t (LifecycleModuleBuilder.java:117) > > > > at > > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > > dBuilder$1.call (MultiThreadedBuilder.java:196) > > > > at > > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > > dBuilder$1.call (MultiThreadedBuilder.java:186) > > > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > > > at java.util.concurrent.Executors$RunnableAdapter.call > > (Executors.java:511) > > > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker > > (ThreadPoolExecutor.java:1149) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run > > (ThreadPoolExecutor.java:624) > > > > at java.lang.Thread.run (Thread.java:748) > > > > [ERROR] > > > > [ERROR] > > > > [ERROR] For more information about the errors and possible solutions, > please read the following articles: > > > > [ERROR] [Help 1] > > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionExceptio > > n > > > > [ERROR] > > > > [ERROR] After correcting the problems, you can resume the build with > > the command > > > > [ERROR] mvn <args> -rf :nifi-kite-processors > > > > > > > > Regards, > > > > Mike >
