Joe, Does it mean ConvertCSVToAvro will disappear too? Or you discontinue only a part of org.apache.nifi.processors.kite schema and the rest will be somehow moved to other schemas?
Regards, Mike From: Joe Witt <joe.w...@gmail.com> Sent: środa, 19 stycznia 2022 15:57 To: users@nifi.apache.org Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors We should discontinue Kite processors from the build at all in 1.16.0. Ill file a Jira Thanks On Wed, Jan 19, 2022 at 7:47 AM David Handermann <exceptionfact...@apache.org<mailto:exceptionfact...@apache.org>> wrote: Mike, Thanks for the additional details. According to the original output, the problem appears to be in the nifi-kite-processors module. The nifi-kite-processors module includes a dependency on hive-hcatalog-core, the version of which is based on the hive.version property. We should be able to add an exclusion for log4j-core within that dependency definition, which will allow it to compile with Hive 3.1.2. For the moment, if you don't need the Kite processors, you could skip building nifi-kite-nar, or you could manually add the following exclusion: <exclusion> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <exclusion> Regards, David Handermann On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski <michal.tomaszew...@cca.pl<mailto:michal.tomaszew...@cca.pl>> wrote: Compilation with -Dhive.version=3.1.0 makes the problem. mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 is not successful. We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive site. Also without success. Please note that without this parameter probably hive2 version is compiled into nifi and HiveStreaming is not working then. And hive 3 is most popular novadays. Regards, Mike -----Original Message----- From: Bryan Bende <bbe...@gmail.com<mailto:bbe...@gmail.com>> Sent: Wednesday, January 19, 2022 3:03 PM To: users@nifi.apache.org<mailto:users@nifi.apache.org> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors We did ban all use of log4j-core in favor of log4j-to-slf4j... https://issues.apache.org/jira/browse/NIFI-9483 With your build specifying the -D versions, it created a different classpath during your build than what existed during the normal build of the release. We should try to fix any issues like this though, so if you can figure out which dependency path is the issue, the fix is to add some exclusions and then to ensure that log4j-to-slf4j dependency is added. On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski <michal.tomaszew...@cca.pl<mailto:michal.tomaszew...@cca.pl>> wrote: > > OK, but there is information that log4j-core:jar:2.17.1 is blocked: > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed > with message: > > Found Banned Dependency: > org.apache.logging.log4j:log4j-core:jar:2.17.1 > > while this version is not impacted by log4j security vulnerability: > > https://logging.apache.org/log4j/2.x/security.html > > > > So for sure it shouldn’t be banned. > > > > Regards, > > Mike > > From: Pierre Villard > <pierre.villard...@gmail.com<mailto:pierre.villard...@gmail.com>> > Sent: Wednesday, January 19, 2022 1:11 PM > To: users@nifi.apache.org<mailto:users@nifi.apache.org> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors > > > > We recently changed the build to fail if it's including problematic > transitive dependencies for log4j. I think those are coming with the specific > versions of HBase/Hive you're using. Not a maven expert but I guess there is > a way to exclude the step "enforce-banned-dependencies" from the build. > > > > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski > <michal.tomaszew...@cca.pl<mailto:michal.tomaszew...@cca.pl>> a écrit : > > Hi Edward, > > Full mvn dependency:tree log enclosed. > > We’re compiling using command: > > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 > -Dhbase.version=2.0.2 > > > > Regards, > > Mike > > > > From: Edward Armes <edward.ar...@gmail.com<mailto:edward.ar...@gmail.com>> > Sent: Wednesday, January 19, 2022 12:34 PM > To: users@nifi.apache.org<mailto:users@nifi.apache.org> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors > > > > Hi Mike, > > > > Can you give the full output you get from mvn dependency:tree as we 15.2 and > 3 was done to remove log4j dependences from Nifi as a safety measure due the > recent log4shell issues. > > > > Thanks > > > > Edward > > > > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski > <michal.tomaszew...@cca.pl<mailto:michal.tomaszew...@cca.pl>> wrote: > > Hi, > > We are trying to compile nifi from sources. > > There is no problem with 1.15.1 compilation but in case of 1.15.2 and 1.15.3 > there are compilation errors. > > > > Can you suggest how to solve this problem? > > > > mvn dependency:tree has no errors: > > > > > > $ tail log1.txt > > [INFO] nifi-toolkit-flowfile-repo ......................... SUCCESS [ > 0.005 s] > > [INFO] nifi-toolkit-flowanalyzer .......................... SUCCESS [ > 0.008 s] > > [INFO] nifi-toolkit-assembly .............................. SUCCESS [ > 0.463 s] > > [INFO] nifi-toolkit-api ................................... SUCCESS [ > 0.023 s] > > [INFO] > ---------------------------------------------------------------------- > -- > > [INFO] BUILD SUCCESS > > [INFO] > ---------------------------------------------------------------------- > -- > > [INFO] Total time: 02:41 min > > [INFO] Finished at: 2022-01-19T08:51:46+01:00 > > > > > > Compilation of branch rel/nifi-1.15.3: > > > > [DEBUG] Adding ERROR message due to exception > > org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned > Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1 > > Use 'mvn dependency:tree' to locate the source of the banned dependencies. > > at > org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute > (AbstractBanDependencies.java:113) > > at org.apache.maven.plugins.enforcer.EnforceMojo.execute > (EnforceMojo.java:200) > > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > (DefaultBuildPluginManager.java:137) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:210) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:156) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:148) > > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec > t (LifecycleModuleBuilder.java:117) > > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > dBuilder$1.call (MultiThreadedBuilder.java:196) > > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > dBuilder$1.call (MultiThreadedBuilder.java:186) > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > at java.util.concurrent.Executors$RunnableAdapter.call > (Executors.java:511) > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1149) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run > (ThreadPoolExecutor.java:624) > > at java.lang.Thread.run (Thread.java:748) > > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed > with message: > > Found Banned Dependency: > org.apache.logging.log4j:log4j-core:jar:2.17.1 > > Use 'mvn dependency:tree' to locate the source of the banned dependencies. > > [INFO] > > > > …. > > > > [INFO] > ---------------------------------------------------------------------- > -- > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce > (enforce-banned-dependencies) on project nifi-kite-processors: Some > Enforcer rules have failed. Look above for specific messages > explaining why the rule failed. -> [Help 1] > > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce > (enforce-banned-dependencies) on project nifi-kite-processors: Some Enforcer > rules have failed. Look above for specific messages explaining why the rule > failed. > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:215) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:156) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:148) > > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec > t (LifecycleModuleBuilder.java:117) > > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > dBuilder$1.call (MultiThreadedBuilder.java:196) > > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > dBuilder$1.call (MultiThreadedBuilder.java:186) > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > at java.util.concurrent.Executors$RunnableAdapter.call > (Executors.java:511) > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1149) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run > (ThreadPoolExecutor.java:624) > > at java.lang.Thread.run (Thread.java:748) > > Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer > rules have failed. Look above for specific messages explaining why the rule > failed. > > at org.apache.maven.plugins.enforcer.EnforceMojo.execute > (EnforceMojo.java:255) > > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo > (DefaultBuildPluginManager.java:137) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:210) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:156) > > at org.apache.maven.lifecycle.internal.MojoExecutor.execute > (MojoExecutor.java:148) > > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec > t (LifecycleModuleBuilder.java:117) > > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > dBuilder$1.call (MultiThreadedBuilder.java:196) > > at > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade > dBuilder$1.call (MultiThreadedBuilder.java:186) > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > at java.util.concurrent.Executors$RunnableAdapter.call > (Executors.java:511) > > at java.util.concurrent.FutureTask.run (FutureTask.java:266) > > at java.util.concurrent.ThreadPoolExecutor.runWorker > (ThreadPoolExecutor.java:1149) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run > (ThreadPoolExecutor.java:624) > > at java.lang.Thread.run (Thread.java:748) > > [ERROR] > > [ERROR] > > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionExceptio > n > > [ERROR] > > [ERROR] After correcting the problems, you can resume the build with > the command > > [ERROR] mvn <args> -rf :nifi-kite-processors > > > > Regards, > > Mike
