Mike Yeah a bit more than two years ago we stopped including the nifi-kite-nar by default. https://cwiki.apache.org/confluence/display/NIFI/Migration+Guidance
That means the processors in it are no longer included by default. Since that time though we still build and published the nar to maven artifact repos. What we've done now is we've removed the source from our repo and will not publish the nar going forward. That said you'll be able to keep using the already published nars to do this as you need. Until we release a nifi 2.0 you should be safe from them breaking. But as Chris Sampsson just replied you almost certainly want to discontinue using them anyway and switch to using the record based processors. They are far more efficient, far more powerful, and are extremely actively developed and maintained. Thanks On Thu, Jan 20, 2022 at 11:32 AM Chris Sampson <chris.samp...@naimuri.com> wrote: > > If the kite nar is refined then that processor would indeed no longer be > present. > > A good alternative would be to use a Record based processor such as > ConvertRecord with a CSVReader and an AvroWriter. Such processors and > controller services typically offer much more powerful options than the older > processors such as those in the kite nar. > > > Cheers, > > Chris Sampson > > On Thu, 20 Jan 2022, 18:08 Michal Tomaszewski, <michal.tomaszew...@cca.pl> > wrote: >> >> Joe, >> >> Does it mean ConvertCSVToAvro will disappear too? >> >> Or you discontinue only a part of org.apache.nifi.processors.kite schema and >> the rest will be somehow moved to other schemas? >> >> >> >> Regards, >> >> Mike >> >> >> >> >> >> From: Joe Witt <joe.w...@gmail.com> >> Sent: środa, 19 stycznia 2022 15:57 >> To: users@nifi.apache.org >> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> >> >> >> We should discontinue Kite processors from the build at all in 1.16.0. >> >> >> >> Ill file a Jira >> >> >> >> Thanks >> >> >> >> On Wed, Jan 19, 2022 at 7:47 AM David Handermann >> <exceptionfact...@apache.org> wrote: >> >> Mike, >> >> >> >> Thanks for the additional details. According to the original output, the >> problem appears to be in the nifi-kite-processors module. The >> nifi-kite-processors module includes a dependency on hive-hcatalog-core, the >> version of which is based on the hive.version property. We should be able >> to add an exclusion for log4j-core within that dependency definition, which >> will allow it to compile with Hive 3.1.2. For the moment, if you don't need >> the Kite processors, you could skip building nifi-kite-nar, or you could >> manually add the following exclusion: >> >> <exclusion> >> >> <groupId>org.apache.logging.log4j</groupId> >> >> <artifactId>log4j-core</artifactId> >> >> <exclusion> >> >> Regards, >> >> David Handermann >> >> >> >> On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski >> <michal.tomaszew...@cca.pl> wrote: >> >> Compilation with -Dhive.version=3.1.0 makes the problem. >> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 >> is not successful. >> We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive >> site. Also without success. >> >> Please note that without this parameter probably hive2 version is compiled >> into nifi and HiveStreaming is not working then. >> And hive 3 is most popular novadays. >> >> Regards, >> Mike >> >> -----Original Message----- >> From: Bryan Bende <bbe...@gmail.com> >> Sent: Wednesday, January 19, 2022 3:03 PM >> To: users@nifi.apache.org >> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> >> We did ban all use of log4j-core in favor of log4j-to-slf4j... >> >> https://issues.apache.org/jira/browse/NIFI-9483 >> >> With your build specifying the -D versions, it created a different classpath >> during your build than what existed during the normal build of the release. >> >> We should try to fix any issues like this though, so if you can figure out >> which dependency path is the issue, the fix is to add some exclusions and >> then to ensure that log4j-to-slf4j dependency is added. >> >> On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski >> <michal.tomaszew...@cca.pl> wrote: >> > >> > OK, but there is information that log4j-core:jar:2.17.1 is blocked: >> > >> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies >> > failed with message: >> > >> > Found Banned Dependency: >> > org.apache.logging.log4j:log4j-core:jar:2.17.1 >> > >> > while this version is not impacted by log4j security vulnerability: >> > >> > https://logging.apache.org/log4j/2.x/security.html >> > >> > >> > >> > So for sure it shouldn’t be banned. >> > >> > >> > >> > Regards, >> > >> > Mike >> > >> > From: Pierre Villard <pierre.villard...@gmail.com> >> > Sent: Wednesday, January 19, 2022 1:11 PM >> > To: users@nifi.apache.org >> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> > >> > >> > >> > We recently changed the build to fail if it's including problematic >> > transitive dependencies for log4j. I think those are coming with the >> > specific versions of HBase/Hive you're using. Not a maven expert but I >> > guess there is a way to exclude the step "enforce-banned-dependencies" >> > from the build. >> > >> > >> > >> > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski >> > <michal.tomaszew...@cca.pl> a écrit : >> > >> > Hi Edward, >> > >> > Full mvn dependency:tree log enclosed. >> > >> > We’re compiling using command: >> > >> > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 >> > -Dhbase.version=2.0.2 >> > >> > >> > >> > Regards, >> > >> > Mike >> > >> > >> > >> > From: Edward Armes <edward.ar...@gmail.com> >> > Sent: Wednesday, January 19, 2022 12:34 PM >> > To: users@nifi.apache.org >> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors >> > >> > >> > >> > Hi Mike, >> > >> > >> > >> > Can you give the full output you get from mvn dependency:tree as we 15.2 >> > and 3 was done to remove log4j dependences from Nifi as a safety measure >> > due the recent log4shell issues. >> > >> > >> > >> > Thanks >> > >> > >> > >> > Edward >> > >> > >> > >> > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski >> > <michal.tomaszew...@cca.pl> wrote: >> > >> > Hi, >> > >> > We are trying to compile nifi from sources. >> > >> > There is no problem with 1.15.1 compilation but in case of 1.15.2 and >> > 1.15.3 there are compilation errors. >> > >> > >> > >> > Can you suggest how to solve this problem? >> > >> > >> > >> > mvn dependency:tree has no errors: >> > >> > >> > >> > >> > >> > $ tail log1.txt >> > >> > [INFO] nifi-toolkit-flowfile-repo ......................... SUCCESS [ >> > 0.005 s] >> > >> > [INFO] nifi-toolkit-flowanalyzer .......................... SUCCESS [ >> > 0.008 s] >> > >> > [INFO] nifi-toolkit-assembly .............................. SUCCESS [ >> > 0.463 s] >> > >> > [INFO] nifi-toolkit-api ................................... SUCCESS [ >> > 0.023 s] >> > >> > [INFO] >> > ---------------------------------------------------------------------- >> > -- >> > >> > [INFO] BUILD SUCCESS >> > >> > [INFO] >> > ---------------------------------------------------------------------- >> > -- >> > >> > [INFO] Total time: 02:41 min >> > >> > [INFO] Finished at: 2022-01-19T08:51:46+01:00 >> > >> > >> > >> > >> > >> > Compilation of branch rel/nifi-1.15.3: >> > >> > >> > >> > [DEBUG] Adding ERROR message due to exception >> > >> > org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned >> > Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1 >> > >> > Use 'mvn dependency:tree' to locate the source of the banned dependencies. >> > >> > at >> > org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute >> > (AbstractBanDependencies.java:113) >> > >> > at org.apache.maven.plugins.enforcer.EnforceMojo.execute >> > (EnforceMojo.java:200) >> > >> > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo >> > (DefaultBuildPluginManager.java:137) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:210) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:156) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:148) >> > >> > at >> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec >> > t (LifecycleModuleBuilder.java:117) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:196) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:186) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.Executors$RunnableAdapter.call >> > (Executors.java:511) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker >> > (ThreadPoolExecutor.java:1149) >> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run >> > (ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run (Thread.java:748) >> > >> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies >> > failed with message: >> > >> > Found Banned Dependency: >> > org.apache.logging.log4j:log4j-core:jar:2.17.1 >> > >> > Use 'mvn dependency:tree' to locate the source of the banned dependencies. >> > >> > [INFO] >> > >> > >> > >> > …. >> > >> > >> > >> > [INFO] >> > ---------------------------------------------------------------------- >> > -- >> > >> > [ERROR] Failed to execute goal >> > org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce >> > (enforce-banned-dependencies) on project nifi-kite-processors: Some >> > Enforcer rules have failed. Look above for specific messages >> > explaining why the rule failed. -> [Help 1] >> > >> > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute >> > goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce >> > (enforce-banned-dependencies) on project nifi-kite-processors: Some >> > Enforcer rules have failed. Look above for specific messages explaining >> > why the rule failed. >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:215) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:156) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:148) >> > >> > at >> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec >> > t (LifecycleModuleBuilder.java:117) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:196) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:186) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.Executors$RunnableAdapter.call >> > (Executors.java:511) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker >> > (ThreadPoolExecutor.java:1149) >> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run >> > (ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run (Thread.java:748) >> > >> > Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer >> > rules have failed. Look above for specific messages explaining why the >> > rule failed. >> > >> > at org.apache.maven.plugins.enforcer.EnforceMojo.execute >> > (EnforceMojo.java:255) >> > >> > at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo >> > (DefaultBuildPluginManager.java:137) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:210) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:156) >> > >> > at org.apache.maven.lifecycle.internal.MojoExecutor.execute >> > (MojoExecutor.java:148) >> > >> > at >> > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec >> > t (LifecycleModuleBuilder.java:117) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:196) >> > >> > at >> > org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade >> > dBuilder$1.call (MultiThreadedBuilder.java:186) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.Executors$RunnableAdapter.call >> > (Executors.java:511) >> > >> > at java.util.concurrent.FutureTask.run (FutureTask.java:266) >> > >> > at java.util.concurrent.ThreadPoolExecutor.runWorker >> > (ThreadPoolExecutor.java:1149) >> > >> > at java.util.concurrent.ThreadPoolExecutor$Worker.run >> > (ThreadPoolExecutor.java:624) >> > >> > at java.lang.Thread.run (Thread.java:748) >> > >> > [ERROR] >> > >> > [ERROR] >> > >> > [ERROR] For more information about the errors and possible solutions, >> > please read the following articles: >> > >> > [ERROR] [Help 1] >> > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionExceptio >> > n >> > >> > [ERROR] >> > >> > [ERROR] After correcting the problems, you can resume the build with >> > the command >> > >> > [ERROR] mvn <args> -rf :nifi-kite-processors >> > >> > >> > >> > Regards, >> > >> > Mike
