Re: Nifi 1.15.2 and 1.15.3 compilation errors

[David Handermann]

Mike,

You're welcome! The nifi-kite-bundle has been removed from the main branch
of Apache NiFi as of the following commit, so this shouldn't be a problem
for future builds.

https://github.com/apache/nifi/commit/8edb5faac9ff469e4c810891c30892962ed83b29

Regards,
David Handermann

On Wed, Jan 19, 2022 at 10:56 AM Michal Tomaszewski <
michal.tomaszew...@cca.pl> wrote:

> David,
>
> Thank you for your help.
>
> We made a change and compilation was successful.
>
>
>
> Regards,
>
> Mike
>
>
>
> *From:* David Handermann 
> *Sent:* środa, 19 stycznia 2022 15:47
> *To:* users@nifi.apache.org
> *Subject:* Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> Mike,
>
>
>
> Thanks for the additional details. According to the original output, the
> problem appears to be in the nifi-kite-processors module.  The
> nifi-kite-processors module includes a dependency on hive-hcatalog-core,
> the version of which is based on the hive.version property.  We should be
> able to add an exclusion for log4j-core within that dependency definition,
> which will allow it to compile with Hive 3.1.2.  For the moment, if you
> don't need the Kite processors, you could skip building nifi-kite-nar, or
> you could manually add the following exclusion:
>
> 
>
>   org.apache.logging.log4j
>
>   log4j-core
>
> 
>
> Regards,
>
> David Handermann
>
>
>
> On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
>
> Compilation with -Dhive.version=3.1.0 makes the problem.
> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
> is not successful.
> We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive
> site. Also without success.
>
> Please note that without this parameter probably hive2 version is compiled
> into nifi and HiveStreaming is not working then.
> And hive 3 is most popular novadays.
>
> Regards,
> Mike
>
> -Original Message-
> From: Bryan Bende 
> Sent: Wednesday, January 19, 2022 3:03 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
> We did ban all use of log4j-core in favor of log4j-to-slf4j...
>
> https://issues.apache.org/jira/browse/NIFI-9483
>
> With your build specifying the -D versions, it created a different
> classpath during your build than what existed during the normal build of
> the release.
>
> We should try to fix any issues like this though, so if you can figure out
> which dependency path is the issue, the fix is to add some exclusions and
> then to ensure that log4j-to-slf4j dependency is added.
>
> On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
> >
> > OK, but there is information that log4j-core:jar:2.17.1 is blocked:
> >
> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies
> failed with message:
> >
> > Found Banned Dependency:
> > org.apache.logging.log4j:log4j-core:jar:2.17.1
> >
> > while this version is not impacted by log4j security vulnerability:
> >
> > https://logging.apache.org/log4j/2.x/security.html
> >
> >
> >
> > So for sure it shouldn’t be banned.
> >
> >
> >
> > Regards,
> >
> > Mike
> >
> > From: Pierre Villard 
> > Sent: Wednesday, January 19, 2022 1:11 PM
> > To: users@nifi.apache.org
> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
> >
> >
> >
> > We recently changed the build to fail if it's including problematic
> transitive dependencies for log4j. I think those are coming with the
> specific versions of HBase/Hive you're using. Not a maven expert but I
> guess there is a way to exclude the step "enforce-banned-dependencies" from
> the build.
> >
> >
> >
> > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski <
> michal.tomaszew...@cca.pl> a écrit :
> >
> > Hi Edward,
> >
> > Full mvn dependency:tree log enclosed.
> >
> > We’re compiling using command:
> >
> > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
> > -Dhbase.version=2.0.2
> >
> >
> >
> > Regards,
> >
> > Mike
> >
> >
> >
> > From: Edward Armes 
> > Sent: Wednesday, January 19, 2022 12:34 PM
> > To: users@nifi.apache.org
> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
> >
> >
> >
> > Hi Mike,
> >
> >
> >
> > Can you give the full output you get from mvn dependency:tree as we 15.2
> and 3 was done to remove log4j dependences from Nifi as a safety measure
> due the recent log4shell issues.
> >
> >
> >
> > Thanks
> >
> >
> >
> > Edward
> >
> >
> >
> > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
> >
> > Hi,
> >
> > We are trying to compile nifi from sources.
> >
> > There is no problem with 1.15.1 compilation but in case of 1.15.2 and
> 1.15.3 there are compilation errors.
> >
> >
> >
> > Can you suggest how to solve this problem?
> >
> >
> >
> > mvn dependency:tree has no errors:
> >
> >
> >
> >
> >
> > $ tail log1.txt
> >
> > [INFO] nifi-toolkit-flowfile-repo . SUCCESS [
> > 0.005 s]
> >
> > [INFO] nifi-toolkit-flowanalyzer .. 

RE: Nifi 1.15.2 and 1.15.3 compilation errors

[Michal Tomaszewski]

David,
Thank you for your help.
We made a change and compilation was successful.

Regards,
Mike

From: David Handermann 
Sent: środa, 19 stycznia 2022 15:47
To: users@nifi.apache.org
Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors

Mike,

Thanks for the additional details. According to the original output, the 
problem appears to be in the nifi-kite-processors module.  The 
nifi-kite-processors module includes a dependency on hive-hcatalog-core, the 
version of which is based on the hive.version property.  We should be able to 
add an exclusion for log4j-core within that dependency definition, which will 
allow it to compile with Hive 3.1.2.  For the moment, if you don't need the 
Kite processors, you could skip building nifi-kite-nar, or you could manually 
add the following exclusion:

  org.apache.logging.log4j
  log4j-core

Regards,
David Handermann

On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski 
mailto:michal.tomaszew...@cca.pl>> wrote:
Compilation with -Dhive.version=3.1.0 makes the problem.
mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
is not successful.
We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive site. 
Also without success.

Please note that without this parameter probably hive2 version is compiled into 
nifi and HiveStreaming is not working then.
And hive 3 is most popular novadays.

Regards,
Mike

-Original Message-
From: Bryan Bende mailto:bbe...@gmail.com>>
Sent: Wednesday, January 19, 2022 3:03 PM
To: users@nifi.apache.org
Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors

We did ban all use of log4j-core in favor of log4j-to-slf4j...

https://issues.apache.org/jira/browse/NIFI-9483

With your build specifying the -D versions, it created a different classpath 
during your build than what existed during the normal build of the release.

We should try to fix any issues like this though, so if you can figure out 
which dependency path is the issue, the fix is to add some exclusions and then 
to ensure that log4j-to-slf4j dependency is added.

On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski 
mailto:michal.tomaszew...@cca.pl>> wrote:
>
> OK, but there is information that log4j-core:jar:2.17.1 is blocked:
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
> with message:
>
> Found Banned Dependency:
> org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> while this version is not impacted by log4j security vulnerability:
>
> https://logging.apache.org/log4j/2.x/security.html
>
>
>
> So for sure it shouldn’t be banned.
>
>
>
> Regards,
>
> Mike
>
> From: Pierre Villard 
> mailto:pierre.villard...@gmail.com>>
> Sent: Wednesday, January 19, 2022 1:11 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> We recently changed the build to fail if it's including problematic 
> transitive dependencies for log4j. I think those are coming with the specific 
> versions of HBase/Hive you're using. Not a maven expert but I guess there is 
> a way to exclude the step "enforce-banned-dependencies" from the build.
>
>
>
> Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski 
> mailto:michal.tomaszew...@cca.pl>> a écrit :
>
> Hi Edward,
>
> Full mvn dependency:tree log enclosed.
>
> We’re compiling using command:
>
> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
> -Dhbase.version=2.0.2
>
>
>
> Regards,
>
> Mike
>
>
>
> From: Edward Armes mailto:edward.ar...@gmail.com>>
> Sent: Wednesday, January 19, 2022 12:34 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> Hi Mike,
>
>
>
> Can you give the full output you get from mvn dependency:tree as we 15.2 and 
> 3 was done to remove log4j dependences from Nifi as a safety measure due the 
> recent log4shell issues.
>
>
>
> Thanks
>
>
>
> Edward
>
>
>
> On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski 
> mailto:michal.tomaszew...@cca.pl>> wrote:
>
> Hi,
>
> We are trying to compile nifi from sources.
>
> There is no problem with 1.15.1 compilation but in case of 1.15.2 and 1.15.3 
> there are compilation errors.
>
>
>
> Can you suggest how to solve this problem?
>
>
>
> mvn dependency:tree has no errors:
>
>
>
>
>
> $ tail log1.txt
>
> [INFO] nifi-toolkit-flowfile-repo . SUCCESS [
> 0.005 s]
>
> [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [
> 0.008 s]
>
> [INFO] nifi-toolkit-assembly .. SUCCESS [
> 0.463 s]
>
> [INFO] nifi-toolkit-api ... SUCCESS [
> 0.023 s]
>
> [INFO]
> --
> --
>
> [INFO] BUILD SUCCESS
>
> [INFO]
> --
> --
>
> [INFO] Total time:  02:41 min
>
> [INFO] Finished at: 2022-01-19T08:51:46+01:00
>
>
>
>
>
> Compilation of 

Re: NiFi V1.15.2 Conversion to Avro from JSON

[Mark Payne]

Thanks Nathan. I created a Jira [1] for this. I was able to easily replicate 
with your template. Thanks for including that. Just put up a Pull Request for 
it, as well.

Thanks
-Mark



[1] https://issues.apache.org/jira/browse/NIFI-9594


On Jan 18, 2022, at 4:49 AM, 
nathan.engl...@bt.com wrote:

Hi There,

We've recently upgraded to NiFi V1.15.2 from V1.12.1 and have noticed a 
difference between the JSON Tree Reader Controller Service, which is a slight 
annoyance.

When we transform our records, we have some fields that will exist or won't 
after the transformation, depending on whether there is a value for the field. 
Previously with NiFi v1.12.1, when we would convert these records to Avro using 
the Avro Writer Controller and JSON Tree Reader Services (Both with Schemas 
Set), these fields would be created and set with the default value in the Avro 
Schema. But in NiFi v1.15.2, the conversion fails with "null of boolean in 
field field_name of Avro.Schema.Name", it doesn't matter which field, type or 
default value. It all seems to fail if the field doesn't exist.

Interestingly, there isn't an issue when I set the JSON Tree Reader to Infer 
Schema instead of 'use Schema Name' with a Schema Registry.

Annoyingly before we convert to Avro from JSON, we go through a Validate Schema 
Processor, which uses the same Schema, and JSON Tree Reader but instead uses a 
JSON Writer and successfully validates the record against Schema!

It seems similar to the NIFI-9335 issue on Jira [1], but maybe with the JSON 
Tree Reader Controller service instead?

I have attached a sample flow to reproduce the issue.

I didn't know if it was worth me raising a bug ticket?

Kind Regards,

Nathan

[1] https://issues.apache.org/jira/browse/NIFI-9335

RE: DistributedMapCacheServer not included when making copy of ProcessGroup

[Isha Lamboo]

Sure, I will create a ticket

Thanks for looking into it!

Isha

Van: Mark Payne 
Verzonden: woensdag 19 januari 2022 15:56
Aan: users 
Onderwerp: Re: DistributedMapCacheServer not included when making copy of 
ProcessGroup

OK thanks. So that is not intended to happen. It’s a bug. Good news is that I 
was able to easily replicate by creating a new process group, just adding the 
distributed map cache server, and then copy & paste. Not sure why it would do 
that, but we can certainly look into it. You mind filing a Jira for it?

Thanks
-Mark



On Jan 19, 2022, at 9:52 AM, Isha Lamboo 
mailto:isha.lam...@virtualsciences.nl>> wrote:

Hi Mark,

It happens when using copy & paste to duplicate the Process Group on the canvas.

We’ve since committed the fixed process group to the registry from the dev 
server and the DistributedMapCacheServer is created successfully when changing 
version on the production server, so the component must be present in the 
flowdefinition.

Thanks,

Isha

Van: Mark Payne mailto:marka...@hotmail.com>>
Verzonden: woensdag 19 januari 2022 15:46
Aan: users@nifi.apache.org
Onderwerp: Re: DistributedMapCacheServer not included when making copy of 
ProcessGroup

Hi Isha,

When you say it was “copied” what precisely do you mean? Did you copy & paste 
the Process Group? Did you create a template and then instantiate it? Did you 
download the flow definition and then upload it again? There are a few 
different mechanism that you can use for duplicating process groups.

Thanks
-Mark




On Jan 19, 2022, at 7:25 AM, Isha Lamboo 
mailto:isha.lam...@virtualsciences.nl>> wrote:

Hi all,

I’ve encountered an issue today where a DistributedMapCacheServer controller 
service was missing from a process group with a flow.
It turns out that during development the entire process group was copied to 
test some changes and then the copy was used from there on.

A bit of testing shows that in NiFi 1.15.2 the DistributedMapCacheServer is 
silently excluded from the copy operation. I found no warnings or errors in the 
log concerning the operation.

I can imagine this is done on purpose since a copy using the same port could 
never work, but I would expect either a warning, or for the validation/enabling 
of the copy to fail.



Is this by design and are the DistributedMapCacheServer services supposed to be 
created at the root level, or can I create a bug/improvement ticket for this?

Kind regards,

Isha Lamboo

Re: Nifi 1.15.2 and 1.15.3 compilation errors

[Joe Witt]

We should discontinue Kite processors from the build at all in 1.16.0.

Ill file a Jira

Thanks

On Wed, Jan 19, 2022 at 7:47 AM David Handermann <
exceptionfact...@apache.org> wrote:

> Mike,
>
> Thanks for the additional details. According to the original output, the
> problem appears to be in the nifi-kite-processors module.  The
> nifi-kite-processors module includes a dependency on hive-hcatalog-core,
> the version of which is based on the hive.version property.  We should be
> able to add an exclusion for log4j-core within that dependency definition,
> which will allow it to compile with Hive 3.1.2.  For the moment, if you
> don't need the Kite processors, you could skip building nifi-kite-nar, or
> you could manually add the following exclusion:
>
> 
>   org.apache.logging.log4j
>   log4j-core
> 
>
> Regards,
> David Handermann
>
> On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
>
>> Compilation with -Dhive.version=3.1.0 makes the problem.
>> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
>> is not successful.
>> We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive
>> site. Also without success.
>>
>> Please note that without this parameter probably hive2 version is
>> compiled into nifi and HiveStreaming is not working then.
>> And hive 3 is most popular novadays.
>>
>> Regards,
>> Mike
>>
>> -Original Message-
>> From: Bryan Bende 
>> Sent: Wednesday, January 19, 2022 3:03 PM
>> To: users@nifi.apache.org
>> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>>
>> We did ban all use of log4j-core in favor of log4j-to-slf4j...
>>
>> https://issues.apache.org/jira/browse/NIFI-9483
>>
>> With your build specifying the -D versions, it created a different
>> classpath during your build than what existed during the normal build of
>> the release.
>>
>> We should try to fix any issues like this though, so if you can figure
>> out which dependency path is the issue, the fix is to add some exclusions
>> and then to ensure that log4j-to-slf4j dependency is added.
>>
>> On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski <
>> michal.tomaszew...@cca.pl> wrote:
>> >
>> > OK, but there is information that log4j-core:jar:2.17.1 is blocked:
>> >
>> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies
>> failed with message:
>> >
>> > Found Banned Dependency:
>> > org.apache.logging.log4j:log4j-core:jar:2.17.1
>> >
>> > while this version is not impacted by log4j security vulnerability:
>> >
>> > https://logging.apache.org/log4j/2.x/security.html
>> >
>> >
>> >
>> > So for sure it shouldn’t be banned.
>> >
>> >
>> >
>> > Regards,
>> >
>> > Mike
>> >
>> > From: Pierre Villard 
>> > Sent: Wednesday, January 19, 2022 1:11 PM
>> > To: users@nifi.apache.org
>> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>> >
>> >
>> >
>> > We recently changed the build to fail if it's including problematic
>> transitive dependencies for log4j. I think those are coming with the
>> specific versions of HBase/Hive you're using. Not a maven expert but I
>> guess there is a way to exclude the step "enforce-banned-dependencies" from
>> the build.
>> >
>> >
>> >
>> > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski <
>> michal.tomaszew...@cca.pl> a écrit :
>> >
>> > Hi Edward,
>> >
>> > Full mvn dependency:tree log enclosed.
>> >
>> > We’re compiling using command:
>> >
>> > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
>> > -Dhbase.version=2.0.2
>> >
>> >
>> >
>> > Regards,
>> >
>> > Mike
>> >
>> >
>> >
>> > From: Edward Armes 
>> > Sent: Wednesday, January 19, 2022 12:34 PM
>> > To: users@nifi.apache.org
>> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>> >
>> >
>> >
>> > Hi Mike,
>> >
>> >
>> >
>> > Can you give the full output you get from mvn dependency:tree as we
>> 15.2 and 3 was done to remove log4j dependences from Nifi as a safety
>> measure due the recent log4shell issues.
>> >
>> >
>> >
>> > Thanks
>> >
>> >
>> >
>> > Edward
>> >
>> >
>> >
>> > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski <
>> michal.tomaszew...@cca.pl> wrote:
>> >
>> > Hi,
>> >
>> > We are trying to compile nifi from sources.
>> >
>> > There is no problem with 1.15.1 compilation but in case of 1.15.2 and
>> 1.15.3 there are compilation errors.
>> >
>> >
>> >
>> > Can you suggest how to solve this problem?
>> >
>> >
>> >
>> > mvn dependency:tree has no errors:
>> >
>> >
>> >
>> >
>> >
>> > $ tail log1.txt
>> >
>> > [INFO] nifi-toolkit-flowfile-repo . SUCCESS [
>> > 0.005 s]
>> >
>> > [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [
>> > 0.008 s]
>> >
>> > [INFO] nifi-toolkit-assembly .. SUCCESS [
>> > 0.463 s]
>> >
>> > [INFO] nifi-toolkit-api ... SUCCESS [
>> > 0.023 s]
>> >
>> > [INFO]
>> > --
>> > --
>> >
>> > [INFO] BUILD SUCCESS
>> 

Re: DistributedMapCacheServer not included when making copy of ProcessGroup

[Mark Payne]

OK thanks. So that is not intended to happen. It’s a bug. Good news is that I 
was able to easily replicate by creating a new process group, just adding the 
distributed map cache server, and then copy & paste. Not sure why it would do 
that, but we can certainly look into it. You mind filing a Jira for it?

Thanks
-Mark


On Jan 19, 2022, at 9:52 AM, Isha Lamboo 
mailto:isha.lam...@virtualsciences.nl>> wrote:

Hi Mark,

It happens when using copy & paste to duplicate the Process Group on the canvas.

We’ve since committed the fixed process group to the registry from the dev 
server and the DistributedMapCacheServer is created successfully when changing 
version on the production server, so the component must be present in the 
flowdefinition.

Thanks,

Isha

Van: Mark Payne mailto:marka...@hotmail.com>>
Verzonden: woensdag 19 januari 2022 15:46
Aan: users@nifi.apache.org
Onderwerp: Re: DistributedMapCacheServer not included when making copy of 
ProcessGroup

Hi Isha,

When you say it was “copied” what precisely do you mean? Did you copy & paste 
the Process Group? Did you create a template and then instantiate it? Did you 
download the flow definition and then upload it again? There are a few 
different mechanism that you can use for duplicating process groups.

Thanks
-Mark



On Jan 19, 2022, at 7:25 AM, Isha Lamboo 
mailto:isha.lam...@virtualsciences.nl>> wrote:

Hi all,

I’ve encountered an issue today where a DistributedMapCacheServer controller 
service was missing from a process group with a flow.
It turns out that during development the entire process group was copied to 
test some changes and then the copy was used from there on.

A bit of testing shows that in NiFi 1.15.2 the DistributedMapCacheServer is 
silently excluded from the copy operation. I found no warnings or errors in the 
log concerning the operation.

I can imagine this is done on purpose since a copy using the same port could 
never work, but I would expect either a warning, or for the validation/enabling 
of the copy to fail.


Is this by design and are the DistributedMapCacheServer services supposed to be 
created at the root level, or can I create a bug/improvement ticket for this?

Kind regards,

Isha Lamboo

RE: DistributedMapCacheServer not included when making copy of ProcessGroup

[Isha Lamboo]

Hi Mark,

It happens when using copy & paste to duplicate the Process Group on the canvas.

We’ve since committed the fixed process group to the registry from the dev 
server and the DistributedMapCacheServer is created successfully when changing 
version on the production server, so the component must be present in the 
flowdefinition.

Thanks,

Isha

Van: Mark Payne 
Verzonden: woensdag 19 januari 2022 15:46
Aan: users@nifi.apache.org
Onderwerp: Re: DistributedMapCacheServer not included when making copy of 
ProcessGroup

Hi Isha,

When you say it was “copied” what precisely do you mean? Did you copy & paste 
the Process Group? Did you create a template and then instantiate it? Did you 
download the flow definition and then upload it again? There are a few 
different mechanism that you can use for duplicating process groups.

Thanks
-Mark



On Jan 19, 2022, at 7:25 AM, Isha Lamboo 
mailto:isha.lam...@virtualsciences.nl>> wrote:

Hi all,

I’ve encountered an issue today where a DistributedMapCacheServer controller 
service was missing from a process group with a flow.
It turns out that during development the entire process group was copied to 
test some changes and then the copy was used from there on.

A bit of testing shows that in NiFi 1.15.2 the DistributedMapCacheServer is 
silently excluded from the copy operation. I found no warnings or errors in the 
log concerning the operation.

I can imagine this is done on purpose since a copy using the same port could 
never work, but I would expect either a warning, or for the validation/enabling 
of the copy to fail.


Is this by design and are the DistributedMapCacheServer services supposed to be 
created at the root level, or can I create a bug/improvement ticket for this?

Kind regards,

Isha Lamboo

Re: Nifi 1.15.2 and 1.15.3 compilation errors

[David Handermann]

Mike,

Thanks for the additional details. According to the original output, the
problem appears to be in the nifi-kite-processors module.  The
nifi-kite-processors module includes a dependency on hive-hcatalog-core,
the version of which is based on the hive.version property.  We should be
able to add an exclusion for log4j-core within that dependency definition,
which will allow it to compile with Hive 3.1.2.  For the moment, if you
don't need the Kite processors, you could skip building nifi-kite-nar, or
you could manually add the following exclusion:


  org.apache.logging.log4j
  log4j-core


Regards,
David Handermann

On Wed, Jan 19, 2022 at 8:27 AM Michal Tomaszewski <
michal.tomaszew...@cca.pl> wrote:

> Compilation with -Dhive.version=3.1.0 makes the problem.
> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
> is not successful.
> We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive
> site. Also without success.
>
> Please note that without this parameter probably hive2 version is compiled
> into nifi and HiveStreaming is not working then.
> And hive 3 is most popular novadays.
>
> Regards,
> Mike
>
> -Original Message-
> From: Bryan Bende 
> Sent: Wednesday, January 19, 2022 3:03 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
> We did ban all use of log4j-core in favor of log4j-to-slf4j...
>
> https://issues.apache.org/jira/browse/NIFI-9483
>
> With your build specifying the -D versions, it created a different
> classpath during your build than what existed during the normal build of
> the release.
>
> We should try to fix any issues like this though, so if you can figure out
> which dependency path is the issue, the fix is to add some exclusions and
> then to ensure that log4j-to-slf4j dependency is added.
>
> On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
> >
> > OK, but there is information that log4j-core:jar:2.17.1 is blocked:
> >
> > [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies
> failed with message:
> >
> > Found Banned Dependency:
> > org.apache.logging.log4j:log4j-core:jar:2.17.1
> >
> > while this version is not impacted by log4j security vulnerability:
> >
> > https://logging.apache.org/log4j/2.x/security.html
> >
> >
> >
> > So for sure it shouldn’t be banned.
> >
> >
> >
> > Regards,
> >
> > Mike
> >
> > From: Pierre Villard 
> > Sent: Wednesday, January 19, 2022 1:11 PM
> > To: users@nifi.apache.org
> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
> >
> >
> >
> > We recently changed the build to fail if it's including problematic
> transitive dependencies for log4j. I think those are coming with the
> specific versions of HBase/Hive you're using. Not a maven expert but I
> guess there is a way to exclude the step "enforce-banned-dependencies" from
> the build.
> >
> >
> >
> > Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski <
> michal.tomaszew...@cca.pl> a écrit :
> >
> > Hi Edward,
> >
> > Full mvn dependency:tree log enclosed.
> >
> > We’re compiling using command:
> >
> > mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
> > -Dhbase.version=2.0.2
> >
> >
> >
> > Regards,
> >
> > Mike
> >
> >
> >
> > From: Edward Armes 
> > Sent: Wednesday, January 19, 2022 12:34 PM
> > To: users@nifi.apache.org
> > Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
> >
> >
> >
> > Hi Mike,
> >
> >
> >
> > Can you give the full output you get from mvn dependency:tree as we 15.2
> and 3 was done to remove log4j dependences from Nifi as a safety measure
> due the recent log4shell issues.
> >
> >
> >
> > Thanks
> >
> >
> >
> > Edward
> >
> >
> >
> > On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
> >
> > Hi,
> >
> > We are trying to compile nifi from sources.
> >
> > There is no problem with 1.15.1 compilation but in case of 1.15.2 and
> 1.15.3 there are compilation errors.
> >
> >
> >
> > Can you suggest how to solve this problem?
> >
> >
> >
> > mvn dependency:tree has no errors:
> >
> >
> >
> >
> >
> > $ tail log1.txt
> >
> > [INFO] nifi-toolkit-flowfile-repo . SUCCESS [
> > 0.005 s]
> >
> > [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [
> > 0.008 s]
> >
> > [INFO] nifi-toolkit-assembly .. SUCCESS [
> > 0.463 s]
> >
> > [INFO] nifi-toolkit-api ... SUCCESS [
> > 0.023 s]
> >
> > [INFO]
> > --
> > --
> >
> > [INFO] BUILD SUCCESS
> >
> > [INFO]
> > --
> > --
> >
> > [INFO] Total time:  02:41 min
> >
> > [INFO] Finished at: 2022-01-19T08:51:46+01:00
> >
> >
> >
> >
> >
> > Compilation of branch rel/nifi-1.15.3:
> >
> >
> >
> > [DEBUG] Adding ERROR message due to exception
> >
> > org.apache.maven.enforcer.rule.api.EnforcerRuleException: 

Re: DistributedMapCacheServer not included when making copy of ProcessGroup

[Mark Payne]

Hi Isha,

When you say it was “copied” what precisely do you mean? Did you copy & paste 
the Process Group? Did you create a template and then instantiate it? Did you 
download the flow definition and then upload it again? There are a few 
different mechanism that you can use for duplicating process groups.

Thanks
-Mark


On Jan 19, 2022, at 7:25 AM, Isha Lamboo 
mailto:isha.lam...@virtualsciences.nl>> wrote:

Hi all,

I’ve encountered an issue today where a DistributedMapCacheServer controller 
service was missing from a process group with a flow.
It turns out that during development the entire process group was copied to 
test some changes and then the copy was used from there on.

A bit of testing shows that in NiFi 1.15.2 the DistributedMapCacheServer is 
silently excluded from the copy operation. I found no warnings or errors in the 
log concerning the operation.

I can imagine this is done on purpose since a copy using the same port could 
never work, but I would expect either a warning, or for the validation/enabling 
of the copy to fail.

Is this by design and are the DistributedMapCacheServer services supposed to be 
created at the root level, or can I create a bug/improvement ticket for this?

Kind regards,

Isha Lamboo

RE: Nifi 1.15.2 and 1.15.3 compilation errors

[Michal Tomaszewski]

Compilation with -Dhive.version=3.1.0 makes the problem.
mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
is not successful.
We also tested -Dhive.version=3.1.2, that is most recent on Apache Hive site. 
Also without success.

Please note that without this parameter probably hive2 version is compiled into 
nifi and HiveStreaming is not working then.
And hive 3 is most popular novadays.

Regards,
Mike

-Original Message-
From: Bryan Bende  
Sent: Wednesday, January 19, 2022 3:03 PM
To: users@nifi.apache.org
Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors

We did ban all use of log4j-core in favor of log4j-to-slf4j...

https://issues.apache.org/jira/browse/NIFI-9483

With your build specifying the -D versions, it created a different classpath 
during your build than what existed during the normal build of the release.

We should try to fix any issues like this though, so if you can figure out 
which dependency path is the issue, the fix is to add some exclusions and then 
to ensure that log4j-to-slf4j dependency is added.

On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski  
wrote:
>
> OK, but there is information that log4j-core:jar:2.17.1 is blocked:
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
> with message:
>
> Found Banned Dependency: 
> org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> while this version is not impacted by log4j security vulnerability:
>
> https://logging.apache.org/log4j/2.x/security.html
>
>
>
> So for sure it shouldn’t be banned.
>
>
>
> Regards,
>
> Mike
>
> From: Pierre Villard 
> Sent: Wednesday, January 19, 2022 1:11 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> We recently changed the build to fail if it's including problematic 
> transitive dependencies for log4j. I think those are coming with the specific 
> versions of HBase/Hive you're using. Not a maven expert but I guess there is 
> a way to exclude the step "enforce-banned-dependencies" from the build.
>
>
>
> Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski  
> a écrit :
>
> Hi Edward,
>
> Full mvn dependency:tree log enclosed.
>
> We’re compiling using command:
>
> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 
> -Dhbase.version=2.0.2
>
>
>
> Regards,
>
> Mike
>
>
>
> From: Edward Armes 
> Sent: Wednesday, January 19, 2022 12:34 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> Hi Mike,
>
>
>
> Can you give the full output you get from mvn dependency:tree as we 15.2 and 
> 3 was done to remove log4j dependences from Nifi as a safety measure due the 
> recent log4shell issues.
>
>
>
> Thanks
>
>
>
> Edward
>
>
>
> On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski 
>  wrote:
>
> Hi,
>
> We are trying to compile nifi from sources.
>
> There is no problem with 1.15.1 compilation but in case of 1.15.2 and 1.15.3 
> there are compilation errors.
>
>
>
> Can you suggest how to solve this problem?
>
>
>
> mvn dependency:tree has no errors:
>
>
>
>
>
> $ tail log1.txt
>
> [INFO] nifi-toolkit-flowfile-repo . SUCCESS [  
> 0.005 s]
>
> [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [  
> 0.008 s]
>
> [INFO] nifi-toolkit-assembly .. SUCCESS [  
> 0.463 s]
>
> [INFO] nifi-toolkit-api ... SUCCESS [  
> 0.023 s]
>
> [INFO] 
> --
> --
>
> [INFO] BUILD SUCCESS
>
> [INFO] 
> --
> --
>
> [INFO] Total time:  02:41 min
>
> [INFO] Finished at: 2022-01-19T08:51:46+01:00
>
>
>
>
>
> Compilation of branch rel/nifi-1.15.3:
>
>
>
> [DEBUG] Adding ERROR message due to exception
>
> org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned 
> Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
>
> at 
> org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute 
> (AbstractBanDependencies.java:113)
>
> at org.apache.maven.plugins.enforcer.EnforceMojo.execute 
> (EnforceMojo.java:200)
>
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
> (DefaultBuildPluginManager.java:137)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:210)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:156)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:148)
>
> at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProjec
> t (LifecycleModuleBuilder.java:117)
>
> at 
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade
> dBuilder$1.call (MultiThreadedBuilder.java:196)
>
> at 
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreade
> 

Re: Nifi 1.15.2 and 1.15.3 compilation errors

[Bryan Bende]

We did ban all use of log4j-core in favor of log4j-to-slf4j...

https://issues.apache.org/jira/browse/NIFI-9483

With your build specifying the -D versions, it created a different
classpath during your build than what existed during the normal build
of the release.

We should try to fix any issues like this though, so if you can figure
out which dependency path is the issue, the fix is to add some
exclusions and then to ensure that log4j-to-slf4j dependency is added.

On Wed, Jan 19, 2022 at 8:48 AM Michal Tomaszewski
 wrote:
>
> OK, but there is information that log4j-core:jar:2.17.1 is blocked:
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
> with message:
>
> Found Banned Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> while this version is not impacted by log4j security vulnerability:
>
> https://logging.apache.org/log4j/2.x/security.html
>
>
>
> So for sure it shouldn’t be banned.
>
>
>
> Regards,
>
> Mike
>
> From: Pierre Villard 
> Sent: Wednesday, January 19, 2022 1:11 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> We recently changed the build to fail if it's including problematic 
> transitive dependencies for log4j. I think those are coming with the specific 
> versions of HBase/Hive you're using. Not a maven expert but I guess there is 
> a way to exclude the step "enforce-banned-dependencies" from the build.
>
>
>
> Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski  
> a écrit :
>
> Hi Edward,
>
> Full mvn dependency:tree log enclosed.
>
> We’re compiling using command:
>
> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 
> -Dhbase.version=2.0.2
>
>
>
> Regards,
>
> Mike
>
>
>
> From: Edward Armes 
> Sent: Wednesday, January 19, 2022 12:34 PM
> To: users@nifi.apache.org
> Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> Hi Mike,
>
>
>
> Can you give the full output you get from mvn dependency:tree as we 15.2 and 
> 3 was done to remove log4j dependences from Nifi as a safety measure due the 
> recent log4shell issues.
>
>
>
> Thanks
>
>
>
> Edward
>
>
>
> On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski 
>  wrote:
>
> Hi,
>
> We are trying to compile nifi from sources.
>
> There is no problem with 1.15.1 compilation but in case of 1.15.2 and 1.15.3 
> there are compilation errors.
>
>
>
> Can you suggest how to solve this problem?
>
>
>
> mvn dependency:tree has no errors:
>
>
>
>
>
> $ tail log1.txt
>
> [INFO] nifi-toolkit-flowfile-repo . SUCCESS [  0.005 
> s]
>
> [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [  0.008 
> s]
>
> [INFO] nifi-toolkit-assembly .. SUCCESS [  0.463 
> s]
>
> [INFO] nifi-toolkit-api ... SUCCESS [  0.023 
> s]
>
> [INFO] 
> 
>
> [INFO] BUILD SUCCESS
>
> [INFO] 
> 
>
> [INFO] Total time:  02:41 min
>
> [INFO] Finished at: 2022-01-19T08:51:46+01:00
>
>
>
>
>
> Compilation of branch rel/nifi-1.15.3:
>
>
>
> [DEBUG] Adding ERROR message due to exception
>
> org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned 
> Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
>
> at org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute 
> (AbstractBanDependencies.java:113)
>
> at org.apache.maven.plugins.enforcer.EnforceMojo.execute 
> (EnforceMojo.java:200)
>
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
> (DefaultBuildPluginManager.java:137)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:210)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:156)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
> (MojoExecutor.java:148)
>
> at 
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
> (LifecycleModuleBuilder.java:117)
>
> at 
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
>  (MultiThreadedBuilder.java:196)
>
> at 
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
>  (MultiThreadedBuilder.java:186)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.Executors$RunnableAdapter.call 
> (Executors.java:511)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker 
> (ThreadPoolExecutor.java:1149)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run 
> (ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run (Thread.java:748)
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
> with 

RE: Nifi 1.15.2 and 1.15.3 compilation errors

[Michal Tomaszewski]

OK, but there is information that log4j-core:jar:2.17.1 is blocked:
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
with message:
Found Banned Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
while this version is not impacted by log4j security vulnerability:
https://logging.apache.org/log4j/2.x/security.html

So for sure it shouldn’t be banned.

Regards,
Mike
From: Pierre Villard 
Sent: Wednesday, January 19, 2022 1:11 PM
To: users@nifi.apache.org
Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors

We recently changed the build to fail if it's including problematic transitive 
dependencies for log4j. I think those are coming with the specific versions of 
HBase/Hive you're using. Not a maven expert but I guess there is a way to 
exclude the step "enforce-banned-dependencies" from the build.

Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski 
mailto:michal.tomaszew...@cca.pl>> a écrit :
Hi Edward,
Full mvn dependency:tree log enclosed.
We’re compiling using command:
mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0 
-Dhbase.version=2.0.2

Regards,
Mike

From: Edward Armes mailto:edward.ar...@gmail.com>>
Sent: Wednesday, January 19, 2022 12:34 PM
To: users@nifi.apache.org
Subject: Re: Nifi 1.15.2 and 1.15.3 compilation errors

Hi Mike,

Can you give the full output you get from mvn dependency:tree as we 15.2 and 3 
was done to remove log4j dependences from Nifi as a safety measure due the 
recent log4shell issues.

Thanks

Edward

On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski 
mailto:michal.tomaszew...@cca.pl>> wrote:
Hi,
We are trying to compile nifi from sources.
There is no problem with 1.15.1 compilation but in case of 1.15.2 and 1.15.3 
there are compilation errors.

Can you suggest how to solve this problem?

mvn dependency:tree has no errors:


$ tail log1.txt
[INFO] nifi-toolkit-flowfile-repo . SUCCESS [  0.005 s]
[INFO] nifi-toolkit-flowanalyzer .. SUCCESS [  0.008 s]
[INFO] nifi-toolkit-assembly .. SUCCESS [  0.463 s]
[INFO] nifi-toolkit-api ... SUCCESS [  0.023 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time:  02:41 min
[INFO] Finished at: 2022-01-19T08:51:46+01:00


Compilation of branch rel/nifi-1.15.3:

[DEBUG] Adding ERROR message due to exception
org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned 
Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
at org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute 
(AbstractBanDependencies.java:113)
at org.apache.maven.plugins.enforcer.EnforceMojo.execute 
(EnforceMojo.java:200)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
(DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
(LifecycleModuleBuilder.java:117)
at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
 (MultiThreadedBuilder.java:196)
at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
 (MultiThreadedBuilder.java:186)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker 
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
with message:
Found Banned Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
[INFO]

….

[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce 
(enforce-banned-dependencies) on project nifi-kite-processors: Some Enforcer 
rules have failed. Look above for specific messages explaining why the rule 
failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal 
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce 
(enforce-banned-dependencies) on project nifi-kite-processors: Some Enforcer 
rules have failed. Look above for specific 

Re: Nifi 1.15.2 and 1.15.3 compilation errors

[Pierre Villard]

We recently changed the build to fail if it's including problematic
transitive dependencies for log4j. I think those are coming with the
specific versions of HBase/Hive you're using. Not a maven expert but I
guess there is a way to exclude the step "enforce-banned-dependencies" from
the build.

Le mer. 19 janv. 2022 à 12:57, Michal Tomaszewski 
a écrit :

> Hi Edward,
>
> Full mvn dependency:tree log enclosed.
>
> We’re compiling using command:
>
> mvn -e -X -T C2.0 clean install -DskipTests -Dhive.version=3.1.0
> -Dhbase.version=2.0.2
>
>
>
> Regards,
>
> Mike
>
>
>
> *From:* Edward Armes 
> *Sent:* Wednesday, January 19, 2022 12:34 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Nifi 1.15.2 and 1.15.3 compilation errors
>
>
>
> Hi Mike,
>
>
>
> Can you give the full output you get from mvn dependency:tree as we 15.2
> and 3 was done to remove log4j dependences from Nifi as a safety measure
> due the recent log4shell issues.
>
>
>
> Thanks
>
>
>
> Edward
>
>
>
> On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski <
> michal.tomaszew...@cca.pl> wrote:
>
> Hi,
>
> We are trying to compile nifi from sources.
>
> There is no problem with 1.15.1 compilation but in case of 1.15.2 and
> 1.15.3 there are compilation errors.
>
>
>
> Can you suggest how to solve this problem?
>
>
>
> mvn dependency:tree has no errors:
>
>
>
>
>
> $ tail log1.txt
>
> [INFO] nifi-toolkit-flowfile-repo . SUCCESS [
> 0.005 s]
>
> [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [
> 0.008 s]
>
> [INFO] nifi-toolkit-assembly .. SUCCESS [
> 0.463 s]
>
> [INFO] nifi-toolkit-api ... SUCCESS [
> 0.023 s]
>
> [INFO]
> 
>
> [INFO] BUILD SUCCESS
>
> [INFO]
> 
>
> [INFO] Total time:  02:41 min
>
> [INFO] Finished at: 2022-01-19T08:51:46+01:00
>
>
>
>
>
> Compilation of branch rel/nifi-1.15.3:
>
>
>
> [DEBUG] Adding ERROR message due to exception
>
> org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned
> Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
>
> at org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute
> (AbstractBanDependencies.java:113)
>
> at org.apache.maven.plugins.enforcer.EnforceMojo.execute
> (EnforceMojo.java:200)
>
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:137)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:210)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
>
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
>
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:196)
>
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:186)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.Executors$RunnableAdapter.call
> (Executors.java:511)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1149)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run (Thread.java:748)
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies
> failed with message:
>
> Found Banned Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
>
> [INFO]
>
>
>
> ….
>
>
>
> [INFO]
> 
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce
> (enforce-banned-dependencies) on project nifi-kite-processors: Some
> Enforcer rules have failed. Look above for specific messages explaining why
> the rule failed. -> [Help 1]
>
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce
> (enforce-banned-dependencies) on project nifi-kite-processors: Some
> Enforcer rules have failed. Look above for specific messages explaining why
> the rule failed.
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:215)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
>
> at
> 

Re: Nifi 1.15.2 and 1.15.3 compilation errors

[Edward Armes]

Hi Mike,

Can you give the full output you get from mvn dependency:tree as we 15.2
and 3 was done to remove log4j dependences from Nifi as a safety measure
due the recent log4shell issues.

Thanks

Edward

On Wed, Jan 19, 2022 at 10:24 AM Michal Tomaszewski <
michal.tomaszew...@cca.pl> wrote:

> Hi,
>
> We are trying to compile nifi from sources.
>
> There is no problem with 1.15.1 compilation but in case of 1.15.2 and
> 1.15.3 there are compilation errors.
>
>
>
> Can you suggest how to solve this problem?
>
>
>
> mvn dependency:tree has no errors:
>
>
>
>
>
> $ tail log1.txt
>
> [INFO] nifi-toolkit-flowfile-repo . SUCCESS [
> 0.005 s]
>
> [INFO] nifi-toolkit-flowanalyzer .. SUCCESS [
> 0.008 s]
>
> [INFO] nifi-toolkit-assembly .. SUCCESS [
> 0.463 s]
>
> [INFO] nifi-toolkit-api ... SUCCESS [
> 0.023 s]
>
> [INFO]
> 
>
> [INFO] BUILD SUCCESS
>
> [INFO]
> 
>
> [INFO] Total time:  02:41 min
>
> [INFO] Finished at: 2022-01-19T08:51:46+01:00
>
>
>
>
>
> Compilation of branch rel/nifi-1.15.3:
>
>
>
> [DEBUG] Adding ERROR message due to exception
>
> org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned
> Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
>
> at org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute
> (AbstractBanDependencies.java:113)
>
> at org.apache.maven.plugins.enforcer.EnforceMojo.execute
> (EnforceMojo.java:200)
>
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:137)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:210)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
>
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
>
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:196)
>
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:186)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.Executors$RunnableAdapter.call
> (Executors.java:511)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1149)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run
> (ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run (Thread.java:748)
>
> [WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies
> failed with message:
>
> Found Banned Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
>
> Use 'mvn dependency:tree' to locate the source of the banned dependencies.
>
> [INFO]
>
>
>
> ….
>
>
>
> [INFO]
> 
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce
> (enforce-banned-dependencies) on project nifi-kite-processors: Some
> Enforcer rules have failed. Look above for specific messages explaining why
> the rule failed. -> [Help 1]
>
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
> goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce
> (enforce-banned-dependencies) on project nifi-kite-processors: Some
> Enforcer rules have failed. Look above for specific messages explaining why
> the rule failed.
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:215)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:156)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:148)
>
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:117)
>
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:196)
>
> at
> org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
> (MultiThreadedBuilder.java:186)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.Executors$RunnableAdapter.call
> (Executors.java:511)
>
> at java.util.concurrent.FutureTask.run (FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker
> (ThreadPoolExecutor.java:1149)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run
> 

Nifi 1.15.2 and 1.15.3 compilation errors

[Michal Tomaszewski]

Hi,
We are trying to compile nifi from sources.
There is no problem with 1.15.1 compilation but in case of 1.15.2 and 1.15.3 
there are compilation errors.

Can you suggest how to solve this problem?

mvn dependency:tree has no errors:


$ tail log1.txt
[INFO] nifi-toolkit-flowfile-repo . SUCCESS [  0.005 s]
[INFO] nifi-toolkit-flowanalyzer .. SUCCESS [  0.008 s]
[INFO] nifi-toolkit-assembly .. SUCCESS [  0.463 s]
[INFO] nifi-toolkit-api ... SUCCESS [  0.023 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time:  02:41 min
[INFO] Finished at: 2022-01-19T08:51:46+01:00


Compilation of branch rel/nifi-1.15.3:

[DEBUG] Adding ERROR message due to exception
org.apache.maven.enforcer.rule.api.EnforcerRuleException: Found Banned 
Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
at org.apache.maven.plugins.enforcer.AbstractBanDependencies.execute 
(AbstractBanDependencies.java:113)
at org.apache.maven.plugins.enforcer.EnforceMojo.execute 
(EnforceMojo.java:200)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
(DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
(LifecycleModuleBuilder.java:117)
at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
 (MultiThreadedBuilder.java:196)
at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
 (MultiThreadedBuilder.java:186)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker 
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
[WARNING] Rule 0: org.apache.maven.plugins.enforcer.BannedDependencies failed 
with message:
Found Banned Dependency: org.apache.logging.log4j:log4j-core:jar:2.17.1
Use 'mvn dependency:tree' to locate the source of the banned dependencies.
[INFO]

….

[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce 
(enforce-banned-dependencies) on project nifi-kite-processors: Some Enforcer 
rules have failed. Look above for specific messages explaining why the rule 
failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal 
org.apache.maven.plugins:maven-enforcer-plugin:3.0.0:enforce 
(enforce-banned-dependencies) on project nifi-kite-processors: Some Enforcer 
rules have failed. Look above for specific messages explaining why the rule 
failed.
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject 
(LifecycleModuleBuilder.java:117)
at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
 (MultiThreadedBuilder.java:196)
at 
org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call
 (MultiThreadedBuilder.java:186)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)
at java.util.concurrent.FutureTask.run (FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker 
(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:624)
at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer rules 
have failed. Look above for specific messages explaining why the rule failed.
at org.apache.maven.plugins.enforcer.EnforceMojo.execute 
(EnforceMojo.java:255)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo 
(DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute 
(MojoExecutor.java:156)