Re: PULL ProvenanceEvent

[Joe Witt]

These distinctions may be meaningful.  Adding them as an attribute lets the
meaning convey but not introduce complexity for the majority case which is
the distinction isnt key.

thanks

On Thu, Oct 31, 2019 at 4:05 PM Nissim Shiman 
wrote:

>  Mike,
> I like the QUERY type as well.  Basically a more refined PULL.  Very nice.
>
>
> Part of the challenge of adding PULL as a type is that there are currently
> two flavors of RECEIVEs.
> RECEIVE and FETCH [1]
>
> So any addition of a PULL would need a second flavor of PULL to match the
> case where a flowfile's contents are being overwritten as well (i.e. as
> FETCH is currently doing)
>
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/provenance/ProvenanceEventType.java#L42
>
>
> Thanks,
> Nissim
>
>
> On Wednesday, October 30, 2019, 6:41:04 PM EDT, Mike Thomsen <
> mikerthom...@gmail.com> wrote:
>
>  I like the idea of creating PULL as a type. In fact, I'd propose that
> there
> are three scenarios here:
>
> RECEIVE - Passively acquire in a sort of hand-off situation. Ex: Kafka
> subscription
> PULL - Direct operations to seek out and fetch something in a targeted
> fashion. Ex. GetHttp
> QUERY - Go looking for the data and take what matches your search. Ex.
> JsonQueryElasticsearch, GetMongo, any SQL query processor, etc.
>
>
>
> On Wed, Oct 30, 2019 at 1:31 PM Nissim Shiman 
> wrote:
>
> >  Joe,
> >
> >
> > It is hard to say how much value transit URI would bring to clarify a
> > RECEIVE.
> > For example a RECEIVE with transit URI of https: could be either a
> > GetHTTP (i.e. active) or ListenHTTP (i.e. passive)
> >
> > but your idea of "a metadata item specifying active vs passive" is a very
> > clever way to make this work with mimimal disruptions.
> >
> > My understanding of this is that the current receive() calls in
> > ProvenanceReporter [1] will remain the same, but news ones will be added
> > with a boolean parameter reflecting if the receive is active or passive.
> > This will allow the current list of Provenance Events [2] to remain the
> > same.  So third party/custom processors can continue working as is
> >
> > Does this sound like what you are thinking?
> >
> >
> > [1]
> >
> https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/provenance/ProvenanceReporter.java#L46
> >
> > [2]
> > apache/nifi
> >
> >
> > Thanks,
> >
> > Nissim
> >On Tuesday, October 29, 2019, 12:47:40 PM EDT, Joe Witt <
> > joe.w...@gmail.com> wrote:
> >
> >  Nissim
> >
> > I like the idea to introduce a more refined type of event for how data is
> > brought into nifi (active - PULL, passive - RECEIVE).
> >
> > That said it might be sufficient to simply have this distinction be on
> the
> > "RECEIVE" event as a metadata item specifying active vs passive.  The
> > protocol utilized as mentioned in the transport URI should clarify this
> > though.
> >
> > In short - i think there is a way here that is all opt-in for existing
> > users and components.
> >
> > Thanks
> >
> > On Tue, Oct 29, 2019 at 9:41 AM Nissim Shiman  >
> > wrote:
> >
> > >  Adam,
> > > good points...
> > > I missed a step in explaining the use case where Provenance Events is
> > > incomplete...
> > > Where the second nifi does a GetSFTP from the *filesytem* that the
> first
> > > nifi is located on
> > > So the second nifi currently sends a RECEIVE event, but there is no
> > > corresponding SEND event from the first nifi (nor should there be)
> > > If the second nifi sent a PULL event, it would be easier for a system
> > > overseer to know that there should be no corresponding SEND event
> > >
> > > Currently send/receive works well when nifi 1 does a PostHTTP and nifi
> 2
> > > does a ListenHTTP, but not in the case above.
> > >
> > > The ERROR case you mention is a nice point as well, although not my
> > > specific issue at the moment.
> > > Thanks,
> > > Nissim
> > >
> > >
> > >
> > >
> > >
> > >On Monday, October 28, 2019, 11:52:57 PM EDT, Adam Taft <
> > > a...@adamtaft.com> wrote:
> > >
> > >  > But a flowfile that was PULLed by the second nifi (from the first
> > nifi)
> > > will not necessarily have any provenance event generated by the first
> > nifi.
> > >
> > > Isn't this the fault of the first NiFi to fail to emit a SEND event in
> > > response to the second NiFi's request?  In this scenario, shouldn't the
> > > send/receive pair be:
> > > NiFi-1 [SEND] :: NIFI-2 [RECEIVE]?
> > >
> > > What you describe is an odd use case for NiFi.  NiFi is usually not in
> > the
> > > business of acting as a file server daemon in order to "send" flowfiles
> > to
> > > other systems.  As you mention, HandleHttpResponse may be a lone wolf
> > > example processor which generates a SEND event whose input originates
> > from
> > > a "listener". [1]  The other ListenXYZ processors generally issue
> RECEIVE
> > > events because they are receiving bytes, not generating them.
> > >
> > > Are there other processors in question? Something 

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Robert Fellows]

+1 (non-binding)

[X] Verified the checksums
[X] Verified signature
[X] Verify build (java 11)
[X] Verify general functionality (including parameter contexts and
back-pressure prediction UI)

On Thu, Oct 31, 2019 at 10:00 AM Peter Turcsanyi 
wrote:

> +1 (non-binding)
>
> Verified checksums and signatures.
> Built source with empty Maven repo (tests ok, contrib-check ok).
> Tested some flows (all AWS processors, focusing on the new S3 features:
> encryption, storage classes, requester pays).
>
> Peter
>
> On Thu, Oct 31, 2019 at 2:28 PM Bryan Bende  wrote:
>
> > +1 (binding)
> >
> > Everything in release helper checked out. I did run into a test
> > failure the first time building the source, but then trying again
> > succeeded. I think it is a sporadic failure we have seen before...
> >
> > [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time
> > elapsed: 1.415 s <<< FAILURE! - in
> > org.apache.nifi.wali.TestLengthDelimitedJournal
> > [ERROR]
> >
> testFailureDuringWriteCannotCauseCorruption(org.apache.nifi.wali.TestLengthDelimitedJournal)
> >  Time elapsed: 1.022 s  <<< FAILURE!
> > java.lang.AssertionError: expected:<0> but was:<2>
> > at
> >
> org.apache.nifi.wali.TestLengthDelimitedJournal.testFailureDuringWriteCannotCauseCorruption(TestLengthDelimitedJournal.java:585)
> >
> > On Thu, Oct 31, 2019 at 6:42 AM Kotaro Terada 
> wrote:
> > >
> > > Thanks for taking care of it, Pierre and Joe!
> > >
> > > > Please note only PMC votes are binding.
> > >
> > > I got it,  I'll be careful next time.
> > >
> > > Thanks,
> > > Kotaro
> > >
> > >
> > > On Thu, Oct 31, 2019 at 7:32 PM Joe Witt  wrote:
> > >
> > > > Id just flag this on release notes and we can fix for the next
> release.
> > > >
> > > > Kotaro good find.  Please note only PMC votes are binding.
> > > >
> > > > Thanks
> > > >
> > > > On Thu, Oct 31, 2019 at 6:27 AM Pierre Villard <
> > > > pierre.villard...@gmail.com>
> > > > wrote:
> > > >
> > > > > Good catch Kotaro! I merged the fix for #2. The fix for #1 LGTM but
> > I'll
> > > > > let the guys who worked on the Java 11 bits look into it.
> > > > >
> > > > > Joe, I didn't set the fix version on the JIRA and will wait until
> we
> > > > reach
> > > > > a decision regarding this RC.
> > > > >
> > > > > Le jeu. 31 oct. 2019 à 11:08, Kotaro Terada  a
> > > > écrit :
> > > > >
> > > > > > -1 (binding), due to the following problems I found.
> > > > > >
> > > > > > (1) NiFi doesn't start up from the rpm build on Java 11.
> > > > > >
> > > > > > We have a mismatch in the "lib" directories of rpm and non-rpm
> > build.
> > > > > This
> > > > > > causes that NiFi (with Java 11) doesn't start up from the rpm
> > build.
> > > > Some
> > > > > > jar files should be located in the "lib/bootstrap/" but they are
> > > > > currently
> > > > > > in "lib/".
> > > > > >
> > > > > > I checked the following behaviors:
> > > > > > - Non-RPM (usual) build with Java 8 and run on Java 8... OK
> > > > > > - Non-RPM (usual) build with Java 11 and run on Java 11... OK
> > > > > > - RPM build with Java 8 and run on Java 8... OK (isn't affected
> by
> > the
> > > > > lib
> > > > > > mismatch)
> > > > > > - RPM build with Java 11 and run on Java 11... NiFi doesn't start
> > (this
> > > > > > problem)
> > > > > >
> > > > > > I created a JIRA and a PR to fix it:
> > > > > > https://issues.apache.org/jira/browse/NIFI-6827
> > > > > >
> > > > > > (2) There is an unintentional string in "conf/logback.xml".
> > > > > >
> > > > > > The "conf/logback.xml" seems to have an unintentional string
> > > > > > "MockProcessContext". I guess this was slipped into it by the
> > merging
> > > > > > commit of https://github.com/apache/nifi/pull/3773
> > unintentionally.
> > > > > >
> > > > > > I created a JIRA and a PR to fix it:
> > > > > > https://issues.apache.org/jira/browse/NIFI-6828
> > > > > >
> > > > > > Could you take a look at them, please?
> > > > > >
> > > > > > Thanks for working for the release!
> > > > > > Kotaro
> > > > > >
> > > > > >
> > > > > > On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard <
> > > > > > pierre.villard...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Went through the release helper guide and it LGTM. Deployed
> > > > > > > secured/unsecured standalone/cluster instances on Google Cloud
> > with
> > > > > > various
> > > > > > > flows including the new BigQuery processor and everything is
> > looking
> > > > > > good.
> > > > > > >
> > > > > > > Thanks Joe for taking care of the release!
> > > > > > >
> > > > > > > Pierre
> > > > > > >
> > > > > > > Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess <
> > > > shayne.burg...@gmail.com
> > > > > >
> > > > > > a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > Deployed cluster in Azure and smoke tested a fairly
> complicated
> > > > data
> > > > > > > flow.
> > > > > > > >
> > > > > > > > Verified the fixes for the Azure EventHub Processor that were
> 

Cert Rotation and Revocation nifi and minifi

[Craig Knell]

Hi folks

What options and protocols are there to revoke certificates and cycle 
certificates.  I’m especially interested in cycling MiNiFi certificates?

Best regards

Craig Knell

Re: Issues while configuring nifi cluster

[Andy LoPresto]

Images do not come through consistently on this email list, so please host that 
elsewhere and link to it, or copy/paste the content of it. 

Without seeing the image, I am not positive, but based on the other text in 
your email, I suspect your nifi.sensitive.props.key value is empty in 
nifi.properties. This is not a blocker but is strongly discouraged. 

The second issue is likely a malformed XML file in users.xml or 
authorizers.xml. It could also be the flow.xml.gz file. Without additional 
debugging information (further stack trace, those XML files), we will not be 
able to help more than that. If you can share those, please do. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Oct 31, 2019, at 1:24 PM, Venkata Raghava Raju Bhupathiraju 
>  wrote:
> 
> Hello,
> 
> I am getting the below while configuring nifi-cluster,
> 
> A blank sensitive properties key was provided. Specify a unique key in 
> nifi.properties for nifi.sensitive.props.key
> 
> 
> 
> We are getting the below exception as well,
> 
> Caused by: org.springframework.beans.factory.BeanCreationException: Error 
> creating bean with name 'flowService': FactoryBean threw exception on object 
> creation; nested exception is 
> org.springframework.beans.factory.BeanCreationException: Error creating bean 
> with name 'flowController': FactoryBean threw exception on object creation; 
> nested exception is java.lang.ArrayIndexOutOfBoundsException: 1
> 
> Thanks,
> Raghava Raju
> Softility, Inc. | E: vbhupathir...@softilityinc.com 
> 

Issues while configuring nifi cluster

[Venkata Raghava Raju Bhupathiraju]

Hello,

I am getting the below while configuring nifi-cluster,

A blank sensitive properties key was provided. Specify a unique key in
nifi.properties for nifi.sensitive.props.key

[image: image.png]

We are getting the below exception as well,

Caused by: org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'flowService': FactoryBean threw exception on
object creation; nested exception is
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name 'flowController': FactoryBean threw exception on object
creation; nested exception is java.lang.ArrayIndexOutOfBoundsException: 1

Thanks,
*Raghava Raju*
Softility, Inc. | E: vbhupathir...@softilityinc.com

Re: PULL ProvenanceEvent

[Nissim Shiman]

 Mike,
I like the QUERY type as well.  Basically a more refined PULL.  Very nice.


Part of the challenge of adding PULL as a type is that there are currently two 
flavors of RECEIVEs.  
RECEIVE and FETCH [1]

So any addition of a PULL would need a second flavor of PULL to match the case 
where a flowfile's contents are being overwritten as well (i.e. as FETCH is 
currently doing)


[1] 
https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/provenance/ProvenanceEventType.java#L42


Thanks,
Nissim


On Wednesday, October 30, 2019, 6:41:04 PM EDT, Mike Thomsen 
 wrote:  
 
 I like the idea of creating PULL as a type. In fact, I'd propose that there
are three scenarios here:

RECEIVE - Passively acquire in a sort of hand-off situation. Ex: Kafka
subscription
PULL - Direct operations to seek out and fetch something in a targeted
fashion. Ex. GetHttp
QUERY - Go looking for the data and take what matches your search. Ex.
JsonQueryElasticsearch, GetMongo, any SQL query processor, etc.



On Wed, Oct 30, 2019 at 1:31 PM Nissim Shiman 
wrote:

>  Joe,
>
>
> It is hard to say how much value transit URI would bring to clarify a
> RECEIVE.
> For example a RECEIVE with transit URI of https: could be either a
> GetHTTP (i.e. active) or ListenHTTP (i.e. passive)
>
> but your idea of "a metadata item specifying active vs passive" is a very
> clever way to make this work with mimimal disruptions.
>
> My understanding of this is that the current receive() calls in
> ProvenanceReporter [1] will remain the same, but news ones will be added
> with a boolean parameter reflecting if the receive is active or passive.
> This will allow the current list of Provenance Events [2] to remain the
> same.  So third party/custom processors can continue working as is
>
> Does this sound like what you are thinking?
>
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/provenance/ProvenanceReporter.java#L46
>
> [2]
> apache/nifi
>
>
> Thanks,
>
> Nissim
>    On Tuesday, October 29, 2019, 12:47:40 PM EDT, Joe Witt <
> joe.w...@gmail.com> wrote:
>
>  Nissim
>
> I like the idea to introduce a more refined type of event for how data is
> brought into nifi (active - PULL, passive - RECEIVE).
>
> That said it might be sufficient to simply have this distinction be on the
> "RECEIVE" event as a metadata item specifying active vs passive.  The
> protocol utilized as mentioned in the transport URI should clarify this
> though.
>
> In short - i think there is a way here that is all opt-in for existing
> users and components.
>
> Thanks
>
> On Tue, Oct 29, 2019 at 9:41 AM Nissim Shiman 
> wrote:
>
> >  Adam,
> > good points...
> > I missed a step in explaining the use case where Provenance Events is
> > incomplete...
> > Where the second nifi does a GetSFTP from the *filesytem* that the first
> > nifi is located on
> > So the second nifi currently sends a RECEIVE event, but there is no
> > corresponding SEND event from the first nifi (nor should there be)
> > If the second nifi sent a PULL event, it would be easier for a system
> > overseer to know that there should be no corresponding SEND event
> >
> > Currently send/receive works well when nifi 1 does a PostHTTP and nifi 2
> > does a ListenHTTP, but not in the case above.
> >
> > The ERROR case you mention is a nice point as well, although not my
> > specific issue at the moment.
> > Thanks,
> > Nissim
> >
> >
> >
> >
> >
> >    On Monday, October 28, 2019, 11:52:57 PM EDT, Adam Taft <
> > a...@adamtaft.com> wrote:
> >
> >  > But a flowfile that was PULLed by the second nifi (from the first
> nifi)
> > will not necessarily have any provenance event generated by the first
> nifi.
> >
> > Isn't this the fault of the first NiFi to fail to emit a SEND event in
> > response to the second NiFi's request?  In this scenario, shouldn't the
> > send/receive pair be:
> > NiFi-1 [SEND] :: NIFI-2 [RECEIVE]?
> >
> > What you describe is an odd use case for NiFi.  NiFi is usually not in
> the
> > business of acting as a file server daemon in order to "send" flowfiles
> to
> > other systems.  As you mention, HandleHttpResponse may be a lone wolf
> > example processor which generates a SEND event whose input originates
> from
> > a "listener". [1]  The other ListenXYZ processors generally issue RECEIVE
> > events because they are receiving bytes, not generating them.
> >
> > Are there other processors in question? Something custom? Or is this
> > related to site-to-site transfers?
> >
> > I still kind of question the motive of a provenance event pair that is
> > trying to establish "who called who first".  Honestly just trying to
> > understand the use case where a matching SEND/RECEIVE pair doesn't give
> you
> > what you need.
> >
> > The only thing I could see would be a processor that asks for data, but
> > then doesn't receive it due to some error condition.  In this case,
> adding
> > some sort of ERROR event might be useful.  "I attempted 

invokeHttp routing of exceptions like ConnectException and IOException to failure instead of retry

[David Caldwell]

Hi,
While testing invokeHttp retry logic when the destination endpoint is offline, 
I learned that invokeHttp processor routes exceptions caused by the offline 
endpoint to the failure relationship instead of the retry relationship.
That surprised me since those types of errors are exactly what I would normally 
like to retry.  What is the the reasoning for routing them to failure instead 
of retry?
Furthermore, when I routed the failure relationship back into invokeHttp to 
retry, I then found that nifi cpu usage stays around 150-160% until the remote 
endpoint comes back online.
Additional digging showed that invokeHttp penalizes retry, no_retry & failure 
scenarios, but yields only for retry and no_retry.  IOW, the failure scenario 
doesn't yield.

I don't really have a problem with failure not yielding.  That's just what I 
suspect may be causing the excessive cpu utilization.  My real problem is that 
I think recoverable communications exceptions should be routed to retry instead 
of failure.  Not only would that avoid developer surprise, but it would include 
yield which I hope would prevent the high cpu utilization.
Reading the topic 
https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/html/developer-guide.html#penalization-vs-yielding,
 the following points reinforced my thinking:   
   - Yield when processor won't be able to perform any useful function for some 
period of time
   
   - This tells framework, don't waste resources triggering the processor to 
run, because there's nothing it can do for a while
   - The topic actually uses a processor communicating with remote resource as 
the example for yield
Thoughts?
David

Re: [ANNOUNCE] New Apache NiFi Committer Peter Turcsanyi

[Jeff]

Congrats and welcome, Peter!

On Tue, Oct 29, 2019 at 4:44 PM Peter Turcsanyi 
wrote:

> Thank you everyone!
>
> Peter
>
> On Mon, Oct 28, 2019 at 6:06 PM Andy LoPresto 
> wrote:
>
> > Congratulations Peter. Looking forward to your continued contributions.
> >
> > Andy LoPresto
> > alopre...@apache.org
> > alopresto.apa...@gmail.com
> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >
> > > On Oct 28, 2019, at 9:39 AM, Matt Burgess 
> wrote:
> > >
> > > Congratulations Peter!
> > >
> > > On Sun, Oct 27, 2019 at 9:05 PM Aldrin Piri 
> > wrote:
> > >>
> > >> Apache NiFi community,
> > >>
> > >> On behalf of the Apache NiFI PMC, I am very pleased to announce that
> > Peter
> > >> has accepted the PMC's invitation to become a committer on the Apache
> > NiFi
> > >> project. We greatly appreciate all of Peter's hard work and generous
> > >> contributions to the project. We look forward to continued involvement
> > >> in the project.
> > >>
> > >> Peter has provided several new extensions and improvements enhancing
> > NiFi's
> > >> interoperability with cloud services.  He has also found and remedied
> > >> several bugs, is a regular participant in code reviews and an active
> > >> presence on the mailing lists helping out the community.
> > >>
> > >> Welcome and congratulations!
> >
> >
>

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Peter Turcsanyi]

+1 (non-binding)

Verified checksums and signatures.
Built source with empty Maven repo (tests ok, contrib-check ok).
Tested some flows (all AWS processors, focusing on the new S3 features:
encryption, storage classes, requester pays).

Peter

On Thu, Oct 31, 2019 at 2:28 PM Bryan Bende  wrote:

> +1 (binding)
>
> Everything in release helper checked out. I did run into a test
> failure the first time building the source, but then trying again
> succeeded. I think it is a sporadic failure we have seen before...
>
> [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time
> elapsed: 1.415 s <<< FAILURE! - in
> org.apache.nifi.wali.TestLengthDelimitedJournal
> [ERROR]
> testFailureDuringWriteCannotCauseCorruption(org.apache.nifi.wali.TestLengthDelimitedJournal)
>  Time elapsed: 1.022 s  <<< FAILURE!
> java.lang.AssertionError: expected:<0> but was:<2>
> at
> org.apache.nifi.wali.TestLengthDelimitedJournal.testFailureDuringWriteCannotCauseCorruption(TestLengthDelimitedJournal.java:585)
>
> On Thu, Oct 31, 2019 at 6:42 AM Kotaro Terada  wrote:
> >
> > Thanks for taking care of it, Pierre and Joe!
> >
> > > Please note only PMC votes are binding.
> >
> > I got it,  I'll be careful next time.
> >
> > Thanks,
> > Kotaro
> >
> >
> > On Thu, Oct 31, 2019 at 7:32 PM Joe Witt  wrote:
> >
> > > Id just flag this on release notes and we can fix for the next release.
> > >
> > > Kotaro good find.  Please note only PMC votes are binding.
> > >
> > > Thanks
> > >
> > > On Thu, Oct 31, 2019 at 6:27 AM Pierre Villard <
> > > pierre.villard...@gmail.com>
> > > wrote:
> > >
> > > > Good catch Kotaro! I merged the fix for #2. The fix for #1 LGTM but
> I'll
> > > > let the guys who worked on the Java 11 bits look into it.
> > > >
> > > > Joe, I didn't set the fix version on the JIRA and will wait until we
> > > reach
> > > > a decision regarding this RC.
> > > >
> > > > Le jeu. 31 oct. 2019 à 11:08, Kotaro Terada  a
> > > écrit :
> > > >
> > > > > -1 (binding), due to the following problems I found.
> > > > >
> > > > > (1) NiFi doesn't start up from the rpm build on Java 11.
> > > > >
> > > > > We have a mismatch in the "lib" directories of rpm and non-rpm
> build.
> > > > This
> > > > > causes that NiFi (with Java 11) doesn't start up from the rpm
> build.
> > > Some
> > > > > jar files should be located in the "lib/bootstrap/" but they are
> > > > currently
> > > > > in "lib/".
> > > > >
> > > > > I checked the following behaviors:
> > > > > - Non-RPM (usual) build with Java 8 and run on Java 8... OK
> > > > > - Non-RPM (usual) build with Java 11 and run on Java 11... OK
> > > > > - RPM build with Java 8 and run on Java 8... OK (isn't affected by
> the
> > > > lib
> > > > > mismatch)
> > > > > - RPM build with Java 11 and run on Java 11... NiFi doesn't start
> (this
> > > > > problem)
> > > > >
> > > > > I created a JIRA and a PR to fix it:
> > > > > https://issues.apache.org/jira/browse/NIFI-6827
> > > > >
> > > > > (2) There is an unintentional string in "conf/logback.xml".
> > > > >
> > > > > The "conf/logback.xml" seems to have an unintentional string
> > > > > "MockProcessContext". I guess this was slipped into it by the
> merging
> > > > > commit of https://github.com/apache/nifi/pull/3773
> unintentionally.
> > > > >
> > > > > I created a JIRA and a PR to fix it:
> > > > > https://issues.apache.org/jira/browse/NIFI-6828
> > > > >
> > > > > Could you take a look at them, please?
> > > > >
> > > > > Thanks for working for the release!
> > > > > Kotaro
> > > > >
> > > > >
> > > > > On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard <
> > > > > pierre.villard...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Went through the release helper guide and it LGTM. Deployed
> > > > > > secured/unsecured standalone/cluster instances on Google Cloud
> with
> > > > > various
> > > > > > flows including the new BigQuery processor and everything is
> looking
> > > > > good.
> > > > > >
> > > > > > Thanks Joe for taking care of the release!
> > > > > >
> > > > > > Pierre
> > > > > >
> > > > > > Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess <
> > > shayne.burg...@gmail.com
> > > > >
> > > > > a
> > > > > > écrit :
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > Deployed cluster in Azure and smoke tested a fairly complicated
> > > data
> > > > > > flow.
> > > > > > >
> > > > > > > Verified the fixes for the Azure EventHub Processor that were
> > > pulled
> > > > > into
> > > > > > > RC3 are working as expected.
> > > > > > >
> > > > > > > Shayne
> > > > > > >
> > > > > > > On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick <
> > > jzemer...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > +1 (non-binding)
> > > > > > > >
> > > > > > > > Built and ran successfully and built custom processor with
> 1.10.0
> > > > > > > > dependency and successfully uploaded NAR to NiFi Registry
> 0.5.0.
> > > > > > > >
> > > > > > > > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne <

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Bryan Bende]

+1 (binding)

Everything in release helper checked out. I did run into a test
failure the first time building the source, but then trying again
succeeded. I think it is a sporadic failure we have seen before...

[ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time
elapsed: 1.415 s <<< FAILURE! - in
org.apache.nifi.wali.TestLengthDelimitedJournal
[ERROR] 
testFailureDuringWriteCannotCauseCorruption(org.apache.nifi.wali.TestLengthDelimitedJournal)
 Time elapsed: 1.022 s  <<< FAILURE!
java.lang.AssertionError: expected:<0> but was:<2>
at 
org.apache.nifi.wali.TestLengthDelimitedJournal.testFailureDuringWriteCannotCauseCorruption(TestLengthDelimitedJournal.java:585)

On Thu, Oct 31, 2019 at 6:42 AM Kotaro Terada  wrote:
>
> Thanks for taking care of it, Pierre and Joe!
>
> > Please note only PMC votes are binding.
>
> I got it,  I'll be careful next time.
>
> Thanks,
> Kotaro
>
>
> On Thu, Oct 31, 2019 at 7:32 PM Joe Witt  wrote:
>
> > Id just flag this on release notes and we can fix for the next release.
> >
> > Kotaro good find.  Please note only PMC votes are binding.
> >
> > Thanks
> >
> > On Thu, Oct 31, 2019 at 6:27 AM Pierre Villard <
> > pierre.villard...@gmail.com>
> > wrote:
> >
> > > Good catch Kotaro! I merged the fix for #2. The fix for #1 LGTM but I'll
> > > let the guys who worked on the Java 11 bits look into it.
> > >
> > > Joe, I didn't set the fix version on the JIRA and will wait until we
> > reach
> > > a decision regarding this RC.
> > >
> > > Le jeu. 31 oct. 2019 à 11:08, Kotaro Terada  a
> > écrit :
> > >
> > > > -1 (binding), due to the following problems I found.
> > > >
> > > > (1) NiFi doesn't start up from the rpm build on Java 11.
> > > >
> > > > We have a mismatch in the "lib" directories of rpm and non-rpm build.
> > > This
> > > > causes that NiFi (with Java 11) doesn't start up from the rpm build.
> > Some
> > > > jar files should be located in the "lib/bootstrap/" but they are
> > > currently
> > > > in "lib/".
> > > >
> > > > I checked the following behaviors:
> > > > - Non-RPM (usual) build with Java 8 and run on Java 8... OK
> > > > - Non-RPM (usual) build with Java 11 and run on Java 11... OK
> > > > - RPM build with Java 8 and run on Java 8... OK (isn't affected by the
> > > lib
> > > > mismatch)
> > > > - RPM build with Java 11 and run on Java 11... NiFi doesn't start (this
> > > > problem)
> > > >
> > > > I created a JIRA and a PR to fix it:
> > > > https://issues.apache.org/jira/browse/NIFI-6827
> > > >
> > > > (2) There is an unintentional string in "conf/logback.xml".
> > > >
> > > > The "conf/logback.xml" seems to have an unintentional string
> > > > "MockProcessContext". I guess this was slipped into it by the merging
> > > > commit of https://github.com/apache/nifi/pull/3773 unintentionally.
> > > >
> > > > I created a JIRA and a PR to fix it:
> > > > https://issues.apache.org/jira/browse/NIFI-6828
> > > >
> > > > Could you take a look at them, please?
> > > >
> > > > Thanks for working for the release!
> > > > Kotaro
> > > >
> > > >
> > > > On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard <
> > > > pierre.villard...@gmail.com>
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Went through the release helper guide and it LGTM. Deployed
> > > > > secured/unsecured standalone/cluster instances on Google Cloud with
> > > > various
> > > > > flows including the new BigQuery processor and everything is looking
> > > > good.
> > > > >
> > > > > Thanks Joe for taking care of the release!
> > > > >
> > > > > Pierre
> > > > >
> > > > > Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess <
> > shayne.burg...@gmail.com
> > > >
> > > > a
> > > > > écrit :
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Deployed cluster in Azure and smoke tested a fairly complicated
> > data
> > > > > flow.
> > > > > >
> > > > > > Verified the fixes for the Azure EventHub Processor that were
> > pulled
> > > > into
> > > > > > RC3 are working as expected.
> > > > > >
> > > > > > Shayne
> > > > > >
> > > > > > On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick <
> > jzemer...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > +1 (non-binding)
> > > > > > >
> > > > > > > Built and ran successfully and built custom processor with 1.10.0
> > > > > > > dependency and successfully uploaded NAR to NiFi Registry 0.5.0.
> > > > > > >
> > > > > > > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne  > >
> > > > > wrote:
> > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > Was able to build and verify that all of the Jiras that I
> > raised
> > > > last
> > > > > > > time
> > > > > > > > around have been addressed in this RC. Left a cluster of 10
> > nodes
> > > > > > running
> > > > > > > > for a day or two, pretty heavily taxed, and ran into no issues.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > -Mark
> > > > > > > >
> > > > > > > >
> > > > > > > > > On Oct 30, 2019, at 3:20 PM, Matt Gilman <
> > > > matt.c.gil...@gmail.com>
> > > > > > > 

Re: Setup a call

[Pierre Villard]

Hi Pushkar,

This is the mailing list of a project belonging to the Apache Software
Foundation. Using the mailing lists (or the Slack channel) allows you to
reach out to the community and ask questions but this is based on a best
effort basis in line with the community guidelines of the foundation [1].

I invite you to ask your questions here and some people may be able to help
you (you might want to use the users@ mailing list instead of the dev@
one). Otherwise, if you are looking for a more "enterprise supported"
discussion, I'd encourage you getting in touch with a vendor providing
services and solutions around Apache NiFi.

[1] https://www.apache.org/theapacheway/index.html

Hope this helps,
Pierre

Le jeu. 31 oct. 2019 à 11:24, Pushkar Pokharna (External - Capgemini
Sverige AB)  a écrit :

> Hi,
>
>   Can we have a call regarding Apache Nifi. We are currently
> evaluating Apache Nifi and we came across some doubts. Can we have a call
> to discuss in details about following pointers.
>
> 1] Upsert operation in Apache Nifi.
> 2] How to capture updated record in one database and send the one in
> target system.
> 3] More on Product capability as ETL tool in replacement of Oracle Data
> Integrator.
>
> Regards,
> Pushkar.
>

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Kotaro Terada]

Thanks for taking care of it, Pierre and Joe!

> Please note only PMC votes are binding.

I got it,  I'll be careful next time.

Thanks,
Kotaro


On Thu, Oct 31, 2019 at 7:32 PM Joe Witt  wrote:

> Id just flag this on release notes and we can fix for the next release.
>
> Kotaro good find.  Please note only PMC votes are binding.
>
> Thanks
>
> On Thu, Oct 31, 2019 at 6:27 AM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > Good catch Kotaro! I merged the fix for #2. The fix for #1 LGTM but I'll
> > let the guys who worked on the Java 11 bits look into it.
> >
> > Joe, I didn't set the fix version on the JIRA and will wait until we
> reach
> > a decision regarding this RC.
> >
> > Le jeu. 31 oct. 2019 à 11:08, Kotaro Terada  a
> écrit :
> >
> > > -1 (binding), due to the following problems I found.
> > >
> > > (1) NiFi doesn't start up from the rpm build on Java 11.
> > >
> > > We have a mismatch in the "lib" directories of rpm and non-rpm build.
> > This
> > > causes that NiFi (with Java 11) doesn't start up from the rpm build.
> Some
> > > jar files should be located in the "lib/bootstrap/" but they are
> > currently
> > > in "lib/".
> > >
> > > I checked the following behaviors:
> > > - Non-RPM (usual) build with Java 8 and run on Java 8... OK
> > > - Non-RPM (usual) build with Java 11 and run on Java 11... OK
> > > - RPM build with Java 8 and run on Java 8... OK (isn't affected by the
> > lib
> > > mismatch)
> > > - RPM build with Java 11 and run on Java 11... NiFi doesn't start (this
> > > problem)
> > >
> > > I created a JIRA and a PR to fix it:
> > > https://issues.apache.org/jira/browse/NIFI-6827
> > >
> > > (2) There is an unintentional string in "conf/logback.xml".
> > >
> > > The "conf/logback.xml" seems to have an unintentional string
> > > "MockProcessContext". I guess this was slipped into it by the merging
> > > commit of https://github.com/apache/nifi/pull/3773 unintentionally.
> > >
> > > I created a JIRA and a PR to fix it:
> > > https://issues.apache.org/jira/browse/NIFI-6828
> > >
> > > Could you take a look at them, please?
> > >
> > > Thanks for working for the release!
> > > Kotaro
> > >
> > >
> > > On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard <
> > > pierre.villard...@gmail.com>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Went through the release helper guide and it LGTM. Deployed
> > > > secured/unsecured standalone/cluster instances on Google Cloud with
> > > various
> > > > flows including the new BigQuery processor and everything is looking
> > > good.
> > > >
> > > > Thanks Joe for taking care of the release!
> > > >
> > > > Pierre
> > > >
> > > > Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess <
> shayne.burg...@gmail.com
> > >
> > > a
> > > > écrit :
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Deployed cluster in Azure and smoke tested a fairly complicated
> data
> > > > flow.
> > > > >
> > > > > Verified the fixes for the Azure EventHub Processor that were
> pulled
> > > into
> > > > > RC3 are working as expected.
> > > > >
> > > > > Shayne
> > > > >
> > > > > On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick <
> jzemer...@apache.org>
> > > > > wrote:
> > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Built and ran successfully and built custom processor with 1.10.0
> > > > > > dependency and successfully uploaded NAR to NiFi Registry 0.5.0.
> > > > > >
> > > > > > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne  >
> > > > wrote:
> > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Was able to build and verify that all of the Jiras that I
> raised
> > > last
> > > > > > time
> > > > > > > around have been addressed in this RC. Left a cluster of 10
> nodes
> > > > > running
> > > > > > > for a day or two, pretty heavily taxed, and ran into no issues.
> > > > > > >
> > > > > > > Thanks
> > > > > > > -Mark
> > > > > > >
> > > > > > >
> > > > > > > > On Oct 30, 2019, at 3:20 PM, Matt Gilman <
> > > matt.c.gil...@gmail.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > Verified signatures, hashes, build, etc. Tested both
> standalone
> > > and
> > > > > > > > clustered and secure and unsecured.
> > > > > > > >
> > > > > > > > Thanks for RMing Joe!
> > > > > > > >
> > > > > > > > Matt
> > > > > > > >
> > > > > > > > On Wed, Oct 30, 2019 at 1:51 PM Adam Taft  >
> > > > wrote:
> > > > > > > >
> > > > > > > >> +1 (binding)
> > > > > > > >>
> > > > > > > >> Signatures verified.
> > > > > > > >> Hashes verified.
> > > > > > > >> Tests pass, source builds cleanly.
> > > > > > > >> I used both Java 11 & Java 8 to build.
> > > > > > > >>
> > > > > > > >> I did run into a problem compiling with Java 11 and running
> > with
> > > > > Java
> > > > > > > 8.  I
> > > > > > > >> don't believe this was a goal of the Java 11 compatibility
> > > > changes,
> > > > > so
> > > > > > > >> nothing unexpected about this. But it's possibly worth
> > > discussion
> > > > in
> > > > > > > >> another 

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Joe Witt]

Id just flag this on release notes and we can fix for the next release.

Kotaro good find.  Please note only PMC votes are binding.

Thanks

On Thu, Oct 31, 2019 at 6:27 AM Pierre Villard 
wrote:

> Good catch Kotaro! I merged the fix for #2. The fix for #1 LGTM but I'll
> let the guys who worked on the Java 11 bits look into it.
>
> Joe, I didn't set the fix version on the JIRA and will wait until we reach
> a decision regarding this RC.
>
> Le jeu. 31 oct. 2019 à 11:08, Kotaro Terada  a écrit :
>
> > -1 (binding), due to the following problems I found.
> >
> > (1) NiFi doesn't start up from the rpm build on Java 11.
> >
> > We have a mismatch in the "lib" directories of rpm and non-rpm build.
> This
> > causes that NiFi (with Java 11) doesn't start up from the rpm build. Some
> > jar files should be located in the "lib/bootstrap/" but they are
> currently
> > in "lib/".
> >
> > I checked the following behaviors:
> > - Non-RPM (usual) build with Java 8 and run on Java 8... OK
> > - Non-RPM (usual) build with Java 11 and run on Java 11... OK
> > - RPM build with Java 8 and run on Java 8... OK (isn't affected by the
> lib
> > mismatch)
> > - RPM build with Java 11 and run on Java 11... NiFi doesn't start (this
> > problem)
> >
> > I created a JIRA and a PR to fix it:
> > https://issues.apache.org/jira/browse/NIFI-6827
> >
> > (2) There is an unintentional string in "conf/logback.xml".
> >
> > The "conf/logback.xml" seems to have an unintentional string
> > "MockProcessContext". I guess this was slipped into it by the merging
> > commit of https://github.com/apache/nifi/pull/3773 unintentionally.
> >
> > I created a JIRA and a PR to fix it:
> > https://issues.apache.org/jira/browse/NIFI-6828
> >
> > Could you take a look at them, please?
> >
> > Thanks for working for the release!
> > Kotaro
> >
> >
> > On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard <
> > pierre.villard...@gmail.com>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Went through the release helper guide and it LGTM. Deployed
> > > secured/unsecured standalone/cluster instances on Google Cloud with
> > various
> > > flows including the new BigQuery processor and everything is looking
> > good.
> > >
> > > Thanks Joe for taking care of the release!
> > >
> > > Pierre
> > >
> > > Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess  >
> > a
> > > écrit :
> > >
> > > > +1 (non-binding)
> > > >
> > > > Deployed cluster in Azure and smoke tested a fairly complicated data
> > > flow.
> > > >
> > > > Verified the fixes for the Azure EventHub Processor that were pulled
> > into
> > > > RC3 are working as expected.
> > > >
> > > > Shayne
> > > >
> > > > On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick 
> > > > wrote:
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Built and ran successfully and built custom processor with 1.10.0
> > > > > dependency and successfully uploaded NAR to NiFi Registry 0.5.0.
> > > > >
> > > > > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne 
> > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Was able to build and verify that all of the Jiras that I raised
> > last
> > > > > time
> > > > > > around have been addressed in this RC. Left a cluster of 10 nodes
> > > > running
> > > > > > for a day or two, pretty heavily taxed, and ran into no issues.
> > > > > >
> > > > > > Thanks
> > > > > > -Mark
> > > > > >
> > > > > >
> > > > > > > On Oct 30, 2019, at 3:20 PM, Matt Gilman <
> > matt.c.gil...@gmail.com>
> > > > > > wrote:
> > > > > > >
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > Verified signatures, hashes, build, etc. Tested both standalone
> > and
> > > > > > > clustered and secure and unsecured.
> > > > > > >
> > > > > > > Thanks for RMing Joe!
> > > > > > >
> > > > > > > Matt
> > > > > > >
> > > > > > > On Wed, Oct 30, 2019 at 1:51 PM Adam Taft 
> > > wrote:
> > > > > > >
> > > > > > >> +1 (binding)
> > > > > > >>
> > > > > > >> Signatures verified.
> > > > > > >> Hashes verified.
> > > > > > >> Tests pass, source builds cleanly.
> > > > > > >> I used both Java 11 & Java 8 to build.
> > > > > > >>
> > > > > > >> I did run into a problem compiling with Java 11 and running
> with
> > > > Java
> > > > > > 8.  I
> > > > > > >> don't believe this was a goal of the Java 11 compatibility
> > > changes,
> > > > so
> > > > > > >> nothing unexpected about this. But it's possibly worth
> > discussion
> > > in
> > > > > > >> another thread (which I'll send out).  The convenience binary
> > was
> > > > > > compiled
> > > > > > >> with Java 8, so no problems with compatibility either way.
> > > > > > >>
> > > > > > >> On Tue, Oct 29, 2019 at 11:32 AM Joe Witt  >
> > > > wrote:
> > > > > > >>
> > > > > > >>> Hello,
> > > > > > >>>
> > > > > > >>> I am pleased to be calling this vote for the source release
> of
> > > > Apache
> > > > > > >> NiFi
> > > > > > >>> nifi-1.10.0.
> > > > > > >>>
> > > > > > >>> As they say 'third time's a charm'.
> > > > > > >>>
> > > > > > >>> The source zip, including signatures, 

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Pierre Villard]

Good catch Kotaro! I merged the fix for #2. The fix for #1 LGTM but I'll
let the guys who worked on the Java 11 bits look into it.

Joe, I didn't set the fix version on the JIRA and will wait until we reach
a decision regarding this RC.

Le jeu. 31 oct. 2019 à 11:08, Kotaro Terada  a écrit :

> -1 (binding), due to the following problems I found.
>
> (1) NiFi doesn't start up from the rpm build on Java 11.
>
> We have a mismatch in the "lib" directories of rpm and non-rpm build. This
> causes that NiFi (with Java 11) doesn't start up from the rpm build. Some
> jar files should be located in the "lib/bootstrap/" but they are currently
> in "lib/".
>
> I checked the following behaviors:
> - Non-RPM (usual) build with Java 8 and run on Java 8... OK
> - Non-RPM (usual) build with Java 11 and run on Java 11... OK
> - RPM build with Java 8 and run on Java 8... OK (isn't affected by the lib
> mismatch)
> - RPM build with Java 11 and run on Java 11... NiFi doesn't start (this
> problem)
>
> I created a JIRA and a PR to fix it:
> https://issues.apache.org/jira/browse/NIFI-6827
>
> (2) There is an unintentional string in "conf/logback.xml".
>
> The "conf/logback.xml" seems to have an unintentional string
> "MockProcessContext". I guess this was slipped into it by the merging
> commit of https://github.com/apache/nifi/pull/3773 unintentionally.
>
> I created a JIRA and a PR to fix it:
> https://issues.apache.org/jira/browse/NIFI-6828
>
> Could you take a look at them, please?
>
> Thanks for working for the release!
> Kotaro
>
>
> On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard <
> pierre.villard...@gmail.com>
> wrote:
>
> > +1 (binding)
> >
> > Went through the release helper guide and it LGTM. Deployed
> > secured/unsecured standalone/cluster instances on Google Cloud with
> various
> > flows including the new BigQuery processor and everything is looking
> good.
> >
> > Thanks Joe for taking care of the release!
> >
> > Pierre
> >
> > Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess 
> a
> > écrit :
> >
> > > +1 (non-binding)
> > >
> > > Deployed cluster in Azure and smoke tested a fairly complicated data
> > flow.
> > >
> > > Verified the fixes for the Azure EventHub Processor that were pulled
> into
> > > RC3 are working as expected.
> > >
> > > Shayne
> > >
> > > On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick 
> > > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > Built and ran successfully and built custom processor with 1.10.0
> > > > dependency and successfully uploaded NAR to NiFi Registry 0.5.0.
> > > >
> > > > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne 
> > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Was able to build and verify that all of the Jiras that I raised
> last
> > > > time
> > > > > around have been addressed in this RC. Left a cluster of 10 nodes
> > > running
> > > > > for a day or two, pretty heavily taxed, and ran into no issues.
> > > > >
> > > > > Thanks
> > > > > -Mark
> > > > >
> > > > >
> > > > > > On Oct 30, 2019, at 3:20 PM, Matt Gilman <
> matt.c.gil...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > Verified signatures, hashes, build, etc. Tested both standalone
> and
> > > > > > clustered and secure and unsecured.
> > > > > >
> > > > > > Thanks for RMing Joe!
> > > > > >
> > > > > > Matt
> > > > > >
> > > > > > On Wed, Oct 30, 2019 at 1:51 PM Adam Taft 
> > wrote:
> > > > > >
> > > > > >> +1 (binding)
> > > > > >>
> > > > > >> Signatures verified.
> > > > > >> Hashes verified.
> > > > > >> Tests pass, source builds cleanly.
> > > > > >> I used both Java 11 & Java 8 to build.
> > > > > >>
> > > > > >> I did run into a problem compiling with Java 11 and running with
> > > Java
> > > > > 8.  I
> > > > > >> don't believe this was a goal of the Java 11 compatibility
> > changes,
> > > so
> > > > > >> nothing unexpected about this. But it's possibly worth
> discussion
> > in
> > > > > >> another thread (which I'll send out).  The convenience binary
> was
> > > > > compiled
> > > > > >> with Java 8, so no problems with compatibility either way.
> > > > > >>
> > > > > >> On Tue, Oct 29, 2019 at 11:32 AM Joe Witt 
> > > wrote:
> > > > > >>
> > > > > >>> Hello,
> > > > > >>>
> > > > > >>> I am pleased to be calling this vote for the source release of
> > > Apache
> > > > > >> NiFi
> > > > > >>> nifi-1.10.0.
> > > > > >>>
> > > > > >>> As they say 'third time's a charm'.
> > > > > >>>
> > > > > >>> The source zip, including signatures, digests, etc. can be
> found
> > > at:
> > > > > >>>
> > > >
> https://repository.apache.org/content/repositories/orgapachenifi-1151
> > > > > >>>
> > > > > >>> The source being voted upon and the convenience binaries can be
> > > found
> > > > > at:
> > > > > >>> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.10.0/
> > > > > >>>
> > > > > >>> The Git tag is nifi-1.10.0-RC3
> > > > > >>> The Git commit ID is b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > 

Setup a call

[Pushkar Pokharna (External - Capgemini Sverige AB)]

Hi,

  Can we have a call regarding Apache Nifi. We are currently 
evaluating Apache Nifi and we came across some doubts. Can we have a call to 
discuss in details about following pointers.

1] Upsert operation in Apache Nifi.
2] How to capture updated record in one database and send the one in target 
system.
3] More on Product capability as ETL tool in replacement of Oracle Data 
Integrator.

Regards,
Pushkar.

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Kotaro Terada]

-1 (binding), due to the following problems I found.

(1) NiFi doesn't start up from the rpm build on Java 11.

We have a mismatch in the "lib" directories of rpm and non-rpm build. This
causes that NiFi (with Java 11) doesn't start up from the rpm build. Some
jar files should be located in the "lib/bootstrap/" but they are currently
in "lib/".

I checked the following behaviors:
- Non-RPM (usual) build with Java 8 and run on Java 8... OK
- Non-RPM (usual) build with Java 11 and run on Java 11... OK
- RPM build with Java 8 and run on Java 8... OK (isn't affected by the lib
mismatch)
- RPM build with Java 11 and run on Java 11... NiFi doesn't start (this
problem)

I created a JIRA and a PR to fix it:
https://issues.apache.org/jira/browse/NIFI-6827

(2) There is an unintentional string in "conf/logback.xml".

The "conf/logback.xml" seems to have an unintentional string
"MockProcessContext". I guess this was slipped into it by the merging
commit of https://github.com/apache/nifi/pull/3773 unintentionally.

I created a JIRA and a PR to fix it:
https://issues.apache.org/jira/browse/NIFI-6828

Could you take a look at them, please?

Thanks for working for the release!
Kotaro


On Thu, Oct 31, 2019 at 6:10 PM Pierre Villard 
wrote:

> +1 (binding)
>
> Went through the release helper guide and it LGTM. Deployed
> secured/unsecured standalone/cluster instances on Google Cloud with various
> flows including the new BigQuery processor and everything is looking good.
>
> Thanks Joe for taking care of the release!
>
> Pierre
>
> Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess  a
> écrit :
>
> > +1 (non-binding)
> >
> > Deployed cluster in Azure and smoke tested a fairly complicated data
> flow.
> >
> > Verified the fixes for the Azure EventHub Processor that were pulled into
> > RC3 are working as expected.
> >
> > Shayne
> >
> > On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick 
> > wrote:
> >
> > > +1 (non-binding)
> > >
> > > Built and ran successfully and built custom processor with 1.10.0
> > > dependency and successfully uploaded NAR to NiFi Registry 0.5.0.
> > >
> > > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne 
> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Was able to build and verify that all of the Jiras that I raised last
> > > time
> > > > around have been addressed in this RC. Left a cluster of 10 nodes
> > running
> > > > for a day or two, pretty heavily taxed, and ran into no issues.
> > > >
> > > > Thanks
> > > > -Mark
> > > >
> > > >
> > > > > On Oct 30, 2019, at 3:20 PM, Matt Gilman 
> > > > wrote:
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > Verified signatures, hashes, build, etc. Tested both standalone and
> > > > > clustered and secure and unsecured.
> > > > >
> > > > > Thanks for RMing Joe!
> > > > >
> > > > > Matt
> > > > >
> > > > > On Wed, Oct 30, 2019 at 1:51 PM Adam Taft 
> wrote:
> > > > >
> > > > >> +1 (binding)
> > > > >>
> > > > >> Signatures verified.
> > > > >> Hashes verified.
> > > > >> Tests pass, source builds cleanly.
> > > > >> I used both Java 11 & Java 8 to build.
> > > > >>
> > > > >> I did run into a problem compiling with Java 11 and running with
> > Java
> > > > 8.  I
> > > > >> don't believe this was a goal of the Java 11 compatibility
> changes,
> > so
> > > > >> nothing unexpected about this. But it's possibly worth discussion
> in
> > > > >> another thread (which I'll send out).  The convenience binary was
> > > > compiled
> > > > >> with Java 8, so no problems with compatibility either way.
> > > > >>
> > > > >> On Tue, Oct 29, 2019 at 11:32 AM Joe Witt 
> > wrote:
> > > > >>
> > > > >>> Hello,
> > > > >>>
> > > > >>> I am pleased to be calling this vote for the source release of
> > Apache
> > > > >> NiFi
> > > > >>> nifi-1.10.0.
> > > > >>>
> > > > >>> As they say 'third time's a charm'.
> > > > >>>
> > > > >>> The source zip, including signatures, digests, etc. can be found
> > at:
> > > > >>>
> > > https://repository.apache.org/content/repositories/orgapachenifi-1151
> > > > >>>
> > > > >>> The source being voted upon and the convenience binaries can be
> > found
> > > > at:
> > > > >>> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.10.0/
> > > > >>>
> > > > >>> The Git tag is nifi-1.10.0-RC3
> > > > >>> The Git commit ID is b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > > > >>>
> > > > >>> Checksums of nifi-1.10.0-source-release.zip:
> > > > >>> SHA256:
> > > > e9b0a14b3029acd69c6693781b6b6487c14dda12676db8b4a015bce23b1029c1
> > > > >>> SHA512:
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> b07258cbc21d2e529a1aa3098449917e2d059e6b45ffcfcb6df094931cf16caa8970576555164d3f2290cfe064b5780ba1a8bf63dad04d20100ed559a1cfe133
> > > > >>>
> > > > >>> Release artifacts are signed with the following key:
> > > > >>> https://people.apache.org/keys/committer/joewitt.asc
> > > > >>>
> > > > >>> KEYS file 

Re: [VOTE] Release Apache NiFi 1.10.0 (rc3)

[Pierre Villard]

+1 (binding)

Went through the release helper guide and it LGTM. Deployed
secured/unsecured standalone/cluster instances on Google Cloud with various
flows including the new BigQuery processor and everything is looking good.

Thanks Joe for taking care of the release!

Pierre

Le jeu. 31 oct. 2019 à 03:12, Shayne Burgess  a
écrit :

> +1 (non-binding)
>
> Deployed cluster in Azure and smoke tested a fairly complicated data flow.
>
> Verified the fixes for the Azure EventHub Processor that were pulled into
> RC3 are working as expected.
>
> Shayne
>
> On Wed, Oct 30, 2019 at 5:59 PM Jeff Zemerick 
> wrote:
>
> > +1 (non-binding)
> >
> > Built and ran successfully and built custom processor with 1.10.0
> > dependency and successfully uploaded NAR to NiFi Registry 0.5.0.
> >
> > On Wed, Oct 30, 2019 at 4:06 PM Mark Payne  wrote:
> >
> > > +1 (binding)
> > >
> > > Was able to build and verify that all of the Jiras that I raised last
> > time
> > > around have been addressed in this RC. Left a cluster of 10 nodes
> running
> > > for a day or two, pretty heavily taxed, and ran into no issues.
> > >
> > > Thanks
> > > -Mark
> > >
> > >
> > > > On Oct 30, 2019, at 3:20 PM, Matt Gilman 
> > > wrote:
> > > >
> > > > +1 (binding)
> > > >
> > > > Verified signatures, hashes, build, etc. Tested both standalone and
> > > > clustered and secure and unsecured.
> > > >
> > > > Thanks for RMing Joe!
> > > >
> > > > Matt
> > > >
> > > > On Wed, Oct 30, 2019 at 1:51 PM Adam Taft  wrote:
> > > >
> > > >> +1 (binding)
> > > >>
> > > >> Signatures verified.
> > > >> Hashes verified.
> > > >> Tests pass, source builds cleanly.
> > > >> I used both Java 11 & Java 8 to build.
> > > >>
> > > >> I did run into a problem compiling with Java 11 and running with
> Java
> > > 8.  I
> > > >> don't believe this was a goal of the Java 11 compatibility changes,
> so
> > > >> nothing unexpected about this. But it's possibly worth discussion in
> > > >> another thread (which I'll send out).  The convenience binary was
> > > compiled
> > > >> with Java 8, so no problems with compatibility either way.
> > > >>
> > > >> On Tue, Oct 29, 2019 at 11:32 AM Joe Witt 
> wrote:
> > > >>
> > > >>> Hello,
> > > >>>
> > > >>> I am pleased to be calling this vote for the source release of
> Apache
> > > >> NiFi
> > > >>> nifi-1.10.0.
> > > >>>
> > > >>> As they say 'third time's a charm'.
> > > >>>
> > > >>> The source zip, including signatures, digests, etc. can be found
> at:
> > > >>>
> > https://repository.apache.org/content/repositories/orgapachenifi-1151
> > > >>>
> > > >>> The source being voted upon and the convenience binaries can be
> found
> > > at:
> > > >>> https://dist.apache.org/repos/dist/dev/nifi/nifi-1.10.0/
> > > >>>
> > > >>> The Git tag is nifi-1.10.0-RC3
> > > >>> The Git commit ID is b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=b217ae20ad6a04cac874b2b00d93b7f7514c0b88
> > > >>>
> > > >>> Checksums of nifi-1.10.0-source-release.zip:
> > > >>> SHA256:
> > > e9b0a14b3029acd69c6693781b6b6487c14dda12676db8b4a015bce23b1029c1
> > > >>> SHA512:
> > > >>>
> > > >>>
> > > >>
> > >
> >
> b07258cbc21d2e529a1aa3098449917e2d059e6b45ffcfcb6df094931cf16caa8970576555164d3f2290cfe064b5780ba1a8bf63dad04d20100ed559a1cfe133
> > > >>>
> > > >>> Release artifacts are signed with the following key:
> > > >>> https://people.apache.org/keys/committer/joewitt.asc
> > > >>>
> > > >>> KEYS file available here:
> > > >>> https://dist.apache.org/repos/dist/release/nifi/KEYS
> > > >>>
> > > >>> 384 issues were closed/resolved for this release:
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12344993
> > > >>>
> > > >>> Release note highlights can be found here:
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.10.0
> > > >>>
> > > >>> The vote will be open for 72 hours.
> > > >>> Please download the release candidate and evaluate the necessary
> > items
> > > >>> including checking hashes, signatures, build
> > > >>> from source, and test. Then please vote:
> > > >>>
> > > >>> [ ] +1 Release this package as nifi-1.10.0
> > > >>> [ ] +0 no opinion
> > > >>> [ ] -1 Do not release this package because...
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Minifi C2 - connection to secured NIFI API

[Pierre Villard]

Hi Ali,

If I remember correctly this should be possible. Can you share the
configuration you tried and potential errors you got?

Pierre

Le mer. 30 oct. 2019 à 18:02, Ali C  a écrit :

> Hi All,
> I'm trying to setup Minifi C2 with a connection to the Nifi API to pull
> templates and distribute them to MINIFI nodes. In my case the NIFI server
> is running in certificate / SSL auth mode (using the NIFI toolkit for
> testing). Is it possible to get C2 to connect to the API in this case? I
> have been able to configure C2 to require ssl connections to it, but cannot
> find a way to get it to authenticate to NIFI in order to access the API.
>
> Any help would be appreciated,
> Ali
> This information is exempt under the Freedom of Information Act 2000
> (FOIA) and may be exempt under other UK information legislation. Refer any
> FOIA queries to ncscinfo...@ncsc.gov.uk. All material is UK Crown
> Copyright (c)
>