Re: NiFi, Docker Environment Variable for enabling debugging of NiFi inside a Docker container

[Andrew Grande]

Here's what I did previously, Erik. A user only had to specify an
additional -debug flag for the main shell script which would in turn take
care of any bootstrap rewriting/generation for allowing for the remote JVM
debug session to connect. Maybe it could give a few ideas.

Andrew

On Wed, Feb 13, 2019, 7:34 AM Erik Anderson  wrote:

> > I was reading that email and was thinking of JVM debug options, with
> > suspend y/n. I guess it just shows we meant very different things by
> debug
> > mode. Maybe you could incorporate those into a PR too?
> >
> > Andrew
>
> Good point Andrew. and sorry for the slow response. I had to look at how
> NiFi set the JVM properties.
>
> We had issues with Java and needed to manually set env Java flags.
>
> It seems all of the Java debugging flags are set in the bootstrap.conf,
> located here.
>
>
> https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf
>
> For me, I chained together the Dockerfile
> ENTRYPOINT ["my_company_entry_point.sh"]
> then had it call the original entry point
> In short, all of NiFi properties are set via environment variables in a
> /etc/profile.d/nifi.sh on the host and propogate throughout the NiFi
> system. Like I listed in
>
> https://github.com/dprophet/nifi/blob/master/nifi-docker/dockerhub/CONFIGURATION.md
>
> I normalized all NIFI specific environment variables to start with NIFI_
>
> Why? Because you set the
>
> export NIFI_FOO="my foo"
>
> and its directly passed into the container and into the NiFi role account
> used to start the container. You now have access to the environment
> variables throughout the NiFi system.
>
> Example:
> docker run --name nifi --env-file <(env | grep NIFI_) --hostname nifi
>
> IMO, the Dockerfile entry point should allow a plug and play script so you
> can set these custom behaviors (both enterprise behaviors and custom
> developer/debugging). I doubt any enterprise will blindly pull a DockerHub
> container and run it. From my experience, a public container isnt
> enterprise friendly.
>
> Andrew, define what you want for JVM debugging, what you would want to set
> (and unset), and I will take a look.
>
> Erik Anderson
> Bloomberg
> https://www.linkedin.com/in/erikanderson/
>

Re: Controlling number of threads in distributed mode

[Joe Witt]

Ok.  So looks like the information needed to make a request comes from a
Kafka topic.
Then requests are made against some HTTP endpoint for which it wants a max
number of requests at once.
Given how small that number is I'd not worry at all about nifi being
overloaded but perhaps there is more depth to this.

I'd just turn on load balancing on the connection between ConsumeKafka and
InvokeHttpRequest and let the messages be consumed and requests sent
naturally.

Thanks

On Wed, Feb 13, 2019 at 9:52 PM Milan Das  wrote:

> Hi Joe,
> Use case is I have to query HTTP request.  The HTTP  request  from  NIFI
> can only run 3 queries at a time.
>
> The queries to be executed are coming from Kafka topic and we have 6 node
> cluster.
> Ideally would like to run the 3 http queries from 3 different nodes
> otherwise one of the node (primary node) will be overloaded with data.
> Thanks,
> Milan Das
>
>
>
> > On Feb 13, 2019, at 4:38 PM, Joe Witt  wrote:
> >
> > Milan
> >
> > The only mechanism to limit whether a given component executes on a given
> > nifi node or not is whether it runs on all or whether it runs on primary
> > node.
> >
> > Can you describe the use case that would lead to wanting to run a
> component
> > on a specified number of nodes as opposed to all nodes?  It would defy
> the
> > clustering logic/data distribution mechanisms of site to site and load
> > balanced connections for example.
> >
> > Thanks
> >
> > On Wed, Feb 13, 2019 at 4:34 PM Milan Das  wrote:
> >
> >> Hello,
> >> Is there a way to control number of thread less than cluster member ?
> >> I have a 5 node NiFI cluster and I want to run 3 instances of the
> >> processor in distributed mode.
> >>
> >> Thanks,
> >> Milan Das
> >>
> >>
>
>

Apache NiFi 1.9.0 RC1 Release Helper Guide

[Joe Witt]

Hello Apache NiFi community,

Please find the associated guidance to help those interested in
validating/verifying the release so they can vote.

# Download latest KEYS file:
https://dist.apache.org/repos/dist/dev/nifi/KEYS

# Import keys file:
gpg --import KEYS

# [optional] Clear out local maven artifact repository

# Pull down nifi-1.9.0 source release artifacts for review:

wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.9.0-rc1/nifi-1.9.0-source-release.zip
wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.9.0-rc1/nifi-1.9.0-source-release.zip.asc
wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.9.0-rc1/nifi-1.9.0-source-release.zip.sha256
wget
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.9.0-rc1/nifi-1.9.0-source-release.zip.sha512

# Verify the signature
gpg --verify nifi-1.9.0-source-release.zip.asc

# Verify the hashes (sha256, sha512) match the source and what was provided
in the vote email thread
shasum -a 256 nifi-1.9.0-source-release.zip
shasum -a 512 nifi-1.9.0-source-release.zip

# Unzip nifi-1.9.0-source-release.zip

# Verify the build works including release audit tool (RAT) checks
cd nifi-1.9.0
mvn clean install -Pcontrib-check,include-grpc

# Verify the contents contain a good README, NOTICE, and LICENSE.

# Verify the git commit ID is correct

# Verify the RC was branched off the correct git commit ID

# Look at the resulting convenience binary as found in nifi-assembly/target

# Make sure the README, NOTICE, and LICENSE are present and correct

# Run the resulting convenience binary and make sure it works as expected

# Send a response to the vote thread indicating a +1, 0, -1 based on your
findings.

Thank you for your time and effort to validate the release!

[VOTE] Release Apache NiFi 1.9.0 (rc1)

[Joe Witt]

Hello,

I am pleased to be calling this vote for the source release of Apache NiFi
nifi-1.9.0.

The source zip, including signatures, digests, etc. can be found at:
https://repository.apache.org/content/repositories/orgapachenifi-1136

The Git tag is nifi-1.9.0-RC1
The Git commit ID is 75233d0c05275a587bcf4b7496f4bb797f5026d6
https://gitbox.apache.org/repos/asf?p=nifi.git;a=commit;h=75233d0c05275a587bcf4b7496f4bb797f5026d6

Checksums of nifi-1.9.0-source-release.zip:
SHA256:
a29244e195c4d4d862fc4209fde3103828fbd5795fa022a3732b337c020d3603
SHA512:
bcba2545337cf657f24b1ff3e85f7538b75e36b133256b95a726a62cdf10c2baf89c4d12411fc2cea442a76c65f4e4119976e536ed548138d1820c323d7d9c27

Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/joewitt.asc

KEYS file available here:
https://dist.apache.org/repos/dist/release/nifi/KEYS

147 issues were closed/resolved for this release:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12316020=12344357

Release note highlights can be found here:
https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version1.9.0
https://dist.apache.org/repos/dist/dev/nifi/nifi-1.9.0-rc1/

The vote will be open for 72 hours.
Please download the release candidate and evaluate the necessary items
including checking hashes, signatures, build
from source, and test. Then please vote:

[ ] +1 Release this package as nifi-1.9.0
[ ] +0 no opinion
[ ] -1 Do not release this package because...

Re: Controlling number of threads in distributed mode

[Milan Das]

Hi Joe,
Use case is I have to query HTTP request.  The HTTP  request  from  NIFI can 
only run 3 queries at a time.

The queries to be executed are coming from Kafka topic and we have 6 node 
cluster.
Ideally would like to run the 3 http queries from 3 different nodes otherwise 
one of the node (primary node) will be overloaded with data.
Thanks,
Milan Das



> On Feb 13, 2019, at 4:38 PM, Joe Witt  wrote:
> 
> Milan
> 
> The only mechanism to limit whether a given component executes on a given
> nifi node or not is whether it runs on all or whether it runs on primary
> node.
> 
> Can you describe the use case that would lead to wanting to run a component
> on a specified number of nodes as opposed to all nodes?  It would defy the
> clustering logic/data distribution mechanisms of site to site and load
> balanced connections for example.
> 
> Thanks
> 
> On Wed, Feb 13, 2019 at 4:34 PM Milan Das  wrote:
> 
>> Hello,
>> Is there a way to control number of thread less than cluster member ?
>> I have a 5 node NiFI cluster and I want to run 3 instances of the
>> processor in distributed mode.
>> 
>> Thanks,
>> Milan Das
>> 
>> 

Re: [EXT] [discuss] release apache nifi 1.9.0

[Joe Percivall]

I appreciate bringing it up Mike but +1 to not including it. The PR is not
a small number of lines and the ticket is not blocking priority so we don't
want to hold off the RC for it.

Cheers,
Joe

On Wed, Feb 13, 2019 at 5:48 PM Joe Witt  wrote:

> No the release build is underway.
>
> If this fails for some reason or RC1 fails we can pick up what new
> happens.  Else we can get them in the next rel.
>
> Thanks
>
> On Wed, Feb 13, 2019 at 5:46 PM Mike Thomsen 
> wrote:
>
> > Before I forget, Joe P dropped this:
> >
> > https://github.com/apache/nifi/pull/3299
> >
> > If I understand it correctly, the ticket aims to address the ugly side
> of a
> > bulk put partly failing w/ Elastic.
> >
> > Should we tag that 1.9?
> >
> > On Wed, Feb 13, 2019 at 5:44 PM Mike Thomsen 
> > wrote:
> >
> > > Would also be good to give Sivaprasanna some time to finish
> > > reviewing/merging 5987 and 5988 which are the two GetMongo fixes that
> > were
> > > requested on users about two weeks ago.
> > >
> > > On Wed, Feb 13, 2019 at 5:31 PM Joe Witt  wrote:
> > >
> > >> no worries - go for it.
> > >>
> > >> On Wed, Feb 13, 2019 at 5:31 PM Matt Burgess 
> > >> wrote:
> > >>
> > >> > Joe,
> > >> >
> > >> > Mike Thomsen's GridFS PR is ready to go, can we get it in real
> quick?
> > >> >
> > >> > Thanks,
> > >> > Matt
> > >> >
> > >> > On Wed, Feb 13, 2019 at 5:06 PM Joe Witt 
> wrote:
> > >> > >
> > >> > > Alright !  1.9.0 looks ready for an RC build.  Will start working
> on
> > >> this
> > >> > > tonight ideally.
> > >> > >
> > >> > > Thanks
> > >> > >
> > >> > > On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:
> > >> > >
> > >> > > > Joe,
> > >> > > >
> > >> > > > Yes, I'll review it today.
> > >> > > >
> > >> > > > On Mon, Feb 11, 2019, 8:44 AM Joe Witt 
> > wrote:
> > >> > > >
> > >> > > > > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going
> to
> > >> > > > > review/merge that then?
> > >> > > > >
> > >> > > > > There are 5 remaining JIRAs tagged to 1.9.0 that aren't
> > obviously
> > >> > > > puntable
> > >> > > > > and nearing completion of review/merge.  Will monitor/poke
> those
> > >> with
> > >> > > > > intent to kick off RC once they're all done.
> > >> > > > >
> > >> > > > > Thanks
> > >> > > > >
> > >> > > > > On Fri, Feb 8, 2019 at 3:38 PM Jeff 
> wrote:
> > >> > > > >
> > >> > > > > > https://issues.apache.org/jira/browse/NIFI-5575 (
> > >> > > > > > https://github.com/apache/nifi/pull/3252) would be a good
> > >> > addition in
> > >> > > > > > 1.9.0, in my opinion.  It attempts to address PutHDFS
> ignoring
> > >> the
> > >> > file
> > >> > > > > > mask default value from hdfs-site.xml when no default mask
> is
> > >> > specified
> > >> > > > > in
> > >> > > > > > the PutHDFS configuration.
> > >> > > > > >
> > >> > > > > > I can do the review on it, if we want to include it.
> > >> > > > > >
> > >> > > > > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto <
> > >> alopre...@apache.org
> > >> > >
> > >> > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Ah, sorry, I misunderstood. You are not the PR author; you
> > >> want
> > >> > those
> > >> > > > > > same
> > >> > > > > > > comments to be addressed.
> > >> > > > > > >
> > >> > > > > > > Andy LoPresto
> > >> > > > > > > alopre...@apache.org
> > >> > > > > > > alopresto.apa...@gmail.com
> > >> > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B
> > 2F7D
> > >> > EF69
> > >> > > > > > >
> > >> > > > > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto <
> > >> > alopre...@apache.org>
> > >> > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > Charlie, I see comments from Matt Burgess (especially
> > >> > considering
> > >> > > > > > > exceptions & stack traces) from 9 days ago that don’t
> appear
> > >> to
> > >> > be
> > >> > > > > > > addressed. Were you waiting for additional information?
> Am I
> > >> > missing
> > >> > > > > > > something? Thanks.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Andy LoPresto
> > >> > > > > > > > alopre...@apache.org 
> > >> > > > > > > > alopresto.apa...@gmail.com
> > >> > > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E
> F65B
> > >> 2F7D
> > >> > EF69
> > >> > > > > > > >
> > >> > > > > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
> > >> > > > > > > charlie.me...@civitaslearning.com  > >> > > > > > > charlie.me...@civitaslearning.com>> wrote:
> > >> > > > > > > >>
> > >> > > > > > > >> Hoping I am not too late to the party, but it would be
> > >> > fantastic
> > >> > > > if
> > >> > > > > > > >> https://github.com/apache/nifi/pull/3253 <
> > >> > > > > > > https://github.com/apache/nifi/pull/3253> could have the
> > >> > comments on
> > >> > > > > it
> > >> > > > > > > >> addressed and included the 1.9 release
> > >> > > > > > > >>
> > >> > > > > > > >> -Charlie
> > >> > > > > > > >>
> > >> > > > > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt <
> > >> joe.w...@gmail.com
> > >> > > > >  > >> > > > > > > joe.w...@gmail.com>> wrote:

Re: [EXT] [discuss] release apache nifi 1.9.0

[Joe Witt]

No the release build is underway.

If this fails for some reason or RC1 fails we can pick up what new
happens.  Else we can get them in the next rel.

Thanks

On Wed, Feb 13, 2019 at 5:46 PM Mike Thomsen  wrote:

> Before I forget, Joe P dropped this:
>
> https://github.com/apache/nifi/pull/3299
>
> If I understand it correctly, the ticket aims to address the ugly side of a
> bulk put partly failing w/ Elastic.
>
> Should we tag that 1.9?
>
> On Wed, Feb 13, 2019 at 5:44 PM Mike Thomsen 
> wrote:
>
> > Would also be good to give Sivaprasanna some time to finish
> > reviewing/merging 5987 and 5988 which are the two GetMongo fixes that
> were
> > requested on users about two weeks ago.
> >
> > On Wed, Feb 13, 2019 at 5:31 PM Joe Witt  wrote:
> >
> >> no worries - go for it.
> >>
> >> On Wed, Feb 13, 2019 at 5:31 PM Matt Burgess 
> >> wrote:
> >>
> >> > Joe,
> >> >
> >> > Mike Thomsen's GridFS PR is ready to go, can we get it in real quick?
> >> >
> >> > Thanks,
> >> > Matt
> >> >
> >> > On Wed, Feb 13, 2019 at 5:06 PM Joe Witt  wrote:
> >> > >
> >> > > Alright !  1.9.0 looks ready for an RC build.  Will start working on
> >> this
> >> > > tonight ideally.
> >> > >
> >> > > Thanks
> >> > >
> >> > > On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:
> >> > >
> >> > > > Joe,
> >> > > >
> >> > > > Yes, I'll review it today.
> >> > > >
> >> > > > On Mon, Feb 11, 2019, 8:44 AM Joe Witt 
> wrote:
> >> > > >
> >> > > > > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going to
> >> > > > > review/merge that then?
> >> > > > >
> >> > > > > There are 5 remaining JIRAs tagged to 1.9.0 that aren't
> obviously
> >> > > > puntable
> >> > > > > and nearing completion of review/merge.  Will monitor/poke those
> >> with
> >> > > > > intent to kick off RC once they're all done.
> >> > > > >
> >> > > > > Thanks
> >> > > > >
> >> > > > > On Fri, Feb 8, 2019 at 3:38 PM Jeff  wrote:
> >> > > > >
> >> > > > > > https://issues.apache.org/jira/browse/NIFI-5575 (
> >> > > > > > https://github.com/apache/nifi/pull/3252) would be a good
> >> > addition in
> >> > > > > > 1.9.0, in my opinion.  It attempts to address PutHDFS ignoring
> >> the
> >> > file
> >> > > > > > mask default value from hdfs-site.xml when no default mask is
> >> > specified
> >> > > > > in
> >> > > > > > the PutHDFS configuration.
> >> > > > > >
> >> > > > > > I can do the review on it, if we want to include it.
> >> > > > > >
> >> > > > > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto <
> >> alopre...@apache.org
> >> > >
> >> > > > > wrote:
> >> > > > > >
> >> > > > > > > Ah, sorry, I misunderstood. You are not the PR author; you
> >> want
> >> > those
> >> > > > > > same
> >> > > > > > > comments to be addressed.
> >> > > > > > >
> >> > > > > > > Andy LoPresto
> >> > > > > > > alopre...@apache.org
> >> > > > > > > alopresto.apa...@gmail.com
> >> > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B
> 2F7D
> >> > EF69
> >> > > > > > >
> >> > > > > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto <
> >> > alopre...@apache.org>
> >> > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > Charlie, I see comments from Matt Burgess (especially
> >> > considering
> >> > > > > > > exceptions & stack traces) from 9 days ago that don’t appear
> >> to
> >> > be
> >> > > > > > > addressed. Were you waiting for additional information? Am I
> >> > missing
> >> > > > > > > something? Thanks.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Andy LoPresto
> >> > > > > > > > alopre...@apache.org 
> >> > > > > > > > alopresto.apa...@gmail.com
> >> > > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B
> >> 2F7D
> >> > EF69
> >> > > > > > > >
> >> > > > > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
> >> > > > > > > charlie.me...@civitaslearning.com  >> > > > > > > charlie.me...@civitaslearning.com>> wrote:
> >> > > > > > > >>
> >> > > > > > > >> Hoping I am not too late to the party, but it would be
> >> > fantastic
> >> > > > if
> >> > > > > > > >> https://github.com/apache/nifi/pull/3253 <
> >> > > > > > > https://github.com/apache/nifi/pull/3253> could have the
> >> > comments on
> >> > > > > it
> >> > > > > > > >> addressed and included the 1.9 release
> >> > > > > > > >>
> >> > > > > > > >> -Charlie
> >> > > > > > > >>
> >> > > > > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt <
> >> joe.w...@gmail.com
> >> > > > >  >> > > > > > > joe.w...@gmail.com>> wrote:
> >> > > > > > > >>
> >> > > > > > > >>> MarkB - i dont have any issue with that being include
> but
> >> we
> >> > need
> >> > > > > to
> >> > > > > > > find
> >> > > > > > > >>> someone familiar enough with that to review it.  If that
> >> > happens
> >> > > > in
> >> > > > > > the
> >> > > > > > > >>> next couple days then great.  Any takers familiar with
> JMS
> >> > and
> >> > > > that
> >> > > > > > > class?
> >> > > > > > > >>> The changes are few but the implications are unclear to
> >> me.
> >> > > > > > > >>>
> >> > > > > > > >>> Thanks

Re: [EXT] [discuss] release apache nifi 1.9.0

[Mike Thomsen]

Before I forget, Joe P dropped this:

https://github.com/apache/nifi/pull/3299

If I understand it correctly, the ticket aims to address the ugly side of a
bulk put partly failing w/ Elastic.

Should we tag that 1.9?

On Wed, Feb 13, 2019 at 5:44 PM Mike Thomsen  wrote:

> Would also be good to give Sivaprasanna some time to finish
> reviewing/merging 5987 and 5988 which are the two GetMongo fixes that were
> requested on users about two weeks ago.
>
> On Wed, Feb 13, 2019 at 5:31 PM Joe Witt  wrote:
>
>> no worries - go for it.
>>
>> On Wed, Feb 13, 2019 at 5:31 PM Matt Burgess 
>> wrote:
>>
>> > Joe,
>> >
>> > Mike Thomsen's GridFS PR is ready to go, can we get it in real quick?
>> >
>> > Thanks,
>> > Matt
>> >
>> > On Wed, Feb 13, 2019 at 5:06 PM Joe Witt  wrote:
>> > >
>> > > Alright !  1.9.0 looks ready for an RC build.  Will start working on
>> this
>> > > tonight ideally.
>> > >
>> > > Thanks
>> > >
>> > > On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:
>> > >
>> > > > Joe,
>> > > >
>> > > > Yes, I'll review it today.
>> > > >
>> > > > On Mon, Feb 11, 2019, 8:44 AM Joe Witt  wrote:
>> > > >
>> > > > > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going to
>> > > > > review/merge that then?
>> > > > >
>> > > > > There are 5 remaining JIRAs tagged to 1.9.0 that aren't obviously
>> > > > puntable
>> > > > > and nearing completion of review/merge.  Will monitor/poke those
>> with
>> > > > > intent to kick off RC once they're all done.
>> > > > >
>> > > > > Thanks
>> > > > >
>> > > > > On Fri, Feb 8, 2019 at 3:38 PM Jeff  wrote:
>> > > > >
>> > > > > > https://issues.apache.org/jira/browse/NIFI-5575 (
>> > > > > > https://github.com/apache/nifi/pull/3252) would be a good
>> > addition in
>> > > > > > 1.9.0, in my opinion.  It attempts to address PutHDFS ignoring
>> the
>> > file
>> > > > > > mask default value from hdfs-site.xml when no default mask is
>> > specified
>> > > > > in
>> > > > > > the PutHDFS configuration.
>> > > > > >
>> > > > > > I can do the review on it, if we want to include it.
>> > > > > >
>> > > > > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto <
>> alopre...@apache.org
>> > >
>> > > > > wrote:
>> > > > > >
>> > > > > > > Ah, sorry, I misunderstood. You are not the PR author; you
>> want
>> > those
>> > > > > > same
>> > > > > > > comments to be addressed.
>> > > > > > >
>> > > > > > > Andy LoPresto
>> > > > > > > alopre...@apache.org
>> > > > > > > alopresto.apa...@gmail.com
>> > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
>> > EF69
>> > > > > > >
>> > > > > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto <
>> > alopre...@apache.org>
>> > > > > > wrote:
>> > > > > > > >
>> > > > > > > > Charlie, I see comments from Matt Burgess (especially
>> > considering
>> > > > > > > exceptions & stack traces) from 9 days ago that don’t appear
>> to
>> > be
>> > > > > > > addressed. Were you waiting for additional information? Am I
>> > missing
>> > > > > > > something? Thanks.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Andy LoPresto
>> > > > > > > > alopre...@apache.org 
>> > > > > > > > alopresto.apa...@gmail.com
>> > > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B
>> 2F7D
>> > EF69
>> > > > > > > >
>> > > > > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
>> > > > > > > charlie.me...@civitaslearning.com > > > > > > > charlie.me...@civitaslearning.com>> wrote:
>> > > > > > > >>
>> > > > > > > >> Hoping I am not too late to the party, but it would be
>> > fantastic
>> > > > if
>> > > > > > > >> https://github.com/apache/nifi/pull/3253 <
>> > > > > > > https://github.com/apache/nifi/pull/3253> could have the
>> > comments on
>> > > > > it
>> > > > > > > >> addressed and included the 1.9 release
>> > > > > > > >>
>> > > > > > > >> -Charlie
>> > > > > > > >>
>> > > > > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt <
>> joe.w...@gmail.com
>> > > > > > > > > > > > joe.w...@gmail.com>> wrote:
>> > > > > > > >>
>> > > > > > > >>> MarkB - i dont have any issue with that being include but
>> we
>> > need
>> > > > > to
>> > > > > > > find
>> > > > > > > >>> someone familiar enough with that to review it.  If that
>> > happens
>> > > > in
>> > > > > > the
>> > > > > > > >>> next couple days then great.  Any takers familiar with JMS
>> > and
>> > > > that
>> > > > > > > class?
>> > > > > > > >>> The changes are few but the implications are unclear to
>> me.
>> > > > > > > >>>
>> > > > > > > >>> Thanks
>> > > > > > > >>>
>> > > > > > > >>> On Wed, Feb 6, 2019 at 9:55 PM Mark Bean <
>> > mark.o.b...@gmail.com
>> > > > > > > > wrote:
>> > > > > > > >>>
>> > > > > > >  I have an open PR I'd like included in 1.9. It's a very
>> > simple
>> > > > > one.
>> > > > > > > 
>> > > > > > >  https://github.com/apache/nifi/pull/3053 <
>> > > > > > > https://github.com/apache/nifi/pull/3053>
>> > > > > > > 
>> > > > > > >  Thanks,
>> > > > > > >  Mark
>> > > > > > > 

Re: [EXT] [discuss] release apache nifi 1.9.0

[Mike Thomsen]

Would also be good to give Sivaprasanna some time to finish
reviewing/merging 5987 and 5988 which are the two GetMongo fixes that were
requested on users about two weeks ago.

On Wed, Feb 13, 2019 at 5:31 PM Joe Witt  wrote:

> no worries - go for it.
>
> On Wed, Feb 13, 2019 at 5:31 PM Matt Burgess  wrote:
>
> > Joe,
> >
> > Mike Thomsen's GridFS PR is ready to go, can we get it in real quick?
> >
> > Thanks,
> > Matt
> >
> > On Wed, Feb 13, 2019 at 5:06 PM Joe Witt  wrote:
> > >
> > > Alright !  1.9.0 looks ready for an RC build.  Will start working on
> this
> > > tonight ideally.
> > >
> > > Thanks
> > >
> > > On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:
> > >
> > > > Joe,
> > > >
> > > > Yes, I'll review it today.
> > > >
> > > > On Mon, Feb 11, 2019, 8:44 AM Joe Witt  wrote:
> > > >
> > > > > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going to
> > > > > review/merge that then?
> > > > >
> > > > > There are 5 remaining JIRAs tagged to 1.9.0 that aren't obviously
> > > > puntable
> > > > > and nearing completion of review/merge.  Will monitor/poke those
> with
> > > > > intent to kick off RC once they're all done.
> > > > >
> > > > > Thanks
> > > > >
> > > > > On Fri, Feb 8, 2019 at 3:38 PM Jeff  wrote:
> > > > >
> > > > > > https://issues.apache.org/jira/browse/NIFI-5575 (
> > > > > > https://github.com/apache/nifi/pull/3252) would be a good
> > addition in
> > > > > > 1.9.0, in my opinion.  It attempts to address PutHDFS ignoring
> the
> > file
> > > > > > mask default value from hdfs-site.xml when no default mask is
> > specified
> > > > > in
> > > > > > the PutHDFS configuration.
> > > > > >
> > > > > > I can do the review on it, if we want to include it.
> > > > > >
> > > > > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto <
> alopre...@apache.org
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Ah, sorry, I misunderstood. You are not the PR author; you want
> > those
> > > > > > same
> > > > > > > comments to be addressed.
> > > > > > >
> > > > > > > Andy LoPresto
> > > > > > > alopre...@apache.org
> > > > > > > alopresto.apa...@gmail.com
> > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> > EF69
> > > > > > >
> > > > > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto <
> > alopre...@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > Charlie, I see comments from Matt Burgess (especially
> > considering
> > > > > > > exceptions & stack traces) from 9 days ago that don’t appear to
> > be
> > > > > > > addressed. Were you waiting for additional information? Am I
> > missing
> > > > > > > something? Thanks.
> > > > > > > >
> > > > > > > >
> > > > > > > > Andy LoPresto
> > > > > > > > alopre...@apache.org 
> > > > > > > > alopresto.apa...@gmail.com
> > > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B
> 2F7D
> > EF69
> > > > > > > >
> > > > > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
> > > > > > > charlie.me...@civitaslearning.com  > > > > > > charlie.me...@civitaslearning.com>> wrote:
> > > > > > > >>
> > > > > > > >> Hoping I am not too late to the party, but it would be
> > fantastic
> > > > if
> > > > > > > >> https://github.com/apache/nifi/pull/3253 <
> > > > > > > https://github.com/apache/nifi/pull/3253> could have the
> > comments on
> > > > > it
> > > > > > > >> addressed and included the 1.9 release
> > > > > > > >>
> > > > > > > >> -Charlie
> > > > > > > >>
> > > > > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt  > > > >  > > > > > > joe.w...@gmail.com>> wrote:
> > > > > > > >>
> > > > > > > >>> MarkB - i dont have any issue with that being include but
> we
> > need
> > > > > to
> > > > > > > find
> > > > > > > >>> someone familiar enough with that to review it.  If that
> > happens
> > > > in
> > > > > > the
> > > > > > > >>> next couple days then great.  Any takers familiar with JMS
> > and
> > > > that
> > > > > > > class?
> > > > > > > >>> The changes are few but the implications are unclear to me.
> > > > > > > >>>
> > > > > > > >>> Thanks
> > > > > > > >>>
> > > > > > > >>> On Wed, Feb 6, 2019 at 9:55 PM Mark Bean <
> > mark.o.b...@gmail.com
> > > > > > > > wrote:
> > > > > > > >>>
> > > > > > >  I have an open PR I'd like included in 1.9. It's a very
> > simple
> > > > > one.
> > > > > > > 
> > > > > > >  https://github.com/apache/nifi/pull/3053 <
> > > > > > > https://github.com/apache/nifi/pull/3053>
> > > > > > > 
> > > > > > >  Thanks,
> > > > > > >  Mark
> > > > > > > 
> > > > > > >  On Wed, Feb 6, 2019 at 7:43 PM Joe Witt <
> joe.w...@gmail.com
> > >
> > > > > wrote:
> > > > > > > 
> > > > > > > > i have that one ready to go andy.  ill take care of the
> dev
> > > > docs
> > > > > > one.
> > > > > > > >
> > > > > > > > thanks
> > > > > > > >
> > > > > > > > On Wed, Feb 6, 2019, 7:24 PM Andy LoPresto <
> > > > alopre...@apache.org
> > > > > > > >>> wrote:
> > > > > > > >
> > > > > > 

Re: [EXT] [discuss] release apache nifi 1.9.0

[Joe Witt]

no worries - go for it.

On Wed, Feb 13, 2019 at 5:31 PM Matt Burgess  wrote:

> Joe,
>
> Mike Thomsen's GridFS PR is ready to go, can we get it in real quick?
>
> Thanks,
> Matt
>
> On Wed, Feb 13, 2019 at 5:06 PM Joe Witt  wrote:
> >
> > Alright !  1.9.0 looks ready for an RC build.  Will start working on this
> > tonight ideally.
> >
> > Thanks
> >
> > On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:
> >
> > > Joe,
> > >
> > > Yes, I'll review it today.
> > >
> > > On Mon, Feb 11, 2019, 8:44 AM Joe Witt  wrote:
> > >
> > > > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going to
> > > > review/merge that then?
> > > >
> > > > There are 5 remaining JIRAs tagged to 1.9.0 that aren't obviously
> > > puntable
> > > > and nearing completion of review/merge.  Will monitor/poke those with
> > > > intent to kick off RC once they're all done.
> > > >
> > > > Thanks
> > > >
> > > > On Fri, Feb 8, 2019 at 3:38 PM Jeff  wrote:
> > > >
> > > > > https://issues.apache.org/jira/browse/NIFI-5575 (
> > > > > https://github.com/apache/nifi/pull/3252) would be a good
> addition in
> > > > > 1.9.0, in my opinion.  It attempts to address PutHDFS ignoring the
> file
> > > > > mask default value from hdfs-site.xml when no default mask is
> specified
> > > > in
> > > > > the PutHDFS configuration.
> > > > >
> > > > > I can do the review on it, if we want to include it.
> > > > >
> > > > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto  >
> > > > wrote:
> > > > >
> > > > > > Ah, sorry, I misunderstood. You are not the PR author; you want
> those
> > > > > same
> > > > > > comments to be addressed.
> > > > > >
> > > > > > Andy LoPresto
> > > > > > alopre...@apache.org
> > > > > > alopresto.apa...@gmail.com
> > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> EF69
> > > > > >
> > > > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto <
> alopre...@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > > Charlie, I see comments from Matt Burgess (especially
> considering
> > > > > > exceptions & stack traces) from 9 days ago that don’t appear to
> be
> > > > > > addressed. Were you waiting for additional information? Am I
> missing
> > > > > > something? Thanks.
> > > > > > >
> > > > > > >
> > > > > > > Andy LoPresto
> > > > > > > alopre...@apache.org 
> > > > > > > alopresto.apa...@gmail.com
> > > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> EF69
> > > > > > >
> > > > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
> > > > > > charlie.me...@civitaslearning.com  > > > > > charlie.me...@civitaslearning.com>> wrote:
> > > > > > >>
> > > > > > >> Hoping I am not too late to the party, but it would be
> fantastic
> > > if
> > > > > > >> https://github.com/apache/nifi/pull/3253 <
> > > > > > https://github.com/apache/nifi/pull/3253> could have the
> comments on
> > > > it
> > > > > > >> addressed and included the 1.9 release
> > > > > > >>
> > > > > > >> -Charlie
> > > > > > >>
> > > > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt  > > >  > > > > > joe.w...@gmail.com>> wrote:
> > > > > > >>
> > > > > > >>> MarkB - i dont have any issue with that being include but we
> need
> > > > to
> > > > > > find
> > > > > > >>> someone familiar enough with that to review it.  If that
> happens
> > > in
> > > > > the
> > > > > > >>> next couple days then great.  Any takers familiar with JMS
> and
> > > that
> > > > > > class?
> > > > > > >>> The changes are few but the implications are unclear to me.
> > > > > > >>>
> > > > > > >>> Thanks
> > > > > > >>>
> > > > > > >>> On Wed, Feb 6, 2019 at 9:55 PM Mark Bean <
> mark.o.b...@gmail.com
> > > > > > > wrote:
> > > > > > >>>
> > > > > >  I have an open PR I'd like included in 1.9. It's a very
> simple
> > > > one.
> > > > > > 
> > > > > >  https://github.com/apache/nifi/pull/3053 <
> > > > > > https://github.com/apache/nifi/pull/3053>
> > > > > > 
> > > > > >  Thanks,
> > > > > >  Mark
> > > > > > 
> > > > > >  On Wed, Feb 6, 2019 at 7:43 PM Joe Witt  >
> > > > wrote:
> > > > > > 
> > > > > > > i have that one ready to go andy.  ill take care of the dev
> > > docs
> > > > > one.
> > > > > > >
> > > > > > > thanks
> > > > > > >
> > > > > > > On Wed, Feb 6, 2019, 7:24 PM Andy LoPresto <
> > > alopre...@apache.org
> > > > > > >>> wrote:
> > > > > > >
> > > > > > >> I also want to merge
> https://github.com/apache/nifi/pull/3283
> > > <
> > > > > > https://github.com/apache/nifi/pull/3283> <
> > > > > > >> https://github.com/apache/nifi/pull/3283 <
> > > > > > https://github.com/apache/nifi/pull/3283>> so we can get the
> > > > > > >>> Developer
> > > > > > >> Guide more visibility in the docs. Will do today.
> > > > > > >>
> > > > > > >> Andy LoPresto
> > > > > > >> alopre...@apache.org 
> > > > > > >> alopresto.apa...@gmail.com
> > > > > > >> PGP Fingerprint: 70EC B3E5 

Re: [EXT] [discuss] release apache nifi 1.9.0

[Matt Burgess]

Joe,

Mike Thomsen's GridFS PR is ready to go, can we get it in real quick?

Thanks,
Matt

On Wed, Feb 13, 2019 at 5:06 PM Joe Witt  wrote:
>
> Alright !  1.9.0 looks ready for an RC build.  Will start working on this
> tonight ideally.
>
> Thanks
>
> On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:
>
> > Joe,
> >
> > Yes, I'll review it today.
> >
> > On Mon, Feb 11, 2019, 8:44 AM Joe Witt  wrote:
> >
> > > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going to
> > > review/merge that then?
> > >
> > > There are 5 remaining JIRAs tagged to 1.9.0 that aren't obviously
> > puntable
> > > and nearing completion of review/merge.  Will monitor/poke those with
> > > intent to kick off RC once they're all done.
> > >
> > > Thanks
> > >
> > > On Fri, Feb 8, 2019 at 3:38 PM Jeff  wrote:
> > >
> > > > https://issues.apache.org/jira/browse/NIFI-5575 (
> > > > https://github.com/apache/nifi/pull/3252) would be a good addition in
> > > > 1.9.0, in my opinion.  It attempts to address PutHDFS ignoring the file
> > > > mask default value from hdfs-site.xml when no default mask is specified
> > > in
> > > > the PutHDFS configuration.
> > > >
> > > > I can do the review on it, if we want to include it.
> > > >
> > > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto 
> > > wrote:
> > > >
> > > > > Ah, sorry, I misunderstood. You are not the PR author; you want those
> > > > same
> > > > > comments to be addressed.
> > > > >
> > > > > Andy LoPresto
> > > > > alopre...@apache.org
> > > > > alopresto.apa...@gmail.com
> > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > > > >
> > > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto 
> > > > wrote:
> > > > > >
> > > > > > Charlie, I see comments from Matt Burgess (especially considering
> > > > > exceptions & stack traces) from 9 days ago that don’t appear to be
> > > > > addressed. Were you waiting for additional information? Am I missing
> > > > > something? Thanks.
> > > > > >
> > > > > >
> > > > > > Andy LoPresto
> > > > > > alopre...@apache.org 
> > > > > > alopresto.apa...@gmail.com
> > > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > > > > >
> > > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
> > > > > charlie.me...@civitaslearning.com  > > > > charlie.me...@civitaslearning.com>> wrote:
> > > > > >>
> > > > > >> Hoping I am not too late to the party, but it would be fantastic
> > if
> > > > > >> https://github.com/apache/nifi/pull/3253 <
> > > > > https://github.com/apache/nifi/pull/3253> could have the comments on
> > > it
> > > > > >> addressed and included the 1.9 release
> > > > > >>
> > > > > >> -Charlie
> > > > > >>
> > > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt  > >  > > > > joe.w...@gmail.com>> wrote:
> > > > > >>
> > > > > >>> MarkB - i dont have any issue with that being include but we need
> > > to
> > > > > find
> > > > > >>> someone familiar enough with that to review it.  If that happens
> > in
> > > > the
> > > > > >>> next couple days then great.  Any takers familiar with JMS and
> > that
> > > > > class?
> > > > > >>> The changes are few but the implications are unclear to me.
> > > > > >>>
> > > > > >>> Thanks
> > > > > >>>
> > > > > >>> On Wed, Feb 6, 2019 at 9:55 PM Mark Bean  > > > > > wrote:
> > > > > >>>
> > > > >  I have an open PR I'd like included in 1.9. It's a very simple
> > > one.
> > > > > 
> > > > >  https://github.com/apache/nifi/pull/3053 <
> > > > > https://github.com/apache/nifi/pull/3053>
> > > > > 
> > > > >  Thanks,
> > > > >  Mark
> > > > > 
> > > > >  On Wed, Feb 6, 2019 at 7:43 PM Joe Witt 
> > > wrote:
> > > > > 
> > > > > > i have that one ready to go andy.  ill take care of the dev
> > docs
> > > > one.
> > > > > >
> > > > > > thanks
> > > > > >
> > > > > > On Wed, Feb 6, 2019, 7:24 PM Andy LoPresto <
> > alopre...@apache.org
> > > > > >>> wrote:
> > > > > >
> > > > > >> I also want to merge https://github.com/apache/nifi/pull/3283
> > <
> > > > > https://github.com/apache/nifi/pull/3283> <
> > > > > >> https://github.com/apache/nifi/pull/3283 <
> > > > > https://github.com/apache/nifi/pull/3283>> so we can get the
> > > > > >>> Developer
> > > > > >> Guide more visibility in the docs. Will do today.
> > > > > >>
> > > > > >> Andy LoPresto
> > > > > >> alopre...@apache.org 
> > > > > >> alopresto.apa...@gmail.com
> > > > > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> > > > EF69
> > > > > >>
> > > > > >>> On Feb 7, 2019, at 10:12 AM, Ed B  wrote:
> > > > > >>>
> > > > > >>> Haha, just using this opportunity to pull attention and get
> > > > > >>> reviewer
> > > > >  :D
> > > > > >>>
> > > > > >>> But totally agreed, it's time for 1.9 to be finalized.
> > > > > >>>
> > > > > >>> Thanks,
> > > > > >>> Ed.
> > > > > 

Re: [EXT] [discuss] release apache nifi 1.9.0

[Joe Witt]

Alright !  1.9.0 looks ready for an RC build.  Will start working on this
tonight ideally.

Thanks

On Mon, Feb 11, 2019 at 10:28 AM Jeff  wrote:

> Joe,
>
> Yes, I'll review it today.
>
> On Mon, Feb 11, 2019, 8:44 AM Joe Witt  wrote:
>
> > Jeff - i added NIFI-5575 to 1.9.0 fix version. Are you going to
> > review/merge that then?
> >
> > There are 5 remaining JIRAs tagged to 1.9.0 that aren't obviously
> puntable
> > and nearing completion of review/merge.  Will monitor/poke those with
> > intent to kick off RC once they're all done.
> >
> > Thanks
> >
> > On Fri, Feb 8, 2019 at 3:38 PM Jeff  wrote:
> >
> > > https://issues.apache.org/jira/browse/NIFI-5575 (
> > > https://github.com/apache/nifi/pull/3252) would be a good addition in
> > > 1.9.0, in my opinion.  It attempts to address PutHDFS ignoring the file
> > > mask default value from hdfs-site.xml when no default mask is specified
> > in
> > > the PutHDFS configuration.
> > >
> > > I can do the review on it, if we want to include it.
> > >
> > > On Thu, Feb 7, 2019 at 1:17 AM Andy LoPresto 
> > wrote:
> > >
> > > > Ah, sorry, I misunderstood. You are not the PR author; you want those
> > > same
> > > > comments to be addressed.
> > > >
> > > > Andy LoPresto
> > > > alopre...@apache.org
> > > > alopresto.apa...@gmail.com
> > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > > >
> > > > > On Feb 7, 2019, at 5:16 PM, Andy LoPresto 
> > > wrote:
> > > > >
> > > > > Charlie, I see comments from Matt Burgess (especially considering
> > > > exceptions & stack traces) from 9 days ago that don’t appear to be
> > > > addressed. Were you waiting for additional information? Am I missing
> > > > something? Thanks.
> > > > >
> > > > >
> > > > > Andy LoPresto
> > > > > alopre...@apache.org 
> > > > > alopresto.apa...@gmail.com
> > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> > > > >
> > > > >> On Feb 7, 2019, at 3:24 PM, Charlie Meyer <
> > > > charlie.me...@civitaslearning.com  > > > charlie.me...@civitaslearning.com>> wrote:
> > > > >>
> > > > >> Hoping I am not too late to the party, but it would be fantastic
> if
> > > > >> https://github.com/apache/nifi/pull/3253 <
> > > > https://github.com/apache/nifi/pull/3253> could have the comments on
> > it
> > > > >> addressed and included the 1.9 release
> > > > >>
> > > > >> -Charlie
> > > > >>
> > > > >> On Wed, Feb 6, 2019 at 9:08 PM Joe Witt  >  > > > joe.w...@gmail.com>> wrote:
> > > > >>
> > > > >>> MarkB - i dont have any issue with that being include but we need
> > to
> > > > find
> > > > >>> someone familiar enough with that to review it.  If that happens
> in
> > > the
> > > > >>> next couple days then great.  Any takers familiar with JMS and
> that
> > > > class?
> > > > >>> The changes are few but the implications are unclear to me.
> > > > >>>
> > > > >>> Thanks
> > > > >>>
> > > > >>> On Wed, Feb 6, 2019 at 9:55 PM Mark Bean  > > > > wrote:
> > > > >>>
> > > >  I have an open PR I'd like included in 1.9. It's a very simple
> > one.
> > > > 
> > > >  https://github.com/apache/nifi/pull/3053 <
> > > > https://github.com/apache/nifi/pull/3053>
> > > > 
> > > >  Thanks,
> > > >  Mark
> > > > 
> > > >  On Wed, Feb 6, 2019 at 7:43 PM Joe Witt 
> > wrote:
> > > > 
> > > > > i have that one ready to go andy.  ill take care of the dev
> docs
> > > one.
> > > > >
> > > > > thanks
> > > > >
> > > > > On Wed, Feb 6, 2019, 7:24 PM Andy LoPresto <
> alopre...@apache.org
> > > > >>> wrote:
> > > > >
> > > > >> I also want to merge https://github.com/apache/nifi/pull/3283
> <
> > > > https://github.com/apache/nifi/pull/3283> <
> > > > >> https://github.com/apache/nifi/pull/3283 <
> > > > https://github.com/apache/nifi/pull/3283>> so we can get the
> > > > >>> Developer
> > > > >> Guide more visibility in the docs. Will do today.
> > > > >>
> > > > >> Andy LoPresto
> > > > >> alopre...@apache.org 
> > > > >> alopresto.apa...@gmail.com
> > > > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D
> > > EF69
> > > > >>
> > > > >>> On Feb 7, 2019, at 10:12 AM, Ed B  wrote:
> > > > >>>
> > > > >>> Haha, just using this opportunity to pull attention and get
> > > > >>> reviewer
> > > >  :D
> > > > >>>
> > > > >>> But totally agreed, it's time for 1.9 to be finalized.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> Ed.
> > > > >>>
> > > > >>>
> > > > >>> On Wed, Feb 6, 2019 at 5:08 PM Joe Witt  > > > >
> > > > >>> wrote:
> > > > >>>
> > > >  Thanks Andy sounds good.
> > > > 
> > > >  EdB: Sure we just need to find a reviewer.  There are tons
> of
> > > good
> > > >  prs
> > > > >> out
> > > >  there its just about getting good review cycles/progress.
> > > > 

Re: Controlling number of threads in distributed mode

[Joe Witt]

Milan

The only mechanism to limit whether a given component executes on a given
nifi node or not is whether it runs on all or whether it runs on primary
node.

Can you describe the use case that would lead to wanting to run a component
on a specified number of nodes as opposed to all nodes?  It would defy the
clustering logic/data distribution mechanisms of site to site and load
balanced connections for example.

Thanks

On Wed, Feb 13, 2019 at 4:34 PM Milan Das  wrote:

> Hello,
> Is there a way to control number of thread less than cluster member ?
> I have a 5 node NiFI cluster and I want to run 3 instances of the
> processor in distributed mode.
>
> Thanks,
> Milan Das
>
>

Controlling number of threads in distributed mode

[Milan Das]

Hello,
Is there a way to control number of thread less than cluster member ?
I have a 5 node NiFI cluster and I want to run 3 instances of the  processor in 
distributed mode.

Thanks,
Milan Das

Re: NiFi toolkit (tls-toolkit.sh server mode) and internally facing Web interface issue + nifi-cli to automate flows

[Andy LoPresto]

I am currently working on these issues and will share the PR when it is in a 
stable and acceptable state. Because of the way the toolkit is currently 
structured, 5460 needs 5462 to happen first. This is why the feature parity 
doesn’t exist; I added the feature to standalone mode but client/server mode 
needs additional changes to make it work. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Feb 13, 2019, at 7:55 AM, Erik Anderson  wrote:
> 
>> Andy LoPresto
>> alopre...@apache.org
>> alopresto.apa...@gmail.com
> 
> Thanks for the reply. I tried the steps you lined but but the only feasible 
> option is as lined out below, feature parity with the standalone mode.
> 
>> The quick answer is that there is currently a feature availability 
>> difference between the standalone mode and the client/server mode of 
>> the TLS Toolkit. The behavior you are requesting is currently in the 
>> standalone mode but not in the client/server mode. This is due to a 
>> difference in their implementation logic, which is being addressed as 
>> part of a larger refactoring under NIFI-5462 [1], which is a component 
>> of the TLS Toolkit epic NIFI-5458 [2]. 
> 
> Is this in progress? I am happy to pick this up and do it per guidance. Its 
> an important feature.
> 
>> Example keytool output for keystore with external CA public certificate 
>> in chain:  
> 
>> Owner: CN=intermediate_signed.nifi.apache.org, OU=NIFI
>> Issuer: CN=Intermediate CA
> 
>> Owner: CN=Intermediate CA
>> Issuer: CN=Root CA
> 
> In my experience, getting a Sub-CA in an enterprise organization is a 
> challenge and it exposes other risks. Its easier for me to code up a solution 
> for everyone. :)
> 
>> The NiFi TLS Toolkit was designed to facilitate the deployment and 
>> configuration of TLS behavior _in an environment lacking an enterprise 
>> security team_.
> 
> I wish an enterprise was that simple. Many times people need to take 
> ownership themselves. Thats why offer to code up feature parity.
> 
>> I am sorry the PKIX error message is unclear. The error is a 
>> Java-native exception and we can make an effort to capture it and 
>> provide a more user-friendly message in the future. However, searching 
>> "PKIX path building failed nifi” [7] does return a number of helpful 
>> articles and Q/A posts with instructions to resolve the issue. 
> 
> I usually catch the error, add a little more content to the error message, 
> and re-throw it. It does give the bes of both worlds.
> 
>> I encourage you to share further thoughts on concrete steps to improve 
>> this process that do not open users up to unseen security risks, and to 
>> follow the Jiras discussed above for further developments in improving 
>> the behavior and ease-of-use of the TLS Toolkit. 
> 
> I agree. If there is no one working on
>  - https://issues.apache.org/jira/browse/NIFI-5460
> 
> I will happily help and coordinate with whomever is working on 
>  - https://issues.apache.org/jira/browse/NIFI-5462
> 
> Erik Anderson
> Bloomberg
> https://www.linkedin.com/in/erikanderson/

Re: NiFi toolkit (tls-toolkit.sh server mode) and internally facing Web interface issue + nifi-cli to automate flows

[Erik Anderson]

> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com

Thanks for the reply. I tried the steps you lined but but the only feasible 
option is as lined out below, feature parity with the standalone mode.

> The quick answer is that there is currently a feature availability 
> difference between the standalone mode and the client/server mode of 
> the TLS Toolkit. The behavior you are requesting is currently in the 
> standalone mode but not in the client/server mode. This is due to a 
> difference in their implementation logic, which is being addressed as 
> part of a larger refactoring under NIFI-5462 [1], which is a component 
> of the TLS Toolkit epic NIFI-5458 [2]. 

Is this in progress? I am happy to pick this up and do it per guidance. Its an 
important feature.

> Example keytool output for keystore with external CA public certificate 
> in chain:  
  
> Owner: CN=intermediate_signed.nifi.apache.org, OU=NIFI
> Issuer: CN=Intermediate CA

> Owner: CN=Intermediate CA
> Issuer: CN=Root CA

In my experience, getting a Sub-CA in an enterprise organization is a challenge 
and it exposes other risks. Its easier for me to code up a solution for 
everyone. :)

> The NiFi TLS Toolkit was designed to facilitate the deployment and 
> configuration of TLS behavior _in an environment lacking an enterprise 
> security team_.

I wish an enterprise was that simple. Many times people need to take ownership 
themselves. Thats why offer to code up feature parity.

> I am sorry the PKIX error message is unclear. The error is a 
> Java-native exception and we can make an effort to capture it and 
> provide a more user-friendly message in the future. However, searching 
> "PKIX path building failed nifi” [7] does return a number of helpful 
> articles and Q/A posts with instructions to resolve the issue. 

I usually catch the error, add a little more content to the error message, and 
re-throw it. It does give the bes of both worlds.

> I encourage you to share further thoughts on concrete steps to improve 
> this process that do not open users up to unseen security risks, and to 
> follow the Jiras discussed above for further developments in improving 
> the behavior and ease-of-use of the TLS Toolkit. 

I agree. If there is no one working on
  - https://issues.apache.org/jira/browse/NIFI-5460

I will happily help and coordinate with whomever is working on 
  - https://issues.apache.org/jira/browse/NIFI-5462

Erik Anderson
Bloomberg
https://www.linkedin.com/in/erikanderson/

Re: NiFi, Docker Environment Variable for enabling debugging of NiFi inside a Docker container

[Erik Anderson]

> I was reading that email and was thinking of JVM debug options, with
> suspend y/n. I guess it just shows we meant very different things by debug
> mode. Maybe you could incorporate those into a PR too?
> 
> Andrew

Good point Andrew. and sorry for the slow response. I had to look at how NiFi 
set the JVM properties.

We had issues with Java and needed to manually set env Java flags.

It seems all of the Java debugging flags are set in the bootstrap.conf, located 
here.

https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/bootstrap.conf

For me, I chained together the Dockerfile
ENTRYPOINT ["my_company_entry_point.sh"]
then had it call the original entry point
In short, all of NiFi properties are set via environment variables in a 
/etc/profile.d/nifi.sh on the host and propogate throughout the NiFi system. 
Like I listed in
https://github.com/dprophet/nifi/blob/master/nifi-docker/dockerhub/CONFIGURATION.md

I normalized all NIFI specific environment variables to start with NIFI_

Why? Because you set the 

export NIFI_FOO="my foo"

and its directly passed into the container and into the NiFi role account used 
to start the container. You now have access to the environment variables 
throughout the NiFi system.

Example:
docker run --name nifi --env-file <(env | grep NIFI_) --hostname nifi

IMO, the Dockerfile entry point should allow a plug and play script so you can 
set these custom behaviors (both enterprise behaviors and custom 
developer/debugging). I doubt any enterprise will blindly pull a DockerHub 
container and run it. From my experience, a public container isnt enterprise 
friendly.

Andrew, define what you want for JVM debugging, what you would want to set (and 
unset), and I will take a look.

Erik Anderson
Bloomberg
https://www.linkedin.com/in/erikanderson/

Re: Proposing NiFi-Fn

[David Kegley]

Hey Peter,

That image was from a different project I was working on and I forgot to add
it to the nififn repo. 
It just has a couple additional nifi properties so that kubectl port-forward
could be used to expose the pods over the proxy. I'll update the repo to
include the nifi dockerfile as well and publish a new image 

David



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: Proposing NiFi-Fn

[Peter Wilcsinszky]

Hi David!

That is a nice one as well! I can see you have a custom NiFi image, but can
not track back to the Dockerfile, could you point to where we can find it?
(I understand it's marginal, just curious)

Cheers,
Peter

On Tue, Feb 12, 2019 at 8:04 PM David Kegley  wrote:

> Sam,
>
> This is really exciting stuff, thanks for contributing.
>
> I took a run at the proposed Kubernetes runtime using a custom NiFiFn
> operator, source is here: https://github.com/b23llc/nifi-fn-operator
>
> I also have some related feature requests, is JIRA the preferred place to
> track feature requests or should I wait until the initial PR is merged?
>
> Thanks,
> David
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>