Re: Closing in on a NiFi 1.2.0 release?

[Andy LoPresto]

As someone who has surely been guilty of optimistically setting fix versions 
and then not meeting them, I second Joe's point about it holding up releases. 
Better to get the PR out, reviewed, and merged *before* setting the fix version 
in my opinion. 

Andy LoPresto
alopre...@apache.org
alopresto.apa...@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Feb 22, 2017, at 19:39, Joe Witt  wrote:
> 
> Peter,
> 
> This is just my preference so discussion is certainly open.  But the
> way I see it we should not set the fix version on JIRAs unless they
> really should block a release without resolution or if due to some
> roadmap/planning/discussion it is a new feature/improvement that is
> tied to a release.  Otherwise, for the many things which pop up
> throughout a given release cycle they should be avoided.  That is to
> say the majority of the time we'd avoid fix versions until the act of
> merging a contribution which also means it has been reviewed.
> 
> From the release management point of view:
> This approach helps greatly as until now it is has been really
> difficult and time consuming to pull together/close down a release as
> pretty much anyone can set these fix versions and make it appear as
> though the release is not ready when in reality it is perfectly
> releasable as-is but might miss out on some contribs that someone
> would like to see in the release but has as of yet not gotten the PR
> and/or review traction necessary.
> 
> From the contributor point of view:
> If someone makes a contribution they obviously want that code to end
> up in a release.  But being an RTC community we need and want peer
> review before the code is submitted.  Some contributions are frankly
> hard to get peer review on or simply take time for someone to
> volunteer to do.  PRs which are difficult to test, lack testing, are
> related to systems or environments which are not easily replicated,
> etc.. are inherently harder to get peer review for.  Also, the
> community has grown quite rapidly and sometimes the hygiene of a given
> PR isn't great.  So our 'patch available' and 'open PR' count ticks
> up.  We need reviews/feedback as much as we need contributions so it
> is important for folks that want those contributions in to build
> meritocracy as well in reviewing others contributions.  This helps
> build a network of contributors/reviewers and improves the timeliness
> of reviews.  Long story short here is that because at times PRs can
> sit too long sometimes people put a fix version on the JIRA so it acts
> as a sort of 'gating function' on the release.  This I am saying is
> the practice that should not occur (given the thoughts above).  We
> should instead take the issue of how to more effectively
> triage/review/provide feedback/and manage expectations for
> contributions so contributors don't feel like their stuff will just
> sit forever.
> 
> Does that make sense and seem fair?
> 
> Thanks
> Joe
> 
> 
> 
>> On Wed, Feb 22, 2017 at 2:39 PM, Peter Wicks (pwicks)  
>> wrote:
>> Just for clarification, "We really need to avoid the practice of setting fix 
>> versions without traction", would mean don't set a version number until 
>> after we've submitted a PR? Until after the PR has been closed? Other?
>> 
>> Thanks,
>>  Peter
>> 
>> -Original Message-
>> From: Joe Witt [mailto:joe.w...@gmail.com]
>> Sent: Wednesday, February 22, 2017 12:55 PM
>> To: dev@nifi.apache.org
>> Subject: Closing in on a NiFi 1.2.0 release?
>> 
>> team,
>> 
>> On the users lists we had an ask of when we are planning to cut a
>> 1.2.0 release.  And someone else asked me recently off-list.
>> 
>> There are 45 open JIRAs tagged to it as of now.
>> 
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20NIFI%20AND%20fixVersion%20%3D%201.2.0%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20key%20DESC
>> 
>> I'd be favorable to going through and seeing if we can start the motions for 
>> a 1.2.0 release and which are ones we can wait for and which we should have 
>> in 1.2.0 for sure.
>> 
>> Is there any reason folks can think of to hold off on a 1.2.0 release?
>> 
>> A non trivial number of the JIRAs are for things which have or do not have 
>> PRs but have no review traction.  We really need to avoid the practice of 
>> setting fix versions without traction on this as otherwise it holds up the 
>> releases.
>> 
>> Thanks
>> Joe

Re: Closing in on a NiFi 1.2.0 release?

[Joe Gresock]

Slightly off topic, but your statement "we need reviews/feedback as much as
we need contributions" made me think that the project could somehow benefit
from the concept of Kanban Work In Progress limits.  Not sure how you'd
implement this with an open source project, but just a thought.

On Thu, Feb 23, 2017 at 8:37 AM, Andy LoPresto 
wrote:

> As someone who has surely been guilty of optimistically setting fix
> versions and then not meeting them, I second Joe's point about it holding
> up releases. Better to get the PR out, reviewed, and merged *before*
> setting the fix version in my opinion.
>
> Andy LoPresto
> alopre...@apache.org
> alopresto.apa...@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Feb 22, 2017, at 19:39, Joe Witt  wrote:
> >
> > Peter,
> >
> > This is just my preference so discussion is certainly open.  But the
> > way I see it we should not set the fix version on JIRAs unless they
> > really should block a release without resolution or if due to some
> > roadmap/planning/discussion it is a new feature/improvement that is
> > tied to a release.  Otherwise, for the many things which pop up
> > throughout a given release cycle they should be avoided.  That is to
> > say the majority of the time we'd avoid fix versions until the act of
> > merging a contribution which also means it has been reviewed.
> >
> > From the release management point of view:
> > This approach helps greatly as until now it is has been really
> > difficult and time consuming to pull together/close down a release as
> > pretty much anyone can set these fix versions and make it appear as
> > though the release is not ready when in reality it is perfectly
> > releasable as-is but might miss out on some contribs that someone
> > would like to see in the release but has as of yet not gotten the PR
> > and/or review traction necessary.
> >
> > From the contributor point of view:
> > If someone makes a contribution they obviously want that code to end
> > up in a release.  But being an RTC community we need and want peer
> > review before the code is submitted.  Some contributions are frankly
> > hard to get peer review on or simply take time for someone to
> > volunteer to do.  PRs which are difficult to test, lack testing, are
> > related to systems or environments which are not easily replicated,
> > etc.. are inherently harder to get peer review for.  Also, the
> > community has grown quite rapidly and sometimes the hygiene of a given
> > PR isn't great.  So our 'patch available' and 'open PR' count ticks
> > up.  We need reviews/feedback as much as we need contributions so it
> > is important for folks that want those contributions in to build
> > meritocracy as well in reviewing others contributions.  This helps
> > build a network of contributors/reviewers and improves the timeliness
> > of reviews.  Long story short here is that because at times PRs can
> > sit too long sometimes people put a fix version on the JIRA so it acts
> > as a sort of 'gating function' on the release.  This I am saying is
> > the practice that should not occur (given the thoughts above).  We
> > should instead take the issue of how to more effectively
> > triage/review/provide feedback/and manage expectations for
> > contributions so contributors don't feel like their stuff will just
> > sit forever.
> >
> > Does that make sense and seem fair?
> >
> > Thanks
> > Joe
> >
> >
> >
> >> On Wed, Feb 22, 2017 at 2:39 PM, Peter Wicks (pwicks) <
> pwi...@micron.com> wrote:
> >> Just for clarification, "We really need to avoid the practice of
> setting fix versions without traction", would mean don't set a version
> number until after we've submitted a PR? Until after the PR has been
> closed? Other?
> >>
> >> Thanks,
> >>  Peter
> >>
> >> -Original Message-
> >> From: Joe Witt [mailto:joe.w...@gmail.com]
> >> Sent: Wednesday, February 22, 2017 12:55 PM
> >> To: dev@nifi.apache.org
> >> Subject: Closing in on a NiFi 1.2.0 release?
> >>
> >> team,
> >>
> >> On the users lists we had an ask of when we are planning to cut a
> >> 1.2.0 release.  And someone else asked me recently off-list.
> >>
> >> There are 45 open JIRAs tagged to it as of now.
> >>
> >> https://issues.apache.org/jira/issues/?jql=project%20%
> 3D%20NIFI%20AND%20fixVersion%20%3D%201.2.0%20AND%20resolution%20%3D%
> 20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20key%20DESC
> >>
> >> I'd be favorable to going through and seeing if we can start the
> motions for a 1.2.0 release and which are ones we can wait for and which we
> should have in 1.2.0 for sure.
> >>
> >> Is there any reason folks can think of to hold off on a 1.2.0 release?
> >>
> >> A non trivial number of the JIRAs are for things which have or do not
> have PRs but have no review traction.  We really need to avoid the practice
> of setting fix versions without traction on this as otherwise it holds up
> the releases.
> >>
> >> Thanks
> >> Joe
>



-- 
I know what it is to 

Re: Minifi receiving flowflie through Remote process

[Jeremy Dyer]

Hey Rafael 

Can you please attach the error that you are seeing? It would also be helpful 
to see your minifi YAML configuration file if there is nothing sensitive in 
there. 

Are you using the java or c++ minifi version?

Thanks

Sent from my iPhone

> On Feb 22, 2017, at 11:20 PM, Rafael Carlos Rosa  
> wrote:
> 
> Hi,
> Atarted using Minifi.
> Tried with remote Input Ports and worked like a charm.
> Tried with remote Output Ports  and got an error.
> I'm doing something wrong or we didnt have this feature yet?
> 
> Thanks,
> Rafael Carlos
> 
> "Impossible is just a word people use to feel better about themselves when
> they give up..."
> --Somewhere on the internet--

Re: Minifi receiving flowflie through Remote process

[Bryan Rosander]

Hey Rafael Carlos,

Unfortunately, remote process group output ports aren't currently supported
in MiNiFi [1].  There is currently a Jira case to support this
functionality [2].

Thanks,
Bryan

[1]
https://nifi.apache.org/minifi/system-admin-guide.html#remote-process-groups
[2] https://issues.apache.org/jira/browse/MINIFI-176

On Thu, Feb 23, 2017 at 8:18 AM, Jeremy Dyer  wrote:

> Hey Rafael
>
> Can you please attach the error that you are seeing? It would also be
> helpful to see your minifi YAML configuration file if there is nothing
> sensitive in there.
>
> Are you using the java or c++ minifi version?
>
> Thanks
>
> Sent from my iPhone
>
> > On Feb 22, 2017, at 11:20 PM, Rafael Carlos Rosa <
> rafaelcarl...@gmail.com> wrote:
> >
> > Hi,
> > Atarted using Minifi.
> > Tried with remote Input Ports and worked like a charm.
> > Tried with remote Output Ports  and got an error.
> > I'm doing something wrong or we didnt have this feature yet?
> >
> > Thanks,
> > Rafael Carlos
> >
> > "Impossible is just a word people use to feel better about themselves
> when
> > they give up..."
> > --Somewhere on the internet--
>

Re: Minifi receiving flowflie through Remote process

[Jeremy Dyer]

Just to clarify it will work in the c++ version just not the java version

> On Feb 23, 2017, at 9:14 AM, Bryan Rosander  wrote:
> 
> Hey Rafael Carlos,
> 
> Unfortunately, remote process group output ports aren't currently supported
> in MiNiFi [1].  There is currently a Jira case to support this
> functionality [2].
> 
> Thanks,
> Bryan
> 
> [1]
> https://nifi.apache.org/minifi/system-admin-guide.html#remote-process-groups
> [2] https://issues.apache.org/jira/browse/MINIFI-176
> 
>> On Thu, Feb 23, 2017 at 8:18 AM, Jeremy Dyer  wrote:
>> 
>> Hey Rafael
>> 
>> Can you please attach the error that you are seeing? It would also be
>> helpful to see your minifi YAML configuration file if there is nothing
>> sensitive in there.
>> 
>> Are you using the java or c++ minifi version?
>> 
>> Thanks
>> 
>> Sent from my iPhone
>> 
>>> On Feb 22, 2017, at 11:20 PM, Rafael Carlos Rosa <
>> rafaelcarl...@gmail.com> wrote:
>>> 
>>> Hi,
>>> Atarted using Minifi.
>>> Tried with remote Input Ports and worked like a charm.
>>> Tried with remote Output Ports  and got an error.
>>> I'm doing something wrong or we didnt have this feature yet?
>>> 
>>> Thanks,
>>> Rafael Carlos
>>> 
>>> "Impossible is just a word people use to feel better about themselves
>> when
>>> they give up..."
>>> --Somewhere on the internet--
>> 

I don't want the ENTIRE remaining flow after the split execute that many times

[srini]

Hi,
Here, the number of times the process group and the LogAttribute executes is
depends on the number of elements in the split. If there are 5 elements due
to the split, then the Process Group and the LogAttribute executes 5 times.

But I don't want the ENTIRE remaining flow after the split execute 5 times.
I want to restrict to some where.
In this flow I want LogAttribute execute only once. How?


 

thanks
Srini



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/I-don-t-want-the-ENTIRE-remaining-flow-after-the-split-execute-that-many-times-tp14920.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

How to keep the original flowfile?

[srini]

Hello,
In the attached flow, the InvokeHTTP replaces the original flowfile. But the
SplitJSON needs the original flowfile instead of the flowfile given by the
InvokeHTTP. How to do that?


 

Thanks
Srini



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-keep-the-original-flowfile-tp14921.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: How to keep the original flowfile?

[Pierre Villard]

Hi Srini,

You have a relationship "original" that will route the original flow file
used to perform the request. Have a look here:
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi.processors.standard.InvokeHTTP/

Thanks

2017-02-23 18:20 GMT+01:00 srini :

> Hello,
> In the attached flow, the InvokeHTTP replaces the original flowfile. But
> the
> SplitJSON needs the original flowfile instead of the flowfile given by the
> InvokeHTTP. How to do that?
>
>  n14921/Screenshot_2017-02-23_12.png>
>
> Thanks
> Srini
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/How-to-keep-the-original-flowfile-tp14921.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

site-to-site configuration

[Mark Bean]

I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
secured NiFi, and am able to access the UI securely via HTTPS. I have set
the following security-related properties:

nifi.sensitive.props.key=
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.aditional.keys=

nifi.security.keystore=
nifi.security.keystoreType=JKS
nifi.security.keystorePasswd=
nifi.security.keyPasswd=
nifi.security.truststore=
nifi.security.truststoreType=JKS
nifi.security.trsustorePasswd=
nifi.security.needClientAuth=true
nifi.security.user.authorizer=file-provider
nifi.security.user.login.identity.provider=

I also set the site-to-site properties:
nifi.remote.input.host=
nifi.remote.input.secure=true
nifi.remote.input.socket.port=
nifi.remote.input.http.enabled=true
nifi.remote.input.http.tansaction.ttl=30 sec

The authorizers.xml has been setup to import the legacy
authorized-users.xml. And, this correctly populated the users.xml to
include the remote server for the site-to-site. It also added users to the
authorizations.xml file to include the user (i.e.server ) with site-to-site
resource (both R and W).

Despite this setup, the Input Port on the UI does not show an Access
Control tab as in NiFi 0.x. I am not sure how to authorize the remote
server such that the Input Port will be displayed in the remote server's
Remote Process Group's list of ports.

Have I missed a step in the security and/or user authentication setup?

Thanks,
Mark

Re: site-to-site configuration

[Bryan Bende]

Hi Mark,

There are two policies needed for secure site-to-site...

In the global policies there needs to be a policy for "retrieve
site-to-site details" with the user of the server added.

In the policies for the port (from the palette on the left when the
port is selected) there needs to be a policy for "receive data via
site-to-site" with user of the server added.

Thanks,

Bryan

On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean  wrote:
> I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
> secured NiFi, and am able to access the UI securely via HTTPS. I have set
> the following security-related properties:
>
> nifi.sensitive.props.key=
> nifi.sensitive.props.key.protected=
> nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> nifi.sensitive.props.provider=BC
> nifi.sensitive.props.aditional.keys=
>
> nifi.security.keystore=
> nifi.security.keystoreType=JKS
> nifi.security.keystorePasswd=
> nifi.security.keyPasswd=
> nifi.security.truststore=
> nifi.security.truststoreType=JKS
> nifi.security.trsustorePasswd=
> nifi.security.needClientAuth=true
> nifi.security.user.authorizer=file-provider
> nifi.security.user.login.identity.provider=
>
> I also set the site-to-site properties:
> nifi.remote.input.host=
> nifi.remote.input.secure=true
> nifi.remote.input.socket.port=
> nifi.remote.input.http.enabled=true
> nifi.remote.input.http.tansaction.ttl=30 sec
>
> The authorizers.xml has been setup to import the legacy
> authorized-users.xml. And, this correctly populated the users.xml to
> include the remote server for the site-to-site. It also added users to the
> authorizations.xml file to include the user (i.e.server ) with site-to-site
> resource (both R and W).
>
> Despite this setup, the Input Port on the UI does not show an Access
> Control tab as in NiFi 0.x. I am not sure how to authorize the remote
> server such that the Input Port will be displayed in the remote server's
> Remote Process Group's list of ports.
>
> Have I missed a step in the security and/or user authentication setup?
>
> Thanks,
> Mark

Re: How to keep the original flowfile?

[srini]

Hi Pierre,
But I want the response from InvokeHTTP also. How can I use the response and
keep the original flowfile so that SplitJson in the flow can use it.

thanks
Srini



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/How-to-keep-the-original-flowfile-tp14921p14925.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: site-to-site configuration

[Mark Bean]

Bryan,

The server is listed on the global policy for "retrieve site-to-site
details". However, I am not able to add users to the "receive data via
site-to-site" policy for the given Input Port (the add user button is
grayed out.) Under global access policies, "access all policies/modify", I
am listed as a user. Shouldn't this allow me to modify the policy (i.e. add
a user) on the Input Port?

Thanks again,
Mark


On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende  wrote:

> Hi Mark,
>
> There are two policies needed for secure site-to-site...
>
> In the global policies there needs to be a policy for "retrieve
> site-to-site details" with the user of the server added.
>
> In the policies for the port (from the palette on the left when the
> port is selected) there needs to be a policy for "receive data via
> site-to-site" with user of the server added.
>
> Thanks,
>
> Bryan
>
> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean  wrote:
> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
> > secured NiFi, and am able to access the UI securely via HTTPS. I have set
> > the following security-related properties:
> >
> > nifi.sensitive.props.key=
> > nifi.sensitive.props.key.protected=
> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> > nifi.sensitive.props.provider=BC
> > nifi.sensitive.props.aditional.keys=
> >
> > nifi.security.keystore=
> > nifi.security.keystoreType=JKS
> > nifi.security.keystorePasswd=
> > nifi.security.keyPasswd=
> > nifi.security.truststore=
> > nifi.security.truststoreType=JKS
> > nifi.security.trsustorePasswd=
> > nifi.security.needClientAuth=true
> > nifi.security.user.authorizer=file-provider
> > nifi.security.user.login.identity.provider=
> >
> > I also set the site-to-site properties:
> > nifi.remote.input.host=
> > nifi.remote.input.secure=true
> > nifi.remote.input.socket.port=
> > nifi.remote.input.http.enabled=true
> > nifi.remote.input.http.tansaction.ttl=30 sec
> >
> > The authorizers.xml has been setup to import the legacy
> > authorized-users.xml. And, this correctly populated the users.xml to
> > include the remote server for the site-to-site. It also added users to
> the
> > authorizations.xml file to include the user (i.e.server ) with
> site-to-site
> > resource (both R and W).
> >
> > Despite this setup, the Input Port on the UI does not show an Access
> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
> > server such that the Input Port will be displayed in the remote server's
> > Remote Process Group's list of ports.
> >
> > Have I missed a step in the security and/or user authentication setup?
> >
> > Thanks,
> > Mark
>

Re: How to keep the original flowfile?

[Aldrin Piri]

Hi Srini,

Depending on the typical size of the response, it is possible to store the
response in an attribute.  This would append the content onto the
previously existing FlowFile in an attribute of your choosing.

On Thu, Feb 23, 2017 at 12:55 PM, srini  wrote:

> Hi Pierre,
> But I want the response from InvokeHTTP also. How can I use the response
> and
> keep the original flowfile so that SplitJson in the flow can use it.
>
> thanks
> Srini
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/How-to-keep-the-original-flowfile-
> tp14921p14925.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: site-to-site configuration

[Bryan Bende]

Mark,

When you are looking at the "receive data via site-to-site" for the
input port, is there a link across the top to "Create Policy"?

I think you need to create a policy first then you can add users.

Thanks,

Bryan

On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean  wrote:
> Bryan,
>
> The server is listed on the global policy for "retrieve site-to-site
> details". However, I am not able to add users to the "receive data via
> site-to-site" policy for the given Input Port (the add user button is
> grayed out.) Under global access policies, "access all policies/modify", I
> am listed as a user. Shouldn't this allow me to modify the policy (i.e. add
> a user) on the Input Port?
>
> Thanks again,
> Mark
>
>
> On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende  wrote:
>
>> Hi Mark,
>>
>> There are two policies needed for secure site-to-site...
>>
>> In the global policies there needs to be a policy for "retrieve
>> site-to-site details" with the user of the server added.
>>
>> In the policies for the port (from the palette on the left when the
>> port is selected) there needs to be a policy for "receive data via
>> site-to-site" with user of the server added.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean  wrote:
>> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
>> > secured NiFi, and am able to access the UI securely via HTTPS. I have set
>> > the following security-related properties:
>> >
>> > nifi.sensitive.props.key=
>> > nifi.sensitive.props.key.protected=
>> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>> > nifi.sensitive.props.provider=BC
>> > nifi.sensitive.props.aditional.keys=
>> >
>> > nifi.security.keystore=
>> > nifi.security.keystoreType=JKS
>> > nifi.security.keystorePasswd=
>> > nifi.security.keyPasswd=
>> > nifi.security.truststore=
>> > nifi.security.truststoreType=JKS
>> > nifi.security.trsustorePasswd=
>> > nifi.security.needClientAuth=true
>> > nifi.security.user.authorizer=file-provider
>> > nifi.security.user.login.identity.provider=
>> >
>> > I also set the site-to-site properties:
>> > nifi.remote.input.host=
>> > nifi.remote.input.secure=true
>> > nifi.remote.input.socket.port=
>> > nifi.remote.input.http.enabled=true
>> > nifi.remote.input.http.tansaction.ttl=30 sec
>> >
>> > The authorizers.xml has been setup to import the legacy
>> > authorized-users.xml. And, this correctly populated the users.xml to
>> > include the remote server for the site-to-site. It also added users to
>> the
>> > authorizations.xml file to include the user (i.e.server ) with
>> site-to-site
>> > resource (both R and W).
>> >
>> > Despite this setup, the Input Port on the UI does not show an Access
>> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
>> > server such that the Input Port will be displayed in the remote server's
>> > Remote Process Group's list of ports.
>> >
>> > Have I missed a step in the security and/or user authentication setup?
>> >
>> > Thanks,
>> > Mark
>>

Re: site-to-site configuration

[Mark Bean]

Ok. Understood. I created the policy and added the user (server.) All is
working as expected now.

Is this process of manipulating policies required for secure site-to-site
documented anywhere? The User Guide still talked about Access Control and
the NiFi Role which seems to apply only to 0.x.

Thanks,
Mark


On Thu, Feb 23, 2017 at 1:11 PM, Bryan Bende  wrote:

> Mark,
>
> When you are looking at the "receive data via site-to-site" for the
> input port, is there a link across the top to "Create Policy"?
>
> I think you need to create a policy first then you can add users.
>
> Thanks,
>
> Bryan
>
> On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean  wrote:
> > Bryan,
> >
> > The server is listed on the global policy for "retrieve site-to-site
> > details". However, I am not able to add users to the "receive data via
> > site-to-site" policy for the given Input Port (the add user button is
> > grayed out.) Under global access policies, "access all policies/modify",
> I
> > am listed as a user. Shouldn't this allow me to modify the policy (i.e.
> add
> > a user) on the Input Port?
> >
> > Thanks again,
> > Mark
> >
> >
> > On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende  wrote:
> >
> >> Hi Mark,
> >>
> >> There are two policies needed for secure site-to-site...
> >>
> >> In the global policies there needs to be a policy for "retrieve
> >> site-to-site details" with the user of the server added.
> >>
> >> In the policies for the port (from the palette on the left when the
> >> port is selected) there needs to be a policy for "receive data via
> >> site-to-site" with user of the server added.
> >>
> >> Thanks,
> >>
> >> Bryan
> >>
> >> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean 
> wrote:
> >> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
> >> > secured NiFi, and am able to access the UI securely via HTTPS. I have
> set
> >> > the following security-related properties:
> >> >
> >> > nifi.sensitive.props.key=
> >> > nifi.sensitive.props.key.protected=
> >> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> >> > nifi.sensitive.props.provider=BC
> >> > nifi.sensitive.props.aditional.keys=
> >> >
> >> > nifi.security.keystore=
> >> > nifi.security.keystoreType=JKS
> >> > nifi.security.keystorePasswd=
> >> > nifi.security.keyPasswd=
> >> > nifi.security.truststore=
> >> > nifi.security.truststoreType=JKS
> >> > nifi.security.trsustorePasswd=
> >> > nifi.security.needClientAuth=true
> >> > nifi.security.user.authorizer=file-provider
> >> > nifi.security.user.login.identity.provider=
> >> >
> >> > I also set the site-to-site properties:
> >> > nifi.remote.input.host=
> >> > nifi.remote.input.secure=true
> >> > nifi.remote.input.socket.port=
> >> > nifi.remote.input.http.enabled=true
> >> > nifi.remote.input.http.tansaction.ttl=30 sec
> >> >
> >> > The authorizers.xml has been setup to import the legacy
> >> > authorized-users.xml. And, this correctly populated the users.xml to
> >> > include the remote server for the site-to-site. It also added users to
> >> the
> >> > authorizations.xml file to include the user (i.e.server ) with
> >> site-to-site
> >> > resource (both R and W).
> >> >
> >> > Despite this setup, the Input Port on the UI does not show an Access
> >> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
> >> > server such that the Input Port will be displayed in the remote
> server's
> >> > Remote Process Group's list of ports.
> >> >
> >> > Have I missed a step in the security and/or user authentication setup?
> >> >
> >> > Thanks,
> >> > Mark
> >>
>

Re: I don't want the ENTIRE remaining flow after the split execute that many times

[Aldrin Piri]

Hi Srini,

You could make use of a RouteOnAttribute to only pass those fragments of
the split that have a given 'fragment.index' of 0 (at least I believe these
are zero indexed) where all the others are dropped.  In this case, you
would get one and only one of each "batch."

On Thu, Feb 23, 2017 at 12:07 PM, srini  wrote:

> Hi,
> Here, the number of times the process group and the LogAttribute executes
> is
> depends on the number of elements in the split. If there are 5 elements due
> to the split, then the Process Group and the LogAttribute executes 5 times.
>
> But I don't want the ENTIRE remaining flow after the split execute 5 times.
> I want to restrict to some where.
> In this flow I want LogAttribute execute only once. How?
>
>  n14920/Screenshot_2017-02-23_11.png>
>
> thanks
> Srini
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/I-don-t-want-the-ENTIRE-remaining-flow-after-the-
> split-execute-that-many-times-tp14920.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: site-to-site configuration

[Bryan Bende]

Mark,

I think you are correct that the paragraph in the user guide should be
updated for 1.x.

I know the admin guide has a section about users and policies in
general, but not necessarily specific to site-to-site:

https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies

I also have a blog post here, but I realize it is not official documentation:

http://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site

Thanks,

Bryan

On Thu, Feb 23, 2017 at 1:33 PM, Mark Bean  wrote:
> Ok. Understood. I created the policy and added the user (server.) All is
> working as expected now.
>
> Is this process of manipulating policies required for secure site-to-site
> documented anywhere? The User Guide still talked about Access Control and
> the NiFi Role which seems to apply only to 0.x.
>
> Thanks,
> Mark
>
>
> On Thu, Feb 23, 2017 at 1:11 PM, Bryan Bende  wrote:
>
>> Mark,
>>
>> When you are looking at the "receive data via site-to-site" for the
>> input port, is there a link across the top to "Create Policy"?
>>
>> I think you need to create a policy first then you can add users.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean  wrote:
>> > Bryan,
>> >
>> > The server is listed on the global policy for "retrieve site-to-site
>> > details". However, I am not able to add users to the "receive data via
>> > site-to-site" policy for the given Input Port (the add user button is
>> > grayed out.) Under global access policies, "access all policies/modify",
>> I
>> > am listed as a user. Shouldn't this allow me to modify the policy (i.e.
>> add
>> > a user) on the Input Port?
>> >
>> > Thanks again,
>> > Mark
>> >
>> >
>> > On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende  wrote:
>> >
>> >> Hi Mark,
>> >>
>> >> There are two policies needed for secure site-to-site...
>> >>
>> >> In the global policies there needs to be a policy for "retrieve
>> >> site-to-site details" with the user of the server added.
>> >>
>> >> In the policies for the port (from the palette on the left when the
>> >> port is selected) there needs to be a policy for "receive data via
>> >> site-to-site" with user of the server added.
>> >>
>> >> Thanks,
>> >>
>> >> Bryan
>> >>
>> >> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean 
>> wrote:
>> >> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
>> >> > secured NiFi, and am able to access the UI securely via HTTPS. I have
>> set
>> >> > the following security-related properties:
>> >> >
>> >> > nifi.sensitive.props.key=
>> >> > nifi.sensitive.props.key.protected=
>> >> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>> >> > nifi.sensitive.props.provider=BC
>> >> > nifi.sensitive.props.aditional.keys=
>> >> >
>> >> > nifi.security.keystore=
>> >> > nifi.security.keystoreType=JKS
>> >> > nifi.security.keystorePasswd=
>> >> > nifi.security.keyPasswd=
>> >> > nifi.security.truststore=
>> >> > nifi.security.truststoreType=JKS
>> >> > nifi.security.trsustorePasswd=
>> >> > nifi.security.needClientAuth=true
>> >> > nifi.security.user.authorizer=file-provider
>> >> > nifi.security.user.login.identity.provider=
>> >> >
>> >> > I also set the site-to-site properties:
>> >> > nifi.remote.input.host=
>> >> > nifi.remote.input.secure=true
>> >> > nifi.remote.input.socket.port=
>> >> > nifi.remote.input.http.enabled=true
>> >> > nifi.remote.input.http.tansaction.ttl=30 sec
>> >> >
>> >> > The authorizers.xml has been setup to import the legacy
>> >> > authorized-users.xml. And, this correctly populated the users.xml to
>> >> > include the remote server for the site-to-site. It also added users to
>> >> the
>> >> > authorizations.xml file to include the user (i.e.server ) with
>> >> site-to-site
>> >> > resource (both R and W).
>> >> >
>> >> > Despite this setup, the Input Port on the UI does not show an Access
>> >> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
>> >> > server such that the Input Port will be displayed in the remote
>> server's
>> >> > Remote Process Group's list of ports.
>> >> >
>> >> > Have I missed a step in the security and/or user authentication setup?
>> >> >
>> >> > Thanks,
>> >> > Mark
>> >>
>>

Re: site-to-site configuration

[Andrew Lim]

I created a Jira to make sure we update that paragraph in the 1.x User Guide:

https://issues.apache.org/jira/browse/NIFI-3526

-Drew

> On Feb 23, 2017, at 1:48 PM, Bryan Bende  wrote:
> 
> Mark,
> 
> I think you are correct that the paragraph in the user guide should be
> updated for 1.x.
> 
> I know the admin guide has a section about users and policies in
> general, but not necessarily specific to site-to-site:
> 
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies
> 
> I also have a blog post here, but I realize it is not official documentation:
> 
> http://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site
> 
> Thanks,
> 
> Bryan
> 
> On Thu, Feb 23, 2017 at 1:33 PM, Mark Bean  wrote:
>> Ok. Understood. I created the policy and added the user (server.) All is
>> working as expected now.
>> 
>> Is this process of manipulating policies required for secure site-to-site
>> documented anywhere? The User Guide still talked about Access Control and
>> the NiFi Role which seems to apply only to 0.x.
>> 
>> Thanks,
>> Mark
>> 
>> 
>> On Thu, Feb 23, 2017 at 1:11 PM, Bryan Bende  wrote:
>> 
>>> Mark,
>>> 
>>> When you are looking at the "receive data via site-to-site" for the
>>> input port, is there a link across the top to "Create Policy"?
>>> 
>>> I think you need to create a policy first then you can add users.
>>> 
>>> Thanks,
>>> 
>>> Bryan
>>> 
>>> On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean  wrote:
 Bryan,
 
 The server is listed on the global policy for "retrieve site-to-site
 details". However, I am not able to add users to the "receive data via
 site-to-site" policy for the given Input Port (the add user button is
 grayed out.) Under global access policies, "access all policies/modify",
>>> I
 am listed as a user. Shouldn't this allow me to modify the policy (i.e.
>>> add
 a user) on the Input Port?
 
 Thanks again,
 Mark
 
 
 On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende  wrote:
 
> Hi Mark,
> 
> There are two policies needed for secure site-to-site...
> 
> In the global policies there needs to be a policy for "retrieve
> site-to-site details" with the user of the server added.
> 
> In the policies for the port (from the palette on the left when the
> port is selected) there needs to be a policy for "receive data via
> site-to-site" with user of the server added.
> 
> Thanks,
> 
> Bryan
> 
> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean 
>>> wrote:
>> I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
>> secured NiFi, and am able to access the UI securely via HTTPS. I have
>>> set
>> the following security-related properties:
>> 
>> nifi.sensitive.props.key=
>> nifi.sensitive.props.key.protected=
>> nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>> nifi.sensitive.props.provider=BC
>> nifi.sensitive.props.aditional.keys=
>> 
>> nifi.security.keystore=
>> nifi.security.keystoreType=JKS
>> nifi.security.keystorePasswd=
>> nifi.security.keyPasswd=
>> nifi.security.truststore=
>> nifi.security.truststoreType=JKS
>> nifi.security.trsustorePasswd=
>> nifi.security.needClientAuth=true
>> nifi.security.user.authorizer=file-provider
>> nifi.security.user.login.identity.provider=
>> 
>> I also set the site-to-site properties:
>> nifi.remote.input.host=
>> nifi.remote.input.secure=true
>> nifi.remote.input.socket.port=
>> nifi.remote.input.http.enabled=true
>> nifi.remote.input.http.tansaction.ttl=30 sec
>> 
>> The authorizers.xml has been setup to import the legacy
>> authorized-users.xml. And, this correctly populated the users.xml to
>> include the remote server for the site-to-site. It also added users to
> the
>> authorizations.xml file to include the user (i.e.server ) with
> site-to-site
>> resource (both R and W).
>> 
>> Despite this setup, the Input Port on the UI does not show an Access
>> Control tab as in NiFi 0.x. I am not sure how to authorize the remote
>> server such that the Input Port will be displayed in the remote
>>> server's
>> Remote Process Group's list of ports.
>> 
>> Have I missed a step in the security and/or user authentication setup?
>> 
>> Thanks,
>> Mark
> 
>>> 

Configuration Suggestion for NiFi HandleHTTPResponse Processor

[nairy991]

Greetings,

I cam across an issue with the HandleHTTPResponse Processor in which the
processor sends the response but it also sends the body of the message along
with the response code. I wanted to ask if it was possible for future
development to add a configuration in the processor that allows the option
of sending the response without the body of the message, since this can
affect the time that it takes for the client to receive the response from
us, because NiFi is sending the full message again, along with the response.
As of now I have been unable to find an optimal solution that does not
impact our flow performance in regards with removing the body of the message
and only sending the response. I wanted to see if this could be a
configuration that could be looked into for this processor, since I think it
would be a useful one that could provide positive impact on the response
time of the NiFi’s HandleHTTPResponse processor.



--
View this message in context: 
http://apache-nifi-developer-list.39713.n7.nabble.com/Configuration-Suggestion-for-NiFi-HandleHTTPResponse-Processor-tp14933.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Re: Configuration Suggestion for NiFi HandleHTTPResponse Processor

[Aldrin Piri]

Hello!

You can get the results you are looking for by using ReplaceText to modify
the body.  In your case, you could completely replace the text with an
empty string.  There is an example of this in the Example Dataflow
Templates [1] with the Hello NiFi Web Service template.  The performance
impact in this scenario should be minimal.

Please let us know if this fulfills what you are attempting to do.  If not,
if you could provide some additional details on where this comes up short,
we can certainly discuss a better approach.

[1]
https://cwiki.apache.org/confluence/display/NIFI/Example+Dataflow+Templates

On Thu, Feb 23, 2017 at 6:15 PM, nairy991  wrote:

> Greetings,
>
> I cam across an issue with the HandleHTTPResponse Processor in which the
> processor sends the response but it also sends the body of the message
> along
> with the response code. I wanted to ask if it was possible for future
> development to add a configuration in the processor that allows the option
> of sending the response without the body of the message, since this can
> affect the time that it takes for the client to receive the response from
> us, because NiFi is sending the full message again, along with the
> response.
> As of now I have been unable to find an optimal solution that does not
> impact our flow performance in regards with removing the body of the
> message
> and only sending the response. I wanted to see if this could be a
> configuration that could be looked into for this processor, since I think
> it
> would be a useful one that could provide positive impact on the response
> time of the NiFi’s HandleHTTPResponse processor.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/Configuration-Suggestion-for-
> NiFi-HandleHTTPResponse-Processor-tp14933.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>