get access token inside custom processor

[尹文才]

Hi guys, I'm trying to invoke some nifi rest apis inside my custom
processor, the nifi I'm using is nifi 1.4.0 and it's a 3 node secured
cluster., the username and password are kept inside a ldap server.
I know that in a secured nifi cluster, in order to make any request I need
the access token, my question is how could I get the access token in my
custom processor? Thanks. (I think the token should be
available somewhere after successful login right?)

regards,
ben

Re: Policies for root Process Group.

[Daniel Hernandez]

Thanks Matt,

I get now what is the problem, in order to exhaust all my possibilities I
may ask, is there a way using the API to get the root UUID from the
flow.xml.gz file? Because I see the file there after running the tests.

Thanks,


On Mon, Feb 26, 2018 at 3:26 PM, Daniel Hernandez <
daniel.hernan...@civitaslearning.com> wrote:

> Hi Matt,
>
> Thanks for your answer.
>
> Do you know if there is a way to preconfigure this value when running
> Nifi's Docker image? I am making the calls from an integration test that
> runs a docker container with the Nifi server. I already check and the value
> under  in the flow.xml.gz file changes everytime I deploy
> the container, I guess it is created at startup.  Is it possible that I can
> change my docker image to get a fix root group value?
>
> Thanks,
>
> Daniel
>
> On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez  civitaslearning.com> wrote:
>
>> Hi,
>>
>> I am currently working on calling the Nifi REST API to get the 'root'
>> process group and setting it as parent for a new process-group.
>>
>> However I am getting the next messages:
>>
>> Attempting GET request to: JerseyWebTarget {
>> https://127.0.0.1:8443/nifi-api/process-groups/root }
>> 2018-02-26 11:06:55.341 DEBUG  --- [   main]
>> c.c.p.n.c.i.b.BootApiClient  :
>> 2018-02-26 11:06:55.341 DEBUG  --- [   main]
>> c.c.p.n.c.i.b.BootApiClient  : Received 403 response from GET
>> to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root
>> }
>>
>> com.civitaslearning.platform.nifi.client.invoker.boot.exception.NifiForbiddenException:
>> No applicable policies could be found. Contact the system administrator.
>>
>> This is the content of my authorizations.xml file:
>>
>> 
>>
>> 
>>
>> 
>>
>> > resource="/flow" action="R">
>>
>> 
>>
>> 
>>
>> > resource="/restricted-components" action="W">
>>
>> 
>>
>> 
>>
>> > resource="/tenants" action="R">
>>
>> 
>>
>> 
>>
>> > resource="/tenants" action="W">
>>
>> 
>>
>> 
>>
>> > resource="/policies" action="R">
>>
>> 
>>
>> 
>>
>> > resource="/policies" action="W">
>>
>> 
>>
>> 
>>
>> > resource="/controller" action="R">
>>
>> 
>>
>> 
>>
>> > resource="/controller" action="W">
>>
>> 
>>
>> 
>>
>> > resource="/process-groups/root" action="R">
>>
>> 
>>
>> 
>>
>> > resource="/process-groups/root" action="W">
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> And this is the content of authorizations.xml
>>
>> 
>>
>> 
>>
>> file-access-policy-provider
>>
>> org.apache.nifi.authorization.FileAccessPolicyProvide
>> r
>>
>> file-user-group-prov
>> ider
>>
>> ./conf/authorizations.xm
>> l
>>
>> CN=civitas,
>> OU=ApacheNifi
>>
>> 
>>
>>
>> 
>>
>> 
>>
>> 
>>
>> managed-authorizer
>>
>> org.apache.nifi.authorization.StandardManagedAuthoriz
>> er
>>
>> file-access-policy-p
>> rovider
>>
>> 
>>
>> 
>>
>>
>> And users.xml
>>
>>
>> 
>>
>> 
>>
>> 
>>
>> 
>>
>> > identity="CN=civitas, OU=ApacheNifi"/>
>>
>> 
>>
>> 
>>
>> I already create a policy using the same user cert so I guess the DN is
>> valid.
>> Am I defining the policy or making the call in a wrong way?
>>
>> Thanks in advance,
>>
>> Daniel Hernandez
>>
>>
>>
>

Re: Policies for root Process Group.

[Bryan Bende]

You should be able to include a canned flow.xml.gz in your in your
container, just have nothing under the root group.


On Mon, Feb 26, 2018 at 3:50 PM, Matt Gilman  wrote:
> Daniel,
>
> Unfortunately, there is no way to set this currently. This is ultimately a
> lifecycle issue. The UUID of the root group may be inherited from a cluster
> or randomly generated if a node is standalone. From the admin guide:
>
> "For a brand new secure flow, providing the "Initial Admin Identity" gives
> that user access to get into the UI and to manage users, groups and
> policies. But if that user wants to start modifying the flow, they need to
> grant themselves policies for the root process group. The system is unable
> to do this automatically because in a new flow the UUID of the root process
> group is not permanent until the flow.xml.gz is generated. If the NiFi
> instance is an upgrade from an existing flow.xml.gz or a 1.x instance going
> from unsecure to secure, then the "Initial Admin Identity" user is
> automatically given the privileges to modify the flow."
>
> Because of this, when there is no existing flow, granting permissions to
> the root group would need to happen after this initial startup.
>
> Matt
>
>
> On Mon, Feb 26, 2018 at 3:26 PM, Daniel Hernandez <
> daniel.hernan...@civitaslearning.com> wrote:
>
>> Hi Matt,
>>
>> Thanks for your answer.
>>
>> Do you know if there is a way to preconfigure this value when running
>> Nifi's Docker image? I am making the calls from an integration test that
>> runs a docker container with the Nifi server. I already check and the value
>> under  in the flow.xml.gz file changes everytime I deploy
>> the container, I guess it is created at startup.  Is it possible that I can
>> change my docker image to get a fix root group value?
>>
>> Thanks,
>>
>> Daniel
>>
>> On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez <
>> daniel.hernan...@civitaslearning.com> wrote:
>>
>> > Hi,
>> >
>> > I am currently working on calling the Nifi REST API to get the 'root'
>> > process group and setting it as parent for a new process-group.
>> >
>> > However I am getting the next messages:
>> >
>> > Attempting GET request to: JerseyWebTarget {
>> https://127.0.0.1:8443/nifi-
>> > api/process-groups/root }
>> > 2018-02-26 11:06:55.341 DEBUG  --- [   main]
>> > c.c.p.n.c.i.b.BootApiClient  :
>> > 2018-02-26 11:06:55.341 DEBUG  --- [   main]
>> > c.c.p.n.c.i.b.BootApiClient  : Received 403 response from GET
>> > to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root
>> }
>> >
>> > com.civitaslearning.platform.nifi.client.invoker.boot.exception.
>> NifiForbiddenException:
>> > No applicable policies could be found. Contact the system administrator.
>> >
>> > This is the content of my authorizations.xml file:
>> >
>> > 
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/flow" action="R">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/restricted-components" action="W">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/tenants" action="R">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/tenants" action="W">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/policies" action="R">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/policies" action="W">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/controller" action="R">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/controller" action="W">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/process-groups/root" action="R">
>> >
>> > 
>> >
>> > 
>> >
>> > > > resource="/process-groups/root" action="W">
>> >
>> > 
>> >
>> > 
>> >
>> > 
>> >
>> > 
>> >
>> > And this is the content of authorizations.xml
>> >
>> > 
>> >
>> > 
>> >
>> > file-access-policy-provider
>> >
>> > org.apache.nifi.authorization.FileAccessPolicyProvider> > class>
>> >
>> > file-user-group-
>> > provider
>> >
>> > ./conf/authorizations.
>> > xml
>> >
>> > CN=civitas,
>> > OU=ApacheNifi
>> >
>> > 
>> >
>> >
>> > 
>> >
>> > 
>> >
>> > 
>> >
>> > managed-authorizer
>> >
>> > org.apache.nifi.authorization.StandardManagedAuthorizer> > class>
>> >
>> > file-access-policy-
>> > provider
>> >
>> > 
>> >
>> > 
>> >
>> >
>> > And users.xml
>> >
>> >
>> > 
>> >
>> > 
>> >
>> > 
>> >
>> > 
>> >
>> > > > identity="CN=civitas, OU=ApacheNifi"/>
>> >
>> > 
>> >
>> > 
>> >
>> > I already create a policy using the same user cert so I guess the DN is
>> > valid.
>> > Am I defining the policy or making the call in a wrong way?
>> >
>> > Thanks in advance,
>> >
>> > Daniel Hernandez
>> >
>> >
>> >
>>

Re: Policies for root Process Group.

[Matt Gilman]

Daniel,

Unfortunately, there is no way to set this currently. This is ultimately a
lifecycle issue. The UUID of the root group may be inherited from a cluster
or randomly generated if a node is standalone. From the admin guide:

"For a brand new secure flow, providing the "Initial Admin Identity" gives
that user access to get into the UI and to manage users, groups and
policies. But if that user wants to start modifying the flow, they need to
grant themselves policies for the root process group. The system is unable
to do this automatically because in a new flow the UUID of the root process
group is not permanent until the flow.xml.gz is generated. If the NiFi
instance is an upgrade from an existing flow.xml.gz or a 1.x instance going
from unsecure to secure, then the "Initial Admin Identity" user is
automatically given the privileges to modify the flow."

Because of this, when there is no existing flow, granting permissions to
the root group would need to happen after this initial startup.

Matt


On Mon, Feb 26, 2018 at 3:26 PM, Daniel Hernandez <
daniel.hernan...@civitaslearning.com> wrote:

> Hi Matt,
>
> Thanks for your answer.
>
> Do you know if there is a way to preconfigure this value when running
> Nifi's Docker image? I am making the calls from an integration test that
> runs a docker container with the Nifi server. I already check and the value
> under  in the flow.xml.gz file changes everytime I deploy
> the container, I guess it is created at startup.  Is it possible that I can
> change my docker image to get a fix root group value?
>
> Thanks,
>
> Daniel
>
> On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez <
> daniel.hernan...@civitaslearning.com> wrote:
>
> > Hi,
> >
> > I am currently working on calling the Nifi REST API to get the 'root'
> > process group and setting it as parent for a new process-group.
> >
> > However I am getting the next messages:
> >
> > Attempting GET request to: JerseyWebTarget {
> https://127.0.0.1:8443/nifi-
> > api/process-groups/root }
> > 2018-02-26 11:06:55.341 DEBUG  --- [   main]
> > c.c.p.n.c.i.b.BootApiClient  :
> > 2018-02-26 11:06:55.341 DEBUG  --- [   main]
> > c.c.p.n.c.i.b.BootApiClient  : Received 403 response from GET
> > to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root
> }
> >
> > com.civitaslearning.platform.nifi.client.invoker.boot.exception.
> NifiForbiddenException:
> > No applicable policies could be found. Contact the system administrator.
> >
> > This is the content of my authorizations.xml file:
> >
> > 
> >
> > 
> >
> > 
> >
> >  > resource="/flow" action="R">
> >
> > 
> >
> > 
> >
> >  > resource="/restricted-components" action="W">
> >
> > 
> >
> > 
> >
> >  > resource="/tenants" action="R">
> >
> > 
> >
> > 
> >
> >  > resource="/tenants" action="W">
> >
> > 
> >
> > 
> >
> >  > resource="/policies" action="R">
> >
> > 
> >
> > 
> >
> >  > resource="/policies" action="W">
> >
> > 
> >
> > 
> >
> >  > resource="/controller" action="R">
> >
> > 
> >
> > 
> >
> >  > resource="/controller" action="W">
> >
> > 
> >
> > 
> >
> >  > resource="/process-groups/root" action="R">
> >
> > 
> >
> > 
> >
> >  > resource="/process-groups/root" action="W">
> >
> > 
> >
> > 
> >
> > 
> >
> > 
> >
> > And this is the content of authorizations.xml
> >
> > 
> >
> > 
> >
> > file-access-policy-provider
> >
> > org.apache.nifi.authorization.FileAccessPolicyProvider > class>
> >
> > file-user-group-
> > provider
> >
> > ./conf/authorizations.
> > xml
> >
> > CN=civitas,
> > OU=ApacheNifi
> >
> > 
> >
> >
> > 
> >
> > 
> >
> > 
> >
> > managed-authorizer
> >
> > org.apache.nifi.authorization.StandardManagedAuthorizer > class>
> >
> > file-access-policy-
> > provider
> >
> > 
> >
> > 
> >
> >
> > And users.xml
> >
> >
> > 
> >
> > 
> >
> > 
> >
> > 
> >
> >  > identity="CN=civitas, OU=ApacheNifi"/>
> >
> > 
> >
> > 
> >
> > I already create a policy using the same user cert so I guess the DN is
> > valid.
> > Am I defining the policy or making the call in a wrong way?
> >
> > Thanks in advance,
> >
> > Daniel Hernandez
> >
> >
> >
>

Re: Policies for root Process Group.

[Daniel Hernandez]

Hi Matt,

Thanks for your answer.

Do you know if there is a way to preconfigure this value when running
Nifi's Docker image? I am making the calls from an integration test that
runs a docker container with the Nifi server. I already check and the value
under  in the flow.xml.gz file changes everytime I deploy
the container, I guess it is created at startup.  Is it possible that I can
change my docker image to get a fix root group value?

Thanks,

Daniel

On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez <
daniel.hernan...@civitaslearning.com> wrote:

> Hi,
>
> I am currently working on calling the Nifi REST API to get the 'root'
> process group and setting it as parent for a new process-group.
>
> However I am getting the next messages:
>
> Attempting GET request to: JerseyWebTarget { https://127.0.0.1:8443/nifi-
> api/process-groups/root }
> 2018-02-26 11:06:55.341 DEBUG  --- [   main]
> c.c.p.n.c.i.b.BootApiClient  :
> 2018-02-26 11:06:55.341 DEBUG  --- [   main]
> c.c.p.n.c.i.b.BootApiClient  : Received 403 response from GET
> to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root }
>
> com.civitaslearning.platform.nifi.client.invoker.boot.exception.NifiForbiddenException:
> No applicable policies could be found. Contact the system administrator.
>
> This is the content of my authorizations.xml file:
>
> 
>
> 
>
> 
>
>  resource="/flow" action="R">
>
> 
>
> 
>
>  resource="/restricted-components" action="W">
>
> 
>
> 
>
>  resource="/tenants" action="R">
>
> 
>
> 
>
>  resource="/tenants" action="W">
>
> 
>
> 
>
>  resource="/policies" action="R">
>
> 
>
> 
>
>  resource="/policies" action="W">
>
> 
>
> 
>
>  resource="/controller" action="R">
>
> 
>
> 
>
>  resource="/controller" action="W">
>
> 
>
> 
>
>  resource="/process-groups/root" action="R">
>
> 
>
> 
>
>  resource="/process-groups/root" action="W">
>
> 
>
> 
>
> 
>
> 
>
> And this is the content of authorizations.xml
>
> 
>
> 
>
> file-access-policy-provider
>
> org.apache.nifi.authorization.FileAccessPolicyProvider class>
>
> file-user-group-
> provider
>
> ./conf/authorizations.
> xml
>
> CN=civitas,
> OU=ApacheNifi
>
> 
>
>
> 
>
> 
>
> 
>
> managed-authorizer
>
> org.apache.nifi.authorization.StandardManagedAuthorizer class>
>
> file-access-policy-
> provider
>
> 
>
> 
>
>
> And users.xml
>
>
> 
>
> 
>
> 
>
> 
>
>  identity="CN=civitas, OU=ApacheNifi"/>
>
> 
>
> 
>
> I already create a policy using the same user cert so I guess the DN is
> valid.
> Am I defining the policy or making the call in a wrong way?
>
> Thanks in advance,
>
> Daniel Hernandez
>
>
>

Re: Policies for root Process Group.

[Matt Gilman]

Daniel,

The policy should use the UUID of the root Process Group.

If your seeding the authorizations.xml as part of your initial start-up,
these policies will be automatically applied to your initial admin if there
is an existing flow.xml.gz. If there is no flow.xml.gz, you'll need to
define these policies manually after starting up. You can see these
endpoints in action if you open your browser's Developer Tools and you
perform these actions in the UI. You should be able to update your client
following these examples.

Matt

On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez <
daniel.hernan...@civitaslearning.com> wrote:

> Hi,
>
> I am currently working on calling the Nifi REST API to get the 'root'
> process group and setting it as parent for a new process-group.
>
> However I am getting the next messages:
>
> Attempting GET request to: JerseyWebTarget {
> https://127.0.0.1:8443/nifi-api/process-groups/root }
> 2018-02-26 11:06:55.341 DEBUG  --- [   main]
> c.c.p.n.c.i.b.BootApiClient  :
> 2018-02-26 11:06:55.341 DEBUG  --- [   main]
> c.c.p.n.c.i.b.BootApiClient  : Received 403 response from GET
> to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root }
>
> com.civitaslearning.platform.nifi.client.invoker.boot.exception.
> NifiForbiddenException:
> No applicable policies could be found. Contact the system administrator.
>
> This is the content of my authorizations.xml file:
>
> 
>
> 
>
> 
>
>  resource="/flow" action="R">
>
> 
>
> 
>
>  resource="/restricted-components" action="W">
>
> 
>
> 
>
>  resource="/tenants" action="R">
>
> 
>
> 
>
>  resource="/tenants" action="W">
>
> 
>
> 
>
>  resource="/policies" action="R">
>
> 
>
> 
>
>  resource="/policies" action="W">
>
> 
>
> 
>
>  resource="/controller" action="R">
>
> 
>
> 
>
>  resource="/controller" action="W">
>
> 
>
> 
>
>  resource="/process-groups/root" action="R">
>
> 
>
> 
>
>  resource="/process-groups/root" action="W">
>
> 
>
> 
>
> 
>
> 
>
> And this is the content of authorizations.xml
>
> 
>
> 
>
> file-access-policy-provider
>
>
> org.apache.nifi.authorization.FileAccessPolicyProvider
>
> file-user-group-provider
>
> ./conf/authorizations.xml
>
> CN=civitas,
> OU=ApacheNifi
>
> 
>
>
> 
>
> 
>
> 
>
> managed-authorizer
>
>
> org.apache.nifi.authorization.StandardManagedAuthorizer
>
> file-access-policy-provider
>
> 
>
> 
>
>
> And users.xml
>
>
> 
>
> 
>
> 
>
> 
>
>  identity="CN=civitas, OU=ApacheNifi"/>
>
> 
>
> 
>
> I already create a policy using the same user cert so I guess the DN is
> valid.
> Am I defining the policy or making the call in a wrong way?
>
> Thanks in advance,
>
> Daniel Hernandez
>

Policies for root Process Group.

[Daniel Hernandez]

Hi,

I am currently working on calling the Nifi REST API to get the 'root'
process group and setting it as parent for a new process-group.

However I am getting the next messages:

Attempting GET request to: JerseyWebTarget {
https://127.0.0.1:8443/nifi-api/process-groups/root }
2018-02-26 11:06:55.341 DEBUG  --- [   main]
c.c.p.n.c.i.b.BootApiClient  :
2018-02-26 11:06:55.341 DEBUG  --- [   main]
c.c.p.n.c.i.b.BootApiClient  : Received 403 response from GET
to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root }

com.civitaslearning.platform.nifi.client.invoker.boot.exception.NifiForbiddenException:
No applicable policies could be found. Contact the system administrator.

This is the content of my authorizations.xml file:







































































And this is the content of authorizations.xml





file-access-policy-provider


org.apache.nifi.authorization.FileAccessPolicyProvider

file-user-group-provider

./conf/authorizations.xml

CN=civitas,
OU=ApacheNifi










managed-authorizer


org.apache.nifi.authorization.StandardManagedAuthorizer

file-access-policy-provider






And users.xml
















I already create a policy using the same user cert so I guess the DN is
valid.
Am I defining the policy or making the call in a wrong way?

Thanks in advance,

Daniel Hernandez

Re: Create custom content viewer

[Matt Gilman]

There are defined UI extension points [1]. One of these is for the content
viewer. Here is the standard content viewer [2]. Check out this guide and
the referenced examples.

Matt

[1]
https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#ui-extensions
[2]
https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-content-viewer

On Sun, Feb 25, 2018 at 12:13 AM, Дмитрий Круглов <
dimpalych...@mail.ru.invalid> wrote:

>
> Is it possible to create custom content viewer, to extend functionality,
> or i have to append some code to exist standart comtent viewer and
> recompile?
> --
> Отправлено из Mail.Ru для Android

PutElasticsearchHttpRecord writing null values

[Mike Thomsen]

I have a client that uses that processor and doesn't like the fact that
null record fields get sent to ElasticSearch instead of excluded from the
payload. Does anyone know if there is a good reason to keep that behavior?
It doesn't seem to add anything except line noise when looking at the
results from ElasticSearch.

Thanks,

Mike