PutElasticsearchHttpRecord writing null values
I have a client that uses that processor and doesn't like the fact that null record fields get sent to ElasticSearch instead of excluded from the payload. Does anyone know if there is a good reason to keep that behavior? It doesn't seem to add anything except line noise when looking at the results from ElasticSearch. Thanks, Mike
Re: Create custom content viewer
There are defined UI extension points [1]. One of these is for the content viewer. Here is the standard content viewer [2]. Check out this guide and the referenced examples. Matt [1] https://nifi.apache.org/docs/nifi-docs/html/developer-guide.html#ui-extensions [2] https://github.com/apache/nifi/tree/master/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-content-viewer On Sun, Feb 25, 2018 at 12:13 AM, Дмитрий Круглов < dimpalych...@mail.ru.invalid> wrote: > > Is it possible to create custom content viewer, to extend functionality, > or i have to append some code to exist standart comtent viewer and > recompile? > -- > Отправлено из Mail.Ru для Android
Policies for root Process Group.
Hi, I am currently working on calling the Nifi REST API to get the 'root' process group and setting it as parent for a new process-group. However I am getting the next messages: Attempting GET request to: JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root } 2018-02-26 11:06:55.341 DEBUG --- [ main] c.c.p.n.c.i.b.BootApiClient : 2018-02-26 11:06:55.341 DEBUG --- [ main] c.c.p.n.c.i.b.BootApiClient : Received 403 response from GET to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root } com.civitaslearning.platform.nifi.client.invoker.boot.exception.NifiForbiddenException: No applicable policies could be found. Contact the system administrator. This is the content of my authorizations.xml file: And this is the content of authorizations.xml file-access-policy-provider org.apache.nifi.authorization.FileAccessPolicyProvider file-user-group-provider ./conf/authorizations.xml CN=civitas, OU=ApacheNifi managed-authorizer org.apache.nifi.authorization.StandardManagedAuthorizer file-access-policy-provider And users.xml I already create a policy using the same user cert so I guess the DN is valid. Am I defining the policy or making the call in a wrong way? Thanks in advance, Daniel Hernandez
Re: Policies for root Process Group.
Daniel, The policy should use the UUID of the root Process Group. If your seeding the authorizations.xml as part of your initial start-up, these policies will be automatically applied to your initial admin if there is an existing flow.xml.gz. If there is no flow.xml.gz, you'll need to define these policies manually after starting up. You can see these endpoints in action if you open your browser's Developer Tools and you perform these actions in the UI. You should be able to update your client following these examples. Matt On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez < daniel.hernan...@civitaslearning.com> wrote: > Hi, > > I am currently working on calling the Nifi REST API to get the 'root' > process group and setting it as parent for a new process-group. > > However I am getting the next messages: > > Attempting GET request to: JerseyWebTarget { > https://127.0.0.1:8443/nifi-api/process-groups/root } > 2018-02-26 11:06:55.341 DEBUG --- [ main] > c.c.p.n.c.i.b.BootApiClient : > 2018-02-26 11:06:55.341 DEBUG --- [ main] > c.c.p.n.c.i.b.BootApiClient : Received 403 response from GET > to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root } > > com.civitaslearning.platform.nifi.client.invoker.boot.exception. > NifiForbiddenException: > No applicable policies could be found. Contact the system administrator. > > This is the content of my authorizations.xml file: > > > > > > > > resource="/flow" action="R"> > > > > > > resource="/restricted-components" action="W"> > > > > > > resource="/tenants" action="R"> > > > > > > resource="/tenants" action="W"> > > > > > > resource="/policies" action="R"> > > > > > > resource="/policies" action="W"> > > > > > > resource="/controller" action="R"> > > > > > > resource="/controller" action="W"> > > > > > > resource="/process-groups/root" action="R"> > > > > > > resource="/process-groups/root" action="W"> > > > > > > > > > > And this is the content of authorizations.xml > > > > > > file-access-policy-provider > > > org.apache.nifi.authorization.FileAccessPolicyProvider > > file-user-group-provider > > ./conf/authorizations.xml > > CN=civitas, > OU=ApacheNifi > > > > > > > > > > > managed-authorizer > > > org.apache.nifi.authorization.StandardManagedAuthorizer > > file-access-policy-provider > > > > > > > And users.xml > > > > > > > > > > > identity="CN=civitas, OU=ApacheNifi"/> > > > > > > I already create a policy using the same user cert so I guess the DN is > valid. > Am I defining the policy or making the call in a wrong way? > > Thanks in advance, > > Daniel Hernandez >
Re: Policies for root Process Group.
Hi Matt, Thanks for your answer. Do you know if there is a way to preconfigure this value when running Nifi's Docker image? I am making the calls from an integration test that runs a docker container with the Nifi server. I already check and the value under in the flow.xml.gz file changes everytime I deploy the container, I guess it is created at startup. Is it possible that I can change my docker image to get a fix root group value? Thanks, Daniel On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez < daniel.hernan...@civitaslearning.com> wrote: > Hi, > > I am currently working on calling the Nifi REST API to get the 'root' > process group and setting it as parent for a new process-group. > > However I am getting the next messages: > > Attempting GET request to: JerseyWebTarget { https://127.0.0.1:8443/nifi- > api/process-groups/root } > 2018-02-26 11:06:55.341 DEBUG --- [ main] > c.c.p.n.c.i.b.BootApiClient : > 2018-02-26 11:06:55.341 DEBUG --- [ main] > c.c.p.n.c.i.b.BootApiClient : Received 403 response from GET > to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root } > > com.civitaslearning.platform.nifi.client.invoker.boot.exception.NifiForbiddenException: > No applicable policies could be found. Contact the system administrator. > > This is the content of my authorizations.xml file: > > > > > > > > resource="/flow" action="R"> > > > > > > resource="/restricted-components" action="W"> > > > > > > resource="/tenants" action="R"> > > > > > > resource="/tenants" action="W"> > > > > > > resource="/policies" action="R"> > > > > > > resource="/policies" action="W"> > > > > > > resource="/controller" action="R"> > > > > > > resource="/controller" action="W"> > > > > > > resource="/process-groups/root" action="R"> > > > > > > resource="/process-groups/root" action="W"> > > > > > > > > > > And this is the content of authorizations.xml > > > > > > file-access-policy-provider > > org.apache.nifi.authorization.FileAccessPolicyProvider class> > > file-user-group- > provider > > ./conf/authorizations. > xml > > CN=civitas, > OU=ApacheNifi > > > > > > > > > > > managed-authorizer > > org.apache.nifi.authorization.StandardManagedAuthorizer class> > > file-access-policy- > provider > > > > > > > And users.xml > > > > > > > > > > > identity="CN=civitas, OU=ApacheNifi"/> > > > > > > I already create a policy using the same user cert so I guess the DN is > valid. > Am I defining the policy or making the call in a wrong way? > > Thanks in advance, > > Daniel Hernandez > > >
Re: Policies for root Process Group.
Daniel, Unfortunately, there is no way to set this currently. This is ultimately a lifecycle issue. The UUID of the root group may be inherited from a cluster or randomly generated if a node is standalone. From the admin guide: "For a brand new secure flow, providing the "Initial Admin Identity" gives that user access to get into the UI and to manage users, groups and policies. But if that user wants to start modifying the flow, they need to grant themselves policies for the root process group. The system is unable to do this automatically because in a new flow the UUID of the root process group is not permanent until the flow.xml.gz is generated. If the NiFi instance is an upgrade from an existing flow.xml.gz or a 1.x instance going from unsecure to secure, then the "Initial Admin Identity" user is automatically given the privileges to modify the flow." Because of this, when there is no existing flow, granting permissions to the root group would need to happen after this initial startup. Matt On Mon, Feb 26, 2018 at 3:26 PM, Daniel Hernandez < daniel.hernan...@civitaslearning.com> wrote: > Hi Matt, > > Thanks for your answer. > > Do you know if there is a way to preconfigure this value when running > Nifi's Docker image? I am making the calls from an integration test that > runs a docker container with the Nifi server. I already check and the value > under in the flow.xml.gz file changes everytime I deploy > the container, I guess it is created at startup. Is it possible that I can > change my docker image to get a fix root group value? > > Thanks, > > Daniel > > On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez < > daniel.hernan...@civitaslearning.com> wrote: > > > Hi, > > > > I am currently working on calling the Nifi REST API to get the 'root' > > process group and setting it as parent for a new process-group. > > > > However I am getting the next messages: > > > > Attempting GET request to: JerseyWebTarget { > https://127.0.0.1:8443/nifi- > > api/process-groups/root } > > 2018-02-26 11:06:55.341 DEBUG --- [ main] > > c.c.p.n.c.i.b.BootApiClient : > > 2018-02-26 11:06:55.341 DEBUG --- [ main] > > c.c.p.n.c.i.b.BootApiClient : Received 403 response from GET > > to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root > } > > > > com.civitaslearning.platform.nifi.client.invoker.boot.exception. > NifiForbiddenException: > > No applicable policies could be found. Contact the system administrator. > > > > This is the content of my authorizations.xml file: > > > > > > > > > > > > > > > > > resource="/flow" action="R"> > > > > > > > > > > > > > resource="/restricted-components" action="W"> > > > > > > > > > > > > > resource="/tenants" action="R"> > > > > > > > > > > > > > resource="/tenants" action="W"> > > > > > > > > > > > > > resource="/policies" action="R"> > > > > > > > > > > > > > resource="/policies" action="W"> > > > > > > > > > > > > > resource="/controller" action="R"> > > > > > > > > > > > > > resource="/controller" action="W"> > > > > > > > > > > > > > resource="/process-groups/root" action="R"> > > > > > > > > > > > > > resource="/process-groups/root" action="W"> > > > > > > > > > > > > > > > > > > > > And this is the content of authorizations.xml > > > > > > > > > > > > file-access-policy-provider > > > > org.apache.nifi.authorization.FileAccessPolicyProvider > class> > > > > file-user-group- > > provider > > > > ./conf/authorizations. > > xml > > > > CN=civitas, > > OU=ApacheNifi > > > > > > > > > > > > > > > > > > > > > > managed-authorizer > > > > org.apache.nifi.authorization.StandardManagedAuthorizer > class> > > > > file-access-policy- > > provider > > > > > > > > > > > > > > And users.xml > > > > > > > > > > > > > > > > > > > > > > > identity="CN=civitas, OU=ApacheNifi"/> > > > > > > > > > > > > I already create a policy using the same user cert so I guess the DN is > > valid. > > Am I defining the policy or making the call in a wrong way? > > > > Thanks in advance, > > > > Daniel Hernandez > > > > > > >
Re: Policies for root Process Group.
You should be able to include a canned flow.xml.gz in your in your container, just have nothing under the root group. On Mon, Feb 26, 2018 at 3:50 PM, Matt Gilman wrote: > Daniel, > > Unfortunately, there is no way to set this currently. This is ultimately a > lifecycle issue. The UUID of the root group may be inherited from a cluster > or randomly generated if a node is standalone. From the admin guide: > > "For a brand new secure flow, providing the "Initial Admin Identity" gives > that user access to get into the UI and to manage users, groups and > policies. But if that user wants to start modifying the flow, they need to > grant themselves policies for the root process group. The system is unable > to do this automatically because in a new flow the UUID of the root process > group is not permanent until the flow.xml.gz is generated. If the NiFi > instance is an upgrade from an existing flow.xml.gz or a 1.x instance going > from unsecure to secure, then the "Initial Admin Identity" user is > automatically given the privileges to modify the flow." > > Because of this, when there is no existing flow, granting permissions to > the root group would need to happen after this initial startup. > > Matt > > > On Mon, Feb 26, 2018 at 3:26 PM, Daniel Hernandez < > daniel.hernan...@civitaslearning.com> wrote: > >> Hi Matt, >> >> Thanks for your answer. >> >> Do you know if there is a way to preconfigure this value when running >> Nifi's Docker image? I am making the calls from an integration test that >> runs a docker container with the Nifi server. I already check and the value >> under in the flow.xml.gz file changes everytime I deploy >> the container, I guess it is created at startup. Is it possible that I can >> change my docker image to get a fix root group value? >> >> Thanks, >> >> Daniel >> >> On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez < >> daniel.hernan...@civitaslearning.com> wrote: >> >> > Hi, >> > >> > I am currently working on calling the Nifi REST API to get the 'root' >> > process group and setting it as parent for a new process-group. >> > >> > However I am getting the next messages: >> > >> > Attempting GET request to: JerseyWebTarget { >> https://127.0.0.1:8443/nifi- >> > api/process-groups/root } >> > 2018-02-26 11:06:55.341 DEBUG --- [ main] >> > c.c.p.n.c.i.b.BootApiClient : >> > 2018-02-26 11:06:55.341 DEBUG --- [ main] >> > c.c.p.n.c.i.b.BootApiClient : Received 403 response from GET >> > to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root >> } >> > >> > com.civitaslearning.platform.nifi.client.invoker.boot.exception. >> NifiForbiddenException: >> > No applicable policies could be found. Contact the system administrator. >> > >> > This is the content of my authorizations.xml file: >> > >> > >> > >> > >> > >> > >> > >> > > > resource="/flow" action="R"> >> > >> > >> > >> > >> > >> > > > resource="/restricted-components" action="W"> >> > >> > >> > >> > >> > >> > > > resource="/tenants" action="R"> >> > >> > >> > >> > >> > >> > > > resource="/tenants" action="W"> >> > >> > >> > >> > >> > >> > > > resource="/policies" action="R"> >> > >> > >> > >> > >> > >> > > > resource="/policies" action="W"> >> > >> > >> > >> > >> > >> > > > resource="/controller" action="R"> >> > >> > >> > >> > >> > >> > > > resource="/controller" action="W"> >> > >> > >> > >> > >> > >> > > > resource="/process-groups/root" action="R"> >> > >> > >> > >> > >> > >> > > > resource="/process-groups/root" action="W"> >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > And this is the content of authorizations.xml >> > >> > >> > >> > >> > >> > file-access-policy-provider >> > >> > org.apache.nifi.authorization.FileAccessPolicyProvider> > class> >> > >> > file-user-group- >> > provider >> > >> > ./conf/authorizations. >> > xml >> > >> > CN=civitas, >> > OU=ApacheNifi >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > managed-authorizer >> > >> > org.apache.nifi.authorization.StandardManagedAuthorizer> > class> >> > >> > file-access-policy- >> > provider >> > >> > >> > >> > >> > >> > >> > And users.xml >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > > > identity="CN=civitas, OU=ApacheNifi"/> >> > >> > >> > >> > >> > >> > I already create a policy using the same user cert so I guess the DN is >> > valid. >> > Am I defining the policy or making the call in a wrong way? >> > >> > Thanks in advance, >> > >> > Daniel Hernandez >> > >> > >> > >>
Re: Policies for root Process Group.
Thanks Matt, I get now what is the problem, in order to exhaust all my possibilities I may ask, is there a way using the API to get the root UUID from the flow.xml.gz file? Because I see the file there after running the tests. Thanks, On Mon, Feb 26, 2018 at 3:26 PM, Daniel Hernandez < daniel.hernan...@civitaslearning.com> wrote: > Hi Matt, > > Thanks for your answer. > > Do you know if there is a way to preconfigure this value when running > Nifi's Docker image? I am making the calls from an integration test that > runs a docker container with the Nifi server. I already check and the value > under in the flow.xml.gz file changes everytime I deploy > the container, I guess it is created at startup. Is it possible that I can > change my docker image to get a fix root group value? > > Thanks, > > Daniel > > On Mon, Feb 26, 2018 at 11:35 AM, Daniel Hernandez civitaslearning.com> wrote: > >> Hi, >> >> I am currently working on calling the Nifi REST API to get the 'root' >> process group and setting it as parent for a new process-group. >> >> However I am getting the next messages: >> >> Attempting GET request to: JerseyWebTarget { >> https://127.0.0.1:8443/nifi-api/process-groups/root } >> 2018-02-26 11:06:55.341 DEBUG --- [ main] >> c.c.p.n.c.i.b.BootApiClient : >> 2018-02-26 11:06:55.341 DEBUG --- [ main] >> c.c.p.n.c.i.b.BootApiClient : Received 403 response from GET >> to JerseyWebTarget { https://127.0.0.1:8443/nifi-api/process-groups/root >> } >> >> com.civitaslearning.platform.nifi.client.invoker.boot.exception.NifiForbiddenException: >> No applicable policies could be found. Contact the system administrator. >> >> This is the content of my authorizations.xml file: >> >> >> >> >> >> >> >> > resource="/flow" action="R"> >> >> >> >> >> >> > resource="/restricted-components" action="W"> >> >> >> >> >> >> > resource="/tenants" action="R"> >> >> >> >> >> >> > resource="/tenants" action="W"> >> >> >> >> >> >> > resource="/policies" action="R"> >> >> >> >> >> >> > resource="/policies" action="W"> >> >> >> >> >> >> > resource="/controller" action="R"> >> >> >> >> >> >> > resource="/controller" action="W"> >> >> >> >> >> >> > resource="/process-groups/root" action="R"> >> >> >> >> >> >> > resource="/process-groups/root" action="W"> >> >> >> >> >> >> >> >> >> >> And this is the content of authorizations.xml >> >> >> >> >> >> file-access-policy-provider >> >> org.apache.nifi.authorization.FileAccessPolicyProvide >> r >> >> file-user-group-prov >> ider >> >> ./conf/authorizations.xm >> l >> >> CN=civitas, >> OU=ApacheNifi >> >> >> >> >> >> >> >> >> >> >> managed-authorizer >> >> org.apache.nifi.authorization.StandardManagedAuthoriz >> er >> >> file-access-policy-p >> rovider >> >> >> >> >> >> >> And users.xml >> >> >> >> >> >> >> >> >> >> >> > identity="CN=civitas, OU=ApacheNifi"/> >> >> >> >> >> >> I already create a policy using the same user cert so I guess the DN is >> valid. >> Am I defining the policy or making the call in a wrong way? >> >> Thanks in advance, >> >> Daniel Hernandez >> >> >> >
get access token inside custom processor
Hi guys, I'm trying to invoke some nifi rest apis inside my custom processor, the nifi I'm using is nifi 1.4.0 and it's a 3 node secured cluster., the username and password are kept inside a ldap server. I know that in a secured nifi cluster, in order to make any request I need the access token, my question is how could I get the access token in my custom processor? Thanks. (I think the token should be available somewhere after successful login right?) regards, ben