RE: Easiest way to list all NiFi processors in a release.
I develop in Java. Thinking of switching to Kotlin (rather than Scala).
Here is how the relevant code looks in my NifiRestClient, which uses Swagger
generated REST client to access NiFi REST API.
/*
* List all NiFi processors in this release
*/
public List listAllProcessors() {
List processors = new ArrayList<>();
try {
flowApi.getProcessorTypes()
.getProcessorTypes()
.forEach(dto -> {
// get processor short type from class
name
String type = dto.getType();
type =
type.substring(type.lastIndexOf('.') + 1);
processors.add(type);
}
);
} catch (Exception ae) {
printApiException("NifiRestClient#listAllProcessors", ae);
throw new IllegalArgumentException("Failed to obtain a list
of all available NiFi processors");
}
processors.sort(Comparator.naturalOrder());
return processors;
}
}
From: Daniel Chaffelson
Sent: Tuesday, November 13, 2018 7:57 AM
To: users@nifi.apache.org
Subject: Re: Easiest way to list all NiFi processors in a release.
There is also a shortcut function NiPyApi for this, as it is handy when
procedurally deploying flows:
https://nipyapi.readthedocs.io/en/latest/_modules/nipyapi/canvas.html#list_all_processor_types
On Mon, Nov 12, 2018 at 6:23 PM Pierre Villard
mailto:pierre.villard...@gmail.com>> wrote:
Hi Vitaly,
Yes there is one [1]: GET on /flow/processor-types
Easiest way to find out is to use the developer tool of your browser. Anything
viewable/doable in the NiFi UI is a doable through REST API endpoints.
[1] https://nifi.apache.org/docs/nifi-docs/rest-api/
Thanks,
Pierre
Le lun. 12 nov. 2018 à 18:36, Vitaly Krivoy
mailto:vitaly_kri...@jhancock.com>> a écrit :
Is there a REST API to list all NiFi processors in a current NiFi release?
Thanks.
STATEMENT OF CONFIDENTIALITY The information contained in this email message
and any attachments may be confidential and legally privileged and is intended
for the use of the addressee(s) only. If you are not an intended recipient,
please: (1) notify me immediately by replying to this message; (2) do not use,
disseminate, distribute or reproduce any part of the message or any attachment;
and (3) destroy all copies of this message and any attachments.
STATEMENT OF CONFIDENTIALITY The information contained in this email message
and any attachments may be confidential and legally privileged and is intended
for the use of the addressee(s) only. If you are not an intended recipient,
please: (1) notify me immediately by replying to this message; (2) do not use,
disseminate, distribute or reproduce any part of the message or any attachment;
and (3) destroy all copies of this message and any attachments.
RE: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted
I don't believe I see any error messages on the initial remote process group that gets created. I do see the error message when I do a restart and there are existing remote process groups but they do seem to connect eventually It's only the initial create of the first remote process group that seems to be acting weird. -Original Message- From: Koji Kawamura Sent: Thursday, November 15, 2018 1:05 AM To: users@nifi.apache.org Subject: Re: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted [CAUTION: This email originated from outside of Kodak Alaris. Do not click links or open attachments unless you recognize the sender and know the content is safe.] Hello William, > fails to connect to the existing input port until I do a restart of > NiFi Is there any error message when it fails? Connection refused? It should not require a NiFi restart to establish connection. Thanks, Koji On Thu, Nov 15, 2018 at 1:38 AM William Gosse wrote: > > I'm using NiFi's restful api to create a process group from a template > that contains a file folder listener and a remote group. I also have > an existing input port going to a process group that has the file > fetch and everything else for processing the file that was fetched. > This is not added by a restful api but is loaded from a template after > the initial startup of Nifi. What I wind up with is a list fetch > pattern that should cluster ready/ > > My problem is that the first time I use the rest api to add the listener > process group the remote group that's inside of it fails to connect to the > existing input port until I do a restart of NiFi. At that point all > subsequent listeners I add seem to connect just fine. > > Is there way that I can cause the initial connection to occur without having > to perform a restart? > >
Re: Custom processors/controller services without Maven
Hello, It would really be best to use Maven as NiFi's NAR Maven plugin will ensure that you are building the proper structure and will make things easy for you. Otherwise it would be up to you to somehow create the appropriate NAR structure (and if it ever changes you would have to update your process). A NAR is an archive with the following layout: META-INF/ META-INF/MANIFEST.MF META-INF/bundled-dependencies The MANIFEST file contains metadata about the NAR, and the bundled-dependencies directory contains all of the required JARs (your JAR + all transitive dependencies). You can take any NAR from NiFi and unzip it to see an example. -Bryan On Thu, Nov 15, 2018 at 4:54 PM Arun kumar wrote: > > Hi, > > I am looking for ways to write custom processors and controller services > and generate NAR from eclipse without maven dependencies. Pls provide > pointers. > > Thanks.
Re: Custom processors/controller services without Maven
I would recommend ( if you want to do this ) looking at the source code for the nar maven plugin, since you will have to re-implement it some how. On November 15, 2018 at 18:19:36, Milan Das (m...@interset.com) wrote: Trust me, you are bringing trouble. I am not sure if there is any user have done that. You may practice the following and try building it the nar (zip on your own). https://www.nifi.rocks/developing-a-custom-apache-nifi-processor-json/ Thanks, Milan Das On 11/15/18, 4:54 PM, "Arun kumar" wrote: Hi, I am looking for ways to write custom processors and controller services and generate NAR from eclipse without maven dependencies. Pls provide pointers. Thanks.
Re: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted
Hi William, > I do see the error message when I do a restart and there are existing remote > process groups but they do seem to connect eventually This is a known behavior for RPGs connecting to the same NiFi. During start up process, RemoteProcessGroups try to get remote info before NiFi API gets ready. They connect eventually. But again, you don't have to do restart to make RPGs work after you instantiated it by REST API. > It's only the initial create of the first remote process group that seems to > be acting weird. Please elaborate how weird it behaves. If the added RPG doesn't recognize the existing ports, then I'd add some wait logic as Chris suggested. Thanks, Koji On Fri, Nov 16, 2018 at 2:03 AM William Gosse wrote: > > I don't believe I see any error messages on the initial remote process group > that gets created. > > I do see the error message when I do a restart and there are existing remote > process groups but they do seem to connect eventually > > It's only the initial create of the first remote process group that seems to > be acting weird. > > -Original Message- > From: Koji Kawamura > Sent: Thursday, November 15, 2018 1:05 AM > To: users@nifi.apache.org > Subject: Re: NiFi 1.7.1 remote group not connecting when added through > restful api until nifi restarted > > [CAUTION: This email originated from outside of Kodak Alaris. Do not click > links or open attachments unless you recognize the sender and know the > content is safe.] > > Hello William, > > > fails to connect to the existing input port until I do a restart of > > NiFi > Is there any error message when it fails? Connection refused? > It should not require a NiFi restart to establish connection. > > Thanks, > Koji > > > > On Thu, Nov 15, 2018 at 1:38 AM William Gosse > wrote: > > > > I'm using NiFi's restful api to create a process group from a template > > that contains a file folder listener and a remote group. I also have > > an existing input port going to a process group that has the file > > fetch and everything else for processing the file that was fetched. > > This is not added by a restful api but is loaded from a template after > > the initial startup of Nifi. What I wind up with is a list fetch > > pattern that should cluster ready/ > > > > My problem is that the first time I use the rest api to add the listener > > process group the remote group that's inside of it fails to connect to the > > existing input port until I do a restart of NiFi. At that point all > > subsequent listeners I add seem to connect just fine. > > > > Is there way that I can cause the initial connection to occur without > > having to perform a restart? > > > >
RE: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted
Hi William My experience with using the REST API to create RPG is that you must wait “some time” before the port becomes available. A restart is not needed though. My workaround is to repeatedly call GET on the RPG until the port becomes available. It typically takes more than 10 seconds on a 5 nodes cluster. Chris -Original Message- From: William Gosse Sent: Donnerstag, 15 November 2018 18:03 To: users@nifi.apache.org Subject: RE: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted I don't believe I see any error messages on the initial remote process group that gets created. I do see the error message when I do a restart and there are existing remote process groups but they do seem to connect eventually It's only the initial create of the first remote process group that seems to be acting weird. -Original Message- From: Koji Kawamura Sent: Thursday, November 15, 2018 1:05 AM To: users@nifi.apache.org Subject: Re: NiFi 1.7.1 remote group not connecting when added through restful api until nifi restarted [CAUTION: This email originated from outside of Kodak Alaris. Do not click links or open attachments unless you recognize the sender and know the content is safe.] Hello William, > fails to connect to the existing input port until I do a restart of > NiFi Is there any error message when it fails? Connection refused? It should not require a NiFi restart to establish connection. Thanks, Koji On Thu, Nov 15, 2018 at 1:38 AM William Gosse wrote: > > I'm using NiFi's restful api to create a process group from a template > that contains a file folder listener and a remote group. I also have > an existing input port going to a process group that has the file > fetch and everything else for processing the file that was fetched. > This is not added by a restful api but is loaded from a template after > the initial startup of Nifi. What I wind up with is a list fetch > pattern that should cluster ready/ > > My problem is that the first time I use the rest api to add the listener > process group the remote group that's inside of it fails to connect to the > existing input port until I do a restart of NiFi. At that point all > subsequent listeners I add seem to connect just fine. > > Is there way that I can cause the initial connection to occur without having > to perform a restart? > >
