RE: [EXT] Re: Automated Testing of Flows
Hi Peter, One thing I have had success with in the past is very similar to what you are stating here. Basically a test framework built in python that is just applying the flow to a processor group for a test, and then performing the tests. NiFi it self is stood up inside of a docker container to verify the flows, and then just reporting back errors when a test fails (via Jenkins pipeline). Specifically I leveraged, in separate instances, the Robot and RedwoodHQ frameworks. I have been interested in building a flow testing framework from Gauge (the new thoughtworks test framework), but have not yet found the time. If you are thinking about working on something to industrialize this, I would be happy to work with you on it. Thanks and Regards, Chris On Nov 17, 2017, 12:15 AM -0600, Peter Wicks (pwicks) , wrote: > I don't even know if I need Jenkins to setup a fresh instance, that could be > nice, but I could use my TEST instance and just upgrade it by hand anytime I > want to test a new version. I could create a dedicated Process Group that my > test scripts would use for deploying the templates. > > I've worked with the REST API quite a bit, so I'd probably use that to > configure the data sources, etc... Have the test runner be Java or Python. > > From a completely different approach, I saw a recent PR to allow for NiFi to > be started up from within another process. This could be an interesting > approach if the API's are easily accessible. > https://github.com/apache/nifi/pull/2251 > > > > > -Original Message- > From: Scott Aslan [mailto:scottyas...@gmail.com] > Sent: Friday, November 17, 2017 12:39 > To: dev@nifi.apache.org > Subject: [EXT] Re: Automated Testing of Flows > > Just brain storming here > > You could use travis or Jenkins to setup a job where you use mvn and npm to > build and start NiFi. Assuming the test template was available from within > the environment maybe you could use the protractor e2e testing suite to > script the clicks necessary to load the template and run the flow... > > On Wed, Nov 15, 2017 at 8:07 PM, Peter Wicks (pwicks) wrote: > > > I would like to find a way to automatically run integration/upgrade > > tests on flows, rather than just one or two processors like the mock > > framework supports. Preferably something Template driven, where I can > > provide a template xml file, and an accompanying config file to setup > > the test, then run the flow and examine the results. > > > > Has anyone worked on something like this? > >
RE: [EXT] Re: Automated Testing of Flows
I don't even know if I need Jenkins to setup a fresh instance, that could be nice, but I could use my TEST instance and just upgrade it by hand anytime I want to test a new version. I could create a dedicated Process Group that my test scripts would use for deploying the templates. I've worked with the REST API quite a bit, so I'd probably use that to configure the data sources, etc... Have the test runner be Java or Python. From a completely different approach, I saw a recent PR to allow for NiFi to be started up from within another process. This could be an interesting approach if the API's are easily accessible. https://github.com/apache/nifi/pull/2251 -Original Message- From: Scott Aslan [mailto:scottyas...@gmail.com] Sent: Friday, November 17, 2017 12:39 To: dev@nifi.apache.org Subject: [EXT] Re: Automated Testing of Flows Just brain storming here You could use travis or Jenkins to setup a job where you use mvn and npm to build and start NiFi. Assuming the test template was available from within the environment maybe you could use the protractor e2e testing suite to script the clicks necessary to load the template and run the flow... On Wed, Nov 15, 2017 at 8:07 PM, Peter Wicks (pwicks) wrote: > I would like to find a way to automatically run integration/upgrade > tests on flows, rather than just one or two processors like the mock > framework supports. Preferably something Template driven, where I can > provide a template xml file, and an accompanying config file to setup > the test, then run the flow and examine the results. > > Has anyone worked on something like this? >
Re: Automated Testing of Flows
Just brain storming here You could use travis or Jenkins to setup a job where you use mvn and npm to build and start NiFi. Assuming the test template was available from within the environment maybe you could use the protractor e2e testing suite to script the clicks necessary to load the template and run the flow... On Wed, Nov 15, 2017 at 8:07 PM, Peter Wicks (pwicks) wrote: > I would like to find a way to automatically run integration/upgrade tests > on flows, rather than just one or two processors like the mock framework > supports. Preferably something Template driven, where I can provide a > template xml file, and an accompanying config file to setup the test, then > run the flow and examine the results. > > Has anyone worked on something like this? >
Automated Testing of Flows
I would like to find a way to automatically run integration/upgrade tests on flows, rather than just one or two processors like the mock framework supports. Preferably something Template driven, where I can provide a template xml file, and an accompanying config file to setup the test, then run the flow and examine the results. Has anyone worked on something like this?
