Definitely agree and can see value in those as well. The core issue I am trying to address with this is that as highlighted by Joe.
As an intermediate step, I have created a PR [1] that shows how I envision a baseline of this. Currently this treats all ITs (in this external resource context) the same without separate profiles. What I am trying to do is not impede a place for unit tests (or, perhaps more generically, those that can run as part of the build). [1] https://github.com/apache/nifi/pull/173 On Wed, Jan 13, 2016 at 8:19 AM, Oleg Zhurakousky < ozhurakou...@hortonworks.com> wrote: > Aldrin > > IMHO there are two types of integration testing; 1. Integration testing a > Processor/ControllerService with the actual target system, 2. Integration > testing of the flow or part of the flow. > The second one essentially is the same as the first one but the target > system is being NiFi itself and we are seriously lacking on that type of > testing since NiFi is highly modularized and there is not a single module > where such testing could be performed. As I’ve mentioned earlier in this > list, I’ve already started such module on my fork > https://github.com/olegz/nifi/tree/int-test which allowed me already to > discover a few bugs and I think we need to start talking about pushing it > to the trunk. > > So what I am essentially saying is that while profile and other maven > tricks may set a convention to allow one to provide "conditional target > system testing” (which is essentially what you mean by integration test > with AWS etc), we still need to think about automating internal NiFi > integration testing. > > Cheers > Oleg > > > On Jan 12, 2016, at 10:26 PM, Aldrin Piri <aldrinp...@gmail.com<mailto: > aldrinp...@gmail.com>> wrote: > > All, > > We have had a few conversations regarding integration tests in the past In > part, this issue has resurfaced thanks to the work going on with NIFI-1325 > in extending AWS related processors and extensions. The issue at hand is > that, In this particular case, the contributor made test cases to > supplement their contribution but, because the integration level test > classes are ignored, had to create separate unit test classes to get these > tests to run. AbstractS3Test lays the ground work quite appropriately for > integration testing, but doesn't so much apply to unit tests where > possible. > > What I am interested in is establishing direction for how these integration > level tests should work within our codebase; superficially, what are the > conventions that make sense and how do we treat these within our codebase. > There is certainly grey area between unit and integration level tests at > spots within our codebase, but for the immediate future, I am looking to > establish a basis in terms of conventions so that we can avoid similar > issues moving forward and perhaps get some guidelines that the community > can use to provide these. > > Following the pattern that is present in support of using the > maven-surefire-plugin, I could see a similar pattern of IT<Component> to > complement the Test<Component>. This is a default pattern that is > supported by the maven-failsafe-plugin, a likely candidate to drive this in > our build process under some associated profile [1]. > > In conjunction with this, I would like to see a profile that defaults to > being deactivated that runs integration level tests and migrate the > integration tests (AbstractS3Test and its subclasses) to an agreed upon > convention as previously mentioned. > > We have an issue associated with this unit/integration level isolation, > NIFI-569 [2]. > > Any thoughts toward this would be most appreciated and I shall look to > incorporate the points made in the associated issues in JIRA. > > Thanks! > > [1] > > http://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html > [2] https://issues.apache.org/jira/browse/NIFI-569 > >