Hi:
My thought is that we can classify tests into 3 major categories -
- Pure unit test - class isolated using mocks etc
- Integration with no external dependencies - some interaction with other
classes/in memory db, or mocks, but no external resources.
- Integration with external resource upto and including end-to-end - Which
require real external resources (http endpoint, aws etc)
In this way, we can always keep unit and integration w/o ext dependencies tests
while conditionally enable integration tests.
Mans
On Wednesday, January 13, 2016 11:24 AM, Aldrin Piri <aldrinp...@gmail.com>
wrote:
Definitely agree and can see value in those as well. The core issue I am
trying to address with this is that as highlighted by Joe.
As an intermediate step, I have created a PR [1] that shows how I envision
a baseline of this. Currently this treats all ITs (in this external
resource context) the same without separate profiles. What I am trying to
do is not impede a place for unit tests (or, perhaps more generically,
those that can run as part of the build).
[1] https://github.com/apache/nifi/pull/173
On Wed, Jan 13, 2016 at 8:19 AM, Oleg Zhurakousky <
ozhurakou...@hortonworks.com> wrote:
> Aldrin
>
> IMHO there are two types of integration testing; 1. Integration testing a
> Processor/ControllerService with the actual target system, 2. Integration
> testing of the flow or part of the flow.
> The second one essentially is the same as the first one but the target
> system is being NiFi itself and we are seriously lacking on that type of
> testing since NiFi is highly modularized and there is not a single module
> where such testing could be performed. As I’ve mentioned earlier in this
> list, I’ve already started such module on my fork
> https://github.com/olegz/nifi/tree/int-test which allowed me already to
> discover a few bugs and I think we need to start talking about pushing it
> to the trunk.
>
> So what I am essentially saying is that while profile and other maven
> tricks may set a convention to allow one to provide "conditional target
> system testing” (which is essentially what you mean by integration test
> with AWS etc), we still need to think about automating internal NiFi
> integration testing.
>
> Cheers
> Oleg
>
>
> On Jan 12, 2016, at 10:26 PM, Aldrin Piri <aldrinp...@gmail.com<mailto:
> aldrinp...@gmail.com>> wrote:
>
> All,
>
> We have had a few conversations regarding integration tests in the past In
> part, this issue has resurfaced thanks to the work going on with NIFI-1325
> in extending AWS related processors and extensions. The issue at hand is
> that, In this particular case, the contributor made test cases to
> supplement their contribution but, because the integration level test
> classes are ignored, had to create separate unit test classes to get these
> tests to run. AbstractS3Test lays the ground work quite appropriately for
> integration testing, but doesn't so much apply to unit tests where
> possible.
>
> What I am interested in is establishing direction for how these integration
> level tests should work within our codebase; superficially, what are the
> conventions that make sense and how do we treat these within our codebase.
> There is certainly grey area between unit and integration level tests at
> spots within our codebase, but for the immediate future, I am looking to
> establish a basis in terms of conventions so that we can avoid similar
> issues moving forward and perhaps get some guidelines that the community
> can use to provide these.
>
> Following the pattern that is present in support of using the
> maven-surefire-plugin, I could see a similar pattern of IT<Component> to
> complement the Test<Component>. This is a default pattern that is
> supported by the maven-failsafe-plugin, a likely candidate to drive this in
> our build process under some associated profile [1].
>
> In conjunction with this, I would like to see a profile that defaults to
> being deactivated that runs integration level tests and migrate the
> integration tests (AbstractS3Test and its subclasses) to an agreed upon
> convention as previously mentioned.
>
> We have an issue associated with this unit/integration level isolation,
> NIFI-569 [2].
>
> Any thoughts toward this would be most appreciated and I shall look to
> incorporate the points made in the associated issues in JIRA.
>
> Thanks!
>
> [1]
>
> http://maven.apache.org/surefire/maven-failsafe-plugin/examples/inclusion-exclusion.html
> [2] https://issues.apache.org/jira/browse/NIFI-569
>
>
