Yeah, my vote would definitely be to have the separate targets. I like having a lot of options for testing but definitely like being able to minimize needed dependencies appropriate to desired level of evaluation.
In terms of naming, would likely avoid calling these integration given their special nature, but otherwise have no strong preferences. On Wed, Aug 9, 2017 at 1:42 PM, Andy Christianson < achristian...@hortonworks.com> wrote: > These are good points. Having a separate target makes sense, plus it'll > reduce risk of interfering with the existing native development workflow. > > What shall we call the new target? Some possibilities: > > - make sip (system integration tests) > - make docker-verify > - make verify-docker > - make integration > > I'm open to ideas. > > -Andy I.C. > ________________________________________ > From: Marc <phroc...@apache.org> > Sent: Wednesday, August 09, 2017 1:16 PM > To: dev@nifi.apache.org > Subject: Re: MINIFI-350 CMake target for docker integration tests > > Andy, > This is great stuff. To facilitate use by developers we should probably > come to agreement on terminology. In my opinion, it seems that you are > facilitating more system integration testing (SIT) based on our > discussions. While what you are doing doesn't preclude you from integrating > one or two components it does cause confusion on what to use when I want to > test something. I think the same exists for our unit and integration tests > that we have now, but those are a function of design decisions made vice a > focus on testability. I think most of the tests that use the docker > framework will be system tests, and the importance of this is that we > should probably isolate these to a separate target. I think better > isolation of what these test facilities do will end users, and limiting > what ctest executes to pure unit and modular integration tests will be very > useful. > I think that also helps sell the story that one must install docker if > they want to run system integration tests, but you can still integrate > several components with ctest if you want to limit what you are testing. I > personally look forward to these changes as I think having multi host tests > will help find more bugs, but I wouldn't want this to run when I type make > test. > > On Wed, Aug 9, 2017 at 12:39 PM, Andy Christianson < > achristian...@hortonworks.com> wrote: > > > MiNiFi cpp team, > > > > > > I am currently working on MINIFI-350. I have an integration test > framework > > plus a few integration tests which are invoked via a new script called > > docker/DockerVerify.sh (https://github.com/achristian > son/nifi-minifi-cpp/ > > blob/MINIFI-350/docker/DockerVerify.sh?). This is intended to be > > consistent in naming and structure to the extant DockerBuild.sh. > > > > > > My question resides in a final bit of integration. I see that there is a > > custom CMake target in cmake/DockerConfig.cmake that calls > DockerBuild.sh. > > It seems like the best thing to integrate DockerVerify.sh is to add a > > sibling CMake custom target, perhaps called something like > 'docker-verify.' > > > > > > A few stylistic/structural questions come to mind. Do we want these > > integration tests to be invoked directly via a custom target, or do we > want > > that custom target to be an intermediate target which is called by some > > existing 'make test' or 'make verify' target? The biggest risk, from > what I > > can see, is that if we hooked into 'make test,' then it would fail if the > > user hadn't already run 'make docker.' > > > > > > Please advise on preferred style and structure with regard to integration > > of the new docker integration tests with CMake. > > > > > > -Andy I.C. > > > > >
