Hi, Categorization is for the purpose you mentioned. For each PR we generally need to run all unit tests (includes tests written using mocks). While the tests which depend on external systems or do end to end testing and are time consuming should be categorized as something like functional tests. By default we run only unit tests (which have no external dependency) and there should be a flag to run functional tests as well. We can schedule to run these functional tests may be once in couple of days on our test cluster.
-Priyanka On Thu, Mar 31, 2016 at 10:21 PM, Chinmay Kolhatkar <[email protected]> wrote: > Yes. That is possible. test property of surefire overrides include and > exclude property. > > - Chinmay. > > --- > Sent from mobile. > On 31 Mar 2016 9:09 p.m., "Thomas Weise" <[email protected]> wrote: > > > Also, it should be able to run tests that aren't enabled by default. > > > > Right now, we can do that via: > > > > mvn -DskipTests=false -Dtest=<testname> > > > > On Thu, Mar 31, 2016 at 8:33 AM, Chinmay Kolhatkar <[email protected]> > > wrote: > > > > > I think the test writer is the best person to make the choice between > > mock > > > and real based on the functionality. > > > > > > Also, this Jira is more about running tests that can be run without > > > external setup during Jenkin's build for PRs and not about categorizing > > > them. > > > Even if we categorize, we possibly would need to provide > include/exclude > > > switch to surefire maven plugin for which tests to run. > > > > > > -Chinmay. > > > > > > > > > On Thu, Mar 31, 2016 at 6:07 PM, Priyanka Gugale < > > [email protected] > > > > > > > wrote: > > > > > > > It's a good idea to separate out tests which has external > dependencies. > > > We > > > > can find other solutions than using "exclude" tag. > > > > We can actually categorize them as different tests, say functional > > tests > > > > and those should be invoked only if invoked explicitly. > > > > > > > > Most of times we should avoid using outside dependencies by using > mock > > or > > > > embedded servers, only for exceptional cases we can do something like > > > > categorizing tests. > > > > > > > > -Priyanka > > > > > > > > On Thu, Mar 31, 2016 at 5:38 PM, Chinmay Kolhatkar < > [email protected] > > > > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > Currently all tests in malhar contrib are skipped by skipTests=true > > > > > property. > > > > > But there are certain tests which can run without external setup. > > > > > > > > > > I idea here is to use "include" configuration property of surefire > > > plugin > > > > > and include only those which can run in jenkins build. > > > > > > > > > > Please share your opinion on following questions: > > > > > > > > > > 1. What are the test that should be included for the run? > > > > > To start with, there are com.datatorrent.formatter.* & > > > > > com.datatorrent.parser.* that can run in without external setup. > > > > > > > > > > 2. Should we use "exclude" tests which needs external setup instead > > > > > include? > > > > > This way, any test that gets added to contrib will run by default > and > > > it > > > > > won't get missed. > > > > > Test writer would need to exclude only those tests which needs > > external > > > > > setup. > > > > > > > > > > Please share your opinion. > > > > > > > > > > Thanks, > > > > > Chinmay. > > > > > > > > > > > > > > >
