Agreed. We can do that way. But for the starter I suggest we go ahead with the approach mentioned. This way we can atleast get going with proper tests being run for PR. We can later have a thorough discussion about what you mentioned and device a approach.
Thanks, Chinmay. On Fri, Apr 1, 2016 at 11:09 AM, Priyanka Gugale <[email protected]> wrote: > Hi, > > Categorization is for the purpose you mentioned. For each PR we generally > need to run all unit tests (includes tests written using mocks). While the > tests which depend on external systems or do end to end testing and are > time consuming should be categorized as something like functional tests. By > default we run only unit tests (which have no external dependency) and > there should be a flag to run functional tests as well. We can schedule to > run these functional tests may be once in couple of days on our test > cluster. > > -Priyanka > > On Thu, Mar 31, 2016 at 10:21 PM, Chinmay Kolhatkar <[email protected]> > wrote: > > > Yes. That is possible. test property of surefire overrides include and > > exclude property. > > > > - Chinmay. > > > > --- > > Sent from mobile. > > On 31 Mar 2016 9:09 p.m., "Thomas Weise" <[email protected]> wrote: > > > > > Also, it should be able to run tests that aren't enabled by default. > > > > > > Right now, we can do that via: > > > > > > mvn -DskipTests=false -Dtest=<testname> > > > > > > On Thu, Mar 31, 2016 at 8:33 AM, Chinmay Kolhatkar <[email protected] > > > > > wrote: > > > > > > > I think the test writer is the best person to make the choice between > > > mock > > > > and real based on the functionality. > > > > > > > > Also, this Jira is more about running tests that can be run without > > > > external setup during Jenkin's build for PRs and not about > categorizing > > > > them. > > > > Even if we categorize, we possibly would need to provide > > include/exclude > > > > switch to surefire maven plugin for which tests to run. > > > > > > > > -Chinmay. > > > > > > > > > > > > On Thu, Mar 31, 2016 at 6:07 PM, Priyanka Gugale < > > > [email protected] > > > > > > > > > wrote: > > > > > > > > > It's a good idea to separate out tests which has external > > dependencies. > > > > We > > > > > can find other solutions than using "exclude" tag. > > > > > We can actually categorize them as different tests, say functional > > > tests > > > > > and those should be invoked only if invoked explicitly. > > > > > > > > > > Most of times we should avoid using outside dependencies by using > > mock > > > or > > > > > embedded servers, only for exceptional cases we can do something > like > > > > > categorizing tests. > > > > > > > > > > -Priyanka > > > > > > > > > > On Thu, Mar 31, 2016 at 5:38 PM, Chinmay Kolhatkar < > > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > Hi All, > > > > > > > > > > > > Currently all tests in malhar contrib are skipped by > skipTests=true > > > > > > property. > > > > > > But there are certain tests which can run without external setup. > > > > > > > > > > > > I idea here is to use "include" configuration property of > surefire > > > > plugin > > > > > > and include only those which can run in jenkins build. > > > > > > > > > > > > Please share your opinion on following questions: > > > > > > > > > > > > 1. What are the test that should be included for the run? > > > > > > To start with, there are com.datatorrent.formatter.* & > > > > > > com.datatorrent.parser.* that can run in without external setup. > > > > > > > > > > > > 2. Should we use "exclude" tests which needs external setup > instead > > > > > > include? > > > > > > This way, any test that gets added to contrib will run by default > > and > > > > it > > > > > > won't get missed. > > > > > > Test writer would need to exclude only those tests which needs > > > external > > > > > > setup. > > > > > > > > > > > > Please share your opinion. > > > > > > > > > > > > Thanks, > > > > > > Chinmay. > > > > > > > > > > > > > > > > > > > > >
