On Fri, Apr 12, 2013 at 12:54:17PM -0400, Chip Childers wrote: > > > > > >Koushik, > > > > > >I think that it will require working with Prasanna to get a Marvin test > > >suite for the feature ready to go. I'd personally like to see that > > >*prior* to the merge into master. We need to "up our game" on automated > > >testing of new features, so that we stop digging ourselves deeper and > > >deeper into a hole. > > > > > >-chip > > > > But this approach isn't scalable either. > > > > Every new whiz bang network device will require infrastructure / licenses > > / etc. > > > > > > I think if there's component level tests of the code introduced (at least > > the network elements) that should be a good first step. > > > > -- > > Chiradeep > > Agreed that it would be a good first step, absolutely. We really do > need to think about how we deal with all of these integrations... and > how we do ongoing testing. It's going to be a challenge either way. > > The point of having a marvin test, IMO, isn't necessarily tied to the CI > infrastructure. Instead, it allows for a regression test to be run when > and if there is an appropriate target infrastructure. I'm sort of > breaking the problem into two parts: having tests to run if you have the > required setup, and figuring out how we get the required setup. >
In general the way we've taken care of external devices for other network appliances, NetScaler for instance, is to assume the deployment contains the network element present (inline, side-by-side, thingamajig mode) and test the services it provides as a tenant would use it. We don't test the actual deployment itself and assume the admin has provisioned and registered the appliance appropriately. -- Prasanna.,
