On Fri, Jun 15, 2012 at 3:06 AM, Andrea Aime
<[email protected]>wrote:
> On Fri, Jun 15, 2012 at 4:38 AM, Justin Deoliveira <[email protected]>
> wrote:
> >
> > (1) Keeping the same pattern as the existing test setup, we could simply
> > serialize out the new structure rather than the old structure. Should be
> > relatively straight forward and can probably use the facilities in
> > XStreamPersister to do so. An alternative would be to have a pre-canned
> data
> > directory and copy/unpack it when we have to do test setup.
>
> The precanned one would be good for the tests that use the standard
> mock data set, which is, I guess, the majority, so it sounds like a good
> idea.
>
> However some setup only a handlful of types skipping the whole data dir
> setup, so we need to be able to recognize if a test is setting up the base
> data dir or not. Many tests add on top of it, but that's something we can
> handle after having setup the precanned one right?
>
When I did this locally i sort of ditched the pre-canned idea and just
build up the resources programatically like we do now, just with the new
structure. It was actually quite easy, just create a local catalog
instance, attach a persister to it and start populating it.
>
> > (2) This is the interesting/tricky one. If tests that actually modify
> data
> > are going to utilize a one-time setup then what we need is way for a test
> > case to reload or refresh single layers at a time, returning both
> > configuration and data to original state. Or in cases where new layers
> are
> > added, remove them. Or if existing layers deleted, restore them. All of
> the
> > above I believe should really just amount to some utility classes on the
> > base test classes. However it does add a burden on the test writer since
> > they now have to worry about initialization/cleanup of state. Also there
> is
> > the issue of test order. The order of tests run in an IDE is not always
> the
> > same as running from maven and since state is maintained across test
> methods
> > i can see this becoming a mess in which failures occur with maven but not
> > from the ide.
> >
> > Anyways, as an exercise I decided to locally port all the restconfig
> tests
> > to a one-time setup. As expected the results are significant. Currently
> on
> > trunk the full restconfig test run takes approximately 5 minutes on my
> local
> > machine. Utilizing the one time setup this drops down to about 1 minute
> 20
> > seconds.
> >
> > So can we find a middle ground between using a one-time setup and making
> > test writers not have to worry too much about state between test methods?
> > One of the ideas i have is to use annotations to help with this. With
> Junit4
> > we essentially have the ability to define our own custom annotations,
> called
> > "rules". So we could do something like this:
> >
> > @Test
> > @PostReload
> > public void testFoo() {
> > ...
> > }
> >
> > The "PostReload" annotation would essentially tell the framework that the
> > test modifies configuration/data and after the test is run the
> configuration
> > must be reloaded. This would allow tests mixing test methods that are
> read
> > only with ones that modify configuration only pay the price when things
> are
> > changed. Unfortunately this could still result in a lot of configuration
> > reloads for a single test class, like in the case for restconfig.
> >
> > Pushing the idea a bit further, i had the idea of being more granular
> about
> > it. Doing something like this:
> >
> > @Test
> > @PostReload(store=sf)
> > @PostRemove(layer=sf:newLayer)
> > public void testFoo() {
> > ...
> > }
> >
> > So instead of reloading the entire setup only a single store/layer/etc...
> > would have to be reloaded, or deleted, etc...
> >
> > Anyways, just a rough idea at this point. Adopting helper annotations
> like
> > this would imply an upgrade of Junit to version 4. Which some tests are
> > actually already using afaik. There is also TestNG, which from a
> > customization point of view is much more flexible than Junit 4. Downside
> is
> > it is really a completely different framework, the upgrade would be much
> > less seamless and it has less built in support from our current maven and
> > eclipse tooling.
>
> Are there tests using junit 4 in GeoServer?
>
Maybe not, i was probably thinking of geotools.
> Not against using Junit 4 or TestNG btw, but wondering how the current
> opt-in "one time setup" mechanism can be replicated there.
>
> I guess we might just not need the opt-in anymore if we switch to use
> the annotations you're proposing though.
>
Right, this is more of what I was thinking. Basically I was thinking that
you would have to explicitly "opt-out" of the one time setup and by default
have it used. The idea with the annotations would give you control and if
need be trigger a reload before a test. Or refresh particular resources
after a test, etc...
>
> Eclipse wise there is a decent plugin for TestNG, thought I'm not too
> happy about the way it works (maybe it's just a familiarity thing).
> Don't know about the build server.
>
Yeah, i haven't tried the plugin at all. TestNG does seem to offer a lot
more options though. Of particular interest to me are test groups
and parallelism.
> Regardless the switch to both JUnit 4 and TestNG would be a rather big
> task. I can help some during the weekend if we're going there, but
> it sounds like something that would require "all hands on board" to
> be carried out (plus probably a GSIP to vote the switch).
>
A big task and proposal worthy indeed. At this point just something i am
doing some initial experimentation on brought about by the fact that doing
full test runs are pretty painful.
As for how to migrate I am sort of thinking an incremental approach here.
Basically the idea would be come up with a new base test class that tests
would extend that implement the new testing setup. And port modules over it
to one-by-one.
>
> Cheers
> Andrea
>
> --
> Ing. Andrea Aime
> GeoSolutions S.A.S.
> Tech lead
>
> Via Poggio alle Viti 1187
> 55054 Massarosa (LU)
> Italy
>
> phone: +39 0584 962313
> fax: +39 0584 962313
> mob: +39 339 8844549
>
> http://www.geo-solutions.it
> http://geo-solutions.blogspot.com/
> http://www.youtube.com/user/GeoSolutionsIT
> http://www.linkedin.com/in/andreaaime
> http://twitter.com/geowolf
>
--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Geoserver-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/geoserver-devel
