Re: [Qgis-developer] rendering tests, what's the latest approach ?
On 15 November 2015 at 01:55, Sandro Santilli wrote: > On Sat, Nov 14, 2015 at 08:14:39PM +1100, Nyall Dawson wrote: >> On 13 November 2015 at 02:48, Sandro Santilli wrote: > >> > To be honest, I like the idea of rendering tests. I know they are >> > fragile in their current incarnation, but being a graphical >> > application having tests actually check what the user would see >> > is a good way to catch many issues. >> >> I personally think we've reached a good level of stability with the >> rendering tests. They now perform consistently across all platforms >> (passing when they should and failing when they shouldn't). So I'm >> keen to hear any thoughts on why you think these are still fragile? > > CDash shows many failures here (check 2.8.3-* builds): > http://dash.orfeo-toolbox.org/index.php?project=QGIS > > All of them are rendering-based tests. > > I'm sure the recent introduction of masks helps with that, but > it is evidently not easy enough to use. Ah. There's an easy answer for that - all the rendering test related improvements landed in QGIS 2.10, and have not been backported to 2.8. I started to do this at one stage, but in the end it's a considerable amount of work which I couldn't find time to complete. Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
On Sat, Nov 14, 2015 at 08:14:39PM +1100, Nyall Dawson wrote: > On 13 November 2015 at 02:48, Sandro Santilli wrote: > > To be honest, I like the idea of rendering tests. I know they are > > fragile in their current incarnation, but being a graphical > > application having tests actually check what the user would see > > is a good way to catch many issues. > > I personally think we've reached a good level of stability with the > rendering tests. They now perform consistently across all platforms > (passing when they should and failing when they shouldn't). So I'm > keen to hear any thoughts on why you think these are still fragile? CDash shows many failures here (check 2.8.3-* builds): http://dash.orfeo-toolbox.org/index.php?project=QGIS All of them are rendering-based tests. I'm sure the recent introduction of masks helps with that, but it is evidently not easy enough to use. --strk; ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
On 13 November 2015 at 02:48, Sandro Santilli wrote: > On Thu, Nov 12, 2015 at 04:42:07PM +0100, Matthias Kuhn wrote: >> Simple answer: don't use rendering tests and find other means to check >> it ;-) > > How could I check that an arror is not flipped and does not move > to the other side of the line ? > > To be honest, I like the idea of rendering tests. I know they are > fragile in their current incarnation, but being a graphical > application having tests actually check what the user would see > is a good way to catch many issues. I personally think we've reached a good level of stability with the rendering tests. They now perform consistently across all platforms (passing when they should and failing when they shouldn't). So I'm keen to hear any thoughts on why you think these are still fragile? Nyall > >> If you still need, use the MultiRenderChecker, it is based on the >> RenderChecker but adds more possibilities for anomalies. At least that's >> the latest state of rendering tests I'm aware of... > > Ok, great. How about the distinction between these two files ? > > * testqgsstylev2.cpp > * testqgssymbolv2.cpp > > Actually, the second one is _not_ in the 2.8 branch (which I'm > targetting). Should it be back-ported ? > > --strk; > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
On 13 November 2015 at 02:42, Matthias Kuhn wrote: > Simple answer: don't use rendering tests and find other means to check > it ;-) Hmm... for something like this (testing rendering of a line style), I think a rendering test is perfectly acceptable! It's only when rendering tests are used for things like checking geometry operations that it's an issue IMO. Nyall > > If you still need, use the MultiRenderChecker, it is based on the > RenderChecker but adds more possibilities for anomalies. At least that's > the latest state of rendering tests I'm aware of... > > Best > > On 11/12/2015 04:37 PM, Sandro Santilli wrote: >> I've found a bug in rendering line-offsetted marker line symbols [1] >> and wondering what's the currently recommended way to add a testcase. >> >> [1] http://hub.qgis.org/issues/13811 or >> https://github.com/qgis/QGIS/pull/2442 >> >> Looking around I see many test files using the QgsRenderChecker >> class in tests/src/core but the rationale is not clear to me. >> >> Here's the list: >> >>regression992.cpp >>testqgsblendmodes.cpp >>testqgscomposerobject.cpp >>testqgscomposerutils.cpp >>testqgsdiagram.cpp >>testqgsgeometry.cpp >>testqgsgradients.cpp >>testqgsimageoperation.cpp >>testqgsinvertedpolygonrenderer.cpp >>testqgslabelingenginev2.cpp >>testqgslegendrenderer.cpp >>testqgslinefillsymbol.cpp >>testqgsmaprenderer.cpp >>testqgsmaprotation.cpp >>testqgspainteffect.cpp >>testqgspointpatternfillsymbol.cpp >>testqgsrasterfill.cpp >>testqgsrasterlayer.cpp >> * testqgsrenderers.cpp >>testqgsshapeburst.cpp >> >> Then other files use the QgsMultiRenderChecker class instead: >> >>qgscompositionchecker.cpp >>testqgsblendmodes.cpp >>testqgsdiagram.cpp >>testqgsinvertedpolygonrenderer.cpp >>testqgspainteffect.cpp >>testqgsrasterfill.cpp >>testqgsrenderers.cpp >>testqgsshapeburst.cpp >> * testqgsstylev2.cpp >> * testqgssymbolv2.cpp >> >> What's the difference between the two classes ? Which one should I use ? >> Should I put the new case in any existing file ? >> (I've tagged the possible candidates with a star) >> >> --strk; >> >> () Free GIS & Flash consultant/developer >> /\ http://strk.keybit.net/services.html >> ___ >> Qgis-developer mailing list >> Qgis-developer@lists.osgeo.org >> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer >> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer > > -- > Matthias Kuhn > OPENGIS.ch - https://www.opengis.ch > Spatial • (Q)GIS • PostGIS • Open Source > > > > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
On Thu, Nov 12, 2015 at 05:01:21PM +0100, Matthias Kuhn wrote: > On 11/12/2015 04:48 PM, Sandro Santilli wrote: > > > > To be honest, I like the idea of rendering tests. I know they are > > fragile in their current incarnation, but being a graphical > > application having tests actually check what the user would see > > is a good way to catch many issues. > > They won't go away. And there are good reasons for them sometimes. They > just have been used as roundhousekick tests for everything in the past. > But if every developer shortly asks himself if > * there is no other possibility to test a particular thing > * tries to reduce the noise in the picture as much as possible >it's fine to have some of them. I was actually thinking it would be nice to make producing such tests easier, by allowing to just drop a .qgs project file in a dir and have that trigger rendering on a small output image to compare against a reference image. Right now it takes sooo much time to write such tests --strk; ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
On 11/12/2015 04:48 PM, Sandro Santilli wrote: > On Thu, Nov 12, 2015 at 04:42:07PM +0100, Matthias Kuhn wrote: >> Simple answer: don't use rendering tests and find other means to check >> it ;-) > How could I check that an arror is not flipped and does not move > to the other side of the line ? No idea, didn't check ;) > > To be honest, I like the idea of rendering tests. I know they are > fragile in their current incarnation, but being a graphical > application having tests actually check what the user would see > is a good way to catch many issues. They won't go away. And there are good reasons for them sometimes. They just have been used as roundhousekick tests for everything in the past. But if every developer shortly asks himself if * there is no other possibility to test a particular thing * tries to reduce the noise in the picture as much as possible it's fine to have some of them. > >> If you still need, use the MultiRenderChecker, it is based on the >> RenderChecker but adds more possibilities for anomalies. At least that's >> the latest state of rendering tests I'm aware of... > Ok, great. How about the distinction between these two files ? > > * testqgsstylev2.cpp > * testqgssymbolv2.cpp > > Actually, the second one is _not_ in the 2.8 branch (which I'm > targetting). Should it be back-ported ? Backporting tests should always be fine. They don't break the application but guarantee quality for subsequent point releases. Best -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
On Thu, Nov 12, 2015 at 04:42:07PM +0100, Matthias Kuhn wrote: > Simple answer: don't use rendering tests and find other means to check > it ;-) How could I check that an arror is not flipped and does not move to the other side of the line ? To be honest, I like the idea of rendering tests. I know they are fragile in their current incarnation, but being a graphical application having tests actually check what the user would see is a good way to catch many issues. > If you still need, use the MultiRenderChecker, it is based on the > RenderChecker but adds more possibilities for anomalies. At least that's > the latest state of rendering tests I'm aware of... Ok, great. How about the distinction between these two files ? * testqgsstylev2.cpp * testqgssymbolv2.cpp Actually, the second one is _not_ in the 2.8 branch (which I'm targetting). Should it be back-ported ? --strk; ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] rendering tests, what's the latest approach ?
Simple answer: don't use rendering tests and find other means to check it ;-) If you still need, use the MultiRenderChecker, it is based on the RenderChecker but adds more possibilities for anomalies. At least that's the latest state of rendering tests I'm aware of... Best On 11/12/2015 04:37 PM, Sandro Santilli wrote: > I've found a bug in rendering line-offsetted marker line symbols [1] > and wondering what's the currently recommended way to add a testcase. > > [1] http://hub.qgis.org/issues/13811 or > https://github.com/qgis/QGIS/pull/2442 > > Looking around I see many test files using the QgsRenderChecker > class in tests/src/core but the rationale is not clear to me. > > Here's the list: > >regression992.cpp >testqgsblendmodes.cpp >testqgscomposerobject.cpp >testqgscomposerutils.cpp >testqgsdiagram.cpp >testqgsgeometry.cpp >testqgsgradients.cpp >testqgsimageoperation.cpp >testqgsinvertedpolygonrenderer.cpp >testqgslabelingenginev2.cpp >testqgslegendrenderer.cpp >testqgslinefillsymbol.cpp >testqgsmaprenderer.cpp >testqgsmaprotation.cpp >testqgspainteffect.cpp >testqgspointpatternfillsymbol.cpp >testqgsrasterfill.cpp >testqgsrasterlayer.cpp > * testqgsrenderers.cpp >testqgsshapeburst.cpp > > Then other files use the QgsMultiRenderChecker class instead: > >qgscompositionchecker.cpp >testqgsblendmodes.cpp >testqgsdiagram.cpp >testqgsinvertedpolygonrenderer.cpp >testqgspainteffect.cpp >testqgsrasterfill.cpp >testqgsrenderers.cpp >testqgsshapeburst.cpp > * testqgsstylev2.cpp > * testqgssymbolv2.cpp > > What's the difference between the two classes ? Which one should I use ? > Should I put the new case in any existing file ? > (I've tagged the possible candidates with a star) > > --strk; > > () Free GIS & Flash consultant/developer > /\ http://strk.keybit.net/services.html > ___ > Qgis-developer mailing list > Qgis-developer@lists.osgeo.org > List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer > Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Matthias Kuhn OPENGIS.ch - https://www.opengis.ch Spatial • (Q)GIS • PostGIS • Open Source signature.asc Description: OpenPGP digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] rendering tests, what's the latest approach ?
I've found a bug in rendering line-offsetted marker line symbols [1] and wondering what's the currently recommended way to add a testcase. [1] http://hub.qgis.org/issues/13811 or https://github.com/qgis/QGIS/pull/2442 Looking around I see many test files using the QgsRenderChecker class in tests/src/core but the rationale is not clear to me. Here's the list: regression992.cpp testqgsblendmodes.cpp testqgscomposerobject.cpp testqgscomposerutils.cpp testqgsdiagram.cpp testqgsgeometry.cpp testqgsgradients.cpp testqgsimageoperation.cpp testqgsinvertedpolygonrenderer.cpp testqgslabelingenginev2.cpp testqgslegendrenderer.cpp testqgslinefillsymbol.cpp testqgsmaprenderer.cpp testqgsmaprotation.cpp testqgspainteffect.cpp testqgspointpatternfillsymbol.cpp testqgsrasterfill.cpp testqgsrasterlayer.cpp * testqgsrenderers.cpp testqgsshapeburst.cpp Then other files use the QgsMultiRenderChecker class instead: qgscompositionchecker.cpp testqgsblendmodes.cpp testqgsdiagram.cpp testqgsinvertedpolygonrenderer.cpp testqgspainteffect.cpp testqgsrasterfill.cpp testqgsrenderers.cpp testqgsshapeburst.cpp * testqgsstylev2.cpp * testqgssymbolv2.cpp What's the difference between the two classes ? Which one should I use ? Should I put the new case in any existing file ? (I've tagged the possible candidates with a star) --strk; () Free GIS & Flash consultant/developer /\ http://strk.keybit.net/services.html ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
