[Qgis-developer] Testing infrastructure (was: QGIS 2 64bits, is it stable ?)
On Fri, Oct 04, 2013 at 04:19:38PM +0200, Paolo Cavallini wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Il 04/10/2013 14:33, Régis Haubourg ha scritto: > > > What about starting the infrastructure for unit test and commit validation > > and testing it now? > > AFAIK the infrastructure is already in place and running. No reference ? Any documentation for developers about how to create and run tests ? All I could find on the website is this interesting page: http://hub.qgis.org/projects/quantum-gis/wiki/Developers_Manual#Unit-Testing "As of November 2007 we require all new features going into trunk to be accompanied with a unit test" Is the document still valid ? Any page reporting test run status ? I think these are all part of an infrastructure (among other aspects) > > I must say I could be very interested in a commercial support for this, if > > this is sticking to community version and not a separate branch. > > If you set up a list of requirements, I'm sure there will be people willing to > undertake the task. Yes please ! But probably also coming up with a "list of requirements" would be part of such project, as it will require finding an agreement among developers to cope with policies that will need to be defined (I guess). What about funding the new QA manager for working on such a plan ? --strk; PS: "make check" output against * master 6d5cf6f : 82% tests passed, 14 tests failed out of 78 Total Test time (real) = 471.88 sec The following tests FAILED: 4 - qgis_diagramtest (Failed) 14 - qgis_rendererstest (Failed) 25 - qgis_composermaptest (Failed) 30 - qgis_stylev2test (Failed) 31 - qgis_composerhtmltest (Failed) 33 - qgis_composerscalebartest (OTHER_FAULT) 45 - qgis_wcsprovidertest (Failed) 51 - PyQgsVectorLayer (Failed) 52 - PyQgsRasterLayer (Failed) 56 - PyQgsDelimitedTextProvider (Failed) 64 - PyQgsComposerMap (Failed) 72 - PyQgsPalLabelingBase (Failed) 73 - PyQgsPalLabelingCanvas (Failed) 78 - PyQgsAppStartup (Failed) ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Testing infrastructure (was: QGIS 2 64bits, is it stable ?)
Hi Sandro, On Fri, 04. Oct 2013 at 17:50:05 +0200, Sandro Santilli wrote: > On Fri, Oct 04, 2013 at 04:19:38PM +0200, Paolo Cavallini wrote: > > AFAIK the infrastructure is already in place and running. > > No reference ? The nightly builds run tests. And AFAIK http://jenkins.linfiniti.com/job/QGIS/ builds QGIS and runs tests after each commit. > Any page reporting test run status? In jenkins and for the nightlies on: http://dash.orfeo-toolbox.org/index.php?project=QGIS We might move that to our new server at some point. > All I could find on the website is this interesting page: > http://hub.qgis.org/projects/quantum-gis/wiki/Developers_Manual#Unit-Testing > > "As of November 2007 we require all new features going into trunk to be > accompanied with a unit test" > > Is the document still valid ? Never heard of that - or successfully repressed it. > Any page reporting test run status ? s.a. > PS: "make check" output against * master 6d5cf6f : > 82% tests passed, 14 tests failed out of 78 > Total Test time (real) = 471.88 sec > The following tests FAILED: > 4 - qgis_diagramtest (Failed) > 14 - qgis_rendererstest (Failed) > 25 - qgis_composermaptest (Failed) > 30 - qgis_stylev2test (Failed) > 31 - qgis_composerhtmltest (Failed) > 33 - qgis_composerscalebartest (OTHER_FAULT) > 45 - qgis_wcsprovidertest (Failed) > 51 - PyQgsVectorLayer (Failed) > 52 - PyQgsRasterLayer (Failed) > 56 - PyQgsDelimitedTextProvider (Failed) > 64 - PyQgsComposerMap (Failed) > 72 - PyQgsPalLabelingBase (Failed) > 73 - PyQgsPalLabelingCanvas (Failed) > 78 - PyQgsAppStartup (Failed) And most of these tests are not really freshly failing (see dash above). Jürgen -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS PSC member (RM) IRC: jef on FreeNode -- norBIT Gesellschaft fuer Unternehmensberatung und Informationssysteme mbH Rheinstrasse 13, 26506 Norden GF: Jelto Buurman, HR: Amtsgericht Emden, HRB 5502 ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] Testing infrastructure (was: QGIS 2 64bits, is it stable ?)
To get back to the first question (stability of QGIS 2 64bits/ Win), my feelings are that a lot of bugs had been corrected before the release announcement, but package phase did not go to the same level of control and showed many regression. For instance, French IGN projections bug in master was a GRASS one, and that bug came back with the released version (but that does'nt seem to be Grass's fault this time). Old or removed packages had been inserted in the installer in the first version. And so on. My self-compiled-from-release QGIS brick is strong. Other bricks are separately strong too. But the wall made of those bricks isn't. Hien TRAN-QUANG > > > -- Message transféré -- > From: Sandro Santilli > To: qgis-developer@lists.osgeo.org > Cc: > Date: Fri, 4 Oct 2013 17:50:05 +0200 > Subject: [Qgis-developer] Testing infrastructure (was: QGIS 2 64bits, is it > stable ?) > On Fri, Oct 04, 2013 at 04:19:38PM +0200, Paolo Cavallini wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> Il 04/10/2013 14:33, Régis Haubourg ha scritto: >> >> > What about starting the infrastructure for unit test and commit validation >> > and testing it now? >> >> AFAIK the infrastructure is already in place and running. > > No reference ? > > Any documentation for developers about how to create and run tests ? > > All I could find on the website is this interesting page: > http://hub.qgis.org/projects/quantum-gis/wiki/Developers_Manual#Unit-Testing > > "As of November 2007 we require all new features going into trunk to be > accompanied with a unit test" > > Is the document still valid ? > > Any page reporting test run status ? > > I think these are all part of an infrastructure (among other aspects) > >> > I must say I could be very interested in a commercial support for this, if >> > this is sticking to community version and not a separate branch. >> >> If you set up a list of requirements, I'm sure there will be people willing >> to >> undertake the task. > > Yes please ! > > But probably also coming up with a "list of requirements" would be part of > such project, as it will require finding an agreement among developers to > cope with policies that will need to be defined (I guess). > > What about funding the new QA manager for working on such a plan ? > > --strk; > > PS: "make check" output against * master 6d5cf6f : > 82% tests passed, 14 tests failed out of 78 > Total Test time (real) = 471.88 sec > The following tests FAILED: > 4 - qgis_diagramtest (Failed) > 14 - qgis_rendererstest (Failed) > 25 - qgis_composermaptest (Failed) > 30 - qgis_stylev2test (Failed) > 31 - qgis_composerhtmltest (Failed) > 33 - qgis_composerscalebartest (OTHER_FAULT) > 45 - qgis_wcsprovidertest (Failed) > 51 - PyQgsVectorLayer (Failed) > 52 - PyQgsRasterLayer (Failed) > 56 - PyQgsDelimitedTextProvider (Failed) > 64 - PyQgsComposerMap (Failed) > 72 - PyQgsPalLabelingBase (Failed) > 73 - PyQgsPalLabelingCanvas (Failed) > 78 - PyQgsAppStartup (Failed) > > > > > > -- Message transféré -- > From: "Jürgen E." Fischer > To: qgis-developer@lists.osgeo.org > Cc: > Date: Fri, 4 Oct 2013 20:03:44 +0200 > Subject: Re: [Qgis-developer] Testing infrastructure (was: QGIS 2 64bits, is > it stable ?) > Hi Sandro, > > On Fri, 04. Oct 2013 at 17:50:05 +0200, Sandro Santilli wrote: >> On Fri, Oct 04, 2013 at 04:19:38PM +0200, Paolo Cavallini wrote: >> > AFAIK the infrastructure is already in place and running. >> >> No reference ? > > The nightly builds run tests. And AFAIK > http://jenkins.linfiniti.com/job/QGIS/ > builds QGIS and runs tests after each commit. > >> Any page reporting test run status? > > In jenkins and for the nightlies on: > > http://dash.orfeo-toolbox.org/index.php?project=QGIS > > We might move that to our new server at some point. > >> All I could find on the website is this interesting page: >> http://hub.qgis.org/projects/quantum-gis/wiki/Developers_Manual#Unit-Testing >> >> "As of November 2007 we require all new features going into trunk to be >> accompanied with a unit test" >> >> Is the document still valid ? > > Never heard of that - or successfully repressed it. > >> Any page reporting test run status ? > > s.a. > >> PS: "make check" output against * master 6d5cf6f : >> 82% tests passed, 14 tests failed out of