Re: [Qgis-developer] QGIS and Coverity scans
Hi Jürgen, Le 11/02/2015 14:18, Jürgen E. Fischer a écrit : Hi Hugo, On Wed, 11. Feb 2015 at 14:04:56 +0100, Hugo Mercier wrote: What would be very good is to call static analyzers during the build/testing process (as an option). Has somebody experiences with such things ? CLang static analyzer ? We have used cppcheck - but not integrated into the build process. There are also some good options for gcc (eg. -Weffc++). Good to know. A CMake option with such compilation flags can be a first pass. Using another compiler (Clang) may also help to exhibit different wwarning messages. I didn't know clang's scan-build[1] before Even Rouault mentioned it on #qgis. Maybe we could use that with travis. But it's not packaged in ubuntu and apparently you have to build clang to get it. It's shipped with the clang-3.4 package on ubuntu (14.04 here). ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
Hi Hugo, On Wed, 11. Feb 2015 at 17:18:03 +0100, Hugo Mercier wrote: We have used cppcheck - but not integrated into the build process. There are also some good options for gcc (eg. -Weffc++). Good to know. A CMake option with such compilation flags can be a first pass. Using another compiler (Clang) may also help to exhibit different wwarning messages. Sure, also MSVC has shown other warnings. Yes, another proprietary tool we are currently free to use. I didn't know clang's scan-build[1] before Even Rouault mentioned it on #qgis. Maybe we could use that with travis. But it's not packaged in ubuntu and apparently you have to build clang to get it. It's shipped with the clang-3.4 package on ubuntu (14.04 here). Oh, nice. I just apt-cache searched for scan-build. Didn't check if I already have it. :) Jürgen -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS release manager (PSC) GermanyIRC: jef on FreeNode signature.asc Description: Digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
On 12 February 2015 at 08:45, Jürgen E. j...@norbit.de wrote: I didn't know clang's scan-build[1] before Even Rouault mentioned it on #qgis. Maybe we could use that with travis. But it's not packaged in ubuntu and apparently you have to build clang to get it. It's shipped with the clang-3.4 package on ubuntu (14.04 here). Oh, nice. I just apt-cache searched for scan-build. Didn't check if I already have it. :) I just tried a scan-build over master and nothing was flagged. I'll try with a more recent clang release just in case, but anyway, it's good to have more tools in the arsenal! Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
On 12 February 2015 at 00:18, Jürgen E. j...@norbit.de wrote: Hi Hugo, On Wed, 11. Feb 2015 at 14:04:56 +0100, Hugo Mercier wrote: I am curious about Coverity. AFAIK, the static analyzer is not open source, right ? So this works as long as they offer if freely for open source projects. Right. It's propietary. And IMHO it's also a heavy depencency (IIRC the download of the tool needed to collect the build logs was 150MB). What would be very good is to call static analyzers during the build/testing process (as an option). Has somebody experiences with such things ? CLang static analyzer ? We have used cppcheck - but not integrated into the build process. There are also some good options for gcc (eg. -Weffc++). I also don't see the non-open source nature of this tool as an issue. I think with things like this the more analysers we can throw at the code the better. So cppcheck, scan-build etc should all be used in parallel to Coverity. Coverity has the positive that it's seen as a highly respected, industry-leading tool. So being able to show that it reports our code as clean (or nearly clean) is a strong argument to QGIS' code quality. One thing I forgot to mention initially: if anyone's working on addressing/checking issues on Coverity, please make sure to use the triage status tools and flag issues as Fix Submitted/False Positive so that we don't duplicate effort. Nyall ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
[Qgis-developer] QGIS and Coverity scans
Hi all, If you've been following recent git commits, you'll have noticed a lot of Coverity related commits, and may be wondering what these are all about. Coverity Scan ( https://scan.coverity.com/ ) is a powerful automated static code analyser which is able to detect a large number of code errors, such as memory leaks and potential crashes, and even things like accidental copy/paste errors. It's a well respected service and fortunately offers free testing of open source projects (it's quite expensive for commercial software). When we first ran Coverity over the QGIS codebase about 2 weeks ago it picked up just over 1000 potential issues, with a defect density of about 1 error per 1000 lines of code. Apparently the standard for good software is a defect density of 1. For comparison, python sits at 0.08, and the Linux kernel at 0.53. Libreoffice's latest release hit 0.02, and they used this as a big highlight of their press release [1]. Thanks to Jürgen and Martin's assistance we're now down to a defect density of 0.26. I'm hoping that with a bit more work we can smash this down even further and possibly even reach the coveted Coverity Clean status [2] for 2.8. In any case this is a great demonstration that we are serious about code quality and stable releases, and is a good selling point for our first LTS release (alongside the expanding test suite and Travis CI testing). Unfortunately we can't automate submission to Coverity via Travis builds due to the compilation time required to build QGIS using Coverity exceeding Travis' limits, so I'm currently manually submitting builds to Coverity on a semi-regular schedule. The full Coverity defect reports are available by invitation only. If you're a developer and want to view them, let me know and I'll add you to the group. Nyall [1] http://blog.documentfoundation.org/2015/01/29/libreoffice-4-4-the-most-beautiful-libreoffice-ever/ [2] Why is this important? well... I really want to beat MapInfo there! http://www.pb.com/pbs-voc/product-improvements.shtml ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
That's a great thing Nyall. It proves QGIS is an industry level product from the side of code quality! Thanks from me and Gis3W. giovanni 2015-02-11 12:29 GMT+01:00 Nathan Woodrow madman...@gmail.com: Thanks to Jürgen and Martin's assistance And yours of course. This is some great stabilization to the product. - Natahn On Wed Feb 11 2015 at 9:13:32 PM Nyall Dawson nyall.daw...@gmail.com wrote: Hi all, If you've been following recent git commits, you'll have noticed a lot of Coverity related commits, and may be wondering what these are all about. Coverity Scan ( https://scan.coverity.com/ ) is a powerful automated static code analyser which is able to detect a large number of code errors, such as memory leaks and potential crashes, and even things like accidental copy/paste errors. It's a well respected service and fortunately offers free testing of open source projects (it's quite expensive for commercial software). When we first ran Coverity over the QGIS codebase about 2 weeks ago it picked up just over 1000 potential issues, with a defect density of about 1 error per 1000 lines of code. Apparently the standard for good software is a defect density of 1. For comparison, python sits at 0.08, and the Linux kernel at 0.53. Libreoffice's latest release hit 0.02, and they used this as a big highlight of their press release [1]. Thanks to Jürgen and Martin's assistance we're now down to a defect density of 0.26. I'm hoping that with a bit more work we can smash this down even further and possibly even reach the coveted Coverity Clean status [2] for 2.8. In any case this is a great demonstration that we are serious about code quality and stable releases, and is a good selling point for our first LTS release (alongside the expanding test suite and Travis CI testing). Unfortunately we can't automate submission to Coverity via Travis builds due to the compilation time required to build QGIS using Coverity exceeding Travis' limits, so I'm currently manually submitting builds to Coverity on a semi-regular schedule. The full Coverity defect reports are available by invitation only. If you're a developer and want to view them, let me know and I'll add you to the group. Nyall [1] http://blog.documentfoundation.org/2015/01/29/libreoffice-4-4-the- most-beautiful-libreoffice-ever/ [2] Why is this important? well... I really want to beat MapInfo there! http://www.pb.com/pbs-voc/product-improvements.shtml ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Giovanni Allegri http://about.me/giovanniallegri Twitter: https://twitter.com/_giohappy_ blog: http://blog.spaziogis.it GEO+ geomatica in Italia http://bit.ly/GEOplus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
On 11-02-15 12:29, Nathan Woodrow wrote: Thanks to Jürgen and Martin's assistance And yours of course. This is some great stabilization to the product. Yes, thanks Nyall !! Also for communicating this to us. What about (/me not pushing, I'm a very slow writer myself...) writing a small article on your blog (with a score screenie), so the 'general public' knows about it too? Another option (not talking to Nyall anymore now ;-) ) would be to write up about our other 'quality'-thingies we do (tests, automatic testing etc etc) somewhere on our website. Regards, Richard Duivenvoorde ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
Thanks to Jürgen and Martin's assistance And yours of course. This is some great stabilization to the product. - Natahn On Wed Feb 11 2015 at 9:13:32 PM Nyall Dawson nyall.daw...@gmail.com wrote: Hi all, If you've been following recent git commits, you'll have noticed a lot of Coverity related commits, and may be wondering what these are all about. Coverity Scan ( https://scan.coverity.com/ ) is a powerful automated static code analyser which is able to detect a large number of code errors, such as memory leaks and potential crashes, and even things like accidental copy/paste errors. It's a well respected service and fortunately offers free testing of open source projects (it's quite expensive for commercial software). When we first ran Coverity over the QGIS codebase about 2 weeks ago it picked up just over 1000 potential issues, with a defect density of about 1 error per 1000 lines of code. Apparently the standard for good software is a defect density of 1. For comparison, python sits at 0.08, and the Linux kernel at 0.53. Libreoffice's latest release hit 0.02, and they used this as a big highlight of their press release [1]. Thanks to Jürgen and Martin's assistance we're now down to a defect density of 0.26. I'm hoping that with a bit more work we can smash this down even further and possibly even reach the coveted Coverity Clean status [2] for 2.8. In any case this is a great demonstration that we are serious about code quality and stable releases, and is a good selling point for our first LTS release (alongside the expanding test suite and Travis CI testing). Unfortunately we can't automate submission to Coverity via Travis builds due to the compilation time required to build QGIS using Coverity exceeding Travis' limits, so I'm currently manually submitting builds to Coverity on a semi-regular schedule. The full Coverity defect reports are available by invitation only. If you're a developer and want to view them, let me know and I'll add you to the group. Nyall [1] http://blog.documentfoundation.org/2015/01/29/libreoffice-4-4-the- most-beautiful-libreoffice-ever/ [2] Why is this important? well... I really want to beat MapInfo there! http://www.pb.com/pbs-voc/product-improvements.shtml ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
Congratulations to everyone. Code quality is something that matters on the long run. Since QGIS only gets better, I envision a big marathon :P. Anyone who develops software and understands what these metrics mean know they are significant, so congratulations again. George On Wed, Feb 11, 2015 at 9:29 AM, Nathan Woodrow madman...@gmail.com wrote: Thanks to Jürgen and Martin's assistance And yours of course. This is some great stabilization to the product. - Natahn On Wed Feb 11 2015 at 9:13:32 PM Nyall Dawson nyall.daw...@gmail.com wrote: Hi all, If you've been following recent git commits, you'll have noticed a lot of Coverity related commits, and may be wondering what these are all about. Coverity Scan ( https://scan.coverity.com/ ) is a powerful automated static code analyser which is able to detect a large number of code errors, such as memory leaks and potential crashes, and even things like accidental copy/paste errors. It's a well respected service and fortunately offers free testing of open source projects (it's quite expensive for commercial software). When we first ran Coverity over the QGIS codebase about 2 weeks ago it picked up just over 1000 potential issues, with a defect density of about 1 error per 1000 lines of code. Apparently the standard for good software is a defect density of 1. For comparison, python sits at 0.08, and the Linux kernel at 0.53. Libreoffice's latest release hit 0.02, and they used this as a big highlight of their press release [1]. Thanks to Jürgen and Martin's assistance we're now down to a defect density of 0.26. I'm hoping that with a bit more work we can smash this down even further and possibly even reach the coveted Coverity Clean status [2] for 2.8. In any case this is a great demonstration that we are serious about code quality and stable releases, and is a good selling point for our first LTS release (alongside the expanding test suite and Travis CI testing). Unfortunately we can't automate submission to Coverity via Travis builds due to the compilation time required to build QGIS using Coverity exceeding Travis' limits, so I'm currently manually submitting builds to Coverity on a semi-regular schedule. The full Coverity defect reports are available by invitation only. If you're a developer and want to view them, let me know and I'll add you to the group. Nyall [1] http://blog.documentfoundation.org/2015/01/29/libreoffice-4-4-the- most-beautiful-libreoffice-ever/ [2] Why is this important? well... I really want to beat MapInfo there! http://www.pb.com/pbs-voc/product-improvements.shtml ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- George R. C. Silva SIGMA Consultoria http://www.consultoriasigma.com.br/ ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
Hi, These are great news, thanks ! I am curious about Coverity. AFAIK, the static analyzer is not open source, right ? So this works as long as they offer if freely for open source projects. What would be very good is to call static analyzers during the build /testing process (as an option). Has somebody experiences with such things ? CLang static analyzer ? Valgrind (not static) ? It would be nice to have these run before each pull request. What would be also interesting after this coverity pass, is to have a QGIS developer guideline that show frequent mistakes to avoid (beware of classes with states, use smart pointers, enforce constness, etc.) Le 11/02/2015 12:47, G. Allegri a écrit : That's a great thing Nyall. It proves QGIS is an industry level product from the side of code quality! Thanks from me and Gis3W. giovanni 2015-02-11 12:29 GMT+01:00 Nathan Woodrow madman...@gmail.com mailto:madman...@gmail.com: Thanks to Jürgen and Martin's assistance And yours of course. This is some great stabilization to the product. - Natahn On Wed Feb 11 2015 at 9:13:32 PM Nyall Dawson nyall.daw...@gmail.com mailto:nyall.daw...@gmail.com wrote: Hi all, If you've been following recent git commits, you'll have noticed a lot of Coverity related commits, and may be wondering what these are all about. Coverity Scan ( https://scan.coverity.com/ ) is a powerful automated static code analyser which is able to detect a large number of code errors, such as memory leaks and potential crashes, and even things like accidental copy/paste errors. It's a well respected service and fortunately offers free testing of open source projects (it's quite expensive for commercial software). When we first ran Coverity over the QGIS codebase about 2 weeks ago it picked up just over 1000 potential issues, with a defect density of about 1 error per 1000 lines of code. Apparently the standard for good software is a defect density of 1. For comparison, python sits at 0.08, and the Linux kernel at 0.53. Libreoffice's latest release hit 0.02, and they used this as a big highlight of their press release [1]. Thanks to Jürgen and Martin's assistance we're now down to a defect density of 0.26. I'm hoping that with a bit more work we can smash this down even further and possibly even reach the coveted Coverity Clean status [2] for 2.8. In any case this is a great demonstration that we are serious about code quality and stable releases, and is a good selling point for our first LTS release (alongside the expanding test suite and Travis CI testing). Unfortunately we can't automate submission to Coverity via Travis builds due to the compilation time required to build QGIS using Coverity exceeding Travis' limits, so I'm currently manually submitting builds to Coverity on a semi-regular schedule. The full Coverity defect reports are available by invitation only. If you're a developer and want to view them, let me know and I'll add you to the group. Nyall [1] http://blog.__documentfoundation.org/2015/__01/29/libreoffice-4-4-the-__most-beautiful-libreoffice-__ever/ http://blog.documentfoundation.org/2015/01/29/libreoffice-4-4-the-most-beautiful-libreoffice-ever/ [2] Why is this important? well... I really want to beat MapInfo there! http://www.pb.com/pbs-voc/__product-improvements.shtml http://www.pb.com/pbs-voc/product-improvements.shtml _ Qgis-developer mailing list Qgis-developer@lists.osgeo.org mailto:Qgis-developer@lists.osgeo.org http://lists.osgeo.org/__mailman/listinfo/qgis-__developer http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org mailto:Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer -- Giovanni Allegri http://about.me/giovanniallegri Twitter: https://twitter.com/_giohappy_ blog: http://blog.spaziogis.it GEO+ geomatica in Italia http://bit.ly/GEOplus ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
Hi Hugo, On Wed, 11. Feb 2015 at 14:04:56 +0100, Hugo Mercier wrote: I am curious about Coverity. AFAIK, the static analyzer is not open source, right ? So this works as long as they offer if freely for open source projects. Right. It's propietary. And IMHO it's also a heavy depencency (IIRC the download of the tool needed to collect the build logs was 150MB). What would be very good is to call static analyzers during the build/testing process (as an option). Has somebody experiences with such things ? CLang static analyzer ? We have used cppcheck - but not integrated into the build process. There are also some good options for gcc (eg. -Weffc++). I didn't know clang's scan-build[1] before Even Rouault mentioned it on #qgis. Maybe we could use that with travis. But it's not packaged in ubuntu and apparently you have to build clang to get it. Jürgen [1] http://clang-analyzer.llvm.org/ -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS release manager (PSC) GermanyIRC: jef on FreeNode signature.asc Description: Digital signature ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
I am curious about Coverity. AFAIK, the static analyzer is not open source, right ? So this works as long as they offer if freely for open source projects. Right. It's propietary. And IMHO it's also a heavy depencency Not sure I find that to be much of a issue. Use what ever works, it seems to be free for us at the moment and is improving our code a lot, if there is a open version that works as well that is fine but it's definitely not something that I personally worry about. I also wouldn't really call it a dependency when we are not forced or forcing anyone else to use it. - Nathan On Wed Feb 11 2015 at 11:19:09 PM Jürgen E. j...@norbit.de wrote: Hi Hugo, On Wed, 11. Feb 2015 at 14:04:56 +0100, Hugo Mercier wrote: I am curious about Coverity. AFAIK, the static analyzer is not open source, right ? So this works as long as they offer if freely for open source projects. Right. It's propietary. And IMHO it's also a heavy depencency (IIRC the download of the tool needed to collect the build logs was 150MB). What would be very good is to call static analyzers during the build/testing process (as an option). Has somebody experiences with such things ? CLang static analyzer ? We have used cppcheck - but not integrated into the build process. There are also some good options for gcc (eg. -Weffc++). I didn't know clang's scan-build[1] before Even Rouault mentioned it on #qgis. Maybe we could use that with travis. But it's not packaged in ubuntu and apparently you have to build clang to get it. Jürgen [1] http://clang-analyzer.llvm.org/ -- Jürgen E. Fischer norBIT GmbH Tel. +49-4931-918175-31 Dipl.-Inf. (FH) Rheinstraße 13 Fax. +49-4931-918175-50 Software Engineer D-26506 Norden http://www.norbit.de QGIS release manager (PSC) GermanyIRC: jef on FreeNode ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
Re: [Qgis-developer] QGIS and Coverity scans
Le 11/02/2015 14:24, Nathan Woodrow a écrit : I am curious about Coverity. AFAIK, the static analyzer is not open source, right ? So this works as long as they offer if freely for open source projects. Right. It's propietary. And IMHO it's also a heavy depencency Not sure I find that to be much of a issue. Use what ever works, it seems to be free for us at the moment and is improving our code a lot, if there is a open version that works as well that is fine but it's definitely not something that I personally worry about. I also wouldn't really call it a dependency when we are not forced or forcing anyone else to use it. Sure, on the short term, this is not an issue. It really valuable. And this is not really a dependency since we are not forcing anyone. I agree. But the day the service stops to be free, we should still deliver code with a high quality :) Either because we have a free alternative to static checkers or if every devs know how to avoid code with potential problems (or if we decide to pay for the service). ___ Qgis-developer mailing list Qgis-developer@lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/qgis-developer
