Re: [Clamav-users] Thoughts on software QA Testing (or lack thereof...)
On 2009-11-05 18:27, George R. Kasica wrote: >> On Mon, 2 Nov 2009 16:32:57 + (GMT), you wrote: >> > > >> Hi there, >> >> On Thu, 29 Oct 2009 aCaB wrote: >> >> >>> On our side we do a lot of QA >>> ... >>> I really believe something needs to happen here so that these type of >>> bugs can be caught quickly before they affect a number of users. >>> >>> Thoughts? >>> >> I suspect that rather than QA, what you do is just a lot of hap-hazard >> testing. That's why, whenever I see a new release of ClamAV, first I >> will suppress a groan and then, before I risk it on any of my servers, >> I'll wait a while and watch the users' list to see how much trouble it >> causes. This approach serves me well, although I can't say I'm proud >> of the fact that I'm letting a lot of poor innocents do my acceptance >> testing for me. >> > > OK...I'm probably going to PO a few folks here but you asked for > opinions so I'll share one > > We run about 50 Unix/Linux servers here...almost all are 32-bit RHEL4 > not an unknown distro by any means and what I'd suspect is easy to > test on. Several more are Solaris 9, again a well known OS, and > several more are Fedora Core 10, again a well known common OS build. > Hi, We test ClamAV on several architectures and OSes: Debian (etch, lenny, unstable) on various architectures (x86_64, powerpc, sparc64, armv5tel, ...), RHEL5.2 (x86-64), Solaris 10 (sparc), and Mac OS X (ppc). Some of these tests are running on the GCC compile farm, some on our own boxes, some on machines we have been kindly offered access for testing. This of course can always be improved. One of the build problems was an error when 'git' was not installed. Unfortunately all of the machines we test ClamAV on have git in $PATH (it is how it checks out the latest version). For the next release we'll setup a buildhost that doesn't have git installed. The other problem (missing include unistd.h) only occurs on old distributions, which we lack in our buildfarm. We're currently investigating the possibility of using the OpenSUSE build service to test the next ClamAV release on multiple Linux distributions, including many old ones: openSUSE 11.x, SLES/SLED 9/10/11, Fedora 10/11, RHEL 4/5, CentOS 5, Mandriva 2009, xUbuntu 6.06/8.04/8.10/9.04 > This latest release had basic make issues in ALL THREE OS versions > that should have been caught by simply typing make as it wouldn't even > finish the make step without various patchesin my mind that > doesn't qualify it as even ALPHA software much less BETA or a Release > Candidate and surely not a Production Release version...please don't > embarrass yourselves by saying you do alot of QAthat obviously > didn't happen in this release and has been a problem in the last few > releases > We do more than just build tests for QA, but as you've realized the build tests for old(er) OSes are currently insufficient. > I expect better from a commercial company like SourceFire now that > they own the product as a commercial concern...its the same codebase > for free or paid users as far as I know, the differences is paying > users get to call for support. > There are also certified binary packages available. Best regards, --Edwin ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [Clamav-users] Thoughts on software QA Testing (or lack thereof...)
>We're currently investigating the possibility of using the OpenSUSE >build service to test the next ClamAV release on multiple Linux >distributions, including many old ones: >openSUSE 11.x, SLES/SLED 9/10/11, Fedora 10/11, RHEL 4/5, CentOS 5, >Mandriva 2009, xUbuntu 6.06/8.04/8.10/9.04 OK...I'm not going to debate "old" vs. "new" here but I'm fairly sure the installed base of Fedora Core 10 and RHEL4 and Solaris 9 which are all actively supported by the various Vendors/groups would disagree with your assessment as would alot of businesses that are running them in a day to day production setting for front line work. Frankly, Don't think I would be able to get a "new" OS such as FC-11 or RHEL5 OK 'd to go into production at our company due to lack of sufficient background from a security standpoint etc. In any case, if you're looking for a test spot for FC10, Solaris 9, RHEL4 I'd be happy to try to run some stuff here on a box - I'm not a programmer but I can do basic things if given clear steps or test the ability to at least get it to make etc in our QA/Test environment. Let me know. George ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [Clamav-users] Thoughts on software QA Testing (or lack thereof...)
George R. Kasica wrote: > In any case, if you're looking for a test spot for FC10, Solaris 9, > RHEL4 I'd be happy to try to run some stuff here on a box - I'm not a > programmer but I can do basic things if given clear steps or test the > ability to at least get it to make etc in our QA/Test environment. Hi George, That would be cool! There are basically two options. The least intrusive is a small shell script to be run daily or so from cron which posts resuts available here: http://farm.0xacab.net/ This only requires git, a compatible compiler and an ftp client. The other one is to run a buildbot slave. Results are available at http://www.0xacab.net:8010/waterfall If you want to help with either, please mail Edwin or me off list. Thanks, -acab ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
Re: [Clamav-users] Thoughts on software QA Testing (or lack thereof...)
2009/11/6 Török Edwin : > One of the build problems was an error when 'git' was not installed. > Unfortunately all of the machines we test ClamAV on have git in $PATH > (it is how it checks out the latest version). I usually do software compilation/packaging on specially-prepared chroot environment, which I can tune according to that software's special needs. For most packages it's simply a new directory created with "yum --installroot=... install ..." Fedora has Mock, which does the similar thing. > For the next release we'll setup a buildhost that doesn't have git > installed. > The other problem (missing include unistd.h) only occurs on old > distributions, which we lack in our buildfarm. > > We're currently investigating the possibility of using the OpenSUSE > build service to test the next ClamAV release on multiple Linux > distributions, including many old ones: > openSUSE 11.x, SLES/SLED 9/10/11, Fedora 10/11, RHEL 4/5, CentOS 5, > Mandriva 2009, xUbuntu 6.06/8.04/8.10/9.04 My build host is actually just a Xen domU running RHEL4 64bit, with chroot environments for RHEL4 and 5 32 and 64bit (which is what we use). It saves the number of host and resources needed for testing. For (open)Solaris, you could probably make use of branded zones which can give you a Linux 2.4 (e.g. Centos3), Solaris 8, Solaris 9, and Solaris 10/Opensolaris build environment. -- Fajar ___ Help us build a comprehensive ClamAV guide: visit http://wiki.clamav.net http://www.clamav.net/support/ml
