On a regular basis, Gna! people pick and interview one vivid and mature project to highlight the software and personality that drive Libre Software community, similarly to SF.net project of the month -- yes, we assume the fact that we grab good ideas from others places.
The second hotspot is dedicated to the Test-Autobuild project. Test-Autobuild Description: --------------------------- Test-Autobuild check out pristine sources of a software from a version control repository, invoked a script to build the software and install it in a virtual root. Snapshots of the virtual root are taken before and after build to identify which files were installed. Snapshots of any designated 'package' directories are also taken to identify any RPM, Debian PKG, Tar, ZIP files which were built. Finally a set of output modules are run to generate HTML status page, copy packages and build logs to a Web / FTP server, send email notifications of build status, create ISO images. Test-Autobuild was registered on Gna! on March 2004 and is leaded by Daniel Berrange and Dennis Gregorovic. An Interview with Daniel Berrange and Dennis Gregorovic: --------------------------------------------------------- | Gna!: Hello, | Can you give a two line description of Test-Autobuild your grandma(s) could | understand? What is the intended audience of Test-Autobuild? Dan: Test-AutoBuild is a tool for automatically running build and test processes, without need for user interaction. It checks code out of version control, invokes the build & test processes on each project and then creates status pages, send email alerts and uploads build logs & packages to FTP/HTTP download sites. The target audience for use of Test-AutoBuild is any project with more than one developer working on the codebase, so pretty much any non-trivial project! The value it brings to the development process is to increase awareness of the state of the codebase. At any given time you should be able to know with confidence that a code tree is compiling and passing all available test cases. When a developer checks in a piece of code that breaks the build, there is a direct impact on the productivity of other team members. At the simplest level they may be unable to compile their code after pulling down updates from version control. More seriously, unexpected changes in the runtime behavior of the program can result in many hours wasted debugging time. By automatically running the build processes continuously (24x7), problems are detected and dealt with as soon as possible, minimizing the disruption for developers. Even greater benefit is seen when unit test cases are incorporated in the build cycle, and parallel Test-AutoBuild instances are setup across different platforms (GNU/Linux, Solaris, etc), since its not feasible to expect developers to test their code submissions across all supported configurations. Dennis: Test-AutoBuild can even be useful for a project with just one developer. As Dan points out, with multiple Test-AutoBuild instances, the developer can avoid the laborious task of building and testing the project code across different platforms and configurations. Additionally, with Test-AutoBuild you can set up "clean-room" where the builds and test runs will performed consistently in a controlled environment. This is often more reliable than using the same environment where you do your development. | Gna!: Do you propose/maintain a collection of recipes (in the Gentoo | or uOS style) to "Test-Autobuild" some popular packages? Dan: When creating the initial content for our GNA webpages (now aliased from www.autobuild.org), I started on a tutorial illustrating how to create control files for use with common build tools. So far I've illustrated use of AutoMake/AutoConf in the C/C++ world, ExtUtils::MakeMaker in the Perl world, and ANT in the Java world. Java is in an interesting situation because while ANT is the de facto standard low level build tool, there is yet to arrive any clear favourite higher level language similar to AutoMake. Although Maven is starting to become more widespread, there are a lot of projects which don't want to be constrained by its particular view of how Java should be built. Thus I think there is still a gap in the Java toolspace for a flexible AutoMake style layer around ANT. | Gna!: What is the difference with other popular unattended build packages, | like eg. Tinderbox? Dan: Tinderbox has a quite a different design to Test-AutoBuild, with a client-server architecture. The client component performs the build of a module and sends the results across to the server when complete. Multiple clients can be setup to build the software across many platforms passing results to a single server. As such its probably more comparable to a combination of Test-AutoBuild with the Bicycle aggregation service. The client component, however, does not have the same level of functionality as Test-AutoBuild, with no direct support for caching builds state of multiple related projects. My preference is for separate pieces of infrastructure each performing a simple, well defined task, since this makes it easier to integrate with existing / legacy software without having to replace an entire system. The most frequently quoted competitor to Test-AutoBuild is CruiseControl. At an implementation level, the latter is written in Java, while Test-AutoBuild is Perl - I'll leave it up to readers to decide which is preferable ;-) Much of the build functionality is very similar, both being able to checkout from a variety of source control repositories, both can send email alerts, generate status pages, make build logs available, etc. The differentiating factor is that Test-AutoBuild has direct support for managing multiple projects in a single build instance, while CruiseControl pushes this knowledge down to the underlying build tool such as ANT. The advantage for Test-AutoBuild here is that with this knowledge it can do intelligent build caching of individual modules which haven't changed, provide incremental alerts and status pages and grouped package downloads. Other competitors include LuntBuild (luntbuild.sourceforge.net) and AntHill or their proprietary version AntHill pro. I consider AntHill Pro to be the biggest competitor in terms of functionality, but then it is closed source and with the work we're currently doing the gap will be much less significant. More detaileds comparisons against our competitors are being gradually put up on ourwebsite. | Gna!: Aren't you overlapping some distro packaging tools | functionality (Eg. RedHat and Debian also use procedures to package | from "pristine sources" and make automation easy; including chroots, | installed files discovery, autodeps, etc)? Dan: I consider Test-AutoBuild to be an important value-add on top of the packaging tools used by the distributions such as Red Hat and Debian. When building an RPM of their software, a developer still has the manual task of checking the code out of version control, running whatever scripts are need to create the source tar.gz distribution, and then invoking rpmbuild. When building multiple related RPMs, it quickly becomes a large burden on the developer to repeat these steps over and over, not to mention the care required to ensure you are building against the correct pre-requisite RPMs. When building RPMs for an entire GNU/Linux distribution comprising many hundreds, or even thousands of RPMs, automation on-top of the basic packaging tools is absolutely essential. This is where a tool such as Test-AutoBuild comes into its own. During day-to-day work, the developer only has to concern themselves with writing code & checking it into version control. A short while later, after the next build cycle has completed, binary packages of their software will be uploaded to the build status pages for anyone to access. A fundamental rule in writing Test-AutoBuild is that we should not make any assumptions about how a project's development process may operate. So we have to be operate with any combination of programming language, build tool, packaging format, source control repository within a single Test-AutoBuild instance. So while RPM provides functionality to detect installed files, we have to cater for projects who do not use RPM or an equivalent. This is a particularly common situation in Java projects most of which simply distribute JARs, or development undertaken by companies for internal use only, for which a custom deployment solution has been written. That said, one of the ideas for enhancing Test-AutoBuild still further is to enable automatic extraction build metadata when already defined in the packaging tool files. For example, rather than have the administrator list build dependencies in the Test-AutoBuild configuration file, it could be determined from 'BuildRequires' in the RPM spec file. | Gna!: Who are you? How many developers contribute to Test-Autobuild | regularly? | How the project is leaded? Dan: I'm currently working for Red Hat, in London, where I'm a member of the Global Professional Services team. Prior to that I was in their Content Management team (CCM), as the technical lead on the APLAWS+ project. Naturally during the course of my work, I've introduced use of Test-AutoBuild to nearly all the development teams I've been involved with, since very few have any build automation processes. At this point in time nearly all development is undertaken by Dennis and myself. One of our prime motivations for setting up a public presence on the GNA service was to solicit contributions and feedback from developers in the Open Source community, as well as spread use of Test-AutoBuild beyond Red Hat and our own projects. The leadership of project development is very informal, we try and start off important design changes with discussion via email (& to a less extent on IRC). Dennis is doing much of the big work on the development branch, while I'm doing low risk changes to the stable branch, along with maintaining the website. Dennis: I'm in Boston, also working for Red Hat, doing release management for Red Hat Enterprise Linux. Previously, I was doing release and build management in the CCM engineering group. I'm a developer at heart and focus most of my energies on tools that are SCM-related. I really enjoy doing Test-AutoBuild development because as it gets better, my life gets easier. | Gna!: When was started the project and why? The first version, which was a single Perl script, was written by Richard Jones (www.annexia.org) back in the year 2000 while we were both working at BiblioTech Ltd (www.bibliotech.net). We had a large team of up to 18 developers working on numerous projects for our online community sites, so build and test automation was key to ensuring code quality was maintained. Richard setup the control files to build the project, run unit tests, and package it all in RPMs. On the development servers, Rich had setup a related script that would periodically grab the latest RPMs from the automated build website & install them - a task typically performed by APT/YUM these days. So not only was it providing us with a build, test and packaging system, it enabled integration of related tools to lower administrative overhead of keeping all the servers uptodate with the latest RPMs. | Gna!: How things have gone for the project since then? | What are your plans? How do you see the future of Test-Autobuild? Dan: Test-AutoBuild has gone through several iterations since Richard's original Perl script. When I left BiblioTech to work at ArsDigita, I started the process of gradually refactoring the Perl script to allow more general usage. The original version only dealt with CVS and had a fixed set of tasks it ran at the end of each build cycle. So I introduced module APIs for obtaining code checkouts from source control repositories, and for performing post-build output tasks, such as sending email alerts, generating HTML status pages, copying packages and build logs to a web directory, etc. I introduced use of this refactored version to the project I was working on at the time & after a while our Engineering team decided it would be useful for their needs too. In the meantime we were acquired by Red Hat, who were more than happy for us to develop AutoBuild during course of our day-to-day work. After working on Test-AutoBuild on & off for a couple of years in Red Hat, we decided it was time to try and engage people in community, hence our launch on GNA earlier this year. The current work that's happening on the development branch of Test-AutoBuild will be a very significant milestone, providing a completely customizable build work-flow engine. The current stable release will always perform the build cycle in three stages, first checkout all the code from source control, then build each project, then run the output modules to generate HTML pages, send alerts, etc. With the new customizable work-flow engine it'll be possible to customize this process in any way whatsoever. One example benefit is that it will be possible to generate incremental HTML pages after each project has been built, rather than only at the end of the complete cycle. When building a large number of projects, this will reduce the time between code check-in and build feedback still further. It will also be possible synchronize multiple Test-AutoBuild instances, for example, one would perform the basic build and unit tests. The second instance, instead of checking out code from version control, could download RPMs generated by the first RPM & perform integration tests on them. This is taking Test-AutoBuild into the realm of being a general purpose development process automation framework, rather than being targeted puring to the build stage. Another very interesting development is the work Red Hat are doing on the Bicycle project. This is provides a aggregation service for continuous integration tools such as Test-AutoBuild, CruiseControl and BugZilla. Amongst other things it will display a historical view of the build status. It will also be a valuable add-on for situations in which multiple Test-AutoBuild instances are run against the same code, but across different hardware architectures and OS platforms. Dennis: Over the last few years we have seen an impressive growth in the functionality, maturity, and number of open source version control systems, such as Subversion and GNU Arch. I expect that in the next few years we will see a similar growth in the area of open source process automation tools, and Test-AutoBuild will be right there among them. The items that Dan mentions for the future of Test-AutoBuild are exiting, but I think they may just be the tip of the iceberg. | Gna!: Which license did you choose for Test-Autobuild and why? Dan: The original version of Richard's autobuild script was released under the GNU GPL. Since I chose to incrementally refactor this script, rather than re-write from scratch, the decision to cover Test-AutoBuild under GNU GPL was made for me. | Gna!: Can you mention some real world use cases of Test-Autobuild? Dan: I have an instance of Test-AutoBuild setup to perform nightly builds of all my random, open source projects, so anyone who may come across one can always get the latest code I have. It is also a useful sanity check in case I forget to check one or more files into CVS, since I'll have an email alert reminding me the next morning. Within the CCM group at Red Hat, our engineering team had an instance of Test-AutoBuild building all the applications they maintained. Any member of the services team, which is spread out across London, Munich and Boston could easily obtain builds of the latest development branch and all supported stable branches, without need to direct communication with the engineers. We also have a nightly process which pushed the latest builds out to the public ftp.redhat.com server. Dennis has also been introducing use of Test-AutoBuild into some of the development groups within Red Hat. I'm also in process of setting up a continuous builder for a project I'm involved in which has to build, test and integrate a mixture of Java, Perl and C++ code. Dennis: In addition to the Test-AutoBuild instances in the CCM group that Dan mentions, we also had an instance for building documentation and another for release builds. I point this out because it shows the versatility of Test-AutoBuild. In the documentation instance, we produced HTML, PDF, and Postscript output from SGML source; the HTML output was packaged up, but it was also copied to a directory served by Apache. This way, the latest CCM documentation was always available online. The release build instance of Test-AutoBuild was on a separate machine with limited access to ensure a consistent build environment. This instance was also configured accordingly. For example, it never used cached builds, and it would pull from Perforce using a specific tag or changelist number, instead of just grabbing the latest code. | Gna!: Do you have any industrial or institutional support? | If any, how so? Dan: Both myself and Dennis are fortunate enough to be employed at Red Hat who are more than happy for us to further develop Test-AutoBuild and related tools such as Bicycle. Situations arising within Red Hat have also provided motivation for developing new capabilities in the Test-AutoBuild framework, adding support for Perforce source control system, caching of builds to speed up cycles when no changes have been made in source control. | Gna!: Are you looking for contributions? | If so, what kind of contributions could be of use to the project? Dan: Our prime motivation for making Test-AutoBuild available on the GNA project service was to try and encourage involvement of other open source developers. We've got a fair few ideas floating around that we so far haven't had time to work on, so we're open to offers from anyone interested in writing some Perl. One significant development task I'd like to see some work on at some point is a web based UI for configuring and managing the builder instance. This would lower the learning curve for users trying to setup a build instance for their projects, as well as addressing the most significant feature gap to the proprietary AntHill Pro tool. Such a configuration tool, wouldn't even necessarily need to be written in Perl, or directly integrated in the main codebase, rather it could evolve independently as a layer on-top of the basic Test-AutoBuild code. Dennis: The feature that I most look forward to is coordination between separate Test-AutoBuild instances. For example, you could have one instance just building and packaging the software, and other instances grabbing those packages and testing them. Also, with the archiving and generic work-flow work being done on the development branch, the web UI is going to need improvements. So, to answer your question, yes, we're looking for contributions and help. :) | Gna!: What tools do you use when working on Test-Autobuild? Why? Dan: It goes without saying that we have an instance of Test-AutoBuild building itself (demo.autobuild.org). Personally I don't find a need for fancy integrated development environments such as Eclipse, so do nearly all my work in Emacs, or very rarely Vi. Dennis: Emacs, and occasionally the Perl profiler and debugger | Gna!: Why did you choose Gna! as host? Dan: I took a look at most of the open source hosting services available, and really liked the speed of the GNA website, and the clarity of its layout. I also think its important not to concentrate too many projects with a single service, since its creates a large single point of failure. That pretty much ruled out SourceForge. At the time I was looking the GNU Savannah site was also still not accepting new project registrations. | Gna!: Anything else? Dan: Something I observe time and again when getting involved in new projects is the lack of good quality development infrastructure. Build and test automation, just one small part of the solution, is typically missing altogether. The need for a good version control system goes without saying & the open source community is spoiled for choice with the de facto standard of CVS, and now the advanced capabilities offered by Subversion and GNU Arch. In contrast in the closed source world, companies are still paying large sums of money for vastly inferior & unreliable tools, which are killing productivity of their developers. The tools used for performing formal builds also play an important role. While many developers will find an IDE great for day-to-day development work, they should not be used at the expense of a solid build tool like Make or ANT. There is no way an IDE can match these when it comes to providing 100% reliable and repeatable release builds. | Gna!: Are there links of interest that should come along this interview? Our website is the first port of call, www.autobuild.org. Since we're written in Perl, the formal releases are all uploaded to CPAN, which generates HTML API documentation, so for interested developers this is worth a visit (http://search.cpan.org/~danberr/Test-AutoBuild-1.0.2/) Also worth keeping an eye on in the future is Bicycle, which although still in an alpha phase of development, is very functional. Dennis: Dan mentioned the Bicycle project several times. It also came out of the CCM group at Red Hat at can be found at the Gna! site. It's a bit young, but well designed and full of potential. Finally, for those who like to communicate over IRC, I can usually be found in #autobuild on the Freenode network. Links: ------ http://www.autobuild.org https://gna.org/projects/testautobuild http://search.cpan.org/~danberr/Test-AutoBuild-1.0.2/ https://gna.org/projects/bicycle _______________________________________________ Message sent via/by Gna! http://gna.org/
