Hi Bill! Thanks a lot for your answer. Quick remarks. > IMHO more suites are always a benefit, not a competition between test tools.
Completely agree with you > that it will be a challenge to bring the resource to bear to replicate what is being done right now I see. I am not going to replace your current framework by mine, sure. But what I can suggest. There are some gaps I may be covered by swat. Let me say a couple: 1) consider INTEGRATION TESTING under various platforms and environments ( debian, ubuntu, centos, gentoo ... ). Chef folks use tool called test-kitchen which is simple way to get desired OS and install something there and then run some tests against installed product. They use this extensively in chef cookbook development. Indeed the same workflow could be used for testing apache. Consider this: - install apache ( via vendored package ) on specific platform (centos, ubuntu, fedora, suse slackware - whatever ) - and the run some (smoke? sanity ) integration tests so to discovery environment specific gray zones / issues with apache - which of course might arise ! Having a extensive experiences in devops,chef, configuration management I may give you a good help in writing such tests (using swat) and automating tests environments ( using chef, test kitchen, travis, vagrant ) 2) Cover unresolved issues or test cases ( listed at "Tests to be written:" http://svn.apache.org/repos/asf/httpd/test/framework/trunk/STATUS ) using swat. Sometimes there is no much time to write a tests, is it ? (: Swat test development in _most cases_ take a minimal amount of time. What if I try to "prove" it? ((: What about writing tests from "Tests to be written:" section ( with some moderate help of apache gurus on apache specific details if need ): * t/apache - simulations of network failures (incomplete POST bodies, chunked and unchunked; missing POST bodies; slooow client connexions, such as taking 1 minute to send 1KiB; ...) * t/modules/autoindex - something seems possibly broken with inheritance on 2.0 * t/ssl - SSLPassPhraseDialog exec: - SSLRandomSeed exec: That's it. Please follow up. Thanks 2016-01-28 5:33 GMT+03:00 William A Rowe Jr <[email protected]>: > On Wed, Jan 27, 2016 at 2:53 PM, Alexey Melezhik <[email protected]> wrote: >> >> Hi! I am the author of swat - https://github.com/melezhik/swat - >> perl/curl based DSL for web services test automation. >> I'd like to contribute in at apache server automated testing. Please >> take a look at _simple_ example of swat test suite for apache web >> server - https://sparrowhub.org/info/swat-apache?v=0.000004 >> >> Some swat features ( more info could be found at >> https://github.com/melezhik/swat ) which possibly could be of >> interested for you: >> >> * swat is DSL for rapid test scenarios development for various web >> services and application >> * swat is written on Perl and could be extended by Perl if you need >> something more complicated when DSL is not enough >> * swat has minimal dependencies ( mostly core perl modules plus few >> CPAN ones ) and easy to install >> * swat relies on perl prove as internal test runner and so yield test >> reports in portable TAP format >> * swat relies on curl to make http requests against tested >> application, so there is almost no limit for you - sending, saving >> cookies, complicated http headers, make html form uploads, etc >> * swat allow both simple (smoke) tests development and complicated >> integration tests - with sequential, coupled http requests, saving >> intermediate state, code reuse, etc. This is classical program API for >> complex things combined with intuitive simple DSL ( for simple and >> medium things ) >> >> * not only httpd server but other a Apache Foundation products could >> be tested with swat ( tomcat, ... ) - take a look at >> https://sparrowhub.org/ - central swat test suite repository to get >> more examples. >> >> I'd be glad to get feedback from you guys. >> >> Feel free to ask any further questions. >> >> Alexey Melezhik - the author of swat and sparrow. > > > Thanks for the pointers and your efforts. IMHO more suites are > always a benefit, not a competition between test tools. > > For a sense of what we actively review today, please do take a quick > look at http://svn.apache.org/repos/asf/httpd/test/framework/trunk/ > to understand the scope of the current framework. > > I personally don't care precisely which tool or framework I use, as long > as it is comprehensive. I'm looking forward to looking at what you have > put together, but wanted you to be aware that it will be a challenge to > bring the resource to bear to replicate what is being done right now > under our existing framework. > > Props and cheers, > > Bill >
