Hi everybody, I spent some time reading the previous discussions around the concept of "CI" and from what I gather, it seems that we didn't reach an agreement about how to proceed and who is working on it (but I might be wrong, in case apologies!). From what I can see, there are two possible working fronts:
1) Simplify the usage of the current perl testing suite, adding docs/tests/etc.. Some people expressed the desire for a more friendly framework, especially when adding new tests. 2) Run the testing framework automatically on different environments to spot anomalies/bugs/etc.. in a timely manner. I am a bit ignorant about how to run a proper CI but I created a little prototype of a Dockerfile able to bootstrap a testing environment on Debian 10 (Buster) and run the perl test suite: https://github.com/elukey/httpd_integration_testing/blob/master/docker/Dockerfile The above is only an example, it is missing a lot of things and some follow up work is needed. But with the following commands, on my laptop I was able to create a docker image and run the test suite: docker build . docker run $id-of-the-image make check Some thoughts: 1) The above Dockerfile is really handy since I can easily switch between Debian versions (Jessie/Stretch/Buster to name the last three) and run the test suite with different package versions (openssl, nghttp2, etc..). It should be also easy to create Dockerfiles for other OS/environments, and run make check in a similar way. 1-bis) Testing on Windows would still need to be solved, Docker probably it is not the right solution but we could find something else to integrate for this specific use case. 2) Docker also offers a way to open a bash shell in interactive mode, so it could be easy to run tests on a certain platform when somebody reports a problem. Or make sure that a new set of tests runs correctly everywhere. 3) Another use case could be to create a Dockerfile that pulls a specific new release of httpd, installing it and running the test suite on multiple platforms. 4) The same Docker image could also run tests suites like https://github.com/icing/mod_h2/tree/master/test/e2e (that is really nice, I suggest to check it if you haven't done it) to run HTTP/2 tests as well. 5) We could even think about having daily docker image builds that take a snapshot of trunk/2.4.x and run the test suites, reporting back any problem. Does the above make sense or it is completely out of scope for our project? In the former case I could spend some time on figuring out how to create what I proposed above, otherwise I'll stop and drop the idea :) Last but not the least, this would be a way to help us testing changes in a more automated way, not a replacement of community based testing and bug reports (stating the obvious for an Apache project but better safe than sorry :). Luca
