Follow up after some work in https://github.com/elukey/httpd_integration_testing:
* Use only one Docker image for ubuntu/debian * Created two use cases: code snapshot testing (latest trunk and 2.4.x) vs release candidate testing * Still some issues in installing perl deps on older Debian distro (like Stretch), but the perl suite seems to run fine in both use cases for Debian Buster. * Some issues while running the HTTP/2 test suite, will follow up with Stefan. * I'll try to add a Docker image for CentOS. Please open pull requests if you have ideas/comments/suggestions/etc.. :) Thanks! Luca Il giorno mer 25 set 2019 alle ore 11:52 Luca Toscano <toscano.l...@gmail.com> ha scritto: > > Hi everybody, > > I spent some time reading the previous discussions around the concept > of "CI" and from what I gather, it seems that we didn't reach an > agreement about how to proceed and who is working on it (but I might > be wrong, in case apologies!). From what I can see, there are two > possible working fronts: > > 1) Simplify the usage of the current perl testing suite, adding > docs/tests/etc.. Some people expressed the desire for a more friendly > framework, especially when adding new tests. > 2) Run the testing framework automatically on different environments > to spot anomalies/bugs/etc.. in a timely manner. > > I am a bit ignorant about how to run a proper CI but I created a > little prototype of a Dockerfile able to bootstrap a testing > environment on Debian 10 (Buster) and run the perl test suite: > > https://github.com/elukey/httpd_integration_testing/blob/master/docker/Dockerfile > > The above is only an example, it is missing a lot of things and some > follow up work is needed. But with the following commands, on my > laptop I was able to create a docker image and run the test suite: > > docker build . > docker run $id-of-the-image make check > > Some thoughts: > > 1) The above Dockerfile is really handy since I can easily switch > between Debian versions (Jessie/Stretch/Buster to name the last three) > and run the test suite with different package versions (openssl, > nghttp2, etc..). It should be also easy to create Dockerfiles for > other OS/environments, and run make check in a similar way. > 1-bis) Testing on Windows would still need to be solved, Docker > probably it is not the right solution but we could find something else > to integrate for this specific use case. > 2) Docker also offers a way to open a bash shell in interactive mode, > so it could be easy to run tests on a certain platform when somebody > reports a problem. Or make sure that a new set of tests runs correctly > everywhere. > 3) Another use case could be to create a Dockerfile that pulls a > specific new release of httpd, installing it and running the test > suite on multiple platforms. > 4) The same Docker image could also run tests suites like > https://github.com/icing/mod_h2/tree/master/test/e2e (that is really > nice, I suggest to check it if you haven't done it) to run HTTP/2 > tests as well. > 5) We could even think about having daily docker image builds that > take a snapshot of trunk/2.4.x and run the test suites, reporting back > any problem. > > Does the above make sense or it is completely out of scope for our > project? In the former case I could spend some time on figuring out > how to create what I proposed above, otherwise I'll stop and drop the > idea :) > > Last but not the least, this would be a way to help us testing changes > in a more automated way, not a replacement of community based testing > and bug reports (stating the obvious for an Apache project but better > safe than sorry :). > > Luca
