+1 It’s a virtual standard. Lots of tools support in Jenkins.
On 1/25/19, 10:03 AM, "Gray, Jonathan" <[email protected]> wrote:
+1 on using a standard test output format for tests
Integrating those reports into each job will have to be done by one of the
privileged few who can modify the jobs since we havn't yet converted to
Jenkins2 Pipeline jobs due to our dependency on the Github-PR-Builder Jenkins
plugin. As such, I'd recommend we also keep around our classical text file
format. Come find me on slack if you want more details on status/progress here.
Also, I'd like to see us standardize that when performing tests we use exit
code != 0 to mean the tests couldn't be run/completed as opposed to test
failure. With the adoption of JUnit, that will unlock a new Jenkins build
status "UNSTABLE" automatically if the tests fail yet the exit codes are 0.
This means that you have better visibility into test infrastructure failures
versus test failures and allows you to choose if some failure is ok. I believe
currently the Github-PR-Builder plugin will treat UNSTABLE the same as failure
automatically.
Jonathan G
On 1/25/19, 9:41 AM, "Jeremy Mitchell" <[email protected]> wrote:
works for me
On Thu, Jan 24, 2019 at 4:13 PM Fieck, Brennan
<[email protected]>
wrote:
> I'm +1. If nothing else, it doesn't add any dependencies to actually
> *running* ATC or CIAB,
> and JUnit files are technically human-readable (though I certainly
> wouldn't love doing that
> often)>
> ________________________________________
> From: Robert Butts <[email protected]>
> Sent: Thursday, January 24, 2019 2:35 PM
> To: [email protected]
> Subject: [EXTERNAL] CiaB Test Compose Outputting JUnit?
>
> Does anyone object to making the cdn-in-a-box test compose files (e.g.
>
>
https://github.com/apache/trafficcontrol/blob/master/infrastructure/cdn-in-a-box/docker-compose.traffic-ops-test.yml
> )
> output JUnit format?
>
> I made an internal fork to do that for
> `docker-compose.traffic-ops-test.yml` as well as complete
> Dockerfiles+Compose for the TM, Portal, and Router tests. At the
time, I
> didn't OSS them, because I thought we shouldn't impose JUnit on TC.
>
> But in retrospect, it limits our contributions to TC for no good
reason.
> Further, TC uses the Apache Jenkins server, which expects JUnit. So
I'm not
> seeing a reason not to standardize all of our docker-compose test
> infrastructure around JUnit.
>
> Note I'm only talking about the `docker-compose` infrastructure to run
> tests, not the tests themselves. The tests themselves inside each
project
> won't change -- the Router already outputs JUnit, TO and TM output Go
test
> format, etc.
>
> Does anyone object to that? Are we ok with adding `docker-compose`
> infrastructure for running TO API, TO Unit, Monitor, Router, and
Portal
> tests outputting JUnit?
>
