Every time we run a test cycle on our Jenkins cluster, we generate hundreds
of XML reports covering all the tests we have (e.g.
`streaming/target/test-reports/org.apache.spark.streaming.util.WriteAheadLogSuite.xml`).

These reports contain interesting information about whether tests succeeded
or failed, and how long they took to complete. There is also detailed
information about the environment they ran in.

It might be valuable to have a window into all these reports across all
Jenkins builds and across all time, and use that to track basic statistics
about our tests. That could give us basic insight into what tests are flaky
or slow, and perhaps drive other improvements to our testing infrastructure
that we can't see just yet.

Do people think that would be valuable? Do we already have something like
this?

I'm thinking for starters it might be cool if we automatically uploaded all
the XML test reports from the Master and the Pull Request builders to an S3
bucket and just opened it up for the dev community to analyze.

Nick

Reply via email to