I was thinking that our test infrastructure could use an upgrade to pytest.
Some advantages: - It'd allow setting the test suite name. For example, if you look at this page <https://builds.apache.org/job/beam_PreCommit_Python_Commit/7043/testReport/apache_beam.io.fileio_test/MatchTest/> you'll find 3 sets of 4 identically named tests with no way to tell which tox environment they were run on (all marked as "nosetests"). - It will hopefully allow a degree of parallelism (if we can solve some pickling errors). This will make running unit tests locally much faster. - pytest has cleaner progress reporting - no more BeamTestPlugin - easier inclusion/exclusion of tests (using markers such as: precommit, postcommit, no_direct, no_dataflow, etc.) On Tue, Jun 25, 2019 at 10:50 AM Udi Meiri <eh...@google.com> wrote: > Yes. It only outputs to one filename though, so it'd need some working > around (our ITs might have more than one nose run). > Some tests run in docker, so that might need work to get the xml out. > > On Tue, Jun 25, 2019 at 10:11 AM Ahmet Altay <al...@google.com> wrote: > >> There is a nose plugin [1] for outputting test results in xunit format. >> Would that work? >> >> [1] https://nose.readthedocs.io/en/latest/plugins/xunit.html >> >> On Tue, Jun 25, 2019 at 10:04 AM Udi Meiri <eh...@google.com> wrote: >> >>> The current state of Python post-commit tests is pretty flaky. >>> I was wondering if we had any stats for integration tests, to help >>> identify which tests are causing the most failures. Jenkins keeps some >>> history for tests (example >>> <https://builds.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/testReport/apache_beam.coders.avro_coder_test/CodersTest/test_avro_record_coder/history/>), >>> but it requires junit-style .xml output. >>> >>> Would it be possible to get our integration test results into Jenkins? >>> >>
smime.p7s
Description: S/MIME Cryptographic Signature
