Hi Seems we are hitting some problems in CI. I propose the following action items to remedy the situation and accelerate turn around times in CI, reduce cost, complexity and probability of failure blocking PRs and frustrating developers:
* Upgrade Windows visual studio from VS 2015 to VS 2017. The build_windows.py infrastructure should easily work with the new version. Currently some PRs are blocked by this: https://github.com/apache/incubator-mxnet/issues/13958 * Move Gluon Model zoo tests to nightly. Tracked at https://github.com/apache/incubator-mxnet/issues/15295 * Move non-python bindings tests to nightly. If a commit is touching other bindings, the reviewer should ask for a full run which can be done locally, use the label bot to trigger a full CI build, or defer to nightly. * Provide a couple of basic sanity performance tests on small models that are run on CI and can be echoed by the label bot as a comment for PRs. * Address unit tests that take more than 10-20s, streamline them or move them to nightly if it can't be done. * Open sourcing the remaining CI infrastructure scripts so the community can contribute. I think our goal should be turnaround under 30min. I would also like to touch base with the community that some PRs are not being followed up by committers asking for changes. For example this PR is importtant and is hanging for a long time. https://github.com/apache/incubator-mxnet/pull/15051 This is another, less important but more trivial to review: https://github.com/apache/incubator-mxnet/pull/14940 I think comitters requesting changes and not folllowing up in reasonable time is not healthy for the project. I suggest configuring github Notifications for a good SNR and following up. Regards. Pedro.
