Also please take note that there's a stage building TVM which is executing compilation serially and takes a lot of time which impacts CI turnaround time:
https://github.com/apache/incubator-mxnet/issues/16962 Pedro On Tue, Dec 3, 2019 at 9:49 AM Pedro Larroy <pedro.larroy.li...@gmail.com> wrote: > Hi MXNet community. We are in the process of updating the base AMIs for CI > with an updated CUDA driver to fix the CI blockage. > > We would need help from the community to diagnose some of the build errors > which don't seem related to the infrastructure. > > I have observed this build failure with tvm when not installing the cuda > driver in the container: > > > https://pastebin.com/bQA0W2U4 > > centos gpu builds and tests seem to run with the updated AMI and changes > to the container. > > > Thanks. > > > On Mon, Dec 2, 2019 at 12:11 PM Pedro Larroy <pedro.larroy.li...@gmail.com> > wrote: > >> Small update about CI, which is blocked. >> >> Seems there's a nvidia driver compatibility problem in the base AMI that >> is running in GPU instances and the nvidia docker images that we use for >> building and testing. >> >> We are working on providing a fix by updating the base images as doesn't >> seem to be easy to fix by just changing the container. >> >> Thanks. >> >> Pedro. >> >
