Re: CI impaired

2018-11-30 Thread Marco de Abreu
Hello, I'm now moving forward with #1. I will try to get to #3 as soon as possible to reduce parallel jobs in our CI. You might notice some unfinished jobs. I will let you know as soon as this process has been completed. Until then, please bare with me since we have hundreds of jobs to run in orde

Re: CI impaired

2018-11-30 Thread Naveen Swamy
There are still pending PRs pending that needs to be merged and cherry picked to the branch > On Nov 30, 2018, at 6:53 AM, Marco de Abreu > wrote: > > Hello, > > I'm now moving forward with #1. I will try to get to #3 as soon as possible > to reduce parallel jobs in our CI. You might notice s

Re: CI impaired

2018-11-30 Thread Naveen Swamy
Marco, run your experiments on a branch - set up, test it well and then bring it to the master. > On Nov 30, 2018, at 6:53 AM, Marco de Abreu > wrote: > > Hello, > > I'm now moving forward with #1. I will try to get to #3 as soon as possible > to reduce parallel jobs in our CI. You might not

Re: CI impaired

2018-11-30 Thread Marco de Abreu
Hello Naveen, this is not an experiment. Everything has been tested in our test system and is considered working 100%. This is not a test but actually the move into production - the merge into master happened a week ago. We now just have to put all PRs into the catalogue, which means that all PRs h

Re: CI impaired

2018-11-30 Thread Gavin M. Bell
Hey Folks, Marco has been running this change in dev, with flying colors, for some time. This is not an experiment but a roll out that was announced. We also decided to make this change post the release cut so limit the blast radius from any critical obligations to the community. Marco is accoun

Re: Adding AMD CPU to CI

2018-11-30 Thread Tianqi Chen
I still think it is overkill to add AMD CPU to the CI, given the additional cost it could bring and little additional information we can get out from it. A middle group is to add AMD CPU to a nightly build or final sweep before release. If there is a case that we find that AMD CPU really makes a d

Re: Adding AMD CPU to CI

2018-11-30 Thread Kumar, Vikas
I don't think there is any downside to this proposal. I think a basic sanity CI testing on AMD processors will give extra boost to our tests. This adds to developer productivity and they have one less thing to worry about. Developers have spent time in past where they had to manually test on AMD

Re: Adding AMD CPU to CI

2018-11-30 Thread kellen sunderland
Regarding cost, yes we could run this nightly or simply make it run an existing test suite that would make sense rather than having it duplicate a suite. On Fri, Nov 30, 2018 at 9:26 AM Kumar, Vikas wrote: > I don't think there is any downside to this proposal. I think a basic > sanity CI testin

Re: Adding AMD CPU to CI

2018-11-30 Thread Tianqi Chen
+1 for nightly for pre-release suit, but not the CI that triggered in every test. The best engineering practice is not to add things, but to remove things so that there is nothing can be removed. In terms of MLDNN, since it is an Intel product, I doubt optimizing for AMD CPUs is its goal, adding

Re: Adding AMD CPU to CI

2018-11-30 Thread Kumar, Vikas
I concur. +1 for nightly for pre-release suit. On 11/30/18, 9:49 AM, "Tianqi Chen" wrote: +1 for nightly for pre-release suit, but not the CI that triggered in every test. The best engineering practice is not to add things, but to remove things so that there is nothing can be remo

Re: Adding AMD CPU to CI

2018-11-30 Thread kellen sunderland
+1 to nightly. Given the awesome results shown by Alex for AMD cpus I think MKLDNN actually would probably be something I'd use, even on my AMD machines. Kudos to Intel for releasing this lib which works great on their hardware, but still pretty well w/ AMD. The upshot of MKLDNN supporting AMD to

Re: Adding AMD CPU to CI

2018-11-30 Thread Marco de Abreu
Kellen we run CI in us-west-2, Oregon :P sorry, Environment :( -Marco Am Fr., 30. Nov. 2018, 18:58 hat kellen sunderland < kellen.sunderl...@gmail.com> geschrieben: > +1 to nightly. > > Given the awesome results shown by Alex for AMD cpus I think MKLDNN > actually would probably be something I'd

Re: Adding AMD CPU to CI

2018-11-30 Thread Pedro Larroy
Agee with Tianqi and Hao. Adding AMD brings no value and increases complexity and CI cost. The instructions sets are the same. For benchmarking it might make sense though. Pedro > On 30. Nov 2018, at 18:19, Tianqi Chen wrote: > > I still think it is overkill to add AMD CPU to the CI, given th

Re: Adding AMD CPU to CI

2018-11-30 Thread kellen sunderland
Damn, knew i should have double-checked! Oh well it's also carbon neutral. On Fri, Nov 30, 2018 at 10:27 AM Pedro Larroy wrote: > Agee with Tianqi and Hao. Adding AMD brings no value and increases > complexity and CI cost. The instructions sets are the same. For > benchmarking it might make sen

Re: Adding AMD CPU to CI

2018-11-30 Thread Pedro Larroy
I think just Adding AMD is not the right abstraction level. Testing and benchmarking with different cpu flags / march ie AVX2 sse2 brings value in my opinion. Just testing another vendor of a compatible cpu doesn’t. Pedro > On 30. Nov 2018, at 19:32, kellen sunderland > wrote: > > Damn, knew

Re: CI impaired

2018-11-30 Thread Naveen Swamy
Hi Marco/Gavin, Thanks for the clarification. I was not aware that it has been tested on a separate test environment(this is what I was suggesting and make the changes in a more controlled manner), last time the change was made, many PRs were left dangling and developers had to go trigger and I tr

Re: v1.4.0 status 11/29

2018-11-30 Thread Alex Zai
PR is here https://github.com/apache/incubator-mxnet/pull/13497. On Thu, Nov 29, 2018 at 8:56 PM Lv, Tao A wrote: > Credit belongs to Alex. > > Hi Alex, would you mind porting your fix to the v1.4.x branch? > > Thanks, > -Tao > > -Original Message- > From: Steffen Rochel [mailto:steffenr

Re: CI impaired

2018-11-30 Thread Marco de Abreu
Thanks Naveen and Gavin! #1 has been completed and every job has finished its processing. #2 is the ticket with infra: https://issues.apache.org/jira/browse/INFRA-17346 I'm now waiting for their response. -Marco On Fri, Nov 30, 2018 at 8:25 PM Naveen Swamy wrote: > Hi Marco/Gavin, > > Thanks