[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-361762157 @piiswrong @eric-haibin-lin @marcoabreu @cjolivier01 @szha @rahul003 @reminisce Thank you very much for your comments. Do you have more

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [x] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [x] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-359960347 Does it run the model in the inference mode? On Tue, Jan 23, 2018 at 1:25 PM, Sheng Zha wrote: >

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-359960347 Does it run the model in the inference mode? On Tue, Jan 23, 2018 at 1:25 PM, Sheng Zha wrote: >

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-358894035 @pengzhao-intel could you also put up your evaluation on the speed of this MKLDNN integration?

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-358145000 @eric-haibin-lin the illegal instruction issue in CI has been resolved. The solution is in the latest commit.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-358086101 The problem that mkldnn+GPU can't run on G3 is caused by the fact that mkldnn is compiled with avx512 instructions on C5 but G3 doesn't support

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357818985 I'm not sure. you think avx2 or avx512 or sse instructions have conflict with nvidia driver?

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357815594 I think I have figured out the problems with mkldnn+gpu in Jenkins. If I build everything in Dockerfile.gpu_mklml and test all unit tests in

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357815594 I think I have figured out the problems with mkldnn+gpu in Jenkins. If I build everything in Dockerfile.gpu_mklml and test all unit tests in

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357815594 I think I have figured out the problems with mkldnn+gpu in Jenkins. If I build everything in Dockerfile.gpu_mklml and test all unit tests in

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357815594 I think I have figured out the problems with mkldnn+gpu in Jenkins. If I build everything in Dockerfile.gpu_mklml and test all unit tests in

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357815594 I think I have figured out the problems with mkldnn+gpu in Jenkins. If I build everything in Dockerfile.gpu_mklml and test all unit tests in

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357805365 @marcoabreu it seems that deep learning AMI uses nvidia 384. ``` ubuntu@ip-172-31-28-11:~$ cat /proc/driver/nvidia/version NVRM version:

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357798062 @marcoabreu i can't modify any Dockerfile right now. It seems nvidia docker is completely broken. I tried a few different versions of their docker

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357798062 @marcoabreu i can't modify any Dockerfile right now. It seems nvidia docker is completely broken. I tried a few different versions of their docker

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357797587 folks. I have squashed all modifications into three commits. This is an automated

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [x] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [x] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357794398 @cjolivier01 the problem with MKLDNN and GPU only happens in the Jenkins tests. I tested it locally on G3 with deep learning AMI and all tests are

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357788446 A known issue: mkldnn can't work with GPU. I did some test and it shows that the problem is most likely caused by the conflict between mkldnn and

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-357778866 Thanks for helping review the code. @rahul003 @cjolivier01 @reminisce @eric-haibin-lin @piiswrong @szha The design document of this

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [x] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. - [ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. -[ ] Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 Remaining MKLDNN-related tasks before merging the PR. * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-353705559 * Speedup inference with batch size 1. The goal is to reach the speed of old.mkldnn. The main work in this task is to speed up activation and BN.

[GitHub] zheng-da commented on issue #8302: Refactor operators & MKLDNN

zheng-da commented on issue #8302: Refactor operators & MKLDNN URL: https://github.com/apache/incubator-mxnet/pull/8302#issuecomment-350410800 @cjolivier01 the reason code refactoring and MKLDNN are put in the same PR is that we don't want to refactor original MKL code. We don't want a PR