[GitHub] [incubator-mxnet] jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default
jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default URL: https://github.com/apache/incubator-mxnet/issues/17331#issuecomment-582752477 @apeforest Oh sorry, so I'm multiplying by only for the samples/second column -1 to keep the meaning consistent with everything else. The rest of the columns depict the correct positive percentage improvement and negative percentage degradation. For example if MKL_LT gives 66 samples/sec and MKL gives 70 samples/sec that will be: 1-(66/70) or 6%. Because it's positive, we think that it's better but actually it's worse because the throughput has gone down. On the other hand if MKL_LT gives 74 samples/sec and MKL gives 70 samples/sec that will be: 1-(74/70) or -5%. Because it's negative, we think it's worse but actually it's better because our throughput has gone up. So I multiply by -1 to give it the same meaning as the rest of the percentages, where positive is better and negative is worse. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default
jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default URL: https://github.com/apache/incubator-mxnet/issues/17331#issuecomment-582702619 @eric-haibin-lin Yes I am calculating this by: 1 - ( / ). For the samples/sec I am multiplying by -1. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default
jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default URL: https://github.com/apache/incubator-mxnet/issues/17331#issuecomment-582689067 Training Benchmarks comparing LT_MKL with just MKL Enabled. Speed measured seconds per Epoch. GPU Memory measured in MB. Note: Samples/Second are opposite so I have multiple the percentages by -1. A quick explanation: The number should be going higher so a positive percentage change means there are now less samples/second. A negative percentage change means there are more samples/second. Model | Speed P50 LT | Speed P50 No LT | GPU Memory LT | GPU Memory No LT | Samples/Second P50 LT | Samples/Second P50 no LT | Speed Percentage Change | GPU Memory Percentage Change | Samples/Second Percentage Change -- | -- | -- | -- | -- | -- | -- | -- | -- | -- xception | 19247.12517 | 18935.02989 | 15304 | 15320 | 67.51961 | 68.61849 | -1.65% | 0.10% | -1.60% resnet50_v2 | 4342.953992 | 4342.899322 | 6892 | 6762 | 299.0174 | 299.1728 | 0.00% | -1.92% | -0.05% gnmt | N/A | N/A | 4244 | 4112 | 7.65 | 7.675 | | -3.21% | -0.33% vgg16 | 5680.658345 | 5641.058277 | 9480 | 9496 | 228.4218 | 230.0739 | -0.70% | 0.17% | -0.72% bert | 20.66 | 16.8 | 4684 | 4050 | 38.1 | 46.7 | -22.98% | -15.65% | -18.42% yolo3_darknet53_custom | 517.4205 | 454.908 | 7304 | 12436 | 31.6145 | 40.65 | -13.74% | 41.27% | -22.23% inceptionv3 | 5765.122603 | 5723.867063 | 8318 | 8304 | 225.4025 | 227.1884 | -0.72% | -0.17% | -0.79% se_resnet152_v1 | 10497.33863 | 10465.23692 | 11290 | 10568 | 123.7371 | 124.1493 | -0.31% | -6.83% | -0.33% word_language_model | 141.125 | 142.3 | 8846 | 7426 | 15651.19 | 15524.71 | 0.83% | -19.12% | 0.81% mobilenet0.25_cifar10 | 56.6609205 | 60.5992765 | 1234 | 1134 | N/A | N/A | 6.50% | -8.82% | resnet101_v1 | 7354.353666 | 7329.202738 | 8118 | 8022 | 176.6355 | 177.3132 | -0.34% | -1.20% | -0.38% squeezenet1.0 | 1677.752777 | 1678.684668 | 3770 | 3590 | 790.7722 | 790.1395 | 0.06% | -5.01% | 0.08% mobilenetv2_0.75 | 1938.194231 | 1968.429737 | 5078 | 5008 | 680.4143 | 672.2202 | 1.54% | -1.40% | 1.22% ssd | 424.28 | 254.9485 | 4702 | 4592 | 66.2365 | 67.56 | -66.42% | -2.40% | -1.96% Average Percentage Change: Speed: -7.53% GPU Memory: -1.73% Samples / Second: -3.44% This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-mxnet] jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default
jonatan1626 commented on issue #17331: [mxnet 2.0] [item 2.4] Turning on large tensor support by default URL: https://github.com/apache/incubator-mxnet/issues/17331#issuecomment-582591625 Inference Benchmarks comparing LT_MKL with just MKL Enabled. All Time in MS. MXNet Type | Model | Mode | average | p50 | p90 | p99 | std-dev | p50 Improvement | p50 Improvement Percentage -- | -- | -- | -- | -- | -- | -- | -- | -- | -- mxnet_LT_MKL | resnext101_64x4d | gluon | 50.1521455 | 47.3425388 | 53.8454056 | 191.146851 | 25.6092029 | 2.12430954 | 4% mxnet_MKL | resnext101_64x4d | gluon | 50.6807831 | 49.4668484 | 53.098917 | 192.113161 | 26.3168013 | | LT_MKL / MKL | | 0.98956927 | 0.95705589 | 1.01405845 | 0.9949701 | 0.9731123 | | | | | | | | | | | mxnet_LT_MKL | resnext101_64x4d | module | 33.3508749 | 28.8367271 | 35.504818 | 151.783705 | 27.1059643 | -0.3488064 | -1% mxnet_MKL | resnext101_64x4d | module | 32.4659094 | 28.4879208 | 36.0739231 | 99.5042324 | 20.7594173 | | LT_MKL / MKL | | 1.0272583 | 1.01224401 | 0.98422392 | 1.52539948 | 1.30571894 | | | | | | | | | | | mxnet_LT_MKL | resnext50 | gluon | 18.7125455 | 17.1453953 | 21.2440491 | 23.4313011 | 5.93459749 | 0.91052055 | 5% mxnet_MKL | resnext50 | gluon | 19.1066013 | 18.0559158 | 20.870924 | 24.1959095 | 7.32521894 | | LT_MKL / MKL | | 0.97937593 | 0.94957218 | 1.01787775 | 0.96839927 | 0.81015974 | | | | | | | | | | | mxnet_LT_MKL | resnext50 | module | 11.056515 | 10.0550652 | 12.2823715 | 13.5610104 | 6.54063042 | -0.4184246 | -4% mxnet_MKL | resnext50 | module | 10.7395838 | 9.63664055 | 12.3991966 | 13.109684 | 6.55145709 | | LT_MKL / MKL | | 1.02951057 | 1.04342017 | 0.99057801 | 1.03442695 | 0.99834744 | | | | | | | | | | | mxnet_LT_MKL | nin | gluon | 2.59861518 | 2.57444382 | 2.81214714 | 3.3082962 | 1.0480078 | 0.03361702 | 1% mxnet_MKL | nin | gluon | 2.67292721 | 2.60806084 | 2.96282768 | 3.25322151 | 1.40067806 | | LT_MKL / MKL | | 0.97219826 | 0.98711034 | 0.949143 | 1.01692928 | 0.74821462 | | | | | | | | | | | mxnet_LT_MKL | nin | module | 2.44113064 | 2.43210793 | 2.50315666 | 2.71081924 | 1.25599202 | 0.30565262 | 11% mxnet_MKL | nin | module | 2.78057171 | 2.73776054 | 2.78377533 | 3.11684608 | 2.75584319 | | LT_MKL / MKL | | 0.877924 | 0.8883567 | 0.89919493 | 0.86973151 | 0.45575598 | | | | | | | | | | | mxnet_LT_MKL | resnet18 | gluon | 3.86981446 | 3.89575958 | 4.06169891 | 4.43696976 | 2.31413846 | -0.2574921 | -7% mxnet_MKL | resnet18 | gluon | 3.69065023 | 3.63826752 | 3.96156311 | 4.49037552 | 1.61035003 | | LT_MKL / MKL | | 1.04854544 | 1.07077326 | 1.02527684 | 0.98810662 | 1.43704065 | | | | | | | | | | | mxnet_LT_MKL | resnet18 | module | 2.99924937 | 2.95495987 | 3.18932533 | 3.74364853 | 3.15199014 | 0.22792816 | 7% mxnet_MKL | resnet18 | module | 3.20062987 | 3.18288803 | 3.33738327 | 3.64685059 | 1.98630643 | | LT_MKL / MKL | | 0.93708098 | 0.92838951 | 0.95563652 | 1.02654289 | 1.58685996 | | | | | | | | | | | mxnet_LT_MKL | wavernn | gluon | 299.9031 | 262.938976 | 365.684748 | 861.40275 | 121.629071 | -6.3843727 | -2% mxnet_MKL | wavernn | gluon | 280.279419 | 256.554604 | 327.608585 | 785.755396 | 113.065038 | | LT_MKL / MKL | | 1.0700147 | 1.02488504 | 1.11622456 | 1.09627341 | 1.0757443 | | | | | | | | | | | mxnet_LT_MKL | caffenet | gluon | 2.94373734 | 2.93087959 | 3.27038765 | 3.6482811 | 2.00530845 | 0.15687943 | 5% mxnet_MKL | caffenet | gluon | 3.28968997 | 3.08775902 | 3.67426872 | 4.28771973 | 2.42602299 | | LT_MKL / MKL | | 0.89483732 | 0.94919311 | 0.89007852 | 0.85086744 | 0.82658262 | | | | | | | | | | | mxnet_LT_MKL | caffenet | module | 3.23916318 | 3.16953659 | 3.4327507 | 3.99780273 | 3.02630778 | 0.05578995 | 2% mxnet_MKL | caffenet | module | 3.3244319 | 3.22532654 | 3.74293327 | 4.31966782 | 2.66768369 | | LT_MKL / MKL | | 0.97435089 | 0.98270254 | 0.91712848 | 0.92548846 | 1.13443276 | | | | | | | | | | | mxnet_LT_MKL | vgg19 | gluon | 18.4855731 | 14.1830444 | 22.5963593 | 36.3841057 | 11.3180528 | -0.2920628 | -2% mxnet_MKL | vgg19 | gluon | 18.1985553 | 13.8909817 | 22.2861767 | 31.3508511 | 9.6094407 | | LT_MKL / MKL | | 1.01577146 | 1.02102535 | 1.01391816 | 1.16054603 | 1.17780557 | | | | | | | | | | | mxnet_LT_MKL | vgg19 | module | 17.794488 | 13.8015747 | 21.9786167 | 44.4116592 | 12.0074793 | 0.53334236 | 4% mxnet_MKL | vgg19 | module | 19.0417649 | 14.3349171 | 22.4690437 | 52.0129204 | 14.4256122 | | LT_MKL /
