kpuatamazon commented on issue #17559: [MXNET-1446] Quantization: intgemm matrix multiply wrappers URL: https://github.com/apache/incubator-mxnet/pull/17559#issuecomment-587073135 Also, OMP performance is very bad. (NB: intgemm is running single-threaded here, partly because OMP is bad at this problem) ```bash export MXNET_ENGINE_TYPE=NaiveEngine; export OMP_NUM_THREADS=2; taskset --cpu-list 0,1 ./quant_bench.py ``` ``` [16:18:40] src/engine/engine.cc:55: MXNet start using engine: NaiveEngine Shape (128, 128) 0.0008789 seconds for quantize 0.0008693 seconds for quantize_v2 0.0000175 seconds for intgemm intgemm is 49.7x faster Shape (256, 256) 0.0034812 seconds for quantize 0.0034044 seconds for quantize_v2 0.0000212 seconds for intgemm intgemm is 161.0x faster Shape (512, 512) 0.0138909 seconds for quantize 0.0138283 seconds for quantize_v2 0.0000731 seconds for intgemm intgemm is 189.3x faster Shape (1024, 1024) 0.0557616 seconds for quantize 0.0553598 seconds for quantize_v2 0.0002330 seconds for intgemm intgemm is 237.6x faster Shape (2048, 2048) 0.2225617 seconds for quantize 0.2196410 seconds for quantize_v2 0.0008387 seconds for intgemm intgemm is 261.9x faster Shape (8, 4096) 0.0017372 seconds for quantize 0.0017434 seconds for quantize_v2 0.0000183 seconds for intgemm intgemm is 94.8x faster ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
