safrooze commented on issue #9705: Added unittest for benchmarking metric performance URL: https://github.com/apache/incubator-mxnet/pull/9705#issuecomment-363977854 OK I think I addressed all the feedback: * random is seeded * nd.wait_all() used before starting timing and before ending timing * Added batch-size values of 16, 64, 256, and 1024 * Datasize varies by number of output channels to keep total runtime down to a few minutes The modified code output looks like this: ``` Metric Data-Ctx Label-Ctx Data Size Batch Size Output Dim Elapsed Time ------------------------------------------------------------------------------------------ acc cpu(0) cpu(0) 131072 16 128 1.0015 acc cpu(0) gpu(0) 131072 16 128 1.682 acc gpu(0) cpu(0) 131072 16 128 2.6263 acc gpu(0) gpu(0) 131072 16 128 3.3028 ------------------------------------------------------------------------------------------ acc cpu(0) cpu(0) 131072 64 128 0.42843 acc cpu(0) gpu(0) 131072 64 128 0.568 acc gpu(0) cpu(0) 131072 64 128 0.78586 acc gpu(0) gpu(0) 131072 64 128 0.94317 ------------------------------------------------------------------------------------------ ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
