matteosal commented on issue #20675: URL: https://github.com/apache/incubator-mxnet/issues/20675#issuecomment-965208956
@szha this is the python equivalent of one of our cases (a single large GRU and a few other operations): [net.zip](https://github.com/apache/incubator-mxnet/files/7513140/net.zip) ``` import mxnet as mx import time print(mx.__version__) batch_size = 64 n_iter = 20 sym = mx.symbol.load('net.json') shapes = {'Input': [batch_size, 100, 16], ':FusedWeights': [3201024], 'State': [64, 1024]} def gen_arrays(): return {name: mx.nd.ones(shape) for (name, shape) in shapes.items()} if(mx.__version__ == '2.0.0'): ex = sym._bind(mx.cpu(), gen_arrays(), args_grad=gen_arrays()) else: ex = sym.bind(mx.cpu(), gen_arrays(), args_grad=gen_arrays()) input_data = [mx.ndarray.random.uniform(0, 1, [batch_size, 100, 16]) for i in range(n_iter)] outgrad = [mx.nd.ones([batch_size, 100, 1024]), mx.nd.ones([batch_size, 1024])] start = time.time() for i in range(n_iter): ex.arg_dict['Input'] = input_data[i] ex.forward() ex.backward(outgrad) mx.ndarray.waitall() end = time.time() inputs_per_sec = batch_size * n_iter / (end - start) print(inputs_per_sec) ``` With v1.6 I get around 70 inputs/sec and only 22 with v2.0. With v2.0 I have used static allocation to stay as close as possible to v1.6 (although it didn't seem to matter much), but I am on fabcd145cd496628791f9f2ea813048360ac33ca and `_bind` doesn't expose static allocation there (it was added later in https://github.com/apache/incubator-mxnet/pull/20462), so I had to manually pass `flags=[("static_alloc", "true")]` [here](https://github.com/apache/incubator-mxnet/blob/master/python/mxnet/executor.py#L125). Also, back to `MXNET_ENGINE_TYPE="NaiveEngine"`: I have tried to run the benchmark with this setting again and this time I have found to difference in both v1.6 and v2.0, so I'm not sure that's a thing anymore. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@mxnet.apache.org For additional commands, e-mail: issues-h...@mxnet.apache.org
