ptrendx opened a new pull request #14067: Increase perfomance of BulkAppend and BulkFlush URL: https://github.com/apache/incubator-mxnet/pull/14067 ## Description ## Increase the performance of `BulkAppend` and `BulkFlush` methods used in Gluon hybridized models with `static_alloc=True, static_shape=False`. ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [x] Changes are complete (i.e. I finished coding on this PR) - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x] The previous way of bulking ops in Gluon was creating a new lambda during every BulkAppend function in kind of recursive scheme, which involved copying all the environment every time. This PR changes that by instead introducing a vector of lambda functions to the `BulkStatus` and populating that vector. - [x] That change improves the time to perform `BulkAppend`, but `BulkFlush` still needed to perform the copy of the entire vector of lambdas, including all of the environment. This is alleviated by, instead of passing the vector by value, a `shared_ptr` is passed instead, increasing the performance of `BulkFlush` function by ~3.5x from 70us to ~20us. ## Comments ## - Those changes have the biggest impact in multi-GPU or small model scenario when the speed of scheduling work is the most important (in multi-GPU all work is scheduled by a single Python thread). - The speed of multiGPU launches should be improved further by going to multiprocess model where each process handles a single GPU. @eric-haibin-lin
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
