jiarenyf commented on issue #7710: CTC ERROR WITH CUDA ILLEGAL MEMORY ACCESS ERROR URL: https://github.com/apache/incubator-mxnet/issues/7710#issuecomment-328841747 @szha I found that the problem is not because of the data, but maybe because of the local variables in python functions that are collected by the gc, such that the cuda access illegal memory. And I add "gc.disable()" at the beginning of each epoch, and then add "gc.enable()" at the end of each epoch, the problem never occurs at least for 5 days . I think maybe this could help you to debug ... ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
With regards, Apache Git Services
