jiarenyf commented on issue #7710: CTC ERROR WITH CUDA ILLEGAL MEMORY ACCESS ERROR URL: https://github.com/apache/incubator-mxnet/issues/7710#issuecomment-330248910 @szha I found another reason of that problem. As in "https://github.com/jiarenyf/mxWrapper/blob/02fd9b0fcd37f7224648efad651a6f83a1f06d78/mxHelper/mxData.py#L158";, the labels is initialize to empty, while in "https://github.com/jiarenyf/mxWrapper/blob/02fd9b0fcd37f7224648efad651a6f83a1f06d78/mxHelper/model/model.py#L98";, it directly uses batch.label without considering the batch.pad. So if not meets "data size % batch size ==0", the error occurs when accessing empty labels. And the data set I offered you to debug has 800 images, and it meets: "800 % 100 == 0" (100 is batch size). So the problem never occurs on your workplace. I found this problem because I happen to change the train set size to 111 and the test size to 33, where "111+33 % 100 != 0". And after I add "https://github.com/jiarenyf/mxWrapper/blob/02fd9b0fcd37f7224648efad651a6f83a1f06d78/mxHelper/mxData.py#L169"; (remove empty), and change "https://github.com/jiarenyf/mxWrapper/blob/02fd9b0fcd37f7224648efad651a6f83a1f06d78/mxHelper/mxData.py#L170"; (set "pad" to 0), then the problem does not occur. I wonder is there some error or bug in my implementation: "https://github.com/jiarenyf/mxWrapper/blob/master/mxHelper/model/model.py"; ? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
With regards, Apache Git Services
