mstokes42 commented on issue #8601: Why is the accuracy overestimated? URL: https://github.com/apache/incubator-mxnet/issues/8601#issuecomment-343499688 I figured out the problem was with my batch_size on the val_iter. The sample size was not a multiple of the batch size, so some samples were being discarded by the model.score call that uses val_iter. By changing the batch size to equal the number of samples, the accuracy comes out correct. Now I'm having an issue where by model performs well on the training data, but performs worse than random on the test data. I'd be better off taking the output of the classifier and predicting the opposite. When I apply the model to a different held-out validation set, though, the model again performs almost perfectly... what could explain this behavior?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
