gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-665976180
Apologies for the delay in answering. I was offline for a few days.
@TristonC
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-665436383
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662756717
I suspect that the behaviour is corrected when the update of moving_mean and
moving_var on GPU is done in the backward pass, like it is on CPU. It will
solve the
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662755383
Coming back to mxnet: it looks like it is possible to do a forward pass
(inference mode) on cpu when the BatchNorm is placed with Dense layers. But on
gpu, it
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662750082
Ok, I see that. But I guess it is the same intended behaviour as pyTorch
nn.BatchNorm1d, which takes as input (N, C). The normalization is done over C
features.
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662729925
Hi @TristonC, the project is for training. I adapted the script. It It does
now one training step, with forward - backward pass and a validation step.
The
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662179530
Many thanks for all the help and the swift responses.
TL;DR
Adding BatchNorm at the end of a feature extractor, and computing on GPU,
and having a batch
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-661884869
@TristonC Yes, I have a project. Let me adapt it with an open dataset, so
that I can make it public.
gilbertfrancois commented on issue #18751:
URL:
https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-660447079
@szha I've just tested it against **mxnet-cu102 v2.0.0b20200716** and it has
the same problem. See below:
```
gamma on CPU and GPU are
