gilbertfrancois edited a comment on issue #18751: URL: https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662750082
Ok, I see that. But I guess it is the same intended behaviour as pyTorch nn.BatchNorm1d for Dense layers, which takes as input (N, C). The normalization is done over C features. E.g.: ``` >>> import torch >>> import torch.nn as nn >>> bn = nn.BatchNorm1d(32) >>> x = torch.randn(2, 32) >>> y = bn(x) >>> bn.running_mean.shape torch.Size([32]) >>> bn.running_var.shape torch.Size([32]) ``` This proofs that the BN layer has C running means and C running vars. When defining `x = torch.randn(1, 32)` and do a forward pass in train mode, it gives an error, like expected. Same as @TristonC mentioned in https://github.com/apache/incubator-mxnet/issues/18751#issuecomment-662730136. When running in eval mode, it does a forward pass: ``` >>> bn.eval() >>> x = torch.randn(1, 32) >>> bn(x) tensor([[-0.6506, 0.9170, -0.8054, 1.4432, -0.2060, -0.5880, -0.0175, -0.8634, 1.0829, 1.1367, 0.6461, 1.1443, -2.3419, 0.0311, -1.2087, 0.1624, -1.4499, -1.2504, -0.0466, 1.5738, 1.3166, 0.7299, 0.0930, 2.1064, -0.0726, 1.6979, -0.9420, -0.1544, 0.8395, 1.9348, 0.1931, 0.9253]], grad_fn=<NativeBatchNormBackward>) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
