There was a bug on the new GPU back-end related to BN. It was fixed probably last Friday. Make sure to update Theano again to the dev version to be sure to have the fix.
Fred On Mon, Feb 20, 2017 at 3:21 AM Ragav Venkatesan <ragav.venkate...@gmail.com> wrote: > Is this a correct usage of theano's dev version batchnorm? > > > > w = theano.shared(... ) > b = theano.shared(...) > gamma = > theano.shared(value=numpy.ones((1,channels,width,height), > dtype=theano.config.floatX), > name = 'gamma', borrow = borrow) > beta = > theano.shared(value=numpy.zeros((1,channels,width,height), > dtype=theano.config.floatX), name = 'beta', borrow=borrow) > running_mean = theano.shared( > value=numpy.zeros((1,channels,height,width), > dtype=theano.config.floatX), name = 'population_mean', borrow = borrow) > running_var = theano.shared( > value=numpy.ones((1,channels,height,width), dtype=theano.config.floatX), > name = 'population_var', borrow=borrow) > > # Perform conv and pool but before activation ..... > > batch_norm_out,_,_,mean,var = batch_normalization_train( > inputs = pool_out, > gamma = gamma, > beta = beta, > running_mean = > running_mean, > running_var = > running_var ) > > mean = theano.tensor.unbroadcast(mean,0) > var = theano.tensor.unbroadcast(var,0) > updates[running_mean] = mean > updates[running_var] = var + 0.001 # to avoid variance being > zero. > > batch_norm_inference = batch_normalization_test ( > inputs = pool_out > gamma = gamma, > beta = beta, > mean = running_mean, > var = running_var ) > > > Of course I am learning the beta and gamma along with w and b and I am > passing the updates to the train theano function. It seems to work fine > with dot product type layers, but convolutional layers always runs into > NaNs for me. I was wondering if I was using the layer right. I use the > batch_norm_out for training and I use batch_norm_inference for testing. > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to theano-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
