The + of the + T.dot(u, v). The debugprint command I gave you will help separate the forward computation from the grad computation.
The grad of a dot is a another dot. So what would explain a 0 outputs would be too many or only zeros in the inputs. Can you very the values of m and n? Make sure there is no zeros in them. On Thu, Jun 29, 2017 at 9:05 AM Mohamed Akrout <mohammed.akr...@gmail.com> wrote: > Yes I printed the gradient function of m but it is extremely big. I find > it unreadable (file attached). I don't know how this tree will help me find > the problem. There are nodes who are Alloc and second but I don't know how > to change and/or control them. > > When you say: "Only the extra addition will be done at each iterations", > about which extra addition are you talking? > > Thank you Fred. > > Med > > Regarding your notice, if m and n are non sequence, Theano will not updat > > > On Thursday, June 29, 2017 at 8:34:32 AM UTC-4, nouiz wrote: > >> I don't know, but you can use theano.printing.debugprint([cost, grads...]) >> >> To see the gradient function. Maybe it will help you understand what is >> going on. >> >> Don't forget m and n are non sequence. This mean the dot will be lifted >> out of the loop by Theano. Only the extra addition will be done at each >> iterations. >> >> Fred >> >> Le mer. 28 juin 2017 19:12, Mohamed Akrout <mohamme...@gmail.com> a >> écrit : >> > Hi all, >>> >>> I am running a neuroscience with an recurrent neural network model with >>> Theano: >>> >>> >>> >>> def rnn(u_t, x_tm1, r_tm1, Wrec): >>> x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec ) + brec + >>> u_t[:,Nin:]) ) >>> r_t = f_hidden(x_t) >>> >>> >>> then I define the scan function to iterate at each time step iteration >>> >>> [x, r], _ = theano.scan(fn=rnn, >>> outputs_info=[x0_, f_hidden(x0_)], >>> sequences=u, >>> non_sequences=[Wrec]) >>> >>> Wrec and brec are learnt by stochastic gradient descent: g = T.grad(cost >>> , [Wrec, brec]) >>> >>> where cost is the cost function: T.sum(f_loss(z, target[:,:,:Nout])) >>> with z = f_output(T.dot(r, Wout_.T) + bout ) >>> >>> Until now, everything works good. >>> >>> >>> >>> Now I want to add two new vectors, let's call them u and v so that the >>> initial rnn function becomes: >>> >>> >>> def rnn(u_t, x_tm1, r_tm1, Wrec, *u, v*): >>> x_t = ( (1 - alpha)*x_tm1 + alpha*(T.dot(r_tm1, Wrec + *T.dot(u, >>> v)* ) + brec + u_t[:,Nin:]) ) >>> r_t = f_hidden(x_t) >>> >>> [x, r], _ = theano.scan(fn=rnn, >>> outputs_info=[x0_, f_hidden(x0_)], >>> sequences=u, >>> non_sequences=[Wrec,* m, n*]) >>> >>> m and n are the variables corresponding to u and v in the main function. >>> >>> and suddenly, the gradient T.grad(cost, m) and T.grad(cost, n) are zeros >>> >>> I am blocked since 2 weeks now on this problem. I verified that the >>> values are not integer by using dtype=theano.config.floatX every where in >>> the definition of the variables. >>> >>> As you can see the link between the cost and m (or n) is: the cost >>> function depends on z, and z depends on r and r is one of the outputs of >>> the rnn function that uses m and n in the equation. >>> >>> Do you have any ideas why this does not work ? >>> >>> Any idea is welcome. I hope I can unblock this problem soon. >>> Thank you! >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "theano-users" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> email to theano-users...@googlegroups.com. >> >> >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to theano-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
