Re: [theano-users] Shared variable dynamic resize in computation graph

[Zachary Kulis]

Thank you for you reply, but I'm trying to understand why:

hid_init = T.ones((input.shape[0], self.num_units), 
dtype=theano.config.floatX) * self.hid_init.get_value()

always uses the initial value assigned to self.hid_init (which is a row of 
zeros). The computation graph seems not to actually evaluate the 
get_value() call at all.

It's true that the size of the weight doesn't depend on the minibatch size, 
but I'm initializing the LSTM's hidden and cell states here.

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Shared variable dynamic resize in computation graph

[Zachary Kulis]

I'm trying to implement a stateful LSTM in Lasagne, and I'm having trouble 
dynamically sizing the matrix of initial states.

Basically, I'm defining the initial hidden and cell states as shared 
variables, each with length equal to number of neurons. It's done this way 
because the batch size is not known at this time (so we assume a batch size 
of 1):

(1) self.hid_init = theano.shared(np.ones((1,self.num_units), 
dtype=theano.config.floatX)
(2) self.cell_init = theano.shared(np.ones((1,self.num_units), 
dtype=theano.config.floatX)

Now, in the computation graph, I want to resize hid_init and cell_init to 
batch_size 
x self.num_units dynamically since the shape info is known. So, I have:

(3) hid_init = T.ones((input.shape[0], self.num_units), 
dtype=theano.config.floatX) * self.hid_init.get_value()
(4) cell_init = T.ones((input.shape[0], self.num_units), 
dtype=theano.config.floatX) * self.cell_init.get_value()

Both hid_init and cell_init are provided to the LSTM scan() function as the 
initial states. I also have a default_update set for self.hid_init and 
self.cell_init 
that sets each of these values to the last hidden state/cell state returned 
from the LSTM scan() function:

(5) self.hid_init.default_update = hid_out[:,-1] if not 
self.only_return_final else hid_out[-1]
(6) self.cell_init.default_update = cell_out[-1]

I've confirmed that the default_update() portion is working; however, (3) 
and (4) always returns a matrix of zeros (of the correct size), which 
indicates that self.hid_init.get_value()in (3) and (4) is perhaps being 
optimized out of the computation graph. In other words, get_value()always 
returns the initial row vector of zeros, even though the shared variable is 
correctly being updated by default_update().

I've spent far too long on this, and would appreciate if anyone could share 
what might be wrong.

Many thanks! 

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Apply generic element wise function against tensor

[Corey Nolet]

Here's what my code in numpy looks like:

tensor = shared(np.random.randn(7, 16, 16))

e_tensor = tensor.eval()

tensor2 = e_tensor[0,:,:]
tensor2[tensor2 < 1] = 0.0
tensor2[tensor2 > 0 = 1.0

new_tensor = [tensor2]
for i in range(1, e_tensor.shape[0]):
new_tensor.append(np.multiply(tensor2, e_tensor[i,:,:]))

output = np.array(new_tensor).reshape(7,16,16)




On Tuesday, January 17, 2017 at 10:47:24 AM UTC-5, Corey Nolet wrote:
>
> I have a tensor which is (7,16,16), let's denote as Tijk. I need to make 
> the following function:
>
> for j in range(0, 16):
> for k in range(0, 16):
> if T0jk == 0:
> for i in range(1, 7):
> Tijk = 0
>
>
> Any ideas on how this can be done with Theano's tensor API?
>
>
> Thanks! :
> 
>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Apply generic element wise function against tensor

[Corey Nolet]

I have a tensor which is (7,16,16), let's denote as Tijk. I need to make 
the following function:

for j in range(0, 16):
for k in range(0, 16):
if T0jk == 0:
for i in range(1, 7):
Tijk = 0


Any ideas on how this can be done with Theano's tensor API?


Thanks! :


-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[theano-users] Re: Save and load MLP model

[Desy rona]

I mean datasets were used for training and testing, ie mnist_train.data, 
mnist_test.data, mnist_valid.data and mnist_train.solution I tried to 
download from https://competitions.codalab.org/competitions/4061#participate
, but the link is broken.









Pada Senin, 16 Januari 2017 14.56.35 UTC+7, krenova Math menulis:
>
> Hi, it should be working, i just accessed it.
>
> You may want to try this link to directly access the zip file:
>
> https://sites.google.com/a/chalearn.org/automl/general-mnist---cnn-example/MNISTrelease2.zip?attredirects=0=1
>
> On Thursday, January 12, 2017 at 4:19:46 PM UTC+8, Desy rona wrote:
>>
>> Thanks for the solution ..
>> but I had a little trouble downloading the dataset, the link is broken, is 
>> there any other links?
>>
>> Pada Selasa, 10 Januari 2017 12.47.20 UTC+7, krenova Math menulis:
>>>
>>> Hi Desy,
>>>
>>> I faced this issue before.
>>>
>>> One way to do this is to save and pickle all the weights. A good 
>>> reference would be to download and study the codes from the following:
>>>
>>> https://sites.google.com/a/chalearn.org/automl/general-mnist---cnn-example
>>>
>>> On Tuesday, January 10, 2017 at 1:22:22 PM UTC+8, Desy rona wrote:

  

 I am using the cnn text classification written by Yoo Kim 
  for sentiment analysis. This 
 code applies cross validation to check the quality of learned model. 
 However, I want to save the learned weights and biases, so I can apply the 
 learned model on new instances one by one for the prediction purpose. I 
 appreciate if someone provides an example of how I can do that. I know 
 that 
 I should use pickle load and dumb to do that, but I am not sure exactly 
 which part of the code I should use them I want to have a separate test.py 
 file so I can only test the trained model on a test sample without 
 training 
 the model again. how I should save and then predict based on saved model?
 I am new to both python and theano. So I appreciate it if someone can 
 provide an example.

>>>

-- 

--- 
You received this message because you are subscribed to the Google Groups 
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to theano-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.