Re: [theano-users] Return a tuple of length-1 from scan?
I answered in the other email on this subject. On Mon, Jan 16, 2017 at 9:57 AM, Peter O'Connor wrote: > Hi all. > > There's some really unintuitive/inconsistent behaviour with theano's scan > and I'm wondering how I should work around it. > > Take the following test, which I would expect to pass: > > import theano > import theano.tensor as tt > > > def test_scan_tuple_output(): > > x = tt.matrix() > > def mul_by_2_and_3(data): > return (data*2, data*3) > > y, updates = theano.scan(mul_by_2_and_3, sequences=x) > > assert len(y)==2 > assert len(updates)==0 > > def mul_by_2(data): > return (data*2, ) > > y, updates = theano.scan(mul_by_2, sequences = [x]) > > assert len(y)==1 > assert len(updates) == 0 > > > if __name__ == '__main__': > test_scan_tuple_output() > > It fails on > It > It fails on > > assert len(y)==1 > > with "TypeError: object of type 'TensorVariable' has no len()" because if > scan detects a tuple of length-1, it just returns the data inside this > tuple. > > This becomes annoying when trying to make a scannable function with > variable return length, (one of which can be one). > > Is there a way I can make scan not totally change its behaviour when the > tuple of returned arguments happens to have length 1? > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to theano-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [theano-users] Scan: Return tuple of length 1?
Thanks for the idea. At a few places, we allow to pass return_list=True to force the return a of list. I added that option to scan in that PR: https://github.com/Theano/Theano/pull/5420 Why we don't always return a list? It is because the best interface depend of what you want. If you use Theano normally not returning a list is better, but when you try to automatize stuff, always having a list is better. That is why we provide both at a few places. Can you confirm this do what you want? Fred On Mon, Jan 16, 2017 at 9:59 AM, Peter O'Connor wrote: > Hi all. > > There's some really unintuitive/inconsistent behaviour with theano's scan > and I'm wondering how I should work around it. > > Take the following test, which I would expect to pass: > > import theano > import theano.tensor as tt > > > def test_scan_tuple_output(): > > x = tt.matrix() > > def mul_by_2_and_3(data): > return (data*2, data*3) > > y, updates = theano.scan(mul_by_2_and_3, sequences=[x]) > > assert len(y)==2 > assert len(updates)==0 > > def mul_by_2(data): > return (data*2, ) > > y, updates = theano.scan(mul_by_2, sequences = [x]) > > assert len(y)==1 > assert len(updates) == 0 > > > if __name__ == '__main__': > test_scan_tuple_output() > > It fails on > It > It fails on > > assert len(y)==1 > > with "TypeError: object of type 'TensorVariable' has no len()" because if > scan detects a tuple of length-1, it just returns the data inside this > tuple. > > This becomes annoying when trying to make a scannable function with > variable return length, (one of which can be one). > > Is there a way I can make scan not totally change its behaviour when the > tuple of returned arguments happens to have length 1? > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to theano-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] Network does not update its weights
Hi. I have been trying to develop imagenet, I have the base of LENET and modifying it and adding things I would like to get Imagenet. My problem is that weights are not updated so the network does not learn and I can not figure out where the problem is and I do not know what to do! I am really stressed and blocked. So any help or idea would be welcome!! I have attached two files, "layers.py" where the layers are described and "network.py" where the architecture and the learning/testing processing is described and carried out. Thank you very much in advance. Regards. Beatriz -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. import numpy as np import theano import theano.tensor as T from theano.tensor.signal import pool from theano.tensor.nnet import conv2d from pylearn2.expr.normalize import CrossChannelNormalization class Fully_Connected_Dropout(object): # http://christianherta.de/lehre/dataScience/machineLearning/neuralNetworks/Dropout.php def __init__(self, rng, is_train, input, n_in, n_out, W=None, b=None, p=0.5): self.input = input # end-snippet-1 rng = np.random.RandomState(1234) srng = T.shared_randomstreams.RandomStreams(rng.randint(99)) # for a discussion of the initialization, see # https://plus.google.com/+EricBattenberg/posts/f3tPKjo7LFa if W is None: W_values = np.asarray( rng.uniform( low=-np.sqrt(6. / (n_in + n_out)), high=np.sqrt(6. / (n_in + n_out)), size=(n_in, n_out) ), dtype=theano.config.floatX ) W = theano.shared(value=W_values, name='W', borrow=True) # init biases to positive values, so we should be initially in the linear regime of the linear rectified function if b is None: b_values = np.ones((n_out,), dtype=theano.config.floatX) * np.cast[theano.config.floatX](0.01) b = theano.shared(value=b_values, name='b', borrow=True) self.W = W self.b = b lin_output = T.dot(input, self.W) + self.b output = theano.tensor.nnet.relu(lin_output) # multiply output and drop -> in an approximation the scaling effects cancel out input_drop = np.cast[theano.config.floatX](1. / p) * output mask = srng.binomial(n=1, p=p, size=input_drop.shape, dtype=theano.config.floatX) train_output = input_drop * mask # is_train is a pseudo boolean theano variable for switching between training and prediction self.output = T.switch(T.neq(is_train, 0), train_output, output) # parameters of the model self.params = [self.W, self.b] class Fully_Connected_Softmax(object): def __init__(self, rng, input, n_in, n_out, W=None, b=None, activation=theano.tensor.nnet.relu): self.input = input if W is None: W_values = np.asarray( rng.uniform( low=-np.sqrt(6. / (n_in + n_out)), high=np.sqrt(6. / (n_in + n_out)), size=(n_in, n_out) ), dtype=theano.config.floatX ) if activation == theano.tensor.nnet.sigmoid: W_values *= 4 W = theano.shared(value=W_values, name='W', borrow=True) if b is None: b_values = np.zeros((n_out,), dtype=theano.config.floatX) b = theano.shared(value=b_values, name='b', borrow=True) self.W = W self.b = b lin_output = T.nnet.softmax(T.dot(input, self.W) + self.b) self.output = ( lin_output if activation is None else activation(lin_output) ) self.params = [self.W, self.b] class LeNetConvPoolLRNLayer(object): def __init__(self, rng, input, filter_shape, image_shape, poolsize=(2, 2), stride=(1, 1), lrn=False): """ Allocate a LeNetConvPoolLayer with shared variable internal parameters. :type rng: numpy.random.RandomState :param rng: a random number generator used to initialize weights :type input: theano.tensor.dtensor4 :param input: symbolic image tensor, of shape image_shape :type filter_shape: tuple or list of length 4 :param filter_shape: (number of filters, num input feature maps, filter height, filter width) :type image_shape: tuple or list of length 4 :param image_shape: (batch size, num input feature maps, image height, image width) :type poolsize: tuple or list of
[theano-users] Scan: Return tuple of length 1?
Hi all. There's some really unintuitive/inconsistent behaviour with theano's scan and I'm wondering how I should work around it. Take the following test, which I would expect to pass: import theano import theano.tensor as tt def test_scan_tuple_output(): x = tt.matrix() def mul_by_2_and_3(data): return (data*2, data*3) y, updates = theano.scan(mul_by_2_and_3, sequences=[x]) assert len(y)==2 assert len(updates)==0 def mul_by_2(data): return (data*2, ) y, updates = theano.scan(mul_by_2, sequences = [x]) assert len(y)==1 assert len(updates) == 0 if __name__ == '__main__': test_scan_tuple_output() It fails on It It fails on assert len(y)==1 with "TypeError: object of type 'TensorVariable' has no len()" because if scan detects a tuple of length-1, it just returns the data inside this tuple. This becomes annoying when trying to make a scannable function with variable return length, (one of which can be one). Is there a way I can make scan not totally change its behaviour when the tuple of returned arguments happens to have length 1? -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] Scan: Return tuple of length 1?
Hi all. There's some really unintuitive/inconsistent behaviour with theano's scan and I'm wondering how I should work around it. Take the following test, which I would expect to pass: import theano import theano.tensor as tt def test_scan_tuple_output(): x = tt.matrix() def mul_by_2_and_3(data): return (data*2, data*3) y, updates = theano.scan(mul_by_2_and_3, sequences=x) assert len(y)==2 assert len(updates)==0 def mul_by_2(data): return (data*2, ) y, updates = theano.scan(mul_by_2, sequences = [x]) assert len(y)==1 assert len(updates) == 0 if __name__ == '__main__': test_scan_tuple_output() It fails on It It fails on assert len(y)==1 with "TypeError: object of type 'TensorVariable' has no len()" because if scan detects a tuple of length-1, it just returns the data inside this tuple. This becomes annoying when trying to make a scannable function with variable return length, (one of which can be one). Is there a way I can make scan not totally change its behaviour when the tuple of returned arguments happens to have length 1? -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] Return a tuple of length-1 from scan?
Hi all. There's some really unintuitive/inconsistent behaviour with theano's scan and I'm wondering how I should work around it. Take the following test, which I would expect to pass: import theano import theano.tensor as tt def test_scan_tuple_output(): x = tt.matrix() def mul_by_2_and_3(data): return (data*2, data*3) y, updates = theano.scan(mul_by_2_and_3, sequences=x) assert len(y)==2 assert len(updates)==0 def mul_by_2(data): return (data*2, ) y, updates = theano.scan(mul_by_2, sequences = [x]) assert len(y)==1 assert len(updates) == 0 if __name__ == '__main__': test_scan_tuple_output() It fails on It It fails on assert len(y)==1 with "TypeError: object of type 'TensorVariable' has no len()" because if scan detects a tuple of length-1, it just returns the data inside this tuple. This becomes annoying when trying to make a scannable function with variable return length, (one of which can be one). Is there a way I can make scan not totally change its behaviour when the tuple of returned arguments happens to have length 1? -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [theano-users] nvidia TX1 Embedded Board not support
Which version of Theano do you use? Can you confirm you use Theano development version? 0.9.devN? Fred On Mon, Jan 16, 2017 at 3:11 AM, sang hyun Yoon wrote: > can i help.. > Thank you.. > > WARNING (theano.gof.cmodule): OPTIMIZATION WARNING: Theano was not able to > find the default g++ parameters. This is needed to tune the compilation to > your specific CPU. This can slow down the execution of Theano functions. > Please submit the following lines to Theano's mailing list so that we can > fix this problem: > ['# 1 ""\n', '# 1 ""\n', '# 1 ""\n', '# 1 > "/usr/include/stdc-predef.h" 1 3 4\n', '# 1 "" 2\n', '# 1 > ""\n', 'Using built-in specs.\n', 'COLLECT_GCC=/usr/bin/g++\n', > 'Target: aarch64-linux-gnu\n', "Configured with: ../src/configure -v > --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4' > --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs > --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr > --program-suffix=-5 --enable-shared --enable-linker-build-id > --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix > --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu > --enable-libstdcxx-debug --enable-libstdcxx-time=yes > --with-default-libstdcxx-abi=new --enable-gnu-unique-object > --disable-libquadmath --enable-plugin --with-system-zlib > --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre > --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64 > --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64 > --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar > --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror > --enable-checking=release --build=aarch64-linux-gnu > --host=aarch64-linux-gnu --target=aarch64-linux-gnu\n", 'Thread model: > posix\n', 'gcc version 5.4.0 20160609 (Ubuntu/Linaro > 5.4.0-6ubuntu1~16.04.4) \n', "COLLECT_GCC_OPTIONS='-E' '-v' > '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'\n", ' > /usr/lib/gcc/aarch64-linux-gnu/5/cc1 -E -quiet -v -imultiarch > aarch64-linux-gnu - -mlittle-endian -mabi=lp64 -fstack-protector-strong > -Wformat -Wformat-security\n', 'ignoring nonexistent directory > "/usr/local/include/aarch64-linux-gnu"\n', 'ignoring nonexistent > directory > "/usr/lib/gcc/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/include"\n', > '#include "..." search starts here:\n', '#include <...> search starts > here:\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include\n', ' > /usr/local/include\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include-fixed\n', > ' /usr/include/aarch64-linux-gnu\n', ' /usr/include\n', 'End of search > list.\n', 'COMPILER_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/ > gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/:/ > usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/\n', > 'LIBRARY_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/ > gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/:/usr/ > lib/gcc/aarch64-linux-gnu/5/../../../../lib/:/lib/aarch64- > linux-gnu/:/lib/../lib/:/usr/lib/aarch64-linux-gnu/:/usr/ > lib/../lib/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../:/lib/:/usr/lib/\n', > "COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mlittle-endian' > '-mabi=lp64'\n"] > > -- > > --- > You received this message because you are subscribed to the Google Groups > "theano-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to theano-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] Re: How to: data parallelism in new GPU backend
Oh, this can be done by setting, for example, the `data` field of the desired entry of the input_storage list to be the `data` from the corresponding input of another function. Each time the same function is unpickled, the memory usage on the GPU goes up by the size of the shareds. Setting the data object as equivalent between two such functions does not reduce apparent GPU memory usage, but subsequently unpickling the again shows no increase in memory (unpickling once more does increase memory again), so it appears to be working fine at re-using the memory. :) -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [theano-users] Mixture Density Networks
I also need it for some experience. Can u post it to me?thanks. 在 2014年4月22日星期二 UTC+8下午1:29:45,Sank写道: > > I need it. Can u post it? > > On Saturday, 19 April 2014 03:08:14 UTC+5:30, Sid wrote: >> >> Hi Fred, >> >> I implemented it myself, I can post it here if anybody needs. >> Thanks :) >> >> -Sid >> >> >> On Tue, Apr 15, 2014 at 3:52 PM, Frédéric Bastien >> wrote: >> >>> As you didn't got answer in the last 3 days, try to email pylearn >>> mailing list. Maybe you will have an answer there. >>> >>> Also, many people are at ICLR, so the response time can be slower. >>> >>> Fred >>> >>> >>> On Sat, Apr 12, 2014 at 6:58 AM, Sid wrote: >>> Hello, I wanted to run some experiments with Mixture Density Networks. I was wondering if anyone here has tried implementing MDNs themselves or has an implementation of GMMs using theano. I don't mind implementing it myself, but it would be easier if pylearn2 or somebody else already has an implementation of it. Thanks :) -Sid -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. >>> >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "theano-users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to theano-users...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] nvidia TX1 Embedded Board not support
can i help.. Thank you.. WARNING (theano.gof.cmodule): OPTIMIZATION WARNING: Theano was not able to find the default g++ parameters. This is needed to tune the compilation to your specific CPU. This can slow down the execution of Theano functions. Please submit the following lines to Theano's mailing list so that we can fix this problem: ['# 1 ""\n', '# 1 ""\n', '# 1 ""\n', '# 1 "/usr/include/stdc-predef.h" 1 3 4\n', '# 1 "" 2\n', '# 1 ""\n', 'Using built-in specs.\n', 'COLLECT_GCC=/usr/bin/g++\n', 'Target: aarch64-linux-gnu\n', "Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64 --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu\n", 'Thread model: posix\n', 'gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4) \n', "COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'\n", ' /usr/lib/gcc/aarch64-linux-gnu/5/cc1 -E -quiet -v -imultiarch aarch64-linux-gnu - -mlittle-endian -mabi=lp64 -fstack-protector-strong -Wformat -Wformat-security\n', 'ignoring nonexistent directory "/usr/local/include/aarch64-linux-gnu"\n', 'ignoring nonexistent directory "/usr/lib/gcc/aarch64-linux-gnu/5/../../../../aarch64-linux-gnu/include"\n', '#include "..." search starts here:\n', '#include <...> search starts here:\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include\n', ' /usr/local/include\n', ' /usr/lib/gcc/aarch64-linux-gnu/5/include-fixed\n', ' /usr/include/aarch64-linux-gnu\n', ' /usr/include\n', 'End of search list.\n', 'COMPILER_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/:/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/\n', 'LIBRARY_PATH=/usr/lib/gcc/aarch64-linux-gnu/5/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../aarch64-linux-gnu/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../../lib/:/lib/aarch64-linux-gnu/:/lib/../lib/:/usr/lib/aarch64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/aarch64-linux-gnu/5/../../../:/lib/:/usr/lib/\n', "COLLECT_GCC_OPTIONS='-E' '-v' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64'\n"] -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[theano-users] Re: How to: data parallelism in new GPU backend
Thinking of a different design: 1. Master python process builds and compiles all theano functions like normal (for GPU), and pickles them. 2. Worker processes initialize on other GPUs and unpickle all the functions. 3. User calls wrapped theano functions in master process, which signals to workers. 4. Workers run infinite loop, waiting for signal of what to do (some switch statement), e.g.: a. call some function (can take inputs from multiprocessing shared variables) and communicate result b. copy multiprocessing shared variables to update local theano GPU shared variables c. do collective GPU comms. d. etc. The workers are "dumb" and never have to bother with any graphs. It's a bit of a pain to set up the multiprocessing shared variables (have to declare data sizes ahead of time) but not so bad. What I'm running into trouble with now is the theano shared variables. They get unpickled under the function's input_storage, but each function ends up with a separate set of objects here. I can manipulate them individually, but *is there a way to get multiple unpickled functions to refer to the same memory for corresponding shared variables?* (Simply setting the input_storage entries to another function's does not work.) -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to theano-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.