just FYI: we now have support for list and named-list data types in SystemML, which allow passing the entire model as a single handle. For example, you can define the following
l1 = list(W1, b1, W2, b2, W3, b3, W4, b4), or l2 = list(a=W1, b=b1, c=W2, d=b2, e=W3, f=b3, g=W4, h=b4) and access entries via l1[7] or l2['g'] accordingly. We're still working on additional features to make the integration with IPA, functions, and size/type propagation smoother, but the basic functionality is already available. Regards, Matthias On Sun, May 6, 2018 at 1:08 PM, Matthias Boehm <[email protected]> wrote: > Hi Guobao, > > that sounds very good. In general, the "model" refers to the > collection of all weights and bias matrices of a given architecture. > Similar to a classic regression model, we can view the weights as the > "slope", i.e., multiplicative terms, while the biases are the > "intercept", i.e., additive terms that shift the layer output. Both > are subject to training and thus part of the model. > > This implies that the number of matrices in the model depends on the > architecture. Hence, we have two choices here: (a) allow for a > variable number of inputs and outputs, or (b) create a struct-like > data type that allows passing the collection of matrices via a single > handle. We've discussed the second option in other contexts as well > because this would also be useful for reducing the number of > parameters passed through function calls. I'm happy to help out > integrating these struct-like data types if needed. > > Great to see that you're in the process of updating the related JIRAs. > Let us know whenever you think you're ready with an initial draft - > then I'd make a detailed pass over it. > > Furthermore, I would recommend to experiment with running these > existing mnist lenet examples (which is one of our baselines moving > forward): > * Download the "infinite MNIST" data generator > (http://leon.bottou.org/projects/infimnist), and generate a moderately > sized dataset (e.g., 256K instances). > * Convert the input into SystemML's binary block format. The generator > produces the data in libsvm format and we provide a data converter > (see RDDConverterUtils.libsvmToBinaryBlock) to convert this into our > internal binary representation. > * Run the basic mnist lenet example for a few epochs. > * Install the native BLAS libraries mkl or openblas and try using it > for the above example to ensure its setup and configured correctly. > > > Regards, > Matthias > > On Sun, May 6, 2018 at 3:24 AM, Guobao Li <[email protected]> wrote: >> Hi Matthias, >> >> I'm currently reading the dml script MNIST LeNet example and got some >> questions. I hope that you could help me out of them. >> >> 1) Is it possible to define a matrix containing the variables? Because I'm >> wondering how to represent the model as a parameter for the "paramserv" >> function. >> 2) What is the role of bias? Why we need it? >> >> Additionally, I have added some updates in JIRA for SYSTEMML-2083 and hope >> to get some feedback. Thanks! >> >> Regards, >> Guobao >>
