Hi Matthias, Thank you for your feedback. I've just created a PR as requested. Regards Nikolay
"Matthias Boehm" <[email protected]> wrote on 08/07/2016 08:43:55: > From: "Matthias Boehm" <[email protected]> > To: [email protected] > Date: 08/07/2016 08:44 > Subject: Re: Restricted Boltzmann Machine scripts > > thanks for reaching out Nikolay, > > 1) Scripts: Could you please create a PR to add them to /scripts/ > staging? This is the place we typically use to share new scripts. > Once they are tested for accuracy and runtime, we would migrate them > into scripts/algorithms along with some basic documentation. Thanks. > > 2) Matrix/Vector element-wise binary operations: As Fred already > mentioned, SystemML supports both Matrix-Row Vector and Matrix- > Column Vector operations, where the right-hand side input is > logically replicated which often allows us to broadcast the vector > and hence avoid shuffle operations. However, if you prefer to write > it explicitly as repmat like X %*% (v %*% matrix(1, 1, ncols(X))), > that's fine too - the SystemML compiler would then anyway > automatically rewrite this to matrix-vector operations if the repmat > is in the same basic block of statements. > > Regards, > Matthias > > > [image removed] Nikolay Manchev ---07/07/2016 09:44:28 AM---Hi all, > I am dropping a note as suggested by Fred below. > > From: Nikolay Manchev <[email protected]> > To: [email protected] > Date: 07/07/2016 09:44 AM > Subject: Restricted Boltzmann Machine scripts > > > > Hi all, > > I am dropping a note as suggested by Fred below. > > I wrote a couple of DML scripts that implement Restricted Boltzmann > Machines, which are available on GitHub. This one uses CD-1 mini-batch > training to fit the model (weights and biases): > https://github.com/nmanchev/incubator-systemml/blob/neuralnets/ > scripts/algorithms/rbm_minibatch.dml > > This one runs a data sets through the trained RBM and outputs sample of > P(h=1|v) for each observation: > https://github.com/nmanchev/incubator-systemml/blob/neuralnets/ > scripts/algorithms/rbm_run.dml > > Can you please add them to scripts/algorithms? I also created a JIRA issue > for adding them here: https://issues.apache.org/jira/browse/SYSTEMML-777 > > Many thanks > Nikolay > > > ----- Original message ----- > From: Frederick R Reiss/Almaden/IBM > To: Nikolay Manchev/UK/IBM@IBMGB > Cc: [email protected] > Subject: Re: Question on SystemML - RBMs and repmat() > Date: Thu, Jul 7, 2016 3:07 PM > > Hi Nikolay, > > I appreciate your interest in the project. To answer your question: You > should be able to write "X%*%W + B" and get the semantics you want. The > SystemML compiler automatically pads vectors with copies of themselves > when it sees a cellwise operation between a matrix and a vector. So if you > run the DML code: > > > A = matrix (1.0, rows=3, cols=3) > v = matrix (2.0, rows=1, cols=3) > sum = A + v > print(toString(sum)) > > the output will be: > > 3.000 3.000 3.000 > 3.000 3.000 3.000 > 3.000 3.000 3.000 > > > Exposing cellwise matrix-vector operations to the SystemML optimizer in > this way should result in more efficient parallel plans, since it's easier > for the optimizer to detect that it can broadcast the vector and stream > the matrix. > > The PNMF script on the SystemML home page (http://systemml.apache.org) has > a more in-depth example of the same pattern: > > while (iter < max_iterations) { > iter = iter + 1; > H = (H * (t(W) %*% (V/(W%*%H)))) / t(colSums(W)); > W = (W * ((V/(W%*%H)) %*% t(H))) / t(rowSums(H)); > obj = as.scalar(colSums(W) %*% rowSums(H)) - sum(V * log(W%*%H)); > print("iter=" + iter + " obj=" + obj); > } > > The part in red divides the matrix (H * (t(W) %*% (V/(W%*%H)))) by the > vector t(colSums(W)). In R, the divisor in this expression would need to > be (t(matrix(colSums(W),nrow=1))%*%matrix(rep(1,m),nrow=1)) or something > equivalent. > > > > I think that an example script for training Boltzmann machines would be a > useful addition to the SystemML distribution. Would you mind opening a > JIRA issue for adding this script and posting a link to the JIRA on the > SystemML mailing list? Our JIRA instance is at > https://issues.apache.org/jira/browse/SYSTEMML, and our mailing list is at > http://systemml.apache.org/community. By the way, it's good to post > questions like your question below the mailing list so that others who run > into the same issue will have an easier time finding the solution; I'm > CCing the list with my response here. > > Fred > Unless stated otherwise above: > IBM United Kingdom Limited - Registered in England and Wales with number > 741598. > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU > Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
