Hi all, I am dropping a note as suggested by Fred below.
I wrote a couple of DML scripts that implement Restricted Boltzmann Machines, which are available on GitHub. This one uses CD-1 mini-batch training to fit the model (weights and biases): https://github.com/nmanchev/incubator-systemml/blob/neuralnets/scripts/algorithms/rbm_minibatch.dml This one runs a data sets through the trained RBM and outputs sample of P(h=1|v) for each observation: https://github.com/nmanchev/incubator-systemml/blob/neuralnets/scripts/algorithms/rbm_run.dml Can you please add them to scripts/algorithms? I also created a JIRA issue for adding them here: https://issues.apache.org/jira/browse/SYSTEMML-777 Many thanks Nikolay ----- Original message ----- From: Frederick R Reiss/Almaden/IBM To: Nikolay Manchev/UK/IBM@IBMGB Cc: dev@systemml.incubator.apache.org Subject: Re: Question on SystemML - RBMs and repmat() Date: Thu, Jul 7, 2016 3:07 PM Hi Nikolay, I appreciate your interest in the project. To answer your question: You should be able to write "X%*%W + B" and get the semantics you want. The SystemML compiler automatically pads vectors with copies of themselves when it sees a cellwise operation between a matrix and a vector. So if you run the DML code: A = matrix (1.0, rows=3, cols=3) v = matrix (2.0, rows=1, cols=3) sum = A + v print(toString(sum)) the output will be: 3.000 3.000 3.000 3.000 3.000 3.000 3.000 3.000 3.000 Exposing cellwise matrix-vector operations to the SystemML optimizer in this way should result in more efficient parallel plans, since it's easier for the optimizer to detect that it can broadcast the vector and stream the matrix. The PNMF script on the SystemML home page (http://systemml.apache.org) has a more in-depth example of the same pattern: while (iter < max_iterations) { iter = iter + 1; H = (H * (t(W) %*% (V/(W%*%H)))) / t(colSums(W)); W = (W * ((V/(W%*%H)) %*% t(H))) / t(rowSums(H)); obj = as.scalar(colSums(W) %*% rowSums(H)) - sum(V * log(W%*%H)); print("iter=" + iter + " obj=" + obj); } The part in red divides the matrix (H * (t(W) %*% (V/(W%*%H)))) by the vector t(colSums(W)). In R, the divisor in this expression would need to be (t(matrix(colSums(W),nrow=1))%*%matrix(rep(1,m),nrow=1)) or something equivalent. I think that an example script for training Boltzmann machines would be a useful addition to the SystemML distribution. Would you mind opening a JIRA issue for adding this script and posting a link to the JIRA on the SystemML mailing list? Our JIRA instance is at https://issues.apache.org/jira/browse/SYSTEMML, and our mailing list is at http://systemml.apache.org/community. By the way, it's good to post questions like your question below the mailing list so that others who run into the same issue will have an easier time finding the solution; I'm CCing the list with my response here. Fred Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
