I pretty much agree with Niketan and Deron. In general, it would be useful to provide an R API as well. However, I'm a bit concerned for two reasons:
* Looking over the github repo, apparently R4ML is not under active development/maintenance anymore (last commit Jul 20). So who would be willing to maintain and extend it? * Providing wrappers for our algorithm scripts would be just a start because it hides our core value proposition of custom large-scale ML. Hence, we would also need an MLContext equivalent that allows to execute arbitrary DML scripts or R functions. Is there already a tentative design of such an API and if not, who would like to take it over? Regards, Matthias On Thu, Sep 21, 2017 at 3:43 PM, Deron Eriksson <[email protected]> wrote: > I agree with Niketan. An R interface definitely makes sense for SystemML. > DML itself is based on R, so it's surprising we have Java/Scala/Python > interfaces to SystemML but we don't have an R interface. > > Perhaps R4ML committers could supply a little more info? For instance: > 1) Would they like to merge R4ML code into the main SystemML project > itself? (Currently we have no modules.) > 2) What would they like to merge? > 3) If so, how do they propose to do so? > 4) Who will do the majority of the work to add R4ML code to SystemML? Or > who would like to volunteer to do this? > 5) Who will maintain the contributed code? Or who would like to volunteer > to do this? > 6) Documentation is needed (fit in SystemML documentation framework). > 7) Testing is needed (fit into SystemML testing framework). > 8) How is this packaged? > > From a technology standpoint, I think an R interface totally makes sense. > As for a minor criticism (which I apply to other parts of SystemML too), I > see script wrappers at https://github.com/SparkTC/r4ml/tree/master/R4ML/R. > This tightly binds the existing DML scripts to R, which means DML > input/output modifications could potentially require modifications to R > code. > > Deron > > > > On Thu, Sep 21, 2017 at 11:00 AM, Niketan Pansare <[email protected]> > wrote: > > > Janardhan: I believe this is the R4ML repo: https://github.com/SparkTC/ > > r4ml . Arvind: please correct me if I am wrong. > > > > Overall, having a R interface for SystemML is an awesome idea. Since I am > > not an R4ML expert, may be R4ML committers can comment on how they > envision > > "two code streams to work together". > > > > Also, comparing the features of R4ML with that of our Python APIs will be > > useful as it might make a stronger case for R4ML. > > > > As an FYI, here are different ways Python users can use SystemML: > > - Using MLContext to invoke DML script (http://apache.github.io/ > > systemml/beginners-guide-python#invoking-dmlpydml- > scripts-using-mlcontext > > and http://apache.github.io/systemml/spark-mlcontext- > > programming-guide.html) > > - Python algorithms wrappers (http://apache.github.io/ > > systemml/beginners-guide-python#invoke-systemmls-algorithms) > > - (not important for R4ML discussion): Python DSL ( > > http://apache.github.io/systemml/beginners-guide- > python#matrix-operations) > > > > Thanks, > > > > Niketan Pansare > > IBM Almaden Research Center > > E-mail: npansar At us.ibm.com > > http://researcher.watson.ibm.com/researcher/view.php?person=us-npansar > > > > [image: Inactive hide details for Janardhan ---09/21/2017 04:44:02 > AM---Hi > > Arvind, This is a great idea. One question: the R4ML generat]Janardhan > > ---09/21/2017 04:44:02 AM---Hi Arvind, This is a great idea. One > question: > > the R4ML generates any plan like the SystemML with `D > > > > From: Janardhan <[email protected]> > > To: Arvind Surve <[email protected]>, "[email protected]" < > > [email protected]> > > Date: 09/21/2017 04:44 AM > > Subject: Re: [DISCUSS] R-Interface to SystemML > > ------------------------------ > > > > > > > > Hi Arvind, > > > > This is a great idea. One question: the R4ML generates any plan like the > > SystemML with `DML` Or with providing some interface we leverage this > > feature ?. Community effort in the sense of collaborative algorithm > > implementation.(?) > > > > Is this the Spark-R repo ( https://urldefense.proofpoint. > > com/v2/url?u=https-3A__github.com_rstudio_sparklyr&d=DwIGaQ& > > c=jf_iaSHvJObTbx-siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_ > > 6a6SaOkZrfiSrYZVby0&m=uxG7P-4VuICwg6yatnAEX5JBdZ- > PSwyvQzq5gwX1GL0&s=6VRs_ > > J7zXj9jZEavEP8iNvVfISAjDJeM8wFL2sBnb0g&e= ) ? > > > > Thanks, > > Janardhan > > > > Sent with [ProtonMail](https://urldefense.proofpoint.com/v2/ > > url?u=https-3A__protonmail.com&d=DwIGaQ&c=jf_iaSHvJObTbx- > > siA1ZOg&r=HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m= > > uxG7P-4VuICwg6yatnAEX5JBdZ-PSwyvQzq5gwX1GL0&s=khkGV3oXz1W5m_ > > ueQRuKWlNMVOXXCVhV_ytNCINjJWY&e= ) Secure Email. > > > > > -------- Original Message -------- > > > Subject: [DISCUSS] R-Interface to SystemML > > > Local Time: September 20, 2017 12:50 PM > > > UTC Time: September 20, 2017 4:50 PM > > > From: [email protected] > > > To: [email protected] <[email protected]> > > > > > > Hi, > > > R4ML is an open source project which provides a R interface to > > SystemML.Its a bridge between SystemML and Spark-R. > > > Lets discuss here if and how we can get two code streams work together > > to benefit development/community effort. > > > > > > Arvind Surve | Spark Technology Center | > https://urldefense.proofpoint. > > com/v2/url?u=http-3A__www.spark.tc_&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r= > > HzVC6v79boGYQrpc383_Kao_6a6SaOkZrfiSrYZVby0&m=uxG7P- > 4VuICwg6yatnAEX5JBdZ- > > PSwyvQzq5gwX1GL0&s=Yj8qfo7sjGBxX547UMGfLHRZFUxCAjZDTHWe8B7jLxI&e= > > > > > > >
