Here are Niketan's question
Thanks for taking time to answer our questions and also for considering to help SystemML community. I have couple more questions: Niketan:1. In case there is inconsistency, do you (as R4ML developers) feel comfortable changing R4ML interface to be compatible with our other APIs ? May be you can go over the below two links and imagine adding a corresponding R tab: - MLContext Programming guide: http://apache.github.io/systemml/spark-mlcontext-programming-guide apache.github.io<http://apache.github.io/systemml/spark-mlcontext-programming-guide> apache.github.io Spark MLContext Programming Guide. Overview; Spark Shell Example. Start Spark Shell with SystemML; Create MLContext; Hello World; LeNet on MNIST Example; DataFrame ... - Algorithm wrappers: http://apache.github.io/systemml/algorithms-classification.html#multinomial-logistic-regression ALOK: Hi Niketan As pointed out earlier, R4ML is not just R interface so it is based on the earlier product of IBM on R and it has many product feature. Also note that the pure ML Ctx and the cmd options for dml is not ideally allow all the things user want to do in his ML code. The solution could be to create wrapper to make user happy . but we have created those wrapper but those are in R and from user point for view it feels that are just writing the R code see some of the examples at https://github.com/SparkTC/r4ml/tree/master/R4ML/inst/examples https://github.com/SparkTC/r4ml/blob/master/R4ML/inst/examples/r4ml.demo.mlogit.R NOTE: that R4ML uses combination of SparkR and DML and R to make user experience best. If the ultimate goal is to have just MLCtx based R interface than I think it undermines and R4ML value proposition. (We can definitely just expose MLCtx api. However calling Logistic Regression example just for the purpose of MLCtx won't be best) R4ML.mlogit has better apis 2. Classification - GitHub Pages<http://apache.github.io/systemml/algorithms-classification.html#multinomial-logistic-regression> apache.github.io SystemML Algorithms Reference 2. Classification 2.1. Multinomial Logistic Regression Description. The MultiLogReg.dml script performs both binomial and multinomial ... Niketan: 2. Other than providing R interface to SystemML as the above APIs, what additional features/code R4ML plans to add in SystemML ? Just like we want the R API to be functionally complete with our Python and Scala API, we want Python and Scala APIs to be functionally complete with the R API. So a discussion on supporting the additional features in Python and Scala APIs is required :) ALOK: as talked in point 1) I think it will require a lot of work for scala and python api to be in sync with r4ml api. Also I feel that if the goal is too have just python, scala than we have to do the coding at R4ML. but I think goals was to merge this project. I think @Fred if he can comment also that would be nice Thanks Alok From: alok singh <singh_a...@hotmail.com> Sent: Thursday, September 21, 2017 7:32 PM To: dev@systemml.apache.org; de...@apache.org Subject: Re: [PROPOSAL] R4ML Integration with SystemML Hi We (me and Brendan) has been focusing on other things like journeys apart from new MLCtx changes. R4ML commits and PR you can also review, I think code will definitely be maintained. Alok From: Deron Eriksson <deroneriks...@gmail.com> Sent: Thursday, September 21, 2017 6:03 PM To: dev@systemml.apache.org Subject: Re: [PROPOSAL] R4ML Integration with SystemML > > * Looking over the github repo, apparently R4ML is not under active > development/maintenance anymore (last commit Jul 20). So who would be > willing to maintain and extend it? > > ALOK: We will doing development into it . there are open PR already. > > No commits since Jul 20 does raise warning flags, as Matthias pointed out. For some perspective, SystemML has 1013 commits in the last year (~2.78 per day). No R4ML commits in 2 months is concerning for obvious reasons. It implies no real work has been done on the project for months. > * Providing wrappers for our algorithm scripts would be just a start > because it hides our core value proposition of custom large-scale ML. > Hence, we would also need an MLContext equivalent that allows to execute > arbitrary DML scripts or R functions. Is there already a tentative design > of such an API and if not, who would like to take it over? > > ALOK: Currently no out of box MLCtx. > > I believe this also raises some warning flags. Looking over the code at https://github.com/SparkTC/r4ml/blob/master/R4ML/R/sysml.bridge.R, it looks https://avatars2.githubusercontent.com/u/13631156?v=4&s=400 SparkTC/r4ml github.com r4ml - Scalable R for Machine Learning like the code in the R4ML master branch utilizes an old API that does not currently exist in SystemML. As Matthias pointed out, a key value proposition of SystemML is customizable machine learning, which would require an API that currently exists in the project. That said, I believe an R API interface to SystemML is extremely valuable and I think the whole SystemML community would benefit from the R API, and I hope you will pursue the issue further. It looks like it has been in development since June (https://github.com/SparkTC/r4ml/pull/50). https://avatars2.githubusercontent.com/u/12959246?v=4&s=400 [WIP][I-50][R4ML-123] new MLContext API by aloknsingh · Pull Request #50 · SparkTC/r4ml github.com Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to subm... Deron
