In case anyone wants to chime in, the conversation is still ongoing: https://issues.apache.org/jira/browse/SPARK-13573
Current argument is that we just use SparkR::: as a prefix to access any private SparkR methods and continue to use reflection to access the RBackend class on the JVM. They are hesitant to support the API outside of internal use and didn't consider our use case of running the kernel and R backend in the same JVM. My main fear with staying private is that they remove the functionality entirely without considering external consumers like Toree. On Mon, Feb 29, 2016 at 2:14 PM Chip Senkbeil <chip.senkb...@gmail.com> wrote: > Just a heads up. I opened an issue against Spark about exposing some of > the SparkR APIs. They are too restrictive on what the SparkR libraries > allows (no custom Java methods) and the R backend is package protected. > > https://issues.apache.org/jira/browse/SPARK-13573 > > Our SparkR implementation is a fork of that package that exposes those > APIs, which allows us to use an existing Spark Context as well as retrieve > execution requests from the kernel (same way as we and Zeppelin do it in > PySpark) and more easily interact with the remote Java process containing > the Spark Context. > > We've been needing to open this issue for awhile, so I figured I'd do it > now. >
