I think the difference is py4j is a public library whereas the R backend is specific to SparkR.
Can you elaborate what you need JVMObjectTracker for? We have provided R convenient APIs to call into JVM: sparkR.callJMethod for example _____________________________ From: Jeremy Liu <jeremy.jl....@gmail.com> Sent: Tuesday, March 27, 2018 12:20 PM Subject: Re: [Spark R] Proposal: Exposing RBackend in RRunner To: <dev@spark.apache.org> Spark Dev, On second thought, the below topic seems more appropriate for spark-dev rather than spark-users: Spark Users, In SparkR, RBackend is created in RRunner.main(). This in particular makes it difficult to control or use the RBackend. For my use case, I am looking to access the JVMObjectTracker that RBackend maintains for SparkR dataframes. Analogously, pyspark starts a py4j.GatewayServer in PythonRunner.main(). It's then possible to start a ClientServer that then has access to the object bindings between Python/Java. Is there something similar for SparkR? Or a reasonable way to expose RBackend? Thanks! -- ----- Jeremy Liu jeremy.jl....@gmail.com<mailto:jeremy.jl....@gmail.com>
