Hi Alek, Thanks for the info. You are correct ,that using the three colons does work. Admittedly I am a R novice, but since the three colons is used to access hidden methods, it seems pretty dirty.
Can someone shed light on the design direction being taken with SparkR? Should I really be accessing hidden methods or will better approach prevail? For instance, it feels like the k-means sample should really use MLlib and not just be a port the k-means sample using hidden methods. Am I looking at this incorrectly? Thanks, Andrew On Tue, May 26, 2015 at 6:56 AM, Eskilson,Aleksander < alek.eskil...@cerner.com> wrote: > From the changes to the namespace file, that appears to be correct, all > methods of the RDD API have been made private, which in R means that you > may still access them by using the namespace prefix SparkR with three > colons, e.g. SparkR:::func(foo, bar). > > So a starting place for porting old SparkR scripts from before the merge > could be to identify those methods in the script belonging to the RDD class > and be sure they have the namespace identifier tacked on the front. I hope > that helps. > > Regards, > Alek Eskilson > > From: Andrew Psaltis <psaltis.and...@gmail.com> > Date: Monday, May 25, 2015 at 6:25 PM > To: "dev@spark.apache.org" <dev@spark.apache.org> > Subject: SparkR and RDDs > > Hi, > I understand from SPARK-6799[1] and the respective merge commit [2] that > the RDD class is private in Spark 1.4 . If I wanted to modify the old > Kmeans and/or LR examples so that the computation happened in Spark what is > the best direction to go? Sorry if I am missing something obvious, but > based on the NAMESPACE file [3] in the SparkR codebase I am having trouble > seeing the obvious direction to go. > > Thanks in advance, > Andrew > > [1] https://issues.apache.org/jira/browse/SPARK-6799 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SPARK-2D6799&d=AwMFaQ&c=NRtzTzKNaCCmhN_9N2YJR-XrNU1huIgYP99yDsEzaJo&r=0vZw1rBdgaYvDJYLyKglbrax9kvQfRPdzxLUyWSyxPM&m=T9sfWUgCtxLUJ9F4B-MAmBhrH4e3aGvb_hbrENoIKho&s=bawjeA3Y9me3xXGxKghL4_dlf7vHdFHtiV5IhMlOmtc&e=> > [2] > https://github.com/apache/spark/commit/4b91e18d9b7803dbfe1e1cf20b46163d8cb8716c > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_spark_commit_4b91e18d9b7803dbfe1e1cf20b46163d8cb8716c&d=AwMFaQ&c=NRtzTzKNaCCmhN_9N2YJR-XrNU1huIgYP99yDsEzaJo&r=0vZw1rBdgaYvDJYLyKglbrax9kvQfRPdzxLUyWSyxPM&m=T9sfWUgCtxLUJ9F4B-MAmBhrH4e3aGvb_hbrENoIKho&s=Hc7ijtxcnrZ7wSOStlz0-BHH-rUXSFowCpJuNGYu5eo&e=> > [3] https://github.com/apache/spark/blob/branch-1.4/R/pkg/NAMESPACE > <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apache_spark_blob_branch-2D1.4_R_pkg_NAMESPACE&d=AwMFaQ&c=NRtzTzKNaCCmhN_9N2YJR-XrNU1huIgYP99yDsEzaJo&r=0vZw1rBdgaYvDJYLyKglbrax9kvQfRPdzxLUyWSyxPM&m=T9sfWUgCtxLUJ9F4B-MAmBhrH4e3aGvb_hbrENoIKho&s=l64LUOvbJ53qsVYphkYJ7_kbNptBdEhsSRSWBg5zqn8&e=> > > CONFIDENTIALITY NOTICE This message and any included attachments are > from Cerner Corporation and are intended only for the addressee. The > information contained in this message is confidential and may constitute > inside or non-public information under international, federal, or state > securities laws. Unauthorized forwarding, printing, copying, distribution, > or use of such information is strictly prohibited and may be unlawful. If > you are not the addressee, please promptly delete this message and notify > the sender of the delivery error by e-mail or you may call Cerner's > corporate offices in Kansas City, Missouri, U.S.A at (+1) (816)221-1024. >
