Re: SparkR package path

2015-09-24 Thread Shivaram Venkataraman
; >> >> >> >> From: Hossein [mailto:fal...@gmail.com] >> Sent: Thursday, September 24, 2015 1:42 AM >> To: shiva...@eecs.berkeley.edu >> Cc: Sun, Rui; dev@spark.apache.org >> Subject: Re: SparkR package path >> >> >> >> Yes, I t

Re: SparkR package path

2015-09-24 Thread Hossein
hursday, September 24, 2015 1:42 AM > *To:* shiva...@eecs.berkeley.edu > *Cc:* Sun, Rui; dev@spark.apache.org > *Subject:* Re: SparkR package path > > > > Yes, I think exposing SparkR in CRAN can significantly expand the reach of > both SparkR and Spark itself to a larg

Re: SparkR package path

2015-09-24 Thread Hossein
ark > distribution, > >> and then use SparkR. > >> > >> > >> > >> For using SparkR in Rstudio, there is a documentation at > >> https://github.com/apache/spark/tree/master/R > >> > >> > >> > >> > >>

Re: SparkR package path

2015-09-24 Thread Luciano Resende
ageing with Spark), is this desirable? Actually, for normal users >> who >> >> are not developers, they are not required to download Spark source, >> build >> >> and install SparkR package. They just need to download a Spark >> distribution, >> >>

RE: SparkR package path

2015-09-24 Thread Sun, Rui
Subject: Re: SparkR package path Right now in sparkR.R the backend hostname is hard coded to "localhost" (https://github.com/apache/spark/blob/master/R/pkg/R/sparkR.R#L156). If we make that address configurable / parameterized, then a user can connect a remote Spark cluster with no ne

RE: SparkR package path

2015-09-24 Thread Sun, Rui
AM To: Sun, Rui Cc: shiva...@eecs.berkeley.edu; dev@spark.apache.org Subject: Re: SparkR package path Requiring users to download entire Spark distribution to connect to a remote cluster (which is already running Spark) seems an over kill. Even for most spark users who download Spark source

Re: SparkR package path

2015-09-23 Thread Hossein
arkR > > source and binary package, and directly use sparkR. No need to install > > SparkR package at all. > > > > > > > > From: Hossein [mailto:fal...@gmail.com] > > Sent: Tuesday, September 22, 2015 9:19 AM > > To: dev@spark.apache.org > &g

RE: SparkR package path

2015-09-23 Thread Sun, Rui
, there is a documentation at https://github.com/apache/spark/tree/master/R From: Hossein [mailto:fal...@gmail.com] Sent: Thursday, September 24, 2015 1:42 AM To: shiva...@eecs.berkeley.edu Cc: Sun, Rui; dev@spark.apache.org Subject: Re: SparkR package path Yes, I think exposing SparkR in CRAN can significantly

Re: SparkR package path

2015-09-22 Thread Shivaram Venkataraman
> > From: Hossein [mailto:fal...@gmail.com] > Sent: Tuesday, September 22, 2015 9:19 AM > To: dev@spark.apache.org > Subject: SparkR package path > > > > Hi dev list, > > > > SparkR backend assumes SparkR source files are located under > "SPARK_HOME/R

RE: SparkR package path

2015-09-21 Thread Sun, Rui
: Hossein [mailto:fal...@gmail.com] Sent: Tuesday, September 22, 2015 9:19 AM To: dev@spark.apache.org Subject: SparkR package path Hi dev list, SparkR backend assumes SparkR source files are located under "SPARK_HOME/R/lib/." This directory is created by running R/install-dev.sh. This set

SparkR package path

2015-09-21 Thread Hossein
Hi dev list, SparkR backend assumes SparkR source files are located under "SPARK_HOME/R/lib/." This directory is created by running R/install-dev.sh. This setting makes sense for Spark developers, but if an R user downloads and installs SparkR source package, the source files are going to be in