RE: Not able to write output to local filsystem from Standalone mode.

Thanks Mathieu, So either I must have shared filesystem OR Hadoop as filesystem in order to write data from Standalone mode cluster setup environment. Thanks for your input. Regards Stuti Awasthi From: Mathieu Longtin [math...@closetwork.org] Sent: Tuesday, May 24, 2016 7:34 PM To: Stuti

RE: Not able to write output to local filsystem from Standalone mode.

tuti. Thanks Stuti  From: Jacek Laskowski [ja...@japila.pl] Sent: Tuesday, May 24, 2016 5:27 PM To: Stuti Awasthi Cc: user Subject: Re: Not able to write output to local filsystem from Standalone mode. Hi, What happens when you create the parent directory /home/stuti? I think the failu

Not able to write output to local filsystem from Standalone mode.

gest. Thanks Stuti Awasthi ::DISCLAIMER:: The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s)

Joins in Spark

hich I can directly use for join rather than implementing Mapside join using Broadcast on my own or any other better way is also welcome. I assume that this might be very common problem for all and looking out for suggestions. Thanks Stuti Awasthi ::

RE: Launch Spark shell using differnt python version

PATH $ pyspark --master local Python 2.6.6 (r266:84292, Jul 23 2015, 15:22:56) [GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2 …. Thanks Stuti Awasthi From: Prabhu Joseph [mailto:prabhujose.ga...@gmail.com] Sent: Tuesday, March 15, 2016 2:22 PM To: Stuti Awasthi Cc: user@spark.apache.org Subject: R

Launch Spark shell using differnt python version

:$PATH Also it is to be noted that Spark cluster is configured with different user account and I have not installed python2.7 on all the nodes in the cluster as I don't have permission access. So is there any way that I can launch my spark shell using Python2.7. Please suggest Thanks Stuti

Survival Curves using AFT implementation in Spark

853...|48.33071792204102| +-+-+ Can any suggest how to use the developed model for plotting Survival Curves for "test_final" data which is a dataframe feature[vector]. Thanks Stuti Awasthi

RE: mllib:Survival Analysis : assertion failed: AFTAggregator loss sum is infinity. Error for unknown reason.

Thanks a lot Yanbo, this will really help. Since I was unaware of this, I was speculating if my vectors were not getting generated correctly. Thanks !! Thanks Stuti Awasthi From: Yanbo Liang [mailto:yblia...@gmail.com] Sent: Wednesday, February 17, 2016 11:51 AM To: Stuti Awasthi Cc: user

mllib:Survival Analysis : assertion failed: AFTAggregator loss sum is infinity. Error for unknown reason.

0.0,[53.3644,2.0,2.0,1.0]] [431.0,1.0,[50.3397,2.0,1.0,1.0]] Im not able to understand about the error, as if I use same data and create the denseVector as given in Sample example of AFT, then code works completely fine. But I would like to read the data from CSV file and then proceed. Please suggest

Issue in executing Spark Application from Eclipse

: Connecting to master spark://10.112.67.80:7077... 14/12/04 11:06:31 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory Thanks Stuti Awasthi ::DISCLAIMER

RE: Cannot run program Rscript using SparkR

Thanks Shivaram, This was the issue. Now I have installed Rscript on all the nodes in Spark cluster and it works now bith from script as well as R prompt. Thanks Stuti Awasthi From: Shivaram Venkataraman [mailto:shiva...@eecs.berkeley.edu] Sent: Tuesday, August 19, 2014 1:17 PM To: Stuti

RSpark installation on Windows

Hi All Can we install RSpark on windows setup of R and use it to access the remote Spark cluster ? Thanks Stuti Awasthi ::DISCLAIMER

SparkR Installation

Hi All, I wanted to try SparkR. Do we need preinstalled R on all the nodes of the cluster before installing SparkR package ? Please guide me how to proceed with this. As of now, I work with R only on single node. Please suggest Thanks Stuti Awasthi ::DISCLAIMER

Convert text into tfidf vectors for Classification

vectorization. Is there any pre-built api's which can be used or other way in which we can achieve this Please suggest Thanks Stuti Awasthi ::DISCLAIMER

Inter and Inra Cluster Density in KMeans

Hi, I wanted to calculate the InterClusterDensity and IntraClusterDensity from the clusters generated from KMeans. How can I achieve that? Is there any already present code/api to use for this purpose. Thanks Stuti Awasthi ::DISCLAIMER

RE: How to use Mahout VectorWritable in Spark.

, SSH_TTY=/dev/pts/0, SPARK_MASTER_PORT=7077, LOGNAME=hduser, MASTER=spark://VM-52540048731A:7077, SPARK_WORKER_MEMORY=2g, HADOOP_HOME=/usr/lib/hadoop, SS... Still not able to import Mahout Classes.. Any ideas ?? Thanks Stuti Awasthi -Original Message- From: Stuti Awasthi Sent

RE: How to use Mahout VectorWritable in Spark.

that, you can do val data = raw.values.map(_.get) To get an RDD of mahout's Vector. You can use `--jar mahout-math.jar` when you launch spark-shell to include mahout-math. Best, Xiangrui On Tue, May 13, 2014 at 10:37 PM, Stuti Awasthi stutiawas...@hcl.com wrote: Hi All, I am very new