unsubscribe

Thanks & Regards, Meethu M - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Filtering based on a float value with more than one decimal place not working correctly in Pyspark dataframe

n, its working with more than one decimal place numbers [image: image.png] Is this a bug? Regards, Meethu Mathew

Re: Failed to run spark jobs on mesos due to "hadoop" not found.

Hi, Add HADOOP_HOME=/path/to/hadoop/folder in /etc/default/mesos-slave in all mesos agents and restart mesos Regards, Meethu Mathew On Thu, Nov 10, 2016 at 4:57 PM, Yu Wei <yu20...@hotmail.com> wrote: > Hi Guys, > > I failed to launch spark jobs on mesos. Actually I su

Re: Please reply if you use Mesos fine grained mode

Hi, We are using Mesos fine grained mode because we can have multiple instances of spark to share machines and each application get resources dynamically allocated.  Thanks & Regards,  Meethu M On Wednesday, 4 November 2015 5:24 AM, Reynold Xin wrote: If you

Spark 1.6 Release window is not updated in Spark-wiki

Hi, In the https://cwiki.apache.org/confluence/display/SPARK/Wiki+Homepage the current release window has not been changed from 1.5. Can anybody give an idea of the expected dates for 1.6 version? Regards, Meethu Mathew Senior Engineer Flytxt

[MLlib] Contributing algorithm for DP means clustering

, Evan R. Sparks, Andre Wibisono. I have raised a JIRA ticket at https://issues.apache.org/jira/browse/SPARK-8402 Suggestions and guidance are welcome. Regards, Meethu Mathew Senior Engineer Flytxt www.flytxt.com | Visit our blog http://blog.flytxt.com/ | Follow us http://www.twitter.com/flytxt

Re: Anyone facing problem in incremental building of individual project

Hi, I added ​ createDependencyReducedPom in my pom.xml and the problem is solved. !-- Work around MSHADE-148 -- + ​​ createDependencyReducedPomfalse/createDependencyReducedPom ​Thank you @Steve​ and @Ted Regards, Meethu Mathew Senior Engineer Flytxt On Thu, Jun 4, 2015 at 9:51

Anyone facing problem in incremental building of individual project

issue. Regards, Meethu Mathew Senior Engineer Flytxt

Regarding Connecting spark to Mesos documentation

version of Spark, mostly we specify a hadoop version (which is not the default one). In this case, make-distribution.sh should be supplied the same maven options we used for building spark. This is not specified in the documentation. Please correct me , if I am wrong. Regards, Meethu Mathew

Re: Speeding up Spark build during development

* * ** ** ** ** ** ** Hi, Is it really necessary to run **mvn --projects assembly/ -DskipTests install ? Could you please explain why this is needed? I got the changes after running mvn --projects streaming/ -DskipTests package. Regards, Meethu On Monday 04 May 2015 02:20 PM,

Mail to u...@spark.apache.org failing

Hi, The mail id given in https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark seems to be failing. Can anyone tell me how to get added to Powered By Spark list? -- Regards, *Meethu*

Test suites in the python wrapper of kmeans failing

Hi, The test suites in the Kmeans class in clustering.py is not updated to take the seed value and hence it is failing. Shall I make the changes and submit it along with my PR( Python API for Gaussian Mixture Model) or create a JIRA ? Regards, Meethu

Re: Test suites in the python wrapper of kmeans failing

Hi, Sorry it was my mistake. My code was not properly built. Regards, Meethu _http://www.linkedin.com/home?trk=hb_tab_home_top_ On Thursday 22 January 2015 10:39 AM, Meethu Mathew wrote: Hi, The test suites in the Kmeans class in clustering.py is not updated to take the seed value

Use of MapConverter, ListConverter in python to java object conversion

Hi all, In the python object to java conversion done in the method _py2java in spark/python/pyspark/mllib/common.py, why we are doing individual conversion using MpaConverter,ListConverter? The same can be acheived using bytearray(PickleSerializer().dumps(obj)) obj =

Re: Python to Java object conversion of numpy array

PM, Davies Liu wrote: On Sun, Jan 11, 2015 at 10:21 PM, Meethu Mathew meethu.mat...@flytxt.com wrote: Hi, This is the code I am running. mu = (Vectors.dense([0.8786, -0.7855]),Vectors.dense([-0.1863, 0.7799])) membershipMatrix = callMLlibFunc(findPredict, rdd.map(_convert_to_vector), mu) What's

Re: Python to Java object conversion of numpy array

here? On Sun, Jan 11, 2015 at 9:28 PM, Meethu Mathew meethu.mat...@flytxt.com wrote: Hi, Thanks Davies . I added a new class GaussianMixtureModel in clustering.py and the method predict in it and trying to pass numpy array from this method.I converted it to DenseVector and its solved now

Re: Python to Java object conversion of numpy array

, but now the exception is 'list' object has no attribute '_get_object_id' and when I give a tuple input (Vectors.dense([0.8786, -0.7855]),Vectors.dense([-0.1863, 0.7799])) exception is like 'numpy.ndarray' object has no attribute '_get_object_id' Regards, *Meethu Mathew* *Engineer

Re: Problems concerning implementing machine learning algorithm from scratch based on Spark

  use the numpy functions. Will it take too much time? I have found some scripts that are not from Mllib and was created by other developers(credits to Meethu Mathew from Flytxt, thanks for giving me insights!:)) Many thanks and look forward to getting feedbacks! Best, Danqing GMMSpark.py (7K

Re: [MLlib] Contributing Algorithm for Outlier Detection

Hi Ashutosh, Please edit the README file.I think the following function call is changed now. |model = OutlierWithAVFModel.outliers(master:String, input dir:String , percentage:Double||) | Regards, *Meethu Mathew* *Engineer* *Flytxt* _http://www.linkedin.com/home?trk=hb_tab_home_top_

Re: [MLlib] Contributing Algorithm for Outlier Detection

from a file? sc.textFile will simply give us an RDD, how to make it a Vector[String]? Could you plz share any code snippet of this conversion if you have.. Regards, Meethu Mathew On Friday 14 November 2014 10:02 AM, Meethu Mathew wrote: Hi Ashutosh, Please edit the README file.I think

Re: Gaussian Mixture Model clustering

at 10:38 PM, Meethu Mathew meethu.mat...@flytxt.com mailto:meethu.mat...@flytxt.com wrote: Hi all, Please find attached the image of benchmark results. The table in the previous mail got messed up. Thanks. On Friday 19 September 2014 10:55 AM, Meethu Mathew wrote: Hi all

Gaussian Mixture Model clustering

. -- Regards, *Meethu Mathew* *Engineer* *Flytxt* F: +91 471.2700202 www.flytxt.com | Visit our blog http://blog.flytxt.com/ | Follow us http://www.twitter.com/flytxt | _Connect on Linkedin http://www.linkedin.com/home?trk=hb_tab_home_top_

Contribution to MLlib

Hi, I am interested in contributing a clustering algorithm towards MLlib of Spark.I am focusing on Gaussian Mixture Model. But I saw a JIRA @ https://spark-project.atlassian.net/browse/SPARK-952 regrading the same.I would like to know whether Gaussian Mixture Model is  already implemented or

Contributions to MLlib

Hi, I would like to do some contributions towards the MLlib .I've a few concerns regarding the same. 1. Is there any reason for implementing the algorithms supported  by MLlib in Scala 2. Will you accept if  the contributions are done in Python or Java Thanks, Meethu M