Hi Anant,
I have removed the counter and all possible side effects. Now I think we can go
ahead with the testing. I have created another folder for testing. I will add
you as a collaborator in github .
_Ashutosh
________________________________
From: slcclimber [via Apache Spark Developers List]
<ml-node+s1001551n9399...@n3.nabble.com>
Sent: Monday, November 17, 2014 10:45 AM
To: Ashutosh Trivedi (MT2013030)
Subject: Re: [MLlib] Contributing Algorithm for Outlier Detection
Ashutosh,
The counter will certainly be an parellization issue when multiple nodes are
used specially over massive datasets.
A better approach would be to use some thing along these lines:
val index = sc.parallelize(Range.Long(0, rdd.count, 1), rdd.partitions.size)
val rddWithIndex = rdd.zip(index)
Which zips the two RDD's in a parallelizable fashion.
________________________________
If you reply to this email, your message will be added to the discussion below:
http://apache-spark-developers-list.1001551.n3.nabble.com/MLlib-Contributing-Algorithm-for-Outlier-Detection-tp8880p9399.html
To unsubscribe from [MLlib] Contributing Algorithm for Outlier Detection, click
here<http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=8880&code=YXNodXRvc2gudHJpdmVkaUBpaWl0Yi5vcmd8ODg4MHwtMzkzMzE5NzYx>.
NAML<http://apache-spark-developers-list.1001551.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/MLlib-Contributing-Algorithm-for-Outlier-Detection-tp8880p9420.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
