MLlib -Collaborative Filtering

2015-04-18 Thread riginos
Is there any way that i can see the similarity table of 2 users in that algorithm? by that i mean the similarity between 2 users -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLlib-Collaborative-Filtering-tp22553.html Sent from the Apache Spark User List

MLlib -Collaborative Filtering

2015-04-18 Thread riginos
Is there any way that i can see the similarity table of 2 users in that algorithm? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/MLlib-Collaborative-Filtering-tp22552.html Sent from the Apache Spark User List mailing list archive at Nabble.com.

MLlib - Naive Bayes Problem

2015-04-16 Thread riginos
I have a big dataset of categories of cars and descriptions of cars. So i want to give a description of a car and the program to classify the category of that car. So i decided to use multinomial naive Bayes. I created a unique id for each word and replaced my whole category,description data.

Running a script on scala-shell on Spark Standalone Cluster

2015-01-27 Thread riginos
I configured 4 pcs with spark-1.2.0-bin-hadoop2.4.tgz with Spark Standalone on a Cluster. On master pc i executed: ./sbin/start-master.sh ./sbin/start-slaves.sh (4 instances) On datanode1 ,datanode2, secondarymaster pcs i executed: ./bin/spark-class org.apache.spark.deploy.worker.Worker

Eclipse on spark

2015-01-24 Thread riginos
How to compile a Spark project in Scala IDE for Eclipse? I got many scala scripts and i no longer want to load them from scala-shell what can i do? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Eclipse-on-spark-tp21350.html Sent from the Apache Spark User

Installing Spark Standalone to a Cluster

2015-01-23 Thread riginos
Do i need to manually install and configure hadoop before doing anything with spark standalone? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Installing-Spark-Standalone-to-a-Cluster-tp21339.html Sent from the Apache Spark User List mailing list archive

Installing Spark Standalone to a Cluster

2015-01-23 Thread riginos
I need someone to send me a snapshot of his /conf/spark-env.sh file cause i don't understand how to set some vars like SPARK_MASTER etc -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Installing-Spark-Standalone-to-a-Cluster-tp21341.html Sent from the

Installing Spark Standalone to a Cluster

2015-01-22 Thread riginos
I have downloaded spark-1.2.0.tgz on each of my node and execute ./sbt/sbt assembly on each of them. So I execute. /sbin/start-master.sh on my master and ./bin/spark-class org.apache.spark.deploy.worker.Worker spark://IP:PORT. Althought when I got to http://localhost:8080 I cannot see any

How to replace user.id to user.names in a file

2015-01-06 Thread riginos
I work on a user to user recommender for a website using mllib.recommendation. I have created a file (recommends.txt) which contains the top 5 recommendations for each user id. The file's form(recommends.txt) is something like this (user::rec1:rec2:rec3:rec4:rec5): /**file's snapshot**/

Install Apache Spark on a Cluster

2014-12-08 Thread riginos
My thesis is related to big data mining and I have a cluster in the laboratory of my university. My task is to install apache spark on it and use it for extraction purposes. Is there any understandable guidance on how to do this ? -- View this message in context:

Re: Spark SQL Programming Guide - registerTempTable Error

2014-11-24 Thread riginos
be good to go. If its a production environment, it sort of depends on how you are setup (e.g. AWS, Cloudera, etc.) On Sun Nov 23 2014 at 11:27:49 AM riginos [hidden email] http://user/SendEmail.jtp?type=nodenode=19598i=0 wrote: That was the problem ! Thank you Denny for your fast response

Spark SQL Programming Guide - registerTempTable Error

2014-11-23 Thread riginos
Hi guys , Im trying to do the Spark SQL Programming Guide but after the: case class Person(name: String, age: Int) // Create an RDD of Person objects and register it as a table. val people = sc.textFile(examples/src/main/resources/people.txt).map(_.split(,)).map(p = Person(p(0), p(1).trim.toInt))

Re: Spark SQL Programming Guide - registerTempTable Error

2014-11-23 Thread riginos
That was the problem ! Thank you Denny for your fast response! Another quick question: Is there any way to update spark to 1.1.0 fast? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Programming-Guide-registerTempTable-Error-tp19591p19595.html