GraphFrames and IPython notebook issue - No module named graphframes

2016-04-25 Thread Camelia Elena Ciolac
Hello, I work locally on my laptop, not using DataBricks Community edition. I downloaded graphframes-0.1.0-spark1.6.jar from http://spark-packages.org/package/graphframes/graphframes and placed it in a folder named spark_extra_jars where I have other jars too. After executing in a

What happens in the master or slave launch ?

2015-10-07 Thread Camelia Elena Ciolac
Hello, I have the following question: I have two scenarios: 1) in one scenario (if I'm connected on the target node) the master starts successfully. Its log contains: Spark Command: /usr/opt/java/jdk1.7.0_07/jre/bin/java -cp

Need for advice - performance improvement and out of memory resolution

2015-09-30 Thread Camelia Elena Ciolac
Hello, I am working on a machine learning project, currently using spark-1.4.1-bin-hadoop2.6 in local mode on a laptop (Ubuntu 14.04 OS running on a Dell laptop with i7-5600@2.6 GHz * 4 cores, 15.6 GB RAM). I also mention working in Python from an IPython notebook. I face the following

Need for advice - performance improvement and out of memory resolution

2015-09-30 Thread Camelia Elena Ciolac
Hello, I am working on a machine learning project, currently using spark-1.4.1-bin-hadoop2.6 in local mode on a laptop (Ubuntu 14.04 OS running on a Dell laptop with i7-5600@2.6 GHz * 4 cores, 15.6 GB RAM). I also mention working in Python from an IPython notebook. I face the following