RE: PySpark 1.6.1: 'builtin_function_or_method' object has no attribute '__code__' in Pickles

An example (adding a package to the spark submit): bin/spark-submit --packages com.datastax.spark:spark-cassandra-connector_2.10:1.6.0 spark_v3.py From: Bhaarat Sharma [mailto:bhaara...@gmail.com] Sent: 30 July 2016 06:38 To: ayan guha Cc: user

RE: Understanding spark concepts cluster, master, slave, job, stage, worker, executor, task

You have the same as link 1 but in English? * spark-questions-concepts * deep-into-spark-exection-model Seems really interesting post but in Chinese. I suppose

RE: run spark apps in linux crontab

Remember that the you need to souce your .bashrc For your PATH to be set up. From: luohui20...@sina.com [mailto:luohui20...@sina.com] Sent: 20 July 2016 11:01 To: user Subject: run spark apps in linux crontab hi guys: I add a spark-submit job into my Linux crontab

RE: Presentation in London: Running Spark on Hive or Hive on Spark

It is on the 20th (Wednesday) next week. From: Marco Mistroni [mailto:mmistr...@gmail.com] Sent: 15 July 2016 11:04 To: Mich Talebzadeh Cc: user @spark ; user Subject: Re: Presentation in London: Running Spark on Hive or

JAr files into python3

HI List, I have the following script which will be used in Spark. #!/usr/bin/env python3 from pyspark_cassandra import CassandraSparkContext, Row from pyspark import SparkContext, SparkConf from pyspark.sql import SQLContext import os os.environ['CLASSPATH']="/mnt/spark/lib" conf =

RE: Remote RPC client disassociated

HI Akhil I am using: Cassandra: 3.0.5 Spark: 1.6.1 Scala 2.10 Spark-cassandra connector: 1.6.0 From: Akhil Das [mailto:ak...@hacked.work] Sent: 01 July 2016 11:38 To: Joaquin Alzola <joaquin.alz...@lebara.com> Cc: user@spark.apache.org Subject: Re: Remote RPC client disassociated This

RE: Spark jobs

n error occurred while calling o29.load. : java.lang.ClassNotFoundException: Failed to find data source: org.apache.spark.sql.cassandra. Please find packages at http://spark-packages.org Is there a way to load up those jar files into the script Jo From: sujeet jog [mailto:sujeet@gmail.com] Sent: 29 J

RE: Remote RPC client disassociated

>>> 16/06/30 10:44:34 ERROR util.Utils: Uncaught exception in thread stdout >>> writer for python java.lang.AbstractMethodError: pyspark_cassandra.DeferringRowReader.read(Lcom/datastax/driver/core/Row;Lcom/datastax/spark/connector/CassandraRowMetadata;)Ljava/lang/Object; >> You are trying to

Remote RPC client disassociated

HI List, I am launching this spark-submit job: hadoop@testbedocg:/mnt/spark> bin/spark-submit --packages com.datastax.spark:spark-cassandra-connector_2.10:1.6.0 --jars /mnt/spark/lib/TargetHolding_pyspark-cassandra-0.3.5.jar spark_v2.py spark_v2.py is: from pyspark_cassandra import

RE: Unsubscribe - 3rd time

And 3rd time is not enough to know that unsubscribe is done through --> user-unsubscr...@spark.apache.org From: Steve Florence [mailto:sflore...@ypm.com] Sent: 29 June 2016 16:47 To: user@spark.apache.org Subject: Unsubscribe - 3rd time This email is confidential and may be subject to

Spark jobs

Hi, This is a totally newbie question but I seem not to find the link . when I create a spark-submit python script to be launch ... how should I call it from the main python script with a subprocess.popen? BR Joaquin This email is confidential and may be subject to privilege. If you

Spark-Cassandra connector

Hi List I am trying to install the Spark-Cassandra connector through maven or sbt but neither works. Both of them try to connect to the Internet (which I do not have connection) to download certain files. Is there a way to install the files manually? I downloaded from the maven repository -->