Regarding Spark Cassandra Metrics

Hi all, I am developing a spark application where I am loading the data into Cassandra and I am using the Spark Cassandra connector for the same. I have created a FAT jar with all the dependencies and submitted that using spark-submit. I am able to load the data successfully to cassandra, but I

Re: spark cassandra questiom

Yes, it should be good to use Spark for this use case in my opinion. You can look into using the Cassandra Spark connector for persisting your updated data into Cassandra. Cheers, Sonal Nube Technologies Join me at Data Con LA Oct 23 | Big Data Conference Europe. Nov 24

spark cassandra questiom

I an very very new to both spark and spark structured streaming. I have to write an application that receives a very very large csv files in hdfs folder. the app must take the file and on each row it must read from Cassandra data base some rows (not many rows will be returned for each row in csv).

Re: spark-cassandra-connector_2.1 caused java.lang.NoClassDefFoundError under Spark 2.4.2?

gt; On Mon, May 6, 2019, 7:59 PM Richard Xin >> wrote: >> >> >> org.apache.spark >> spark-core_2.12 >> 2.4.0 >> compile >> >> >> org.apache.spark >> spark-sql_2.12 >> 2.4.0 >> >> >>

Re: spark-cassandra-connector_2.1 caused java.lang.NoClassDefFoundError under Spark 2.4.2?

; >> Spark is shown at 2.12, the connector only has a 2.11 release >> >> >> >> On Mon, May 6, 2019, 7:59 PM Richard Xin >> wrote: >> >> >> org.apache.spark >> spark-core_2.12 >> 2.4.0 >> compile >> >>

Re: spark-cassandra-connector_2.1 caused java.lang.NoClassDefFoundError under Spark 2.4.2?

e.spark > spark-core_2.12 > 2.4.0 > compile > > > org.apache.spark > spark-sql_2.12 > 2.4.0 > > > com.datastax.spark > spark-cassandra-connector_2.11 > 2.4.1 > > > > > I run spark-submit I got following

Re: spark-cassandra-connector_2.1 caused java.lang.NoClassDefFoundError under Spark 2.4.2?

org.apache.spark spark-sql_2.12 2.4.0 com.datastax.spark spark-cassandra-connector_2.11 2.4.1 I run spark-submit I got following exceptions on Spark 2.4.2, it works fine when running  spark-submit under Spark 2.4.0 with exact the same command-line call, any idea how do i

Re: spark-cassandra-connector_2.1 caused java.lang.NoClassDefFoundError under Spark 2.4.2?

spark-sql_2.12 > 2.4.0 > > > com.datastax.spark > spark-cassandra-connector_2.11 > 2.4.1 > > > > > I run spark-submit I got following exceptions on Spark 2.4.2, it works > fine when running spark-submit under Spark 2.4.0 with exact the same > command-line call,

spark-cassandra-connector_2.1 caused java.lang.NoClassDefFoundError under Spark 2.4.2?

org.apache.spark spark-core_2.12 2.4.0 compile org.apache.spark spark-sql_2.12 2.4.0 com.datastax.spark spark-cassandra-connector_2.11 2.4.1 I run spark-submit I got following exceptions on Spark 2.4.2, it works fine when running  spark-submit under

read parallel processing spark-cassandra

Folks, I have a time series table with each record being 350 columns. the primary key is ((date, bucket), objectid, timestamp) objective is to read 1 day worth of data, which comes to around 12k partitions, each partition has around 25MB of data, I see only 1 task active during the read

Re: Testing Spark-Cassandra

ocker-test-cluster/> link > to install it, and then follow this > <https://github.com/koeninger/spark-cassandra-example> project, but you > will have to adapt the necessary libraries to use spark 2.0.x version. > > Good luck, i would like to see any blog post using this combi

Re: Testing Spark-Cassandra

Yes, you can use docker to build your own cassandra ring. Depending your SO, instructions may change, so, please, follow this <https://yurisubach.com/2016/03/24/cassandra-docker-test-cluster/> link to install it, and then follow this <https://github.com/koeninger/spark-cassandra-example

Testing Spark-Cassandra

Hello, I'm using spark 2.0 and Cassandra. Is there any util to make unit test easily or which one would be the best way to do it? library? Cassandra with docker?

Lucene Index with Spark Cassandra

Hi everyone, I am trying to run lucene with spark but sparkSQL returns zero results, where when same query is run using cqlsh it returns correct rows. same issue as https://github.com/Stratio/cassandra-lucene-index/issues/79 I can see in spark logs that lucene is working but as mentioned in the

Re: Apache Spark or Spark-Cassandra-Connector doesnt look like it is reading multiple partitions in parallel.

; HDFS and then read it using spark.read.json . > > Cheers, > Anastasios > > > > On Sat, Nov 26, 2016 at 9:34 AM, kant kodali <kanth...@gmail.com> wrote: > >> up vote >> down votefavorite >> <http://stackoverflow.com/questions/40797231/apache-spark

Re: Apache Spark or Spark-Cassandra-Connector doesnt look like it is reading multiple partitions in parallel.

che-spark-or-spark-cassandra-connector-doesnt-look-like-it-is-reading-multipl?noredirect=1#> > > Apache Spark or Spark-Cassandra-Connector doesnt look like it is reading > multiple partitions in parallel. > > Here is my code using spark-shell > &

Apache Spark or Spark-Cassandra-Connector doesnt look like it is reading multiple partitions in parallel.

up vote down votefavorite <http://stackoverflow.com/questions/40797231/apache-spark-or-spark-cassandra-connector-doesnt-look-like-it-is-reading-multipl?noredirect=1#> Apache Spark or Spark-Cassandra-Connector doesnt look like it is reading multiple partitions in parallel. Here is my code

Re: Python - Spark Cassandra Connector on DC/OS

Sorry: Spark 2.0.0 On Tue, Nov 1, 2016 at 10:04 AM, Andrew Holway < andrew.hol...@otternetworks.de> wrote: > Hello, > > I've been getting pretty serious with DC/OS which I guess could be > described as a somewhat polished distribution of Mesos. I'm not sure how > relevant DC/OS is to this

Python - Spark Cassandra Connector on DC/OS

Hello, I've been getting pretty serious with DC/OS which I guess could be described as a somewhat polished distribution of Mesos. I'm not sure how relevant DC/OS is to this problem. I am using this pyspark program to test the cassandra connection: http://bit.ly/2eWAfxm (github) I can that the

Re: unresolved dependency: datastax#spark-cassandra-connector;2.0.0-s_2.11-M3-20-g75719df: not found

3 You can verify the available versions by searching Maven at http://search.maven.org. Thanks, Kevin On Wed, Sep 21, 2016 at 3:38 AM, muhammet pakyürek <mpa...@hotmail.com> wrote: > while i run the spark-shell as below > > spark-shell --jars '/home/ktuser/spark-cassandra- > connector

unresolved dependency: datastax#spark-cassandra-connector;2.0.0-s_2.11-M3-20-g75719df: not found

while i run the spark-shell as below spark-shell --jars '/home/ktuser/spark-cassandra-connector/target/scala-2.11/root_2.11-2.0.0-M3-20-g75719df.jar' --packages datastax:spark-cassandra-connector:2.0.0-s_2.11-M3-20-g75719df --conf spark.cassandra.connection.host=localhost i get the error

cassandra 3.7 is compatible with datastax Spark Cassandra Connector 2.0?

Re: clear steps for installation of spark, cassandra and cassandra connector to run on spyder 2.3.7 using python 3.5 and anaconda 2.4 ipython 4.0

ammet pakyürek" <mpa...@hotmail.com> wrote: > > > could u send me documents and links to satisfy all above requirements of > installation > of spark, cassandra and cassandra connector to run on spyder 2.3.7 using > python 3.5 and anaconda 2.4 ipython 4.0 > > > -- > >

clear steps for installation of spark, cassandra and cassandra connector to run on spyder 2.3.7 using python 3.5 and anaconda 2.4 ipython 4.0

could u send me documents and links to satisfy all above requirements of installation of spark, cassandra and cassandra connector to run on spyder 2.3.7 using python 3.5 and anaconda 2.4 ipython 4.0

Re: spark cassandra issue

un, Sep 4, 2016 at 10:04 PM, Russell Spitzer <russell.spit...@gmail.com> wrote: > This would also be a better question for the SCC user list :) > https://groups.google.com/a/lists.datastax.com/forum/#! > forum/spark-connector-user > > On Sun, Sep 4, 2016 at 9:31 AM Russell Spitzer <r

Re: spark cassandra issue

This would also be a better question for the SCC user list :) https://groups.google.com/a/lists.datastax.com/forum/#!forum/spark-connector-user On Sun, Sep 4, 2016 at 9:31 AM Russell Spitzer <russell.spit...@gmail.com> wrote: > > https://github.com/datastax/spark-cassandra-connector

Re: spark cassandra issue

https://github.com/datastax/spark-cassandra-connector/blob/v1.3.1/doc/14_data_frames.md In Spark 1.3 it was illegal to use "table" as a key in Spark SQL so in that version of Spark the connector needed to use the option "c_table" val df = sqlContext

Re: spark cassandra issue

t; sc.cassandraTable("keyspace name", "table name") > > > On Sun, Sep 4, 2016 at 8:48 PM, Mich Talebzadeh <mich.talebza...@gmail.com > > wrote: > >> Hi Selvan. >> >> I don't deal with Cassandra but have you tried other options as describe

Re: spark cassandra issue

ch.talebza...@gmail.com> wrote: > Hi Selvan. > > I don't deal with Cassandra but have you tried other options as described > here > > https://github.com/datastax/spark-cassandra-connector/ > blob/master/doc/2_loading.md > > To get a Spark RDD that represent

Re: spark cassandra issue

Hi Selvan. I don't deal with Cassandra but have you tried other options as described here https://github.com/datastax/spark-cassandra-connector/blob/master/doc/2_loading.md To get a Spark RDD that represents a Cassandra table, call the cassandraTable method on the SparkContext object. import

Re: spark cassandra issue

its very urgent. please help me guys. On Sun, Sep 4, 2016 at 8:05 PM, Selvam Raman wrote: > Please help me to solve the issue. > > spark-shell --packages com.datastax.spark:spark-cassandra-connector_2.10:1.3.0 > --conf spark.cassandra.connection.host=** > > val df =

spark cassandra issue

Please help me to solve the issue. spark-shell --packages com.datastax.spark:spark-cassandra-connector_2.10:1.3.0 --conf spark.cassandra.connection.host=** val df = sqlContext.read. | format("org.apache.spark.sql.cassandra"). | options(Map( "table" -> "", "keyspace" -> "***")).

Spark-Cassandra connector

Hi List I am trying to install the Spark-Cassandra connector through maven or sbt but neither works. Both of them try to connect to the Internet (which I do not have connection) to download certain files. Is there a way to install the files manually? I downloaded from the maven repository

Re: pyspark spark-cassandra-connector java.io.IOException: Failed to open native connection to Cassandra at {192.168.1.126}:9042

quot;9043") sqlContext.setConf("host","localhost") sqlContext.setConf("port","9043”) Thanks Andy From: Saurabh Bajaj <bajaj.onl...@gmail.com> Date: Tuesday, March 8, 2016 at 9:13 PM To: Andrew Davidson <a...@santacruzintegration.com> Cc: Te

Re: pyspark spark-cassandra-connector java.io.IOException: Failed to open native connection to Cassandra at {192.168.1.126}:9042

onnection.port > > https://github.com/datastax/spark-cassandra-connector/blob/master/doc/reference.md#cassandra-connection-parameters > > Looking at the logs, it seems your port config is not being set and it's > falling back to default. > Let me know if that helps. > > Saurab

Re: pyspark spark-cassandra-connector java.io.IOException: Failed to open native connection to Cassandra at {192.168.1.126}:9042

Hi Andy, I believe you need to set the host and port settings separately spark.cassandra.connection.host spark.cassandra.connection.port https://github.com/datastax/spark-cassandra-connector/blob/master/doc/reference.md#cassandra-connection-parameters Looking at the logs, it seems your port

Re: pyspark spark-cassandra-connector java.io.IOException: Failed to open native connection to Cassandra at {192.168.1.126}:9042

Hi Ted I believe by default cassandra listens on 9042 From: Ted Yu <yuzhih...@gmail.com> Date: Tuesday, March 8, 2016 at 6:11 PM To: Andrew Davidson <a...@santacruzintegration.com> Cc: "user @spark" <user@spark.apache.org> Subject: Re: pyspark spark-cassandra-c

Re: pyspark spark-cassandra-connector java.io.IOException: Failed to open native connection to Cassandra at {192.168.1.126}:9042

Have you contacted spark-cassandra-connector related mailing list ? I wonder where the port 9042 came from. Cheers On Tue, Mar 8, 2016 at 6:02 PM, Andy Davidson <a...@santacruzintegration.com > wrote: > > I am using spark-1.6.0-bin-hadoop2.6. I am trying to write a python > note

pyspark spark-cassandra-connector java.io.IOException: Failed to open native connection to Cassandra at {192.168.1.126}:9042

I am using spark-1.6.0-bin-hadoop2.6. I am trying to write a python notebook that reads a data frame from Cassandra. I connect to cassadra using an ssh tunnel running on port 9043. CQLSH works how ever I can not figure out how to configure my notebook. I have tried various hacks any idea what I

Re: metrics not reported by spark-cassandra-connector

Hi Yin, Thanks for your reply. I didn't realize there is a specific mailing list for spark-Cassandra-connector. I will ask there. Thanks! -Sa On Tuesday, February 23, 2016, Yin Yang <yy201...@gmail.com> wrote: > Hi, Sa: > Have you asked on spark-cassandra-connector mailing list ? &

Re: metrics not reported by spark-cassandra-connector

Hi, Sa: Have you asked on spark-cassandra-connector mailing list ? Seems you would get better response there. Cheers

metrics not reported by spark-cassandra-connector

Hi there, I am trying to enable the metrics collection by spark-cassandra-connector, following the instruction here: https://github.com/datastax/spark-cassandra-connector/blob/master/doc/11_metrics.md However, I was not able to see any metrics reported. I'm using spark-cassandra-connector_2.10

spark-cassandra-connector BulkOutputWriter

Hello all, I looked through the cassandra spark integration ( https://github.com/datastax/spark-cassandra-connector) and couldn't find any usages of the BulkOutputWriter ( http://www.datastax.com/dev/blog/bulk-loading) - an awesome tool for creating local sstables, which could be later uploaded

RE: spark-cassandra-connector BulkOutputWriter

Alex – I suggest posting this question on the Spark Cassandra Connector mailing list. The SCC developers are pretty responsive. Mohammed Author: Big Data Analytics with Spark<http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> From: Alexandr Dzhagriev [mail

Spark Cassandra Atomic Inserts

Cassandra provides "BEGIN BATCH" and "APPLY BATCH" to perform atomic execution of multiple statements as below: BEGIN BATCH INSERT INTO "user_status_updates" ("username", "id", "body") VALUES( 'dave', 16e2f240-2afa-11e4-8069-5f98e903bf02, 'dave update 4' ); INSERT INTO

RE: spark-cassandra

Another thing to check is what version of the Spark-Cassandra-Connector the Spark Job server passing to the workers. It looks like when you use Spark-submit, you are sending the correct SCC jar, but the Spark Job server may be using a different one. Mohammed Author: Big Data Analytics

spark-cassandra

Hi, I am using Spark Jobserver to submit the jobs. I am using spark-cassandra connector to connect to Cassandra. I am getting below exception through spak jobserver. If I submit the job through *Spark-Submit *command it is working fine,. Please let me know how to solve this issue Exception

Re: Spark Cassandra clusters

Thanks mohammed and Ted. I will try out the options and let you all know the progress. Also had posted in spark Cassandra connector community, got similar response. Regards Vivek On Sat, Jan 23, 2016 at 11:37 am, Mohammed Guller <moham...@glassbeam.com<mailto:moham...@glassbeam.com&g

RE: Spark Cassandra clusters

with Spark<http://www.amazon.com/Big-Data-Analytics-Spark-Practitioners/dp/1484209656/> From: Ted Yu [mailto:yuzhih...@gmail.com] Sent: Friday, January 22, 2016 6:37 PM To: vivek.meghanat...@wipro.com Cc: user Subject: Re: Spark Cassandra clusters I am not Cassandra developer :-) Can you us

Re: Spark Cassandra clusters

vivek.meghanat...@wipro.com [mailto:vivek.meghanat...@wipro.com] > *Sent:* Friday, January 22, 2016 5:38 PM > *To:* user@spark.apache.org > *Subject:* Spark Cassandra clusters > > > > Hi All, > What is the right spark Cassandra cluster setup - having Cassandra cluster &

Re: Spark Cassandra clusters

ro.com> > <vivek.meghanat...@wipro.com> wrote: > > Hi All, > What is the right spark Cassandra cluster setup - having Cassandra cluster > and spark cluster in different nodes or they should be on same nodes. > We are having them in different nodes and performance test shows

Re: Spark Cassandra clusters

bit more information ? How much memory does each node have ? What's the current heap allocation for Cassandra process and executor ? Spark / Cassandra release you are using Thanks On Fri, Jan 22, 2016 at 5:37 PM, <vivek.meghanat...@wipro.com<mailto:vivek.meghanat...@wipro.com>>

Re: Spark Cassandra clusters

Can you give us a bit more information ? > > How much memory does each node have ? > What's the current heap allocation for Cassandra process and executor ? > Spark / Cassandra release you are using > > Thanks > > On Fri, Jan 22, 2016 at 5:37 PM, <vivek.meghanat...@wipro.com&g

Re: Spark Cassandra clusters

de have ? What's the current heap allocation for Cassandra process and executor ? Spark / Cassandra release you are using Thanks On Fri, Jan 22, 2016 at 5:37 PM, <vivek.meghanat...@wipro.com<mailto:vivek.meghanat...@wipro.com>> wrote: Hi All, What is the right spark Cassandra cluster

Re: Spark Cassandra clusters

ystems hence was >> planning to have them together on more efficient nodes. >> >> Regards >> Vivek >> On Sat, Jan 23, 2016 at 7:13 am, Ted Yu <yuzhih...@gmail.com> wrote: >> >> Can you give us a bit more information ? >> >> How much memor

Re: Spark Cassandra clusters

Can you give us a bit more information ? How much memory does each node have ? What's the current heap allocation for Cassandra process and executor ? Spark / Cassandra release you are using Thanks On Fri, Jan 22, 2016 at 5:37 PM, <vivek.meghanat...@wipro.com> wrote: > Hi Al

Spark Cassandra clusters

Hi All, What is the right spark Cassandra cluster setup - having Cassandra cluster and spark cluster in different nodes or they should be on same nodes. We are having them in different nodes and performance test shows very bad result for the spark streaming jobs. Please let us know. Regards

Re: Spark Cassandra Java Connector: records missing despite consistency=ALL

Cassandra Java Driver 2.2.0-rc3 Spark Cassandra Java Connector 2.10-1.5.0-M2 - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org <mailto:user-unsubscr...@spark.apache.org> For additional comman

Re: Spark Cassandra Java Connector: records missing despite consistency=ALL

s > problem, thanks! > > Greets, > Dennis > > P.s.: > some information about our setup: > Cassandra 2.1.12 in a two Node configuration with replication factor=2 > Spark 1.5.1 > Cassandra Java Driver 2.2.0-rc3 > Spark Cassandra Java Connector 2.10-1.5.0-M2 > >

Spark Cassandra Java Connector: records missing despite consistency=ALL

ld really appreciate if someone could give me a hint how to fix this problem, thanks! Greets, Dennis P.s.: some information about our setup: Cassandra 2.1.12 in a two Node configuration with replication factor=2 Spark 1.5.1 Cassandra Java Driver 2.2.0-rc3 Spark Cassandra Java Con

Re: Spark 1.5.2 compatible spark-cassandra-connector

Hi, Vivek M I had ever tried 1.5.x spark-cassandra connector and indeed encounter some classpath issues, mainly for the guaua dependency. I believe that can be solved by some maven config, but have not tried that yet. Best, Sun. fightf...@163.com From: vivek.meghanat...@wipro.com Date

Re: Spark 1.5.2 compatible spark-cassandra-connector

t;user@spark.apache.org> 主题: Re: Spark 1.5.2 compatible spark-cassandra-connector Hi, Vivek M I had ever tried 1.5.x spark-cassandra connector and indeed encounter some classpath issues, mainly for the guaua dependency. I believe that can be solved by some maven config, but have not tried

RE: Spark 1.5.2 compatible spark-cassandra-connector

Thank you mwy and Sun for your response. Yes basic things are working for me using this connector(guava issue was encountered earlier but with proper exclusion of old version we have resolved it). The current issue is strange one �C we have a kafka-spark-cassandra streaming job in spark

Spark 1.5.2 compatible spark-cassandra-connector

All, What is the compatible spark-cassandra-connector for spark 1.5.2? I can only find the latest connector version spark-cassandra-connector_2.10-1.5.0-M3 which has dependency with 1.5.1 spark. Can we use the same for 1.5.2? Any classpath issues needs to be handled or any jars needs

Re: error in spark cassandra connector

Mind providing a bit more detail ? Release of Spark version of Cassandra connector How job was submitted complete stack trace Thanks On Thu, Dec 24, 2015 at 2:06 AM, Vijay Kandiboyina wrote: > java.lang.NoClassDefFoundError: >

error in spark cassandra connector

java.lang.NoClassDefFoundError: com/datastax/spark/connector/rdd/CassandraTableScanRDD

Spark- Cassandra Connector Error

erflow.com/questions/33896937/spark-connector-error-warn-nettyutil-found-nettys-native-epoll-transport-but Thank you for your help and for your support. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Cassandra-Connector-Error-tp25483.html Sent from

Spark Cassandra Filtering

Hi , How can i pass an dynamic value inside below function to filter instead of hardcoded if have an existing RDD and i would like to use data in that for filter so instead of doing .where("name=?","Anna") i want to do .where("name=?",someobject.value) Please help JavaRDD rdd3 =

Re: Spark-Cassandra-connector

is YES, is there a way to create a connectionPool for each executor, so that multiple task can dump data to cassandra in parallel? Regards, Samya -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Cassandra-connector-tp24378.html Sent from

Spark-Cassandra-connector

.1001560.n3.nabble.com/Spark-Cassandra-connector-tp24378.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h

Evaluating spark + Cassandra for our use cases

(which is unappealing from a cost standpoint) because it's the only thing that provides the hive spark thrift server to Cassandra. The two top contenders for our solution are Spark+Cassandra and Druid. Neither of these solutions work perfectly out of the box: - Druid would need

Re: Evaluating spark + Cassandra for our use cases

from a cost standpoint) because it’s the only thing that provides the hive spark thrift server to Cassandra. The two top contenders for our solution are Spark+Cassandra and Druid. Neither of these solutions work perfectly out of the box: - Druid would need to be modified, possibly

RE: Evaluating spark + Cassandra for our use cases

From: Jörn Franke [jornfra...@gmail.com] Sent: Tuesday, August 18, 2015 4:14 PM To: Benjamin Ross; user@spark.apache.org Cc: Ron Gonzalez Subject: Re: Evaluating spark + Cassandra for our use cases Hi, First you need to make your SLA clear. It does not sound for me

Re: Spark Cassandra Connector issue

HI All, I have tried Commands as mentioned below but still it is errors dse spark-submit --master spark://10.246.43.15:7077 --class HelloWorld --jars /home/missingmerch/ postgresql-9.4-1201.jdbc41.jar,/home/missingmerch/dse.jar,/home/missingmerch/spark- cassandra-connector-java_2.10-1.1.1.jar

Spark Cassandra Connector issue

HI All, Please help me to fix Spark Cassandra Connector issue, find the details below *Command:* dse spark-submit --master spark://10.246.43.15:7077 --class HelloWorld --jars ///home/missingmerch/postgresql-9.4-1201.jdbc41.jar ///home/missingmerch/etl-0.0.1-SNAPSHOT.jar *Error:* WARN 2015

Re: Spark Cassandra Connector issue

Add the other Cassandra dependencies (dse.jar, spark-cassandra-connect-java_2.10) to your --jars argument on the command line. Dean Wampler, Ph.D. Author: Programming Scala, 2nd Edition http://shop.oreilly.com/product/0636920033073.do (O'Reilly) Typesafe http://typesafe.com @deanwampler http

Re: Spark Cassandra Connector issue

-1201.jdbc41.jar,/home/missingmerch/dse.jar,/home/missingmerch/spark- cassandra-connector-java_2.10-1.1.1.jar /home/missingmerch/etl-0.0. 1-SNAPSHOT.jar I also removed the extra //. Or put file: in front of them so they are proper URLs. Note the snapshot jar isn't in the --jars list. I assume that's

Re: Spark Cassandra Connector issue

Hi, Thanks for quick input, now I am getting class not found error *Command:* dse spark-submit --master spark://10.246.43.15:7077 --class HelloWorld --jars ///home/missingmerch/postgresql-9.4-1201.jdbc41.jar ///home/missingmerch/dse.jar ///home/missingmerch/spark-cassandra-connector-java_2.10

Spark-Cassandra connector DataFrame

Hi, I would like to get the recommendations to use Spark-Cassandra connector DataFrame feature. I was trying to save a Dataframe containing 8 Million rows to Cassandra through the Spark-Cassandra connector. Based on the Spark log, this single action took about 60 minutes to complete. I think

Spark - Cassandra (timestamp question)

Hi, I am using Spark to load data form Cassandra. One of the fields in C* table is timestamp. When queried in C* it looks like this 2015-06-01 02:56:07-0700 After loading data in to Spark DataFrame (using sqlContext) and printing it from there, I lose the last field (4-digit time zone) and than

Spark Cassandra

in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Cassandra-tp23065.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org

Re: Spark Cassandra connector number of Tasks

Looking for help with this. Thank you! -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Cassandra-connector-number-of-Tasks-tp22820p22839.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Spark Cassandra connector number of Tasks

I am using the Spark Cassandra connector to work with a table with 3 million records. Using .where() API to work with only a certain rows in this table. Where clause filters the data to 1 rows. CassandraJavaUtil.javaFunctions(sparkContext) .cassandraTable(KEY_SPACE, MY_TABLE

Spark Cassandra Connector

libraryDependencies += com.datastax.spark %% spark-cassandra-connector % 1.2.0-rc3 in order to create the previous jar version of the connector and not the default one (i.e. spark-cassandra-connector-assembly-1.3.0-SNAPSHOT.jar ) I am really new on working with sbt. Any guidance / help would be really

Hive, Spark, Cassandra, Tableau, BI, etc.

Hi,I've seen a few articles where they CqlStorageHandler to create hive tables referencing Cassandra data using the thriftserver. Is there a secret to getting this to work? I've basically got Spark built with Hive, and a Cassandra cluster. Is there a way to get the hive server to talk to

Python connector for spark-cassandra

Hello everyone, Is there a python connector for Spark and Cassandra as there is one for Java. I found a Java connector by DataStax on github: https://github.com/datastax/spark-cassandra-connector I am looking for something similar in Java. Thanks

RE: Adding Spark Cassandra dependency breaks Spark Streaming?

to 1.1.1 anyway, but I am intrigued...I'm fairly new to sbt, scala and the jvm in general. Any idea how having spark streaming 1.1.0 and spark cassandra connector 1.1.0 together would cause classes in spark streaming to go missing? Here's the full sbt file if anybody is interested: import sbt

RE: Adding Spark Cassandra dependency breaks Spark Streaming?

Hi,Just checked cassandra connector 1.1.0-beta1 runs fine. The issue seems to be 1.1.0 for spark streaming and 1.1.0 cassandra connector (final). Regards,Ashic. Date: Sat, 6 Dec 2014 13:52:20 -0500 Subject: Re: Adding Spark Cassandra dependency breaks Spark Streaming? From: jayunit100.apa

Adding Spark Cassandra dependency breaks Spark Streaming?

version := 1.0 scalaVersion := 2.10.4 val spark = org.apache.spark %% spark-core % 1.1.0val sparkStreaming = org.apache.spark %% spark-streaming % 1.1.0val cassandraConnector = com.datastax.spark %% spark-cassandra-connector % 1.1.0 withSources() withJavadoc() libraryDependencies ++= Seq

Re: Adding Spark Cassandra dependency breaks Spark Streaming?

= com.datastax.spark %% spark-cassandra-connector % 1.1.0 withSources() withJavadoc()* *libraryDependencies ++= Seq(cassandraConnector,spark,sparkStreaming)* *resolvers += Akka Repository at http://repo.akka.io/releases/ http://repo.akka.io/releases/* And here's my code: *import

RE: Adding Spark Cassandra dependency breaks Spark Streaming?

-cassandra-connector_2.10/artifactId version1.1.0/version /dependency /dependencies /project Date: Fri, 5 Dec 2014 10:58:51 -0800 Subject: Re: Adding Spark Cassandra dependency breaks Spark Streaming? From: yuzhih...@gmail.com To: as...@live.com CC: user@spark.apache.org Can

RE: Adding Spark Cassandra dependency breaks Spark Streaming?

Getting this on the home machine as well. Not referencing the spark cassandra connector in libraryDependencies compiles. I've recently updated IntelliJ to 14. Could that be causing an issue? From: as...@live.com To: yuzhih...@gmail.com CC: user@spark.apache.org Subject: RE: Adding Spark

Spark Cassandra Guava version issues

issue. I'm using the Spark Cassandra connector v 1.1.0-rc2 which just came out, though the issue was in rc1 as well. I can't see the cassandra connector using Guava directly, so I guess it's a dependency for some other thing that the cassandra spark connector is using. Does anybody have a workaround

Re: Spark Cassandra Guava version issues

I faced same problem, and s work around solution is here : https://github.com/datastax/spark-cassandra-connector/issues/292 best, /Shahab On Mon, Nov 24, 2014 at 3:21 PM, Ashic Mahtab as...@live.com wrote: I've got a Cassandra 2.1.1 + Spark 1.1.0 cluster running. I'm using sbt-assembly

RE: Spark Cassandra Guava version issues

Did the workaround work for you? Doesn't seem to work for me. Date: Mon, 24 Nov 2014 16:44:17 +0100 Subject: Re: Spark Cassandra Guava version issues From: shahab.mok...@gmail.com To: as...@live.com CC: user@spark.apache.org I faced same problem, and s work around solution is here : https

Spark Cassandra Connector proper usage

I'm looking to use spark for some ETL, which will mostly consist of update statements (a column is a set, that'll be appended to, so a simple insert is likely not going to work). As such, it seems like issuing CQL queries to import the data is the best option. Using the Spark Cassandra

RE: Spark Cassandra Connector proper usage

, Ashic. Date: Thu, 23 Oct 2014 14:27:47 +0200 Subject: Re: Spark Cassandra Connector proper usage From: gerard.m...@gmail.com To: as...@live.com Ashic, With the Spark-cassandra connector you would typically create an RDD from the source table, update what you need, filter out what you don't update

Re: Spark Cassandra Connector proper usage

about session management in order to issue custom update queries. Thanks, Ashic. -- Date: Thu, 23 Oct 2014 14:27:47 +0200 Subject: Re: Spark Cassandra Connector proper usage From: gerard.m...@gmail.com To: as...@live.com Ashic, With the Spark-cassandra connector

RE: Spark Cassandra Connector proper usage

Hi Gerard, I've gone with option 1, and seems to be working well. Option 2 is also quite interesting. Thanks for your help in this. Regards, Ashic. From: gerard.m...@gmail.com Date: Thu, 23 Oct 2014 17:07:56 +0200 Subject: Re: Spark Cassandra Connector proper usage To: as...@live.com CC: user

Spark Cassandra connector issue

Hi, I am creating a cassandra java rdd and transforming it using the where clause. It works fine when I run it outside the mapValues, but when I put the code in mapValues I get an error while creating the transformation. Below is my sample code: CassandraJavaRDDReferenceData

  1   2   >