Writing to mysql from pyspark spark structured streaming

2020-10-15 Thread Krishnanand Khambadkone
Hi,  I am trying to write to mysql from a spark structured streaming kafka source.  Using spark 2.4.0. I get this exception, AnalysisException: u"'write' can not be called on streaming Dataset/DataFrame

Re: Spark Error: Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077

2015-03-03 Thread Krishnanand Khambadkone
) Can you turn on life cycle event logging to see if you would get some more clue ? Cheers On Mon, Mar 2, 2015 at 3:56 PM, Krishnanand Khambadkone kkhambadk...@yahoo.com wrote: I see these messages now, spark.master - spark://krishs-mbp:7077 Classpath elements: Sending launch command to spark

java.lang.NoClassDefFoundError: org/apache/spark/streaming/kafka/KafkaUtils

2015-03-03 Thread Krishnanand Khambadkone
Hi,  When I submit my spark job, I see the following runtime exception in the log, Exception in thread Thread-1 java.lang.NoClassDefFoundError: org/apache/spark/streaming/kafka/KafkaUtils at SparkHdfs.run(SparkHdfs.java:56) Caused by: java.lang.ClassNotFoundException:

Spark Error: Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077

2015-03-02 Thread Krishnanand Khambadkone
Hi,  I am running spark on my mac.   It is reading from a kafka topic and then writes the data to a hbase table.  When I do a spark submit,  I get this error, Error connecting to master spark://localhost:7077 (akka.tcp://sparkMaster@localhost:7077), exiting. Cause was:

Re: Spark Error: Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077

2015-03-02 Thread Krishnanand Khambadkone
:7077 There should be some more output following the above line. Can you post them ? Cheers On Mon, Mar 2, 2015 at 2:06 PM, Krishnanand Khambadkone kkhambadk...@yahoo.com.invalid wrote: Hi,  I am running spark on my mac.   It is reading from a kafka topic and then writes the data to a hbase table

Re: Spark Error: Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077

2015-03-02 Thread Krishnanand Khambadkone
. Can you post them ? Cheers On Mon, Mar 2, 2015 at 2:06 PM, Krishnanand Khambadkone kkhambadk...@yahoo.com.invalid wrote: Hi,  I am running spark on my mac.   It is reading from a kafka topic and then writes the data to a hbase table.  When I do a spark submit,  I get this error, Error connecting

Re: Spark Error: Cause was: akka.remote.InvalidAssociation: Invalid address: akka.tcp://sparkMaster@localhost:7077

2015-03-02 Thread Krishnanand Khambadkone
? Cheers On Mon, Mar 2, 2015 at 2:06 PM, Krishnanand Khambadkone kkhambadk...@yahoo.com.invalid wrote: Hi,  I am running spark on my mac.   It is reading from a kafka topic and then writes the data to a hbase table.  When I do a spark submit,  I get this error, Error connecting to master spark