Hello Ted,  Some progress,  now it seems that the spark job does get submitted, 
 In the spark web UI, I do see this under finished drivers.  However, it seems 
to not go past this step,  JavaPairReceiverInputDStream<String, String> 
messages = KafkaUtils.createStream(jsc, "localhost:2181", "aa", topicMap);   I 
do have several messages in the Kafka topic.  When I run the consumer with 
from-beginning option, I do see all the messages.

krishs-mbp:bin hadoop$ ./spark-submit --jars 
/Users/hadoop/kafka-0.8.1.1-src/core/build/libs/kafka_2.8.0-0.8.1.1.jar --class 
KafkaMain --master spark://krishs-mbp:7077 --deploy-mode cluster 
--num-executors 100 --driver-memory 1g --executor-memory 1g --supervise 
file:///Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar
Sending launch command to spark://krishs-mbp:7077
Driver successfully submitted as driver-20150303085115-0007
... waiting before polling master for driver state
... polling master for driver state
State of driver-20150303085115-0007 is FINISHED
 

     On Monday, March 2, 2015 4:31 PM, Ted Yu <[email protected]> wrote:
   

 In AkkaUtils.scala:

    val akkaLogLifecycleEvents = 
conf.getBoolean("spark.akka.logLifecycleEvents", false)

Can you turn on life cycle event logging to see if you would get some more clue 
?
Cheers
On Mon, Mar 2, 2015 at 3:56 PM, Krishnanand Khambadkone 
<[email protected]> wrote:

I see these messages now,
spark.master -> spark://krishs-mbp:7077
Classpath elements:



Sending launch command to spark://krishs-mbp:7077
Driver successfully submitted as driver-20150302155433-0000
... waiting before polling master for driver state
... polling master for driver state
State of driver-20150302155433-0000 is FAILED
 

     On Monday, March 2, 2015 2:42 PM, Ted Yu <[email protected]> wrote:
   

 bq. Cause was: akka.remote.InvalidAssociation: Invalid address: 
akka.tcp://sparkMaster@localhost:7077
There should be some more output following the above line.
Can you post them ?
Cheers
On Mon, Mar 2, 2015 at 2:06 PM, Krishnanand Khambadkone 
<[email protected]> wrote:

Hi,  I am running spark on my mac.   It is reading from a kafka topic and then 
writes the data to a hbase table.  When I do a spark submit,  I get this error,
Error connecting to master spark://localhost:7077 
(akka.tcp://sparkMaster@localhost:7077), exiting.
Cause was: akka.remote.InvalidAssociation: Invalid address: 
akka.tcp://sparkMaster@localhost:7077
My submit statement looks like this,
./spark-submit --jars 
/Users/hadoop/kafka-0.8.1.1-src/core/build/libs/kafka_2.8.0-0.8.1.1.jar --class 
KafkaMain --master spark://localhost:7077 --deploy-mode cluster --num-executors 
100 --driver-memory 1g --executor-memory 1g 
/Users/hadoop/dev/kafkahbase/kafka/src/sparkhbase.jar





    



   

Reply via email to