RE: Python : StreamingContext isn't created properly

Wed, 03 Aug 2016 07:52:20 -0700 

Sorry I missed the "return" on the last line ... coming from Scala :-)

Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

From: ppatie...@live.com
To: user@spark.apache.org
Subject: Python : StreamingContext isn't created properly
Date: Wed, 3 Aug 2016 14:16:22 +0000




Hi all,

I wrote following function in Python :

def createStreamingContext():
    conf = (SparkConf().setMaster("local[2]").setAppName("my_app"))
    conf.set("spark.streaming.receiver.writeAheadLog.enable", "true")

    sc = SparkContext(conf=conf)
    ssc = StreamingContext(sc, 1)
    ssc.checkpoint("/tmp/spark-streaming")

    ....
    ....

    ssc
But I don't know why launching it with spark_submit gives me following error :

AttributeError: 'NoneType' object has no attribute 'checkpoint'

I can imagine that ssc wasn't created properly but ... why ?

Thanks,
Paolo.


Paolo PatiernoSenior Software Engineer (IoT) @ Red Hat
Microsoft MVP on Windows Embedded & IoTMicrosoft Azure Advisor 
Twitter : @ppatierno
Linkedin : paolopatierno
Blog : DevExperience

Reply via email to