Re: Global Variables in Spark Streaming

Hi Ravi, 2014-09-10 15:55 GMT+02:00 Ravi Sharma : > > > I'm using spark streaming as for kafka consumer. I want to do the CEP by spark. So as for that I need to store my sequence of events. so that I cant detect some pattern. > Depending on what you're trying to accomplish, you might implement th

Re: Global Variables in Spark Streaming

Yes your understanding is correct. In that case one easiest option would be to Serialize the object and dump it somewhere in hdfs so that you will be able to recreate/update the object from the file. We have something similar which you can find over BroadCastServer

Re: Global Variables in Spark Streaming

Akhil, By using broadcast variable Will I be able to change the values of Broadcast variable? As per my understanding It will create final variable to access the value across the cluster. Please correct me if I'm wrong. Thanks, Cheers, Ravi Sharma On Wed, Sep 10, 2014 at 7:31 PM, Akhil Das wro

Re: Global Variables in Spark Streaming

Have a look at Broadcasting variables http://spark.apache.org/docs/latest/programming-guide.html#broadcast-variables Thanks Best Regards On Wed, Sep 10, 2014 at 7:25 PM, Ravi Sharma wrote: > Hi Friends, > > I'm using spark streaming as for kafka consumer. I want to do the CEP by > spark. So as