Reusing HBase connection in transformations

Hi! I'm struggling with the following problem: I have a couple of Spark Streaming jobs that keep state (using mapWithState, and in one case updateStateByKey) and write their results to HBase. One of the Streaming jobs, needs the results that the other Streaming job writes to HBase. How it's

Can mapWithState state func be called every batchInterval?

Hi there, I've built a Spark Streaming app that accepts certain events from Kafka, and I want to keep some state between the events. So I've successfully used mapWithState for that. The problem is, that I want the state for keys to be updated on every batchInterval, because "lack" of events is

Spark Streaming - dividing DStream into mini batches

Hi all! When reading about Spark Streaming and its execution model, I see diagrams like this a lot: It does a fine job explaining how