Theoretically the executor is a long lived container. So you could use some
simple caching library or a simple Singleton to cache the data in your
executors, once they load it from mysql. But note that with lots of
executors you might choke your mysql.
Regards
Sab
On 05-Nov-2015 7:03 pm, "Kay-Uwe
t;"
Subject: How to use data from Database and reload every hour
I have the following problem.
We have MySQL and an Spark cluster.
We need to load 5 different validation-instructions (several thousand of
entries each) and use this information on the executors to decide if content
from
I have the following problem.
We have MySQL and an Spark cluster.
We need to load 5 different validation-instructions (several thousand of
entries each) and use this information on the executors to decide if content
from Kafka-Streaming is for process a or b.
The streaming data from kafka are json
