Thanks. I tried. The problem is I have to updateStatebyKey to maintain
other states related to keys.
Not sure where to pass this accumulator variable into updateStateBykey.
On Tue, Aug 18, 2015 at 2:17 AM, Hemant Bhanawat hemant9...@gmail.com wrote:
See if SparkContext.accumulator helps.
On Tue, Aug 18, 2015 at 2:27 PM, Joanne Contact joannenetw...@gmail.com
wrote:
Hi Gurus,
Please help.
But please don't tell me to use updateStateByKey because I need a
global variable (something like the clock time) across the micro
batches but not depending on key. For my case, it is not acceptable to
maintain a state for each key since each key comes in different times.
Yes my global variable is related to time but cannot use machine
clock.
Any hint? Or is this lack of global variable by design?
Thanks!
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org