Thanks for your answers, and I'm sorry for the sight delay.
I was trying to narrow it down first since I noticed very unpredictable
behaviour in reproducing it. Finally the unpredictability seemed related to
the message format of the messages on Kafka, so I also came to suspect it
had something to
I went through StateMap.scala a few times but didn't find any logic error
yet.
According to the call stack, the following was executed in get(key):
} else {
parentStateMap.get(key)
}
This implies that parentStateMap was null.
But it seems parentStateMap is properly assigned in readO
Hi Jan, could you post your codes? I could not reproduce this issue in my
environment.
Best Regards,
Shixiong Zhu
2015-12-29 10:22 GMT-08:00 Shixiong Zhu :
> Could you create a JIRA? We can continue the discussion there. Thanks!
>
> Best Regards,
> Shixiong Zhu
>
> 2015-12-29 3:42 GMT-08:00 Jan
Could you create a JIRA? We can continue the discussion there. Thanks!
Best Regards,
Shixiong Zhu
2015-12-29 3:42 GMT-08:00 Jan Uyttenhove :
> Hi guys,
>
> I upgraded to the RC4 of Spark (streaming) 1.6.0 to (re)test the new
> mapWithState API, after previously reporting issue SPARK-11932 (
> ht
Hi guys,
I upgraded to the RC4 of Spark (streaming) 1.6.0 to (re)test the new
mapWithState API, after previously reporting issue SPARK-11932 (
https://issues.apache.org/jira/browse/SPARK-11932).
My Spark streaming job involves reading data from a Kafka topic (using
KafkaUtils.createDirectStream),