Github user wobuxiangtong commented on the issue:
https://github.com/apache/spark/pull/16850
Excuse meï¼
I am using mapwithstate to storage data in sparkstreamingãWhat
confused me is that rememberDuration must more than checkpointDurationãI read
the code of MapWithStateRDD and changed is as the follow question and have done
some tests which indicated that it does not matter even though hold only one
RDD in storage and no data will loseãit will cut down a lot of memory if
doing so like this.
Maybe i am wrong. Could you help me? the question is here:
https://stackoverflow.com/questions/47784439/what-will-happen-if-i-change-the-rememberduration-in-dstream-less-than-checkpoin
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
