It is implemented with cogroup. Basically it stores states in a separate RDD and cogroups the target RDD with the state RDD, which is then hidden from you. See StateDStream.scala, there is everything you need to know.
On Fri, Nov 6, 2015 at 6:25 PM Hien Luu <h...@linkedin.com> wrote: > Hi, > > I am interested in learning about the implementation of updateStateByKey. > Does anyone know of a jira or design doc I read? > > I did a quick search and couldn't find much info. on the implementation. > > Thanks in advance, > > Hien >
