Hi all: In the `mapWithState` method in spark streaming, you need to pass in an anonymous function. This function maintains a state and should return a result. It can be said that the final stateful result can be obtained from the state object. So, what is the significance of returning result? I looked up the official API, and it did not specifically say that the result is used for, just give a simple explanation, as follows:
// A mapping function that maintains an integer state and return a String def mappingFunction(key: String, value: Option[Int], state: State[Int]): Option[String] = { // Use state.exists(), state.get(), state.update() and state.remove() // to manage state, and return the necessary string } val spec = StateSpec.function(mappingFunction).numPartitions(10) val mapWithStateDStream = keyValueDStream.mapWithState[StateType, MappedType](spec) Can anyone help me with this problem?Thanks! shicheng31...@gmail.com