+1 Is there any design doc related to API/internal changes ? Will CP be the default in structured streaming or it's a mode in conjunction with exisiting behavior.
Thanks. Deb On Nov 1, 2017 8:37 AM, "Reynold Xin" <r...@databricks.com> wrote: Earlier I sent out a discussion thread for CP in Structured Streaming: https://issues.apache.org/jira/browse/SPARK-20928 It is meant to be a very small, surgical change to Structured Streaming to enable ultra-low latency. This is great timing because we are also designing and implementing data source API v2. If designed properly, we can have the same data source API working for both streaming and batch. Following the SPIP process, I'm putting this SPIP up for a vote. +1: Let's go ahead and design / implement the SPIP. +0: Don't really care. -1: I do not think this is a good idea for the following reasons.
