+1. I think this architecture makes a lot of sense to let executors talk to source/sink directly, and bring very low latency.
On Thu, Nov 2, 2017 at 9:01 AM, Sean Owen <so...@cloudera.com> wrote: > +0 simply because I don't feel I know enough to have an opinion. I have no > reason to doubt the change though, from a skim through the doc. > > > On Wed, Nov 1, 2017 at 3:37 PM Reynold Xin <r...@databricks.com> wrote: > >> Earlier I sent out a discussion thread for CP in Structured Streaming: >> >> https://issues.apache.org/jira/browse/SPARK-20928 >> >> It is meant to be a very small, surgical change to Structured Streaming >> to enable ultra-low latency. This is great timing because we are also >> designing and implementing data source API v2. If designed properly, we can >> have the same data source API working for both streaming and batch. >> >> >> Following the SPIP process, I'm putting this SPIP up for a vote. >> >> +1: Let's go ahead and design / implement the SPIP. >> +0: Don't really care. >> -1: I do not think this is a good idea for the following reasons. >> >> >>
