Please take a look at the attached PDF for the SPIP: Continuous Processing Mode for Structured Streaming
https://issues.apache.org/jira/browse/SPARK-20928 It is meant to be a very small, surgical change to Structured Streaming to enable ultra-low latency. This is great timing because we are also designing and implementing data source API v2. If designed properly, we can have the same data source API working for both streaming and batch.
