Hello Flink, Calcite, and Beam dev lists! Linked below is the second document I promised way back in April regarding a collaborative spec for streaming SQL in Beam/Calcite/Flink (& apologies for the delay; I thought I was nearly done a while back and then temporal joins expanded to something much larger than expected).
To repeat what it says in the doc, my hope is that it can serve various purposes over it's lifetime: - - A discussion ground for ironing out any remaining features necessary for supporting robust streaming semantics in Calcite SQL. - A rough, high-level source of truth for tracking efforts underway in support of this, currently spanning the Calcite, Flink, and Beam projects. - A written specification of the changes that were made, for the sake of understanding the delta after the fact. The first and third points are, IMO, the most important. AFAIK, there are a few features missing still that need to be defined (e.g., triggers equivalents via EMIT, robust temporal join support). I'm also proposing a clear distinction of streams and tables, which I think is important, but which I believe is not the approach most folks have been taking in this area. Sorting out these open issues and then having a concise record of the solutions adopted will be important for providing a solid streaming experience and teaching folks how to use it. At any rate, I would much appreciate it if anyone with an interest in this stuff could please take a look and add comments/suggestions/references to related work in flight/etc as appropriate. For now please use comments/suggestions, but if you really want to dive in with edit access, let me know. The doc: http://s.apache.org/streaming-sql-spec -Tyler
