Re: Calcite Streaming and Mongo

2018-10-03 Thread Shuyi Chen
That will be cool. It can also be used as an example for other projects to use Calcite streaming. I would imagine you need to translate the SQL through the calcite logical plan to a set of pipeline operators and pipeline stages in mongo. You can refer to projects like Flink, Samza or Storm for

Re: Calcite Streaming and Mongo

2018-10-03 Thread Michael Mior
That would be great and an awesome example of an adapter using streaming. I haven't looked closely at Mongo's change streams, but you'd have to see if the semantics of the stream are suitable for exposing existing tables also as streams or if it would make more sense to separate streams and

Re: Calcite Streaming and Mongo

2018-10-03 Thread Julian Hyde
That would be very cool. MongoTable could implement StreamableTable, so queries that want to read change streams could do so. Please log a JIRA case. Julian > On Oct 2, 2018, at 5:42 PM, Andrei Sereda wrote: > > Hello, > > Since version 3.6 mongo introduced collection change streams >

Calcite Streaming and Mongo

2018-10-02 Thread Andrei Sereda
Hello, Since version 3.6 mongo introduced collection change streams which allow to observe real-time changes of a collection. This looks very accommodating to calcite streams which provides more complex