Hi All, this is a great discussion! (I have some thoughts on most of the topics but I'll wait for the separate discussion threads)
@Haibo Will you start a separate threads? I think the separate discussion topics would be (based on Stephans mail but further split up): 1. What should the API stack look like? 2. What should the interface for a single operator look like, i.e. what will StreamOperator look like? 3. What does a job look like, i.e. the graph of operations. Maybe a proper serialized format for DAGs. 4. Modules and dependency structure. This is currently a bit messed up for flink-streaming, which depends on flink-runtime 5. What's special for batch. There's some interdependencies, i.e. 2 depends on 5. and maybe 1. Best, Aljoscha > On 7. Dec 2018, at 10:00, Shuai Xu <chigg...@gmail.com> wrote: > > Hi all > Glad to see the discussion, we are now designing to enhance the scheduling > of batch job, a unified api will help a lot. > > Haibo Sun <sunhaib...@163.com> 于2018年12月5日周三 下午4:45写道： > >> Hi all, >> >> Thank Kurt, you see more benefits of the unification than I do. >> >> I quite agree Kurt's views. DataStream, DataSet and Table are remained >> independent for now, and subsumed DataSet in data stream in the future. The >> collection execution mode is replaced by mini cluster. The high-level >> semantic APIs have their own optimizations, but StreamTransformation does >> not. >> >> About iterations, I have not more ideas at the moment. >> >> >> Best, >> Haibo >> >> >> >> -- >> Sent from: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/ >>
