Re: Spark Structured Streaming Runner Roadmap

2021-08-03 Thread Etienne Chauchot
Hi, Sorry for the late answer: the streaming mode in spark structured streaming runner is stuck because of spark structured streaming framework implementation of watermark at the apache spark project side. See https://echauchot.blogspot.com/2020/11/watermark-architecture-proposal-for.html be

Spark Structured Streaming Runner Roadmap

2021-05-20 Thread Yu Zhang
Hi Beam Community, Would there be any roadmap for Spark Structured Runner to support streaming and Splittable DoFn API? Like the specific timeline or release version. Thanks, Yu