As a reminder, the list contains two correctness bugs: stream-stream outer join, and multiple stateful operations with watermark.
Regarding common theme, yes that's somewhat I'd rather avoid to say, but honestly I feel there's shortage on active committers on 'structured streaming'. Many of them I know as relevant to SS area didn't show up themselves in Spark community for around half a year (maybe even more), and unfortunately even active committers seem to have struggled with shortage of time doing their own works (that's natural) and haven't found time to focus reviewing other PRs (provide valuable comments but not leading PRs as shepherd to be merged). I hoped that's temporary issue for some important events like Spark+AI summit, and turned out it's not. Spark has no replacement of SS, DStream is now even cared less than SS. Does Spark community not feeling important from streaming area? I might not agree, as there're reports from end users and patches proposed so far from contributors. I wouldn't the right one to say how can solve the issue, but I hope we would handle the main issue nicely and less painful way. On Tue, Aug 13, 2019 at 10:42 PM Sean Owen <sro...@gmail.com> wrote: > General tips: > > - dev@ is not usually the right place to discuss _specific_ changes > except once in a while to call attention > - Ping the authors of the code being changed directly > - Tighten the change if possible > - Tests, reproductions, docs, etc help prove the change > - Bugs are more important than new marginal features > > If there has been some feedback that's just skeptical about the > approach or value, that may be the answer, it won't be merged. > If there is no feedback and it seems important (correctness bugs) it's > OK to raise that here once in a while. > > One common theme here is 'structured streaming' -- who amongst the > committers feels they are able to review these changes? I sense we > have a shortage there. > -- Name : Jungtaek Lim Blog : http://medium.com/@heartsavior Twitter : http://twitter.com/heartsavior LinkedIn : http://www.linkedin.com/in/heartsavior
